ADR 0005: Ollama Integration Strategy

Status

Accepted

Context

For our market analysis feature, we needed to integrate Ollama to perform AI-powered market comparisons. We had two main options:

1. Ollama REST API: Direct HTTP calls to Ollama's REST API endpoints
2. OllamaJS: A Node.js client library for Ollama

Both options provide access to Ollama's functionality, but they differ in several aspects that influenced our decision.

Decision

We chose to use the direct Ollama REST API approach over OllamaJS.

Rationale

Advantages of Ollama REST API

1. Direct Control

   • Full control over request/response handling
   • Ability to implement custom retry logic
   • Easier to debug network issues
   • No additional dependency layer

2. Performance

   • No overhead from client library abstractions
   • Direct HTTP calls are more efficient
   • Better control over timeouts and connection pooling

3. Flexibility

   • Easier to adapt to API changes
   • Can implement custom caching strategies
   • More control over error handling

4. Deployment

   • No additional npm dependencies
   • Simpler deployment process
   • Reduced package size

Disadvantages of OllamaJS

1. Additional Abstraction Layer

   • Potential version compatibility issues
   • Less control over underlying HTTP requests
   • Dependency on third-party maintenance

2. Limited Customization

   • Predefined error handling
   • Fixed retry strategies
   • Less flexibility in request configuration

3. Version Lock-in

   • Need to wait for library updates for new Ollama features
   • Potential conflicts with other dependencies

Implementation Details

We implemented the Ollama integration using:

• Axios for HTTP requests
• Custom retry logic with exponential backoff
• Configurable timeouts
• Environment-based URL configuration
• Response validation and error handling

Example configuration:

OLLAMA_URL: 'http://ollama:11434';
OLLAMA_API_TIMEOUT: '60000';

Consequences

Positive

1. Better control over the integration
2. Improved performance
3. Easier to maintain and debug
4. More flexible error handling
5. Direct access to new Ollama features

Negative

1. Need to implement our own client abstractions
2. More responsibility for handling edge cases
3. Need to stay updated with Ollama API changes

Future Considerations

1. Monitoring

   • Implement detailed request/response logging
   • Add performance metrics tracking
   • Monitor model usage patterns

2. Optimization

   • Cache frequently used prompts
   • Implement connection pooling
   • Add request queuing for high load

3. Features

   • Support for streaming responses
   • Batch request handling
   • Model performance tracking