Skip to main content

0001. OpenWebUI vs LibreChat Distribution

Date: 2024-03-19
Status: Proposed
Deciders: [Team]
Tags: [infra]

Context​

We have two Ollama servers:

  • arcana-cpu: CPU-based server
  • arcana-gpu: GPU-based server

We need to decide which server should host:

  • OpenWebUI: A modern, minimalist UI for Ollama
  • LibreChat: A more feature-rich ChatGPT-like interface

Key considerations:

  • Resource utilization (CPU vs GPU)
  • UI complexity and requirements
  • Expected usage patterns
  • Server capabilities

Decision​

Proposed distribution:

arcana-gpu: Install OpenWebUI

  • OpenWebUI is more lightweight and focused on direct model interaction
  • GPU server's primary role is model execution, making it a natural fit for a streamlined UI
  • Keeps the GPU server focused on core model operations

arcana-cpu: Install LibreChat

  • LibreChat has more features and overhead, better suited for CPU server
  • CPU server can handle the additional backend processing required by LibreChat
  • Separates complex UI operations from model execution resources

Consequences​

Positive​

  • Clear separation of concerns between servers
  • GPU resources remain primarily focused on model execution
  • Better resource utilization based on UI characteristics
  • Easier monitoring and scaling based on usage patterns

Negative​

  • Need to maintain two different UI systems
  • Users need to know which interface to use for what purpose
  • Potential complexity in documentation and access management

Neutral​

  • Need to set up proper DNS and routing for both UIs
  • Required monitoring setup for both systems
  • Documentation needs for both interfaces

References​