Mintplex-Labs/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-07-19 14:13:35 -04:00
Code Issues 328 Packages Projects Releases 10 Wiki Activity

[GH-ISSUE #4057] Backend Architecture: Performance Monitoring and Observability #2586

New Issue
Closed
opened 2026-02-22 18:30:21 -05:00 by yindo · 0 comments
yindo commented 2026-02-22 18:30:21 -05:00
Owner
Copy Link
Copy Source

Originally created by @jezweb on GitHub (Jun 26, 2025).
Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/4057

Overview

Implement comprehensive monitoring, logging, and observability features for better system insights and debugging.

Related to upstream discussion: https://github.com/Mintplex-Labs/anything-llm/issues/2797

Current State

  • Basic console logging
  • Limited performance metrics
  • No distributed tracing
  • Minimal error tracking

Proposed Solution

1. Structured Logging

Log Format Standardization

// Structured log entry
{
  "timestamp": "2024-01-01T00:00:00.000Z",
  "level": "info",
  "service": "api",
  "traceId": "abc-123-def",
  "spanId": "456-789",
  "userId": "user-123",
  "method": "POST",
  "path": "/api/v1/chat",
  "duration": 145,
  "status": 200,
  "metadata": {
    "workspaceId": "ws-123",
    "model": "gpt-4",
    "tokens": 1250
  }
}

Log Levels and Categories

  • Error: System errors, exceptions
  • Warn: Degraded performance, retry attempts
  • Info: Request/response, state changes
  • Debug: Detailed execution flow
  • Trace: Full request details

2. Metrics Collection

Application Metrics

// Prometheus-style metrics
http_requests_total{method="GET",endpoint="/api/chat",status="200"} 1234
http_request_duration_seconds{quantile="0.99"} 0.145
active_websocket_connections 127
document_processing_queue_size 45
vector_search_latency_milliseconds{provider="qdrant"} 23

Business Metrics

  • Chat completions per minute
  • Token usage by workspace
  • Document processing throughput
  • Search accuracy scores
  • User engagement metrics

3. Distributed Tracing

OpenTelemetry Integration

const { trace } = require('@opentelemetry/api');

async function processChat(request) {
  const span = tracer.startSpan('chat.process');
  
  span.setAttributes({
    'user.id': request.userId,
    'workspace.id': request.workspaceId,
    'chat.model': request.model
  });
  
  try {
    // Trace through entire request lifecycle
    const context = await span.traceContext(validateRequest)(request);
    const embedding = await span.traceContext(generateEmbedding)(context);
    const results = await span.traceContext(searchVectors)(embedding);
    const response = await span.traceContext(generateResponse)(results);
    
    span.setStatus({ code: StatusCode.OK });
    return response;
  } catch (error) {
    span.recordException(error);
    span.setStatus({ code: StatusCode.ERROR });
    throw error;
  } finally {
    span.end();
  }
}

4. Error Tracking

Enhanced Error Handling

class ErrorTracker {
  captureError(error, context) {
    const errorInfo = {
      message: error.message,
      stack: error.stack,
      type: error.constructor.name,
      timestamp: new Date().toISOString(),
      
      // Context
      user: context.user,
      request: this.sanitizeRequest(context.request),
      environment: process.env.NODE_ENV,
      
      // Metadata
      service: 'anythingllm-backend',
      version: process.env.APP_VERSION,
      hostname: os.hostname()
    };
    
    // Send to error tracking service
    this.send(errorInfo);
  }
}

5. Performance Monitoring

Real-time Dashboards

  • Request latency percentiles
  • Error rate trends
  • Resource utilization
  • Queue depths
  • Cache hit rates

Alerting Rules

alerts:
  - name: HighErrorRate
    condition: rate(errors[5m]) > 0.05
    severity: critical
    
  - name: SlowResponse
    condition: http_request_duration_p99 > 2
    severity: warning
    
  - name: HighMemoryUsage
    condition: memory_usage_percent > 90
    severity: critical

6. Implementation Architecture

Components

Application -> OpenTelemetry Collector -> Backends
                     < /dev/null | 
                    ├── Prometheus (Metrics)
                    ├── Elasticsearch (Logs)
                    ├── Jaeger (Traces)
                    └── Sentry (Errors)

SDK Integration

// Unified observability setup
const { setupObservability } = require('./observability');

setupObservability({
  serviceName: 'anythingllm-backend',
  
  // Metrics
  metrics: {
    enabled: true,
    endpoint: process.env.METRICS_ENDPOINT,
    interval: 60000
  },
  
  // Tracing
  tracing: {
    enabled: true,
    endpoint: process.env.TRACING_ENDPOINT,
    sampleRate: 0.1
  },
  
  // Logging
  logging: {
    level: process.env.LOG_LEVEL || 'info',
    format: 'json'
  }
});

Benefits

Operational

  • Faster issue diagnosis
  • Proactive problem detection
  • Performance optimization insights
  • Capacity planning data

Business

  • User experience metrics
  • Cost optimization data
  • Feature usage analytics
  • SLA monitoring

Implementation Phases

Phase 1: Foundation

  1. Implement structured logging
  2. Add basic metrics collection
  3. Set up error tracking
  4. Create initial dashboards

Phase 2: Advanced Features

  1. Add distributed tracing
  2. Implement custom metrics
  3. Create alerting rules
  4. Build analytics dashboards

Phase 3: Optimization

  1. Fine-tune sampling rates
  2. Optimize storage retention
  3. Add anomaly detection
  4. Create runbooks

Labels: enhancement, monitoring, observability, backend

Originally created by @jezweb on GitHub (Jun 26, 2025). Original GitHub issue: https://github.com/Mintplex-Labs/anything-llm/issues/4057 ## Overview Implement comprehensive monitoring, logging, and observability features for better system insights and debugging. Related to upstream discussion: https://github.com/Mintplex-Labs/anything-llm/issues/2797 ## Current State - Basic console logging - Limited performance metrics - No distributed tracing - Minimal error tracking ## Proposed Solution ### 1. Structured Logging #### Log Format Standardization ```javascript // Structured log entry { "timestamp": "2024-01-01T00:00:00.000Z", "level": "info", "service": "api", "traceId": "abc-123-def", "spanId": "456-789", "userId": "user-123", "method": "POST", "path": "/api/v1/chat", "duration": 145, "status": 200, "metadata": { "workspaceId": "ws-123", "model": "gpt-4", "tokens": 1250 } } ``` #### Log Levels and Categories - **Error**: System errors, exceptions - **Warn**: Degraded performance, retry attempts - **Info**: Request/response, state changes - **Debug**: Detailed execution flow - **Trace**: Full request details ### 2. Metrics Collection #### Application Metrics ```javascript // Prometheus-style metrics http_requests_total{method="GET",endpoint="/api/chat",status="200"} 1234 http_request_duration_seconds{quantile="0.99"} 0.145 active_websocket_connections 127 document_processing_queue_size 45 vector_search_latency_milliseconds{provider="qdrant"} 23 ``` #### Business Metrics - Chat completions per minute - Token usage by workspace - Document processing throughput - Search accuracy scores - User engagement metrics ### 3. Distributed Tracing #### OpenTelemetry Integration ```javascript const { trace } = require('@opentelemetry/api'); async function processChat(request) { const span = tracer.startSpan('chat.process'); span.setAttributes({ 'user.id': request.userId, 'workspace.id': request.workspaceId, 'chat.model': request.model }); try { // Trace through entire request lifecycle const context = await span.traceContext(validateRequest)(request); const embedding = await span.traceContext(generateEmbedding)(context); const results = await span.traceContext(searchVectors)(embedding); const response = await span.traceContext(generateResponse)(results); span.setStatus({ code: StatusCode.OK }); return response; } catch (error) { span.recordException(error); span.setStatus({ code: StatusCode.ERROR }); throw error; } finally { span.end(); } } ``` ### 4. Error Tracking #### Enhanced Error Handling ```javascript class ErrorTracker { captureError(error, context) { const errorInfo = { message: error.message, stack: error.stack, type: error.constructor.name, timestamp: new Date().toISOString(), // Context user: context.user, request: this.sanitizeRequest(context.request), environment: process.env.NODE_ENV, // Metadata service: 'anythingllm-backend', version: process.env.APP_VERSION, hostname: os.hostname() }; // Send to error tracking service this.send(errorInfo); } } ``` ### 5. Performance Monitoring #### Real-time Dashboards - Request latency percentiles - Error rate trends - Resource utilization - Queue depths - Cache hit rates #### Alerting Rules ```yaml alerts: - name: HighErrorRate condition: rate(errors[5m]) > 0.05 severity: critical - name: SlowResponse condition: http_request_duration_p99 > 2 severity: warning - name: HighMemoryUsage condition: memory_usage_percent > 90 severity: critical ``` ### 6. Implementation Architecture #### Components ``` Application -> OpenTelemetry Collector -> Backends < /dev/null | ├── Prometheus (Metrics) ├── Elasticsearch (Logs) ├── Jaeger (Traces) └── Sentry (Errors) ``` #### SDK Integration ```javascript // Unified observability setup const { setupObservability } = require('./observability'); setupObservability({ serviceName: 'anythingllm-backend', // Metrics metrics: { enabled: true, endpoint: process.env.METRICS_ENDPOINT, interval: 60000 }, // Tracing tracing: { enabled: true, endpoint: process.env.TRACING_ENDPOINT, sampleRate: 0.1 }, // Logging logging: { level: process.env.LOG_LEVEL || 'info', format: 'json' } }); ``` ## Benefits ### Operational - Faster issue diagnosis - Proactive problem detection - Performance optimization insights - Capacity planning data ### Business - User experience metrics - Cost optimization data - Feature usage analytics - SLA monitoring ## Implementation Phases ### Phase 1: Foundation 1. Implement structured logging 2. Add basic metrics collection 3. Set up error tracking 4. Create initial dashboards ### Phase 2: Advanced Features 1. Add distributed tracing 2. Implement custom metrics 3. Create alerting rules 4. Build analytics dashboards ### Phase 3: Optimization 1. Fine-tune sampling rates 2. Optimize storage retention 3. Add anomaly detection 4. Create runbooks Labels: enhancement, monitoring, observability, backend
yindo closed this issue 2026-02-22 18:30:21 -05:00
yindo changed title from Backend Architecture: Performance Monitoring and Observability to [GH-ISSUE #4057] Backend Architecture: Performance Monitoring and Observability 2026-06-05 14:47:24 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Desktop
Docker
Integration Request
Integration Request
OS: Linux
OS: Mobile
OS: Windows
UI/UX
blocked
bug
bug
core-team-only
documentation
duplicate
embed-widget
enhancement
feature request
github_actions
good first issue
investigating
needs info / can't replicate
possible bug
pull-request
Mirrored from GitHub Pull Request
 question
stage: specifications
wontfix
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Mintplex-Labs/anything-llm#2586
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?