18 Commits
Author SHA1 Message Date
Bentlybro 4935119ac2 Default OPENAI_REASONING_EFFORT to 'low' (not 'minimal')
'minimal' is only valid on older gpt-5 / o-series models and is rejected
by gpt-5.4 with a 400 ("Supported values are: 'none', 'low', 'medium',
'high', 'xhigh'"). Our fail-safe silently approved every request when the
API errored, which presented as "moderation is fast but broken" in real
tests — scam / phishing / misinfo content was getting through.

'low' is the lowest common value accepted by every reasoning model in the
family so the default works regardless of which model you point the
OPENAI_CHAT_MODEL at. Empirically verified: 8/8 real-world classification
accuracy on gpt-5.4-nano with reasoning=low, vs 2/4 with reasoning=none
(false negatives on obvious scams).
2026-04-22 14:03:38 +01:00
Bentlybro 478ca18ad8 Phase 0 hardening + batched AI moderation + websocket/perf fixes
Security / correctness
- Privilege escalation: rule/api-key/project-update routes now gate on
  can_manage_members instead of is_member; plain members could previously
  rewrite or disable moderation rules and delete API keys.
- Project.is_owner() AttributeError on non-admin API-user deletion (route
  + template) replaced with user_id comparison.
- Flask-Limiter on /auth/login, /auth/register, /auth/change-password,
  and /api/moderate (per API key id).
- Remove silent SECRET_KEY fallback: docker-compose fails hard if unset;
  production config validator refuses to boot without SECRET_KEY and
  OPENAI_API_KEY.
- Require verified email in Google OAuth auto-link flow.
- User-supplied regex now evaluated via the regex package with a 1s
  timeout, closing the ReDoS vector.
- Cap batched AI chunk count (max 40) and concurrent workers (200 -> 10).
- openai>=2.32 (from 1.55.3 so reasoning_effort kwarg works); add regex,
  Flask-Limiter, gunicorn, eventlet, psycogreen; drop the stdlib-shadowing
  pypi asyncio package.

Production runtime
- New wsgi.py: eventlet.monkey_patch + psycogreen.patch_psycopg so DB
  queries cooperate with the event loop; sets SOCKETIO_ASYNC_MODE=eventlet
  for app startup.
- Dockerfiles run as non-root app user and CMD gunicorn with
  --worker-class eventlet. run.py refuses FLASK_CONFIG=production so a
  misconfigured deploy fails loudly.
- cloudbuild.yaml stops managing env vars/secrets on deploy — it was
  about to clobber secret-backed env vars with plain ones. Runtime config
  is now owned by the Cloud Run service.

SocketIO async-mode fix (was causing full site lockup during API calls)
- async_mode picked via SOCKETIO_ASYNC_MODE env var, default "threading".
  The previous sys.modules auto-detect false-positived because
  flask_socketio / python-engineio import eventlet on load whenever it is
  installed, pinning a non-monkey-patched eventlet server in front of
  blocking stdlib I/O.

Real-time dashboard updates
- New content_received WebSocket event fires when the API accepts a
  submission; UI renders a "Processing…" row immediately and updates it
  in place when moderation_update arrives.
- Fix stale updated_at timestamp in moderation_update payloads.

Batched AI rule evaluation
- Replace the per-rule parallel fan-out (N OpenAI calls) with a single
  batched call evaluating all AI-prompt rules at once. On a 6-rule
  default project this drops safe-content cost from 5-12s to ~1-4s.
- response_format=json_object + max_completion_tokens=4000 so the model
  emits strict JSON and stops at the close-brace.
- OPENAI_REASONING_EFFORT config: minimal/low/medium/high for gpt-5 /
  o-series; accepts "none" on gpt-5.4. Works around the gpt-5-nano case
  where reasoning tokens ate the full output budget and returned empty
  responses.

Client errors
- Malformed / non-UTF-8 JSON bodies now return 400 with a clear message
  about UTF-8 encoding instead of a generic 500.

Logging
- Single INFO line per moderation: the existing
  "Content <id>: decision in Zs" summary.
- Per-stage timing, websocket scheduling, and batched-AI call details
  moved to DEBUG so they are available when diagnosing latency without
  cluttering steady-state logs.
2026-04-21 16:01:18 +01:00
Bentlybro 1d82daefc9 Increase concurrency and DB pool sizes for scalability
Raised ThreadPoolExecutor worker limits from 50 to 200 in AI moderation and rule processing to improve OpenAI API concurrency. Increased SQLAlchemy pool_size to 50, pool_timeout to 60, max_overflow to 100, and DB_THREAD_POOL_WORKERS to 100 for better database performance under load.
2025-10-22 16:23:12 +01:00
Bentlybro 1bbd830910 Refine AI moderator token calculation and config
Improved the token calculation logic in AIModerator to better estimate prompt and message overhead, ensuring more accurate content token allocation. Updated OPENAI_MAX_OUTPUT_TOKENS default in config.py to 500, reflecting the typical size of AI moderator responses.
2025-10-14 14:22:27 +01:00
Bentlybro c8522fba14 Refactor token limit calculation for AI moderation
Replaces static max_content_tokens logic with dynamic calculation based on prompt size, ensuring content fits within the model's context window. Updates default context window from 400,000 to 272,000 tokens in both config and service. Improves chunking and moderation handling for custom prompts and default flows.
2025-10-03 22:39:35 +01:00
Bentlybro 4a999fa369 Integrate Sentry error tracking with Flask app
Added Sentry DSN configuration to environment and config files. Initialized Sentry SDK in the Flask app if DSN is provided, and updated requirements to include sentry-sdk[flask] for error monitoring.
2025-10-02 21:13:53 +01:00
Bentlybro 2a98fc7228 Add Google OAuth login and user linking
Integrates Google OAuth for user authentication, including login, account creation, and linking to existing accounts. Updates User model to support Google ID, adds relevant database service methods, and modifies login template to include Google sign-in. Also adds necessary configuration for Google OAuth and middleware for proxy headers.
2025-09-30 13:11:00 +01:00
Bentlybro 78fd3f6036 Increase content size limits and add logging
Raised content moderation limit from 1MB to 5MB and updated MAX_CONTENT_LENGTH to 10MB in config. Added logging for content size, missing JSON data, and validation errors to improve monitoring and debugging.
2025-09-26 15:51:58 +01:00
Bentlybro c78aca02ae Refactor database operations and add error handling utilities
Centralizes database operations in db_service for user and API key creation, and project statistics. Introduces app/utils/error_handlers.py for consistent API and web error responses, and updates API routes to use these utilities. Adds thread pool configuration for async DB operations in config. Improves maintainability and error handling across admin, dashboard, and API routes.
2025-09-15 09:48:20 +01:00
Bentlybro bc370f2cc1 Remove default admin credentials from config and compose
Eliminates hardcoded default values for ADMIN_EMAIL and ADMIN_PASSWORD in both config.py and docker-compose.yml, requiring explicit environment variable configuration for improved security.
2025-09-11 09:35:49 +01:00
Bentlybro dd455d4dd3 Refactor project and moderation rule creation logic
Project and moderation rule creation methods now return dictionaries instead of ORM instances to avoid detached instance issues. Updated related usages in dashboard route and default rules configuration to match new return types and argument names. Also added cascade delete-orphan to APIUser's project relationship.
2025-09-11 09:11:37 +01:00
Bentlybro b478a7661c Refactor moderation rules and minor code cleanup
Updated config/default_rules.py to use multi-line strings for rule descriptions and prompts for improved readability. Minor code cleanups in admin.py, openai_client.py, result_cache.py, and websocket_notifier.py for consistency and error handling.
2025-08-25 19:29:13 +01:00
Bentlybro 1c8dbe36ef Refactor default moderation rules to config module
Moved default moderation rules to a new config/default_rules.py module and replaced inline rule creation in dashboard.py with a call to create_default_rules. This improves maintainability and centralizes rule configuration.
2025-08-21 21:21:27 +01:00
Bentlybro c78017eb3d autopep8 + flake8 + isort
Reorganized imports and formatting across models and routes for better readability and maintainability. Improved consistency in model field definitions, relationship declarations, and route logic. No functional changes, but code is now easier to follow and modify.
2025-08-20 16:19:26 +01:00
Bentlybro 7a8a0bec77 Refactor AI moderation for model config and performance
Updated AI moderation pipeline to support configurable OpenAI model, context window, and output tokens via environment variables. Improved chunking logic for large content, streamlined prompt templates, and enhanced connection pooling for OpenAI client. Reduced logging verbosity in core services and added retry logic for database initialization. Updated documentation to reflect new architecture and configuration options.
2025-08-13 16:41:49 +01:00
Bentlybro 6eeddaf3be Add chunked AI moderation and DB pool config improvements
Introduces chunking logic to AI moderation for handling large content, ensuring token limits are respected and results are combined for final decisions. Adds retry logic for database initialization to handle connection pool issues, and refines SQLAlchemy pool configuration for different environments. Improves logging for rule processing and moderation orchestration. Adds tiktoken dependency for token counting.
2025-08-06 19:11:10 +01:00
Bentlybro 01c2814a5b Enhance database config and logging, add Jinja2 filter
Added USE_DIRECT_POSTGRES config to support direct PostgreSQL connections and updated requirements with supabase and psycopg2-binary. Refactored app initialization to improve database setup, admin user creation, and added a custom Jinja2 filter for object-to-dict conversion. Improved logging configuration in run.py to reduce noise from Flask and SocketIO.
2025-07-29 19:39:06 +01:00
Bentlybro 635805353d Initial code! 2025-07-29 12:52:14 +01:00