Keys are now stored as a SHA-256 digest in the existing key column (64
chars, no schema change). Lookups hash the presented key; rows still
holding plaintext are matched and upgraded to their hash on first use.
The full key is shown once at creation and the dashboard displays a
masked placeholder thereafter.
- run.py and the __init__ SocketIO comment referenced the eventlet
worker; production uses gthread. Copy-pasting the old run.py guidance
would have redeployed the exact config wsgi.py warns against.
- README/CLAUDE tech-stack tables said Flask 2.3.3 (actual: 3.1.3);
CLAUDE architecture tree listed a rule_cache.py that doesn't exist.
- Drop the per-request token count that was computed and stored on the
content object but never read (the AI paths recount as needed).
- Remove the conflicting-AI-decisions manual-review branch: the pipeline
early-exits on the first AI match, so it could never see >1 AI result.
- Read/reset the process-global cache-store counter under the cache lock
so it can't tear against the locked increment.
- Make API-key and API-user usage counters atomic (single UPDATE ...
SET col = col + 1) so concurrent moderations for the same key/user
don't lose increments to a read-modify-write race.
- Persist each moderation outcome (content status + user counters +
result rows) in a single transaction via save_moderation_outcome,
replacing three independent commits plus a no-op commit that could
leave a decided status with no result rows on partial failure.
- Include rule.action in the batched AI cache fingerprint and clear the
shared result cache when a rule is created/updated/toggled/deleted, so
edits take effect immediately instead of after the 1h TTL.
- Flag (not approve) content that exceeds the per-request chunk cap:
partial coverage must not yield a clean approval.
Concurrency / performance
- Drop the global threading.RLock in DatabaseService. It was wrapping every
_safe_execute and turning the 8-wide thread pool into a 1-wide one;
Flask-SQLAlchemy's scoped_session already gives each thread its own
session, so the lock was protecting nothing while serialising every DB
op in the process.
- moderate_content now awaits _process_rules and _handle_no_matches; both
push the blocking OpenAI call to asyncio.to_thread so the event loop
stays responsive while waiting on the model.
- DiscordNotifier shares a single module-level requests.Session (instead
of one per instance) and exposes an async wrapper that runs the POST
via asyncio.to_thread, so the moderation request doesn't pin its loop
on Discord latency / 429 retry backoff.
- load_user simplified to a single User.query.get(user_id). Previously it
spun a fresh ThreadPoolExecutor and asyncio event loop on every
authenticated request just to await an async wrapper around the same
primary-key lookup.
- ResultCache now LRU-evicts the oldest entries when at capacity with no
expired entries to free, instead of silently dropping new writes.
_cleanup_expired_entries now sets _last_cleanup_time so the periodic
throttle applies even when called from the hot-path fallback.
Data integrity
- APIUser gained UniqueConstraint(project_id, external_user_id).
get_or_create_api_user catches IntegrityError, rolls back, and reads the
winner's row so concurrent submits for the same external user no longer
produce duplicate APIUser rows that split stats.
- ProjectMember gained UniqueConstraint(project_id, user_id). The
invitation accept flow catches the IntegrityError and treats it as a
benign duplicate-click.
- New scripts/dedup_before_unique_constraints.py to merge any pre-existing
duplicate rows so the new constraints can be added cleanly. Dry-run by
default, --apply to commit.
- Fix the API-user stats detached-instance bug: increment_api_user_stats
in the DB service now does query + mutation + commit in one
_safe_execute call. Previously the orchestrator mutated a detached ORM
instance outside any session and the changes silently never persisted,
so per-user counters drifted.
- Manual-review decision flow now commits the moderation result first,
then atomically commits the APIUser stat bump in the same request
context (instead of relying on the detached-instance pattern).
Correctness
- ContentType enum is now {text, markdown, html} to match the route's
whitelist. Previously the schema declared text/image/video/audio while
the route accepted text/markdown/html, so markdown/html submissions
died with a Pydantic 400 before reaching the route.
- Project invitation emails are normalised to lowercase on store and
compared case-insensitively on accept/decline. Mixed-case stored
invitations were silently locking recipients out of accepting their
own invite.
- Admin create_user lowercases the email and requires 8-char passwords
(matching the public registration policy). The previous 6-char floor
was weaker than the public path despite admin-created accounts often
being privileged.
- AI moderator response parsing now uses (content or "").strip() — OpenAI
returns content=None on safety refusals and the raw .strip() raised
AttributeError that the surrounding except didn't catch.
Security / correctness
- Privilege escalation: rule/api-key/project-update routes now gate on
can_manage_members instead of is_member; plain members could previously
rewrite or disable moderation rules and delete API keys.
- Project.is_owner() AttributeError on non-admin API-user deletion (route
+ template) replaced with user_id comparison.
- Flask-Limiter on /auth/login, /auth/register, /auth/change-password,
and /api/moderate (per API key id).
- Remove silent SECRET_KEY fallback: docker-compose fails hard if unset;
production config validator refuses to boot without SECRET_KEY and
OPENAI_API_KEY.
- Require verified email in Google OAuth auto-link flow.
- User-supplied regex now evaluated via the regex package with a 1s
timeout, closing the ReDoS vector.
- Cap batched AI chunk count (max 40) and concurrent workers (200 -> 10).
- openai>=2.32 (from 1.55.3 so reasoning_effort kwarg works); add regex,
Flask-Limiter, gunicorn, eventlet, psycogreen; drop the stdlib-shadowing
pypi asyncio package.
Production runtime
- New wsgi.py: eventlet.monkey_patch + psycogreen.patch_psycopg so DB
queries cooperate with the event loop; sets SOCKETIO_ASYNC_MODE=eventlet
for app startup.
- Dockerfiles run as non-root app user and CMD gunicorn with
--worker-class eventlet. run.py refuses FLASK_CONFIG=production so a
misconfigured deploy fails loudly.
- cloudbuild.yaml stops managing env vars/secrets on deploy — it was
about to clobber secret-backed env vars with plain ones. Runtime config
is now owned by the Cloud Run service.
SocketIO async-mode fix (was causing full site lockup during API calls)
- async_mode picked via SOCKETIO_ASYNC_MODE env var, default "threading".
The previous sys.modules auto-detect false-positived because
flask_socketio / python-engineio import eventlet on load whenever it is
installed, pinning a non-monkey-patched eventlet server in front of
blocking stdlib I/O.
Real-time dashboard updates
- New content_received WebSocket event fires when the API accepts a
submission; UI renders a "Processing…" row immediately and updates it
in place when moderation_update arrives.
- Fix stale updated_at timestamp in moderation_update payloads.
Batched AI rule evaluation
- Replace the per-rule parallel fan-out (N OpenAI calls) with a single
batched call evaluating all AI-prompt rules at once. On a 6-rule
default project this drops safe-content cost from 5-12s to ~1-4s.
- response_format=json_object + max_completion_tokens=4000 so the model
emits strict JSON and stops at the close-brace.
- OPENAI_REASONING_EFFORT config: minimal/low/medium/high for gpt-5 /
o-series; accepts "none" on gpt-5.4. Works around the gpt-5-nano case
where reasoning tokens ate the full output budget and returned empty
responses.
Client errors
- Malformed / non-UTF-8 JSON bodies now return 400 with a clear message
about UTF-8 encoding instead of a generic 500.
Logging
- Single INFO line per moderation: the existing
"Content <id>: decision in Zs" summary.
- Per-stage timing, websocket scheduling, and batched-AI call details
moved to DEBUG so they are available when diagnosing latency without
cluttering steady-state logs.
Raised the maximum number of workers from 150 to 200 in ai_moderator.py and rule_processor.py to improve parallel processing performance for chunk and rule analysis.
Raised the maximum number of workers from 100 to 150 in ai_moderator.py and rule_processor.py to improve parallel processing performance for chunk and rule analysis.
Lowered the maximum number of workers in ThreadPoolExecutor from 200 to 100 in ai_moderator.py and rule_processor.py to optimize resource usage and prevent potential overload during parallel processing.
Raised ThreadPoolExecutor worker limits from 50 to 200 in AI moderation and rule processing to improve OpenAI API concurrency. Increased SQLAlchemy pool_size to 50, pool_timeout to 60, max_overflow to 100, and DB_THREAD_POOL_WORKERS to 100 for better database performance under load.
Raised thread pool worker limits from 10 to 50 in AI moderation and rule processing for improved parallelism under load. Increased max cache size in result_cache.py from 10k to 50k entries to handle higher concurrency. Updated OpenAI client connection pool and limits for better burst handling. Increased max chunk size for content processing to 150k characters for efficiency with large context models.
Refactored content statistics queries in dashboard routes to use a single optimized database query via db_service.get_content_counts_by_status. The new implementation reduces multiple queries to a single grouped query, improving performance when retrieving content status counts for projects.
Simplifies the logging statement when a chunk is rejected by calculating the number of remaining chunks in a variable and using it in the log message. This improves readability and maintainability.
Redirect authenticated users from login/register to dashboard. Increase AI moderator chunk size and add early exit on rejection for faster moderation. Expand result cache size and TTL, refine aggressive cleanup to only remove expired entries, and add warnings when cache is full but no expired entries are found.
Refines the cache information display by checking for stored cache results instead of relying solely on request time and cache hits. This provides more accurate feedback about cache usage in moderation requests.
Improved readability of the debug log statement by calculating the number of chunks in a separate variable before logging. No functional changes to chunking logic.
Introduces a context wrapper in AIModerator to ensure Flask application context is available during parallel chunk processing. Also updates logging for chunking and refines cache info reporting in ModerationOrchestrator for improved clarity and accuracy.
Updated ai_moderator.py to process content chunks in parallel using ThreadPoolExecutor for improved performance. Modified base.html to display the real-time connection indicator only for authenticated users.
Eliminated several info and error log statements related to content size, chunking, and API call details to reduce log verbosity and potential exposure of sensitive content. This streamlines the logging output and focuses on essential warnings and errors.
Adds a hard character count limit (50,000 chars) for content analyzed with custom prompts to address issues with inaccurate token counting. Content exceeding this limit is split by character count to ensure safe processing, improving reliability when tiktoken misestimates token usage.
Replaces token count debug logs with more detailed error logs including content length, prompt length, message lengths, content type, and a preview of the content. This aids in identifying cost-related issues with OpenAI API usage.
Introduces a check to raise a ValueError if a custom moderation prompt is too large to allow sufficient space for content and output. Adds detailed logging for prompt and content token counts to aid debugging and prevent exceeding model context window limits.
Adjusts the calculation of available tokens for content to use a larger safety margin and buffer, accounting for tokenizer inaccuracies and large custom prompts. Adds logging for unusually large prompts and enforces a hard cap on content tokens to prevent exceeding model context limits.
Changed the default value for OPENAI_MAX_OUTPUT_TOKENS from 128000 to 500 in AIModerator. This likely addresses performance or cost concerns by limiting the maximum number of output tokens generated.
Improved the token calculation logic in AIModerator to better estimate prompt and message overhead, ensuring more accurate content token allocation. Updated OPENAI_MAX_OUTPUT_TOKENS default in config.py to 500, reflecting the typical size of AI moderator responses.
Updated docstrings, comments, and UI labels to remove the '(GDPR compliance)' note from user data deletion actions. This change clarifies the purpose of these actions without referencing GDPR specifically.
Implements GDPR-compliant user data deletion in the admin and manual review interfaces, including backend service methods and UI modals for confirmation. Adds search and deletion endpoints, updates templates for user data management, and improves quick search functionality for API users. Also corrects the repository URL in the installation guide.
Refactored app initialization, WebSocket routes, and moderation notifier to use Python's logging module instead of print statements for better log management and consistency. This improves log level control and integration with Flask's logger.
Replaces static max_content_tokens logic with dynamic calculation based on prompt size, ensuring content fits within the model's context window. Updates default context window from 400,000 to 272,000 tokens in both config and service. Improves chunking and moderation handling for custom prompts and default flows.
Eliminates the RuleCache class and its usage, updating moderation logic to fetch rules directly from the database for each request. Simplifies cache management and system health reporting by removing rule cache statistics.
The stats dictionary now includes 'active_users', showing the count of users with is_active=True. This provides more detailed user metrics in the database service.
Removes the 'Moderator' field from the embed and refines the embed title generation to better reflect content status, including specific titles for rejected and flagged content.
Manual review routes, templates, and links now require a project context, restricting access and displaying flagged content per project. Navigation and Discord notification links have been updated to reflect project-specific manual review pages. API Users and review content views now validate project membership and provide improved navigation between project and manual review dashboards.
Introduces Discord webhook notifications for flagged or rejected content. Adds project-level settings for Discord notifications, including enabling/disabling and specifying a webhook URL. Implements backend logic to send notifications and a frontend UI for configuring and testing Discord webhooks in project settings.
Replaces class-level API key storage with thread-local storage to ensure thread safety when managing OpenAI client instances. Also updates client creation and usage to reference the thread-local API key and client.
Added openai.InternalServerError to exception handling and retry logic in AIModerator. This improves robustness against transient server errors from the OpenAI API by treating them similarly to connection and timeout errors.
Added GOOGLE_CLIENT_ID and GOOGLE_CLIENT_SECRET to .env.example for Google OAuth configuration. Refactored OpenAIClient to use thread-local storage for client instances, improving thread safety in multi-threaded environments.
Refactored parallel execution of AI rules to increase timeout to 60 seconds, add granular error logging for individual futures, and handle timeouts by collecting completed results and cancelling unfinished tasks. This enhances reliability and provides better diagnostics for rule processing failures.
Introduces 'error' as a content status throughout the backend and dashboard templates. Moderation errors are now tracked and saved to the database, and the UI displays error status with a distinct badge. This improves visibility and tracking of failed moderation attempts.
Introduces an in-memory error tracker service for monitoring and debugging. Integrates error tracking into API and moderation logic, logs errors with details, and displays error statistics and recent errors in the admin system health page for improved visibility and diagnostics.
Changed the CPU usage interval in system health endpoint from 0.1 to 0.5 seconds for more accurate readings. Updated RuleCache to use shared class-level caches for rules and timestamps, improving cache consistency across instances.
Introduces a system health page in the admin panel displaying process metrics, cache stats, and database pool info. Adds a /health endpoint for basic service checks, optimizes cache management in result_cache and rule_cache, and updates requirements to include psutil.
Replaces loading full user projects with a lightweight stats aggregation to prevent OOM errors on large datasets. Adds get_user_with_project_stats to database_service and updates profile route and template to use aggregated counts for projects, API keys, and content.
Integrates Google OAuth for user authentication, including login, account creation, and linking to existing accounts. Updates User model to support Google ID, adds relevant database service methods, and modifies login template to include Google sign-in. Also adds necessary configuration for Google OAuth and middleware for proxy headers.
Introduces a _retry_api_call method with exponential backoff for OpenAI API connection errors and applies it to all API calls. Error handling is updated to approve content for manual review on API errors or connection failures, with improved logging and structured error categories for moderation results.
The update_api_key_usage method now fetches a fresh APIKey instance from the database before incrementing usage, ensuring changes are committed to the correct object. This prevents issues with stale or detached instances.
Bentlybro