Files
Bentlybro 478ca18ad8 Phase 0 hardening + batched AI moderation + websocket/perf fixes
Security / correctness
- Privilege escalation: rule/api-key/project-update routes now gate on
  can_manage_members instead of is_member; plain members could previously
  rewrite or disable moderation rules and delete API keys.
- Project.is_owner() AttributeError on non-admin API-user deletion (route
  + template) replaced with user_id comparison.
- Flask-Limiter on /auth/login, /auth/register, /auth/change-password,
  and /api/moderate (per API key id).
- Remove silent SECRET_KEY fallback: docker-compose fails hard if unset;
  production config validator refuses to boot without SECRET_KEY and
  OPENAI_API_KEY.
- Require verified email in Google OAuth auto-link flow.
- User-supplied regex now evaluated via the regex package with a 1s
  timeout, closing the ReDoS vector.
- Cap batched AI chunk count (max 40) and concurrent workers (200 -> 10).
- openai>=2.32 (from 1.55.3 so reasoning_effort kwarg works); add regex,
  Flask-Limiter, gunicorn, eventlet, psycogreen; drop the stdlib-shadowing
  pypi asyncio package.

Production runtime
- New wsgi.py: eventlet.monkey_patch + psycogreen.patch_psycopg so DB
  queries cooperate with the event loop; sets SOCKETIO_ASYNC_MODE=eventlet
  for app startup.
- Dockerfiles run as non-root app user and CMD gunicorn with
  --worker-class eventlet. run.py refuses FLASK_CONFIG=production so a
  misconfigured deploy fails loudly.
- cloudbuild.yaml stops managing env vars/secrets on deploy — it was
  about to clobber secret-backed env vars with plain ones. Runtime config
  is now owned by the Cloud Run service.

SocketIO async-mode fix (was causing full site lockup during API calls)
- async_mode picked via SOCKETIO_ASYNC_MODE env var, default "threading".
  The previous sys.modules auto-detect false-positived because
  flask_socketio / python-engineio import eventlet on load whenever it is
  installed, pinning a non-monkey-patched eventlet server in front of
  blocking stdlib I/O.

Real-time dashboard updates
- New content_received WebSocket event fires when the API accepts a
  submission; UI renders a "Processing…" row immediately and updates it
  in place when moderation_update arrives.
- Fix stale updated_at timestamp in moderation_update payloads.

Batched AI rule evaluation
- Replace the per-rule parallel fan-out (N OpenAI calls) with a single
  batched call evaluating all AI-prompt rules at once. On a 6-rule
  default project this drops safe-content cost from 5-12s to ~1-4s.
- response_format=json_object + max_completion_tokens=4000 so the model
  emits strict JSON and stops at the close-brace.
- OPENAI_REASONING_EFFORT config: minimal/low/medium/high for gpt-5 /
  o-series; accepts "none" on gpt-5.4. Works around the gpt-5-nano case
  where reasoning tokens ate the full output budget and returned empty
  responses.

Client errors
- Malformed / non-UTF-8 JSON bodies now return 400 with a clear message
  about UTF-8 encoding instead of a generic 500.

Logging
- Single INFO line per moderation: the existing
  "Content <id>: decision in Zs" summary.
- Per-stage timing, websocket scheduling, and batched-AI call details
  moved to DEBUG so they are available when diagnosing latency without
  cluttering steady-state logs.
2026-04-21 16:01:18 +01:00

238 lines
8.2 KiB
Python

"""
Error handling utilities for consistent error responses
"""
import logging
from functools import wraps
from flask import flash, jsonify, redirect, request, url_for
from pydantic import BaseModel, ValidationError
from werkzeug.exceptions import BadRequest
logger = logging.getLogger(__name__)
class APIError(Exception):
"""Custom exception for API errors with structured response data"""
def __init__(self, message, status_code=400, error_code=None, details=None):
super().__init__(message)
self.message = message
self.status_code = status_code
self.error_code = error_code
self.details = details or {}
def api_error_response(message, status_code=400, error_code=None, details=None):
"""Generate standardized API error response"""
response_data = {
'success': False,
'error': message
}
if error_code:
response_data['error_code'] = error_code
if details:
response_data['details'] = details
return jsonify(response_data), status_code
def api_success_response(data=None, message=None):
"""Generate standardized API success response"""
response_data = {'success': True}
if message:
response_data['message'] = message
if data:
response_data.update(data)
return jsonify(response_data)
def handle_api_error(f):
"""Decorator to handle API errors consistently"""
@wraps(f)
async def decorated_function(*args, **kwargs):
try:
return await f(*args, **kwargs)
except APIError as e:
logger.warning(f"API error in {f.__name__}: {e.message}")
return api_error_response(
e.message,
e.status_code,
e.error_code,
e.details
)
except Exception as e:
# Log full exception details with traceback
logger.error(f"Unexpected error in {f.__name__}: {str(e)}", exc_info=True)
return api_error_response(
"An internal server error occurred",
500,
error_code="INTERNAL_ERROR"
)
return decorated_function
def web_error_handler(f):
"""Decorator to handle web route errors consistently"""
@wraps(f)
async def decorated_function(*args, **kwargs):
try:
return await f(*args, **kwargs)
except Exception as e:
logger.error(f"Error in {f.__name__}: {str(e)}")
flash("An unexpected error occurred. Please try again.", 'error')
# Try to redirect to a sensible default
return redirect(url_for('dashboard.index'))
return decorated_function
def validate_required_fields(data, required_fields):
"""Validate required fields are present in data"""
missing_fields = []
for field in required_fields:
if field not in data or data[field] is None or (isinstance(data[field], str) and not data[field].strip()):
missing_fields.append(field)
if missing_fields:
raise APIError(
f"Missing required fields: {', '.join(missing_fields)}",
status_code=400,
error_code="MISSING_REQUIRED_FIELDS",
details={"missing_fields": missing_fields}
)
def validate_json_request(schema_class: BaseModel):
"""
Decorator to validate JSON request data against a Pydantic schema
Adds 'validated_data' to the route function's keyword arguments
"""
def decorator(f):
@wraps(f)
async def decorated_function(*args, **kwargs):
try:
# Get JSON data from request
json_data = request.get_json()
if json_data is None:
logger.warning(f"Missing JSON data in request to {request.endpoint}")
return api_error_response(
"JSON data required",
400,
"MISSING_JSON_DATA"
)
# Validate against schema
validated_data = schema_class(**json_data)
# Add validated data to kwargs
kwargs['validated_data'] = validated_data
return await f(*args, **kwargs)
except ValidationError as e:
# Format Pydantic validation errors
error_details = []
for error in e.errors():
field = '.'.join(str(loc) for loc in error['loc'])
error_details.append(f"{field}: {error['msg']}")
logger.warning(f"Validation error in {request.endpoint}: {error_details}")
return api_error_response(
"Invalid input data",
400,
"VALIDATION_ERROR",
{"field_errors": error_details}
)
except BadRequest as e:
# werkzeug.exceptions.BadRequest fires when request.get_json()
# can't parse the body — malformed JSON, invalid UTF-8, etc.
# These are client errors, not server errors. Surface a useful
# 400 so clients can debug their encoding/payload instead of
# chasing a phantom "internal server error".
description = getattr(e, 'description', str(e))
logger.warning(
f"Malformed request body in {request.endpoint}: {description}"
)
message = "Malformed JSON body"
if "codec can't decode" in str(description):
message = (
"Request body is not valid UTF-8. Encode your JSON "
"payload as UTF-8 (common culprit: Windows CP-1252 "
"em dash / smart quotes)."
)
return api_error_response(
message,
400,
"MALFORMED_REQUEST_BODY",
{"detail": str(description)}
)
except Exception as e:
logger.error(
f"Unexpected error in validate_json_request for "
f"{f.__name__}: {str(e)}",
exc_info=True,
)
return api_error_response(
"An internal server error occurred",
500,
"INTERNAL_ERROR",
)
return decorated_function
return decorator
def validate_query_params(schema_class: BaseModel):
"""
Decorator to validate query parameters against a Pydantic schema
Adds 'validated_params' to the route function's keyword arguments
"""
def decorator(f):
@wraps(f)
async def decorated_function(*args, **kwargs):
try:
# Get query parameters
query_data = request.args.to_dict()
# Convert common query param types
for key, value in query_data.items():
# Try to convert numbers
if value.isdigit():
query_data[key] = int(value)
elif value.lower() in ('true', 'false'):
query_data[key] = value.lower() == 'true'
# Validate against schema
validated_params = schema_class(**query_data)
# Add validated params to kwargs
kwargs['validated_params'] = validated_params
return await f(*args, **kwargs)
except ValidationError as e:
# Format Pydantic validation errors
error_details = []
for error in e.errors():
field = '.'.join(str(loc) for loc in error['loc'])
error_details.append(f"{field}: {error['msg']}")
return api_error_response(
"Invalid query parameters",
400,
"VALIDATION_ERROR",
{"field_errors": error_details}
)
except Exception as e:
logger.error(f"Query param validation error in {f.__name__}: {str(e)}")
return api_error_response("Internal server error", 500)
return decorated_function
return decorator