Files
Bentlybro b92bdf694e Phase 1: architectural fixes (concurrency, async/sync, data integrity)
Concurrency / performance
- Drop the global threading.RLock in DatabaseService. It was wrapping every
  _safe_execute and turning the 8-wide thread pool into a 1-wide one;
  Flask-SQLAlchemy's scoped_session already gives each thread its own
  session, so the lock was protecting nothing while serialising every DB
  op in the process.
- moderate_content now awaits _process_rules and _handle_no_matches; both
  push the blocking OpenAI call to asyncio.to_thread so the event loop
  stays responsive while waiting on the model.
- DiscordNotifier shares a single module-level requests.Session (instead
  of one per instance) and exposes an async wrapper that runs the POST
  via asyncio.to_thread, so the moderation request doesn't pin its loop
  on Discord latency / 429 retry backoff.
- load_user simplified to a single User.query.get(user_id). Previously it
  spun a fresh ThreadPoolExecutor and asyncio event loop on every
  authenticated request just to await an async wrapper around the same
  primary-key lookup.
- ResultCache now LRU-evicts the oldest entries when at capacity with no
  expired entries to free, instead of silently dropping new writes.
  _cleanup_expired_entries now sets _last_cleanup_time so the periodic
  throttle applies even when called from the hot-path fallback.

Data integrity
- APIUser gained UniqueConstraint(project_id, external_user_id).
  get_or_create_api_user catches IntegrityError, rolls back, and reads the
  winner's row so concurrent submits for the same external user no longer
  produce duplicate APIUser rows that split stats.
- ProjectMember gained UniqueConstraint(project_id, user_id). The
  invitation accept flow catches the IntegrityError and treats it as a
  benign duplicate-click.
- New scripts/dedup_before_unique_constraints.py to merge any pre-existing
  duplicate rows so the new constraints can be added cleanly. Dry-run by
  default, --apply to commit.
- Fix the API-user stats detached-instance bug: increment_api_user_stats
  in the DB service now does query + mutation + commit in one
  _safe_execute call. Previously the orchestrator mutated a detached ORM
  instance outside any session and the changes silently never persisted,
  so per-user counters drifted.
- Manual-review decision flow now commits the moderation result first,
  then atomically commits the APIUser stat bump in the same request
  context (instead of relying on the detached-instance pattern).

Correctness
- ContentType enum is now {text, markdown, html} to match the route's
  whitelist. Previously the schema declared text/image/video/audio while
  the route accepted text/markdown/html, so markdown/html submissions
  died with a Pydantic 400 before reaching the route.
- Project invitation emails are normalised to lowercase on store and
  compared case-insensitively on accept/decline. Mixed-case stored
  invitations were silently locking recipients out of accepting their
  own invite.
- Admin create_user lowercases the email and requires 8-char passwords
  (matching the public registration policy). The previous 6-char floor
  was weaker than the public path despite admin-created accounts often
  being privileged.
- AI moderator response parsing now uses (content or "").strip() — OpenAI
  returns content=None on safety refusals and the raw .strip() raised
  AttributeError that the surrounding except didn't catch.
2026-04-21 16:14:43 +01:00

142 lines
5.1 KiB
Python

import uuid
from datetime import datetime
from sqlalchemy import func
from app import db
class APIUser(db.Model):
__tablename__ = 'api_users'
# An external user is uniquely identified by (project_id, external_user_id).
# Without this constraint two concurrent /api/moderate requests for the
# same external user could both miss the SELECT and both INSERT, producing
# duplicate rows that split stats across them.
__table_args__ = (
db.UniqueConstraint('project_id', 'external_user_id',
name='uq_api_users_project_external'),
)
id = db.Column(db.String(36), primary_key=True,
default=lambda: str(uuid.uuid4()))
# The user ID passed in metadata
external_user_id = db.Column(db.String(255), nullable=False)
project_id = db.Column(db.String(36), db.ForeignKey(
'projects.id'), nullable=False)
first_seen = db.Column(db.DateTime, default=datetime.utcnow)
last_seen = db.Column(db.DateTime, default=datetime.utcnow)
total_requests = db.Column(db.Integer, default=0)
approved_count = db.Column(db.Integer, default=0)
rejected_count = db.Column(db.Integer, default=0)
flagged_count = db.Column(db.Integer, default=0)
created_at = db.Column(db.DateTime, default=datetime.utcnow)
updated_at = db.Column(
db.DateTime, default=datetime.utcnow, onupdate=datetime.utcnow)
# Relationships
project = db.relationship('Project', backref=db.backref('api_users', cascade='all, delete-orphan'))
content_items = db.relationship('Content', backref='api_user', lazy=True)
def to_dict(self):
return {
'id': self.id,
'external_user_id': self.external_user_id,
'project_id': self.project_id,
'first_seen': self.first_seen.isoformat(),
'last_seen': self.last_seen.isoformat(),
'total_requests': self.total_requests,
'approved_count': self.approved_count,
'rejected_count': self.rejected_count,
'flagged_count': self.flagged_count,
'approval_rate': (self.approved_count / self.total_requests * 100) if self.total_requests > 0 else 0,
'created_at': self.created_at.isoformat(),
'updated_at': self.updated_at.isoformat()
}
def update_stats(self, status):
"""Update user statistics based on moderation result"""
self.total_requests += 1
self.last_seen = datetime.utcnow()
if status == 'approved':
self.approved_count += 1
elif status == 'rejected':
self.rejected_count += 1
elif status == 'flagged':
self.flagged_count += 1
def get_current_stats(self):
"""Get current stats by counting from database with caching"""
from app.models.content import Content
# Use cached stats if available in this request
if hasattr(self, '_cached_stats'):
return self._cached_stats
# Use a single efficient query to get all counts at once
result = db.session.query(
func.count(Content.id).label('total'),
func.sum(db.case((Content.status == 'approved', 1), else_=0)).label('approved'),
func.sum(db.case((Content.status == 'rejected', 1), else_=0)).label('rejected'),
func.sum(db.case((Content.status == 'flagged', 1), else_=0)).label('flagged')
).filter(Content.api_user_id == self.id).first()
total_content = result.total or 0
approved = result.approved or 0
rejected = result.rejected or 0
flagged = result.flagged or 0
stats = {
'total_requests': total_content,
'approved_count': approved,
'rejected_count': rejected,
'flagged_count': flagged,
'approval_rate': (approved / total_content * 100) if total_content > 0 else 0
}
# Cache the stats for this instance
self._cached_stats = stats
return stats
@property
def approval_rate(self):
stats = self.get_current_stats()
return stats['approval_rate']
@property
def current_total_requests(self):
stats = self.get_current_stats()
return stats['total_requests']
@property
def current_approved_count(self):
stats = self.get_current_stats()
return stats['approved_count']
@property
def current_rejected_count(self):
stats = self.get_current_stats()
return stats['rejected_count']
@property
def current_flagged_count(self):
stats = self.get_current_stats()
return stats['flagged_count']
@property
def days_active(self):
"""Calculate days active more accurately"""
if not self.last_seen or not self.first_seen:
return 0
# Get date components only (ignore time)
first_date = self.first_seen.date()
last_date = self.last_seen.date()
# Calculate difference in days and add 1 (inclusive)
delta = (last_date - first_date).days + 1
return max(1, delta) # Minimum 1 day
def __repr__(self):
return f'<APIUser {self.external_user_id}>'