openclaw/clawhub
1
0
Fork 0
mirror of https://github.com/openclaw/clawhub.git synced 2026-07-22 03:35:29 -04:00
Code Issues 183 Packages Projects Releases 1 Wiki Activity

[PR #43] [MERGED] fix: rate limit and dedupe downloads #231

New Issue
Closed
opened 2026-02-15 17:16:23 -05:00 by yindo · 0 comments
yindo commented 2026-02-15 17:16:23 -05:00
Owner
Copy Link
Copy Source

📋 Pull Request Information

Original PR: https://github.com/openclaw/clawhub/pull/43
Author: @regenrek
Created: 1/26/2026
Status: ✅ Merged
Merged: 2/13/2026
Merged by: @steipete

Base: main ← Head: fix/download-rate-limit-dedupe

📝 Commits (5)

  • 36fdfe2 fix: centralize rate limiting and CF IP
  • 69bb220 fix: rate limit and dedupe downloads
  • 2a06e34 fix: allow opt-in forwarded IP headers
  • 2387867 chore: merge main into fix/download-rate-limit-dedupe
  • 39be560 docs: acknowledge download hardening contribution (#43)

📊 Changes

10 files changed (+318 additions, -11 deletions)

View changed files

📝 CHANGELOG.md (+1 -0)
📝 convex/_generated/api.d.ts (+2 -0)
📝 convex/crons.ts (+7 -0)
➕ convex/downloads.test.ts (+12 -0)
📝 convex/downloads.ts (+90 -6)
📝 convex/httpApiV1.ts (+18 -5)
➕ convex/lib/httpRateLimit.test.ts (+35 -0)
➕ convex/lib/httpRateLimit.ts (+137 -0)
📝 convex/schema.ts (+10 -0)
📝 docs/http-api.md (+6 -0)

📄 Description

summary

  • Add download rate limiting, per-IP/day dedupe, and cleanup cron.

motivation

  • Stop download count inflation and keep dedupe storage bounded.

what's included

  • Download-specific rate limit tier.
  • Dedupe table + index + cron pruning.
  • Internal mutation to record unique downloads.
  • Opt-in forwarded IP header support via TRUST_FORWARDED_IPS for non-Cloudflare deployments.

what's not included

  • UI changes (file viewer / warnings).

tests

  • bun run test
  • bun run lint

affected files

  • convex/downloads.ts
  • convex/downloads.test.ts
  • convex/schema.ts
  • convex/crons.ts
  • convex/lib/httpRateLimit.ts

prompt

# ClawdHub Security Hardening

## Goal & Success Criteria

- Block download inflation: rate limit + per‑IP/day dedupe on ZIP downloads.
- IP spoofing fixed: trust only cf-connecting-ip.
- Files tab shows full file viewer + warnings for dangerous commands.
- Each PR has Conventional Commits, full test suite runs, PR opened from regenrek fork.

## Non‑goals / Out of Scope

- Replace download stats with installs.
- Auth‑gated downloads or paid access.
- Deep static analysis beyond warning heuristics.

## Assumptions

- Rate limits: new “download” tier tighter than “read”.
- IP trust: CF‑only, no fallback.
- PRs live on regenrek fork, not upstream.

## Proposed Solution

- Single canonical rate‑limit/IP module (Convex best‑practice: no duplicated logic).
- Dedupe table keyed by hashed IP + skill + day bucket; increment only once.
- UI file viewer loads via existing getFileText; warnings from regex scan.

### Alternatives Considered

- Reuse “read” limits in convex/httpApiV1.ts:634 — too permissive.
- Fallback to x-forwarded-for — violates CF‑only requirement.
- Store raw IPs — privacy risk.

## System Design

- Increment flow: new mutation recordDownload handles dedupe + stats increment atomically.
- Cleanup: cron job prunes dedupe rows older than N days.
- UI: SkillDetailPage Files tab extracted to SkillFilesPanel with viewer + warnings.

## Interfaces & Data Contracts

- recordDownload mutation args: { skillId: Id<'skills'>, ipHash?: string, dayStart: number }.
- downloadDedupes schema: { skillId, ipHash, dayStart, createdAt }.
- Rate limit config includes download: { ip, key } in shared helper.

## Execution Details

PR 2 — Download Rate Limit + Dedupe (fix)

- Add downloadDedupes table + index in convex/schema.ts:301.
- Add recordDownload mutation and optional cleanup internal mutation.
- Add cron in convex/crons.ts:6 to prune old dedupe rows.
- Update convex/downloads.ts:7 to call applyRateLimit (download tier) and recordDownload.

## Testing & Quality

- Full suite per PR: bun run test and bun run lint.
- Add unit tests for dedupe/rate limit in convex/downloads.test.ts.
- Extend handler tests to cover CF‑only IP logic.

## Risks & Mitigations

- Missing CF header → “unknown” key. Mitigate by documenting requirement.
- Dedupe table growth → cron pruning.
- Viewer perf → relies on existing 200KB cap.

🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/openclaw/clawhub/pull/43 **Author:** [@regenrek](https://github.com/regenrek) **Created:** 1/26/2026 **Status:** ✅ Merged **Merged:** 2/13/2026 **Merged by:** [@steipete](https://github.com/steipete) **Base:** `main` ← **Head:** `fix/download-rate-limit-dedupe` --- ### 📝 Commits (5) - [`36fdfe2`](https://github.com/openclaw/clawhub/commit/36fdfe2f806d56b48b69337642947ec9944cf478) fix: centralize rate limiting and CF IP - [`69bb220`](https://github.com/openclaw/clawhub/commit/69bb2207f770da86707b72897cb57ead2aa423b2) fix: rate limit and dedupe downloads - [`2a06e34`](https://github.com/openclaw/clawhub/commit/2a06e34b5b6dba3966306297d135273ebd3bde81) fix: allow opt-in forwarded IP headers - [`2387867`](https://github.com/openclaw/clawhub/commit/2387867e9f9f030c675d994169bce5b9e4455038) chore: merge main into fix/download-rate-limit-dedupe - [`39be560`](https://github.com/openclaw/clawhub/commit/39be560207b50a18d3d4f750f366c2a721b12686) docs: acknowledge download hardening contribution (#43) ### 📊 Changes **10 files changed** (+318 additions, -11 deletions) <details> <summary>View changed files</summary> 📝 `CHANGELOG.md` (+1 -0) 📝 `convex/_generated/api.d.ts` (+2 -0) 📝 `convex/crons.ts` (+7 -0) ➕ `convex/downloads.test.ts` (+12 -0) 📝 `convex/downloads.ts` (+90 -6) 📝 `convex/httpApiV1.ts` (+18 -5) ➕ `convex/lib/httpRateLimit.test.ts` (+35 -0) ➕ `convex/lib/httpRateLimit.ts` (+137 -0) 📝 `convex/schema.ts` (+10 -0) 📝 `docs/http-api.md` (+6 -0) </details> ### 📄 Description ### summary - Add download rate limiting, per-IP/day dedupe, and cleanup cron. ### motivation - Stop download count inflation and keep dedupe storage bounded. ### what's included - Download-specific rate limit tier. - Dedupe table + index + cron pruning. - Internal mutation to record unique downloads. - Opt-in forwarded IP header support via `TRUST_FORWARDED_IPS` for non-Cloudflare deployments. ### what's not included - UI changes (file viewer / warnings). ### tests - `bun run test` - `bun run lint` ### affected files - `convex/downloads.ts` - `convex/downloads.test.ts` - `convex/schema.ts` - `convex/crons.ts` - `convex/lib/httpRateLimit.ts` ### prompt ``` # ClawdHub Security Hardening ## Goal & Success Criteria - Block download inflation: rate limit + per‑IP/day dedupe on ZIP downloads. - IP spoofing fixed: trust only cf-connecting-ip. - Files tab shows full file viewer + warnings for dangerous commands. - Each PR has Conventional Commits, full test suite runs, PR opened from regenrek fork. ## Non‑goals / Out of Scope - Replace download stats with installs. - Auth‑gated downloads or paid access. - Deep static analysis beyond warning heuristics. ## Assumptions - Rate limits: new “download” tier tighter than “read”. - IP trust: CF‑only, no fallback. - PRs live on regenrek fork, not upstream. ## Proposed Solution - Single canonical rate‑limit/IP module (Convex best‑practice: no duplicated logic). - Dedupe table keyed by hashed IP + skill + day bucket; increment only once. - UI file viewer loads via existing getFileText; warnings from regex scan. ### Alternatives Considered - Reuse “read” limits in convex/httpApiV1.ts:634 — too permissive. - Fallback to x-forwarded-for — violates CF‑only requirement. - Store raw IPs — privacy risk. ## System Design - Increment flow: new mutation recordDownload handles dedupe + stats increment atomically. - Cleanup: cron job prunes dedupe rows older than N days. - UI: SkillDetailPage Files tab extracted to SkillFilesPanel with viewer + warnings. ## Interfaces & Data Contracts - recordDownload mutation args: { skillId: Id<'skills'>, ipHash?: string, dayStart: number }. - downloadDedupes schema: { skillId, ipHash, dayStart, createdAt }. - Rate limit config includes download: { ip, key } in shared helper. ## Execution Details PR 2 — Download Rate Limit + Dedupe (fix) - Add downloadDedupes table + index in convex/schema.ts:301. - Add recordDownload mutation and optional cleanup internal mutation. - Add cron in convex/crons.ts:6 to prune old dedupe rows. - Update convex/downloads.ts:7 to call applyRateLimit (download tier) and recordDownload. ## Testing & Quality - Full suite per PR: bun run test and bun run lint. - Add unit tests for dedupe/rate limit in convex/downloads.test.ts. - Extend handler tests to cover CF‑only IP logic. ## Risks & Mitigations - Missing CF header → “unknown” key. Mitigate by documenting requirement. - Dedupe table growth → cron pruning. - Viewer perf → relies on existing 200KB cap. ``` --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
yindo added the pull-request label 2026-02-15 17:16:23 -05:00
yindo closed this issue 2026-02-15 17:16:23 -05:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
bug
pull-request
Mirrored from GitHub Pull Request
No Label pull-request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
yindo
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: openclaw/clawhub#231
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?