mirror of
https://github.com/Significant-Gravitas/AutoGPT.git
synced 2026-11-09 04:49:50 -10:00
bba2ff341a
### Why / What / How **Why.** Since Oct 16, 954 people signed up, 444 saw the paywall, 209 opened Stripe and 29 started a trial ([funnel](https://eu.posthog.com/project/117338/insights/ugcxrNQU)). 68% of the people who reached the hire step hired nobody, and only 3 people have ever clicked a suggested first task on Home. The hire step asked people to pick experts before they had seen what those experts would do, the paywall then asked for a card without a reason, and Home's suggestions had no owner and no order. **What.** The paywall becomes the plan: what Otto heard, the first three tasks ranked, each owned by the expert who will do it, beside the 7-day trial. The hire step is gone. The plan's team is hired on the preparing screen once the trial is confirmed, so a user who abandons Stripe hires nobody. Home opens on the same tasks as a list: "Start here" on the first open row, each row naming who takes it, x removes a task, and Hide / Show collapses the list. **How.** - **Backend.** After the greeting and the team are stored, one TypeSafe Jev call ranks the greeting's tasks and picks an owner for each (a call to TypeSafe's hosted model, which carries the tasks, the team and up to 6,000 characters of the transcript); a deterministic fallback keeps the greeting's order and matches owners by icon and title. The team is capped to the three people the step shows. `GET /onboarding/brain-dump/plan` serves the plan (or a role plan for a skipped, failed or empty brain dump), `POST /onboarding/brain-dump/plan/settle` makes the plan the step shows the take's plan when the step stops waiting, and `POST /onboarding/brain-dump/plan/hire` hires its team idempotently. A suggested task's `expert_id` is read fresh: an open task's from the owner's active hire, a started task's from the chat it opened. - **Onboarding.** `PlanTrialStep` (`PlanPanel` beside `TrialPanel`, or `PaidPanel` without a trial offer, or the plan alone with Continue when there is no paywall) replaces the hire step and the trial welcome. `usePlanHire` hires on the preparing screen after the trial (or plan) is confirmed. - **Home.** `OnboardingIntroCard` goes back to `SuggestedTaskList`; the #16236 cards are deleted. A task is sent to its owner the way the recipient picker sends to an expert, so its chat is the expert's. ### Progress - [x] **Phase 0**: branch, baseline counts, before screenshots. Frontend `vitest` 12,282 passed + 4 expected failures (1,127 files); backend `onboarding_dump` + `experts` 904 passed, 29 skipped. - [x] **Phase 1** `bd3dc3cc feat(backend/onboarding): plan with ranked, owned tasks and a hire endpoint` (+ `f879fb20` review follow-ups). - [x] **Phase 2** `ba232616 feat(frontend/onboarding): plan + trial step replaces hire and subscription` (+ `7fb3dce4`, the plan read waits for a Skip still in flight). - [x] **Phase 3** `61708f3c feat(frontend/onboarding): hire the plan's team after the trial is confirmed`. - [x] **Phase 4** `130b92ad feat(frontend/home): suggested tasks ranked and owned by experts, list restored, remove and collapse` (+ `3782dfc4`, a started task names the expert its chat is with). - [x] **Phase 5** `e075809d` tracking audit fixes and the tracking plan, `177f4fad` the Playwright helper, `044c4fdb` edge-case coverage (a Jev outage logs one warning, an unset key warns once at startup, an empty team hires nobody, the walkthrough lands on the pill's row), `6eda537b` focus after Remove never drops to the page (found by the final review), `8badd23e` one ref write out of render (a lint warning this PR added), `d113cf2c` the plan's hire checks the paywall and the plan read is listed open (CI's `paywall_routes_test` requires one or the other for every route). - [x] **The plan step's wait** (found testing the preview) `9532c2b7` the step stops waiting on work the plan does not use and a plan it gives up on is the plan, `2abf8821` the settle route listed open while paywalled, `418ae59c` review fixes. See "The plan step's wait" below. - [x] **Merged with dev again** `8d81826a`, for dev's expert-roster test update (#16443). Checked on the merge: the exported API schema matches `openapi.json`; frontend types and lint pass, and onboarding, Home's task list and analytics run 1,053 tests, all passing; backend `experts` + `onboarding_dump` + `paywall_routes_test` has 1,170 passed, and the one failure needs ClamAV, which this environment does not run. - [x] **Kept current with dev** `075f17a9` (checked: schema matches `openapi.json`, frontend types and lint pass, 506 test files / 6,510 tests pass in onboarding, copilot, services and hooks, backend `onboarding_dump` + `paywall_routes_test` 637 passed), `538606ae`, `28e82949`, `6a8e6052`, `21ec4514`, `89750735`, `28a7f663`, `9a7fa159`, and `c365e277` (a port of #16489, dev's fix for the MCP tests it had broken). `9a7fa159` took dev's colour-role refactor (#16511): the hire-step Playwright fixture it recoloured stays deleted, and its `ds-colour-exempt.json` entry for `OttoCover.tsx`, a file this branch deletes, is removed. Test-only ports for CI flakes outside this PR: `52a5f637` (#16449, the PostHog consent test's capture interval) and `36527ca2` (#16566, the shared auth mock's session exports; #16565, the external API concurrency test's Redis timeout). They no-op once dev carries those PRs. The conflicts: dev's Langfuse prompt client in `intro.py` (#16026); a storage key both sides added (#16527); dev's Text, Icon and Button props on the Suggested Tasks components this branch rewrote (#16513: kept this branch's versions, moved two colour classes to `tone`, regenerated the design-system baseline); dev's `ENABLE_PLATFORM_PAYMENT` setting (#16459); and dev's removal of unused exports (#16163), which dropped `SELF_HOST_STEPS` from the onboarding store while this branch had changed it (`075f17a9`: kept this branch's plan step in `NO_PAYWALL_STEPS` and left `SELF_HOST_STEPS` deleted, since nothing imports it). Dev's palette removal (#16612) also added a lint-rule test case that relies on the exemption for `OttoCover.tsx`, which this branch deletes; `000a3f88` points that case at `CodeRenderer.tsx`'s allow-list instead. The paywall test's PostHog mock now returns the event, as dev's own tests do since #16518. - Since #16459, the no-paywall version of the plan step is what an install gets while `ENABLE_PLATFORM_PAYMENT` is off, which is the default. The plan step is step 5 either way; only the paywall beside it changes. - Checked on the last merge: the exported API schema matches `openapi.json`; frontend types and lint pass, and onboarding, copilot, services, hooks and the paywall gate run 485 test files (6,442 tests), all passing; backend `onboarding_dump`, `experts`, `paywall_routes_test` and dev's payment-setting and trial tests: 1,326 passed, and the one failure needs ClamAV. - [x] **The greeting has a fast backup, so the brain dump shapes the plan** (found testing the preview) `2c416169`, with review fixes in `a50305f3`. After a real brain dump, the plan step kept showing the role's starter tasks. The greeting, which writes the plan's tasks, either failed (the generic greeting) or was still running when the step stopped waiting at 30 s. Sonnet writes long answers for a long take, and this PR turned its thinking off, which is proven on Haiku 5.5 but not on Sonnet 5.5. `greeting_race.py` runs Sonnet first, and Haiku 5.5, which the team job already calls the same way, joins it. Haiku starts at once when Sonnet fails or gives an unusable answer, or alongside Sonnet once it has run 12 s. The first full answer wins and the other call is cancelled. Each attempt logs its model and why it failed, and the winner logs its model and time. Sonnet is now asked for by OpenRouter's own id, `anthropic/claude-sonnet-5.5`. A two-lens adversarial review confirmed three defects, now fixed in `a50305f3`: a malformed reply could escape the race and fail the dump, a test passed for the wrong reason, and the env example was stale. Checked: `onboarding_dump` + `paywall_routes_test`, 654 passed. - [x] **Every plan starts with a team of three** (asked for after testing the preview) `d6ea5524`. Without a brain dump, the team came from the wizard role alone: two experts for Sales/BD, Operations and HR/People, none for Other. Each role now names three skills that cover its starter tasks. Sales/BD adds Quinn for the account research; Operations and HR/People add Kai, for the self-chasing tracker and the interview scheduling. A brain-dump team with fewer than three is topped up with the wizard's picks after the model's own. Checked: `onboarding_dump`, `experts` and `paywall_routes_test`, 1,255 passed. - [x] **Tasks go to the experts whose roles fit them** (found testing the preview) `29476fbb`, with review fixes in `676a6b34`. A product manager's plan named Alex, Quinn and Nadia as the team and every task as Otto's. The owner match compared exact role strings ("research", "bookkeeping") with the catalog's roles, which now read "Research, Data & KPI Analysis", "Market & Competitor Intelligence" and "Product", so nobody matched. Now a role's skills come from the words it contains (`role_skills.py`), and the role sets, the title and icon hints and the fallback team all name skills. Against the live roster, every role's starter plan is its team's work: Product gets Alex, Alex, Nadia, Quinn, Alex. The one exception is Engineering's customer-questions task, which no engineering expert covers. Among teammates with a skill, the most focused one takes the task, so Maya (Marketing) writes the newsletter, not Clip (Motion Graphics). Title hints read the task's subject, not a word in passing: a "feature" in a marketing title or a sales "pitch" no longer decides. New hints cover CRM, contracts, security, bookkeeping and fundraising work. A task whose best match is cut from the three-person team goes to the next match who stays, not to Otto. An empty seat goes to a later task's owner before someone with no task. A four-lens adversarial review checked this against the live roster, with each finding verified. The greeting's failure log now names the exception type, because a timeout's message is empty. - Checked: the onboarding suite (613 tests) passes; experts + `paywall_routes_test` pass (638 passed, 30 skipped); pyright is clean on the changed package. The new `role_skills_test.py` pins owners and teams against the catalog's 33 roles. See "The greeting fell back on the preview" below. - [x] **Review round (kcze)** `4015926a`, then merged with dev `6c365f60`. 18 of the 19 findings fixed, each with a test that failed first; `d286bd1f` fixes the 19th, with `e438b230` and `4ea3cc78` correcting that commit: - Settling keeps the tasks the greeting already wrote instead of swapping in the role plan, and the role plan's template read is cached for 60 s. - `plan_experts_hired` moved to the backend and counts only experts that joined in that call. - Preparing retries a hire that got no answer inside its 15 s window, toasts who couldn't join, refreshes Home's task owners, and waits (bounded) for a ready plan when none was shown. - The plan step bounds and retries its settle, keeps polling behind a fallback the server never stored, ignores stale answers, and remembers only stored plans. - Checkout is back to "within a trial usage allowance". - Home holds owned tasks while the roster read fails, doesn't carry a new-chat mode over to another recipient, and only restores a recipient it changed. - `d286bd1f`, `e438b230` and `4ea3cc78` then cover the last finding and two gaps. `hire_plan` refuses a plan that `settle_plan` could not settle (409, nobody hired, no event) and the client retries it inside its window. This replaces the earlier choice to hire anyway: a user with nobody is now the cost only if every retry loses too, and then the toast says so. A refresh mid-hire re-asks once the first call's time is up, with its own retry window: the answer does the hire if the first call was lost, says who did not join, and refreshes Home. If no answer comes, it shows no toast, because the first call may have worked. Home's owners refresh whatever the answer. A hire nobody could join is not tried again, because the answer can't tell a blip from the expert limit. `plan_experts_hired` counts a failure once, through a Redis claim that gives up after 1 s and reports anyway. - Left as is: when no answer ever comes on the first hire, the toast names the whole plan even though a lost answer may have hired them (rare, and Home's roster still refreshes). A refresh whose first call was lost and whose re-ask never gets an answer leaves nobody hired and shows no toast; only Sentry sees it (rarer still). Failed whole calls (a 409 or 5xx) send no `plan_experts_hired`, as the tracking plan says. - Checked on the merge: the exported API schema matches `openapi.json`; frontend types and lint pass; backend `onboarding_dump`, experts routes and db, and `posthog_events` tests pass (1,025 passed, 19 skipped); frontend Vitest runs 1,220 files (13,685 passed, 4 expected failures). - [x] **Review round (Capy, CodeRabbit)** `32bac59b`, after merging dev in `0d7379cf`. Each of the 31 findings was checked against the code: 8 are fixed, and the other 23 are answered on the PR with the reason each stays as is. - Home's suggested-task reads fail when the roster can't be read and a task names its owner, instead of handing every owned task to Otto. The plan read and the plan hire keep the lenient read. - Each plan-hire try gets a 7 s deadline while another try still fits in the 15 s window, so a stalled request is sent again. The last try waits for its answer; the proxy's 30 s timeout ends it with the failure toast. - The plan step's settle retries stop once the step is left, so they can't land on a newer take. Paid checkout stays locked while the trial is starting. - Removing the last task on a touch screen moves focus to a composer button, not the text box that opens the keyboard. The folded list is inert while it closes. A roster read still running doesn't count against the task-switch give-up. - Checked: CI green on `32bac59b` (project coverage 90.61%, +0.04%). - [x] **Merged with dev** `88fefd94` (the conflicts were dev's design-token and Tailwind 4 refactors on files this branch deletes or rewrote) and `ef8ccdb6` (this branch's components on dev's design-system lint; see "Where this differs"). Final counts: | | Before (dev, Phase 0) | After (`6eda537b`) | After the dev merge (`ef8ccdb6`) | | --- | --- | --- | --- | | Frontend `pnpm test:unit` | 12,282 passed + 4 expected failures, 1,127 files | 12,471 passed + 4 expected failures, 1,135 files | 12,834 passed + 4 expected failures, 1,164 files | | Backend `onboarding_dump` + `experts` | 904 passed, 29 skipped | 1,073 passed, 29 skipped | 1,172 passed, 29 skipped (with `paywall_routes_test` and dev's `onboarding_role` and `onboarding_wizard` tests) | The deleted hire-step, card and trial-welcome tests are inside those numbers; the count went up by 189 regardless. `8badd23e` (after the full run) moves one ref write into an effect; the 622 onboarding tests pass on it. `d113cf2c` adds one backend test; `onboarding_dump` + `paywall_routes_test`: 526 passed. ### Before and after All screenshots, side by side, desktop (1440 × 900) and phone (390 × 844): **[Plan-first trial: before and after](https://claude.ai/artifact/N4aZGHKWh3Ug3upSVKNA83)** (a private Claude page: it opens for people it has been shared with). It covers: | Screen | Before (dev) | After (this PR) | | --- | --- | --- | | The step after the brain dump | Hire step, then the trial welcome | One plan + trial step (brain dump and skipped; paid and no-paywall variants; pending skeleton; 1280 × 720 and 1366 × 768 without scrolling; 320 × 640 with the sticky trial button) | | Preparing | Checklist | Checklist with "Hiring Maria, Max and Mina" | | Home under the greeting | Two-column cards | The ranked, owned list with the Your team row | | Home after the greeting | Hide all | Hide / Show, "{n} more tasks", a started task, the "Task removed" toast | | The walkthrough's last step | Lands on the first card | Lands on the row with the Start here pill | The screenshots come from `next dev` with every API call mocked (no backend, no Stripe here). The app renders light only (`forcedTheme`), so there are no dark captures. ### Changes 🏗️ **Backend (`onboarding_dump`)** - `plan.py`: one Jev call ranks the greeting's tasks and picks owners; every failure (no key, timeout, raise, cut state, malformed answer) returns None and the plan falls back. `plan_owners.py`: the deterministic fallback, the team cap of three, and owners matched by skill among the three the plan keeps. `role_skills.py`: a template's skills, read from the words of its role, so a renamed role still matches. `plan_job.py`: the last step of the background pipeline, a conditional write on `updatedAt`, retried, Jev asked once. `plan_service.py`, `plan_models.py`, routes: `GET /plan` (open while paywalled: it is shown on the paywall step), `POST /plan/hire` (checks the paywall, so a user who left Stripe without paying hires nobody, whatever the client sends). - `role_tasks.py` / `role_task_sets.py`: five starter tasks per wizard role, each naming the skills that take it, owned from the role's fallback team. `recommend_experts.py`: the fallback team is the roster's most focused template for each skill, starting with the role's first skill, then the user's pain points, then the role's other skills. Each role's skills cover its starter tasks. - `SuggestedPrompt` gains `expert_template_id`, `why`, `expert_name`; `SuggestedTask` gains read-time `expert_id`. `suggested_tasks.with_expert_ids` reads a started task's expert from its chat (`get_chat_session_expert_ids`), so firing that expert, or hiring the owner after Otto took the task, does not relabel the chat. `expert_hired` gains `via: plan`. **Onboarding (frontend)** - `SubscriptionStep/PlanTrialStep.tsx` with `PlanPanel` (polls every second and shows the server's progress as a checklist, settles the plan at 30 s, never swaps a shown plan, restores it on a Stripe cancel return), `TrialPanel`, `PaidPanel`. The trial date math moved to `TrialPanel/helpers.ts` with its tests. Deleted: `HireStep/`, `SubscriptionStep.tsx` and the trial welcome components (#16036) the new step does not use. - `store.ts` drops the hire step; saved progress that names it resumes on the plan step. - `usePlanHire.ts` + `usePreparingStep.ts`: one hire after confirmation, retried inside its window (sessionStorage guard against a remount), "Hiring {names}" or "Setting up your team" in the checklist, the bar holds up to 15 s and then goes on, roster queries refreshed. **Home (frontend)** - `SuggestedTaskList` + `components/TaskRow`, `TaskOwner`, `YourTeamRow`; `SuggestedTasksBlock` with the Hide / Show toggle (`useCollapsedSuggestions`, per user in `localStorage` under `home.suggestedTasks.collapsed`); `useSuggestedTasks` split into `useSuggestedTaskList` and `useTaskRecipient`. Deleted: `SuggestedTaskCard.tsx`, `SuggestedTaskCards.tsx`. - Sending: the composer's recipient becomes the owner (`?expertId=`) and the task goes out once the page can send; a switch that does not land in 3 s gives up with a failure toast and puts the recipient back. An owner who is not on the team: Otto takes it, with "{Name} isn't on your team, so Otto has this one." only when the roster actually loaded. - Accessibility: rows are named "Start with {Name}: {title}" / "Open the chat with {Name}: {title}" and described by their pill and why; a starting task is announced; focus moves to the neighbouring row on remove and back to the row when a start fails; inset focus rings; text and icons at AA contrast. **Tracking, docs, tests** - `posthog-events.ts`, `plan-analytics.ts`, `experts-analytics.ts`, `expert-thread-source.ts`; `docs/engineering/tracking-plan.md` updated (below). - `src/playwright/utils/onboarding.ts`: `completeOnboardingWizard` waits for `plan-panel` to lose `plan-panel-pending`, presses Continue, then connect or preparing. - Two new optional settings, both commented out in `.env.default`: `BRAIN_DUMP_GREETING_BACKUP_MODEL` (the greeting's fast backup) and `BRAIN_DUMP_PLAN_JEV_MODEL` (the Jev model; unset means the SDK default). No new feature flag, no data migration. `openapi.json` is regenerated (the generated client is gitignored). ### Tracking Both walks ran in the mocked app with every `posthog.capture()` call recorded (twice each, identical). App events only; `$autocapture` and `$pageleave` are left out. **The walk ran before `e075809d`**, which renamed two properties after the audit (see "Where this differs"); the lists show the names the code sends now, and the unit tests pin them. <details><summary>Path A: brain dump typed (20 app events)</summary> | # | Event | Properties | When | | --- | --- | --- | --- | | 1 | `onboarding_step_viewed` | `step: team` | /onboarding | | 2 | `onboarding_step_viewed` | `step: autopilot` | Next | | 3 | `onboarding_step_viewed` | `step: role` | Next | | 4 | `onboarding_step_viewed` | `step: pain_points` | role Next | | 5 | `brain_dump_typed_fallback` | `reason: chose_to_type, prompt_set: Founder/CEO` | Write | | 6 | `brain_dump_completed` | `input_mode: typed, chars: 283, prompt_set: Founder/CEO, prompts_seen: 1` | I'm done | | 7 | `paywall_viewed` | `surface: onboarding` | plan step opens | | 8 | `onboarding_step_viewed` | `step: subscription` | plan step opens | | 9 | `plan_viewed` | `plan_source: llm, skipped_dump: false, tasks: 3, experts: 3, wait_ms: 84, gave_up: false` | plan step opens | | 10 | `trial_offer_viewed` | `trial_offer_version: trial-v1, subscription_tier: PRO, trial_duration_days: 7, surface: onboarding` | plan step opens | | 11 | `onboarding_step_viewed` | `step: preparing` | Start my free trial → Stripe → back | | 12 | `plan_experts_hired` (backend) | `hired: 3, already_on_team: 0, failed: 0` | preparing | | 13 | `intro_path` | `path: A` | Home | | 14 | `home_suggestion_clicked` | `position: 0, surface: intro, remaining: 5, owner: expert, expert_template_id: tpl-maria, rank: 0` | task 1 | | 15 | `intro_followup_sent` | `chars: 115` | task 1 | | 16 | `expert_thread_created` | `expert_id: exp-maria, thread_source: suggested_task` | task 1 | | 17 | `home_viewed` | — | back to Home | | 18 | `home_suggestion_dismissed` | `position: 1, surface: home, remaining: 4, scope: one` | x on task 2 (Undo sends nothing) | | 19 | `home_suggestions_collapsed` | `open: 5` | Hide | | 20 | `home_suggestions_expanded` | `open: 5` | Show | </details> <details><summary>Path B: brain dump skipped (19 app events)</summary> | # | Event | Properties | When | | --- | --- | --- | --- | | 1–4 | `onboarding_step_viewed` | `step: team`, `autopilot`, `role`, `pain_points` | as in A | | 5 | `brain_dump_skipped` | `prompt_set: Founder/CEO, prompts_seen: 0` | Skip for now | | 6 | `paywall_viewed` | `surface: onboarding` | plan step opens | | 7 | `onboarding_step_viewed` | `step: subscription` | plan step opens | | 8 | `trial_offer_viewed` | `trial_offer_version: trial-v1, subscription_tier: PRO, trial_duration_days: 7, surface: onboarding` | plan step opens | | 9 | `plan_viewed` | `plan_source: fallback, skipped_dump: true, tasks: 3, experts: 2, wait_ms: 103, gave_up: false` | plan step opens | | 10 | `onboarding_step_viewed` | `step: preparing` | Start my free trial → Stripe → back | | 11 | `plan_experts_hired` (backend) | `hired: 2, already_on_team: 0, failed: 0` | preparing | | 12 | `intro_path` | `path: B` | Home | | 13 | `home_suggestion_clicked` | `position: 0, surface: intro, remaining: 4, owner: expert, expert_template_id: tpl-max, rank: 0` | task 1 | | 14 | `intro_followup_sent` | `chars: 63` | task 1 | | 15 | `expert_thread_created` | `expert_id: exp-max, thread_source: suggested_task` | task 1 | | 16 | `home_viewed` | — | back to Home | | 17 | `home_suggestion_dismissed` | `position: 1, surface: home, remaining: 3, scope: one` | x on task 2 (Undo sends nothing) | | 18 | `home_suggestions_collapsed` | `open: 4` | Hide | | 19 | `home_suggestions_expanded` | `open: 4` | Show | </details> None of the removed events appeared. Sent by the backend, so not in a browser walk here: `signup_completed`, `checkout_started` (`checkout_kind: trial, surface: onboarding`), `trial_started`, `trial_ending`, `trial_canceled`, `trial_converted`, `payment_succeeded`, `expert_hired` (3 in A, 2 in B, `surface: onboarding, via: plan`). Not reached by the walk's actions, covered by unit tests: `plan_selected` (paid path, `PlanTrialStep.paywall.test.tsx`), `checkout_abandoned` (cancel return), `welcome_dialog_closed` (the walkthrough was marked seen). `capability_card_*` were retired before this branch (SECRT-3149) and stay reserved. **Removed with the hire step:** `expert_recommended`, `expert_recommendation_clicked`, `hire_all_clicked`, `hire_step_continued` (in `RETIRED_EVENT_NAMES`, rows in the tracking plan's Removed table). `hire_started` loses its `onboarding` surface (the plan's hires have no click); it stays for `expert_page`. **After deploy:** four saved PostHog insights read the removed events (u2ZK2Isk, QfcpXRmM, 2OSZpNcv and ugcxrNQU). SECRT-3257 has the steps to freeze each one at the release and the tested queries for the plan-first versions, to do once this is live on prod. ### Langfuse Tracked in SECRT-3225, to do before this reaches prod. The greeting's instructions come from the Langfuse prompt **Brain Dump Greeting** (`BRAIN_DUMP_GREETING_PROMPT_NAME`) whenever it is reachable; the local copy in `intro.py` is only the fallback. The local copy now asks for one optional key per suggested task: > `"why"` (optional): one short clause, at most 60 characters, second person, naming the thing they said this task fixes ("Because listings eat your evenings."). Leave the key out when nothing they said ties to the task. The same line needs adding to the Langfuse copy, or every task in prod arrives without a why line (nothing breaks; the line is just absent). The parser keeps a `why` of 1 to 60 characters and drops a longer one rather than cutting it. Nothing else in the greeting prompt changes. ### Settled decisions - The paywall is replaced entirely by the plan + trial step; `useSubscriptionStep`, `useTrialCard`, `tracking.ts`, the trial date math (moved, tests kept) and the Stripe return handling stay. - Tasks are ranked and assigned by TypeSafe Jev with a deterministic fallback. Jev is a hosted model call: the request carries the role, pain points, tasks, team and up to 6,000 characters of the transcript, with a 5.5 s budget. - Experts are hired on the preparing screen after the trial is confirmed; a user who abandons Stripe hires nobody (with no paywall, when the user continues past the plan). - The list is the design, the #16236 cards are removed; x removes a task, Hide all became a collapse. ### Where this differs from the handoff, and why - **`plan_source` and `thread_source`, not `source`.** `docs/engineering/tracking-plan.md` reserves `source` for the emitter: every browser event carries `source: web` as a super property, and an event property of the same name replaces it. The repo settled this clash before with `team_source` and `workflow_source`, so `plan_viewed` sends `plan_source` and `expert_thread_created` sends `thread_source`. - **A started task keeps naming who has it.** Edge case 11 (a fired expert's tasks go to Otto) holds for open tasks. A task already started links to a chat with that expert, so its row keeps their name; the server reads it from the chat. Without this, firing Maria relabelled her chat as Otto's, and hiring an owner after Otto took a task relabelled Otto's chat. - **Contrast.** The spec's "muted" grey (`#83838C`) is under 4.5:1 for small text, so the why line, the role, the sent rows' title and the toggle use `foreground-secondary`, and "In progress with {Name}" uses `success-foreground` rather than `success`. - **Undo focus.** Focus moves to the neighbouring row on remove. After Undo, the toast (sonner) returns focus to where it was before the toast, which is that neighbour, rather than to the restored row; fighting the toast's own focus return would make focus jump twice. - **The brain dump step** gained one read-only accessor (`whenSkipSettled`), so the plan read waits for a Skip still in flight (`7fb3dce4`). Its behaviour is unchanged. - **Module split.** The backend plan lives in seven small modules instead of one `plan.py`, because backend AGENTS.md caps files at about 300 lines. - **Jev request budget.** Owner candidates are the recommendation's first 6, each task's prompt is cut to 600 characters / 1,200 bytes, and the transcript to 6,000 characters / 12,000 bytes, then 6,000 and 3,000 bytes if the state still does not fit. A character-only cut sent every long Japanese or Chinese dump to the fallback. - **The plan API's `source` is `llm`** only when the greeting wrote the stored tasks. A completed dump whose greeting fell back to the generic starters gets the role set (owned from the dump's team) and reads as `fallback` with no "heard" line. - **Pending plan.** While the pipeline runs, `GET /plan` answers `ready: false` with the role plan, unstored, and `progress` (tasks written, team chosen). A voice take still transcribing counts as pending. At 30 s the step calls `POST /plan/settle`, which stores that role plan as the take's tasks and team unless a plan is already stored; see "The plan step's wait". - **Reads that store** are built from reads that raise, so a failed template or wizard read fails the request (the step keeps polling) instead of storing a degraded list for good. - **Ids.** Ranked rows keep their original position as `id`, so a task keeps the id Home may already have handed out. Each owned task stores `expert_name`, so the Otto toast can name an expert who has left. - **Hire while pending (no paywall only).** A hire that arrives before the pipeline finishes settles the plan first, so the team hired is the role team whose tasks Home then lists; with the paywall the Stripe round trip makes this window practically zero. - **Archived experts.** A repeat plan hire can bring back an expert the user archived, but only on an onboarding retake whose new plan names them again. - **Merged with dev's design-system lint (#16402, #16429).** The new rules fail any violation in a file the baseline doesn't list, so this branch's new components now use the primitives' own looks (`ef8ccdb6`). Visible differences from the screenshots: avatars keep the design system's own shape (rounded-xl) instead of a full circle, as dev's onboarding does; the row's remove button is the Button's `toggle` variant, quiet grey with no hover fill; the sending spinner is `LoadingSpinner` (16px); the two 48px CTAs set their label at 15px (`textSize="body-md"`) instead of 16px; the step card's corner is 24px instead of 28px; the Start here pill is 11px with `tracking-eyebrow`. The baseline only goes down (the task list 13 → 1, the block 9 → 3, the deleted card's entry gone). Dev's role-draft test (#16398) now steps from the brain dump straight to the plan step. - **Commit trailer.** AGENTS.md asks for a `Co-authored-by` trailer naming the model and the platform; the commits use the session's trailer with the platform added. ### The plan step's wait Found testing the preview: the step waited for the whole brain-dump pipeline. That was the Sonnet greeting with adaptive thinking on, the team and integration picks, the memory extraction, the Jev ranking, then the dump's `completed` status. Past the old 20 s give-up, the step showed the role plan while the hire and Home used the pipeline's, so the team named on the step could differ from the one hired. Fixed in this PR: - **Less to wait for.** The greeting call answers without thinking first (`intro.py`, reasoning off like the pipeline's other calls, `max_tokens` 4,000). The team job asks for the six experts the plan reads (`plan.MAX_OWNER_CANDIDATES`), not sixteen. Only the greeting, team and plan jobs end the step's wait. The provider picks and the memory extraction run beside them and write their own columns; a failed understanding write now costs the context, not the take. - **Progress, not skeletons.** The pending answer carries `progress`. The step shows it as a checklist where the heard card will be: "Turning your brain dump into tasks", "Finding experts who fit your work", "Choosing who takes each task". It polls every second (was 2.5 s) and announces "Your plan is ready" from a live region outside the busy skeleton. - **A plan the step gives up on is the plan.** The step gives up at 30 s (was 20 s). It then calls `POST /plan/settle`, which stores the role plan it showed as the take's tasks (rows with ids and states) and team. The greeting still writes its text, but leaves tasks already stored alone (`db.store_greeting`, conditional on an empty list). The team job leaves a stored team alone (`db.replace_team`, a conditional write on `updatedAt`, because a reset take holds a JSON null and an older row a SQL null). The plan job finds nothing to rank. So the trial hires, and Home lists, what the step showed. The cut-off moved to 30 s because giving up now costs the plan the brain dump would have written. - **Back and re-record.** The remembered plan (for a Stripe cancel return and the preparing screen's label) is forgotten whenever a new take or skip is sent. - **Measuring it.** The preview sends nothing to PostHog, so the new wait isn't measured yet. The backend now logs each leg ("Brain dump greeting for user … took Ns", "Brain dump team for user … took Ns", "Brain dump plan ready for user … Ns after the pipeline started"). `plan_viewed`'s `wait_ms` and `gave_up` give the same in PostHog after release. `gave_up` now means the 30 s wait ran out, and `plan_source` says whose plan was shown. - **Tests.** BE: `service_test.py` (the plan is ready before the provider picks and before the understanding is stored, a failed understanding write costs the context, tasks and team stored while the jobs ran stand, a lost team write is retried), `routes_test.py` (pending progress, settling mid-pipeline, settling a ready plan changes nothing, settling a take still being transcribed reads as recorded, hiring while the jobs run settles first, and the end-to-end "a plan the step gave up on is what the trial hires and Home lists"), `suggested_tasks_db_test.py` (the conditional writes against Postgres), `intro_test.py` (reasoning off). FE: `usePlan.test.tsx` (1 s polls, settles at 30 s, falls back when the settle fails, waits for a skip in flight, keeps a plan that lands during the settle), `PlanPanel.test.tsx` (the checklist and the announcement). ### Not run here, and notes - **Stripe test mode and a real `posthog.debug()` walk.** This environment has no backend stack or Stripe account. The flow was walked in `next dev` with every call mocked (Stripe Checkout intercepted to the success return), with PostHog captured in the page. The Playwright smoke and journeys suites ran in CI and passed with the updated helper. - **CI: two `experts_db_test` failures from skills-catalog `main`.** Catalog PR #24 (30 Oct 15:50 UTC) changed day-one rows and roles, which broke `test_wave_three_experts_are_skills_only_with_ordered_packs` and `test_roster_day_one_promises_match_work_the_expert_can_do_on_request` on every branch ([details](https://github.com/Significant-Gravitas/AutoGPT/pull/16308#issuecomment-6527907044)). Dev fixed them in #16443, merged here in `8d81826a`. - **Out of scope, seen in passing.** `home_viewed` does not fire on the first Home visit, because `HomeRecap` mounts only once the greeting has retired; the same on dev. The app sidebar logs a hydration mismatch on some loads (its mobile/desktop branch differs between server and client); the dev captures show it too. - **`POST /api/experts` stays open while paywalled.** Its entry in `paywall_routes_test.py` dates from when onboarding hired the team before the plan picker. Onboarding no longer calls it; whether it should now check the paywall is a separate decision, so I only updated the comment. - **The greeting fell back on the preview.** In the product manager's test, Home showed the generic greeting ("Thanks for talking me through your work — I've got it…"). That means the greeting call failed twice, and the plan then took the Product/PM starter set instead of tasks written from the brain dump. The owner fix above makes that set the team's work, but the tasks still come from the role, not from what the user said. The cause is not visible from here: no model keys in this environment. The preview backend logs it as `Brain dump greeting generation failed (attempt N): <ErrorType>: <message>`, now with the type. - **`Other` and the team of three.** Every plan now starts with three experts (`d6ea5524`). For Other, or no role, the user's pain points lead, then the experts the Other starter set needs (Quinn, Maya, Nadia), so its research, post, summary and watch tasks have owners; its first task, finding the work to hand off, stays Otto's. On a thin roster, any seat still empty is filled from general skills and never with a role that names no skill, so a filler may own no task. ### Edge cases Each of the handoff's 16 cases, with the tests that cover it. Paths: **BE** `backend/api/features/onboarding_dump/`, **OB** `frontend/src/app/(no-navbar)/onboarding/`, **PP** `OB/steps/SubscriptionStep/components/PlanPanel/__tests__/`, **ST** `frontend/src/app/(platform)/copilot/components/SuggestedTasks/__tests__/`. | # | Case | Covered by | | --- | --- | --- | | 1 | Skipped brain dump | BE `routes_test.py::test_a_skipped_dump_gets_the_role_plan_stored_on_first_read` (fallback, `skipped_dump`, Founder team, Home gets the same titles and ids); PP `PlanPanel.test.tsx` "speaks to the role for a skipped dump" | | 2 | Dump still generating, 30 s give-up settles the plan, no swap | PP `usePlan.test.tsx` "settles the plan at 30 s and shows the plan the server settled on", "keeps the settled plan when a poll in flight at 30 s lands later", "keeps a plan that lands while the settle is on its way"; BE `test_the_plan_is_pending_while_the_pipeline_runs` (a pending read stores nothing), `test_a_plan_the_step_gave_up_on_is_what_the_trial_hires_and_home_lists`. A *failed* dump answers at once with the role plan (`test_a_failed_take_plans_and_hires_from_the_role_not_the_team_it_stored`). | | 3 | Jev unset, timeout, error, cut state | BE `plan_test.py` (no key, timeout or raise, slow call, failed/empty/cut call, oversized state all return None; a timeout or raise logs one warning), `plan_job_test.py::test_without_jev_the_greeting_order_is_kept_with_hinted_owners`. An unset key warns once at import, like the copilot gate. | | 4 | Zero experts | BE `plan_owners_test.py::test_no_recommended_experts_leaves_every_task_with_otto`, `routes_test.py::test_a_plan_with_nobody_on_its_team_hires_nobody`; PP "leaves the tasks with Otto when nobody joins"; OB `usePreparingStep.test.ts` "says it is setting up the team when the plan has nobody to hire" | | 5 | Fewer / more than three | PP "shows fewer cards when the plan has fewer tasks"; ST `suggested-tasks.test.tsx` "keeps the rows after the first three under a closed '2 more tasks' row that opens on click" | | 6 | Owner outside the capped team | BE `plan_owners_test.py::test_a_team_of_four_owners_keeps_three_and_nulls_the_fourth`, `plan_job_test.py::test_a_jev_owner_left_off_the_capped_team_goes_to_otto` | | 7 | Back and re-record | PP `usePlan.test.tsx` "does not show a plan cached before the mount while the refetch is pending", "reads the plan only once a skip still on its way has landed"; BE `service_test.py::test_a_new_take_never_serves_the_previous_takes_answers` | | 8 | Stripe cancel return | PP "shows the remembered plan at once on a checkout return", "does not report a restored plan as a second view"; OB `PlanTrialStep.paywall.test.tsx` "reports checkout_abandoned when Stripe sends the user back without paying"; `usePreparingStep.test.ts` "never hires before the trial is confirmed"; BE `routes_test.py::test_a_user_still_behind_the_paywall_hires_nobody` | | 9 | No trial offer | OB `PlanTrialStep.paywall.test.tsx` "offers one Pro checkout beside the plan…"; PP "names the moment the team joins" ("joins your team when you subscribe") | | 10 | Hire fails for one or all | BE `test_a_failed_hire_is_reported_and_the_rest_still_join`; OB "never blocks Home on a failed hire, and reports it", "moves on after the ceiling when the hire never answers"; ST "shows each row's owner… and Otto… for an expert not on the team" | | 11 | Expert fired between visits | BE `suggested_tasks_test.py::test_each_owned_task_names_its_hired_expert`, `test_a_started_task_stays_with_the_expert_its_chat_is_with_once_fired`; ST "gives Otto a task whose expert has left the team, says so, and shows it in progress with Otto", "keeps naming the expert a started task went to after that expert is fired". The toast fires when such a task is started, once (the row then becomes a link). | | 12 | Old progress with `hire` | OB `progress.test.ts` / `page.test.tsx` "resumes a draft saved on the retired hire step at the plan step"; `store.test.ts` "never lays out the retired hire step" | | 13 | Old rows without owner or why | BE `suggested_tasks_db_test.py::test_prompts_stored_before_owners_existed_still_rehydrate`, `intro_test.py::test_a_prompt_keeps_its_why_and_one_without_still_parses`; ST "shows a task's why under its title, and nothing there when it has none" | | 14 | Role `Other` | BE `plan_owners_test.py::test_other_and_unknown_roles_take_the_other_set`; PP "keeps the line role-free for Other". See the note on `Other` above for the team. | | 15 | Motion, keyboard, viewports, phones, collapsed list | PP "offers nothing to click or focus"; ST `SuggestedTasksBlock.test.tsx` "still hides and shows the list when the user prefers less motion", "keeps the list folded for the same user after a reload…", `helpers.test.ts` "never collapses the list for a signed-out visitor…". By reading the code and in the screenshots: tab order on the step is Back, Start my free trial, Log out (DOM order; the progress bar has nothing focusable); 1280 × 720 and 1366 × 768 fit without scrolling and phones keep the sticky trial button (screenshots); rows stack under `sm` with the x always visible on touch; the collapse is a per-user key that sign-out does not clear. | | 16 | Expert limit mid-hire | BE `test_the_expert_limit_stops_the_hire_and_reports_the_rest`, `test_the_expert_limit_on_the_second_hire_keeps_the_first_and_reports_the_rest`; Home's Otto fallback is case 10's path | Screen reader and keyboard: rows announce "Start with Maria: {title}" and, once sent, "Open the chat with Maria: {title}" (ST); the toggle reports expanded and collapsed (ST); the walkthrough's Start here step lands on the row with the pill (`first-run-tour.test.tsx` "starts on a task's own button, never its Remove button"); focus after Remove goes to the next row, its Remove while locked, or the composer when the list goes (ST "keyboard and screen readers", `SuggestedTasksBlock.test.tsx` "focus after Remove"). ### Ticket lines Resolves SECRT-3201 Resolves SECRT-3132 Resolves SECRT-3131 Progresses SECRT-3134 (Start here and ranking shipped; account requirements per task still open) Progresses SECRT-3142 (owner per task shipped; Otto proposing delegation in chat still open) Related SECRT-3211 (what happens to a task after its chat finishes), out of scope Related SECRT-3212 (integrations on suggested tasks), out of scope Related SECRT-3225 (the Langfuse why line, before release) and SECRT-3257 (the PostHog insights, after release) ### Agents and large language models used - Claude Code (model reported in the commit trailers) ### Checklist 📋 #### For code changes: - [x] I have clearly listed my changes in the PR description - [x] I have made a test plan - [ ] I have tested my changes according to the test plan (all but a Stripe test-mode run, below): - [x] Backend: `poetry run pytest backend/api/features/onboarding_dump backend/api/features/experts backend/api/paywall_routes_test.py` plus dev's `onboarding_role` and `onboarding_wizard` tests: 1,172 passed, 29 skipped after the dev merge. `export-api-schema` matches the committed `openapi.json`. isort, black and pyright clean on every changed file. - [x] Frontend: `pnpm lint` and `pnpm types` exit 0 on the final head (the only warnings left in files this PR touches predate it); `pnpm test:unit`: 12,834 passed + 4 expected failures after the dev merge, including the design-system baseline tests. React Compiler coverage and bailout tests pass with no new bailouts. - [x] Each commit checked on its own in a clean worktree (types and the suites it touches). - [x] Screenshots of every changed screen, desktop and phone, plus 1280 × 720, 1366 × 768 and 320 × 640 for the plan step (linked above). Mocked `next dev`; light theme only. - [x] Both onboarding paths walked to Home with every PostHog `capture()` recorded, twice each (lists above). - [x] Two review passes with independent verification of each finding; the confirmed ones are fixed in `3782dfc4` (a started task's owner), the give-up fix in `130b92ad`, and `6eda537b` (focus after Remove). - [x] Playwright smoke and journeys suites: passed in CI with the updated `completeOnboardingWizard`. - [ ] Stripe test-mode run: not possible in this environment (no backend stack, no Stripe). #### For configuration changes: - [x] `.env.default` is updated or already compatible with my changes (two new optional model settings, commented out; the Jev key is the existing `TYPESAFE_API_KEY`) - [x] `docker-compose.yml` is updated or already compatible with my changes - [x] I have included a list of my configuration changes in the PR description (under **Changes**): none No data migration is required: the new fields are optional, and rows written before this change read as Otto-owned with no `why`. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01G58r45FBHQQQwJGu1SgbxC --------- Co-authored-by: Claude <[email protected]> Co-authored-by: Krzysztof Czerwinski <[email protected]>
Engineering notes
Working documents for the team and the agents that work on AutoGPT: analytics and tracking plans, rollout plans, runbooks, internal references, and design and architecture notes.
Nothing in this folder is on the docs site. The site at agpt.co/docs is built from docs/home/, docs/platform/ and docs/integrations/, and shows the pages listed in each folder's SUMMARY.md. A page written for people outside the team, whether they use, self-host or contribute to AutoGPT, goes there. Everything else goes here.
These notes are still public. The repository is public, and AutoPilot's documentation search indexes every Markdown file under docs/, this folder included.
- Don't list a file from this folder in a
SUMMARY.md. - To link to a note from a published page, use its GitHub URL on
dev. The relative path does not exist on the docs site.