Answers in the store, and one account password made for the person

The answers move from answers.yaml into an `answers` table of data/jobs.db,
the SQLite file that already holds the jobs: name, value, source, when.
An old answers.yaml is brought in once and renamed. At the first start
AIHawk makes `account_password`, twenty characters of every class, and
keeps it there: the one password for sites that need an account, which
the Workday plugin now names. Settings has "Your answers": see, change,
add or remove one. Docs and SECURITY say where the answers and the
password live, and that the password is in clear.

Co-Authored-By: Claude Fable 5.1 <[email protected]>

SECURITY.md

@@ -23,7 +23,7 @@ AIHawk reads job pages written by strangers, fills their forms, and sends applic
 ## Where keys and sign-ins are kept
 
 - The OrcaRouter key comes from the `ORCAROUTER_API_KEY` environment variable, or from a file named `key` in the AIHawk folder.
-- The AIHawk folder is `~/.aihawk`. It also holds the CV, the answers, the job data, what was sent (`applied.csv` and `data/proofs/`) and the browser profile (`browser-profile/`), which keeps the person's sign-ins.
+- The AIHawk folder is `~/.aihawk`. It also holds the CV, the answers and the account password (in clear, in the `answers` table of `data/jobs.db`), the job data, what was sent (`applied.csv` and `data/proofs/`) and the browser profile (`browser-profile/`), which keeps the person's sign-ins.
 - Do not paste a key, a browser profile or a folder from the AIHawk folder into an issue or a pull request.
 
 ## Known and accepted

docs/configuration.md

@@ -4,6 +4,10 @@ Press `e` in AIHawk to change these on the screen. AIHawk reads `aihawk.yaml` in
 
 Loading a resume rewrites this file, and the rewrite drops the comments you put in it, so keep your notes elsewhere.
 
+## Your answers
+
+What the forms are told about you beyond the CV (email, phone, location, links, and anything you add) lives in the `answers` table of `data/jobs.db`, the SQLite file in the AIHawk folder. Press `e`, then "Your answers", to see, change or add them; an answer you gave is kept when a new resume is read. AIHawk makes an `account_password` the first time it starts and keeps it there: the one password it uses on job sites that need an account, such as Workday, so you can sign in to those sites yourself. The `answers.yaml` of earlier versions is read once, brought into the table and renamed `answers.yaml.imported`.
+
 | Key | Default | What it does |
 |---|---|---|
 | `name` | empty | Your name for this search, shown at the top of the list. |

src/aihawk/config.py

@@ -99,9 +99,14 @@ class Config:
         return self.workspace / "cv.md"
 
     @property
-    def answers(self) -> Path:
+    def answers_file(self) -> Path:
+        """Where the answers were kept before they moved into the store: read once, to import them."""
         return self.workspace / "answers.yaml"
 
+    @property
+    def store_file(self) -> Path:
+        return self.data_dir / "jobs.db"
+
     @property
     def data_dir(self) -> Path:
         return self.workspace / "data"

src/aihawk/plugins/workday.md

@@ -7,7 +7,7 @@ A Workday form needs a candidate account on the company's own site: every compan
 
 The page at `/apply` is "Start Your Application" with `applyManually` ("Apply Manually"), `autofillWithResume`, `useMyLastApplication` and a LinkedIn frame: click `applyManually`. Do not use "Autofill with Resume", "Sign in with Google", "Sign in with LinkedIn" or the LinkedIn frame: the account is made with the candidate's email, never through another service.
 
-Sign in or create the account. The step "Create Account/Sign In" shows `SignInWithEmailButton` ("Sign in with email"): click it. The sign-in form has `email`, `password`, `signInSubmitButton` ("Sign In"), `createAccountLink` ("Create Account") and `forgotPasswordLink`. If the answers give a Workday password, sign in with the candidate's email and that password; if the site says the account does not exist, click `createAccountLink`. The create form has `email`, `password`, `verifyPassword`, the checkbox `createAccountCheckbox` ("I agree", the terms and the privacy notice of the company) and `createAccountSubmitButton` ("Create Account"). The password rules are printed above the form: at least 8 characters with upper case, lower case, a digit and a special character. Use the Workday password from the answers for every company, so the candidate can sign in again; if the answers give none, call finish_form with status blocked and say that the person must add a line "Workday password: ..." to the answers, or press `l` and sign in to this company's site.
+Sign in or create the account. The step "Create Account/Sign In" shows `SignInWithEmailButton` ("Sign in with email"): click it. The sign-in form has `email`, `password`, `signInSubmitButton` ("Sign In"), `createAccountLink` ("Create Account") and `forgotPasswordLink`. Sign in with the candidate's email and the `account_password` from the answers; if the site says the account does not exist, click `createAccountLink`. The create form has `email`, `password`, `verifyPassword`, the checkbox `createAccountCheckbox` ("I agree", the terms and the privacy notice of the company) and `createAccountSubmitButton` ("Create Account"). The password rules are printed above the form: at least 8 characters with upper case, lower case, a digit and a special character. Use the `account_password` from the answers, the same on every company, so the candidate can sign in again; if the answers have none, call finish_form with status blocked and say so.
 
 Both forms carry a hidden text field `beecatcher` (name `website`, "This input is for robots only"): never type in it.
 

src/aihawk/profile.py

@@ -3,7 +3,9 @@
 from __future__ import annotations
 
 import re
+import secrets
 import shutil
+import string
 import zipfile
 from pathlib import Path
 from typing import Any
@@ -16,6 +18,12 @@ from pypdf.errors import PyPdfError
 
 from aihawk.config import CONFIG_FILE, Config, ConfigError, load_config
 from aihawk.llm import structured
+from aihawk.store import Store
+
+# The one password for the job sites that need an account, made once and kept with the answers: the person can sign in again.
+ACCOUNT_PASSWORD = "account_password"
+PASSWORD_LENGTH = 20
+PASSWORD_SYMBOLS = "!@#$%&*-_=+?"
 
 _WORD_TEXT = "{http://schemas.openxmlformats.org/wordprocessingml/2006/main}"
 
@@ -189,8 +197,7 @@ async def install_resume(client: AsyncOpenAI, config: Config, path: Path) -> dic
     settings.update({"name": profile["name"], "goals": profile["goals"], "filters": filters})
     settings_path.write_text(yaml.safe_dump(settings, allow_unicode=True, sort_keys=False), encoding="utf-8")
 
-    stated = {key: value for key, value in profile["answers"].items() if value.strip()}
-    add_answers(config, stated)
+    add_answers(config, {key: value for key, value in profile["answers"].items() if value.strip()}, source="cv")
     return profile
 
 
@@ -233,15 +240,50 @@ def save_setting(config: Config, name: str, value: str) -> None:
         raise
 
 
+def import_answers_file(config: Config, store: Store) -> int:
+    """The answers.yaml of earlier versions, brought into the store once; the file is then renamed, not deleted. How many came in."""
+    path = config.answers_file
+    if not path.is_file():
+        return 0
+    data = yaml.safe_load(path.read_text(encoding="utf-8-sig"))
+    answers = {str(name): str(value) for name, value in data.items() if value is not None} if isinstance(data, dict) else {}
+    store.add_answers(answers, source="answers.yaml")
+    path.replace(path.with_suffix(".yaml.imported"))
+    return len(answers)
+
+
+def new_password() -> str:
+    """A password the sites accept: upper and lower case, a digit and a symbol, in a random order."""
+    classes = (string.ascii_uppercase, string.ascii_lowercase, string.digits, PASSWORD_SYMBOLS)
+    chars = [secrets.choice(group) for group in classes]
+    chars += [secrets.choice("".join(classes)) for _ in range(PASSWORD_LENGTH - len(chars))]
+    secrets.SystemRandom().shuffle(chars)
+    return "".join(chars)
+
+
+def ensure_account_password(store: Store) -> bool:
+    """The account password, made the first time it is missing. Whether it was made now."""
+    if ACCOUNT_PASSWORD in store.answers():
+        return False
+    store.set_answer(ACCOUNT_PASSWORD, new_password(), source="generated")
+    return True
+
+
 def read_answers(config: Config) -> dict[str, str]:
-    if not config.answers.is_file():
-        return {}
-    data = yaml.safe_load(config.answers.read_text(encoding="utf-8-sig"))
-    return data if isinstance(data, dict) else {}
+    """The answers in the store, after the old file, if any, has been brought in."""
+    store = Store(config.store_file)
+    try:
+        import_answers_file(config, store)
+        return store.answers()
+    finally:
+        store.close()
 
 
-def add_answers(config: Config, answers: dict[str, str]) -> None:
+def add_answers(config: Config, answers: dict[str, str], source: str = "person") -> None:
     """Remember what the person said. An answer they gave wins over one read again from a CV."""
-    config.workspace.mkdir(parents=True, exist_ok=True)
-    merged = {**answers, **read_answers(config)}
-    config.answers.write_text(yaml.safe_dump(merged, allow_unicode=True, sort_keys=False), encoding="utf-8")
+    store = Store(config.store_file)
+    try:
+        import_answers_file(config, store)
+        store.add_answers(answers, source)
+    finally:
+        store.close()

src/aihawk/session.py

@@ -17,7 +17,7 @@ from aihawk.documents import score_job
 from aihawk.judge import evidence, fuse, questions
 from aihawk.llm import explain, make_client, refused
 from aihawk.plugins import Plugin, always_plugins, load_plugins, plugin_for
-from aihawk.profile import install_resume, save_setting
+from aihawk.profile import ensure_account_password, import_answers_file, install_resume, save_setting
 from aihawk.store import SENT_WORDS, SUBMITTED, Store, Stored, write_applied
 
 # The funnel: the next WIDE postings by title are weighed by the decision model, and the best JUDGED of them are scored by the judge model.
@@ -33,7 +33,9 @@ class Session:
         self.workspace = workspace
         self.config = load_config(workspace)
         self.plugins: tuple[Plugin, ...] = load_plugins(workspace)
-        self.store = Store(self.config.data_dir / "jobs.db")
+        self.store = Store(self.config.store_file)
+        import_answers_file(self.config, self.store)
+        ensure_account_password(self.store)
         self._client: AsyncOpenAI | None = None
 
     def close(self) -> None:
@@ -176,7 +178,7 @@ class Session:
         plugin = plugin_for(item.job.url, self.plugins)
         cv = self.config.cv.read_text(encoding="utf-8-sig")
         job = replace(item.job, apply_url=plugin.form_url(item.job.url) if plugin else item.job.url)
-        answers = self.config.answers.read_text(encoding="utf-8-sig") if self.config.answers.is_file() else ""
+        answers = self.store.answers_text()
         extras = always_plugins(self.plugins)
         guidance = "\n\n".join([*([plugin.instructions] if plugin else []), *(extra.instructions for extra in extras)])
         try:

src/aihawk/store.py

@@ -1,4 +1,4 @@
-"""What AIHawk knows about each job, in one SQLite file of the workspace."""
+"""What AIHawk knows about each job, and the person's answers, in one SQLite file of the workspace."""
 
 from __future__ import annotations
 
@@ -42,6 +42,15 @@ CREATE TABLE IF NOT EXISTS jobs (
 )
 """
 
+ANSWERS_SCHEMA = """
+CREATE TABLE IF NOT EXISTS answers (
+    name TEXT PRIMARY KEY,
+    value TEXT NOT NULL,
+    source TEXT NOT NULL,
+    updated TEXT NOT NULL
+)
+"""
+
 # new -> shortlisted | passed_over; shortlisted -> scored -> refused | blocked | failed | submitted | unconfirmed
 SUBMITTED = ("submitted", "unconfirmed")
 # The jobs the person sees as tried, and what each status says in plain words.
@@ -131,6 +140,7 @@ class Store:
         self._db = sqlite3.connect(path)
         self._db.row_factory = sqlite3.Row
         self._db.execute(SCHEMA)
+        self._db.execute(ANSWERS_SCHEMA)
 
     def close(self) -> None:
         self._db.close()
@@ -186,6 +196,32 @@ class Store:
         rest = [item for _, item in sorted(zip(order, rest, strict=True), key=lambda pair: (-pair[0], pair[1].job.id))]
         return judged + rest
 
+    # The answers: what the person said about themselves, given to the model with the CV when a form is filled.
+    def answers(self) -> dict[str, str]:
+        return {row["name"]: row["value"] for row in self._db.execute("SELECT name, value FROM answers ORDER BY rowid")}
+
+    def answers_text(self) -> str:
+        """The answers as the model reads them: one line each, name and value."""
+        return "\n".join(f"{name}: {value}" for name, value in self.answers().items())
+
+    def add_answers(self, answers: dict[str, str], source: str) -> None:
+        """Answers that are new. One already there is kept: what the person said wins over what is read again from a CV."""
+        rows = [(name, value.strip(), source, _now()) for name, value in answers.items() if value and value.strip()]
+        self._db.executemany("INSERT OR IGNORE INTO answers (name, value, source, updated) VALUES (?, ?, ?, ?)", rows)
+        self._db.commit()
+
+    def set_answer(self, name: str, value: str, source: str = "person") -> None:
+        """One answer as the person changes it; an empty value removes it."""
+        if value.strip():
+            self._db.execute(
+                "INSERT INTO answers (name, value, source, updated) VALUES (?, ?, ?, ?) "
+                "ON CONFLICT(name) DO UPDATE SET value = excluded.value, source = excluded.source, updated = excluded.updated",
+                (name, value.strip(), source, _now()),
+            )
+        else:
+            self._db.execute("DELETE FROM answers WHERE name = ?", (name,))
+        self._db.commit()
+
     def set_evidence(self, job_id: str, evidence: float) -> None:
         self._db.execute("UPDATE jobs SET evidence = ?, updated = ? WHERE id = ?", (evidence, _now(), job_id))
         self._db.commit()

src/aihawk/tui.py

@@ -210,13 +210,14 @@ class Settings(ModalScreen[str | None]):
 
     BINDINGS: ClassVar = [Binding("escape,q,e", "close", "Back")]
 
-    def __init__(self, rows: list[tuple[str, str, str]]) -> None:
+    def __init__(self, rows: list[tuple[str, str, str]], title: str = "Settings") -> None:
         super().__init__()
         self.rows = rows
+        self.title_text = title
 
     def compose(self) -> ComposeResult:
         with Vertical(id="wide"):
-            yield Static("Settings", id="question", markup=False)
+            yield Static(self.title_text, id="question", markup=False)
             yield DataTable(id="settings", cursor_type="row", show_header=False)
             yield Static("Enter changes it · Esc goes back", classes="hint")
             yield Footer()
@@ -512,9 +513,15 @@ class AIHawk(App[None]):
             ("judge_model", "Scoring model", config.judge_model),
             ("quick_model", "Search plan model", config.quick_model),
             ("decision_model", "Shortlist model", config.decision_model),
+            ("answers", "Your answers", f"{len(self.session.store.answers())} saved, given with the CV to every form"),
             ("folder", "Your files", str(config.workspace)),
         ]
 
+    def answer_rows(self) -> list[tuple[str, str, str]]:
+        """What the forms are told about the person: the answers in the store, and a row to add one."""
+        rows = [(f"answer:{name}", name.replace("_", " "), value) for name, value in self.session.store.answers().items()]
+        return [*rows, ("answer:", "Add an answer", "name: value")]
+
     def action_settings(self) -> None:
         if self.applying() or self.signing_in:
             self.say("One moment: the browser is in use.")
@@ -533,6 +540,8 @@ class AIHawk(App[None]):
                 return
             if choice == "folder":
                 webbrowser.open(self.session.config.workspace.as_uri())
+            elif choice == "answers":
+                await self.change_answers()
             elif choice == "key":
                 await self.change_key()
             else:
@@ -549,6 +558,23 @@ class AIHawk(App[None]):
             self.say("Searching again with the new settings.")
             self.start()
 
+    async def change_answers(self) -> None:
+        """One answer per row. Enter edits it, an empty value removes it; the last row adds one as `name: value`."""
+        store = self.session.store
+        while (choice := await self.push_screen_wait(Settings(self.answer_rows(), title="Your answers"))) is not None:
+            name = choice.removeprefix("answer:")
+            if name:
+                value = await self.push_screen_wait(TextPrompt(name.replace("_", " "), value=store.answers().get(name, ""), hint="Enter saves it · empty removes it · Esc leaves it"))
+                if value is not None:
+                    store.set_answer(name, value)
+            else:
+                line = await self.push_screen_wait(TextPrompt("A new answer, as name: value", placeholder="notice period: one month", hint="Enter saves it · Esc goes back"))
+                if line and ":" in line:
+                    new_name, new_value = line.split(":", 1)
+                    store.set_answer(new_name.strip().lower().replace(" ", "_"), new_value)
+                elif line:
+                    self.say("Write it as name: value.")
+
     async def change_key(self) -> None:
         config = self.session.config
         if config.key_from_environment():

tests/test_answers.py

@@ -0,0 +1,53 @@
+"""The answers: in the store's SQLite table, the old file brought in once, and the account password made once."""
+
+from __future__ import annotations
+
+import string
+from pathlib import Path
+
+from aihawk.config import load_config
+from aihawk.profile import ACCOUNT_PASSWORD, PASSWORD_SYMBOLS, add_answers, ensure_account_password, import_answers_file, new_password, read_answers
+from aihawk.store import Store
+
+
+def test_answers_are_kept_in_the_store_and_the_persons_answer_wins_over_the_cv(tmp_path: Path) -> None:
+    store = Store(tmp_path / "jobs.db")
+    store.add_answers({"email": "[email protected]", "phone": "", "location": "Milan"}, source="cv")
+    assert store.answers() == {"email": "[email protected]", "location": "Milan"}
+    store.add_answers({"email": "[email protected]", "website": "ada.example"}, source="cv")
+    assert store.answers()["email"] == "[email protected]" and store.answers()["website"] == "ada.example"
+    store.set_answer("email", "[email protected]")
+    store.set_answer("website", "  ")
+    assert store.answers() == {"email": "[email protected]", "location": "Milan"}
+    assert store.answers_text() == "email: [email protected]\nlocation: Milan"
+    store.close()
+
+
+def test_the_old_answers_file_is_brought_in_once_and_renamed(tmp_path: Path) -> None:
+    home = tmp_path / "home"
+    home.mkdir()
+    (home / "answers.yaml").write_bytes(b"\xef\xbb\xbfemail: [email protected]\nlocation: Milan\n")
+    config = load_config(home)
+    store = Store(config.store_file)
+    store.set_answer("location", "Rome")
+    assert import_answers_file(config, store) == 2
+    assert store.answers() == {"location": "Rome", "email": "[email protected]"}
+    assert not (home / "answers.yaml").exists() and (home / "answers.yaml.imported").is_file()
+    assert import_answers_file(config, store) == 0
+    store.close()
+    add_answers(config, {"phone": "+39 02"})
+    assert read_answers(config) == {"location": "Rome", "email": "[email protected]", "phone": "+39 02"}
+
+
+def test_the_account_password_is_made_once_and_has_every_class_of_character(tmp_path: Path) -> None:
+    store = Store(tmp_path / "jobs.db")
+    assert ensure_account_password(store) is True
+    password = store.answers()[ACCOUNT_PASSWORD]
+    assert ensure_account_password(store) is False and store.answers()[ACCOUNT_PASSWORD] == password
+    for _ in range(20):
+        made = new_password()
+        assert len(made) == 20
+        assert any(c in string.ascii_uppercase for c in made) and any(c in string.ascii_lowercase for c in made)
+        assert any(c in string.digits for c in made) and any(c in PASSWORD_SYMBOLS for c in made)
+    assert len({new_password() for _ in range(10)}) == 10
+    store.close()

tests/test_tui.py

@@ -160,7 +160,10 @@ def test_a_resume_is_all_it_takes_to_get_the_best_jobs_first(home: Path, resume:
     assert config.filters.locations == ("Milan", "Milano", "Italy")
     assert config.filters.cities == ("Milan", "Milano")
     assert config.cv.read_text(encoding="utf-8").startswith("# Ada Rossi")
-    assert read_answers(config) == {"email": "[email protected]", "location": "Milan"}
+    answers = read_answers(config)
+    assert {name: answers[name] for name in ("email", "location")} == {"email": "[email protected]", "location": "Milan"}
+    # The one password for the sites that need an account, made at the first start and kept with the answers.
+    assert len(answers["account_password"]) == 20
 
 
 def test_no_resume_asks_for_one_and_a_wrong_path_asks_again(home: Path, resume: Path) -> None: