mirror of
https://github.com/codewhale-hq/Codewhale.git
synced 2026-11-09 01:49:49 -10:00
fix(session): fence publications to captured native sources [skip ci]
Bind automatic writes and conditional checkpoint retirement to the original physical session/checkpoint pair and an opaque native producer. Keep SDK checkpoint ACKs ordered through the existing actor and preserve newer publications on stale, failed or closed producers. Protect history-operation witness binding with its original reservation and protected re-read. Keep readable transcripts available as data when optional accounting sidecars are damaged, without granting a write receipt. Preserve display titles and lifecycle metadata. Render exact typed compaction carriers as System receipts on reopen so bare undo retains real request boundaries and model history. Verification: normal CLI build 49.878s; affected TUI lib compile 81.067s. 59 Engine and121 TUI unique existing checks pass with frozen source-delta reuse; all originally selected checks now pass. Nine ordinary Cargo-cohort Manager controls pass. Authenticated local HTTP history/recovery/stale-source, unknown-delivery and actual Native SDK checkpoint/cancel/next-user pass. Real tmux TUI SDK/Actor journey passes with3 synthetic model POSTs,1 tool execution, exact retained prefix, graceful exit and actual EOF. HTTP Native SDK is separately3 POSTs; unknown-delivery is1. No external provider calls. Receipts: CW/artifacts/engine-0103-execution-20261102/checkpoint-causal-v7/RECEIPT.json and CHECK-COVERAGE.json; prior failures retained. This is a bounded local source slice, not full P03/P04, native platform, installed-package or release readiness.
This commit is contained in:
1 parent
8225f75b16
commit
20f8244aaf
61 files changed
+3111
-536
No files matched your search
@@ -160,6 +160,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
||||
|
||||
### Fixed
|
||||
|
||||
- Session publication compares the document and crash checkpoint captured before work was queued. A delayed save refuses changed records without replacing newer history or clearing its checkpoint. Ordered completion retains real checkpoint acknowledgments, and native turn/control identity follows restores, undo and terminal saves. Legacy checkpoints require explicit recovery; canonical history recovery retains its original source and exact prepared generation.
|
||||
- Damaged usage sidecars keep valid recovery data readable with incomplete coverage and no publication receipt. Known retirement, changed raw records and unsafe writes still refuse.
|
||||
- Local control and ACP output serialize within the existing 8 MiB frame limit and stop a damaged or stalled output stream. Compatibility replies, events and relay stdout now share the 30-second write deadline; owner forwarding keeps its tighter 5 seconds. Standalone and attached ACP recover after complete invalid UTF-8 or oversized input frames, discard oversized tails through the real newline, and preserve cancellation, permissions and actual Core receipts.
|
||||
- MCP composition records actual schema or description changes before the next request, so a conservative batch withdrawal re-pins the declared tool surface instead of failing the C5 prefix guard. Equal schemas retain their existing order. The `execute_tools` description also explains that visual input needs a direct capture call because nested results do not deliver inline images to the model.
|
||||
- Inference requests stop after uncertain delivery or an interrupted stream instead of silently sending another request. Automatic retries require an ordinary rate limit or a transient connection failure before dispatch. Redirects and HTTP protocol retries cannot bypass this policy; retained output and missing usage coverage remain visible. Terminal `exec` failures return exit 1; error categories no longer imply that repeating a saved command is safe.
|
||||
|
||||
@@ -2745,7 +2745,13 @@ impl Engine {
|
||||
self.apply_runtime_mode_policy(&authority);
|
||||
self.discard_kernels_if_narrowed(&prior).await;
|
||||
|
||||
let previous_source = self.session.snapshot_source.clone();
|
||||
self.session.snapshot_source = Some(crate::core::ops::SessionSnapshotSource::turn(
|
||||
turn_id.clone(),
|
||||
None,
|
||||
));
|
||||
start_permit.send(Event::TurnStarted {
|
||||
previous_source,
|
||||
turn_id: turn_id.clone(),
|
||||
created_at: chrono::Utc::now(),
|
||||
route: None,
|
||||
@@ -3303,6 +3309,7 @@ impl Engine {
|
||||
system_prompt: Option<codewhale_models::SystemPrompt>,
|
||||
system_prompt_override: bool,
|
||||
) {
|
||||
self.session.snapshot_source = None;
|
||||
self.session.tool_activation_cache.clear();
|
||||
let compaction_checkpoint =
|
||||
compaction_checkpoint_for_restore(&messages, system_prompt.clone());
|
||||
@@ -3350,6 +3357,7 @@ impl Engine {
|
||||
let total_tokens =
|
||||
self.session.total_usage.input_tokens + self.session.total_usage.output_tokens;
|
||||
SessionSnapshot {
|
||||
source: self.session.snapshot_source.clone(),
|
||||
session_id: self.session.id.clone(),
|
||||
messages: self.session.messages.to_vec(),
|
||||
total_tokens,
|
||||
@@ -4343,6 +4351,7 @@ impl Engine {
|
||||
self.config.fleet_roster = roster;
|
||||
}
|
||||
Op::SyncSession {
|
||||
control,
|
||||
session_id,
|
||||
messages,
|
||||
system_prompt,
|
||||
@@ -4421,6 +4430,8 @@ impl Engine {
|
||||
system_prompt,
|
||||
system_prompt_override,
|
||||
);
|
||||
self.session.snapshot_source =
|
||||
Some(crate::core::ops::SessionSnapshotSource::control(control));
|
||||
self.session.auto_model = model.trim().eq_ignore_ascii_case("auto");
|
||||
self.session.model = model;
|
||||
self.session.workspace = workspace.clone();
|
||||
@@ -4470,6 +4481,7 @@ impl Engine {
|
||||
self.emit_session_updated().await;
|
||||
}
|
||||
Op::RewindConversation {
|
||||
control,
|
||||
expected,
|
||||
messages,
|
||||
tx,
|
||||
@@ -4489,6 +4501,8 @@ impl Engine {
|
||||
self.session.system_prompt.clone(),
|
||||
self.session.system_prompt_override,
|
||||
);
|
||||
self.session.snapshot_source =
|
||||
Some(crate::core::ops::SessionSnapshotSource::control(control));
|
||||
self.session.rebuild_working_set();
|
||||
self.reconcile_restored_work_bindings().await;
|
||||
let _ = tx.send(Some(self.session_snapshot()));
|
||||
@@ -4862,6 +4876,7 @@ impl Engine {
|
||||
}
|
||||
let _ = self
|
||||
.send_event(Event::SessionUpdated {
|
||||
source: self.session.snapshot_source.clone(),
|
||||
checkpoint: None,
|
||||
session_id: self.session.id.clone(),
|
||||
messages: self.session.messages.snapshot(),
|
||||
@@ -6982,7 +6997,13 @@ impl Engine {
|
||||
// Emit turn started event IMMEDIATELY so the UI knows the turn is
|
||||
// active. The snapshot below can take 30+ seconds on slow filesystems
|
||||
// (e.g. WSL2 /mnt/c) and must not delay the TurnStarted event.
|
||||
let previous_source = self.session.snapshot_source.clone();
|
||||
self.session.snapshot_source = Some(crate::core::ops::SessionSnapshotSource::turn(
|
||||
turn.id.clone(),
|
||||
submission_id.clone(),
|
||||
));
|
||||
start_permit.send(Event::TurnStarted {
|
||||
previous_source,
|
||||
turn_id: turn.id.clone(),
|
||||
created_at: turn_started_at,
|
||||
route: Some(turn_route),
|
||||
|
||||
@@ -64,9 +64,13 @@ impl Engine {
|
||||
id: String,
|
||||
auto: bool,
|
||||
message: String,
|
||||
previous_source: Option<crate::core::ops::SessionSnapshotSource>,
|
||||
) {
|
||||
let _ = self
|
||||
.send_event(Event::CompactionStarted { id, auto, message })
|
||||
.send_event(Event::CompactionStarted {
|
||||
previous_source,
|
||||
id, auto, message,
|
||||
})
|
||||
.await;
|
||||
}
|
||||
|
||||
@@ -315,8 +319,6 @@ impl Engine {
|
||||
// Manual compaction terminals identify the compaction operation, never
|
||||
// a surrounding host submission or a model turn.
|
||||
let compaction_id = Some(id.clone());
|
||||
self.emit_compaction_started(id.clone(), false, "Making room…".to_string())
|
||||
.await;
|
||||
let Some(cancel_token) = self.claim_compaction(&id) else {
|
||||
let message = "Making room stopped before it started".to_string();
|
||||
self.emit_compaction_cancelled(id, false, message).await;
|
||||
@@ -336,6 +338,14 @@ impl Engine {
|
||||
.await;
|
||||
return;
|
||||
};
|
||||
let previous_source = self.session.snapshot_source.clone();
|
||||
self.session.snapshot_source = Some(crate::core::ops::SessionSnapshotSource::compaction(
|
||||
id.clone(),
|
||||
));
|
||||
self.emit_compaction_started(id.clone(), false, "Making room…".to_string(),
|
||||
previous_source,
|
||||
)
|
||||
.await;
|
||||
if let Err(err) = self.install_resolved_runtime_route(route) {
|
||||
let message =
|
||||
format!("Cannot compact context because its provider route is not ready: {err}");
|
||||
@@ -708,7 +718,9 @@ impl Engine {
|
||||
let compaction_cancel = self
|
||||
.claim_compaction(&compaction_id)
|
||||
.expect("a fresh automatic compaction id cannot be pre-canceled");
|
||||
self.emit_compaction_started(compaction_id.clone(), true, "Making room…".to_string())
|
||||
self.emit_compaction_started(compaction_id.clone(), true, "Making room…".to_string(),
|
||||
self.session.snapshot_source.clone(),
|
||||
)
|
||||
.await;
|
||||
let auto_messages_before = self.session.messages.len();
|
||||
let auto_tokens_before = self.estimated_input_tokens();
|
||||
@@ -933,7 +945,9 @@ impl Engine {
|
||||
.emergency_compaction_attempts
|
||||
.saturating_add(1);
|
||||
let start_message = format!("Making room now ({reason})");
|
||||
self.emit_compaction_started(id.clone(), true, start_message)
|
||||
self.emit_compaction_started(id.clone(), true, start_message,
|
||||
self.session.snapshot_source.clone(),
|
||||
)
|
||||
.await;
|
||||
let Some(compaction_cancel) = self.claim_compaction(&id) else {
|
||||
self.emit_compaction_cancelled(
|
||||
|
||||
@@ -94,6 +94,7 @@ async fn queued_manual_compaction_cancellation_is_idempotent_and_skips_route_act
|
||||
.expect("structurally resolve route without credential");
|
||||
let (mut engine, handle) = Engine::new(EngineConfig::default(), &route_config);
|
||||
let id = "compact-cancel-before-start";
|
||||
let before = engine.session_snapshot();
|
||||
|
||||
handle.cancel_compaction(id).expect("first cancel accepted");
|
||||
handle
|
||||
@@ -103,16 +104,17 @@ async fn queued_manual_compaction_cancellation_is_idempotent_and_skips_route_act
|
||||
.handle_manual_compaction_op(id.to_string(), route, CompactionConfig::default())
|
||||
.await;
|
||||
|
||||
assert_eq!(engine.session_snapshot(), before);
|
||||
|
||||
let mut events = handle.rx_event.write().await;
|
||||
let drained = std::iter::from_fn(|| events.try_recv().ok()).collect::<Vec<_>>();
|
||||
assert!(matches!(
|
||||
drained.as_slice(),
|
||||
[
|
||||
Event::CompactionStarted { id: started, auto: false, .. },
|
||||
Event::CompactionCancelled { id: cancelled, auto: false, .. },
|
||||
Event::TurnComplete { turn_id: None, submission_id: None,
|
||||
compaction_id: Some(terminal), status: TurnOutcomeStatus::Interrupted, .. }
|
||||
] if started == id && cancelled == id && terminal == id
|
||||
] if cancelled == id && terminal == id
|
||||
));
|
||||
assert!(
|
||||
!drained
|
||||
|
||||
@@ -631,6 +631,7 @@ async fn edit_last_turn_restores_the_exchange_when_the_replacement_never_starts(
|
||||
};
|
||||
handle
|
||||
.send(Op::SyncSession {
|
||||
control: crate::core::ops::SessionControlId::new(),
|
||||
session_id: Some("edit-restore".to_string()),
|
||||
messages: vec![
|
||||
text(Role::User, "original prompt"),
|
||||
|
||||
@@ -361,7 +361,9 @@ async fn verify_next_user_turn_after_loss(failure: Failure) {
|
||||
),
|
||||
"this regression has no pending approval or user-input gate: {event:?}"
|
||||
);
|
||||
if let Event::SessionUpdated { session_id, .. } = event {
|
||||
if let Event::SessionUpdated {
|
||||
source: _,
|
||||
session_id, .. } = event {
|
||||
assert_eq!(session_id, SESSION_ID);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -64,7 +64,9 @@ async fn queued_goal_clear_refreshes_prompt_and_cancels_stale_continuation() {
|
||||
Event::TurnStarted { .. } => {
|
||||
panic!("queued clear must prevent a stale goal continuation")
|
||||
}
|
||||
Event::SessionUpdated { system_prompt, .. } => {
|
||||
Event::SessionUpdated {
|
||||
source: _,
|
||||
system_prompt, .. } => {
|
||||
let prompt = match system_prompt.expect("clear SessionUpdated prompt") {
|
||||
SystemPrompt::Text(text) => text,
|
||||
SystemPrompt::Blocks(blocks) => blocks
|
||||
|
||||
@@ -430,7 +430,9 @@ async fn truncated_response_continues_turn() {
|
||||
assert_eq!(reported.reasoning_tokens, Some(3));
|
||||
saw_turn_usage = true;
|
||||
}
|
||||
Event::SessionUpdated { messages, .. } => {
|
||||
Event::SessionUpdated {
|
||||
source: _,
|
||||
messages, .. } => {
|
||||
saw_truncation_observation |= messages.iter().any(|message| {
|
||||
message.content.iter().any(|block| {
|
||||
matches!(
|
||||
|
||||
@@ -83,8 +83,8 @@ fn agent_mode_elevates_writes_without_granting_network() {
|
||||
#[test]
|
||||
fn sandbox_policy_for_turn_returns_correct_default_policy_per_mode() {
|
||||
use crate::core::authority::{SandboxNetworkAccess, sandbox_policy_for_turn};
|
||||
use codewhale_sandbox::SandboxPolicy;
|
||||
use ApprovalMode;
|
||||
use codewhale_sandbox::SandboxPolicy;
|
||||
|
||||
let workspace = PathBuf::from("/tmp/example-workspace");
|
||||
|
||||
@@ -181,7 +181,9 @@ async fn session_update_preserves_reasoning_tool_only_turn() {
|
||||
let mut rx = handle.rx_event.write().await;
|
||||
rx.recv().await.expect("session update event")
|
||||
};
|
||||
let Event::SessionUpdated { messages, .. } = event else {
|
||||
let Event::SessionUpdated {
|
||||
source: _,
|
||||
messages, .. } = event else {
|
||||
panic!("expected session update event");
|
||||
};
|
||||
|
||||
@@ -342,8 +344,8 @@ async fn posture_change_status_uses_permissions_and_work() {
|
||||
|
||||
#[tokio::test]
|
||||
async fn live_runtime_authority_applies_latest_posture_and_sandbox_before_tools() {
|
||||
use codewhale_sandbox::SandboxPolicy;
|
||||
use ApprovalMode;
|
||||
use codewhale_sandbox::SandboxPolicy;
|
||||
|
||||
let tmp = tempdir().expect("tempdir");
|
||||
let config = EngineConfig {
|
||||
@@ -1004,6 +1006,7 @@ async fn unchanged_compaction_config_is_acknowledged_silently() {
|
||||
let run = tokio::spawn(engine.run());
|
||||
handle
|
||||
.send(Op::SyncSession {
|
||||
control: crate::core::ops::SessionControlId::new(),
|
||||
session_id: Some("resumed-session".to_string()),
|
||||
messages: restored_messages.clone(),
|
||||
system_prompt: None,
|
||||
@@ -1110,7 +1113,9 @@ async fn change_mode_op_updates_current_mode_and_emits_status() {
|
||||
.await
|
||||
.expect("session update after mode switch")
|
||||
.expect("event");
|
||||
let Event::SessionUpdated { messages, .. } = session_updated else {
|
||||
let Event::SessionUpdated {
|
||||
source: _,
|
||||
messages, .. } = session_updated else {
|
||||
panic!("should emit SessionUpdated after mode change, got: {session_updated:?}");
|
||||
};
|
||||
assert!(
|
||||
@@ -1195,6 +1200,7 @@ async fn sync_session_restores_current_mode() {
|
||||
let run = tokio::spawn(engine.run());
|
||||
handle
|
||||
.send(Op::SyncSession {
|
||||
control: crate::core::ops::SessionControlId::new(),
|
||||
session_id: Some("plan-session".to_string()),
|
||||
messages: Vec::new(),
|
||||
system_prompt: None,
|
||||
@@ -1275,6 +1281,7 @@ async fn sync_session_without_prompt_repins_full_system_prompt_on_next_turn() {
|
||||
|
||||
handle
|
||||
.send(Op::SyncSession {
|
||||
control: crate::core::ops::SessionControlId::new(),
|
||||
session_id: Some("fresh-session".to_string()),
|
||||
messages: Vec::new(),
|
||||
system_prompt: None,
|
||||
@@ -1356,6 +1363,7 @@ async fn sync_session_same_id_does_not_finalize_live_worker() {
|
||||
// construction-time UUID.
|
||||
handle
|
||||
.send(Op::SyncSession {
|
||||
control: crate::core::ops::SessionControlId::new(),
|
||||
session_id: Some("session-keep".to_string()),
|
||||
messages: Vec::new(),
|
||||
system_prompt: None,
|
||||
@@ -1376,6 +1384,7 @@ async fn sync_session_same_id_does_not_finalize_live_worker() {
|
||||
// A same-id re-sync is a reload, not a conversation boundary.
|
||||
handle
|
||||
.send(Op::SyncSession {
|
||||
control: crate::core::ops::SessionControlId::new(),
|
||||
session_id: Some("session-keep".to_string()),
|
||||
messages: Vec::new(),
|
||||
system_prompt: None,
|
||||
@@ -1417,6 +1426,7 @@ async fn sync_session_different_id_finalizes_live_worker() {
|
||||
let run = tokio::spawn(engine.run());
|
||||
handle
|
||||
.send(Op::SyncSession {
|
||||
control: crate::core::ops::SessionControlId::new(),
|
||||
session_id: Some("session-a".to_string()),
|
||||
messages: Vec::new(),
|
||||
system_prompt: None,
|
||||
@@ -1440,6 +1450,7 @@ async fn sync_session_different_id_finalizes_live_worker() {
|
||||
// finalized with the session-closed reason.
|
||||
handle
|
||||
.send(Op::SyncSession {
|
||||
control: crate::core::ops::SessionControlId::new(),
|
||||
session_id: Some("session-b".to_string()),
|
||||
messages: Vec::new(),
|
||||
system_prompt: None,
|
||||
@@ -1496,6 +1507,7 @@ async fn sync_session_migrates_one_checkpoint_and_strips_its_system_carrier() {
|
||||
for round in 0..2 {
|
||||
handle
|
||||
.send(Op::SyncSession {
|
||||
control: crate::core::ops::SessionControlId::new(),
|
||||
session_id: Some("compacted-session".to_string()),
|
||||
messages,
|
||||
system_prompt: Some(carrier.clone()),
|
||||
|
||||
@@ -27,6 +27,7 @@ async fn sync_session_projects_persisted_subagent_handoff_for_headless_restore()
|
||||
let run = tokio::spawn(engine.run());
|
||||
handle
|
||||
.send(Op::SyncSession {
|
||||
control: crate::core::ops::SessionControlId::new(),
|
||||
session_id: Some("headless-resume".to_string()),
|
||||
messages,
|
||||
system_prompt: None,
|
||||
@@ -55,7 +56,8 @@ async fn sync_session_projects_persisted_subagent_handoff_for_headless_restore()
|
||||
|block| matches!(block, ContentBlock::Text { text, .. } if text == "Keep the original task")
|
||||
));
|
||||
let restored =
|
||||
codewhale_models::turn_classify::restored_subagent_checkpoint_display(&snapshot.messages[1])
|
||||
codewhale_models::turn_classify::restored_subagent_checkpoint_display(&snapshot.messages[1],
|
||||
)
|
||||
.expect("projected headless checkpoint");
|
||||
assert!(restored.contains("agent_headless"));
|
||||
assert!(restored.contains("Checkpoint: engine restore is covered."));
|
||||
@@ -164,6 +166,7 @@ async fn edit_last_turn_preserves_current_mode() {
|
||||
];
|
||||
handle
|
||||
.send(Op::SyncSession {
|
||||
control: crate::core::ops::SessionControlId::new(),
|
||||
session_id: Some("edit-mode-test".to_string()),
|
||||
messages: seeded_messages,
|
||||
system_prompt: None,
|
||||
@@ -320,6 +323,7 @@ async fn edit_last_turn_cuts_at_user_prompt_before_tool_results() {
|
||||
];
|
||||
handle
|
||||
.send(Op::SyncSession {
|
||||
control: crate::core::ops::SessionControlId::new(),
|
||||
session_id: Some("edit-cut-test".to_string()),
|
||||
messages: seeded_messages,
|
||||
system_prompt: None,
|
||||
@@ -424,6 +428,7 @@ async fn edit_last_turn_without_user_prompt_errors_and_sends_nothing() {
|
||||
// must surface an error instead of silently appending the message.
|
||||
handle
|
||||
.send(Op::SyncSession {
|
||||
control: crate::core::ops::SessionControlId::new(),
|
||||
session_id: Some("edit-no-user-test".to_string()),
|
||||
messages: vec![Message {
|
||||
role: Role::Assistant,
|
||||
@@ -546,6 +551,7 @@ async fn edit_last_turn_without_user_prompt_errors_and_sends_nothing() {
|
||||
];
|
||||
handle
|
||||
.send(Op::SyncSession {
|
||||
control: crate::core::ops::SessionControlId::new(),
|
||||
session_id: Some("edit-unsupported-user-test".to_string()),
|
||||
messages: image_only_history.clone(),
|
||||
system_prompt: None,
|
||||
@@ -727,7 +733,8 @@ fn context_budget_scenario() {
|
||||
let _lock = lock_test_env();
|
||||
let budget = context_input_budget_for_provider(ProviderKind::OpenaiCodex, "gpt-5.5")
|
||||
.expect("OpenAI Codex should use a conservative fallback without route metadata");
|
||||
let expected = usize::try_from(codewhale_runtime::config::OPENAI_CODEX_EFFECTIVE_CONTEXT_WINDOW_TOKENS)
|
||||
let expected = usize::try_from(codewhale_runtime::config::OPENAI_CODEX_EFFECTIVE_CONTEXT_WINDOW_TOKENS,
|
||||
)
|
||||
.expect("context window fits usize")
|
||||
- crate::provider_lake::provider_capability(ProviderKind::OpenaiCodex, "gpt-5.5")
|
||||
.max_output
|
||||
|
||||
@@ -88,7 +88,9 @@ async fn trust_warning_is_internal_and_tracks_current_state() {
|
||||
engine.refresh_system_prompt_with_reason("model");
|
||||
engine.emit_session_updated().await;
|
||||
let event = handle.rx_event.write().await.recv().await.unwrap();
|
||||
let Event::SessionUpdated { messages, .. } = event else {
|
||||
let Event::SessionUpdated {
|
||||
source: _,
|
||||
messages, .. } = event else {
|
||||
panic!("expected session update");
|
||||
};
|
||||
assert_eq!(
|
||||
|
||||
@@ -325,7 +325,9 @@ async fn headless_turn_retries_mid_stream_network_drop_and_recovers() {
|
||||
let transcript_text = events
|
||||
.iter()
|
||||
.filter_map(|event| match event {
|
||||
Event::SessionUpdated { messages, .. } => Some(messages),
|
||||
Event::SessionUpdated {
|
||||
source: _,
|
||||
messages, .. } => Some(messages),
|
||||
_ => None,
|
||||
})
|
||||
.flat_map(|messages| messages.iter())
|
||||
@@ -1250,7 +1252,9 @@ async fn interactive_turn_preserves_partial_reply_and_recovers_after_network_dro
|
||||
.iter()
|
||||
.rev()
|
||||
.find_map(|event| match event {
|
||||
Event::SessionUpdated { messages, .. } => Some(messages.clone()),
|
||||
Event::SessionUpdated {
|
||||
source: _,
|
||||
messages, .. } => Some(messages.clone()),
|
||||
_ => None,
|
||||
})
|
||||
.expect("final SessionUpdated");
|
||||
|
||||
@@ -111,7 +111,9 @@ async fn interactive_thinking_only_drop_preserves_nothing_and_never_claims_it_di
|
||||
.iter()
|
||||
.rev()
|
||||
.find_map(|event| match event {
|
||||
Event::SessionUpdated { messages, .. } => Some(messages.clone()),
|
||||
Event::SessionUpdated {
|
||||
source: _,
|
||||
messages, .. } => Some(messages.clone()),
|
||||
_ => None,
|
||||
})
|
||||
.expect("final SessionUpdated");
|
||||
|
||||
@@ -3854,6 +3854,7 @@ impl Engine {
|
||||
};
|
||||
let request = crate::core::events::SessionCheckpointRequest::new(caller, snapshot, prefix_sha256, projection, withdraw, reply);
|
||||
let event = Event::SessionUpdated {
|
||||
source: self.session.snapshot_source.clone(),
|
||||
checkpoint: Some(request.clone()),
|
||||
session_id: self.session.id.clone(),
|
||||
messages: self.session.messages.snapshot(),
|
||||
|
||||
@@ -290,6 +290,12 @@ impl SessionCheckpointRequest {
|
||||
return Err("session checkpoint was withdrawn".into());
|
||||
}
|
||||
if self.caller.agent_id.is_some()
|
||||
|| self
|
||||
.snapshot
|
||||
.source
|
||||
.as_ref()
|
||||
.and_then(|source| source.turn_id.as_deref())
|
||||
!= Some(self.caller.turn_id.as_str())
|
||||
|| session_id != self.caller.session_id
|
||||
|| session_id != self.snapshot.session_id
|
||||
|| workspace != self.caller.workspace
|
||||
@@ -489,6 +495,7 @@ pub enum Event {
|
||||
// === Turn Lifecycle ===
|
||||
/// A new turn has started (user sent a message)
|
||||
TurnStarted {
|
||||
previous_source: Option<super::ops::SessionSnapshotSource>,
|
||||
turn_id: String,
|
||||
created_at: DateTime<Utc>,
|
||||
/// Legacy/non-model hosts may still attach a route at start. Model
|
||||
@@ -627,6 +634,7 @@ pub enum Event {
|
||||
|
||||
/// Context compaction started.
|
||||
CompactionStarted {
|
||||
previous_source: Option<super::ops::SessionSnapshotSource>,
|
||||
id: String,
|
||||
auto: bool,
|
||||
message: String,
|
||||
@@ -894,6 +902,7 @@ pub enum Event {
|
||||
/// text block, and that assistant message still has to be persisted for
|
||||
/// later `reasoning_content` replay.
|
||||
SessionUpdated {
|
||||
source: Option<crate::core::ops::SessionSnapshotSource>,
|
||||
checkpoint: Option<Arc<SessionCheckpointRequest>>,
|
||||
session_id: String,
|
||||
/// Shared history snapshot (#6214 T2): the engine hands out an `Arc`
|
||||
|
||||
@@ -18,10 +18,85 @@ pub const USER_SHELL_TOOL_ID_PREFIX: &str = "user_shell_";
|
||||
|
||||
pub type FleetMemberStarted = Result<String, String>;
|
||||
|
||||
#[derive(Debug, Clone, PartialEq, Eq, serde::Serialize)]
|
||||
pub struct SessionControlId(String);
|
||||
|
||||
impl Default for SessionControlId {
|
||||
fn default() -> Self {
|
||||
Self::new()
|
||||
}
|
||||
}
|
||||
|
||||
impl SessionControlId {
|
||||
pub fn new() -> Self {
|
||||
Self(uuid::Uuid::new_v4().to_string())
|
||||
}
|
||||
|
||||
pub(crate) fn for_admitted_history(id: String) -> Self {
|
||||
Self(id)
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, PartialEq, Eq, serde::Serialize)]
|
||||
pub struct SessionSnapshotSource {
|
||||
pub turn_id: Option<String>,
|
||||
pub submission_id: Option<String>,
|
||||
pub compaction_id: Option<String>,
|
||||
#[serde(skip_serializing_if = "Option::is_none")]
|
||||
pub control_id: Option<SessionControlId>,
|
||||
}
|
||||
|
||||
impl SessionSnapshotSource {
|
||||
pub fn turn(turn_id: String, submission_id: Option<String>) -> Self {
|
||||
Self {
|
||||
turn_id: Some(turn_id),
|
||||
submission_id,
|
||||
compaction_id: None,
|
||||
control_id: None,
|
||||
}
|
||||
}
|
||||
|
||||
pub fn control(control_id: SessionControlId) -> Self {
|
||||
Self {
|
||||
turn_id: None,
|
||||
submission_id: None,
|
||||
compaction_id: None,
|
||||
control_id: Some(control_id),
|
||||
}
|
||||
}
|
||||
|
||||
pub fn compaction(compaction_id: String) -> Self {
|
||||
Self {
|
||||
turn_id: None,
|
||||
submission_id: None,
|
||||
compaction_id: Some(compaction_id),
|
||||
control_id: None,
|
||||
}
|
||||
}
|
||||
|
||||
pub fn terminal(
|
||||
turn_id: Option<String>,
|
||||
submission_id: Option<String>,
|
||||
compaction_id: Option<String>,
|
||||
) -> Option<Self> {
|
||||
if turn_id.is_some() == compaction_id.is_some() {
|
||||
return None;
|
||||
}
|
||||
Some(Self {
|
||||
turn_id,
|
||||
submission_id,
|
||||
compaction_id,
|
||||
control_id: None,
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
/// Snapshot of session state for saving to disk.
|
||||
/// Returned by `Op::GetSessionSnapshot` via a oneshot channel.
|
||||
#[derive(Debug, Clone, PartialEq, serde::Serialize)]
|
||||
pub struct SessionSnapshot {
|
||||
#[serde(skip_serializing_if = "Option::is_none")]
|
||||
pub source: Option<SessionSnapshotSource>,
|
||||
/// The live conversation id this engine session is running under.
|
||||
///
|
||||
/// Every workspace snapshot the conversation takes is tagged with it. A
|
||||
@@ -385,6 +460,7 @@ pub enum Op {
|
||||
|
||||
/// Sync engine session state (used for resume/load)
|
||||
SyncSession {
|
||||
control: SessionControlId,
|
||||
session_id: Option<String>,
|
||||
messages: Vec<Message>,
|
||||
system_prompt: Option<SystemPrompt>,
|
||||
@@ -398,6 +474,7 @@ pub enum Op {
|
||||
/// compares the full expected state before changing history or caches.
|
||||
/// A rejected rewind returns None and performs no mutation or inference.
|
||||
RewindConversation {
|
||||
control: SessionControlId,
|
||||
expected: Box<SessionSnapshot>,
|
||||
messages: Vec<Message>,
|
||||
tx: tokio::sync::oneshot::Sender<Option<SessionSnapshot>>,
|
||||
|
||||
@@ -594,6 +594,7 @@ pub fn event_to_protocol(event: &Event, ids: &ProtocolIds) -> wire::EventMsg {
|
||||
created_at,
|
||||
route,
|
||||
submission_id,
|
||||
..
|
||||
} => wire::EventMsg::TurnStarted {
|
||||
thread_id,
|
||||
session_id,
|
||||
@@ -698,7 +699,8 @@ pub fn event_to_protocol(event: &Event, ids: &ProtocolIds) -> wire::EventMsg {
|
||||
interrupted: *interrupted,
|
||||
}
|
||||
}
|
||||
Event::CompactionStarted { id, auto, message } => wire::EventMsg::CompactionStarted {
|
||||
Event::CompactionStarted { id, auto, message, ..
|
||||
} => wire::EventMsg::CompactionStarted {
|
||||
thread_id,
|
||||
session_id,
|
||||
id: id.clone(),
|
||||
@@ -993,6 +995,7 @@ pub fn event_to_protocol(event: &Event, ids: &ProtocolIds) -> wire::EventMsg {
|
||||
},
|
||||
},
|
||||
Event::SessionUpdated {
|
||||
source: _,
|
||||
checkpoint: _,
|
||||
session_id: engine_session_id,
|
||||
messages,
|
||||
@@ -1254,6 +1257,7 @@ pub fn op_to_protocol(op: &Op) -> wire_op::Op {
|
||||
model,
|
||||
workspace,
|
||||
mode,
|
||||
..
|
||||
} => wire_op::Op::SyncSession {
|
||||
engine_session_id: session_id.clone(),
|
||||
messages: messages.iter().map(to_value).collect(),
|
||||
@@ -1267,6 +1271,7 @@ pub fn op_to_protocol(op: &Op) -> wire_op::Op {
|
||||
expected,
|
||||
messages,
|
||||
tx: _,
|
||||
..
|
||||
} => wire_op::Op::RewindConversation {
|
||||
expected: to_value(expected),
|
||||
messages: messages.iter().map(to_value).collect(),
|
||||
@@ -1576,6 +1581,7 @@ mod tests {
|
||||
result: Err(ToolError::Timeout { seconds: 9 }),
|
||||
},
|
||||
Event::TurnStarted {
|
||||
previous_source: None,
|
||||
turn_id: "turn-1".into(),
|
||||
created_at: chrono::Utc::now(),
|
||||
route: None,
|
||||
@@ -1711,6 +1717,8 @@ mod tests {
|
||||
#[test]
|
||||
fn conditional_rewind_projection_preserves_expected_state_and_reply_owner() {
|
||||
let expected = crate::core::ops::SessionSnapshot {
|
||||
source: None,
|
||||
|
||||
session_id: "observed-conversation".into(),
|
||||
messages: vec![],
|
||||
total_tokens: 9,
|
||||
@@ -1724,6 +1732,7 @@ mod tests {
|
||||
};
|
||||
let (tx, mut receive) = tokio::sync::oneshot::channel();
|
||||
let op = Op::RewindConversation {
|
||||
control: crate::core::ops::SessionControlId::new(),
|
||||
expected: Box::new(expected.clone()),
|
||||
messages: vec![],
|
||||
tx,
|
||||
|
||||
@@ -300,6 +300,7 @@ impl ToolActivationCache {
|
||||
/// Session state for the engine.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct Session {
|
||||
pub snapshot_source: Option<crate::core::ops::SessionSnapshotSource>,
|
||||
/// Model being used
|
||||
pub model: String,
|
||||
|
||||
@@ -467,6 +468,7 @@ impl Session {
|
||||
approval_mode: ApprovalMode::Suggest,
|
||||
notes_path,
|
||||
mcp_config_path,
|
||||
snapshot_source: None,
|
||||
id: uuid::Uuid::new_v4().to_string(),
|
||||
project_context: None,
|
||||
last_system_prompt_hash: None,
|
||||
|
||||
@@ -1277,6 +1277,11 @@ fn the_wire_result_is_the_tools_text_and_json_and_says_when_it_was_cut() {
|
||||
SessionCheckpointRequest,
|
||||
};
|
||||
let snapshot = Arc::new(crate::core::ops::SessionSnapshot {
|
||||
source: Some(crate::core::ops::SessionSnapshotSource::turn(
|
||||
"turn".into(),
|
||||
None,
|
||||
)),
|
||||
|
||||
session_id: "session".into(),
|
||||
messages: Vec::new(),
|
||||
total_tokens: 0,
|
||||
|
||||
@@ -411,6 +411,8 @@ pub struct ThreadReadMarker {
|
||||
|
||||
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)]
|
||||
pub struct SavedSessionCheckpoint {
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub publication_observation: Option<crate::session_manager::SessionObservation>,
|
||||
pub covered_turn_id: Option<String>,
|
||||
/// Fingerprint of the document's first `messages_len` messages.
|
||||
pub messages_sha256: String,
|
||||
@@ -4721,6 +4723,10 @@ impl HeldRuntimeStore {
|
||||
let mut thread = store.load_thread(thread_id)?;
|
||||
thread.session_id = Some(session.metadata.id.clone());
|
||||
thread.saved_session_checkpoint = Some(SavedSessionCheckpoint {
|
||||
publication_observation: session
|
||||
.write_receipt
|
||||
.as_ref()
|
||||
.and_then(|receipt| receipt.observation().ok()),
|
||||
covered_turn_id: thread.latest_turn_id.clone(),
|
||||
messages_sha256: session_messages_sha256(&session.messages)?,
|
||||
messages_len: Some(session.messages.len()),
|
||||
@@ -5069,6 +5075,8 @@ pub struct RuntimeHistoryOperation {
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub target_document_digest: Option<String>,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
publication_base: Option<crate::session_manager::SessionObservation>,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
goal_source_digest: Option<String>,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
goal_target_digest: Option<String>,
|
||||
@@ -5091,6 +5099,18 @@ pub struct RuntimeHistoryWitness {
|
||||
pub workspace: PathBuf,
|
||||
}
|
||||
impl RuntimeHistoryOperation {
|
||||
pub fn session_publication_id(&self) -> Result<String> {
|
||||
self.validate()?;
|
||||
Ok(codewhale_runtime::hashing::sha256_hex(
|
||||
codewhale_runtime::hashing::canonical_json(&serde_json::json!({
|
||||
"session_publication": 1,
|
||||
"key_fingerprint": self.key_fingerprint,
|
||||
"receipt": self.receipt,
|
||||
}))
|
||||
.as_bytes(),
|
||||
))
|
||||
}
|
||||
|
||||
fn validate(&self) -> Result<()> {
|
||||
anyhow::ensure!(
|
||||
self.version == 1 && self.receipt.version == 1,
|
||||
|
||||
@@ -8005,7 +8005,7 @@ impl RuntimeThreadManager {
|
||||
"invalid existing history target identity"
|
||||
);
|
||||
}
|
||||
let operation = RuntimeHistoryOperation {
|
||||
let mut operation = RuntimeHistoryOperation {
|
||||
version: 1,
|
||||
key_fingerprint: fingerprint.clone(),
|
||||
committed: false,
|
||||
@@ -8013,6 +8013,7 @@ impl RuntimeThreadManager {
|
||||
journal_witness: None,
|
||||
association: None,
|
||||
target_document_digest: None,
|
||||
publication_base: None,
|
||||
goal_source_digest: None,
|
||||
goal_target_digest: None,
|
||||
session_goal_source_digest: None,
|
||||
@@ -8032,6 +8033,15 @@ impl RuntimeThreadManager {
|
||||
.unwrap_or_else(|| uuid::Uuid::new_v4().to_string()),
|
||||
},
|
||||
};
|
||||
let sessions =
|
||||
crate::session_manager::SessionManager::new(self.sessions_dir().to_path_buf())?;
|
||||
let source = if target.is_some() {
|
||||
sessions.capture_session_write_source(&operation.receipt.session_id)?
|
||||
} else {
|
||||
sessions.expect_absent_session(&operation.receipt.session_id)?
|
||||
};
|
||||
operation.publication_base = Some(source.observation()?);
|
||||
operation.validate()?;
|
||||
self.store.save_history_operation(&operation)?;
|
||||
Ok(operation)
|
||||
})
|
||||
@@ -8076,14 +8086,14 @@ impl RuntimeThreadManager {
|
||||
|
||||
pub fn bind_history_operation_witness(
|
||||
&self,
|
||||
mut operation: RuntimeHistoryOperation,
|
||||
operation: RuntimeHistoryOperation,
|
||||
witness: RuntimeHistoryWitness,
|
||||
association: codewhale_protocol::CanonicalThreadOperationAssociation,
|
||||
) -> Result<RuntimeHistoryOperation> {
|
||||
let fingerprint = operation.key_fingerprint.clone();
|
||||
self.store
|
||||
.with_turn_operation_claim(Some(&fingerprint), || {
|
||||
let current = self
|
||||
let mut current = self
|
||||
.store
|
||||
.load_history_operation(&fingerprint)?
|
||||
.ok_or_else(|| anyhow!("history reservation is absent"))?;
|
||||
@@ -8102,11 +8112,11 @@ impl RuntimeThreadManager {
|
||||
!current.committed,
|
||||
"committed operation has no journal witness"
|
||||
);
|
||||
operation.journal_witness = Some(witness);
|
||||
operation.association = Some(association);
|
||||
operation.validate()?;
|
||||
self.store.save_history_operation(&operation)?;
|
||||
Ok(operation)
|
||||
current.journal_witness = Some(witness);
|
||||
current.association = Some(association);
|
||||
current.validate()?;
|
||||
self.store.save_history_operation(¤t)?;
|
||||
Ok(current)
|
||||
})
|
||||
}
|
||||
|
||||
@@ -8202,6 +8212,7 @@ impl RuntimeThreadManager {
|
||||
};
|
||||
engine
|
||||
.send(Op::SyncSession {
|
||||
control: crate::core::ops::SessionControlId::new(),
|
||||
session_id: Some(thread.id.clone()),
|
||||
messages: session.messages.clone(),
|
||||
system_prompt_override: system_prompt.is_some(),
|
||||
@@ -8456,6 +8467,124 @@ impl RuntimeThreadManager {
|
||||
Ok(())
|
||||
}
|
||||
|
||||
pub fn saved_session_publication_source(
|
||||
&self,
|
||||
thread_id: &str,
|
||||
) -> Result<Option<crate::session_manager::SessionWriteReceipt>> {
|
||||
let _thread_mutation = self.store.thread_mutation.lock();
|
||||
let thread = self.store.load_thread(thread_id)?;
|
||||
let Some(id) = thread.session_id.as_deref() else {
|
||||
return Ok(None);
|
||||
};
|
||||
let Some(observation) = thread
|
||||
.saved_session_checkpoint
|
||||
.as_ref()
|
||||
.and_then(|checkpoint| checkpoint.publication_observation.clone())
|
||||
else {
|
||||
return Ok(None);
|
||||
};
|
||||
let source = crate::session_manager::SessionWriteReceipt::observed(observation);
|
||||
anyhow::ensure!(
|
||||
source.targets(self.sessions_dir(), id)?,
|
||||
"saved publication source belongs to another store or session"
|
||||
);
|
||||
Ok(Some(source))
|
||||
}
|
||||
|
||||
pub fn capture_history_publication_source(
|
||||
&self,
|
||||
operation: &RuntimeHistoryOperation,
|
||||
source: crate::session_manager::SessionWriteReceipt,
|
||||
) -> Result<crate::session_manager::SessionWriteReceipt> {
|
||||
validate_runtime_turn_operation_key(&operation.receipt.operation_key)?;
|
||||
let fingerprint = codewhale_runtime::hashing::sha256_hex(
|
||||
format!(
|
||||
"codewhale:history-operation:v1\0{}",
|
||||
operation.receipt.operation_key
|
||||
)
|
||||
.as_bytes(),
|
||||
);
|
||||
self.store
|
||||
.with_turn_operation_claim(Some(&fingerprint), || {
|
||||
let current = self
|
||||
.lookup_history_operation(
|
||||
&operation.receipt.operation_key,
|
||||
&operation.receipt.request_digest,
|
||||
)?
|
||||
.ok_or_else(|| anyhow!("history reservation is absent"))?;
|
||||
anyhow::ensure!(
|
||||
current.receipt == operation.receipt
|
||||
&& !current.committed
|
||||
&& current.journal_witness == operation.journal_witness
|
||||
&& current.association == operation.association,
|
||||
"history operation changed before native publication admission"
|
||||
);
|
||||
anyhow::ensure!(
|
||||
source.targets(self.sessions_dir(), ¤t.receipt.session_id)?,
|
||||
"history publication source belongs to another store or target"
|
||||
);
|
||||
anyhow::ensure!(
|
||||
current.publication_base.as_ref() == Some(&source.observation()?),
|
||||
"history original publication source is absent or changed; recovery required"
|
||||
);
|
||||
Ok(source.capture_history_control(¤t)?)
|
||||
})
|
||||
}
|
||||
|
||||
pub fn verify_prepared_history_publication(
|
||||
&self,
|
||||
operation: &RuntimeHistoryOperation,
|
||||
session: &crate::session_manager::SavedSession,
|
||||
) -> Result<()> {
|
||||
validate_runtime_turn_operation_key(&operation.receipt.operation_key)?;
|
||||
let fingerprint = codewhale_runtime::hashing::sha256_hex(
|
||||
format!(
|
||||
"codewhale:history-operation:v1\0{}",
|
||||
operation.receipt.operation_key
|
||||
)
|
||||
.as_bytes(),
|
||||
);
|
||||
self.store
|
||||
.with_turn_operation_claim(Some(&fingerprint), || {
|
||||
let current = self
|
||||
.lookup_history_operation(
|
||||
&operation.receipt.operation_key,
|
||||
&operation.receipt.request_digest,
|
||||
)?
|
||||
.ok_or_else(|| anyhow!("history reservation is absent"))?;
|
||||
anyhow::ensure!(
|
||||
current.receipt == operation.receipt && !current.committed,
|
||||
"prepared history operation changed before recovery"
|
||||
);
|
||||
let source = session.write_receipt.as_ref().ok_or_else(|| {
|
||||
anyhow!("prepared target lacks an admitted native read source")
|
||||
})?;
|
||||
anyhow::ensure!(
|
||||
source.targets(self.sessions_dir(), ¤t.receipt.session_id)?,
|
||||
"prepared target belongs to another store or session"
|
||||
);
|
||||
let observed = source.observation()?;
|
||||
anyhow::ensure!(
|
||||
session.metadata.publication_id.as_deref()
|
||||
== Some(current.session_publication_id()?.as_str()),
|
||||
"uncommitted target belongs to another publication generation; recovery required"
|
||||
);
|
||||
anyhow::ensure!(
|
||||
current
|
||||
.publication_base
|
||||
.as_ref()
|
||||
.is_some_and(|base| base.same_checkpoint(&observed)),
|
||||
"prepared target checkpoint changed or its original source is unknown"
|
||||
);
|
||||
anyhow::ensure!(
|
||||
current.target_document_digest.as_ref()
|
||||
== Some(&history::saved_document_digest(session)?),
|
||||
"uncommitted target is not its exact prepared publication; recovery required"
|
||||
);
|
||||
Ok(())
|
||||
})
|
||||
}
|
||||
|
||||
/// Bind full-fidelity saved history to the exact prefix it covers. Callers
|
||||
/// hold session_checkpoint_guard across the snapshot, save and this write.
|
||||
pub async fn set_thread_session_checkpoint(
|
||||
@@ -8470,6 +8599,10 @@ impl RuntimeThreadManager {
|
||||
let mut thread = self.store.load_thread(thread_id)?;
|
||||
thread.session_id = Some(session_id.clone());
|
||||
thread.saved_session_checkpoint = Some(SavedSessionCheckpoint {
|
||||
publication_observation: session
|
||||
.write_receipt
|
||||
.as_ref()
|
||||
.and_then(|receipt| receipt.observation().ok()),
|
||||
covered_turn_id: thread.latest_turn_id.clone(),
|
||||
messages_sha256,
|
||||
messages_len: Some(session.messages.len()),
|
||||
@@ -8831,6 +8964,10 @@ impl RuntimeThreadManager {
|
||||
"native fork source document or local goal changed; source retained"
|
||||
);
|
||||
}
|
||||
let target_id = Uuid::new_v4().to_string();
|
||||
let source = manager
|
||||
.expect_absent_session(&target_id)?
|
||||
.capture(crate::session_manager::SessionWriteOwner::control())?;
|
||||
let mut session = match captured.session {
|
||||
Some(source) => {
|
||||
let parent = source.metadata.clone();
|
||||
@@ -8867,7 +9004,7 @@ impl RuntimeThreadManager {
|
||||
session.leaf_id = journal.leaf_id.clone();
|
||||
session.metadata.spawn_depth = journal.spawn_depth;
|
||||
session.messages = journal.to_messages();
|
||||
session.metadata.id = Uuid::new_v4().to_string();
|
||||
session.metadata.id = target_id;
|
||||
session.metadata.created_at = forked.created_at;
|
||||
session.metadata.updated_at = forked.updated_at;
|
||||
session.metadata.message_count = session.messages.len();
|
||||
@@ -8915,7 +9052,9 @@ impl RuntimeThreadManager {
|
||||
!manager.session_document_exists(&session_id),
|
||||
"native fork target already exists"
|
||||
);
|
||||
manager.save_session(&session)?;
|
||||
session.write_receipt = Some(source);
|
||||
let publication = manager.publish_session(&session)?;
|
||||
session.adopt_publication(&publication);
|
||||
let mut goal = captured.goal;
|
||||
if let Some(goal) = goal.as_mut()
|
||||
&& goal.status == crate::session_manager::SessionGoalStatus::Active
|
||||
@@ -8926,6 +9065,7 @@ impl RuntimeThreadManager {
|
||||
manager.save_session_goal(&session_id, goal.as_ref())?;
|
||||
forked.session_id = Some(session_id);
|
||||
forked.saved_session_checkpoint = Some(SavedSessionCheckpoint {
|
||||
publication_observation: Some(publication.observation().clone()),
|
||||
covered_turn_id,
|
||||
messages_sha256: session_messages_sha256(&session.messages)?,
|
||||
messages_len: Some(session.messages.len()),
|
||||
@@ -9565,6 +9705,8 @@ impl RuntimeThreadManager {
|
||||
),
|
||||
};
|
||||
forked.saved_session_checkpoint = Some(SavedSessionCheckpoint {
|
||||
publication_observation: None,
|
||||
|
||||
covered_turn_id: covered_turns
|
||||
.checked_sub(1)
|
||||
.map(|index| source_turns[index].id.clone()),
|
||||
@@ -10268,6 +10410,8 @@ impl RuntimeThreadManager {
|
||||
);
|
||||
thread.session_id = Some(operation.receipt.session_id.clone());
|
||||
thread.saved_session_checkpoint = Some(SavedSessionCheckpoint {
|
||||
publication_observation: None,
|
||||
|
||||
covered_turn_id: thread.latest_turn_id.clone(),
|
||||
messages_sha256: session_messages_sha256(checkpoint_messages)?,
|
||||
messages_len: Some(checkpoint_messages.len()),
|
||||
@@ -13289,6 +13433,7 @@ impl RuntimeThreadManager {
|
||||
if !session_messages.is_empty() || sys_prompt.is_some() {
|
||||
engine
|
||||
.send(Op::SyncSession {
|
||||
control: crate::core::ops::SessionControlId::new(),
|
||||
// Same identity the engine was built with: a
|
||||
// re-sync of the same conversation, never a
|
||||
// conversation boundary.
|
||||
@@ -13511,7 +13656,8 @@ impl RuntimeThreadManager {
|
||||
permit = forward.reserve() => match permit {
|
||||
Ok(permit) => { permit.send(event); break; },
|
||||
Err(_) => {
|
||||
if let EngineEvent::SessionUpdated { checkpoint: Some(checkpoint), .. } = &event {
|
||||
if let EngineEvent::SessionUpdated {
|
||||
source: _, checkpoint: Some(checkpoint), .. } = &event {
|
||||
checkpoint.fail("Runtime checkpoint event receiver closed");
|
||||
}
|
||||
break 'pump;
|
||||
@@ -13519,7 +13665,8 @@ impl RuntimeThreadManager {
|
||||
},
|
||||
_ = &mut notified => {},
|
||||
_ = stop.cancelled() => {
|
||||
if let EngineEvent::SessionUpdated { checkpoint: Some(checkpoint), .. } = &event {
|
||||
if let EngineEvent::SessionUpdated {
|
||||
source: _, checkpoint: Some(checkpoint), .. } = &event {
|
||||
checkpoint.fail("Runtime checkpoint event owner stopped");
|
||||
}
|
||||
break 'pump;
|
||||
@@ -13813,6 +13960,8 @@ impl RuntimeThreadManager {
|
||||
return;
|
||||
}
|
||||
stored.saved_session_checkpoint = Some(SavedSessionCheckpoint {
|
||||
publication_observation: None,
|
||||
|
||||
covered_turn_id: covered
|
||||
.checked_sub(1)
|
||||
.and_then(|index| turns.get(index))
|
||||
@@ -14519,6 +14668,7 @@ impl RuntimeThreadManager {
|
||||
created_at,
|
||||
route,
|
||||
submission_id: _,
|
||||
..
|
||||
} => {
|
||||
saw_turn_started = true;
|
||||
engine_turn_id = Some(started_turn_id);
|
||||
@@ -15174,7 +15324,8 @@ impl RuntimeThreadManager {
|
||||
)?;
|
||||
}
|
||||
}
|
||||
EngineEvent::CompactionStarted { id, auto, message } => {
|
||||
EngineEvent::CompactionStarted { id, auto, message, ..
|
||||
} => {
|
||||
let item_id = runtime_record_id("item");
|
||||
compaction_items.insert(id.clone(), item_id.clone());
|
||||
let item = TurnItemRecord {
|
||||
@@ -15660,7 +15811,8 @@ impl RuntimeThreadManager {
|
||||
Some(EngineEvent::UserInputClosed { id, outcome }) => {
|
||||
self.user_input_settlement(&thread_id).close(&id, Some(&turn_id), outcome).await?;
|
||||
}
|
||||
Some(EngineEvent::SessionUpdated { checkpoint: Some(checkpoint), .. }) => {
|
||||
Some(EngineEvent::SessionUpdated {
|
||||
source: None, checkpoint: Some(checkpoint), .. }) => {
|
||||
checkpoint.fail("Runtime checkpoint cannot wait behind an approval");
|
||||
}
|
||||
Some(EngineEvent::ApprovalWithdrawn { id: withdrawn }) if withdrawn == id => {
|
||||
|
||||
@@ -535,6 +535,8 @@ mod recovery {
|
||||
];
|
||||
for len in 0..=messages.len() {
|
||||
let checkpoint = SavedSessionCheckpoint {
|
||||
publication_observation: None,
|
||||
|
||||
covered_turn_id: None,
|
||||
messages_sha256: session_messages_sha256(&messages[..len])?,
|
||||
messages_len: None,
|
||||
@@ -548,6 +550,8 @@ mod recovery {
|
||||
assert_eq!(checkpoint_prefix_len(&exact, &messages)?, Some(len));
|
||||
}
|
||||
let diverged = SavedSessionCheckpoint {
|
||||
publication_observation: None,
|
||||
|
||||
covered_turn_id: None,
|
||||
messages_sha256: session_messages_sha256(&[msg(Role::User, "other")])?,
|
||||
messages_len: Some(1),
|
||||
@@ -4552,6 +4556,7 @@ async fn compact_lifecycle_outlives_caller_and_preserves_concurrent_thread_updat
|
||||
harness
|
||||
.tx_event
|
||||
.send(EngineEvent::CompactionStarted {
|
||||
previous_source: None,
|
||||
id: "manual_owned".to_string(),
|
||||
auto: false,
|
||||
message: "compaction started".to_string(),
|
||||
@@ -8666,6 +8671,7 @@ async fn thread_lifecycle_persists_across_restart() -> Result<()> {
|
||||
if matches!(rx_op.recv().await, Some(Op::SendMessage(TurnSpec { .. }))) {
|
||||
let _ = tx_event
|
||||
.send(EngineEvent::TurnStarted {
|
||||
previous_source: None,
|
||||
turn_id: "engine_turn_1".to_string(),
|
||||
created_at: chrono::Utc::now(),
|
||||
route: None,
|
||||
@@ -8837,6 +8843,7 @@ async fn initial_classifier_usage_is_persisted_before_terminal_and_merged_exactl
|
||||
harness
|
||||
.tx_event
|
||||
.send(EngineEvent::TurnStarted {
|
||||
previous_source: None,
|
||||
turn_id: "engine_classifier_receipt".to_string(),
|
||||
created_at: Utc::now(),
|
||||
route: None,
|
||||
@@ -9425,6 +9432,7 @@ async fn monitor_deduplicates_sink_and_metadata_and_persists_metadata_only_missi
|
||||
harness
|
||||
.tx_event
|
||||
.send(EngineEvent::TurnStarted {
|
||||
previous_source: None,
|
||||
turn_id: "engine_metadata_receipt".to_string(),
|
||||
created_at: Utc::now(),
|
||||
route: None,
|
||||
@@ -9512,6 +9520,7 @@ async fn monitor_separates_lifecycle_start_from_billing_dispatch_and_child_usage
|
||||
harness
|
||||
.tx_event
|
||||
.send(EngineEvent::TurnStarted {
|
||||
previous_source: None,
|
||||
turn_id: "engine_route_receipt".to_string(),
|
||||
created_at: started_at,
|
||||
route: None,
|
||||
@@ -9690,6 +9699,7 @@ async fn monitor_separates_lifecycle_start_from_billing_dispatch_and_child_usage
|
||||
harness
|
||||
.tx_event
|
||||
.send(EngineEvent::TurnStarted {
|
||||
previous_source: None,
|
||||
turn_id: second_engine_turn.to_string(),
|
||||
created_at: Utc::now(),
|
||||
route: None,
|
||||
@@ -9773,6 +9783,7 @@ async fn monitor_persists_request_snapshots_and_matching_terminal_diagnostics()
|
||||
harness
|
||||
.tx_event
|
||||
.send(EngineEvent::TurnStarted {
|
||||
previous_source: None,
|
||||
turn_id: engine_turn_id.to_string(),
|
||||
created_at: Utc::now(),
|
||||
route: None,
|
||||
@@ -10103,6 +10114,7 @@ async fn monitor_persists_request_snapshots_and_matching_terminal_diagnostics()
|
||||
harness
|
||||
.tx_event
|
||||
.send(EngineEvent::TurnStarted {
|
||||
previous_source: None,
|
||||
turn_id: second_engine_turn_id.to_string(),
|
||||
created_at: Utc::now(),
|
||||
route: None,
|
||||
@@ -10198,6 +10210,7 @@ async fn completed_turn_without_engine_output_fails() -> Result<()> {
|
||||
if matches!(rx_op.recv().await, Some(Op::SendMessage(TurnSpec { .. }))) {
|
||||
let _ = tx_event
|
||||
.send(EngineEvent::TurnStarted {
|
||||
previous_source: None,
|
||||
turn_id: "engine_empty_turn".to_string(),
|
||||
created_at: chrono::Utc::now(),
|
||||
route: None,
|
||||
@@ -10326,6 +10339,7 @@ async fn worker_lifecycle_receipts_preserve_owner_outcome_and_durable_replay() -
|
||||
}
|
||||
let _ = tx_event
|
||||
.send(EngineEvent::TurnStarted {
|
||||
previous_source: None,
|
||||
turn_id: "engine_worker_lifecycle".into(),
|
||||
created_at: Utc::now(),
|
||||
route: None,
|
||||
@@ -10383,6 +10397,7 @@ async fn worker_lifecycle_receipts_preserve_owner_outcome_and_durable_replay() -
|
||||
}
|
||||
let _ = tx_event
|
||||
.send(EngineEvent::TurnStarted {
|
||||
previous_source: None,
|
||||
turn_id: "engine_next_parent".into(),
|
||||
created_at: Utc::now(),
|
||||
route: None,
|
||||
@@ -10674,6 +10689,7 @@ async fn preturn_control_status_does_not_make_empty_turn_succeed() -> Result<()>
|
||||
.await;
|
||||
let _ = tx_event
|
||||
.send(EngineEvent::TurnStarted {
|
||||
previous_source: None,
|
||||
turn_id: "engine_empty_after_control_status".to_string(),
|
||||
created_at: chrono::Utc::now(),
|
||||
route: None,
|
||||
@@ -10735,6 +10751,7 @@ async fn engine_error_remains_failed_after_nominal_turn_complete() -> Result<()>
|
||||
if matches!(rx_op.recv().await, Some(Op::SendMessage(TurnSpec { .. }))) {
|
||||
let _ = tx_event
|
||||
.send(EngineEvent::TurnStarted {
|
||||
previous_source: None,
|
||||
turn_id: "engine_error_then_complete".to_string(),
|
||||
created_at: chrono::Utc::now(),
|
||||
route: None,
|
||||
@@ -11318,6 +11335,7 @@ async fn compact_interrupt_persists_canceled_item_for_the_exact_request() -> Res
|
||||
harness
|
||||
.tx_event
|
||||
.send(EngineEvent::CompactionStarted {
|
||||
previous_source: None,
|
||||
id: compaction_id.clone(),
|
||||
auto: false,
|
||||
message: "compaction started".to_string(),
|
||||
@@ -11563,6 +11581,7 @@ async fn multi_turn_continuity_same_thread() -> Result<()> {
|
||||
turn_index = turn_index.saturating_add(1);
|
||||
let _ = tx_event
|
||||
.send(EngineEvent::TurnStarted {
|
||||
previous_source: None,
|
||||
turn_id: format!("engine_turn_{turn_index}"),
|
||||
created_at: chrono::Utc::now(),
|
||||
route: None,
|
||||
@@ -11810,6 +11829,7 @@ async fn host_goal_loop_kickoff_arms_one_continuation_and_parks_at_engine_cap()
|
||||
let _ = tx_event.send(EngineEvent::GoalUpdated { snapshot }).await;
|
||||
let _ = tx_event
|
||||
.send(EngineEvent::TurnStarted {
|
||||
previous_source: None,
|
||||
turn_id: format!("engine_goal_{pass}"),
|
||||
created_at: chrono::Utc::now(),
|
||||
route: None,
|
||||
@@ -11932,6 +11952,7 @@ async fn host_goal_loop_skips_rearm_without_update_goal_and_after_failed_pass()
|
||||
let _ = tx_event.send(EngineEvent::GoalUpdated { snapshot }).await;
|
||||
let _ = tx_event
|
||||
.send(EngineEvent::TurnStarted {
|
||||
previous_source: None,
|
||||
turn_id: "engine_goal_no_update".to_string(),
|
||||
created_at: chrono::Utc::now(),
|
||||
route: None,
|
||||
@@ -12026,6 +12047,7 @@ async fn host_goal_loop_skips_rearm_without_update_goal_and_after_failed_pass()
|
||||
.await;
|
||||
let _ = failed_tx_event
|
||||
.send(EngineEvent::TurnStarted {
|
||||
previous_source: None,
|
||||
turn_id: "engine_goal_failed".to_string(),
|
||||
created_at: chrono::Utc::now(),
|
||||
route: None,
|
||||
@@ -12127,6 +12149,7 @@ async fn host_goal_loop_mirrors_terminal_snapshot_and_does_not_rearm() -> Result
|
||||
let _ = tx_event.send(EngineEvent::GoalUpdated { snapshot }).await;
|
||||
let _ = tx_event
|
||||
.send(EngineEvent::TurnStarted {
|
||||
previous_source: None,
|
||||
turn_id: format!("engine_{status}"),
|
||||
created_at: chrono::Utc::now(),
|
||||
route: None,
|
||||
@@ -12238,6 +12261,7 @@ async fn model_created_goal_persists_through_adopted_revision() -> Result<()> {
|
||||
let _ = tx_event.send(EngineEvent::GoalUpdated { snapshot }).await;
|
||||
let _ = tx_event
|
||||
.send(EngineEvent::TurnStarted {
|
||||
previous_source: None,
|
||||
turn_id: "engine_model_created".to_string(),
|
||||
created_at: chrono::Utc::now(),
|
||||
route: None,
|
||||
@@ -12325,6 +12349,7 @@ async fn model_created_goal_never_overwrites_concurrent_explicit_goal() -> Resul
|
||||
let _ = tx_event.send(EngineEvent::GoalUpdated { snapshot }).await;
|
||||
let _ = tx_event
|
||||
.send(EngineEvent::TurnStarted {
|
||||
previous_source: None,
|
||||
turn_id: "engine_concurrent".to_string(),
|
||||
created_at: chrono::Utc::now(),
|
||||
route: None,
|
||||
@@ -13021,6 +13046,7 @@ async fn interrupt_turn_marks_interrupted_after_cleanup() -> Result<()> {
|
||||
if matches!(rx_op.recv().await, Some(Op::SendMessage(TurnSpec { .. }))) {
|
||||
let _ = tx_event
|
||||
.send(EngineEvent::TurnStarted {
|
||||
previous_source: None,
|
||||
turn_id: "engine_turn_interrupt".to_string(),
|
||||
created_at: chrono::Utc::now(),
|
||||
route: None,
|
||||
@@ -14460,6 +14486,7 @@ async fn thread_detail_cursor_precedes_projection_reads_at_terminal_boundary() -
|
||||
harness
|
||||
.tx_event
|
||||
.send(EngineEvent::TurnStarted {
|
||||
previous_source: None,
|
||||
turn_id: "snapshot_terminal".to_string(),
|
||||
created_at: Utc::now(),
|
||||
route: None,
|
||||
@@ -14645,6 +14672,7 @@ async fn thread_detail_materializes_stream_prefixes_before_their_delta_cursor()
|
||||
harness
|
||||
.tx_event
|
||||
.send(EngineEvent::TurnStarted {
|
||||
previous_source: None,
|
||||
turn_id: "delta_snapshot".to_string(),
|
||||
created_at: Utc::now(),
|
||||
route: None,
|
||||
@@ -14783,6 +14811,7 @@ async fn superseded_reasoning_segment_closes_when_the_retried_stream_starts() ->
|
||||
harness
|
||||
.tx_event
|
||||
.send(EngineEvent::TurnStarted {
|
||||
previous_source: None,
|
||||
turn_id: "retried_stream".to_string(),
|
||||
created_at: Utc::now(),
|
||||
route: None,
|
||||
@@ -14932,6 +14961,7 @@ async fn turn_end_closes_a_tool_call_that_never_completed() -> Result<()> {
|
||||
harness
|
||||
.tx_event
|
||||
.send(EngineEvent::TurnStarted {
|
||||
previous_source: None,
|
||||
turn_id: "unanswered_tool_call".to_string(),
|
||||
created_at: Utc::now(),
|
||||
route: None,
|
||||
@@ -15093,6 +15123,7 @@ async fn thread_detail_delta_boundary_is_replay_idempotent() -> Result<()> {
|
||||
harness
|
||||
.tx_event
|
||||
.send(EngineEvent::TurnStarted {
|
||||
previous_source: None,
|
||||
turn_id: "delta_boundary".to_string(),
|
||||
created_at: Utc::now(),
|
||||
route: None,
|
||||
@@ -15452,6 +15483,7 @@ async fn dynamic_tool_result_settles_snapshot_and_emits_one_safe_resolution() ->
|
||||
harness
|
||||
.tx_event
|
||||
.send(EngineEvent::TurnStarted {
|
||||
previous_source: None,
|
||||
turn_id: "dynamic_result".to_string(),
|
||||
created_at: Utc::now(),
|
||||
route: None,
|
||||
@@ -15617,6 +15649,7 @@ async fn dynamic_tool_result_receipt_outlives_canceled_delivery_future() -> Resu
|
||||
harness
|
||||
.tx_event
|
||||
.send(EngineEvent::TurnStarted {
|
||||
previous_source: None,
|
||||
turn_id: "dynamic_detached_settlement".to_string(),
|
||||
created_at: Utc::now(),
|
||||
route: None,
|
||||
@@ -16563,6 +16596,7 @@ async fn dynamic_tool_timeout_clears_snapshot_and_emits_once() -> Result<()> {
|
||||
harness
|
||||
.tx_event
|
||||
.send(EngineEvent::TurnStarted {
|
||||
previous_source: None,
|
||||
turn_id: "dynamic_timeout".to_string(),
|
||||
created_at: Utc::now(),
|
||||
route: None,
|
||||
@@ -16654,6 +16688,7 @@ async fn terminal_turn_cancels_pending_dynamic_tool_exactly_once() -> Result<()>
|
||||
harness
|
||||
.tx_event
|
||||
.send(EngineEvent::TurnStarted {
|
||||
previous_source: None,
|
||||
turn_id: "dynamic_cancel".to_string(),
|
||||
created_at: Utc::now(),
|
||||
route: None,
|
||||
@@ -16765,6 +16800,7 @@ async fn engine_withdrawal_retires_approval_and_unblocks_queued_events() -> Resu
|
||||
harness
|
||||
.tx_event
|
||||
.send(EngineEvent::TurnStarted {
|
||||
previous_source: None,
|
||||
turn_id: "engine_withdrawal".to_string(),
|
||||
created_at: Utc::now(),
|
||||
route: None,
|
||||
@@ -16920,6 +16956,7 @@ async fn approval_wait_heartbeat_is_never_sequenced_after_the_decision() -> Resu
|
||||
harness
|
||||
.tx_event
|
||||
.send(EngineEvent::TurnStarted {
|
||||
previous_source: None,
|
||||
turn_id: "engine_turn_wait".to_string(),
|
||||
created_at: Utc::now(),
|
||||
route: None,
|
||||
@@ -18482,6 +18519,7 @@ async fn steer_turn_on_active_turn_records_item_and_event() -> Result<()> {
|
||||
if matches!(rx_op.recv().await, Some(Op::SendMessage(TurnSpec { .. }))) {
|
||||
let _ = tx_event
|
||||
.send(EngineEvent::TurnStarted {
|
||||
previous_source: None,
|
||||
turn_id: "engine_turn_steer".to_string(),
|
||||
created_at: chrono::Utc::now(),
|
||||
route: None,
|
||||
@@ -19151,6 +19189,8 @@ fn answer_session_snapshot(
|
||||
.expect("snapshot message");
|
||||
if let Some(tx) = tx.lock().expect("snapshot sender").take() {
|
||||
let _ = tx.send(crate::core::ops::SessionSnapshot {
|
||||
source: None,
|
||||
|
||||
session_id: String::new(),
|
||||
messages,
|
||||
total_tokens: 0,
|
||||
@@ -19208,6 +19248,7 @@ async fn compaction_lifecycle_emits_item_events_with_compaction_counts() -> Resu
|
||||
op_count = op_count.saturating_add(1);
|
||||
let _ = tx_event
|
||||
.send(EngineEvent::TurnStarted {
|
||||
previous_source: None,
|
||||
turn_id: "engine_turn_auto".to_string(),
|
||||
created_at: chrono::Utc::now(),
|
||||
route: None,
|
||||
@@ -19224,6 +19265,11 @@ async fn compaction_lifecycle_emits_item_events_with_compaction_counts() -> Resu
|
||||
{"role":"user","content":[{"type":"text","text":"auto checkpoint prefix"}]}
|
||||
])).unwrap();
|
||||
let snapshot = Arc::new(crate::core::ops::SessionSnapshot {
|
||||
source: Some(crate::core::ops::SessionSnapshotSource::turn(
|
||||
"engine_turn_auto".into(),
|
||||
None,
|
||||
)),
|
||||
|
||||
session_id: checkpoint_thread.clone(),
|
||||
messages: messages.clone(),
|
||||
total_tokens: 0,
|
||||
@@ -19249,6 +19295,7 @@ async fn compaction_lifecycle_emits_item_events_with_compaction_counts() -> Resu
|
||||
);
|
||||
tx_event
|
||||
.send(EngineEvent::SessionUpdated {
|
||||
source: request.snapshot.source.clone(),
|
||||
checkpoint: Some(request),
|
||||
session_id: checkpoint_thread.clone(),
|
||||
messages: Arc::new(messages),
|
||||
@@ -19271,6 +19318,7 @@ async fn compaction_lifecycle_emits_item_events_with_compaction_counts() -> Resu
|
||||
}
|
||||
let _ = tx_event
|
||||
.send(EngineEvent::CompactionStarted {
|
||||
previous_source: None,
|
||||
id: "auto_compact_1".to_string(),
|
||||
auto: true,
|
||||
message: "auto compact begin".to_string(),
|
||||
@@ -19314,6 +19362,7 @@ async fn compaction_lifecycle_emits_item_events_with_compaction_counts() -> Resu
|
||||
op_count = op_count.saturating_add(1);
|
||||
let _ = tx_event
|
||||
.send(EngineEvent::CompactionStarted {
|
||||
previous_source: None,
|
||||
id: "manual_compact_1".to_string(),
|
||||
auto: false,
|
||||
message: "manual compact begin".to_string(),
|
||||
@@ -21509,6 +21558,7 @@ mod runtime_image_inputs {
|
||||
let run = tokio::spawn(engine.run());
|
||||
handle
|
||||
.send(Op::SyncSession {
|
||||
control: crate::core::ops::SessionControlId::new(),
|
||||
session_id: stored_thread.session_id.clone(),
|
||||
messages: reopened.restore_thread_messages(&stored_thread)?,
|
||||
system_prompt: None,
|
||||
@@ -21768,6 +21818,7 @@ mod runtime_image_inputs {
|
||||
// and observe its durable settlement before restoring that case.
|
||||
for event in [
|
||||
EngineEvent::TurnStarted {
|
||||
previous_source: None,
|
||||
turn_id: turn.id.clone(),
|
||||
created_at: Utc::now(),
|
||||
route: None,
|
||||
@@ -21956,6 +22007,7 @@ async fn notices_raise_from_engine_events_and_clear_on_settle_or_ack() -> Result
|
||||
harness
|
||||
.tx_event
|
||||
.send(EngineEvent::TurnStarted {
|
||||
previous_source: None,
|
||||
turn_id: turn.id.clone(),
|
||||
created_at: Utc::now(),
|
||||
route: None,
|
||||
@@ -23019,6 +23071,7 @@ async fn engine_plumbing_items_are_tagged_internal_and_retry_hints_are_dropped()
|
||||
harness
|
||||
.tx_event
|
||||
.send(EngineEvent::TurnStarted {
|
||||
previous_source: None,
|
||||
turn_id: "engine_plumbing_visibility".to_string(),
|
||||
created_at: Utc::now(),
|
||||
route: None,
|
||||
@@ -23790,6 +23843,7 @@ async fn runtime_receipts_mask_configured_secrets() -> Result<()> {
|
||||
harness
|
||||
.tx_event
|
||||
.send(EngineEvent::TurnStarted {
|
||||
previous_source: None,
|
||||
turn_id: turn.id.clone(),
|
||||
created_at: Utc::now(),
|
||||
route: None,
|
||||
@@ -24049,6 +24103,7 @@ async fn tool_completion_items_carry_typed_artifact_refs() -> Result<()> {
|
||||
harness
|
||||
.tx_event
|
||||
.send(EngineEvent::TurnStarted {
|
||||
previous_source: None,
|
||||
turn_id: turn.id.clone(),
|
||||
created_at: Utc::now(),
|
||||
route: None,
|
||||
@@ -24292,6 +24347,7 @@ async fn run_workspace_turn(
|
||||
harness
|
||||
.tx_event
|
||||
.send(EngineEvent::TurnStarted {
|
||||
previous_source: None,
|
||||
turn_id: "engine-turn".to_string(),
|
||||
created_at: Utc::now(),
|
||||
route: None,
|
||||
@@ -24903,6 +24959,7 @@ mod execution_identity {
|
||||
harness
|
||||
.tx_event
|
||||
.send(EngineEvent::TurnStarted {
|
||||
previous_source: None,
|
||||
turn_id: turn.id.clone(),
|
||||
created_at: Utc::now(),
|
||||
route: None,
|
||||
@@ -25143,6 +25200,7 @@ mod execution_identity {
|
||||
harness
|
||||
.tx_event
|
||||
.send(EngineEvent::TurnStarted {
|
||||
previous_source: None,
|
||||
turn_id: turn.id.clone(),
|
||||
created_at: Utc::now(),
|
||||
route: None,
|
||||
|
||||
File diff suppressed because it is too large.
Load diff
@@ -314,6 +314,8 @@
|
||||
assert_eq!(restored.metadata.cost.cny_unpriced_turns, 1);
|
||||
assert_eq!(restored.metadata.cost.usage_source_fingerprints.len(), 1);
|
||||
if malformed == "tombstone" {
|
||||
assert!(restored.write_receipt.is_none());
|
||||
assert!(restored.capture_publication(SessionWriteOwner::control()).is_err());
|
||||
assert!(
|
||||
manager.save_session(&restored).is_err(),
|
||||
"an invalid deletion marker must fail closed for writes"
|
||||
|
||||
@@ -7,10 +7,15 @@
|
||||
updated_at: DateTime<Utc>,
|
||||
) {
|
||||
let session = SavedSession {
|
||||
schema_version: CURRENT_SESSION_SCHEMA_VERSION,
|
||||
write_receipt: None,
|
||||
|
||||
schema_version: CURRENT_SESSION_SCHEMA_VERSION,
|
||||
messages: vec![make_test_message("user", "hi")],
|
||||
metadata: SessionMetadata {
|
||||
id: id.to_string(),
|
||||
publication_id: None,
|
||||
native_mutation_observation: None,
|
||||
|
||||
id: id.to_string(),
|
||||
title: format!("session-{id}"),
|
||||
title_is_custom: false,
|
||||
created_at: updated_at,
|
||||
@@ -52,10 +57,15 @@
|
||||
updated_at: DateTime<Utc>,
|
||||
) {
|
||||
let session = SavedSession {
|
||||
schema_version: CURRENT_SESSION_SCHEMA_VERSION,
|
||||
write_receipt: None,
|
||||
|
||||
schema_version: CURRENT_SESSION_SCHEMA_VERSION,
|
||||
messages: Vec::new(),
|
||||
metadata: SessionMetadata {
|
||||
id: id.to_string(),
|
||||
publication_id: None,
|
||||
native_mutation_observation: None,
|
||||
|
||||
id: id.to_string(),
|
||||
title: DEFAULT_SESSION_TITLE.to_string(),
|
||||
title_is_custom: false,
|
||||
created_at: updated_at,
|
||||
|
||||
@@ -346,6 +346,9 @@ mod tests {
|
||||
fn metadata(id: &str, title: &str, workspace: &str, minutes_ago: i64) -> SessionMetadata {
|
||||
let ts = Utc::now() - Duration::minutes(minutes_ago);
|
||||
SessionMetadata {
|
||||
publication_id: None,
|
||||
native_mutation_observation: None,
|
||||
|
||||
id: id.to_string(),
|
||||
title: title.to_string(),
|
||||
title_is_custom: false,
|
||||
|
||||
@@ -995,8 +995,12 @@ impl Run<'_> {
|
||||
// A run that stopped between saving the document and binding the
|
||||
// thread left the document behind at this same id; reuse it.
|
||||
if exists(&id) || self.sessions.session_document_exists(&id) {
|
||||
return Ok(self.sessions.load_session(&id)?);
|
||||
return Ok(self.sessions.resume_session(&id)?.session);
|
||||
}
|
||||
let source = self
|
||||
.sessions
|
||||
.expect_absent_session(&id)?
|
||||
.capture(crate::session_manager::SessionWriteOwner::control())?;
|
||||
let mut session = crate::session_manager::create_saved_session_with_id_and_mode(
|
||||
id,
|
||||
&recovered.messages,
|
||||
@@ -1022,7 +1026,9 @@ impl Run<'_> {
|
||||
.unwrap_or_else(|| session.metadata.title.clone());
|
||||
session.metadata.title = recovered_title(&label);
|
||||
session.metadata.runtime_store = Some(held.binding().clone());
|
||||
self.sessions.save_session(&session)?;
|
||||
session.write_receipt = Some(source);
|
||||
let publication = self.sessions.publish_session(&session)?;
|
||||
session.adopt_publication(&publication);
|
||||
Ok(session)
|
||||
}
|
||||
|
||||
|
||||
@@ -160,6 +160,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
||||
|
||||
### Fixed
|
||||
|
||||
- Session publication compares the document and crash checkpoint captured before work was queued. A delayed save refuses changed records without replacing newer history or clearing its checkpoint. Ordered completion retains real checkpoint acknowledgments, and native turn/control identity follows restores, undo and terminal saves. Legacy checkpoints require explicit recovery; canonical history recovery retains its original source and exact prepared generation.
|
||||
- Damaged usage sidecars keep valid recovery data readable with incomplete coverage and no publication receipt. Known retirement, changed raw records and unsafe writes still refuse.
|
||||
- Local control and ACP output serialize within the existing 8 MiB frame limit and stop a damaged or stalled output stream. Compatibility replies, events and relay stdout now share the 30-second write deadline; owner forwarding keeps its tighter 5 seconds. Standalone and attached ACP recover after complete invalid UTF-8 or oversized input frames, discard oversized tails through the real newline, and preserve cancellation, permissions and actual Core receipts.
|
||||
- MCP composition records actual schema or description changes before the next request, so a conservative batch withdrawal re-pins the declared tool surface instead of failing the C5 prefix guard. Equal schemas retain their existing order. The `execute_tools` description also explains that visual input needs a direct capture call because nested results do not deliver inline images to the model.
|
||||
- Inference requests stop after uncertain delivery or an interrupted stream instead of silently sending another request. Automatic retries require an ordinary rate limit or a transient connection failure before dispatch. Redirects and HTTP protocol retries cannot bypass this policy; retained output and missing usage coverage remain visible. Terminal `exec` failures return exit 1; error categories no longer imply that repeating a saved command is safe.
|
||||
|
||||
@@ -360,10 +360,20 @@ impl CommandSessionLifecycleContext for SessionLifecycleAdapter<'_> {
|
||||
Err(e) => return Err(format!("could not open sessions directory: {e}")),
|
||||
};
|
||||
crate::tui::persistence_actor::flush_before_transition()?;
|
||||
let source = app
|
||||
.session_write_receipt
|
||||
.as_ref()
|
||||
.ok_or_else(|| {
|
||||
"Session source was not admitted; reopen its managed record before branching"
|
||||
.to_string()
|
||||
})?
|
||||
.capture(codewhale_engine::session_manager::SessionWriteOwner::control())
|
||||
.map_err(|error| error.to_string())?;
|
||||
let mut session = match manager.load_session(&session_id) {
|
||||
Ok(s) => s,
|
||||
Err(e) => return Err(format!("could not load session {session_id}: {e}")),
|
||||
};
|
||||
session.write_receipt = Some(source);
|
||||
session.ensure_journal();
|
||||
let journal_len_before = session
|
||||
.journal
|
||||
@@ -379,11 +389,10 @@ impl CommandSessionLifecycleContext for SessionLifecycleAdapter<'_> {
|
||||
}
|
||||
match session.journal_branch_to(entry_id) {
|
||||
Ok(()) => {
|
||||
if let Err(e) = manager.save_session(&session) {
|
||||
return Err(format!(
|
||||
"branch could not be persisted; active conversation unchanged: {e}"
|
||||
));
|
||||
}
|
||||
let publication = manager.publish_session(&session).map_err(|error| {
|
||||
format!("branch could not be persisted; active conversation unchanged: {error}")
|
||||
})?;
|
||||
session.adopt_publication(&publication);
|
||||
app.restore_api_messages(session.messages.clone(), &session);
|
||||
let leaf_display = session
|
||||
.leaf_id
|
||||
@@ -500,6 +509,9 @@ impl CommandSessionLifecycleContext for SessionLifecycleAdapter<'_> {
|
||||
.map_err(|error| format!("Failed to snapshot Work state: {error}"))?;
|
||||
let manager = codewhale_engine::session_manager::SessionManager::default_location()
|
||||
.map_err(|error| format!("could not open sessions directory: {error}"))?;
|
||||
crate::tui::persistence_actor::flush_before_transition()?;
|
||||
let control_owner = codewhale_engine::session_manager::SessionWriteOwner::control();
|
||||
let control_source = app.session_write_receipt.clone();
|
||||
let mut session = crate::tui::ui::build_session_snapshot(&mut app, &manager)?;
|
||||
let work_owner = app.current_session_id.clone();
|
||||
// Snapshots are journal-only (#6214 T3); exports need the legacy
|
||||
@@ -544,11 +556,20 @@ impl CommandSessionLifecycleContext for SessionLifecycleAdapter<'_> {
|
||||
let managed =
|
||||
in_place || manager.owns_session_path(&session.metadata.id, &save_path);
|
||||
let result = if managed {
|
||||
manager.save_session_bounded(session.clone()).and_then(|_| {
|
||||
session.metadata =
|
||||
manager.load_session_metadata_by_id(&session.metadata.id)?;
|
||||
Ok(())
|
||||
})
|
||||
if let Some(source) = control_source {
|
||||
session.write_receipt = Some(
|
||||
source
|
||||
.capture(control_owner)
|
||||
.map_err(|error| error.to_string())?,
|
||||
);
|
||||
} else {
|
||||
session
|
||||
.capture_publication(control_owner)
|
||||
.map_err(|error| error.to_string())?;
|
||||
}
|
||||
manager.publish_session_bounded(session.clone()).map(|publication| {
|
||||
session.adopt_publication(&publication);
|
||||
})
|
||||
} else {
|
||||
let json = match serde_json::to_string_pretty(&session) {
|
||||
Ok(json) => json,
|
||||
@@ -563,6 +584,7 @@ impl CommandSessionLifecycleContext for SessionLifecycleAdapter<'_> {
|
||||
queue_transition,
|
||||
);
|
||||
app.set_current_session_id(Some(session.metadata.id.clone()));
|
||||
app.session_write_receipt = session.write_receipt.clone();
|
||||
app.current_session_metadata = Some(session.metadata.clone());
|
||||
app.session_title = Some(session.metadata.title.clone());
|
||||
if let Err(err) = app.publish_saved_work_state(
|
||||
@@ -601,15 +623,38 @@ impl CommandSessionLifecycleContext for SessionLifecycleAdapter<'_> {
|
||||
}
|
||||
};
|
||||
|
||||
crate::tui::persistence_actor::flush_before_transition()?;
|
||||
let control_owner = codewhale_engine::session_manager::SessionWriteOwner::control();
|
||||
let control_source = app
|
||||
.session_write_receipt
|
||||
.as_ref()
|
||||
.map(|source| source.capture(control_owner.clone()))
|
||||
.transpose()
|
||||
.map_err(|error| error.to_string())?;
|
||||
let mut parent = crate::tui::ui::build_session_snapshot(&mut app, &manager)?;
|
||||
let work_owner = app.current_session_id.clone();
|
||||
parent.make_storage_compatible();
|
||||
if let Err(err) = manager.save_session_bounded(parent.clone()) {
|
||||
return Err(format!("Failed to save parent session: {err}"));
|
||||
if let Some(source) = control_source {
|
||||
parent.write_receipt = Some(source);
|
||||
} else if parent.write_receipt.is_some() {
|
||||
parent
|
||||
.capture_publication(control_owner)
|
||||
.map_err(|error| error.to_string())?;
|
||||
}
|
||||
|
||||
let work_owner = app.current_session_id.clone();
|
||||
parent.make_storage_compatible();
|
||||
let publication = manager.publish_session_bounded(parent.clone())
|
||||
.map_err(|error| format!("Failed to save parent session: {error}"))?;
|
||||
parent.adopt_publication(&publication);
|
||||
app.session_write_receipt = parent.write_receipt.clone();
|
||||
|
||||
let fork_id = uuid::Uuid::new_v4().to_string();
|
||||
let fork_source = manager
|
||||
.expect_absent_session(&fork_id)
|
||||
.map_err(|error| error.to_string())?
|
||||
.capture(codewhale_engine::session_manager::SessionWriteOwner::control())
|
||||
.map_err(|error| error.to_string())?;
|
||||
let mut forked = parent.clone();
|
||||
forked.metadata.id = uuid::Uuid::new_v4().to_string();
|
||||
forked.metadata.id = fork_id;
|
||||
forked.metadata.created_at = chrono::Utc::now();
|
||||
forked.metadata.updated_at = forked.metadata.created_at;
|
||||
forked.metadata.archived = false;
|
||||
@@ -625,9 +670,10 @@ impl CommandSessionLifecycleContext for SessionLifecycleAdapter<'_> {
|
||||
let queue_transition =
|
||||
crate::tui::ui::prepare_offline_queue_transition(&app, &forked.metadata.id)?;
|
||||
|
||||
if let Err(err) = manager.save_session(&forked) {
|
||||
return Err(format!("Failed to save forked session: {err}"));
|
||||
}
|
||||
forked.write_receipt = Some(fork_source);
|
||||
let publication = manager.publish_session(&forked)
|
||||
.map_err(|error| format!("Failed to save forked session: {error}"))?;
|
||||
forked.adopt_publication(&publication);
|
||||
if let Err(err) =
|
||||
app.publish_saved_work_state(work_owner.as_deref(), parent.work_state.as_ref())
|
||||
{
|
||||
@@ -696,6 +742,12 @@ impl CommandSessionLifecycleContext for SessionLifecycleAdapter<'_> {
|
||||
source_session.metadata.spawn_depth,
|
||||
)
|
||||
});
|
||||
let fork_id = uuid::Uuid::new_v4().to_string();
|
||||
let fork_source = manager
|
||||
.expect_absent_session(&fork_id)
|
||||
.map_err(|error| error.to_string())?
|
||||
.capture(codewhale_engine::session_manager::SessionWriteOwner::control())
|
||||
.map_err(|error| error.to_string())?;
|
||||
let forked_journal = journal.fork_from(None).unwrap_or_else(|_| {
|
||||
codewhale_runtime::session_tree::SessionJournal::with_spawn_depth(
|
||||
source_session.metadata.spawn_depth.saturating_add(1),
|
||||
@@ -703,7 +755,7 @@ impl CommandSessionLifecycleContext for SessionLifecycleAdapter<'_> {
|
||||
});
|
||||
let messages = forked_journal.to_messages();
|
||||
let mut forked = codewhale_engine::session_manager::create_saved_session_with_id_and_mode(
|
||||
uuid::Uuid::new_v4().to_string(),
|
||||
fork_id,
|
||||
&messages,
|
||||
&source_session.metadata.model,
|
||||
&app.workspace,
|
||||
@@ -730,9 +782,10 @@ impl CommandSessionLifecycleContext for SessionLifecycleAdapter<'_> {
|
||||
forked.artifacts = source_session.artifacts.clone();
|
||||
forked.work_state = source_session.work_state.clone();
|
||||
forked.last_auto_route = source_session.last_auto_route.clone();
|
||||
if let Err(err) = manager.save_session(&forked) {
|
||||
return Err(format!("Failed to save forked session: {err}"));
|
||||
}
|
||||
forked.write_receipt = Some(fork_source);
|
||||
let publication = manager.publish_session(&forked)
|
||||
.map_err(|error| format!("Failed to save forked session: {error}"))?;
|
||||
forked.adopt_publication(&publication);
|
||||
let parent_label =
|
||||
codewhale_engine::session_manager::truncate_id(&source_session.metadata.id).to_string();
|
||||
let fork_label =
|
||||
@@ -787,6 +840,7 @@ impl CommandSessionLifecycleContext for SessionLifecycleAdapter<'_> {
|
||||
app.session_context_references.clear();
|
||||
app.tool_evidence.clear();
|
||||
app.set_current_session_id(Some(new_id.clone()));
|
||||
crate::tui::ui::expect_fresh_session(&mut app, &new_id)?;
|
||||
app.current_session_metadata = None;
|
||||
app.session_title =
|
||||
Some(codewhale_engine::session_manager::DEFAULT_SESSION_TITLE.to_string());
|
||||
@@ -874,12 +928,14 @@ impl CommandSessionLifecycleContext for SessionLifecycleAdapter<'_> {
|
||||
return Err(format!("could not open sessions directory: {err}"));
|
||||
}
|
||||
};
|
||||
crate::tui::persistence_actor::flush_before_transition()?;
|
||||
match manager.set_session_archived(
|
||||
session_id,
|
||||
archived,
|
||||
codewhale_engine::session_manager::SessionMutator::Owner,
|
||||
) {
|
||||
Ok(metadata) => {
|
||||
app.adopt_session_mutation(&metadata);
|
||||
if let Some(cached) = app.current_session_metadata.as_mut()
|
||||
&& cached.id == metadata.id
|
||||
{
|
||||
@@ -1239,21 +1295,24 @@ impl CommandSessionControlContext for SessionControlAdapter<'_> {
|
||||
Err(e) => return Err(format!("Could not open sessions directory: {e}")),
|
||||
};
|
||||
|
||||
// Mirrors the baseline `/rename` write path exactly: load (with
|
||||
// first-snapshot recovery), sync live state, snapshot Work state,
|
||||
// carry context/artifacts/route/cost/model/workspace/mode metadata,
|
||||
// persist, then publish. Publication failures keep their post-save
|
||||
// partial-success semantics.
|
||||
let mut session = match manager.load_session(&session_id) {
|
||||
Ok(s) => s,
|
||||
Err(err) if err.kind() == std::io::ErrorKind::NotFound => {
|
||||
match live_session_before_first_snapshot(&manager, &session_id, &app) {
|
||||
Some(s) => s,
|
||||
None => return Err(format!("Could not load session: {err}")),
|
||||
}
|
||||
}
|
||||
Err(e) => return Err(format!("Could not load session: {e}")),
|
||||
};
|
||||
crate::tui::persistence_actor::flush_before_transition()?;
|
||||
let control_owner = codewhale_engine::session_manager::SessionWriteOwner::control();
|
||||
let control_source = app
|
||||
.session_write_receipt
|
||||
.as_ref()
|
||||
.map(|source| source.capture(control_owner.clone()))
|
||||
.transpose()
|
||||
.map_err(|error| error.to_string())?;
|
||||
let mut session = crate::tui::ui::build_session_snapshot(&mut app, &manager)?;
|
||||
if let Some(source) = control_source {
|
||||
session.write_receipt = Some(source);
|
||||
} else if session.write_receipt.is_some() {
|
||||
session
|
||||
.capture_publication(control_owner)
|
||||
.map_err(|error| error.to_string())?;
|
||||
}
|
||||
|
||||
session.make_storage_compatible();
|
||||
session = codewhale_engine::session_manager::update_session(
|
||||
session,
|
||||
&app.api_messages,
|
||||
@@ -1281,8 +1340,10 @@ impl CommandSessionControlContext for SessionControlAdapter<'_> {
|
||||
session.metadata.title = new_title.to_string();
|
||||
session.metadata.title_is_custom = true;
|
||||
|
||||
match manager.save_session(&session) {
|
||||
Ok(_) => {
|
||||
match manager.publish_session(&session) {
|
||||
Ok(publication) => {
|
||||
session.adopt_publication(&publication);
|
||||
app.session_write_receipt = session.write_receipt.clone();
|
||||
app.current_session_metadata = Some(session.metadata.clone());
|
||||
app.session_title = Some(new_title.to_string());
|
||||
if let Err(err) =
|
||||
@@ -1421,16 +1482,24 @@ fn persist_session_display(
|
||||
return Err("No active session. Send a message first to start a session.".to_string());
|
||||
}
|
||||
};
|
||||
let mut session = match manager.load_session(&session_id) {
|
||||
Ok(session) => session,
|
||||
Err(error) if error.kind() == std::io::ErrorKind::NotFound => {
|
||||
match live_session_before_first_snapshot(&manager, &session_id, app) {
|
||||
Some(session) => session,
|
||||
None => return Err(format!("Could not load session: {error}")),
|
||||
}
|
||||
}
|
||||
Err(error) => return Err(format!("Could not load session: {error}")),
|
||||
};
|
||||
crate::tui::persistence_actor::flush_before_transition()?;
|
||||
let control_owner = codewhale_engine::session_manager::SessionWriteOwner::control();
|
||||
let control_source = app
|
||||
.session_write_receipt
|
||||
.as_ref()
|
||||
.map(|source| source.capture(control_owner.clone()))
|
||||
.transpose()
|
||||
.map_err(|error| error.to_string())?;
|
||||
let mut session = crate::tui::ui::build_session_snapshot(app, &manager)?;
|
||||
if let Some(source) = control_source {
|
||||
session.write_receipt = Some(source);
|
||||
} else if session.write_receipt.is_some() {
|
||||
session
|
||||
.capture_publication(control_owner)
|
||||
.map_err(|error| error.to_string())?;
|
||||
}
|
||||
|
||||
session.make_storage_compatible();
|
||||
session = codewhale_engine::session_manager::update_session(
|
||||
session,
|
||||
&app.api_messages,
|
||||
@@ -1457,8 +1526,11 @@ fn persist_session_display(
|
||||
app.sync_cost_to_metadata(&mut session.metadata);
|
||||
stage(&mut session);
|
||||
|
||||
match manager.save_session(&session) {
|
||||
Ok(_) => {
|
||||
match manager.publish_session_preserving_lifecycle(&session) {
|
||||
Ok(publication) => {
|
||||
session.adopt_publication(&publication);
|
||||
app.session_write_receipt = session.write_receipt.clone();
|
||||
app.current_session_metadata = Some(session.metadata.clone());
|
||||
commit(app);
|
||||
// The render loop syncs the resolved prefix into the terminal
|
||||
// title; force a frame so the change lands immediately.
|
||||
@@ -1524,26 +1596,6 @@ fn plan_snapshot_to_sections(
|
||||
/// Recover the session document for a live turn that has not completed (and
|
||||
/// therefore persisted) its first snapshot yet (#5430). Mirrors the legacy
|
||||
/// `/rename`/`/title` recovery exactly.
|
||||
fn live_session_before_first_snapshot(
|
||||
manager: &codewhale_engine::session_manager::SessionManager,
|
||||
session_id: &str,
|
||||
app: &App,
|
||||
) -> Option<codewhale_engine::session_manager::SavedSession> {
|
||||
if let Ok(Some(checkpoint)) = manager.load_session_checkpoint(session_id) {
|
||||
return Some(checkpoint);
|
||||
}
|
||||
Some(
|
||||
codewhale_engine::session_manager::create_saved_session_with_id_and_mode(
|
||||
session_id.to_string(),
|
||||
&app.api_messages,
|
||||
&app.model_selection_for_persistence(),
|
||||
&app.workspace,
|
||||
u64::from(app.session.total_tokens),
|
||||
app.system_prompt.as_ref(),
|
||||
Some(app.mode.as_setting()),
|
||||
),
|
||||
)
|
||||
}
|
||||
|
||||
/// `/resume <file>` import: read, parse a container or plain saved session,
|
||||
/// and apply it atomically. Errors are the exact baseline text.
|
||||
@@ -1604,9 +1656,17 @@ fn import_session_container(
|
||||
.set_model_provider_route(&provider, provider_id.as_deref());
|
||||
let new_id = imported.metadata.id.clone();
|
||||
let queue_transition = crate::tui::ui::prepare_offline_queue_transition(app, &new_id)?;
|
||||
if let Err(e) = manager.save_session(&imported) {
|
||||
return Err(format!("imported session could not be saved: {e}"));
|
||||
}
|
||||
imported.write_receipt = Some(
|
||||
manager.expect_absent_session(&new_id)
|
||||
.map_err(|error| error.to_string())?,
|
||||
);
|
||||
imported
|
||||
.capture_publication(codewhale_engine::session_manager::SessionWriteOwner::control())
|
||||
.map_err(|error| error.to_string())?;
|
||||
let publication = manager
|
||||
.publish_session(&imported)
|
||||
.map_err(|error| format!("imported session could not be saved: {error}"))?;
|
||||
imported.adopt_publication(&publication);
|
||||
crate::tui::ui::install_offline_queue_transition(app, queue_transition);
|
||||
app.set_current_session_id(Some(new_id.clone()));
|
||||
app.current_session_metadata = Some(imported.metadata.clone());
|
||||
@@ -7561,6 +7621,7 @@ mod tests {
|
||||
save_control_session(&tmpdir, "rename-1");
|
||||
let mut app = control_test_app(&tmpdir);
|
||||
app.current_session_id = Some("rename-1".to_string());
|
||||
crate::test_support::capture_session_fixture_source(&mut app);
|
||||
|
||||
let receipt = {
|
||||
let mut bundle = app.command_contexts();
|
||||
@@ -7607,6 +7668,7 @@ mod tests {
|
||||
|
||||
let mut app = control_test_app(&tmpdir);
|
||||
app.current_session_id = Some("midturn-1".to_string());
|
||||
crate::test_support::capture_session_fixture_source(&mut app);
|
||||
app.api_messages = std::sync::Arc::new(vec![user_message("first turn still streaming")]);
|
||||
|
||||
let receipt = {
|
||||
@@ -7646,6 +7708,7 @@ mod tests {
|
||||
save_control_session(&tmpdir, "title-1");
|
||||
let mut app = control_test_app(&tmpdir);
|
||||
app.current_session_id = Some("title-1".to_string());
|
||||
crate::test_support::capture_session_fixture_source(&mut app);
|
||||
|
||||
// No session window title and no config default -> unset, no source.
|
||||
let report = {
|
||||
|
||||
@@ -39,7 +39,10 @@ pub fn clear(app: &mut App) -> CommandResult {
|
||||
// than letting the engine generate one the App only learns about from
|
||||
// `SessionUpdated`. Two ids for one conversation orphan the checkpoint.
|
||||
crate::tui::ui::install_offline_queue_transition(app, queue_transition);
|
||||
app.current_session_id = Some(new_id.clone());
|
||||
app.set_current_session_id(Some(new_id.clone()));
|
||||
if let Err(error) = crate::tui::ui::expect_fresh_session(app, &new_id) {
|
||||
return CommandResult::error(error);
|
||||
}
|
||||
app.current_session_metadata = None;
|
||||
app.session_title = None;
|
||||
app.window_title = None;
|
||||
|
||||
@@ -101,8 +101,12 @@ fn persisted_workspace_with_one_user_message(world: &mut SessionCommandWorld) {
|
||||
let original_id = "original-session".to_string();
|
||||
let app = world.app.as_deref_mut().expect("app should exist");
|
||||
app.current_session_id = Some(original_id.clone());
|
||||
crate::test_support::capture_session_fixture_source(app);
|
||||
world.original_session_id = Some(original_id);
|
||||
persist_active_session(world);
|
||||
crate::test_support::capture_session_fixture_source(
|
||||
world.app.as_deref_mut().expect("fixture app"),
|
||||
);
|
||||
}
|
||||
|
||||
#[given("a CodeWhale session workspace with stale and fresh saved sessions")]
|
||||
|
||||
@@ -61,6 +61,7 @@ impl ControlHarness {
|
||||
let session_id = session.metadata.id.clone();
|
||||
self.manager.save_session(&session).expect("save session");
|
||||
self.app.current_session_id = Some(session_id.clone());
|
||||
crate::test_support::capture_session_fixture_source(&mut self.app);
|
||||
session_id
|
||||
}
|
||||
}
|
||||
@@ -126,6 +127,7 @@ fn rename_persists_all_live_metadata_through_public_dispatch() {
|
||||
.save_session(&session)
|
||||
.expect("save session");
|
||||
harness.app.current_session_id = Some(session_id.clone());
|
||||
crate::test_support::capture_session_fixture_source(&mut harness.app);
|
||||
harness
|
||||
.app
|
||||
.set_model_selection("local-code-model".to_string());
|
||||
@@ -216,6 +218,7 @@ fn rename_recovers_first_snapshot_from_checkpoint() {
|
||||
.save_checkpoint(&checkpoint)
|
||||
.expect("save checkpoint");
|
||||
harness.app.current_session_id = Some(session_id.clone());
|
||||
crate::test_support::capture_session_fixture_source(&mut harness.app);
|
||||
harness.app.api_messages =
|
||||
std::sync::Arc::new(vec![user_message("first turn still streaming")]);
|
||||
|
||||
@@ -236,6 +239,7 @@ fn rename_builds_from_app_state_before_any_checkpoint_exists() {
|
||||
let mut harness = ControlHarness::new();
|
||||
let session_id = "live-before-first-checkpoint";
|
||||
harness.app.current_session_id = Some(session_id.to_string());
|
||||
crate::test_support::capture_session_fixture_source(&mut harness.app);
|
||||
harness.app.api_messages =
|
||||
std::sync::Arc::new(vec![user_message("turn one, nothing persisted yet")]);
|
||||
|
||||
@@ -259,6 +263,8 @@ fn title_requires_an_active_session_and_preserves_raw_length_limit() {
|
||||
assert!(result_text(&no_session).contains("No active session"));
|
||||
|
||||
harness.app.current_session_id = Some("any".to_string());
|
||||
|
||||
crate::test_support::capture_session_fixture_source(&mut harness.app);
|
||||
let too_long = "x".repeat(MAX_TITLE_LEN + 1);
|
||||
let result = dispatch(&mut harness.app, "title", Some(&too_long));
|
||||
assert!(result.is_error);
|
||||
@@ -277,6 +283,7 @@ fn title_set_and_clear_persist_without_renaming_the_session() {
|
||||
.save_session(&session)
|
||||
.expect("save session");
|
||||
harness.app.current_session_id = Some("title-test".to_string());
|
||||
crate::test_support::capture_session_fixture_source(&mut harness.app);
|
||||
|
||||
let result = dispatch(&mut harness.app, "title", Some("parallel-task"));
|
||||
assert!(!result.is_error, "{result:?}");
|
||||
@@ -347,6 +354,7 @@ fn title_recovers_first_snapshot_from_checkpoint() {
|
||||
.save_checkpoint(&checkpoint)
|
||||
.expect("save checkpoint");
|
||||
harness.app.current_session_id = Some(session_id.clone());
|
||||
crate::test_support::capture_session_fixture_source(&mut harness.app);
|
||||
harness.app.api_messages =
|
||||
std::sync::Arc::new(vec![user_message("first turn still streaming")]);
|
||||
|
||||
@@ -368,6 +376,7 @@ fn title_builds_from_app_state_before_any_checkpoint_exists() {
|
||||
let mut harness = ControlHarness::new();
|
||||
let session_id = "live-title-before-first-checkpoint";
|
||||
harness.app.current_session_id = Some(session_id.to_string());
|
||||
crate::test_support::capture_session_fixture_source(&mut harness.app);
|
||||
harness.app.api_messages =
|
||||
std::sync::Arc::new(vec![user_message("turn one, nothing persisted yet")]);
|
||||
|
||||
|
||||
@@ -161,6 +161,7 @@ fn fork_saves_parent_and_switches_to_child_session() {
|
||||
let mut app = create_test_app_with_tmpdir(&tmpdir);
|
||||
app.set_provider_identity(codewhale_config::ProviderKind::Custom, "lm-studio");
|
||||
app.current_session_id = Some("parent-session".to_string());
|
||||
crate::test_support::capture_session_fixture_source(&mut app);
|
||||
let mut cached_parent = create_saved_session_with_id_and_mode(
|
||||
"parent-session".to_string(),
|
||||
&[],
|
||||
@@ -264,6 +265,7 @@ fn fork_rejects_active_runtime_without_switching_sessions() {
|
||||
let _home = SealedHome::new();
|
||||
let mut app = create_test_app_with_tmpdir(&tmpdir);
|
||||
app.current_session_id = Some("parent-session".to_string());
|
||||
crate::test_support::capture_session_fixture_source(&mut app);
|
||||
app.api_messages_mut().push(codewhale_models::Message {
|
||||
role: Role::User,
|
||||
content: vec![codewhale_models::ContentBlock::Text {
|
||||
@@ -287,6 +289,7 @@ fn new_session_from_resumed_state_creates_distinct_empty_session() {
|
||||
let _home = SealedHome::new();
|
||||
let mut app = create_test_app_with_tmpdir(&tmpdir);
|
||||
app.current_session_id = Some("old-session".to_string());
|
||||
crate::test_support::capture_session_fixture_source(&mut app);
|
||||
app.session_title = Some("Old Session".to_string());
|
||||
app.api_messages_mut().push(codewhale_models::Message {
|
||||
role: Role::User,
|
||||
@@ -354,6 +357,7 @@ fn conversation_reset_forgets_approvals(command: &str) {
|
||||
let _home = SealedHome::new();
|
||||
let mut app = create_test_app_with_tmpdir(&tmpdir);
|
||||
app.current_session_id = Some("old-session".to_string());
|
||||
crate::test_support::capture_session_fixture_source(&mut app);
|
||||
app.approval_session_denied
|
||||
.insert("shell:rm -rf build:call-1".to_string());
|
||||
app.approval_session_approved
|
||||
@@ -410,6 +414,7 @@ fn new_session_blocks_unsent_input_without_force() {
|
||||
let _home = SealedHome::new();
|
||||
let mut app = create_test_app_with_tmpdir(&tmpdir);
|
||||
app.current_session_id = Some("old-session".to_string());
|
||||
crate::test_support::capture_session_fixture_source(&mut app);
|
||||
app.input = "draft text".to_string();
|
||||
|
||||
let result = new_session(&mut app, None);
|
||||
@@ -433,6 +438,7 @@ fn new_session_force_discards_unsent_input() {
|
||||
let _home = SealedHome::new();
|
||||
let mut app = create_test_app_with_tmpdir(&tmpdir);
|
||||
app.current_session_id = Some("old-session".to_string());
|
||||
crate::test_support::capture_session_fixture_source(&mut app);
|
||||
app.input = "draft text".to_string();
|
||||
|
||||
let result = new_session(&mut app, Some("--force"));
|
||||
@@ -449,6 +455,7 @@ fn new_session_blocks_in_flight_turn_without_force() {
|
||||
let _home = SealedHome::new();
|
||||
let mut app = create_test_app_with_tmpdir(&tmpdir);
|
||||
app.current_session_id = Some("old-session".to_string());
|
||||
crate::test_support::capture_session_fixture_source(&mut app);
|
||||
app.is_loading = true;
|
||||
|
||||
let result = new_session(&mut app, None);
|
||||
@@ -464,6 +471,7 @@ fn new_session_force_cannot_detach_an_in_flight_turn() {
|
||||
let _home = SealedHome::new();
|
||||
let mut app = create_test_app_with_tmpdir(&tmpdir);
|
||||
app.current_session_id = Some("old-session".to_string());
|
||||
crate::test_support::capture_session_fixture_source(&mut app);
|
||||
app.api_messages_mut().push(codewhale_models::Message {
|
||||
role: Role::User,
|
||||
content: vec![],
|
||||
@@ -494,6 +502,7 @@ fn load_rejects_an_active_runtime_before_reading_or_mutating() {
|
||||
let _home = SealedHome::new();
|
||||
let mut app = create_test_app_with_tmpdir(&tmpdir);
|
||||
app.current_session_id = Some("old-session".to_string());
|
||||
crate::test_support::capture_session_fixture_source(&mut app);
|
||||
app.api_messages_mut().push(codewhale_models::Message {
|
||||
role: Role::User,
|
||||
content: vec![],
|
||||
@@ -706,6 +715,7 @@ fn new_session_is_all_or_nothing_when_work_state_is_busy() {
|
||||
content: vec![],
|
||||
});
|
||||
app.current_session_id = Some("current-session".to_string());
|
||||
crate::test_support::capture_session_fixture_source(&mut app);
|
||||
let todos = app.todos.clone();
|
||||
let _held = todos.try_lock().expect("hold todos lock");
|
||||
|
||||
|
||||
+101
-29
@@ -95,11 +95,8 @@ async fn persist_exec_checkpoint(
|
||||
turn_id: Option<&str>,
|
||||
engine: &codewhale_engine::core::engine::EngineHandle,
|
||||
deadline: Instant,
|
||||
source: &mut Option<codewhale_engine::session_manager::SessionWriteReceipt>,
|
||||
) {
|
||||
use codewhale_engine::core::events::{
|
||||
SessionCheckpointProjection, SessionCheckpointReceipt, SessionCheckpointRequest,
|
||||
};
|
||||
|
||||
if turn_id != Some(request.caller.turn_id.as_str()) {
|
||||
request.fail("exec session checkpoint does not match its active turn");
|
||||
return;
|
||||
@@ -122,10 +119,18 @@ async fn persist_exec_checkpoint(
|
||||
request.no_backend();
|
||||
return;
|
||||
};
|
||||
let Some(captured_source) = source.clone() else {
|
||||
request.fail("exec checkpoint has no admitted original session source");
|
||||
return;
|
||||
};
|
||||
let Some(captured_owner) = request.snapshot.source.clone() else {
|
||||
request.fail("exec checkpoint has no native Engine source");
|
||||
return;
|
||||
};
|
||||
let write_request = Arc::clone(&request);
|
||||
let mut writer = tokio::task::spawn_blocking(move || -> Result<SessionCheckpointReceipt> {
|
||||
let mut writer = tokio::task::spawn_blocking(move || -> Result<codewhale_engine::session_manager::SessionPublication> {
|
||||
let snapshot = &write_request.snapshot;
|
||||
let id = persist_exec_session(
|
||||
let publication = persist_exec_session(
|
||||
&snapshot.messages,
|
||||
&snapshot.model,
|
||||
PersistedProviderRoute {
|
||||
@@ -140,24 +145,12 @@ async fn persist_exec_checkpoint(
|
||||
manager.as_ref(),
|
||||
&[],
|
||||
Some(write_request.as_ref()),
|
||||
)?;
|
||||
let persisted = manager
|
||||
.load_session_snapshot(&id)
|
||||
.context("could not read back exec session checkpoint")?;
|
||||
write_request
|
||||
.validate_prefix(
|
||||
&persisted.metadata.id,
|
||||
&persisted.metadata.workspace,
|
||||
&persisted.messages,
|
||||
)
|
||||
.map_err(anyhow::Error::msg)?;
|
||||
Ok(SessionCheckpointReceipt {
|
||||
persisted_sha256: SessionCheckpointRequest::prefix_hash(&persisted.messages)
|
||||
.map_err(anyhow::Error::msg)?,
|
||||
messages_len: persisted.messages.len(),
|
||||
projection: SessionCheckpointProjection::Exact,
|
||||
})
|
||||
});
|
||||
&captured_source,
|
||||
&captured_owner,
|
||||
)?;
|
||||
Ok(publication)
|
||||
},
|
||||
);
|
||||
let result = tokio::select! {
|
||||
biased;
|
||||
() = request.withdraw.cancelled() => {
|
||||
@@ -178,7 +171,10 @@ async fn persist_exec_checkpoint(
|
||||
result = &mut writer => result,
|
||||
};
|
||||
match result {
|
||||
Ok(Ok(receipt)) => request.succeed(receipt),
|
||||
Ok(Ok(publication)) => {
|
||||
*source = Some(publication.write_receipt());
|
||||
request.succeed(publication.checkpoint_receipt);
|
||||
}
|
||||
Ok(Err(error)) => request.fail(format!("exec session checkpoint failed: {error:#}")),
|
||||
Err(error) => request.fail(format!("exec session checkpoint writer failed: {error}")),
|
||||
}
|
||||
@@ -276,6 +272,11 @@ pub(crate) async fn run_exec_agent(
|
||||
_ => anyhow::bail!("incomplete Fleet session capture destination"),
|
||||
};
|
||||
|
||||
let mut exec_write_receipt = resume_session
|
||||
.as_ref()
|
||||
.and_then(|saved| saved.write_receipt.clone());
|
||||
let exec_is_fresh = resume_session.is_none();
|
||||
|
||||
let exec_session_id = fleet_capture
|
||||
.as_ref()
|
||||
.map(|(id, _)| id.clone())
|
||||
@@ -307,7 +308,10 @@ pub(crate) async fn run_exec_agent(
|
||||
))
|
||||
};
|
||||
if should_persist_session {
|
||||
open_exec_session_manager()?;
|
||||
let manager = open_exec_session_manager()?;
|
||||
if exec_is_fresh {
|
||||
exec_write_receipt = Some(manager.expect_absent_session(&exec_session_id)?);
|
||||
}
|
||||
}
|
||||
|
||||
let route = resolve_cli_exec_route(config, model, prompt, force_configured_route).await?;
|
||||
@@ -699,6 +703,7 @@ pub(crate) async fn run_exec_agent(
|
||||
|
||||
engine_handle
|
||||
.send(Op::SyncSession {
|
||||
control: codewhale_engine::core::ops::SessionControlId::new(),
|
||||
session_id: Some(saved_id.clone()),
|
||||
messages: saved.messages,
|
||||
system_prompt: saved.system_prompt.map(SystemPrompt::Text),
|
||||
@@ -800,6 +805,7 @@ pub(crate) async fn run_exec_agent(
|
||||
let mut provider_request_receipts = Vec::new();
|
||||
let mut latest_session_id = loaded_session_id;
|
||||
let mut latest_messages: Arc<Vec<Message>> = Arc::new(Vec::new());
|
||||
let mut latest_source: Option<codewhale_engine::core::ops::SessionSnapshotSource> = None;
|
||||
let mut latest_system_prompt: Option<SystemPrompt> = None;
|
||||
let mut latest_model = effective_model.clone();
|
||||
let mut latest_workspace = workspace.clone();
|
||||
@@ -1207,6 +1213,9 @@ pub(crate) async fn run_exec_agent(
|
||||
}
|
||||
}
|
||||
Event::TurnComplete {
|
||||
turn_id,
|
||||
submission_id,
|
||||
compaction_id,
|
||||
status,
|
||||
error,
|
||||
usage,
|
||||
@@ -1214,6 +1223,25 @@ pub(crate) async fn run_exec_agent(
|
||||
..
|
||||
} => {
|
||||
let _timing = codewhale_runtime::logging::PhaseTimer::start("exec_postprocessing");
|
||||
let terminal_source = codewhale_engine::core::ops::SessionSnapshotSource::terminal(
|
||||
turn_id,
|
||||
submission_id,
|
||||
compaction_id,
|
||||
);
|
||||
let terminal_snapshot = match engine_handle.get_session_snapshot().await {
|
||||
Ok(snapshot) => Some(snapshot),
|
||||
Err(error) => {
|
||||
tracing::warn!(%error, "exec terminal session snapshot could not be read; checkpoint retained");
|
||||
None
|
||||
}
|
||||
}.filter(|snapshot| {
|
||||
snapshot.source.as_ref() == terminal_source.as_ref()
|
||||
&& terminal_source.is_some()
|
||||
&& snapshot.session_id == exec_session_id
|
||||
&& snapshot.workspace == exec_session_workspace
|
||||
&& latest_source.as_ref() == terminal_source.as_ref()
|
||||
&& snapshot.messages.as_slice() == latest_messages.as_slice()
|
||||
});
|
||||
let mut terminal_status = status;
|
||||
let terminal_error = error;
|
||||
#[cfg(unix)]
|
||||
@@ -1297,6 +1325,21 @@ pub(crate) async fn run_exec_agent(
|
||||
}
|
||||
let saved_session_id = if should_persist_session && !latest_messages.is_empty() {
|
||||
let saved = open_exec_session_manager().and_then(|manager| {
|
||||
if exec_write_receipt.is_none() && exec_is_fresh {
|
||||
exec_write_receipt =
|
||||
Some(manager.expect_absent_session(&exec_session_id)?);
|
||||
}
|
||||
let source = exec_write_receipt.as_ref().ok_or_else(|| {
|
||||
anyhow::anyhow!("exec original source was not admitted")
|
||||
})?;
|
||||
let owner = terminal_snapshot
|
||||
.as_ref()
|
||||
.and_then(|snapshot| snapshot.source.as_ref())
|
||||
.ok_or_else(|| {
|
||||
anyhow::anyhow!(
|
||||
"exec terminal snapshot belongs to another native turn"
|
||||
)
|
||||
})?;
|
||||
persist_exec_session(
|
||||
&latest_messages,
|
||||
&latest_model,
|
||||
@@ -1322,10 +1365,13 @@ pub(crate) async fn run_exec_agent(
|
||||
manager.as_ref(),
|
||||
&provider_request_receipts,
|
||||
None,
|
||||
source,
|
||||
owner,
|
||||
)
|
||||
});
|
||||
match saved {
|
||||
Ok(id) => {
|
||||
Ok(publication) => {
|
||||
let id = publication.metadata.id.clone();
|
||||
if output_format == ExecOutputFormat::Text && !json_output {
|
||||
eprintln!("{}", exec_saved_session_line(&id));
|
||||
}
|
||||
@@ -1515,6 +1561,7 @@ pub(crate) async fn run_exec_agent(
|
||||
should_persist_session = true;
|
||||
}
|
||||
Event::SessionUpdated {
|
||||
source,
|
||||
session_id,
|
||||
messages,
|
||||
system_prompt,
|
||||
@@ -1541,7 +1588,13 @@ pub(crate) async fn run_exec_agent(
|
||||
continue;
|
||||
}
|
||||
let manager = if should_persist_session {
|
||||
open_exec_session_manager().map(Some)
|
||||
open_exec_session_manager().and_then(|manager| {
|
||||
if exec_write_receipt.is_none() && exec_is_fresh {
|
||||
exec_write_receipt =
|
||||
Some(manager.expect_absent_session(&exec_session_id)?);
|
||||
}
|
||||
Ok(Some(manager))
|
||||
})
|
||||
} else {
|
||||
Ok(None)
|
||||
};
|
||||
@@ -1555,6 +1608,7 @@ pub(crate) async fn run_exec_agent(
|
||||
active_turn_id.as_deref(),
|
||||
&engine_handle,
|
||||
exec_deadline,
|
||||
&mut exec_write_receipt,
|
||||
)
|
||||
.await
|
||||
}
|
||||
@@ -1562,6 +1616,7 @@ pub(crate) async fn run_exec_agent(
|
||||
.fail(format!("could not open exec checkpoint store: {error:#}")),
|
||||
}
|
||||
}
|
||||
latest_source = source;
|
||||
latest_session_id = Some(session_id);
|
||||
latest_messages = messages;
|
||||
latest_system_prompt = system_prompt;
|
||||
@@ -1856,6 +1911,11 @@ mod tests {
|
||||
{"role":"assistant","content":[{"type":"tool_use","id":"provider-call","execution_id":"exec-call","name":"review","input":{}}]}
|
||||
])).unwrap();
|
||||
let snapshot = Arc::new(SessionSnapshot {
|
||||
source: Some(codewhale_engine::core::ops::SessionSnapshotSource::turn(
|
||||
"exec-turn".into(),
|
||||
None,
|
||||
)),
|
||||
|
||||
session_id: "exec-checkpoint".into(),
|
||||
messages: messages.clone(),
|
||||
total_tokens: 23,
|
||||
@@ -1889,6 +1949,7 @@ mod tests {
|
||||
receive,
|
||||
)
|
||||
};
|
||||
let mut source = Some(manager.expect_absent_session(&snapshot.session_id).unwrap());
|
||||
let (request, receive) = checkpoint(caller.clone(), Arc::clone(&snapshot));
|
||||
super::persist_exec_checkpoint(
|
||||
Arc::clone(&request),
|
||||
@@ -1898,6 +1959,7 @@ mod tests {
|
||||
Some("exec-turn"),
|
||||
&engine.handle,
|
||||
Instant::now() + Duration::from_secs(3),
|
||||
&mut source,
|
||||
)
|
||||
.await;
|
||||
assert!(receive.await.unwrap().unwrap());
|
||||
@@ -1915,7 +1977,7 @@ mod tests {
|
||||
completed.push(serde_json::from_value(serde_json::json!({
|
||||
"role":"user","content":[{"type":"tool_result","tool_use_id":"provider-call","execution_id":"exec-call","content":"review complete"}]
|
||||
})).unwrap());
|
||||
crate::persist_exec_session(
|
||||
let publication = crate::persist_exec_session(
|
||||
&completed,
|
||||
&snapshot.model,
|
||||
crate::PersistedProviderRoute {
|
||||
@@ -1930,8 +1992,12 @@ mod tests {
|
||||
manager.as_ref(),
|
||||
&[],
|
||||
None,
|
||||
source.as_ref()
|
||||
.unwrap(),
|
||||
snapshot.source.as_ref().unwrap(),
|
||||
)
|
||||
.unwrap();
|
||||
source = Some(publication.write_receipt());
|
||||
super::persist_exec_checkpoint(
|
||||
request,
|
||||
Some(Arc::clone(&manager)),
|
||||
@@ -1940,6 +2006,7 @@ mod tests {
|
||||
Some("exec-turn"),
|
||||
&engine.handle,
|
||||
Instant::now() + Duration::from_secs(3),
|
||||
&mut source,
|
||||
)
|
||||
.await;
|
||||
assert_eq!(
|
||||
@@ -1968,6 +2035,7 @@ mod tests {
|
||||
Some("exec-turn"),
|
||||
&engine.handle,
|
||||
Instant::now() + Duration::from_secs(3),
|
||||
&mut source,
|
||||
)
|
||||
.await;
|
||||
assert!(receive.await.unwrap().is_err(), "{mismatch}");
|
||||
@@ -1991,6 +2059,7 @@ mod tests {
|
||||
Some("exec-turn"),
|
||||
&engine.handle,
|
||||
Instant::now() + Duration::from_secs(3),
|
||||
&mut source,
|
||||
)
|
||||
.await;
|
||||
assert!(receive.await.unwrap().is_err());
|
||||
@@ -2003,6 +2072,7 @@ mod tests {
|
||||
Some("exec-turn"),
|
||||
&engine.handle,
|
||||
Instant::now() + Duration::from_secs(3),
|
||||
&mut source,
|
||||
)
|
||||
.await;
|
||||
assert!(!receive.await.unwrap().unwrap());
|
||||
@@ -2021,6 +2091,7 @@ mod tests {
|
||||
Some("exec-turn"),
|
||||
&engine.handle,
|
||||
Instant::now() + Duration::from_secs(3),
|
||||
&mut source,
|
||||
)
|
||||
.await;
|
||||
assert!(receive.await.unwrap().is_err());
|
||||
@@ -2033,6 +2104,7 @@ mod tests {
|
||||
Some("exec-turn"),
|
||||
&engine.handle,
|
||||
Instant::now(),
|
||||
&mut source,
|
||||
)
|
||||
.await;
|
||||
assert!(receive.await.unwrap().is_err());
|
||||
|
||||
+10
-5
@@ -15258,7 +15258,13 @@ fn persist_exec_session(
|
||||
manager: &SessionManager,
|
||||
provider_request_receipts: &[serde_json::Value],
|
||||
checkpoint: Option<&codewhale_engine::core::events::SessionCheckpointRequest>,
|
||||
) -> Result<String> {
|
||||
source: &session_manager::SessionWriteReceipt,
|
||||
owner: &codewhale_engine::core::ops::SessionSnapshotSource,
|
||||
) -> Result<session_manager::SessionPublication> {
|
||||
let captured = source.capture(session_manager::SessionWriteOwner::Engine(owner.clone()))?;
|
||||
if checkpoint.is_some() {
|
||||
captured.require_checkpoint_acknowledgement();
|
||||
}
|
||||
let mut saved = if let Some(id) = session_id.filter(|id| !id.trim().is_empty()) {
|
||||
match manager.load_session_snapshot(id) {
|
||||
Ok(existing) => session_manager::update_session(
|
||||
@@ -15290,6 +15296,7 @@ fn persist_exec_session(
|
||||
Some("exec"),
|
||||
)
|
||||
};
|
||||
saved.write_receipt = Some(captured);
|
||||
stamp_exec_session_metadata(
|
||||
&mut saved,
|
||||
model,
|
||||
@@ -15394,11 +15401,9 @@ fn persist_exec_session(
|
||||
)
|
||||
.map_err(anyhow::Error::msg)?;
|
||||
}
|
||||
let id = saved.metadata.id.clone();
|
||||
manager
|
||||
.save_session(&saved)
|
||||
.context("could not save exec session")?;
|
||||
Ok(id)
|
||||
.publish_session(&saved)
|
||||
.context("could not save exec session")
|
||||
}
|
||||
|
||||
fn stamp_exec_session_metadata(
|
||||
|
||||
@@ -6516,6 +6516,7 @@ mod tests {
|
||||
assert!(worker_rx.try_recv().is_err());
|
||||
|
||||
controller.observe_engine_event(&EngineEvent::TurnStarted {
|
||||
previous_source: None,
|
||||
turn_id: "turn_started_later".to_string(),
|
||||
created_at: chrono::Utc::now(),
|
||||
route: None,
|
||||
|
||||
@@ -526,19 +526,7 @@ pub(super) async fn create_session_from_thread(
|
||||
});
|
||||
}
|
||||
|
||||
let messages =
|
||||
messages_from_thread_detail(&state.runtime_threads, &detail).map_err(|error| {
|
||||
ApiError::internal(format!("Failed to reconstruct thread history: {error}"))
|
||||
})?;
|
||||
if messages.is_empty() {
|
||||
return Err(ApiError::bad_request(format!(
|
||||
"Thread {thread_id} has no user or assistant messages to save"
|
||||
)));
|
||||
}
|
||||
|
||||
let manager = SessionManager::new(state.sessions_dir.clone())
|
||||
.map_err(|e| ApiError::internal(format!("Failed to open sessions dir: {e}")))?;
|
||||
let total_tokens = total_tokens_from_thread_detail(&detail);
|
||||
let manager = SessionManager::new(state.sessions_dir.clone()).map_err(|e| ApiError::internal(format!("Failed to open sessions dir: {e}")))?;
|
||||
// Export is idempotent (#6144). Every POST used to mint a fresh document
|
||||
// and rebind the thread to it, so each re-export left the previous
|
||||
// document unreferenced, and a crash between the save and the bind below
|
||||
@@ -559,6 +547,30 @@ pub(super) async fn create_session_from_thread(
|
||||
Err(error) if error.kind() == std::io::ErrorKind::NotFound => None,
|
||||
Err(error) => return Err(map_session_err(&session_handle, error, "read")),
|
||||
};
|
||||
let control_owner = codewhale_engine::session_manager::SessionWriteOwner::control();
|
||||
let captured_source = if let Some(existing) = existing.as_ref() {
|
||||
existing
|
||||
.write_receipt
|
||||
.as_ref()
|
||||
.ok_or_else(|| ApiError::conflict("Export target source was not admitted"))?
|
||||
.capture(control_owner)
|
||||
} else {
|
||||
manager
|
||||
.expect_absent_session(&session_handle)
|
||||
.and_then(|source| source.capture(control_owner))
|
||||
}
|
||||
.map_err(|error| ApiError::conflict(error.to_string()))?;
|
||||
let messages =
|
||||
messages_from_thread_detail(&state.runtime_threads, &detail).map_err(|error| {
|
||||
ApiError::internal(format!("Failed to reconstruct thread history: {error}"))
|
||||
})?;
|
||||
if messages.is_empty() {
|
||||
return Err(ApiError::bad_request(format!(
|
||||
"Thread {thread_id} has no user or assistant messages to save"
|
||||
)));
|
||||
}
|
||||
|
||||
let total_tokens = total_tokens_from_thread_detail(&detail);
|
||||
let created = existing.is_none();
|
||||
let mut session = match existing {
|
||||
Some(existing) => {
|
||||
@@ -603,11 +615,13 @@ pub(super) async fn create_session_from_thread(
|
||||
let title = session.metadata.title.clone();
|
||||
let message_count = session.metadata.message_count;
|
||||
|
||||
session.write_receipt = Some(captured_source);
|
||||
persist_thread_cost(&state.runtime_threads, thread_id, &mut session).await?;
|
||||
|
||||
manager
|
||||
.save_session(&session)
|
||||
.map_err(|e| ApiError::internal(format!("Failed to save session: {e}")))?;
|
||||
let publication = manager
|
||||
.publish_session(&session)
|
||||
.map_err(|error| map_session_err(&session.metadata.id, error, "save"))?;
|
||||
session.adopt_publication(&publication);
|
||||
|
||||
// Link the session to the thread so that `ensure_engine_loaded` can
|
||||
// restore the full message history from the session file.
|
||||
@@ -836,20 +850,6 @@ pub(crate) async fn save_session_in_runtime(
|
||||
});
|
||||
}
|
||||
|
||||
// Get the engine handle (loads the thread into an engine if needed),
|
||||
// then request a session snapshot. This reuses the same code path as
|
||||
// TUI's `build_session_snapshot`: the engine holds the authoritative
|
||||
// messages and token usage, so we don't need to reconstruct from turns.
|
||||
let engine = runtime
|
||||
.get_engine(&thread_id)
|
||||
.await
|
||||
.map_err(|e| ApiError::internal(format!("Failed to get engine for thread: {e}")))?;
|
||||
|
||||
let snapshot = engine
|
||||
.get_session_snapshot()
|
||||
.await
|
||||
.map_err(|e| ApiError::internal(format!("Failed to get session snapshot: {e}")))?;
|
||||
|
||||
let manager = SessionManager::new(sessions_dir.to_path_buf())
|
||||
.map_err(|e| ApiError::internal(format!("Failed to open sessions dir: {e}")))?;
|
||||
|
||||
@@ -883,23 +883,57 @@ pub(crate) async fn save_session_in_runtime(
|
||||
// collide with another thread's document. Snapshot ownership does not
|
||||
// depend on this binding: a thread owns the restore points recorded on
|
||||
// its turns.
|
||||
let document_id = match req.session_id {
|
||||
Some(named) => named,
|
||||
None => runtime
|
||||
.get_thread(&thread_id)
|
||||
.await
|
||||
.map_err(map_thread_err)?
|
||||
.session_id
|
||||
.unwrap_or_else(|| snapshot.session_id.clone()),
|
||||
};
|
||||
let document_id = req.session_id
|
||||
.clone()
|
||||
.or_else(|| detail.thread.session_id.clone())
|
||||
.unwrap_or_else(|| codewhale_engine::runtime_threads::thread_session_id(&thread_id));
|
||||
let _lease = reserve_session_write(sessions_dir, &document_id, "save").await?;
|
||||
let existing = match manager.load_session_snapshot(&document_id) {
|
||||
Ok(session) => Some(session),
|
||||
Err(error) if error.kind() == std::io::ErrorKind::NotFound => None,
|
||||
Err(error) => return Err(map_session_err(&document_id, error, "read")),
|
||||
};
|
||||
let captured_source = if detail.thread.session_id.as_deref() == Some(document_id.as_str()) {
|
||||
let captured_runtime = std::sync::Arc::clone(runtime);
|
||||
let id = thread_id.clone();
|
||||
codewhale_engine::runtime_threads::manager::history::history_owner_work(move || {
|
||||
captured_runtime.saved_session_publication_source(&id)
|
||||
}).await.map_err(|error| ApiError::conflict(error.to_string()))?
|
||||
} else if let Some(existing) = existing.as_ref() {
|
||||
existing.write_receipt.clone()
|
||||
} else {
|
||||
Some(manager.expect_absent_session(&document_id).map_err(|error| ApiError::conflict(error.to_string()))?)
|
||||
}.ok_or_else(|| ApiError::conflict("This Runtime history has no admitted full-document publication source; reopen its saved session before saving"))?;
|
||||
|
||||
let captured_source = captured_source
|
||||
.capture(codewhale_engine::session_manager::SessionWriteOwner::control())
|
||||
.map_err(|error| ApiError::conflict(error.to_string()))?;
|
||||
// Get the engine handle (loads the thread into an engine if needed),
|
||||
// then request a session snapshot. This reuses the same code path as
|
||||
// TUI's `build_session_snapshot`: the engine holds the authoritative
|
||||
// messages and token usage, so we don't need to reconstruct from turns.
|
||||
let engine = runtime
|
||||
.get_engine(&thread_id)
|
||||
.await
|
||||
.map_err(|e| ApiError::internal(format!("Failed to get engine for thread: {e}")))?;
|
||||
|
||||
let snapshot = engine
|
||||
.get_session_snapshot()
|
||||
.await
|
||||
.map_err(|e| ApiError::internal(format!("Failed to get session snapshot: {e}")))?;
|
||||
|
||||
if snapshot.session_id != detail.thread.id || snapshot.workspace != detail.thread.workspace {
|
||||
return Err(ApiError::conflict(
|
||||
"Engine snapshot source changed before session publication",
|
||||
));
|
||||
}
|
||||
|
||||
// Build or update the session, mirroring TUI's `build_session_snapshot`.
|
||||
// Only `io::ErrorKind::NotFound` falls back to creating a new session;
|
||||
// other I/O errors (e.g. PermissionDenied) are propagated so callers
|
||||
// don't silently overwrite a corrupt or inaccessible session file.
|
||||
let mut session = match manager.load_session(&document_id) {
|
||||
Ok(existing) => {
|
||||
let mut session = match existing {
|
||||
Some(existing) => {
|
||||
let mut updated = codewhale_engine::session_manager::update_session(
|
||||
existing,
|
||||
&snapshot.messages,
|
||||
@@ -914,7 +948,7 @@ pub(crate) async fn save_session_in_runtime(
|
||||
updated.metadata.mode = Some(snapshot.mode.clone());
|
||||
updated
|
||||
}
|
||||
Err(e) if e.kind() == std::io::ErrorKind::NotFound => {
|
||||
None => {
|
||||
let mut session =
|
||||
codewhale_engine::session_manager::create_saved_session_with_id_and_mode(
|
||||
document_id.clone(),
|
||||
@@ -931,12 +965,8 @@ pub(crate) async fn save_session_in_runtime(
|
||||
);
|
||||
session
|
||||
}
|
||||
Err(e) => {
|
||||
return Err(ApiError::internal(format!(
|
||||
"Failed to load session {document_id}: {e}"
|
||||
)));
|
||||
}
|
||||
};
|
||||
session.write_receipt = Some(captured_source);
|
||||
|
||||
persist_thread_cost(runtime, &thread_id, &mut session).await?;
|
||||
|
||||
@@ -945,9 +975,10 @@ pub(crate) async fn save_session_in_runtime(
|
||||
}
|
||||
|
||||
// Save the session.
|
||||
manager
|
||||
.save_session(&session)
|
||||
.map_err(|e| ApiError::internal(format!("Failed to save session: {e}")))?;
|
||||
let publication = manager
|
||||
.publish_session(&session)
|
||||
.map_err(|error| map_session_err(&session.metadata.id, error, "save"))?;
|
||||
session.adopt_publication(&publication);
|
||||
|
||||
// Link the session to the thread so that `ensure_engine_loaded` can
|
||||
// restore the full message history (including thinking/tool blocks)
|
||||
@@ -1817,6 +1848,12 @@ pub(crate) async fn initialize_empty_session(
|
||||
}
|
||||
Err(error) => return Err(map_session_err(session_id, error, "read")),
|
||||
}
|
||||
let source = manager
|
||||
.expect_absent_session(session_id)
|
||||
.and_then(|source| {
|
||||
source.capture(codewhale_engine::session_manager::SessionWriteOwner::control())
|
||||
})
|
||||
.map_err(|error| ApiError::conflict(error.to_string()))?;
|
||||
let mut session = create_saved_session_with_id_and_mode(
|
||||
session_id.to_string(),
|
||||
&[],
|
||||
@@ -1829,9 +1866,11 @@ pub(crate) async fn initialize_empty_session(
|
||||
stamp_session_provider_from_thread(&runtime.read_config(), &detail, &mut session.metadata)
|
||||
.map_err(ApiError::bad_request)?;
|
||||
session.metadata.runtime_store = Some(runtime.session_store_binding());
|
||||
manager
|
||||
.save_session(&session)
|
||||
.map_err(|error| ApiError::internal(error.to_string()))?;
|
||||
session.write_receipt = Some(source);
|
||||
let publication = manager
|
||||
.publish_session(&session)
|
||||
.map_err(|error| map_session_err(session_id, error, "initialize"))?;
|
||||
session.adopt_publication(&publication);
|
||||
runtime
|
||||
.set_thread_session_checkpoint(thread_id, &session)
|
||||
.await
|
||||
|
||||
@@ -305,8 +305,13 @@ impl codewhale_engine::task_manager::TaskExecutor for MockExecutor {
|
||||
|
||||
fn saved_session_with_blocks(blocks: Vec<codewhale_models::ContentBlock>) -> SavedSession {
|
||||
SavedSession {
|
||||
write_receipt: None,
|
||||
|
||||
schema_version: 1,
|
||||
metadata: SessionMetadata {
|
||||
publication_id: None,
|
||||
native_mutation_observation: None,
|
||||
|
||||
id: "session-1".to_string(),
|
||||
title: "test session".to_string(),
|
||||
title_is_custom: false,
|
||||
@@ -3515,6 +3520,7 @@ async fn compatibility_stream_closes_losslessly_across_replay_live_handoff() ->
|
||||
let _ = wait_for_overlap_release.await;
|
||||
let _ = tx_event
|
||||
.send(EngineEvent::TurnStarted {
|
||||
previous_source: None,
|
||||
turn_id: "mock_compat_handoff".to_string(),
|
||||
created_at: chrono::Utc::now(),
|
||||
route: None,
|
||||
@@ -3722,6 +3728,7 @@ async fn compatibility_stream_exposes_and_resolves_user_input_without_answer_ech
|
||||
harness
|
||||
.tx_event
|
||||
.send(EngineEvent::TurnStarted {
|
||||
previous_source: None,
|
||||
turn_id: "mock_compat_input".to_string(),
|
||||
created_at: chrono::Utc::now(),
|
||||
route: None,
|
||||
@@ -4197,6 +4204,7 @@ async fn thread_endpoints_expose_lifecycle_contract() -> Result<()> {
|
||||
Op::SendMessage(TurnSpec { .. }) => {
|
||||
let _ = tx_event
|
||||
.send(EngineEvent::TurnStarted {
|
||||
previous_source: None,
|
||||
turn_id: "mock_lifecycle".to_string(),
|
||||
created_at: chrono::Utc::now(),
|
||||
route: None,
|
||||
@@ -4593,6 +4601,7 @@ async fn turn_operation_lookup_is_authenticated_read_only_and_survives_restart()
|
||||
|
||||
for event in [
|
||||
EngineEvent::TurnStarted {
|
||||
previous_source: None,
|
||||
turn_id: "lookup-fixture-engine-turn".into(),
|
||||
created_at: chrono::Utc::now(),
|
||||
route: None,
|
||||
@@ -4728,6 +4737,7 @@ async fn events_endpoint_respects_since_seq_cursor() -> Result<()> {
|
||||
}
|
||||
let _ = tx_event
|
||||
.send(EngineEvent::TurnStarted {
|
||||
previous_source: None,
|
||||
turn_id: "mock_cursor".to_string(),
|
||||
created_at: chrono::Utc::now(),
|
||||
route: None,
|
||||
@@ -5660,6 +5670,7 @@ async fn steer_and_interrupt_endpoints_work_on_active_turn() -> Result<()> {
|
||||
}
|
||||
let _ = tx_event
|
||||
.send(EngineEvent::TurnStarted {
|
||||
previous_source: None,
|
||||
turn_id: "engine_turn_api".to_string(),
|
||||
created_at: chrono::Utc::now(),
|
||||
route: None,
|
||||
@@ -6218,6 +6229,7 @@ async fn stream_endpoint_remains_backward_compatible() -> Result<()> {
|
||||
}
|
||||
let _ = tx_event
|
||||
.send(EngineEvent::TurnStarted {
|
||||
previous_source: None,
|
||||
turn_id: "mock_stream".to_string(),
|
||||
created_at: chrono::Utc::now(),
|
||||
route: None,
|
||||
@@ -7208,6 +7220,8 @@ async fn session_save_without_an_id_keeps_the_conversation_that_owns_the_snapsho
|
||||
while let Some(op) = rx_op.recv().await {
|
||||
if let Op::GetSessionSnapshot { tx } = op {
|
||||
let snapshot = codewhale_engine::core::ops::SessionSnapshot {
|
||||
source: None,
|
||||
|
||||
session_id: CONVERSATION_ID.to_string(),
|
||||
messages: vec![Message {
|
||||
role: Role::User,
|
||||
@@ -8791,6 +8805,7 @@ async fn session_create_from_thread_rejects_active_turn() -> Result<()> {
|
||||
}
|
||||
let _ = tx_event
|
||||
.send(EngineEvent::TurnStarted {
|
||||
previous_source: None,
|
||||
turn_id: "mock_active_session_save".to_string(),
|
||||
created_at: chrono::Utc::now(),
|
||||
route: None,
|
||||
@@ -20074,6 +20089,7 @@ async fn runtime_image_http_rejects_before_dispatch_and_accepts_large_canonical_
|
||||
harness
|
||||
.tx_event
|
||||
.send(EngineEvent::TurnStarted {
|
||||
previous_source: None,
|
||||
turn_id: "image-http-fixture".into(),
|
||||
created_at: Utc::now(),
|
||||
route: None,
|
||||
|
||||
@@ -314,13 +314,65 @@ async fn import_into_owner(
|
||||
.await?
|
||||
}
|
||||
};
|
||||
let mut session = SavedSession::import_foreign(
|
||||
let recovery_dir = state.sessions_dir.clone();
|
||||
let recovery_operation = operation.clone();
|
||||
let recovery_runtime = Arc::clone(&runtime);
|
||||
let recovered = history_owner_work(move || {
|
||||
let Some(expected_digest) = recovery_operation.target_document_digest.as_ref() else {
|
||||
return Ok(None);
|
||||
};
|
||||
let manager = SessionManager::new(recovery_dir)?;
|
||||
let lease =
|
||||
manager.reserve_session_for_external_write(&recovery_operation.receipt.session_id)?;
|
||||
let saved = match manager.load_session_snapshot_bounded(
|
||||
&recovery_operation.receipt.session_id,
|
||||
MAX_CANONICAL_HISTORY_BYTES,
|
||||
) {
|
||||
Ok(saved) => saved,
|
||||
Err(error) if error.kind() == std::io::ErrorKind::NotFound => return Ok(None),
|
||||
Err(error) => return Err(error.into()),
|
||||
};
|
||||
if saved_document_digest(&saved)? != *expected_digest {
|
||||
return Ok(None);
|
||||
}
|
||||
recovery_runtime.verify_prepared_history_publication(&recovery_operation, &saved)?;
|
||||
Ok(Some((saved, lease)))
|
||||
})
|
||||
.await?;
|
||||
let target_merge = target.is_some();
|
||||
let (session, _target_lease, operation) = if let Some((session, lease)) = recovered {
|
||||
(session, lease, operation)
|
||||
} else {
|
||||
let publication_source = if target.is_none() {
|
||||
let dir = state.sessions_dir.clone();
|
||||
let id = operation.receipt.session_id.clone();
|
||||
let captured_operation = operation.clone();
|
||||
let source_runtime = Arc::clone(&runtime);
|
||||
Some(
|
||||
history_owner_work(move || {
|
||||
let manager = SessionManager::new(dir)?;
|
||||
source_runtime.capture_history_publication_source(
|
||||
&captured_operation,
|
||||
manager.expect_absent_session(&id)?,
|
||||
)
|
||||
})
|
||||
.await?,
|
||||
)
|
||||
} else {
|
||||
None
|
||||
};
|
||||
let mut session = SavedSession::import_foreign(
|
||||
SessionImportContainer::new("legacy-state-sqlite".into(), &journal, None),
|
||||
thread.workspace.clone(),
|
||||
thread.model.clone(),
|
||||
)
|
||||
.map_err(anyhow::Error::msg)?;
|
||||
session.metadata.id = operation.receipt.session_id.clone();
|
||||
session.write_receipt = publication_source;
|
||||
session.metadata.publication_id = session
|
||||
.write_receipt
|
||||
.as_ref()
|
||||
.and_then(|receipt| receipt.publication_id());
|
||||
session.metadata.id = operation.receipt.session_id.clone();
|
||||
session.system_prompt.clone_from(&thread.system_prompt);
|
||||
session.metadata.created_at = operation.created_at;
|
||||
session.metadata.updated_at = operation.created_at;
|
||||
@@ -339,7 +391,7 @@ async fn import_into_owner(
|
||||
!super::sessions::thread_detail_has_live_work(&detail),
|
||||
"canonical target has live work; retry the same import after settlement"
|
||||
);
|
||||
let full = snapshot_in_owner(state.clone(), thread.id.clone())
|
||||
let (full, source) = snapshot_in_owner(state.clone(), thread.id.clone())
|
||||
.await
|
||||
.map_err(|error| anyhow!(error.message))?;
|
||||
captured_target_document_digest = full.saved_document_digest;
|
||||
@@ -348,6 +400,20 @@ async fn import_into_owner(
|
||||
"canonical target full document witness is absent"
|
||||
);
|
||||
let mut observed: SavedSession = serde_json::from_value(full.session)?;
|
||||
let source_runtime = Arc::clone(&runtime);
|
||||
let captured_operation = operation.clone();
|
||||
let source =
|
||||
source.ok_or_else(|| anyhow!("canonical target source was not admitted"))?;
|
||||
observed.write_receipt = Some(
|
||||
history_owner_work(move || {
|
||||
source_runtime.capture_history_publication_source(&captured_operation, source)
|
||||
})
|
||||
.await?,
|
||||
);
|
||||
observed.metadata.publication_id = observed
|
||||
.write_receipt
|
||||
.as_ref()
|
||||
.and_then(|receipt| receipt.publication_id());
|
||||
ensure!(
|
||||
observed.metadata.id == operation.receipt.session_id,
|
||||
"canonical target session changed before source merge"
|
||||
@@ -394,9 +460,19 @@ async fn import_into_owner(
|
||||
serde_json::to_writer(&mut bound, &observed)?;
|
||||
session = observed;
|
||||
}
|
||||
let target_merge = target.is_some();
|
||||
let document_digest = saved_document_digest(&session)?;
|
||||
let prepare_runtime = Arc::clone(&runtime);
|
||||
let prepared_operation = operation.clone();
|
||||
let bound_operation = history_owner_work(move || {
|
||||
prepare_runtime.bind_history_operation_document(prepared_operation, &document_digest)
|
||||
})
|
||||
.await?;
|
||||
let target_merge = target.is_some();
|
||||
let sessions_dir = state.sessions_dir.clone();
|
||||
let (session, _target_lease) = history_owner_work(move || {
|
||||
let verification_runtime = Arc::clone(&runtime);
|
||||
let verification_operation = bound_operation.clone();
|
||||
let (session, target_lease) = history_owner_work(move || {
|
||||
let mut session = session;
|
||||
let manager = SessionManager::new(sessions_dir)?;
|
||||
let lease = manager.reserve_session_for_external_write(&session.metadata.id)?;
|
||||
match manager
|
||||
@@ -413,25 +489,24 @@ async fn import_into_owner(
|
||||
) && observed.metadata.workspace == session.metadata.workspace,
|
||||
"canonical target store binding changed before merge"
|
||||
);
|
||||
manager.save_session(&session)?;
|
||||
let publication = manager.publish_session(&session)?;
|
||||
session.adopt_publication(&publication);
|
||||
return Ok((session, lease));
|
||||
}
|
||||
Ok(observed) => {
|
||||
ensure!(
|
||||
observed.journal == session.journal
|
||||
&& observed.metadata.runtime_store == session.metadata.runtime_store
|
||||
&& observed.metadata.workspace == session.metadata.workspace,
|
||||
"reserved imported session diverged; recovery required"
|
||||
);
|
||||
verification_runtime.verify_prepared_history_publication(&verification_operation, &observed)?;
|
||||
return Ok((observed, lease));
|
||||
}
|
||||
Err(error) if error.kind() == std::io::ErrorKind::NotFound && !target_merge => {}
|
||||
Err(error) => return Err(error.into()),
|
||||
}
|
||||
manager.save_session(&session)?;
|
||||
let publication = manager.publish_session(&session)?;
|
||||
session.adopt_publication(&publication);
|
||||
Ok((session, lease))
|
||||
})
|
||||
.await?;
|
||||
(session, target_lease, bound_operation)
|
||||
};
|
||||
let detail = runtime.get_thread_detail(&thread.id).await?;
|
||||
ensure!(
|
||||
!super::sessions::thread_detail_has_live_work(&detail),
|
||||
@@ -539,13 +614,19 @@ pub(super) async fn snapshot_thread_history(
|
||||
axum::extract::Path(id): axum::extract::Path<String>,
|
||||
) -> Result<Json<codewhale_protocol::CanonicalThreadSnapshot>, ApiError> {
|
||||
let _admission = state.runtime_threads.session_checkpoint_guard().await;
|
||||
snapshot_in_owner(state, id).await.map(Json)
|
||||
snapshot_in_owner(state, id).await.map(|(snapshot, _)| Json(snapshot))
|
||||
}
|
||||
|
||||
async fn snapshot_in_owner(
|
||||
state: RuntimeApiState,
|
||||
id: String,
|
||||
) -> Result<codewhale_protocol::CanonicalThreadSnapshot, ApiError> {
|
||||
) -> Result<
|
||||
(
|
||||
codewhale_protocol::CanonicalThreadSnapshot,
|
||||
Option<codewhale_engine::session_manager::SessionWriteReceipt>,
|
||||
),
|
||||
ApiError,
|
||||
> {
|
||||
snapshot_in_runtime(&state.runtime_threads, &state.sessions_dir, id).await
|
||||
}
|
||||
|
||||
@@ -553,7 +634,13 @@ async fn snapshot_in_runtime(
|
||||
runtime: &Arc<RuntimeThreadManager>,
|
||||
sessions_dir: &Path,
|
||||
id: String,
|
||||
) -> Result<codewhale_protocol::CanonicalThreadSnapshot, ApiError> {
|
||||
) -> Result<
|
||||
(
|
||||
codewhale_protocol::CanonicalThreadSnapshot,
|
||||
Option<codewhale_engine::session_manager::SessionWriteReceipt>,
|
||||
),
|
||||
ApiError,
|
||||
> {
|
||||
let runtime = Arc::clone(runtime);
|
||||
let thread = runtime
|
||||
.get_thread(&id)
|
||||
@@ -673,6 +760,7 @@ async fn snapshot_in_runtime(
|
||||
// result rather than truncating branches or returning a partial graph.
|
||||
let mut size = HistorySizeBound(MAX_CANONICAL_HISTORY_BYTES);
|
||||
serde_json::to_writer(&mut size, &session)?;
|
||||
let source = session.write_receipt.clone();
|
||||
let response = codewhale_protocol::CanonicalThreadSnapshot {
|
||||
version: 1,
|
||||
data_dir: expected_binding.data_dir,
|
||||
@@ -686,7 +774,7 @@ async fn snapshot_in_runtime(
|
||||
};
|
||||
let mut size = HistorySizeBound(MAX_CANONICAL_HISTORY_BYTES);
|
||||
serde_json::to_writer(&mut size, &response)?;
|
||||
Ok(response)
|
||||
Ok((response, source))
|
||||
})
|
||||
.await
|
||||
.map_err(|error| ApiError::conflict(error.to_string()))?;
|
||||
@@ -937,6 +1025,7 @@ async fn settle_prepared_history_operation(
|
||||
let binding = runtime.session_store_binding();
|
||||
let expected = operation.clone();
|
||||
let selected_workspace = workspace.to_path_buf();
|
||||
let verification_runtime = Arc::clone(runtime);
|
||||
let saved = history_owner_work(move || {
|
||||
let manager = SessionManager::new(dir)?;
|
||||
let lease = manager.reserve_session_for_external_write(&expected.receipt.session_id)?;
|
||||
@@ -953,6 +1042,7 @@ async fn settle_prepared_history_operation(
|
||||
if expected.target_document_digest.as_ref() != Some(&saved_document_digest(&saved)?) {
|
||||
return Ok(None);
|
||||
}
|
||||
verification_runtime.verify_prepared_history_publication(&expected, &saved)?;
|
||||
ensure!(
|
||||
saved.metadata.runtime_store.as_ref() == Some(&binding)
|
||||
&& saved.metadata.workspace == selected_workspace,
|
||||
@@ -1141,7 +1231,7 @@ async fn mutate_in_owner(
|
||||
!super::sessions::thread_detail_has_live_work(&detail),
|
||||
"selected history has pending work; wait before resume or fork"
|
||||
);
|
||||
let snapshot =
|
||||
let (snapshot, source) =
|
||||
snapshot_in_runtime(&runtime, &sessions_dir, runtime_thread_id.clone())
|
||||
.await
|
||||
.map_err(|error| anyhow!(error.message))?;
|
||||
@@ -1150,7 +1240,8 @@ async fn mutate_in_owner(
|
||||
"selected full history changed before canonical admission"
|
||||
);
|
||||
source_disk_digest = snapshot.saved_document_digest;
|
||||
let saved: SavedSession = serde_json::from_value(snapshot.session)?;
|
||||
let mut saved: SavedSession = serde_json::from_value(snapshot.session)?;
|
||||
saved.write_receipt = source;
|
||||
let id = saved.metadata.id.clone();
|
||||
let dir = sessions_dir.clone();
|
||||
let expected = snapshot.session_goal_digest;
|
||||
@@ -1264,7 +1355,7 @@ async fn mutate_in_owner(
|
||||
covered.is_some(),
|
||||
"saved holder checkpoint differs from the saved history; refresh history"
|
||||
);
|
||||
let full =
|
||||
let (full, _) =
|
||||
snapshot_in_runtime(&runtime, &sessions_dir, thread.id.clone())
|
||||
.await
|
||||
.map_err(|error| anyhow!(error.message))?;
|
||||
@@ -1535,6 +1626,50 @@ async fn mutate_in_owner(
|
||||
.await?;
|
||||
}
|
||||
let source_id = source.as_ref().map(|session| session.metadata.id.clone());
|
||||
let publication_source = if source_id.as_ref() == Some(&operation.receipt.session_id) {
|
||||
match source
|
||||
.as_ref()
|
||||
.and_then(|session| session.write_receipt.as_ref())
|
||||
{
|
||||
Some(source) => {
|
||||
let source_runtime = Arc::clone(&runtime);
|
||||
let captured_operation = operation.clone();
|
||||
let source = source.clone();
|
||||
history_owner_work(move || {
|
||||
source_runtime.capture_history_publication_source(&captured_operation, source)
|
||||
})
|
||||
.await?
|
||||
}
|
||||
None if source_disk_digest.is_none() => {
|
||||
let dir = sessions_dir.clone();
|
||||
let id = operation.receipt.session_id.clone();
|
||||
let captured_operation = operation.clone();
|
||||
let source_runtime = Arc::clone(&runtime);
|
||||
history_owner_work(move || {
|
||||
let manager = SessionManager::new(dir)?;
|
||||
source_runtime.capture_history_publication_source(
|
||||
&captured_operation,
|
||||
manager.expect_absent_session(&id)?,
|
||||
)
|
||||
})
|
||||
.await?
|
||||
}
|
||||
None => bail!("canonical source lacks its admitted publication receipt"),
|
||||
}
|
||||
} else {
|
||||
let dir = sessions_dir.clone();
|
||||
let id = operation.receipt.session_id.clone();
|
||||
let captured_operation = operation.clone();
|
||||
let source_runtime = Arc::clone(&runtime);
|
||||
history_owner_work(move || {
|
||||
let manager = SessionManager::new(dir)?;
|
||||
source_runtime.capture_history_publication_source(
|
||||
&captured_operation,
|
||||
manager.expect_absent_session(&id)?,
|
||||
)
|
||||
})
|
||||
.await?
|
||||
};
|
||||
let mut session = source.unwrap_or_else(|| {
|
||||
codewhale_engine::session_manager::create_saved_session_with_id_and_mode(
|
||||
operation.receipt.session_id.clone(),
|
||||
@@ -1546,6 +1681,8 @@ async fn mutate_in_owner(
|
||||
Some(&thread.mode),
|
||||
)
|
||||
});
|
||||
session.metadata.publication_id = publication_source.publication_id();
|
||||
session.write_receipt = Some(publication_source);
|
||||
session.metadata.id = operation.receipt.session_id.clone();
|
||||
if !is_resume {
|
||||
session.metadata.created_at = operation.created_at;
|
||||
@@ -1578,11 +1715,13 @@ async fn mutate_in_owner(
|
||||
// Revalidate the exact source file immediately before target publication.
|
||||
// Both existing session leases are held until the target write settles.
|
||||
let dir = sessions_dir.clone();
|
||||
let expected_journal = journal.clone();
|
||||
let source_goal_digest = session_goal_digest(&source_goal)?;
|
||||
let target_goal_digest = session_goal_digest(&target_goal)?;
|
||||
let target_id = session.metadata.id.clone();
|
||||
let verification_runtime = Arc::clone(&runtime);
|
||||
let verification_operation = operation.clone();
|
||||
let (session, _source_lease, _target_lease) = history_owner_work(move || {
|
||||
let mut session = session;
|
||||
let manager = SessionManager::new(dir)?;
|
||||
let source_lease = source_id
|
||||
.as_ref()
|
||||
@@ -1612,11 +1751,8 @@ async fn mutate_in_owner(
|
||||
}
|
||||
match manager.load_session_snapshot_bounded(&target_id, MAX_CANONICAL_HISTORY_BYTES) {
|
||||
Ok(observed) if source_id.as_ref() != Some(&target_id) => {
|
||||
ensure!(
|
||||
observed.journal.as_ref() == Some(&expected_journal)
|
||||
&& observed.metadata.runtime_store == session.metadata.runtime_store,
|
||||
"reserved canonical target changed; recovery required"
|
||||
);
|
||||
verification_runtime
|
||||
.verify_prepared_history_publication(&verification_operation, &observed)?;
|
||||
ensure!(
|
||||
session_goal_digest(&manager.load_session_goal(&target_id)?)?
|
||||
== target_goal_digest,
|
||||
@@ -1625,12 +1761,14 @@ async fn mutate_in_owner(
|
||||
Ok((observed, source_lease, target_lease))
|
||||
}
|
||||
Err(error) if error.kind() == std::io::ErrorKind::NotFound => {
|
||||
manager.save_session(&session)?;
|
||||
let publication = manager.publish_session(&session)?;
|
||||
session.adopt_publication(&publication);
|
||||
manager.save_session_goal(&target_id, target_goal.as_ref())?;
|
||||
Ok((session, source_lease, target_lease))
|
||||
}
|
||||
Ok(_) => {
|
||||
manager.save_session(&session)?;
|
||||
let publication = manager.publish_session(&session)?;
|
||||
session.adopt_publication(&publication);
|
||||
manager.save_session_goal(&target_id, target_goal.as_ref())?;
|
||||
Ok((session, source_lease, target_lease))
|
||||
}
|
||||
|
||||
@@ -1,5 +1,32 @@
|
||||
pub(crate) use codewhale_engine::test_support::*;
|
||||
|
||||
pub(crate) fn capture_session_fixture_source(app: &mut crate::tui::app::App) {
|
||||
use codewhale_engine::session_manager::{SessionManager, SessionWriteOwner};
|
||||
let Some(id) = app.current_session_id.as_deref() else {
|
||||
return;
|
||||
};
|
||||
let manager = SessionManager::default_location().expect("fixture session manager");
|
||||
let source = match manager.load_session_snapshot(id) {
|
||||
Ok(session) => session.write_receipt,
|
||||
Err(error) if error.kind() == std::io::ErrorKind::NotFound => {
|
||||
match manager
|
||||
.load_session_checkpoint(id)
|
||||
.expect("fixture checkpoint source")
|
||||
{
|
||||
Some(session) => session.write_receipt,
|
||||
None => Some(
|
||||
manager
|
||||
.expect_absent_session(id)
|
||||
.expect("fresh fixture source"),
|
||||
),
|
||||
}
|
||||
}
|
||||
Err(error) => panic!("fixture session source: {error}"),
|
||||
};
|
||||
app.session_write_receipt = Some(source.expect("fixture admitted source"));
|
||||
app.session_write_owner = Some(SessionWriteOwner::control());
|
||||
}
|
||||
|
||||
#[cfg(unix)]
|
||||
mod tests {
|
||||
use std::path::Path;
|
||||
|
||||
@@ -2260,6 +2260,9 @@ pub struct App {
|
||||
pub undo_selection: bool,
|
||||
/// Current session ID for auto-save updates
|
||||
pub current_session_id: Option<String>,
|
||||
pub(crate) session_write_receipt:
|
||||
Option<codewhale_engine::session_manager::SessionWriteReceipt>,
|
||||
pub(crate) session_write_owner: Option<codewhale_engine::session_manager::SessionWriteOwner>,
|
||||
/// Exclusive editor ownership, shared with outstanding queue writes.
|
||||
pub(crate) offline_queue_lease:
|
||||
Option<std::sync::Arc<codewhale_engine::session_manager::OfflineQueueLease>>,
|
||||
@@ -5307,9 +5310,38 @@ impl App {
|
||||
self.needs_redraw = true;
|
||||
}
|
||||
|
||||
pub(crate) fn adopt_session_mutation(&mut self, metadata: &SessionMetadata) {
|
||||
if self.current_session_id.as_deref() == Some(metadata.id.as_str())
|
||||
&& let Some(mutation) = metadata.native_mutation_observation.as_ref()
|
||||
&& let Some(receipt) = self.session_write_receipt.as_ref()
|
||||
&& let Ok(receipt) = receipt.adopt_mutation(mutation)
|
||||
{
|
||||
self.session_write_receipt = Some(receipt);
|
||||
}
|
||||
}
|
||||
|
||||
pub(crate) fn owns_session_projection(
|
||||
&self,
|
||||
source: Option<&codewhale_engine::core::ops::SessionSnapshotSource>,
|
||||
) -> bool {
|
||||
self.session_write_owner
|
||||
.as_ref()
|
||||
.zip(source)
|
||||
.is_some_and(|(owner, source)| owner.matches_source(source))
|
||||
}
|
||||
|
||||
pub(crate) fn session_sync_control(&mut self) -> codewhale_engine::core::ops::SessionControlId {
|
||||
let id = codewhale_engine::core::ops::SessionControlId::new();
|
||||
self.session_write_owner =
|
||||
Some(codewhale_engine::session_manager::SessionWriteOwner::Control(id.clone()));
|
||||
id
|
||||
}
|
||||
|
||||
pub(crate) fn set_current_session_id(&mut self, session_id: Option<String>) {
|
||||
if self.current_session_id != session_id {
|
||||
self.view_stack.remove_owned_evidence_pagers();
|
||||
self.session_write_receipt = None;
|
||||
self.session_write_owner = None;
|
||||
}
|
||||
self.current_session_id = session_id;
|
||||
}
|
||||
@@ -5587,6 +5619,11 @@ impl App {
|
||||
messages: Vec<Message>,
|
||||
session: &codewhale_engine::session_manager::SavedSession,
|
||||
) {
|
||||
self.session_write_receipt = session.write_receipt.clone();
|
||||
self.session_write_owner = session
|
||||
.write_receipt
|
||||
.as_ref()
|
||||
.map(|_| codewhale_engine::session_manager::SessionWriteOwner::control());
|
||||
let journal = session.journal.clone().unwrap_or_else(|| {
|
||||
codewhale_runtime::session_tree::SessionJournal::from_messages(
|
||||
session.messages.clone(),
|
||||
@@ -5605,6 +5642,11 @@ impl App {
|
||||
&mut self,
|
||||
session: &mut codewhale_engine::session_manager::SavedSession,
|
||||
) {
|
||||
self.session_write_receipt = session.write_receipt.clone();
|
||||
self.session_write_owner = session
|
||||
.write_receipt
|
||||
.as_ref()
|
||||
.map(|_| codewhale_engine::session_manager::SessionWriteOwner::control());
|
||||
let stamps = session.journal_message_stamps();
|
||||
let journal = match session.journal.take() {
|
||||
Some(journal) => journal,
|
||||
|
||||
@@ -1055,6 +1055,8 @@ impl App {
|
||||
backtrack: crate::tui::backtrack::BacktrackState::new(),
|
||||
undo_selection: false,
|
||||
current_session_id: None,
|
||||
session_write_receipt: None,
|
||||
session_write_owner: None,
|
||||
offline_queue_lease: None,
|
||||
last_known_work_state: None,
|
||||
last_known_goal_state: None,
|
||||
|
||||
@@ -747,6 +747,13 @@ pub fn history_cells_from_message(msg: &Message) -> Vec<HistoryCell> {
|
||||
if codewhale_engine::runtime_handoff::is_operate_contract_message(msg) {
|
||||
return Vec::new();
|
||||
}
|
||||
if codewhale_models::turn_classify::is_wire_compaction_checkpoint_message(msg)
|
||||
&& let Some(ContentBlock::Text { text, .. }) = msg.content.first()
|
||||
{
|
||||
return vec![HistoryCell::System {
|
||||
content: text.clone(),
|
||||
}];
|
||||
}
|
||||
if let Some(display) =
|
||||
codewhale_models::turn_classify::restored_subagent_checkpoint_display(msg)
|
||||
{
|
||||
|
||||
@@ -24,7 +24,7 @@
|
||||
//! - **Bounded command channel with sender-side coalescing** (#6212):
|
||||
//! `try_send` absorbs each request into a shared latest-wins state at send
|
||||
//! time and wakes the actor through a small bounded channel, so a paused
|
||||
//! consumer retains one snapshot per session instead of one per send.
|
||||
//! consumer retains one barrier and one successor per session instead of one per send.
|
||||
//! Queued snapshots retain only the canonical journal; legacy `messages`
|
||||
//! are derived at the disk boundary instead of doubling every paused
|
||||
//! request.
|
||||
@@ -36,7 +36,8 @@ use std::time::Duration;
|
||||
use tokio::sync::{mpsc, oneshot};
|
||||
|
||||
use codewhale_engine::session_manager::{
|
||||
OfflineQueueLease, OfflineQueueState, SavedSession, SessionManager,
|
||||
OfflineQueueLease, OfflineQueueState, SavedSession, SessionManager, SessionPublication,
|
||||
SessionPublicationKind as PublicationKind,
|
||||
};
|
||||
use codewhale_util::utils::spawn_supervised;
|
||||
|
||||
@@ -52,7 +53,7 @@ pub enum PersistRequest {
|
||||
SaveCheckpoint { session: SavedSession },
|
||||
AcknowledgedCheckpoint {
|
||||
session: SavedSession,
|
||||
reply: oneshot::Sender<Result<(), String>>,
|
||||
reply: oneshot::Sender<Result<SessionPublication, String>>,
|
||||
},
|
||||
/// Write a full session snapshot (completed turn, durable save).
|
||||
SessionSnapshot(SavedSession),
|
||||
@@ -86,6 +87,18 @@ pub enum PersistRequest {
|
||||
Shutdown,
|
||||
}
|
||||
|
||||
impl PersistRequest {
|
||||
fn receipt(&self) -> Option<codewhale_engine::session_manager::SessionWriteReceipt> {
|
||||
match self {
|
||||
Self::SaveCheckpoint { session }
|
||||
| Self::AcknowledgedCheckpoint { session, .. }
|
||||
| Self::SessionSnapshot(session)
|
||||
| Self::CompletedCommit { session } => session.write_receipt.clone(),
|
||||
_ => None,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Aggregated durability results: how many writes/removals completed and
|
||||
/// which failed (labelled by what was being persisted, with the I/O error
|
||||
/// kind).
|
||||
@@ -128,7 +141,7 @@ enum PendingOfflineQueue {
|
||||
|
||||
/// Control commands the actor reacts to. Work itself never crosses the
|
||||
/// channel: requests are coalesced into the shared [`PendingState`] at send
|
||||
/// time, so a paused consumer retains at most the latest request per session
|
||||
/// time, so a paused consumer retains one ordered barrier and one latest successor per session
|
||||
/// instead of every snapshot ever sent (#6212).
|
||||
enum ActorCommand {
|
||||
/// The shared pending state has work the actor has not taken yet.
|
||||
@@ -209,10 +222,10 @@ impl Drop for PersistRequestReceiver {
|
||||
.unwrap_or_else(|poisoned| poisoned.into_inner());
|
||||
guard.closed = true;
|
||||
guard.notified = false;
|
||||
for (_, reply) in std::mem::take(&mut guard.pending.checkpoint_replies) {
|
||||
let _ = reply.send(Err(
|
||||
"persistence actor shut down before checkpoint write".into()
|
||||
));
|
||||
for (_, slot) in std::mem::take(&mut guard.pending.publications) {
|
||||
for publication in slot.barrier.into_iter().chain(slot.latest) {
|
||||
publication.refuse("persistence actor shut down before publication");
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -266,9 +279,14 @@ impl PersistActorHandle {
|
||||
/// Queue a persistence request without blocking. The request is
|
||||
/// coalesced into the shared pending state immediately (latest-wins per
|
||||
/// session), so repeated snapshots of one session retain only the
|
||||
/// newest. Returns `false` when the actor is already shut down.
|
||||
/// newest compatible successor. Returns `false` when the actor is closed
|
||||
/// or the native predecessor cannot be retained in order.
|
||||
pub fn try_send(&self, mut request: PersistRequest) -> bool {
|
||||
let receipt = request.receipt();
|
||||
if self.tx.cmd_tx.is_closed() {
|
||||
if let Some(receipt) = receipt {
|
||||
receipt.refuse("persistence actor is closed");
|
||||
}
|
||||
return false;
|
||||
}
|
||||
match &mut request {
|
||||
@@ -287,11 +305,15 @@ impl PersistActorHandle {
|
||||
.lock()
|
||||
.unwrap_or_else(|poisoned| poisoned.into_inner());
|
||||
if guard.closed {
|
||||
if let Some(receipt) = receipt {
|
||||
receipt.refuse("persistence actor is closed");
|
||||
}
|
||||
return false;
|
||||
}
|
||||
guard.pending.absorb(request)
|
||||
};
|
||||
match control {
|
||||
Control::Refused => false,
|
||||
Control::Continue => {
|
||||
// A WorkReady the actor has not taken yet is already queued;
|
||||
// that take will observe this request too. `notified` resets
|
||||
@@ -308,18 +330,20 @@ impl PersistActorHandle {
|
||||
}
|
||||
guard.notified = true;
|
||||
}
|
||||
if self.tx.cmd_tx.try_send(ActorCommand::WorkReady).is_ok() {
|
||||
true
|
||||
} else {
|
||||
// Roll the flag back so later sends re-attempt (and
|
||||
// re-fail) honestly instead of riding a dead wake.
|
||||
let mut guard = self
|
||||
match self.tx.cmd_tx.try_send(ActorCommand::WorkReady) {
|
||||
Ok(()) | Err(mpsc::error::TrySendError::Full(_)) => true,
|
||||
Err(mpsc::error::TrySendError::Closed(_)) => {
|
||||
let mut guard = self
|
||||
.tx
|
||||
.shared
|
||||
.lock()
|
||||
.unwrap_or_else(|poisoned| poisoned.into_inner());
|
||||
.unwrap_or_else(std::sync::PoisonError::into_inner);
|
||||
guard.notified = false;
|
||||
false
|
||||
if let Some(receipt) = receipt {
|
||||
receipt.refuse("persistence actor is closed");
|
||||
}
|
||||
false
|
||||
}
|
||||
}
|
||||
}
|
||||
Control::Flush(reply) => self
|
||||
@@ -401,9 +425,13 @@ pub fn persist(request: PersistRequest) {
|
||||
|
||||
pub(crate) fn persist_checked(request: PersistRequest) -> Result<(), String> {
|
||||
let label = request_label(&request);
|
||||
let receipt = request.receipt();
|
||||
if try_persist(request) {
|
||||
return Ok(());
|
||||
}
|
||||
if let Some(error) = receipt.and_then(|receipt| receipt.observation().err()) {
|
||||
return Err(error.to_string());
|
||||
}
|
||||
if ACTOR_TX.get().is_some() {
|
||||
tracing::warn!(
|
||||
request = label,
|
||||
@@ -415,7 +443,7 @@ pub(crate) fn persist_checked(request: PersistRequest) -> Result<(), String> {
|
||||
request = label,
|
||||
"persistence request dropped: actor not initialised yet"
|
||||
);
|
||||
Ok(())
|
||||
Err("session persistence actor is not initialized".to_string())
|
||||
}
|
||||
}
|
||||
|
||||
@@ -474,9 +502,16 @@ pub(crate) fn session_save_health() -> Option<SaveHealthReading> {
|
||||
}
|
||||
|
||||
pub fn try_persist(request: PersistRequest) -> bool {
|
||||
ACTOR_TX
|
||||
.get()
|
||||
.is_some_and(|handle| handle.try_send(request))
|
||||
match ACTOR_TX
|
||||
.get() {
|
||||
Some(handle) => handle.try_send(request),
|
||||
None => {
|
||||
if let Some(receipt) = request.receipt() {
|
||||
receipt.refuse("session persistence actor is not initialized");
|
||||
}
|
||||
false
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
@@ -508,10 +543,20 @@ pub fn spawn_persistence_actor(
|
||||
health: &SessionSaveHealth,
|
||||
) {
|
||||
let session_ids: Vec<_> = pending
|
||||
.sessions
|
||||
.keys()
|
||||
.chain(pending.completed_commits.keys())
|
||||
.cloned()
|
||||
.publications
|
||||
.iter()
|
||||
.filter(|(_, slot)| {
|
||||
slot.barrier
|
||||
.iter()
|
||||
.chain(slot.latest.iter())
|
||||
.any(|publication| publication.kind != PublicationKind::Checkpoint)
|
||||
})
|
||||
.map(|(id, _)| id.clone())
|
||||
.collect();
|
||||
let receipts: Vec<_> = pending.publications
|
||||
.values()
|
||||
.flat_map(|slot| slot.barrier.iter().chain(slot.latest.iter()))
|
||||
.filter_map(|publication| publication.session.write_receipt.clone())
|
||||
.collect();
|
||||
let manager = manager.clone();
|
||||
let worker_health = health.clone();
|
||||
@@ -522,6 +567,9 @@ pub fn spawn_persistence_actor(
|
||||
match cycle {
|
||||
Ok(cycle) => unreported.merge(cycle),
|
||||
Err(error) => {
|
||||
for receipt in receipts {
|
||||
receipt.refuse(format!("owned persistence batch failed: {error}"));
|
||||
}
|
||||
tracing::warn!(target: "persistence", %error, "owned persistence batch failed");
|
||||
for session_id in session_ids {
|
||||
health.record::<()>(
|
||||
@@ -572,116 +620,203 @@ pub fn spawn_persistence_actor(
|
||||
}
|
||||
|
||||
#[derive(Debug)]
|
||||
enum PendingSession {
|
||||
Snapshot(SavedSession),
|
||||
Completion(SavedSession),
|
||||
struct PendingPublication {
|
||||
kind: PublicationKind,
|
||||
session: SavedSession,
|
||||
reply: Option<oneshot::Sender<Result<SessionPublication, String>>>,
|
||||
}
|
||||
|
||||
impl PendingPublication {
|
||||
fn refuse(self, error: &str) {
|
||||
if let Some(receipt) = self.session.write_receipt.as_ref() {
|
||||
receipt.refuse(error);
|
||||
}
|
||||
if let Some(reply) = self.reply {
|
||||
let _ = reply.send(Err(error.into()));
|
||||
}
|
||||
tracing::warn!(session_id = %self.session.metadata.id, %error, "session publication refused before queueing");
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Debug, Default)]
|
||||
struct PendingSlot {
|
||||
barrier: Option<PendingPublication>,
|
||||
latest: Option<PendingPublication>,
|
||||
}
|
||||
|
||||
/// Coalesced work waiting for the next write cycle.
|
||||
#[derive(Debug, Default)]
|
||||
struct PendingState {
|
||||
/// Latest-wins per session id. Crash checkpoints are keyed per session
|
||||
/// (mirroring `sessions` below) so concurrent sessions can interleave
|
||||
/// saves and clears without clobbering each other.
|
||||
checkpoints: BTreeMap<String, SavedSession>,
|
||||
checkpoint_replies: BTreeMap<String, oneshot::Sender<Result<(), String>>>,
|
||||
/// Latest-wins per session id. Coalescing into one global slot can
|
||||
/// drop session A when an immediate `/new` queues session B before
|
||||
/// the actor drains.
|
||||
sessions: BTreeMap<String, PendingSession>,
|
||||
/// Compound completion commits, latest-wins per session id: the
|
||||
/// completed snapshot body to write, followed by that session's own
|
||||
/// checkpoint clear — the clear only if the write succeeded. A newer
|
||||
/// snapshot can replace the body while retaining the clear intent until
|
||||
/// an explicit clear changes that ordering.
|
||||
completed_commits: BTreeMap<String, PendingSession>,
|
||||
/// Latest-wins per session id, for the same reason `sessions` above is:
|
||||
/// a single global slot dropped session A's queued text when session B
|
||||
/// queued before the actor drained, which defeats the per-session file
|
||||
/// naming entirely. Each pending request retains its editor lease, so a
|
||||
/// window changing session cannot release ownership ahead of its writes.
|
||||
publications: BTreeMap<String, PendingSlot>,
|
||||
offline_queue: BTreeMap<String, PendingOfflineQueue>,
|
||||
}
|
||||
|
||||
/// What the actor loop should do after absorbing a request.
|
||||
enum Control {
|
||||
Continue,
|
||||
Refused,
|
||||
Flush(oneshot::Sender<FlushReport>),
|
||||
Shutdown,
|
||||
}
|
||||
|
||||
impl PendingState {
|
||||
/// True when nothing coalesced is waiting. An empty `WorkReady` take is
|
||||
/// skipped instead of running a no-op flush cycle.
|
||||
fn is_empty(&self) -> bool {
|
||||
self.checkpoints.is_empty()
|
||||
&& self.sessions.is_empty()
|
||||
&& self.completed_commits.is_empty()
|
||||
self.publications.is_empty()
|
||||
&& self.offline_queue.is_empty()
|
||||
}
|
||||
|
||||
fn absorb(&mut self, req: PersistRequest) -> Control {
|
||||
let replaced = match &req {
|
||||
PersistRequest::SaveCheckpoint { session }
|
||||
| PersistRequest::AcknowledgedCheckpoint { session, .. }
|
||||
| PersistRequest::SessionSnapshot(session)
|
||||
| PersistRequest::CompletedCommit { session } => Some(session.metadata.id.as_str()),
|
||||
_ => None,
|
||||
};
|
||||
if let Some(id) = replaced
|
||||
&& let Some(reply) = self.checkpoint_replies.remove(id)
|
||||
fn absorb_publication(&mut self, publication: PendingPublication) -> Control {
|
||||
let id = publication.session.metadata.id.clone();
|
||||
if publication
|
||||
.session
|
||||
.write_receipt
|
||||
.as_ref()
|
||||
.and_then(|receipt| receipt.owner())
|
||||
.is_none()
|
||||
{
|
||||
let _ = reply.send(Err("checkpoint superseded before persistence".into()));
|
||||
publication.refuse("session publication lacks a captured native producer");
|
||||
return Control::Refused;
|
||||
}
|
||||
let slot = self.publications.entry(id).or_default();
|
||||
if slot.latest.is_none()
|
||||
&& let Some(barrier) = slot.barrier.as_ref()
|
||||
&& !publication
|
||||
.session
|
||||
.write_receipt
|
||||
.as_ref()
|
||||
.zip(barrier.session.write_receipt.as_ref())
|
||||
.is_some_and(|(current, prior)| current.is_successor_of(prior))
|
||||
{
|
||||
publication
|
||||
.refuse("session publication is not the captured successor of its pending barrier");
|
||||
return Control::Refused;
|
||||
}
|
||||
if let Some(previous) = slot.latest.take() {
|
||||
if previous.reply.is_some() {
|
||||
slot.latest = Some(previous);
|
||||
publication.refuse("an acknowledged checkpoint must publish before its successor");
|
||||
return Control::Refused;
|
||||
}
|
||||
let receipts = publication
|
||||
.session
|
||||
.write_receipt
|
||||
.as_ref()
|
||||
.zip(previous.session.write_receipt.as_ref());
|
||||
if !receipts
|
||||
.as_ref()
|
||||
.is_some_and(|(current, prior)| current.is_successor_of(prior))
|
||||
{
|
||||
slot.latest = Some(previous);
|
||||
publication
|
||||
.refuse("session publication is not the captured successor of pending work");
|
||||
return Control::Refused;
|
||||
}
|
||||
if previous.kind == PublicationKind::Completion {
|
||||
if slot.barrier.is_some() {
|
||||
slot.latest = Some(previous);
|
||||
publication.refuse(
|
||||
"completion publication is waiting on its checkpoint acknowledgement",
|
||||
);
|
||||
return Control::Refused;
|
||||
}
|
||||
slot.barrier = Some(previous);
|
||||
} else {
|
||||
let same_execution = receipts.as_ref().is_some_and(|(current, prior)| {
|
||||
current
|
||||
.owner()
|
||||
.zip(prior.owner())
|
||||
.is_some_and(|(current, prior)| current.same_execution(&prior))
|
||||
});
|
||||
let history_follows = same_execution
|
||||
|| match (
|
||||
publication.session.journal.as_ref(),
|
||||
previous.session.journal.as_ref(),
|
||||
) {
|
||||
(Some(current), Some(prior)) => {
|
||||
let current_index = current.index();
|
||||
let retained_graph = prior.entries.iter().all(|entry| {
|
||||
current_index
|
||||
.get(entry.id.as_str())
|
||||
.is_some_and(|current| **current == *entry)
|
||||
});
|
||||
let current_path = current.root_to_leaf();
|
||||
let prior_path = prior.root_to_leaf();
|
||||
retained_graph
|
||||
&& current_path.len() >= prior_path.len()
|
||||
&& prior_path
|
||||
.iter()
|
||||
.zip(current_path.iter())
|
||||
.all(|(prior, current)| prior == current)
|
||||
}
|
||||
(None, None) => publication
|
||||
.session
|
||||
.messages
|
||||
.starts_with(&previous.session.messages),
|
||||
_ => false,
|
||||
};
|
||||
let explicit_control = publication.session.write_receipt.as_ref().and_then(|receipt| receipt.owner())
|
||||
.is_some_and(|owner| matches!(owner, codewhale_engine::session_manager::SessionWriteOwner::Control(_))
|
||||
|| matches!(owner, codewhale_engine::session_manager::SessionWriteOwner::Engine(source) if source.compaction_id.is_some()));
|
||||
if !same_execution && !history_follows {
|
||||
if explicit_control && slot.barrier.is_none() {
|
||||
slot.barrier = Some(previous);
|
||||
slot.latest = Some(publication);
|
||||
return Control::Continue;
|
||||
}
|
||||
slot.latest = Some(previous);
|
||||
publication
|
||||
.refuse("pending publication history is incompatible with its successor");
|
||||
return Control::Refused;
|
||||
}
|
||||
if !receipts
|
||||
.is_some_and(|(current, prior)| current.coalesce_after(prior, previous.kind))
|
||||
{
|
||||
slot.latest = Some(previous);
|
||||
publication
|
||||
.refuse("pending publication could not preserve its captured predecessor");
|
||||
return Control::Refused;
|
||||
}
|
||||
tracing::debug!(session_id = %publication.session.metadata.id, previous = ?previous.kind, current = ?publication.kind, "coalesced native publication without acknowledging the predecessor");
|
||||
}
|
||||
}
|
||||
if publication.reply.is_some() && slot.barrier.is_none() {
|
||||
slot.barrier = Some(publication);
|
||||
} else {
|
||||
slot.latest = Some(publication);
|
||||
}
|
||||
Control::Continue
|
||||
}
|
||||
|
||||
fn absorb(&mut self, req: PersistRequest) -> Control {
|
||||
match req {
|
||||
PersistRequest::AcknowledgedCheckpoint { session, reply } => {
|
||||
let id = session.metadata.id.clone();
|
||||
self.absorb(PersistRequest::SaveCheckpoint { session });
|
||||
self.checkpoint_replies.insert(id, reply);
|
||||
if let Some(receipt) = session.write_receipt.as_ref() {
|
||||
receipt.require_checkpoint_acknowledgement();
|
||||
}
|
||||
self.absorb_publication(PendingPublication {
|
||||
kind: PublicationKind::Checkpoint,
|
||||
session,
|
||||
reply: Some(reply),
|
||||
})
|
||||
}
|
||||
PersistRequest::SaveCheckpoint { session } => {
|
||||
// Last-writer-wins per session: a fresh checkpoint supersedes
|
||||
// a pending clear for the same session so the two never both
|
||||
// apply in one drain (which previously cleared then re-wrote
|
||||
// the stale checkpoint, undoing the clear).
|
||||
let id = session.metadata.id.clone();
|
||||
// A new in-flight checkpoint means newer turn work started
|
||||
// after the completion it would have cleared: the compound's
|
||||
// clear-on-success intent is stale now (it would erase the
|
||||
// fresher recovery record), but the completed snapshot still
|
||||
// needs to be saved.
|
||||
if let Some(completed) = self.completed_commits.remove(&id) {
|
||||
tracing::debug!(
|
||||
session_id = %id,
|
||||
"pending completed commit retained beside a newer in-flight checkpoint"
|
||||
);
|
||||
self.sessions.insert(id.clone(), completed);
|
||||
}
|
||||
self.checkpoints.insert(id, session);
|
||||
self.absorb_publication(PendingPublication {
|
||||
kind: PublicationKind::Checkpoint,
|
||||
session,
|
||||
reply: None,
|
||||
})
|
||||
}
|
||||
PersistRequest::SessionSnapshot(session) => {
|
||||
// A newer full snapshot of a session with a pending
|
||||
// completed commit refreshes the commit's body (and keeps
|
||||
// its clear-on-success intent): the in-flight checkpoint it
|
||||
// guards is already captured by the newer completed state.
|
||||
let id = session.metadata.id.clone();
|
||||
if let Some(pending) = self.completed_commits.get_mut(&id) {
|
||||
*pending = PendingSession::Snapshot(session);
|
||||
} else {
|
||||
self.sessions.insert(id, PendingSession::Snapshot(session));
|
||||
}
|
||||
self.absorb_publication(PendingPublication {
|
||||
kind: PublicationKind::Snapshot,
|
||||
session,
|
||||
reply: None,
|
||||
})
|
||||
}
|
||||
PersistRequest::CompletedCommit { session } => {
|
||||
let id = session.metadata.id.clone();
|
||||
// The compound owns this session's snapshot and clear: a
|
||||
// pending plain snapshot is superseded, a pending standalone
|
||||
// clear would erase the recovery record even when the save
|
||||
// fails, and a pending checkpoint write would re-create the
|
||||
// record after the compound cleared it.
|
||||
self.sessions.remove(&id);
|
||||
self.checkpoints.remove(&id);
|
||||
self.completed_commits
|
||||
.insert(id, PendingSession::Completion(session));
|
||||
self.absorb_publication(PendingPublication {
|
||||
kind: PublicationKind::Completion,
|
||||
session,
|
||||
reply: None,
|
||||
})
|
||||
}
|
||||
PersistRequest::OfflineQueue { state, lease } => {
|
||||
self.offline_queue.insert(
|
||||
@@ -691,18 +826,18 @@ impl PendingState {
|
||||
lease,
|
||||
},
|
||||
);
|
||||
Control::Continue
|
||||
}
|
||||
PersistRequest::ClearOfflineQueue { lease } => {
|
||||
// A clear supersedes a pending save for its OWN session only.
|
||||
self.offline_queue.insert(
|
||||
lease.session_id().to_string(),
|
||||
PendingOfflineQueue::Clear { lease },
|
||||
);
|
||||
Control::Continue
|
||||
}
|
||||
PersistRequest::FlushAndReport { reply } => return Control::Flush(reply),
|
||||
PersistRequest::Shutdown => return Control::Shutdown,
|
||||
PersistRequest::FlushAndReport { reply } => Control::Flush(reply),
|
||||
PersistRequest::Shutdown => Control::Shutdown,
|
||||
}
|
||||
Control::Continue
|
||||
}
|
||||
}
|
||||
|
||||
@@ -744,41 +879,46 @@ fn flush_inner(
|
||||
}
|
||||
};
|
||||
|
||||
// Document saves are bounded (#6842): an oversized journal is archived
|
||||
// and pruned here, on the actor, never on the UI loop.
|
||||
for (session_id, pending_session) in std::mem::take(&mut pending.sessions) {
|
||||
let session = match pending_session {
|
||||
PendingSession::Snapshot(session) | PendingSession::Completion(session) => session,
|
||||
};
|
||||
let result = manager.save_session_bounded(session).map(|_| ());
|
||||
for (session_id, slot) in std::mem::take(&mut pending.publications) {
|
||||
for publication in slot.barrier.into_iter().chain(slot.latest) {
|
||||
let PendingPublication {
|
||||
kind,
|
||||
session,
|
||||
reply,
|
||||
} = publication;
|
||||
if reply.as_ref().is_some_and(oneshot::Sender::is_closed) {
|
||||
if let Some(receipt) = session.write_receipt.as_ref() {
|
||||
receipt.refuse("checkpoint acknowledgement was withdrawn before publication");
|
||||
}
|
||||
continue;
|
||||
}
|
||||
match kind {
|
||||
PublicationKind::Checkpoint => {
|
||||
let result = manager.publish_checkpoint_owned(session);
|
||||
if let Some(reply) = reply {
|
||||
let _ = reply.send(result.as_ref().cloned().map_err(ToString::to_string));
|
||||
}
|
||||
record(format!("checkpoint:{session_id}"), result.map(|_| ()));
|
||||
}
|
||||
PublicationKind::Snapshot => {
|
||||
let result = manager.publish_session_bounded(session);
|
||||
health.record(&session_id, &result);
|
||||
record(format!("session:{session_id}"), result);
|
||||
}
|
||||
for (session_id, pending_session) in std::mem::take(&mut pending.completed_commits) {
|
||||
let session = match pending_session {
|
||||
PendingSession::Snapshot(session) | PendingSession::Completion(session) => session,
|
||||
};
|
||||
let commit_result = manager.commit_completed_session(session);
|
||||
health.record(&session_id, &commit_result);
|
||||
match commit_result {
|
||||
Ok(clear) => {
|
||||
record(format!("completed-commit:{session_id}"), Ok(()));
|
||||
record(format!("clear-checkpoint:{session_id}"), clear);
|
||||
record(format!("session:{session_id}"), result.map(|_| ()));
|
||||
}
|
||||
PublicationKind::Completion => {
|
||||
let result = manager.publish_completed_session(session);
|
||||
health.record(&session_id, &result);
|
||||
match result {
|
||||
Ok((_, clear)) => {
|
||||
record(format!("completed-commit:{session_id}"), Ok(()));
|
||||
record(format!("clear-checkpoint:{session_id}"), clear);
|
||||
}
|
||||
Err(error) => record(format!("completed-commit:{session_id}"), Err(error)),
|
||||
}
|
||||
}
|
||||
}
|
||||
Err(error) => record(format!("completed-commit:{session_id}"), Err(error)),
|
||||
}
|
||||
}
|
||||
for (session_id, session) in std::mem::take(&mut pending.checkpoints) {
|
||||
let reply = pending.checkpoint_replies.remove(&session_id);
|
||||
if reply.as_ref().is_some_and(oneshot::Sender::is_closed) {
|
||||
continue;
|
||||
}
|
||||
let result = manager.save_checkpoint_owned(session).map(|_| ());
|
||||
if let Some(reply) = reply {
|
||||
let _ = reply.send(result.as_ref().map(|_| ()).map_err(ToString::to_string));
|
||||
}
|
||||
record(format!("checkpoint:{session_id}"), result);
|
||||
}
|
||||
for (_, request) in std::mem::take(&mut pending.offline_queue) {
|
||||
match request {
|
||||
PendingOfflineQueue::Save { state, lease } => record(
|
||||
@@ -1016,7 +1156,7 @@ mod tests {
|
||||
let sessions_dir = tmp.path().join("sessions");
|
||||
let manager = SessionManager::new(sessions_dir.clone()).expect("manager");
|
||||
let verification_manager = SessionManager::new(sessions_dir).expect("verification manager");
|
||||
let session = codewhale_engine::session_manager::create_saved_session_with_mode(
|
||||
let mut session = codewhale_engine::session_manager::create_saved_session_with_mode(
|
||||
&[],
|
||||
"deepseek-v4-pro",
|
||||
tmp.path(),
|
||||
@@ -1024,6 +1164,14 @@ mod tests {
|
||||
None,
|
||||
Some("agent"),
|
||||
);
|
||||
session.write_receipt = Some(
|
||||
manager
|
||||
.expect_absent_session(&session.metadata.id)
|
||||
.expect("native fresh source"),
|
||||
);
|
||||
session
|
||||
.capture_publication(codewhale_engine::session_manager::SessionWriteOwner::control())
|
||||
.expect("native producer");
|
||||
let (tx, mut receiver) = persistence_request_channel();
|
||||
let retained = PersistActorHandle { tx };
|
||||
assert!(retained.try_send(PersistRequest::Shutdown));
|
||||
@@ -1046,6 +1194,10 @@ mod tests {
|
||||
}));
|
||||
assert!(after_shutdown.await.is_err());
|
||||
let session_id = session.metadata.id.clone();
|
||||
session.write_receipt = Some(manager.expect_absent_session(&session_id).unwrap());
|
||||
session
|
||||
.capture_publication(codewhale_engine::session_manager::SessionWriteOwner::control())
|
||||
.unwrap();
|
||||
let (handle, task) = spawn_persistence_actor(manager);
|
||||
|
||||
handle.try_send(PersistRequest::SessionSnapshot(session));
|
||||
@@ -1072,6 +1224,14 @@ mod tests {
|
||||
None,
|
||||
Some("agent"),
|
||||
);
|
||||
first.write_receipt = Some(
|
||||
manager
|
||||
.expect_absent_session(&first.metadata.id)
|
||||
.expect("native fresh source"),
|
||||
);
|
||||
first
|
||||
.capture_publication(codewhale_engine::session_manager::SessionWriteOwner::control())
|
||||
.expect("native producer");
|
||||
first.metadata.title = "Session A".to_string();
|
||||
let mut second = codewhale_engine::session_manager::create_saved_session_with_mode(
|
||||
&[],
|
||||
@@ -1081,6 +1241,14 @@ mod tests {
|
||||
None,
|
||||
Some("agent"),
|
||||
);
|
||||
second.write_receipt = Some(
|
||||
manager
|
||||
.expect_absent_session(&second.metadata.id)
|
||||
.expect("native fresh source"),
|
||||
);
|
||||
second
|
||||
.capture_publication(codewhale_engine::session_manager::SessionWriteOwner::control())
|
||||
.expect("native producer");
|
||||
second.metadata.title = "Session B".to_string();
|
||||
let first_id = first.metadata.id.clone();
|
||||
let second_id = second.metadata.id.clone();
|
||||
@@ -1115,7 +1283,7 @@ mod tests {
|
||||
let sessions_dir = tmp.path().join("sessions");
|
||||
let manager = SessionManager::new(sessions_dir.clone()).expect("manager");
|
||||
let verification_manager = SessionManager::new(sessions_dir).expect("verification manager");
|
||||
let session = codewhale_engine::session_manager::create_saved_session_with_mode(
|
||||
let mut session = codewhale_engine::session_manager::create_saved_session_with_mode(
|
||||
&[],
|
||||
"deepseek-v4-pro",
|
||||
tmp.path(),
|
||||
@@ -1123,32 +1291,64 @@ mod tests {
|
||||
None,
|
||||
Some("agent"),
|
||||
);
|
||||
for replacement in [
|
||||
PersistRequest::SaveCheckpoint {
|
||||
session: session.clone(),
|
||||
},
|
||||
PersistRequest::CompletedCommit {
|
||||
session: session.clone(),
|
||||
},
|
||||
] {
|
||||
session.write_receipt = Some(
|
||||
manager
|
||||
.expect_absent_session(&session.metadata.id)
|
||||
.expect("native fresh source"),
|
||||
);
|
||||
session
|
||||
.capture_publication(codewhale_engine::session_manager::SessionWriteOwner::control())
|
||||
.expect("native producer");
|
||||
for completion in [false, true] {
|
||||
let mut pending = PendingState::default();
|
||||
let (reply, receiver) = oneshot::channel();
|
||||
let mut checkpoint = session.clone();
|
||||
checkpoint.metadata.id = uuid::Uuid::new_v4().to_string();
|
||||
checkpoint.write_receipt = Some(
|
||||
manager
|
||||
.expect_absent_session(&checkpoint.metadata.id)
|
||||
.unwrap(),
|
||||
);
|
||||
checkpoint
|
||||
.capture_publication(
|
||||
codewhale_engine::session_manager::SessionWriteOwner::control(),
|
||||
)
|
||||
.unwrap();
|
||||
let mut successor = checkpoint.clone();
|
||||
successor
|
||||
.capture_publication(
|
||||
codewhale_engine::session_manager::SessionWriteOwner::control(),
|
||||
)
|
||||
.unwrap();
|
||||
let (reply, mut receiver) = oneshot::channel();
|
||||
pending.absorb(PersistRequest::AcknowledgedCheckpoint {
|
||||
session: session.clone(),
|
||||
session: checkpoint,
|
||||
reply,
|
||||
});
|
||||
pending.absorb(replacement);
|
||||
assert!(receiver.await.expect("supersession reply").is_err());
|
||||
assert!(pending.checkpoint_replies.is_empty());
|
||||
}
|
||||
let (handle, task) = spawn_persistence_actor(manager);
|
||||
pending.absorb(if completion {
|
||||
PersistRequest::CompletedCommit { session: successor }
|
||||
} else {
|
||||
PersistRequest::SaveCheckpoint { session: successor }
|
||||
});
|
||||
assert!(matches!(
|
||||
receiver.try_recv(),
|
||||
Err(oneshot::error::TryRecvError::Empty)
|
||||
));
|
||||
let report = flush_inner(&manager, &mut pending, &SessionSaveHealth::default());
|
||||
assert!(report.failures.is_empty(), "{report:?}");
|
||||
|
||||
assert!(handle.try_send(PersistRequest::SessionSnapshot(session.clone())));
|
||||
assert!(receiver.await.expect("real publication reply").is_ok());
|
||||
}
|
||||
let mut snapshot = session.clone();
|
||||
snapshot
|
||||
.capture_publication(codewhale_engine::session_manager::SessionWriteOwner::control())
|
||||
.unwrap();
|
||||
let (handle, task) = spawn_persistence_actor(manager);
|
||||
let (checkpoint_reply, checkpoint_result) = oneshot::channel();
|
||||
assert!(handle.try_send(PersistRequest::AcknowledgedCheckpoint {
|
||||
session: session.clone(),
|
||||
reply: checkpoint_reply
|
||||
reply: checkpoint_reply,
|
||||
}));
|
||||
assert!(handle.try_send(PersistRequest::SessionSnapshot(snapshot)));
|
||||
let report = handle
|
||||
.flush(Duration::from_secs(2))
|
||||
.await
|
||||
@@ -1182,7 +1382,7 @@ mod tests {
|
||||
let manager = SessionManager::new(sessions_dir.clone()).expect("manager");
|
||||
let checkpoint_blocker = sessions_dir.join("checkpoints");
|
||||
std::fs::write(&checkpoint_blocker, b"not a directory").expect("block checkpoints dir");
|
||||
let session = codewhale_engine::session_manager::create_saved_session_with_mode(
|
||||
let mut session = codewhale_engine::session_manager::create_saved_session_with_mode(
|
||||
&[],
|
||||
"deepseek-v4-pro",
|
||||
tmp.path(),
|
||||
@@ -1190,6 +1390,14 @@ mod tests {
|
||||
None,
|
||||
Some("agent"),
|
||||
);
|
||||
session.write_receipt = Some(
|
||||
manager
|
||||
.expect_absent_session(&session.metadata.id)
|
||||
.expect("native fresh source"),
|
||||
);
|
||||
session
|
||||
.capture_publication(codewhale_engine::session_manager::SessionWriteOwner::control())
|
||||
.expect("native producer");
|
||||
let session_id = session.metadata.id.clone();
|
||||
// A directory where the document goes makes its save fail.
|
||||
let blocker = sessions_dir.join(format!("{session_id}.json"));
|
||||
@@ -1246,7 +1454,7 @@ mod tests {
|
||||
// checkpoint write deterministically fails on all platforms.
|
||||
std::fs::write(sessions_dir.join("checkpoints"), b"not a directory")
|
||||
.expect("block checkpoints dir");
|
||||
let session = codewhale_engine::session_manager::create_saved_session_with_mode(
|
||||
let mut session = codewhale_engine::session_manager::create_saved_session_with_mode(
|
||||
&[],
|
||||
"deepseek-v4-pro",
|
||||
tmp.path(),
|
||||
@@ -1254,6 +1462,14 @@ mod tests {
|
||||
None,
|
||||
Some("agent"),
|
||||
);
|
||||
session.write_receipt = Some(
|
||||
manager
|
||||
.expect_absent_session(&session.metadata.id)
|
||||
.expect("native fresh source"),
|
||||
);
|
||||
session
|
||||
.capture_publication(codewhale_engine::session_manager::SessionWriteOwner::control())
|
||||
.expect("native producer");
|
||||
let session_id = session.metadata.id.clone();
|
||||
let (handle, task) = spawn_persistence_actor(manager);
|
||||
|
||||
@@ -1286,9 +1502,9 @@ mod tests {
|
||||
fn seed_checkpoint_file(
|
||||
sessions_dir: &std::path::Path,
|
||||
session_id: &str,
|
||||
) -> std::path::PathBuf {
|
||||
) -> SessionPublication {
|
||||
let manager = SessionManager::new(sessions_dir.to_path_buf()).expect("checkpoint manager");
|
||||
let session = codewhale_engine::session_manager::create_saved_session_with_id_and_mode(
|
||||
let mut session = codewhale_engine::session_manager::create_saved_session_with_id_and_mode(
|
||||
session_id.to_string(),
|
||||
&[],
|
||||
"deepseek-v4-pro",
|
||||
@@ -1297,8 +1513,13 @@ mod tests {
|
||||
None,
|
||||
Some("agent"),
|
||||
);
|
||||
session.write_receipt = Some(manager
|
||||
.expect_absent_session(session_id).unwrap());
|
||||
session
|
||||
.capture_publication(codewhale_engine::session_manager::SessionWriteOwner::control())
|
||||
.unwrap();
|
||||
manager
|
||||
.save_checkpoint_owned(session)
|
||||
.publish_checkpoint_owned(session)
|
||||
.expect("seed checkpoint file")
|
||||
}
|
||||
|
||||
@@ -1315,7 +1536,7 @@ mod tests {
|
||||
let tmp = tempfile::tempdir().expect("tempdir");
|
||||
let sessions_dir = tmp.path().join("sessions");
|
||||
let manager = SessionManager::new(sessions_dir.clone()).expect("manager");
|
||||
let session = codewhale_engine::session_manager::create_saved_session_with_mode(
|
||||
let mut session = codewhale_engine::session_manager::create_saved_session_with_mode(
|
||||
&[],
|
||||
"deepseek-v4-pro",
|
||||
tmp.path(),
|
||||
@@ -1323,8 +1544,21 @@ mod tests {
|
||||
None,
|
||||
Some("agent"),
|
||||
);
|
||||
session.write_receipt = Some(
|
||||
manager
|
||||
.expect_absent_session(&session.metadata.id)
|
||||
.expect("native fresh source"),
|
||||
);
|
||||
session
|
||||
.capture_publication(codewhale_engine::session_manager::SessionWriteOwner::control())
|
||||
.expect("native producer");
|
||||
let session_id = session.metadata.id.clone();
|
||||
let checkpoint_path = seed_checkpoint_file(&sessions_dir, &session_id);
|
||||
let checkpoint_publication = seed_checkpoint_file(&sessions_dir, &session_id);
|
||||
let checkpoint_path = checkpoint_publication.path.clone();
|
||||
session.write_receipt = Some(checkpoint_publication.write_receipt());
|
||||
session
|
||||
.capture_publication(codewhale_engine::session_manager::SessionWriteOwner::control())
|
||||
.expect("checkpoint successor");
|
||||
block_session_file(&sessions_dir, &session_id);
|
||||
let (handle, task) = spawn_persistence_actor(manager);
|
||||
|
||||
@@ -1364,7 +1598,7 @@ mod tests {
|
||||
let sessions_dir = tmp.path().join("sessions");
|
||||
let manager = SessionManager::new(sessions_dir.clone()).expect("manager");
|
||||
let verification_manager = SessionManager::new(sessions_dir.clone()).expect("verify");
|
||||
let session = codewhale_engine::session_manager::create_saved_session_with_mode(
|
||||
let mut session = codewhale_engine::session_manager::create_saved_session_with_mode(
|
||||
&[],
|
||||
"deepseek-v4-pro",
|
||||
tmp.path(),
|
||||
@@ -1372,8 +1606,21 @@ mod tests {
|
||||
None,
|
||||
Some("agent"),
|
||||
);
|
||||
session.write_receipt = Some(
|
||||
manager
|
||||
.expect_absent_session(&session.metadata.id)
|
||||
.expect("native fresh source"),
|
||||
);
|
||||
session
|
||||
.capture_publication(codewhale_engine::session_manager::SessionWriteOwner::control())
|
||||
.expect("native producer");
|
||||
let session_id = session.metadata.id.clone();
|
||||
let checkpoint_path = seed_checkpoint_file(&sessions_dir, &session_id);
|
||||
let checkpoint_publication = seed_checkpoint_file(&sessions_dir, &session_id);
|
||||
let checkpoint_path = checkpoint_publication.path.clone();
|
||||
session.write_receipt = Some(checkpoint_publication.write_receipt());
|
||||
session
|
||||
.capture_publication(codewhale_engine::session_manager::SessionWriteOwner::control())
|
||||
.expect("checkpoint successor");
|
||||
let (handle, task) = spawn_persistence_actor(manager);
|
||||
|
||||
handle.try_send(PersistRequest::CompletedCommit { session });
|
||||
@@ -1404,7 +1651,7 @@ mod tests {
|
||||
let sessions_dir = tmp.path().join("sessions");
|
||||
let manager = SessionManager::new(sessions_dir.clone()).expect("manager");
|
||||
let verification_manager = SessionManager::new(sessions_dir.clone()).expect("verify");
|
||||
let committed = codewhale_engine::session_manager::create_saved_session_with_mode(
|
||||
let mut committed = codewhale_engine::session_manager::create_saved_session_with_mode(
|
||||
&[],
|
||||
"deepseek-v4-pro",
|
||||
tmp.path(),
|
||||
@@ -1412,7 +1659,15 @@ mod tests {
|
||||
None,
|
||||
Some("agent"),
|
||||
);
|
||||
let inflight = codewhale_engine::session_manager::create_saved_session_with_mode(
|
||||
committed.write_receipt = Some(
|
||||
manager
|
||||
.expect_absent_session(&committed.metadata.id)
|
||||
.expect("native fresh source"),
|
||||
);
|
||||
committed
|
||||
.capture_publication(codewhale_engine::session_manager::SessionWriteOwner::control())
|
||||
.expect("native producer");
|
||||
let mut inflight = codewhale_engine::session_manager::create_saved_session_with_mode(
|
||||
&[],
|
||||
"deepseek-v4-pro",
|
||||
tmp.path(),
|
||||
@@ -1420,9 +1675,22 @@ mod tests {
|
||||
None,
|
||||
Some("agent"),
|
||||
);
|
||||
inflight.write_receipt = Some(
|
||||
manager
|
||||
.expect_absent_session(&inflight.metadata.id)
|
||||
.expect("native fresh source"),
|
||||
);
|
||||
inflight
|
||||
.capture_publication(codewhale_engine::session_manager::SessionWriteOwner::control())
|
||||
.expect("native producer");
|
||||
let committed_id = committed.metadata.id.clone();
|
||||
let inflight_id = inflight.metadata.id.clone();
|
||||
let committed_checkpoint = seed_checkpoint_file(&sessions_dir, &committed_id);
|
||||
let checkpoint_publication = seed_checkpoint_file(&sessions_dir, &committed_id);
|
||||
let committed_checkpoint = checkpoint_publication.path.clone();
|
||||
committed.write_receipt = Some(checkpoint_publication.write_receipt());
|
||||
committed
|
||||
.capture_publication(codewhale_engine::session_manager::SessionWriteOwner::control())
|
||||
.expect("checkpoint successor");
|
||||
// The concurrent session's checkpoint must carry a loadable body too.
|
||||
let inflight_checkpoint = verification_manager
|
||||
.save_checkpoint(&inflight)
|
||||
@@ -1455,7 +1723,7 @@ mod tests {
|
||||
let sessions_dir = tmp.path().join("sessions");
|
||||
let manager = SessionManager::new(sessions_dir.clone()).expect("manager");
|
||||
let verification_manager = SessionManager::new(sessions_dir.clone()).expect("verify");
|
||||
let session = codewhale_engine::session_manager::create_saved_session_with_mode(
|
||||
let mut session = codewhale_engine::session_manager::create_saved_session_with_mode(
|
||||
&[],
|
||||
"deepseek-v4-pro",
|
||||
tmp.path(),
|
||||
@@ -1463,8 +1731,21 @@ mod tests {
|
||||
None,
|
||||
Some("agent"),
|
||||
);
|
||||
session.write_receipt = Some(
|
||||
manager
|
||||
.expect_absent_session(&session.metadata.id)
|
||||
.expect("native fresh source"),
|
||||
);
|
||||
session
|
||||
.capture_publication(codewhale_engine::session_manager::SessionWriteOwner::control())
|
||||
.expect("native producer");
|
||||
let session_id = session.metadata.id.clone();
|
||||
let checkpoint_path = seed_checkpoint_file(&sessions_dir, &session_id);
|
||||
let checkpoint_publication = seed_checkpoint_file(&sessions_dir, &session_id);
|
||||
let checkpoint_path = checkpoint_publication.path.clone();
|
||||
session.write_receipt = Some(checkpoint_publication.write_receipt());
|
||||
session
|
||||
.capture_publication(codewhale_engine::session_manager::SessionWriteOwner::control())
|
||||
.expect("checkpoint successor");
|
||||
let (handle, task) = spawn_persistence_actor(manager);
|
||||
|
||||
handle.try_send(PersistRequest::SaveCheckpoint { session });
|
||||
@@ -1495,6 +1776,14 @@ mod tests {
|
||||
None,
|
||||
Some("agent"),
|
||||
);
|
||||
completed.write_receipt = Some(
|
||||
manager
|
||||
.expect_absent_session(&completed.metadata.id)
|
||||
.expect("native fresh source"),
|
||||
);
|
||||
completed
|
||||
.capture_publication(codewhale_engine::session_manager::SessionWriteOwner::control())
|
||||
.expect("native producer");
|
||||
completed.metadata.title = "Completed turn".to_string();
|
||||
completed.metadata.total_tokens = 7;
|
||||
let mut inflight = codewhale_engine::session_manager::create_saved_session_with_mode(
|
||||
@@ -1505,11 +1794,27 @@ mod tests {
|
||||
None,
|
||||
Some("agent"),
|
||||
);
|
||||
inflight.write_receipt = Some(
|
||||
manager
|
||||
.expect_absent_session(&inflight.metadata.id)
|
||||
.expect("native fresh source"),
|
||||
);
|
||||
inflight
|
||||
.capture_publication(codewhale_engine::session_manager::SessionWriteOwner::control())
|
||||
.expect("native producer");
|
||||
// The new turn reuses the same session id: a fresh in-flight
|
||||
// checkpoint arrives while the previous completion is still queued.
|
||||
inflight.metadata.id = completed.metadata.id.clone();
|
||||
inflight.metadata.title = "Next turn in progress".to_string();
|
||||
inflight.metadata.total_tokens = 11;
|
||||
inflight.write_receipt = completed.write_receipt.clone();
|
||||
inflight
|
||||
.capture_publication(
|
||||
codewhale_engine::session_manager::SessionWriteOwner::Submission(
|
||||
uuid::Uuid::new_v4().to_string(),
|
||||
),
|
||||
)
|
||||
.expect("next native submission");
|
||||
let session_id = completed.metadata.id.clone();
|
||||
let verification_manager = SessionManager::new(sessions_dir.clone()).expect("manager");
|
||||
let (handle, task) = spawn_persistence_actor(manager);
|
||||
|
||||
@@ -147,27 +147,24 @@ fn saved_session_version(session: &SavedSession) -> Option<usize> {
|
||||
fn drain_retained_requests(receiver: &mut PersistRequestReceiver) -> (usize, usize, Option<usize>) {
|
||||
let pending = receiver.take_pending();
|
||||
assert!(
|
||||
pending.checkpoints.is_empty()
|
||||
&& pending.completed_commits.is_empty()
|
||||
&& pending.offline_queue.is_empty(),
|
||||
pending.offline_queue.is_empty(),
|
||||
"backlog fixture retained an unexpected request variant"
|
||||
);
|
||||
let mut retained_queued_requests = 0;
|
||||
let mut estimated_retained_payload_bytes = 0;
|
||||
for session in pending.sessions.values() {
|
||||
let (PendingSession::Snapshot(session) | PendingSession::Completion(session)) = session;
|
||||
for slot in pending.publications.values() {
|
||||
for publication in slot.barrier.iter().chain(slot.latest.iter()) {
|
||||
assert_eq!(publication.kind, PublicationKind::Snapshot);
|
||||
retained_queued_requests += 1;
|
||||
estimated_retained_payload_bytes += retained_session_payload_bytes(session);
|
||||
estimated_retained_payload_bytes += retained_session_payload_bytes(&publication.session);
|
||||
}
|
||||
}
|
||||
let applied_version =
|
||||
pending
|
||||
.sessions
|
||||
.get(BACKLOG_SESSION_ID)
|
||||
.and_then(|session| match session {
|
||||
PendingSession::Snapshot(session) | PendingSession::Completion(session) => {
|
||||
saved_session_version(session)
|
||||
}
|
||||
});
|
||||
.publications
|
||||
.get(BACKLOG_SESSION_ID)
|
||||
.and_then(|slot| slot.latest.as_ref().or(slot.barrier.as_ref()))
|
||||
.and_then(|publication| saved_session_version(&publication.session));
|
||||
(
|
||||
retained_queued_requests,
|
||||
estimated_retained_payload_bytes,
|
||||
@@ -185,12 +182,23 @@ fn measure_paused_persistence_backlog() -> PersistenceBacklogObservation {
|
||||
|
||||
let (tx, mut receiver) = persistence_request_channel();
|
||||
let handle = PersistActorHandle { tx };
|
||||
let manager = SessionManager::default_location().expect("measurement session store");
|
||||
let mut source = manager.expect_absent_session(BACKLOG_SESSION_ID).unwrap();
|
||||
let mut journal: Option<codewhale_runtime::session_tree::SessionJournal> = None;
|
||||
let rss_before_bytes = current_process_rss_bytes();
|
||||
let mut accepted_requests = 0;
|
||||
let mut enqueue_elapsed_ns = 0;
|
||||
|
||||
for index in 0..BACKLOG_REQUESTS_ATTEMPTED {
|
||||
let session = backlog_session(tmp.path(), index);
|
||||
let mut session = backlog_session(tmp.path(), index);
|
||||
if let Some(previous) = journal.as_ref() {
|
||||
session.journal = Some(previous.clone());
|
||||
}
|
||||
journal = session.journal.clone();
|
||||
session.write_receipt = Some(source);
|
||||
source = session
|
||||
.capture_publication(codewhale_engine::session_manager::SessionWriteOwner::control())
|
||||
.unwrap();
|
||||
let started = Instant::now();
|
||||
let accepted = handle.try_send(PersistRequest::SessionSnapshot(session));
|
||||
enqueue_elapsed_ns += started.elapsed().as_nanos();
|
||||
@@ -288,17 +296,20 @@ fn applied_version_follows_production_drain_order_instead_of_numeric_maximum() {
|
||||
let tmp = tempfile::tempdir().expect("isolated measurement workspace");
|
||||
let (tx, mut receiver) = persistence_request_channel();
|
||||
let handle = PersistActorHandle { tx };
|
||||
assert!(
|
||||
handle.try_send(PersistRequest::SessionSnapshot(backlog_session(
|
||||
let manager = SessionManager::new(tmp.path().join("sessions")).unwrap();
|
||||
let mut first = backlog_session(
|
||||
tmp.path(),
|
||||
BACKLOG_EXPECTED_APPLIED_VERSION,
|
||||
)))
|
||||
);
|
||||
assert!(
|
||||
handle.try_send(PersistRequest::SessionSnapshot(backlog_session(
|
||||
tmp.path(),
|
||||
BACKLOG_EXPECTED_APPLIED_VERSION - 1,
|
||||
)))
|
||||
BACKLOG_EXPECTED_APPLIED_VERSION);
|
||||
first.write_receipt = Some(manager.expect_absent_session(BACKLOG_SESSION_ID).unwrap());
|
||||
first
|
||||
.capture_publication(codewhale_engine::session_manager::SessionWriteOwner::control())
|
||||
.unwrap();
|
||||
let mut last = first.clone();
|
||||
last.metadata.total_tokens = (BACKLOG_EXPECTED_APPLIED_VERSION - 1) as u64;
|
||||
last.capture_publication(codewhale_engine::session_manager::SessionWriteOwner::control())
|
||||
.unwrap();
|
||||
assert!(handle.try_send(PersistRequest::SessionSnapshot(first)));
|
||||
assert!(handle.try_send(PersistRequest::SessionSnapshot(last))
|
||||
);
|
||||
|
||||
let (_, _, applied_version) = drain_retained_requests(&mut receiver);
|
||||
|
||||
@@ -868,6 +868,7 @@ mod tests {
|
||||
observe(
|
||||
&mut app,
|
||||
&Event::TurnStarted {
|
||||
previous_source: None,
|
||||
turn_id: "preview-turn".into(),
|
||||
created_at: chrono::Utc::now(),
|
||||
route: None,
|
||||
@@ -953,6 +954,7 @@ mod tests {
|
||||
observe(
|
||||
&mut app,
|
||||
&Event::TurnStarted {
|
||||
previous_source: None,
|
||||
turn_id: "turn".into(),
|
||||
created_at: chrono::Utc::now(),
|
||||
route: None,
|
||||
@@ -1038,6 +1040,7 @@ mod tests {
|
||||
observe(
|
||||
&mut app,
|
||||
&Event::TurnStarted {
|
||||
previous_source: None,
|
||||
turn_id: "turn".into(),
|
||||
created_at: chrono::Utc::now(),
|
||||
route: None,
|
||||
@@ -1066,6 +1069,7 @@ mod tests {
|
||||
|
||||
watch.observe(
|
||||
&Event::TurnStarted {
|
||||
previous_source: None,
|
||||
turn_id: "turn-b".into(),
|
||||
created_at: chrono::Utc::now(),
|
||||
route: None,
|
||||
|
||||
@@ -729,6 +729,13 @@ impl SessionPickerView {
|
||||
} else {
|
||||
!session.archived
|
||||
};
|
||||
if self.current_session_id.as_deref() == Some(session.id.as_str())
|
||||
&& let Err(error) = crate::tui::persistence_actor::flush_before_transition()
|
||||
{
|
||||
self.status =
|
||||
Some(tr(self.locale, MessageId::SessionsArchiveFailed).replace("{error}", &error));
|
||||
return ViewAction::None;
|
||||
}
|
||||
let metadata = match manager.set_session_archived(
|
||||
&session.id,
|
||||
archived,
|
||||
@@ -804,6 +811,13 @@ impl SessionPickerView {
|
||||
return ViewAction::None;
|
||||
}
|
||||
};
|
||||
if self.current_session_id.as_deref() == Some(session.id.as_str())
|
||||
&& let Err(error) = crate::tui::persistence_actor::flush_before_transition()
|
||||
{
|
||||
self.status =
|
||||
Some(tr(self.locale, MessageId::SessionsRenameFailed).replace("{error}", &error));
|
||||
return ViewAction::None;
|
||||
}
|
||||
let metadata = match manager.rename_session(
|
||||
&session.id,
|
||||
new_title,
|
||||
@@ -1824,6 +1838,9 @@ mod tests {
|
||||
|
||||
fn test_session(idx: usize, title: &str) -> SessionMetadata {
|
||||
SessionMetadata {
|
||||
publication_id: None,
|
||||
native_mutation_observation: None,
|
||||
|
||||
id: format!("session-{idx:02}"),
|
||||
title: title.to_string(),
|
||||
title_is_custom: false,
|
||||
|
||||
@@ -565,9 +565,10 @@ async fn spawn_tui_engine_with_session(
|
||||
) -> Result<EngineHandle> {
|
||||
let handle = spawn_tui_engine(app, config, mcp_handoff).await;
|
||||
let restored = async {
|
||||
if !app.api_messages.is_empty() {
|
||||
{
|
||||
handle
|
||||
.send(Op::SyncSession {
|
||||
control: app.session_sync_control(),
|
||||
session_id: app.current_session_id.clone(),
|
||||
messages: app.api_messages.as_ref().clone(),
|
||||
system_prompt: app.system_prompt.clone(),
|
||||
|
||||
@@ -173,6 +173,7 @@ pub(crate) fn apply_picker_session_rename_to_active_app(
|
||||
if app.current_session_id.as_deref() != Some(metadata.id.as_str()) {
|
||||
return false;
|
||||
}
|
||||
app.adopt_session_mutation(&metadata);
|
||||
app.session_title = Some(metadata.title.clone());
|
||||
app.current_session_metadata = Some(metadata);
|
||||
true
|
||||
@@ -459,7 +460,10 @@ fn persist_accepted_goal_state(
|
||||
if app.current_session_id.is_none() {
|
||||
let session = build_session_snapshot(app, &manager)?;
|
||||
let session_id = session.metadata.id.clone();
|
||||
if !persistence_actor::try_persist(PersistRequest::SaveCheckpoint { session }) {
|
||||
if !super::session_state::try_persist_session(
|
||||
app,
|
||||
PersistRequest::SaveCheckpoint { session },
|
||||
) {
|
||||
return Err("the persistence worker is unavailable".to_string());
|
||||
}
|
||||
app.set_current_session_id(Some(session_id));
|
||||
@@ -1311,6 +1315,7 @@ pub(crate) async fn apply_provider_fallback_switch(
|
||||
if !app.api_messages.is_empty() {
|
||||
let _ = engine_handle
|
||||
.send(Op::SyncSession {
|
||||
control: app.session_sync_control(),
|
||||
session_id: app.current_session_id.clone(),
|
||||
messages: app.api_messages.as_ref().clone(),
|
||||
system_prompt: app.system_prompt.clone(),
|
||||
@@ -1408,6 +1413,9 @@ pub(super) async fn apply_conversation_undo(
|
||||
&& !app.remote_control.runtime_chat_blocks_local_dispatch(),
|
||||
"wait for the active turn to finish before undoing its conversation"
|
||||
);
|
||||
let control = codewhale_engine::core::ops::SessionControlId::new();
|
||||
let captured_source = app.session_write_receipt.clone();
|
||||
let captured_owner = app.session_write_owner.clone();
|
||||
let backtrack = !matches!(target, ConversationRewind::Command(_));
|
||||
let before = if backtrack {
|
||||
let (tx, receive) = tokio::sync::oneshot::channel();
|
||||
@@ -1433,7 +1441,11 @@ pub(super) async fn apply_conversation_undo(
|
||||
&& before.mode == app.mode.as_setting()
|
||||
&& codewhale_engine::compaction::strip_compaction_summaries(app.system_prompt.as_ref())
|
||||
== before_prompt
|
||||
&& before.messages.as_slice() == app.api_messages.as_slice(),
|
||||
&& before.messages.as_slice() == app.api_messages.as_slice()
|
||||
&& captured_owner
|
||||
.as_ref()
|
||||
.zip(before.source.as_ref())
|
||||
.is_some_and(|(owner, source)| owner.matches_source(source)),
|
||||
"{}",
|
||||
changed()
|
||||
);
|
||||
@@ -1469,6 +1481,7 @@ pub(super) async fn apply_conversation_undo(
|
||||
codewhale_engine::runtime_handoff::project_owned_messages_for_restore(messages.clone());
|
||||
let (tx, receive) = tokio::sync::oneshot::channel();
|
||||
let rewind = Op::RewindConversation {
|
||||
control: control.clone(),
|
||||
expected: Box::new(before),
|
||||
messages,
|
||||
tx,
|
||||
@@ -1487,7 +1500,11 @@ pub(super) async fn apply_conversation_undo(
|
||||
)
|
||||
})?;
|
||||
anyhow::ensure!(
|
||||
installed.session_id == id && installed.messages == expected,
|
||||
installed.session_id == id && installed.messages == expected
|
||||
&& installed.source.as_ref()
|
||||
== Some(
|
||||
&codewhale_engine::core::ops::SessionSnapshotSource::control(control.clone())
|
||||
),
|
||||
"the Engine did not acknowledge the conversation rollback"
|
||||
);
|
||||
if let Some(depth) = backtrack_depth {
|
||||
@@ -1501,6 +1518,9 @@ pub(super) async fn apply_conversation_undo(
|
||||
}
|
||||
}
|
||||
}
|
||||
app.session_write_owner =
|
||||
Some(codewhale_engine::session_manager::SessionWriteOwner::Control(control));
|
||||
app.session_write_receipt = captured_source;
|
||||
app.set_api_messages(Arc::new(installed.messages));
|
||||
app.set_current_session_id(Some(id));
|
||||
app.tool_cells.clear();
|
||||
@@ -1512,10 +1532,10 @@ pub(super) async fn apply_conversation_undo(
|
||||
let manager = tokio::task::spawn_blocking(SessionManager::default_location).await??;
|
||||
let session = build_session_snapshot(app, &manager).map_err(anyhow::Error::msg)?;
|
||||
let projection = saved_work_projection(app, &session);
|
||||
// CompletedCommit supersedes an older queued checkpoint as well as its
|
||||
// snapshot. A plain snapshot could let crash recovery revive the old turn.
|
||||
anyhow::ensure!(
|
||||
persistence_actor::try_persist(PersistRequest::CompletedCommit { session }),
|
||||
super::session_state::try_persist_session(
|
||||
app,
|
||||
PersistRequest::CompletedCommit { session }),
|
||||
"the session persistence worker is unavailable"
|
||||
);
|
||||
let (reply, receive) = tokio::sync::oneshot::channel();
|
||||
@@ -1801,6 +1821,7 @@ async fn apply_command_result_inner(
|
||||
}
|
||||
let _ = engine_handle
|
||||
.send(Op::SyncSession {
|
||||
control: app.session_sync_control(),
|
||||
session_id: app.current_session_id.clone(),
|
||||
messages: app.api_messages.as_ref().clone(),
|
||||
system_prompt: app.system_prompt.clone(),
|
||||
@@ -1913,6 +1934,7 @@ async fn apply_command_result_inner(
|
||||
.await;
|
||||
let _ = engine_handle
|
||||
.send(Op::SyncSession {
|
||||
control: app.session_sync_control(),
|
||||
session_id,
|
||||
messages,
|
||||
system_prompt,
|
||||
@@ -2030,6 +2052,7 @@ async fn apply_command_result_inner(
|
||||
if !app.api_messages.is_empty() {
|
||||
let _ = engine_handle
|
||||
.send(Op::SyncSession {
|
||||
control: app.session_sync_control(),
|
||||
session_id: app.current_session_id.clone(),
|
||||
messages: app.api_messages.as_ref().clone(),
|
||||
system_prompt: app.system_prompt.clone(),
|
||||
@@ -3257,6 +3280,7 @@ async fn apply_command_result_inner(
|
||||
if !app.api_messages.is_empty() {
|
||||
let _ = engine_handle
|
||||
.send(Op::SyncSession {
|
||||
control: app.session_sync_control(),
|
||||
session_id: app.current_session_id.clone(),
|
||||
messages: app.api_messages.as_ref().clone(),
|
||||
system_prompt: app.system_prompt.clone(),
|
||||
@@ -4727,12 +4751,16 @@ pub(crate) fn apply_loaded_session_with_goal(
|
||||
// is contended, the current conversation stays intact and a retry can
|
||||
// use this durably repaired binding to the same host.
|
||||
let mut recovered = session.clone();
|
||||
recovered
|
||||
.capture_publication(codewhale_engine::session_manager::SessionWriteOwner::control())
|
||||
.map_err(|error| error.to_string())?;
|
||||
let abandoned = recovered.metadata.runtime_store.replace(binding.clone());
|
||||
let manager = SessionManager::default_location()
|
||||
.map_err(|error| format!("Session recovery could not be saved: {error}"))?;
|
||||
manager
|
||||
.save_session(&recovered)
|
||||
let publication = manager
|
||||
.publish_session(&recovered)
|
||||
.map_err(|error| format!("Session recovery could not be saved: {error}"))?;
|
||||
session.adopt_publication(&publication);
|
||||
// The conversation now lives in this host's store. The empty store it
|
||||
// left is set aside here, where it is abandoned, unless another
|
||||
// document still binds it (#6144 P1a) — otherwise it stayed on disk
|
||||
@@ -4961,6 +4989,11 @@ pub(crate) fn apply_loaded_session_with_goal(
|
||||
app.cumulative_turn_duration =
|
||||
std::time::Duration::from_secs(session.metadata.cumulative_turn_secs);
|
||||
app.set_current_session_id(Some(session.metadata.id.clone()));
|
||||
app.session_write_receipt = session.write_receipt.clone();
|
||||
app.session_write_owner = session
|
||||
.write_receipt
|
||||
.as_ref()
|
||||
.map(|_| codewhale_engine::session_manager::SessionWriteOwner::control());
|
||||
app.session_title = Some(session.metadata.title.clone());
|
||||
app.current_session_metadata = Some(session.metadata);
|
||||
reset_approval_scope_for_new_conversation(app);
|
||||
|
||||
@@ -674,6 +674,7 @@ pub(crate) fn prepare_user_dispatch(
|
||||
// Capture the App state before any optimistic mutation so a failure can
|
||||
// roll back cleanly.
|
||||
let snapshot = UserDispatchSnapshot {
|
||||
session_write_owner: app.session_write_owner.clone(),
|
||||
is_loading: app.is_loading,
|
||||
suppress_stream_events_until_turn_complete: app.suppress_stream_events_until_turn_complete,
|
||||
runtime_turn_status: app.runtime_turn_status.clone(),
|
||||
@@ -722,6 +723,7 @@ pub(crate) fn prepare_user_dispatch(
|
||||
let allowed_tools = allowed_tools_for_message(app.active_allowed_tools.clone(), &message);
|
||||
|
||||
Ok(UserDispatchPrepare {
|
||||
submission_id: uuid::Uuid::new_v4().to_string(),
|
||||
message,
|
||||
content,
|
||||
references,
|
||||
@@ -1023,8 +1025,7 @@ pub(crate) async fn spawned_dispatch_inner(
|
||||
hook_executor: prepare.hook_executor.clone(),
|
||||
verbosity: prepare.verbosity.clone(),
|
||||
provenance: prepare.provenance,
|
||||
// Interactive TUI submissions do not correlate submissions.
|
||||
submission_id: None,
|
||||
submission_id: Some(prepare.submission_id.clone()),
|
||||
});
|
||||
// Reserve capacity off the render thread, but do not let Engine start
|
||||
// until the completion callback has installed the UI's acceptance state.
|
||||
@@ -1096,6 +1097,11 @@ pub(crate) fn build_dispatch_success_closure(
|
||||
move |app: &mut App, engine_handle: &EngineHandle, config: &Config| -> anyhow::Result<()> {
|
||||
app.dispatch_in_flight = false;
|
||||
prepare.paused_dispatch.apply(app, engine_handle);
|
||||
app.session_write_owner = Some(
|
||||
codewhale_engine::session_manager::SessionWriteOwner::Submission(
|
||||
prepare.submission_id.clone(),
|
||||
),
|
||||
);
|
||||
|
||||
let dispatch_started_at = Instant::now();
|
||||
app.is_loading = true;
|
||||
@@ -1251,6 +1257,7 @@ pub(crate) fn build_dispatch_error_closure(
|
||||
}
|
||||
app.remote_control.fail_active_dispatch(&error);
|
||||
// Roll back the optimistic sync prepare mutations.
|
||||
app.session_write_owner = prepare.snapshot.session_write_owner.clone();
|
||||
app.is_loading = prepare.snapshot.is_loading;
|
||||
app.runtime_turn_status = prepare.snapshot.runtime_turn_status.clone();
|
||||
app.receipt_text = prepare.snapshot.receipt_text.clone();
|
||||
|
||||
@@ -222,6 +222,7 @@ pub(crate) enum DispatchRecovery {
|
||||
/// dispatch can roll back the optimistic history/api_messages changes.
|
||||
#[derive(Debug, Clone)]
|
||||
pub(crate) struct UserDispatchSnapshot {
|
||||
pub(crate) session_write_owner: Option<codewhale_engine::session_manager::SessionWriteOwner>,
|
||||
pub(crate) is_loading: bool,
|
||||
pub(crate) suppress_stream_events_until_turn_complete: bool,
|
||||
pub(crate) runtime_turn_status: Option<String>,
|
||||
@@ -239,6 +240,7 @@ pub(crate) struct UserDispatchSnapshot {
|
||||
#[allow(clippy::struct_excessive_bools)]
|
||||
#[derive(Debug, Clone)]
|
||||
pub(crate) struct UserDispatchPrepare {
|
||||
pub(super) submission_id: String,
|
||||
pub(super) message: QueuedMessage,
|
||||
pub(super) content: String,
|
||||
pub(super) references: Vec<ContextReference>,
|
||||
|
||||
@@ -128,10 +128,21 @@ pub(super) fn apply_engine_session_projection(
|
||||
model,
|
||||
workspace,
|
||||
checkpoint,
|
||||
source,
|
||||
} = event
|
||||
else {
|
||||
return false;
|
||||
};
|
||||
if !app.owns_session_projection(source.as_ref()) {
|
||||
if let Some(request) = checkpoint.as_ref() {
|
||||
request.fail("checkpoint projection belongs to another native source");
|
||||
}
|
||||
tracing::debug!(
|
||||
?source,
|
||||
"session projection did not match its admitted native owner"
|
||||
);
|
||||
return false;
|
||||
}
|
||||
if let Some(request) = checkpoint.as_ref() {
|
||||
if request.is_stale()
|
||||
|| request.caller.agent_id.is_some()
|
||||
@@ -222,10 +233,12 @@ pub(super) fn apply_engine_session_projection(
|
||||
return false;
|
||||
}
|
||||
let (reply, receiver) = tokio::sync::oneshot::channel();
|
||||
if !crate::tui::persistence_actor::try_persist(PersistRequest::AcknowledgedCheckpoint {
|
||||
if !super::session_state::try_persist_session(
|
||||
app,
|
||||
PersistRequest::AcknowledgedCheckpoint {
|
||||
session,
|
||||
reply,
|
||||
}) {
|
||||
reply },
|
||||
) {
|
||||
request.fail("terminal persistence actor is unavailable");
|
||||
return false;
|
||||
}
|
||||
@@ -259,11 +272,7 @@ pub(super) fn apply_engine_session_projection(
|
||||
biased;
|
||||
() = request.withdraw.cancelled() => request.fail("checkpoint withdrawn; a write already in progress may have completed"),
|
||||
result = receiver => match result {
|
||||
Ok(Ok(())) => request.succeed(codewhale_engine::core::events::SessionCheckpointReceipt {
|
||||
persisted_sha256: request.prefix_sha256.clone(),
|
||||
messages_len: request.snapshot.messages.len(),
|
||||
projection: codewhale_engine::core::events::SessionCheckpointProjection::Exact,
|
||||
}),
|
||||
Ok(Ok(publication)) => request.succeed(publication.checkpoint_receipt),
|
||||
Ok(Err(error)) => request.fail(error),
|
||||
Err(_) => request.fail("terminal persistence actor dropped checkpoint acknowledgement"),
|
||||
},
|
||||
@@ -325,8 +334,8 @@ fn turn_unsettled_for_shutdown(app: &App) -> bool {
|
||||
}
|
||||
|
||||
/// Host state can change without a model turn, including learning the Runtime
|
||||
/// binding of a resumed legacy session. Commit that state before clearing its
|
||||
/// recovery checkpoint; an unfinished turn keeps its checkpoint untouched.
|
||||
/// binding of a resumed legacy session. Save that state through its captured
|
||||
/// source; the shutdown snapshot retains its recovery checkpoint.
|
||||
pub(super) fn persist_settled_session_on_shutdown(
|
||||
app: &mut App,
|
||||
handle: &persistence_actor::PersistActorHandle,
|
||||
@@ -336,7 +345,9 @@ pub(super) fn persist_settled_session_on_shutdown(
|
||||
}
|
||||
let manager = SessionManager::default_location().map_err(|error| error.to_string())?;
|
||||
let session = build_session_snapshot(app, &manager)?;
|
||||
if !handle.try_send(PersistRequest::CompletedCommit { session }) {
|
||||
let mut request = PersistRequest::SessionSnapshot(session);
|
||||
super::session_state::prepare_session_publication(app, &mut request)?;
|
||||
if !handle.try_send(request) {
|
||||
return Err("persistence actor is unavailable during shutdown".into());
|
||||
}
|
||||
Ok(true)
|
||||
@@ -547,6 +558,9 @@ pub(crate) fn ensure_runtime_session_id(app: &mut App) -> String {
|
||||
}
|
||||
let session_id = uuid::Uuid::new_v4().to_string();
|
||||
app.set_current_session_id(Some(session_id.clone()));
|
||||
if let Err(error) = super::session_state::expect_fresh_session(app, &session_id) {
|
||||
tracing::warn!(%error, "new session publication source could not be established");
|
||||
}
|
||||
session_id
|
||||
}
|
||||
|
||||
@@ -905,6 +919,23 @@ pub(crate) fn coalesce_resize_burst(
|
||||
/// Toast identity for a failing session save, so recovery can retire it.
|
||||
const SESSION_SAVE_FAILURE_TOAST: &str = "session-save-failure";
|
||||
|
||||
pub(crate) fn surface_session_save_failure(app: &mut App, session_id: Option<&str>, error: &str) {
|
||||
let text = app
|
||||
.tr(MessageId::SessionSaveFailed)
|
||||
.replace(
|
||||
"{id}",
|
||||
session_id
|
||||
.map(codewhale_engine::session_manager::truncate_id)
|
||||
.unwrap_or("—"),
|
||||
)
|
||||
.replace("{error}", error);
|
||||
app.push_status_toast_record(StatusToast::standing(
|
||||
text,
|
||||
StatusToastLevel::Error,
|
||||
SESSION_SAVE_FAILURE_TOAST,
|
||||
));
|
||||
}
|
||||
|
||||
/// Keep the save-failure notice in step with the persistence actor's session
|
||||
/// save health. Saves run off the UI thread, so a full disk or an unwritable
|
||||
/// sessions directory used to reach only the log while the user kept working
|
||||
@@ -921,20 +952,9 @@ pub(crate) fn surface_session_save_health(
|
||||
*seen = reading.generation;
|
||||
app.retire_event_notices(SESSION_SAVE_FAILURE_TOAST);
|
||||
if let Some((session_id, kind)) = reading.failing {
|
||||
let text = app
|
||||
.tr(MessageId::SessionSaveFailed)
|
||||
.replace(
|
||||
"{id}",
|
||||
codewhale_engine::session_manager::truncate_id(&session_id),
|
||||
)
|
||||
.replace("{error}", &kind.to_string());
|
||||
// Standing, not timed: it must outlast the failure, and only the
|
||||
// recovery reading above withdraws it.
|
||||
app.push_status_toast_record(StatusToast::standing(
|
||||
text,
|
||||
StatusToastLevel::Error,
|
||||
SESSION_SAVE_FAILURE_TOAST,
|
||||
));
|
||||
surface_session_save_failure(app, Some(&session_id), &kind.to_string());
|
||||
}
|
||||
}
|
||||
|
||||
@@ -2254,6 +2274,11 @@ pub(crate) async fn run_event_loop(
|
||||
() = drain => None,
|
||||
};
|
||||
if let Some(snapshot) = snapshot
|
||||
&& snapshot.source.as_ref().is_some_and(|source| {
|
||||
app.session_write_owner
|
||||
.as_ref()
|
||||
.is_some_and(|owner| owner.matches_source(source))
|
||||
})
|
||||
&& !acknowledge.is_closed()
|
||||
&& apply_recovery_messages(
|
||||
app,
|
||||
@@ -2717,7 +2742,10 @@ pub(crate) async fn run_event_loop(
|
||||
apply_engine_session_projection(app, config, event);
|
||||
continue;
|
||||
}
|
||||
capture_turn_started_metadata(app, &event);
|
||||
if !capture_turn_started_metadata(app, &event) {
|
||||
tracing::debug!("ignored a start from another native source");
|
||||
continue;
|
||||
}
|
||||
observe_human_request_settlement(app, &event);
|
||||
// Child approval bookkeeping runs before every filter: it is
|
||||
// keyed by approval id and agent, not by the active session,
|
||||
@@ -2740,8 +2768,10 @@ pub(crate) async fn run_event_loop(
|
||||
workspace,
|
||||
messages,
|
||||
checkpoint: None,
|
||||
source,
|
||||
..
|
||||
} = &event
|
||||
&& app.owns_session_projection(source.as_ref())
|
||||
&& messages.len().checked_sub(app.api_messages.len()) == Some(1)
|
||||
&& messages
|
||||
.last()
|
||||
@@ -3356,7 +3386,9 @@ pub(crate) async fn run_event_loop(
|
||||
}
|
||||
}
|
||||
EngineEvent::TurnComplete {
|
||||
turn_id: _,
|
||||
turn_id,
|
||||
submission_id,
|
||||
compaction_id,
|
||||
usage,
|
||||
parent_route_usage,
|
||||
routed_usage_dropped_records,
|
||||
@@ -3366,6 +3398,18 @@ pub(crate) async fn run_event_loop(
|
||||
base_url,
|
||||
..
|
||||
} => {
|
||||
let terminal_source =
|
||||
codewhale_engine::core::ops::SessionSnapshotSource::terminal(
|
||||
turn_id.clone(),
|
||||
submission_id.clone(),
|
||||
compaction_id.clone(),
|
||||
);
|
||||
let terminal_owner_matches =
|
||||
terminal_source.as_ref().is_some_and(|source| {
|
||||
app.session_write_owner
|
||||
.as_ref()
|
||||
.is_some_and(|owner| owner.matches_source(source))
|
||||
});
|
||||
// A decision whose tool never reported completion
|
||||
// still gets its receipt before the turn closes.
|
||||
if flush_gate_receipts_for(app, None) {
|
||||
@@ -3376,16 +3420,6 @@ pub(crate) async fn run_event_loop(
|
||||
// (#6190).
|
||||
crate::tui::ui::dispatch::settle_unaccepted_steers_at_turn_end(app);
|
||||
let completed_turn = app.active_turn.take();
|
||||
// TUI dispatches carry no submission id, so
|
||||
// TurnComplete cannot name the dispatch it closes.
|
||||
// After a local cancel the UI is idle, so `is_loading` can only be
|
||||
// true again because a newer dispatch set it: this
|
||||
// terminal event belongs to the cancelled turn and
|
||||
// must not close the newer one's loading, dispatch,
|
||||
// status or unanswered-submission state, nor drain
|
||||
// the queue ahead of it (U02-07). The engine runs one
|
||||
// turn at a time, so the newer turn's own TurnStarted
|
||||
// and TurnComplete follow this event.
|
||||
let newer_dispatch_owns_turn_state =
|
||||
app.suppress_stream_events_until_turn_complete && app.is_loading;
|
||||
if !newer_dispatch_owns_turn_state {
|
||||
@@ -3815,7 +3849,47 @@ pub(crate) async fn run_event_loop(
|
||||
// Auto-save completed turn and clear crash checkpoint.
|
||||
// Offloaded to the persistence actor so the UI
|
||||
// stays responsive.
|
||||
if let Ok(manager) = SessionManager::default_location()
|
||||
let mut snapshot_error = None;
|
||||
let terminal_snapshot = if terminal_owner_matches
|
||||
&& !newer_dispatch_owns_turn_state
|
||||
{
|
||||
let snapshot = match engine_handle.get_session_snapshot().await {
|
||||
Ok(snapshot) => Some(snapshot),
|
||||
Err(error) => {
|
||||
tracing::warn!(%error, "terminal session snapshot could not be read; checkpoint retained");
|
||||
snapshot_error = Some(error.to_string());
|
||||
None
|
||||
}
|
||||
};
|
||||
snapshot.filter(|snapshot| {
|
||||
snapshot.source.as_ref() == terminal_source.as_ref()
|
||||
&& app.session_write_owner.as_ref().is_some_and(|owner| {
|
||||
snapshot
|
||||
.source
|
||||
.as_ref()
|
||||
.is_some_and(|source| owner.matches_source(source))
|
||||
})
|
||||
&& app.current_session_id.as_deref()
|
||||
== Some(snapshot.session_id.as_str())
|
||||
&& snapshot.workspace == app.workspace
|
||||
&& snapshot.messages.as_slice() == app.api_messages.as_slice()
|
||||
})
|
||||
} else {
|
||||
None
|
||||
};
|
||||
if terminal_owner_matches && terminal_snapshot.is_none() {
|
||||
tracing::warn!(
|
||||
?terminal_source,
|
||||
"terminal session source no longer matches; checkpoint retained"
|
||||
);
|
||||
let session_id = app.current_session_id.clone();
|
||||
let error = snapshot_error.unwrap_or_else(|| {
|
||||
std::io::ErrorKind::ResourceBusy.to_string()
|
||||
});
|
||||
surface_session_save_failure(app, session_id.as_deref(), &error);
|
||||
}
|
||||
if terminal_snapshot.is_some()
|
||||
&& let Ok(manager) = SessionManager::default_location()
|
||||
&& let Ok(session) = build_session_snapshot(app, &manager)
|
||||
{
|
||||
let projection = saved_work_projection(app, &session);
|
||||
@@ -3825,10 +3899,11 @@ pub(crate) async fn run_event_loop(
|
||||
// crash checkpoint only after that write succeeds.
|
||||
// A failed save now retains the checkpoint as the
|
||||
// sole recovery record instead of erasing it.
|
||||
let queued =
|
||||
persistence_actor::try_persist(PersistRequest::CompletedCommit {
|
||||
session,
|
||||
});
|
||||
let queued = super::session_state::try_persist_session(
|
||||
app,
|
||||
PersistRequest::CompletedCommit {
|
||||
session },
|
||||
);
|
||||
if queued {
|
||||
if let Err(err) =
|
||||
publish_saved_work_projection(app, &projection).await
|
||||
@@ -5035,6 +5110,7 @@ pub(crate) async fn run_event_loop(
|
||||
if !app.api_messages.is_empty() {
|
||||
let _ = engine_handle
|
||||
.send(Op::SyncSession {
|
||||
control: app.session_sync_control(),
|
||||
session_id: app.current_session_id.clone(),
|
||||
messages: app.api_messages.as_ref().clone(),
|
||||
system_prompt: app.system_prompt.clone(),
|
||||
@@ -5430,7 +5506,14 @@ pub(crate) async fn run_event_loop(
|
||||
&& let Ok(session) = build_session_snapshot(app, &manager)
|
||||
{
|
||||
app.set_current_session_id(Some(session.metadata.id.clone()));
|
||||
persistence_actor::persist(PersistRequest::SessionSnapshot(session));
|
||||
if !super::session_state::try_persist_session(
|
||||
app,
|
||||
PersistRequest::SessionSnapshot(session),
|
||||
) {
|
||||
tracing::warn!(
|
||||
"late usage snapshot was not queued; canonical usage evidence retained"
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
// Drain completed file-tree walks (initial build / expands) so the
|
||||
@@ -8786,6 +8869,7 @@ mod launch_session_sync_tests {
|
||||
|
||||
handle
|
||||
.send(Op::SyncSession {
|
||||
control: codewhale_engine::core::ops::SessionControlId::new(),
|
||||
session_id: Some("launch-session".to_string()),
|
||||
messages: Vec::new(),
|
||||
system_prompt: None,
|
||||
|
||||
@@ -1299,6 +1299,15 @@ pub(crate) fn build_session_snapshot(
|
||||
app.system_prompt.as_ref(),
|
||||
Some(app.mode.as_setting()),
|
||||
);
|
||||
session.write_receipt = if app.current_session_id.is_none() {
|
||||
Some(
|
||||
manager
|
||||
.expect_absent_session(&session.metadata.id)
|
||||
.map_err(|error| error.to_string())?,
|
||||
)
|
||||
} else {
|
||||
app.session_write_receipt.clone()
|
||||
};
|
||||
let computed_title = session.metadata.title.clone();
|
||||
if let Some(cached) = app
|
||||
.current_session_metadata
|
||||
|
||||
@@ -2100,6 +2100,7 @@ pub(crate) async fn handle_view_events(
|
||||
}
|
||||
let _ = engine_handle
|
||||
.send(Op::SyncSession {
|
||||
control: app.session_sync_control(),
|
||||
session_id: app.current_session_id.clone(),
|
||||
messages: app.api_messages.as_ref().clone(),
|
||||
system_prompt: app.system_prompt.clone(),
|
||||
@@ -2183,6 +2184,7 @@ pub(crate) async fn handle_view_events(
|
||||
}));
|
||||
}
|
||||
ViewEvent::SessionArchived { metadata } => {
|
||||
app.adopt_session_mutation(&metadata);
|
||||
// The manager already wrote the flag. Keep the active app's
|
||||
// cached metadata in step so the next autosave carries the new
|
||||
// state forward instead of reverting it, and drop the rail
|
||||
|
||||
@@ -908,6 +908,7 @@ async fn switch_provider_inner(
|
||||
if !app.api_messages.is_empty() {
|
||||
let _ = engine_handle
|
||||
.send(Op::SyncSession {
|
||||
control: app.session_sync_control(),
|
||||
session_id: app.current_session_id.clone(),
|
||||
messages: app.api_messages.as_ref().clone(),
|
||||
system_prompt: app.system_prompt.clone(),
|
||||
|
||||
@@ -127,6 +127,64 @@ pub(crate) async fn publish_pending_work_projection(app: &mut App) -> Result<boo
|
||||
work.publish_pending().await
|
||||
}
|
||||
|
||||
pub(crate) fn prepare_session_publication(
|
||||
app: &mut App,
|
||||
request: &mut PersistRequest,
|
||||
) -> Result<(), String> {
|
||||
if app.dispatch_in_flight {
|
||||
return Err("session publication is waiting for native dispatch admission".to_string());
|
||||
}
|
||||
let session = match request {
|
||||
PersistRequest::SaveCheckpoint { session }
|
||||
| PersistRequest::AcknowledgedCheckpoint { session, .. }
|
||||
| PersistRequest::SessionSnapshot(session)
|
||||
| PersistRequest::CompletedCommit { session } => session,
|
||||
_ => return Ok(()),
|
||||
};
|
||||
let owner = app
|
||||
.session_write_owner
|
||||
.clone()
|
||||
.ok_or_else(|| "session publication lacks an admitted native owner".to_string())?;
|
||||
let receipt = session
|
||||
.capture_publication(owner)
|
||||
.map_err(|error| error.to_string())?;
|
||||
if app
|
||||
.current_session_id
|
||||
.as_deref()
|
||||
.is_none_or(|id| id == session.metadata.id)
|
||||
{
|
||||
app.session_write_receipt = Some(receipt);
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
pub(crate) fn try_persist_session(app: &mut App, mut request: PersistRequest) -> bool {
|
||||
if let Err(error) = prepare_session_publication(app, &mut request) {
|
||||
tracing::warn!(%error, "session publication could not capture its native source");
|
||||
let session_id = match &request {
|
||||
PersistRequest::SaveCheckpoint { session }
|
||||
| PersistRequest::AcknowledgedCheckpoint { session, .. }
|
||||
| PersistRequest::SessionSnapshot(session)
|
||||
| PersistRequest::CompletedCommit { session } => Some(session.metadata.id.as_str()),
|
||||
_ => None,
|
||||
};
|
||||
super::event_loop::surface_session_save_failure(app, session_id, &error);
|
||||
return false;
|
||||
}
|
||||
persistence_actor::try_persist(request)
|
||||
}
|
||||
|
||||
pub(crate) fn expect_fresh_session(app: &mut App, id: &str) -> Result<(), String> {
|
||||
let manager = SessionManager::default_location().map_err(|error| error.to_string())?;
|
||||
app.session_write_receipt = Some(
|
||||
manager
|
||||
.expect_absent_session(id)
|
||||
.map_err(|error| error.to_string())?,
|
||||
);
|
||||
app.session_write_owner = Some(codewhale_engine::session_manager::SessionWriteOwner::control());
|
||||
Ok(())
|
||||
}
|
||||
|
||||
pub(crate) async fn persist_pending_work_checkpoint(app: &mut App) -> Result<bool, String> {
|
||||
let Some(work) = app.runtime_services.work.clone() else {
|
||||
return Ok(false);
|
||||
@@ -141,7 +199,7 @@ pub(crate) async fn persist_pending_work_checkpoint(app: &mut App) -> Result<boo
|
||||
if app.current_session_id.is_none() {
|
||||
app.set_current_session_id(Some(session.metadata.id.clone()));
|
||||
}
|
||||
if !persistence_actor::try_persist(PersistRequest::SaveCheckpoint { session }) {
|
||||
if !try_persist_session(app, PersistRequest::SaveCheckpoint { session }) {
|
||||
return Err("persistence actor is unavailable".to_string());
|
||||
}
|
||||
publish_saved_work_projection(app, &projection).await
|
||||
@@ -149,9 +207,10 @@ pub(crate) async fn persist_pending_work_checkpoint(app: &mut App) -> Result<boo
|
||||
|
||||
pub(crate) fn persist_with_pending_work_boundary(
|
||||
app: &mut App,
|
||||
request: PersistRequest,
|
||||
mut request: PersistRequest,
|
||||
projection: &SavedWorkProjection,
|
||||
) -> Result<(), String> {
|
||||
prepare_session_publication(app, &mut request)?;
|
||||
let has_pending = app
|
||||
.runtime_services
|
||||
.work
|
||||
@@ -692,14 +751,36 @@ pub(crate) fn recover_engine_event_disconnect(app: &mut App) -> bool {
|
||||
true
|
||||
}
|
||||
|
||||
pub(crate) fn capture_turn_started_metadata(app: &mut App, event: &EngineEvent) {
|
||||
pub(crate) fn capture_turn_started_metadata(app: &mut App, event: &EngineEvent) -> bool {
|
||||
match event {
|
||||
EngineEvent::TurnStarted {
|
||||
previous_source,
|
||||
turn_id,
|
||||
created_at,
|
||||
route,
|
||||
submission_id: _,
|
||||
submission_id,
|
||||
} => {
|
||||
let source = codewhale_engine::core::ops::SessionSnapshotSource::turn(
|
||||
turn_id.clone(),
|
||||
submission_id.clone(),
|
||||
);
|
||||
let admitted = match app.session_write_owner.as_ref() {
|
||||
Some(codewhale_engine::session_manager::SessionWriteOwner::Submission(id)) => {
|
||||
submission_id.as_ref() == Some(id)
|
||||
}
|
||||
Some(owner) => {
|
||||
owner.matches_source(&source)
|
||||
|| previous_source
|
||||
.as_ref()
|
||||
.is_some_and(|previous| owner.matches_source(previous))
|
||||
}
|
||||
None => true,
|
||||
};
|
||||
if !admitted {
|
||||
return false;
|
||||
}
|
||||
app.session_write_owner =
|
||||
Some(codewhale_engine::session_manager::SessionWriteOwner::Engine(source));
|
||||
app.ocean_completion_started_at = None;
|
||||
let auto_route_receipt = if route.as_ref().is_some_and(|route| route.auto_model) {
|
||||
app.pending_auto_route_receipt.take()
|
||||
@@ -731,14 +812,18 @@ pub(crate) fn capture_turn_started_metadata(app: &mut App, event: &EngineEvent)
|
||||
// already taken at `TurnStarted` are preserved — this event narrows the
|
||||
// route, it never re-opens an authority decision.
|
||||
EngineEvent::RouteDispatched { turn_id, route } => {
|
||||
let newer_submission = matches!(
|
||||
app.session_write_owner.as_ref(),
|
||||
Some(codewhale_engine::session_manager::SessionWriteOwner::Submission(_))
|
||||
);
|
||||
if let Some(active) = app
|
||||
.active_turn
|
||||
.as_mut()
|
||||
.filter(|active| active.turn_id == *turn_id)
|
||||
{
|
||||
if route.auto_model && active.auto_route_receipt.is_none() {
|
||||
if !newer_submission && route.auto_model && active.auto_route_receipt.is_none() {
|
||||
active.auto_route_receipt = app.pending_auto_route_receipt.take();
|
||||
} else if !route.auto_model {
|
||||
} else if !newer_submission && !route.auto_model {
|
||||
app.pending_auto_route_receipt = None;
|
||||
active.auto_route_receipt = None;
|
||||
}
|
||||
@@ -749,8 +834,28 @@ pub(crate) fn capture_turn_started_metadata(app: &mut App, event: &EngineEvent)
|
||||
active.route = Some(route.clone());
|
||||
}
|
||||
}
|
||||
EngineEvent::CompactionStarted {
|
||||
id,
|
||||
previous_source,
|
||||
auto: false,
|
||||
..
|
||||
} => {
|
||||
let source = codewhale_engine::core::ops::SessionSnapshotSource::compaction(id.clone());
|
||||
let admitted = app.session_write_owner.as_ref().is_none_or(|owner| {
|
||||
owner.matches_source(&source)
|
||||
|| previous_source
|
||||
.as_ref()
|
||||
.is_some_and(|previous| owner.matches_source(previous))
|
||||
});
|
||||
if !admitted {
|
||||
return false;
|
||||
}
|
||||
app.session_write_owner =
|
||||
Some(codewhale_engine::session_manager::SessionWriteOwner::Engine(source));
|
||||
}
|
||||
_ => {}
|
||||
}
|
||||
true
|
||||
}
|
||||
|
||||
pub(crate) fn record_turn_activity(app: &mut App, event: &EngineEvent, now: Instant) {
|
||||
@@ -984,6 +1089,9 @@ pub(crate) fn begin_launch_session(
|
||||
app.workspace = workspace;
|
||||
}
|
||||
app.set_current_session_id(Some(session_id.clone()));
|
||||
if let Err(error) = expect_fresh_session(app, &session_id) {
|
||||
return commands::CommandResult::error(error);
|
||||
}
|
||||
app.current_session_metadata = None;
|
||||
app.window_title = None;
|
||||
app.composer_color = None;
|
||||
@@ -1038,6 +1146,7 @@ pub(crate) async fn switch_workspace(
|
||||
if !app.api_messages.is_empty() {
|
||||
let _ = engine_handle
|
||||
.send(Op::SyncSession {
|
||||
control: app.session_sync_control(),
|
||||
session_id: app.current_session_id.clone(),
|
||||
messages: app.api_messages.as_ref().clone(),
|
||||
system_prompt: app.system_prompt.clone(),
|
||||
|
||||
@@ -9500,8 +9500,13 @@ fn authoritative_user_message(text: &str) -> Message {
|
||||
|
||||
fn saved_session_with_messages(messages: Vec<Message>) -> SavedSession {
|
||||
SavedSession {
|
||||
write_receipt: None,
|
||||
|
||||
schema_version: 1,
|
||||
metadata: codewhale_engine::session_manager::SessionMetadata {
|
||||
publication_id: None,
|
||||
native_mutation_observation: None,
|
||||
|
||||
id: uuid::Uuid::new_v4().to_string(),
|
||||
title: "resume recovery".to_string(),
|
||||
title_is_custom: false,
|
||||
@@ -14168,10 +14173,14 @@ async fn a_steer_settles_even_though_the_engine_stored_it_trimmed() {
|
||||
let workspace = app.workspace.clone();
|
||||
// The engine's record carries the trimmed form, which is what
|
||||
// `turn_loop` stores for a committed steer.
|
||||
let projection_control = app.session_sync_control();
|
||||
assert!(super::event_loop::apply_engine_session_projection(
|
||||
&mut app,
|
||||
&Config::default(),
|
||||
EngineEvent::SessionUpdated {
|
||||
source: Some(codewhale_engine::core::ops::SessionSnapshotSource::control(
|
||||
projection_control
|
||||
)),
|
||||
checkpoint: None,
|
||||
session_id: session,
|
||||
messages: std::sync::Arc::new(vec![
|
||||
@@ -14247,10 +14256,14 @@ async fn steer_becomes_the_newest_transcript_entry_only_when_the_engine_records_
|
||||
app.is_loading = false;
|
||||
let model = app.model.clone();
|
||||
let workspace = app.workspace.clone();
|
||||
let projection_control = app.session_sync_control();
|
||||
assert!(super::event_loop::apply_engine_session_projection(
|
||||
&mut app,
|
||||
&Config::default(),
|
||||
EngineEvent::SessionUpdated {
|
||||
source: Some(codewhale_engine::core::ops::SessionSnapshotSource::control(
|
||||
projection_control
|
||||
)),
|
||||
checkpoint: None,
|
||||
session_id: session,
|
||||
messages: std::sync::Arc::new(vec![
|
||||
@@ -19410,6 +19423,7 @@ fn local_cancel_marks_late_stream_events_for_suppression() {
|
||||
));
|
||||
assert!(suppress_engine_event_after_local_cancel(
|
||||
&EngineEvent::SessionUpdated {
|
||||
source: None,
|
||||
checkpoint: None,
|
||||
session_id: "session".to_string(),
|
||||
messages: std::sync::Arc::new(Vec::new()),
|
||||
@@ -19662,6 +19676,10 @@ fn turn_started_route_is_captured_before_cancel_suppression() {
|
||||
});
|
||||
let created_at = chrono::Utc::now();
|
||||
let event = EngineEvent::TurnStarted {
|
||||
previous_source: app
|
||||
.session_write_owner
|
||||
.as_ref()
|
||||
.and_then(|owner| owner.snapshot_source()),
|
||||
turn_id: "turn_cancel_race".to_string(),
|
||||
created_at,
|
||||
submission_id: None,
|
||||
@@ -19731,6 +19749,10 @@ fn turn_started_suggestion_authority_comes_from_the_route_receipt_not_config() {
|
||||
"sk-route-a-secret",
|
||||
);
|
||||
let event = EngineEvent::TurnStarted {
|
||||
previous_source: app
|
||||
.session_write_owner
|
||||
.as_ref()
|
||||
.and_then(|owner| owner.snapshot_source()),
|
||||
turn_id: "turn_route_receipt".to_string(),
|
||||
created_at: chrono::Utc::now(),
|
||||
route: Some(codewhale_engine::core::events::TurnRoute {
|
||||
@@ -19776,6 +19798,10 @@ fn turn_started_suggestion_authority_comes_from_the_route_receipt_not_config() {
|
||||
fn turn_started_without_a_route_receipt_captures_no_suggestion_authority() {
|
||||
let mut app = create_test_app();
|
||||
let event = EngineEvent::TurnStarted {
|
||||
previous_source: app
|
||||
.session_write_owner
|
||||
.as_ref()
|
||||
.and_then(|owner| owner.snapshot_source()),
|
||||
turn_id: "turn_no_receipt".to_string(),
|
||||
created_at: chrono::Utc::now(),
|
||||
route: Some(codewhale_engine::core::events::TurnRoute {
|
||||
@@ -19815,6 +19841,10 @@ fn engine_error_health_accounting_uses_active_turn_route() {
|
||||
app.api_provider = ProviderKind::Deepseek;
|
||||
app.model = "current-model".to_string();
|
||||
let event = EngineEvent::TurnStarted {
|
||||
previous_source: app
|
||||
.session_write_owner
|
||||
.as_ref()
|
||||
.and_then(|owner| owner.snapshot_source()),
|
||||
turn_id: "routed-turn".to_string(),
|
||||
created_at: chrono::Utc::now(),
|
||||
route: Some(codewhale_engine::core::events::TurnRoute {
|
||||
@@ -33199,6 +33229,10 @@ fn fresh_session_turn_lifecycle_leaves_no_orphan_checkpoint() {
|
||||
// Fresh launch (event_loop::run_tui): the App claims the id, the engine
|
||||
// adopts it.
|
||||
let app_session_id = super::event_loop::ensure_runtime_session_id(&mut app);
|
||||
app.session_write_receipt = Some(manager.expect_absent_session(&app_session_id).unwrap());
|
||||
app.session_write_owner = Some(
|
||||
codewhale_engine::session_manager::SessionWriteOwner::Submission("fresh-submission".into()),
|
||||
);
|
||||
let (engine, _handle) =
|
||||
codewhale_engine::core::engine::Engine::new(build_engine_config(&app, &config), &config);
|
||||
|
||||
@@ -33210,11 +33244,16 @@ fn fresh_session_turn_lifecycle_leaves_no_orphan_checkpoint() {
|
||||
cache_control: None,
|
||||
}],
|
||||
});
|
||||
let checkpoint = build_session_snapshot(&mut app, &manager).expect("turn-start snapshot");
|
||||
let mut checkpoint = build_session_snapshot(&mut app, &manager).expect("turn-start snapshot");
|
||||
assert_eq!(checkpoint.metadata.id, app_session_id);
|
||||
manager
|
||||
.save_checkpoint(&checkpoint)
|
||||
let receipt = checkpoint
|
||||
.capture_publication(app.session_write_owner.clone().unwrap())
|
||||
.unwrap();
|
||||
let publication = manager
|
||||
.publish_checkpoint_owned(checkpoint)
|
||||
.expect("save turn-start checkpoint");
|
||||
assert_eq!(receipt.observation().unwrap(), *publication.observation());
|
||||
app.session_write_receipt = Some(publication.write_receipt());
|
||||
|
||||
// The engine reports its conversation id (`SessionUpdated` handler).
|
||||
app.current_session_id = Some(engine.session_id().to_string());
|
||||
@@ -33228,14 +33267,23 @@ fn fresh_session_turn_lifecycle_leaves_no_orphan_checkpoint() {
|
||||
cache_control: None,
|
||||
}],
|
||||
});
|
||||
let completed = build_session_snapshot(&mut app, &manager).expect("completed snapshot");
|
||||
manager.save_session(&completed).expect("save session");
|
||||
manager
|
||||
.clear_session_checkpoint(&completed.metadata.id)
|
||||
.expect("clear checkpoint");
|
||||
let mut completed = build_session_snapshot(&mut app, &manager).expect("completed snapshot");
|
||||
completed
|
||||
.capture_publication(
|
||||
codewhale_engine::session_manager::SessionWriteOwner::Engine(
|
||||
codewhale_engine::core::ops::SessionSnapshotSource::turn(
|
||||
"fresh-turn".into(),
|
||||
Some("fresh-submission".into()),
|
||||
),
|
||||
),
|
||||
)
|
||||
.unwrap();
|
||||
let completed_id = completed.metadata.id.clone();
|
||||
let (_, clear) = manager.publish_completed_session(completed).expect("save session");
|
||||
clear.expect("clear checkpoint");
|
||||
|
||||
assert_eq!(
|
||||
completed.metadata.id, app_session_id,
|
||||
completed_id, app_session_id,
|
||||
"the completed turn must commit under the id the checkpoint was written with"
|
||||
);
|
||||
let orphans = manager.list_checkpoints().expect("list checkpoints");
|
||||
@@ -33990,6 +34038,7 @@ fn stale_session_projection_cannot_rewind_a_host_launch() {
|
||||
&mut app,
|
||||
&Config::default(),
|
||||
EngineEvent::SessionUpdated {
|
||||
source: None,
|
||||
checkpoint: None,
|
||||
session_id: previous,
|
||||
messages: std::sync::Arc::new(vec![text_message("user", "stale transcript")]),
|
||||
@@ -34015,8 +34064,21 @@ fn stale_session_projection_cannot_rewind_a_host_launch() {
|
||||
assert_eq!(app.status_toasts.len(), notice_count);
|
||||
app.runtime_turn_id = Some("checkpoint-turn".into());
|
||||
app.is_loading = true;
|
||||
app.session_write_owner = Some(
|
||||
codewhale_engine::session_manager::SessionWriteOwner::Engine(
|
||||
codewhale_engine::core::ops::SessionSnapshotSource::turn(
|
||||
"checkpoint-turn".into(),
|
||||
None,
|
||||
),
|
||||
),
|
||||
);
|
||||
for invalid in ["session", "turn", "workspace", "child", "withdrawn"] {
|
||||
let snapshot = std::sync::Arc::new(codewhale_engine::core::ops::SessionSnapshot {
|
||||
source: Some(codewhale_engine::core::ops::SessionSnapshotSource::turn(
|
||||
"checkpoint-turn".into(),
|
||||
None,
|
||||
)),
|
||||
|
||||
session_id: if invalid == "session" {
|
||||
"stale-session".into()
|
||||
} else {
|
||||
@@ -34066,6 +34128,7 @@ fn stale_session_projection_cannot_rewind_a_host_launch() {
|
||||
&mut app,
|
||||
&Config::default(),
|
||||
EngineEvent::SessionUpdated {
|
||||
source: snapshot.source.clone(),
|
||||
checkpoint: Some(request),
|
||||
session_id: snapshot.session_id.clone(),
|
||||
messages: std::sync::Arc::new(snapshot.messages.clone()),
|
||||
@@ -34109,10 +34172,14 @@ fn stale_session_projection_cannot_rewind_a_host_launch() {
|
||||
app.active_context_window = Some(accepted_window);
|
||||
app.active_context_window_source = accepted_window.source;
|
||||
app.pending_turn_route = Some((app.api_provider, "canonical-model".into(), false));
|
||||
let projection_control = app.session_sync_control();
|
||||
assert!(super::event_loop::apply_engine_session_projection(
|
||||
&mut app,
|
||||
&Config::default(),
|
||||
EngineEvent::SessionUpdated {
|
||||
source: Some(codewhale_engine::core::ops::SessionSnapshotSource::control(
|
||||
projection_control
|
||||
)),
|
||||
checkpoint: None,
|
||||
session_id: current.clone(),
|
||||
messages: std::sync::Arc::new(vec![
|
||||
|
||||
@@ -41,6 +41,10 @@ async fn settled(app: &mut App, handle: &EngineHandle, answer: &str) {
|
||||
.get_session_snapshot()
|
||||
.await
|
||||
.expect("settled snapshot");
|
||||
app.session_write_owner = snapshot
|
||||
.source
|
||||
.clone()
|
||||
.map(codewhale_engine::session_manager::SessionWriteOwner::Engine);
|
||||
app.current_session_id = Some(snapshot.session_id);
|
||||
app.set_api_messages(Arc::new(snapshot.messages));
|
||||
app.system_prompt = snapshot.system_prompt;
|
||||
@@ -52,8 +56,11 @@ async fn settled(app: &mut App, handle: &EngineHandle, answer: &str) {
|
||||
});
|
||||
}
|
||||
|
||||
fn sync_snapshot(snapshot: &codewhale_engine::core::ops::SessionSnapshot) -> Op {
|
||||
fn sync_snapshot(snapshot: &codewhale_engine::core::ops::SessionSnapshot,
|
||||
control: codewhale_engine::core::ops::SessionControlId,
|
||||
) -> Op {
|
||||
Op::SyncSession {
|
||||
control,
|
||||
session_id: Some(snapshot.session_id.clone()),
|
||||
messages: snapshot.messages.clone(),
|
||||
system_prompt: snapshot.system_prompt.clone(),
|
||||
@@ -99,8 +106,6 @@ fn undo_retry_apply_path_keeps_engine_request_and_reopened_session_consistent()
|
||||
let home = TempDir::new().unwrap();
|
||||
let workspace = TempDir::new().unwrap();
|
||||
let _home = crate::test_support::EnvVarGuard::set("CODEWHALE_HOME", home.path());
|
||||
let _user_home = crate::test_support::EnvVarGuard::set("HOME", home.path());
|
||||
let _profile = crate::test_support::EnvVarGuard::set("USERPROFILE", home.path());
|
||||
let runtime = tokio::runtime::Builder::new_current_thread()
|
||||
.enable_all()
|
||||
.build()
|
||||
@@ -125,7 +130,6 @@ fn undo_retry_apply_path_keeps_engine_request_and_reopened_session_consistent()
|
||||
canned::simple_text_turn("retry answer one"),
|
||||
canned::simple_text_turn("retry answer two"),
|
||||
canned::simple_text_turn("answer after reopen"),
|
||||
canned::simple_text_turn("edited answer"),
|
||||
]));
|
||||
let (engine, mut handle) = Engine::new_with_model_client(
|
||||
EngineConfig {
|
||||
@@ -141,6 +145,9 @@ fn undo_retry_apply_path_keeps_engine_request_and_reopened_session_consistent()
|
||||
);
|
||||
let engine_task = tokio::spawn(engine.run());
|
||||
let manager = SessionManager::default_location().unwrap();
|
||||
let initial_id = handle.get_session_snapshot().await.unwrap().session_id;
|
||||
app.set_current_session_id(Some(initial_id.clone()));
|
||||
super::super::session_state::expect_fresh_session(&mut app, &initial_id).unwrap();
|
||||
let (actor, actor_task) =
|
||||
persistence_actor::spawn_persistence_actor(SessionManager::default_location().unwrap());
|
||||
persistence_actor::init_actor(actor.clone());
|
||||
@@ -195,7 +202,7 @@ fn undo_retry_apply_path_keeps_engine_request_and_reopened_session_consistent()
|
||||
compacted.system_prompt = Some(SystemPrompt::Text(format!(
|
||||
"{stable_prompt}\n\n<!-- compaction-summary:begin -->\n{checkpoint_text}\n<!-- compaction-summary:end -->"
|
||||
)));
|
||||
handle.send(sync_snapshot(&compacted)).await.unwrap();
|
||||
handle.send(sync_snapshot(&compacted, app.session_sync_control())).await.unwrap();
|
||||
loop {
|
||||
let event = handle.rx_event.write().await.recv().await.unwrap();
|
||||
if matches!(&event, EngineEvent::SessionUpdated { messages, .. } if messages.contains(&checkpoint)) {
|
||||
@@ -206,10 +213,15 @@ fn undo_retry_apply_path_keeps_engine_request_and_reopened_session_consistent()
|
||||
assert!(codewhale_engine::compaction::extract_compaction_summary(app.system_prompt.as_ref()).is_none());
|
||||
assert!(app.api_messages.contains(&checkpoint));
|
||||
let id = app.current_session_id.clone().unwrap();
|
||||
let old = build_session_snapshot(&mut app, &manager).unwrap();
|
||||
let path = manager.save_session(&old).unwrap();
|
||||
flush_persistence(&actor).await;
|
||||
let mut old = build_session_snapshot(&mut app, &manager).unwrap();
|
||||
old.capture_publication(app.session_write_owner.clone().unwrap()).unwrap();
|
||||
let publication = manager.publish_session(&old).unwrap();
|
||||
let path = publication.path.clone();
|
||||
old.adopt_publication(&publication);
|
||||
app.session_write_receipt = old.write_receipt.clone();
|
||||
// The stale in-flight recovery record must not revive the undone turn.
|
||||
assert!(actor.try_send(PersistRequest::SaveCheckpoint { session: old }));
|
||||
assert!(super::super::session_state::try_persist_session(&mut app, PersistRequest::SaveCheckpoint { session: old }));
|
||||
flush_persistence(&actor).await;
|
||||
assert!(
|
||||
serde_json::to_string(
|
||||
@@ -297,7 +309,9 @@ fn undo_retry_apply_path_keeps_engine_request_and_reopened_session_consistent()
|
||||
"mode" => newer.mode = AppMode::Plan.as_setting().into(),
|
||||
_ => unreachable!(),
|
||||
}
|
||||
let replacement = sync_snapshot(&newer);
|
||||
let replacement_control = codewhale_engine::core::ops::SessionControlId::new();
|
||||
newer.source = Some(codewhale_engine::core::ops::SessionSnapshotSource::control(replacement_control.clone()));
|
||||
let replacement = sync_snapshot(&newer, replacement_control);
|
||||
let real_handle = handle.clone();
|
||||
let queued = real_handle.clone();
|
||||
let (proxy_tx, mut proxy_rx) = tokio::sync::mpsc::channel(8);
|
||||
@@ -359,10 +373,12 @@ fn undo_retry_apply_path_keeps_engine_request_and_reopened_session_consistent()
|
||||
app.tr(MessageId::ConversationChangedBeforeUndo).as_ref()
|
||||
));
|
||||
// Restore only the fixture between independent race cases.
|
||||
handle.send(sync_snapshot(&original)).await.unwrap();
|
||||
let restore_control = app.session_sync_control();
|
||||
handle.send(sync_snapshot(&original, restore_control.clone())).await.unwrap();
|
||||
assert!(
|
||||
handle.get_session_snapshot().await.unwrap()
|
||||
== codewhale_engine::core::ops::SessionSnapshot {
|
||||
source: Some(codewhale_engine::core::ops::SessionSnapshotSource::control(restore_control)),
|
||||
instruction_assembly: None,
|
||||
..original
|
||||
}
|
||||
@@ -441,6 +457,17 @@ fn undo_retry_apply_path_keeps_engine_request_and_reopened_session_consistent()
|
||||
// File LoadSession always respawns a real client, so use the same
|
||||
// loaded-session projection with our injected model client here.
|
||||
apply_loaded_session(&mut app, &mut config, &reopened).unwrap();
|
||||
assert_eq!(
|
||||
app.history.iter().filter_map(|cell| match cell {
|
||||
HistoryCell::User { content } => Some(content.as_str()),
|
||||
_ => None,
|
||||
}).collect::<Vec<_>>(),
|
||||
["keep this"]
|
||||
);
|
||||
assert!(app.history.iter().any(|cell| {
|
||||
matches!(cell, HistoryCell::System { content } if content == &checkpoint_text)
|
||||
}));
|
||||
assert!(app.api_messages.contains(&checkpoint));
|
||||
let restore = AppAction::SyncSession {
|
||||
session_id: app.current_session_id.clone(),
|
||||
messages: app.api_messages.as_ref().clone(),
|
||||
@@ -485,6 +512,17 @@ fn undo_retry_apply_path_keeps_engine_request_and_reopened_session_consistent()
|
||||
assert!(!loaded.is_error, "{:?}", loaded.message);
|
||||
assert_eq!(app.input, "after reopen");
|
||||
assert!(app.edit_in_progress);
|
||||
let edit_store = Arc::new(SessionManager::default_location().unwrap());
|
||||
let edit_id = id.clone();
|
||||
mock.push_factory(move |_| {
|
||||
let durable = edit_store.load_session_snapshot(&edit_id).unwrap();
|
||||
assert_eq!(
|
||||
prompts(&durable.messages),
|
||||
["keep this"],
|
||||
"the edit rollback is durable before the replacement provider request"
|
||||
);
|
||||
canned::simple_text_turn("edited answer")
|
||||
});
|
||||
let result = super::super::event_loop::edit_replacement_result(&mut app, "edited prompt")
|
||||
.expect("a pending edit stages a replacement");
|
||||
assert!(!app.edit_in_progress);
|
||||
@@ -508,11 +546,18 @@ fn undo_retry_apply_path_keeps_engine_request_and_reopened_session_consistent()
|
||||
let inbound = serde_json::to_string(&requests.last().unwrap().messages).unwrap();
|
||||
assert!(!inbound.contains("after reopen"));
|
||||
assert_eq!(prompts(&app.api_messages), ["keep this", "edited prompt"]);
|
||||
let completed = build_session_snapshot(&mut app, &manager).unwrap();
|
||||
assert!(super::super::session_state::try_persist_session(
|
||||
&mut app,
|
||||
PersistRequest::CompletedCommit { session: completed }
|
||||
));
|
||||
flush_persistence(&actor).await;
|
||||
assert_eq!(
|
||||
prompts(&manager.load_session(&id).unwrap().messages),
|
||||
["keep this"],
|
||||
"the edit rollback is durable before the replacement turn"
|
||||
prompts(&manager.load_session_snapshot(&id).unwrap().messages),
|
||||
["keep this", "edited prompt"],
|
||||
"the completed replacement is durable after its completion save"
|
||||
);
|
||||
assert!(manager.load_session_checkpoint(&id).unwrap().is_none());
|
||||
// With no edit pending, a submit is an ordinary turn.
|
||||
assert!(
|
||||
super::super::event_loop::edit_replacement_result(&mut app, "plain").is_none()
|
||||
@@ -530,7 +575,17 @@ fn undo_retry_apply_path_keeps_engine_request_and_reopened_session_consistent()
|
||||
)
|
||||
.await
|
||||
.unwrap();
|
||||
assert_eq!(prompts(&app.api_messages), ["keep this"]);
|
||||
assert_eq!(
|
||||
prompts(&app.api_messages),
|
||||
["keep this"],
|
||||
"undo status: {:?}; toasts: {:?}; system history: {:?}",
|
||||
app.status_message,
|
||||
app.status_toasts.iter().map(|toast| toast.text.as_str()).collect::<Vec<_>>(),
|
||||
app.history.iter().filter_map(|cell| match cell {
|
||||
HistoryCell::System { content } => Some(content.as_str()),
|
||||
_ => None,
|
||||
}).collect::<Vec<_>>()
|
||||
);
|
||||
|
||||
// Portable save failure: block the canonical file with a directory.
|
||||
// Preserve the last durable snapshot as a sibling for inspection.
|
||||
@@ -554,7 +609,7 @@ fn undo_retry_apply_path_keeps_engine_request_and_reopened_session_consistent()
|
||||
assert!(app.queued_messages.is_empty());
|
||||
assert!(app.history.iter().any(|cell| matches!(cell, HistoryCell::System { content } if content == "Retry prompt was not sent. Copy it from here to retry:\nkeep this")));
|
||||
assert_eq!(mock.captured_requests().len(), 6);
|
||||
handle.send(sync_snapshot(&before_failure)).await.unwrap();
|
||||
handle.send(sync_snapshot(&before_failure, app.session_sync_control())).await.unwrap();
|
||||
assert_eq!(handle.get_session_snapshot().await.unwrap().messages, before_failure.messages);
|
||||
app.set_api_messages(Arc::new(before_failure.messages));
|
||||
app.push_history_cell(HistoryCell::User { content: "keep this".into() });
|
||||
|
||||
@@ -41,7 +41,9 @@ async fn runtime_store_binding_persists_on_exit_without_a_model_turn() -> anyhow
|
||||
);
|
||||
assert!(original.metadata.runtime_store.is_none());
|
||||
sessions.save_session(&original)?;
|
||||
sessions.save_checkpoint(&original)?;
|
||||
let checkpoint = sessions.save_checkpoint(&original)?;
|
||||
let checkpoint_before = std::fs::read(&checkpoint)?;
|
||||
let original = sessions.load_session_snapshot(&original.metadata.id)?;
|
||||
let mut app = Box::new(create_test_app());
|
||||
apply_loaded_session_with_goal(&mut app, &mut config, original.clone(), None)
|
||||
.map_err(anyhow::Error::msg)?;
|
||||
@@ -76,8 +78,9 @@ async fn runtime_store_binding_persists_on_exit_without_a_model_turn() -> anyhow
|
||||
assert!(
|
||||
sessions
|
||||
.load_session_checkpoint(&original.metadata.id)?
|
||||
.is_none()
|
||||
.is_some()
|
||||
);
|
||||
assert_eq!(std::fs::read(checkpoint)?, checkpoint_before);
|
||||
drop(app);
|
||||
drop(tasks);
|
||||
drop(explicit_store);
|
||||
@@ -578,6 +581,7 @@ async fn picker_recovers_missing_store_into_the_idle_host_and_persists_before_re
|
||||
execution_scope: "0".repeat(64),
|
||||
});
|
||||
sessions.save_session(&saved)?;
|
||||
let saved = sessions.load_session_snapshot("picker-interrupted")?;
|
||||
let mut app = Box::new(create_test_app());
|
||||
let tasks = TaskManager::start(
|
||||
TaskManagerConfig::from_runtime(&config, root.path().into(), None, Some(1)),
|
||||
@@ -610,7 +614,17 @@ async fn picker_recovers_missing_store_into_the_idle_host_and_persists_before_re
|
||||
"binding repair survives a contended UI restore"
|
||||
);
|
||||
drop(held);
|
||||
apply_loaded_session_with_goal(&mut app, &mut config, saved.clone(), None)
|
||||
let stale_error =
|
||||
apply_loaded_session_with_goal(&mut app, &mut config, saved.clone(), None)
|
||||
.expect_err("the repaired binding invalidates its original picker source");
|
||||
assert!(
|
||||
stale_error.contains("changed after its captured source"),
|
||||
"{stale_error}"
|
||||
);
|
||||
assert_eq!(app.current_session_id.as_deref(), Some("picker-current"));
|
||||
assert_eq!(app.api_messages, current_messages);
|
||||
let recovered = sessions.load_session_snapshot("picker-interrupted")?;
|
||||
apply_loaded_session_with_goal(&mut app, &mut config, recovered, None)
|
||||
.map_err(anyhow::Error::msg)?;
|
||||
assert_eq!(
|
||||
app.current_session_id.as_deref(),
|
||||
@@ -898,6 +912,7 @@ async fn picker_adopts_existing_empty_unheld_store() -> anyhow::Result<()> {
|
||||
execution_scope: "0".repeat(64),
|
||||
});
|
||||
sessions.save_session(&saved)?;
|
||||
let saved = sessions.load_session_snapshot("picker-adoptable")?;
|
||||
|
||||
let mut app = Box::new(create_test_app());
|
||||
let tasks = TaskManager::start(
|
||||
@@ -974,6 +989,7 @@ async fn picker_adoption_keeps_a_store_another_document_binds() -> anyhow::Resul
|
||||
);
|
||||
saved.metadata.runtime_store = Some(binding.clone());
|
||||
sessions.save_session(&saved)?;
|
||||
let saved = sessions.load_session_snapshot("picker-adoptable")?;
|
||||
let mut sibling = codewhale_engine::session_manager::create_saved_session_with_id_and_mode(
|
||||
"same-host-sibling".into(),
|
||||
&[text_message("user", "saved in the same host")],
|
||||
|
||||
@@ -634,6 +634,12 @@ the original intent from a source that may already have changed. An unprepared
|
||||
target stays pending; a changed or unverifiable target refuses completion.
|
||||
Repeat recovery with the same key to observe the same committed result.
|
||||
|
||||
New history reservations retain their original document/checkpoint observation.
|
||||
An uncommitted recovery must match the complete prepared document and the
|
||||
operation's own publication generation, with its original checkpoint unchanged.
|
||||
An older reservation without that original observation stays pending or refuses
|
||||
recovery; reading current records cannot retrospectively supply it.
|
||||
|
||||
The selected workspace comes from the acknowledged owner or the explicitly
|
||||
admitted request. History and historical receipts cannot supply permissions,
|
||||
credentials, endpoints or a different owner. A missing, changed, busy or
|
||||
@@ -1141,6 +1147,18 @@ unreadable records returns an error rather than an incomplete successful list.
|
||||
The terminal picker retains readable rows and reports its inspection warning.
|
||||
These limits cover metadata inspection, not full transcript loading.
|
||||
|
||||
Saved-document and checkpoint publication uses a native source captured before
|
||||
snapshot preparation or queueing. Both physical records must still match under
|
||||
the existing session write admission before archival, pruning, replacement or
|
||||
checkpoint consumption. A mismatch refuses and retains recovery. Optional
|
||||
serialized `publication_id` metadata identifies a generation; it does not grant
|
||||
write authority. An unknown checkpoint needs explicit held resume/attach/promotion
|
||||
before automatic completion can consume it. Runtime prefix-restore validation
|
||||
remains distinct from the full-document predicate required for a later write.
|
||||
Optional usage-store damage remains visible as incomplete coverage while valid
|
||||
recovery data can be read without a publication receipt. Known retirement and
|
||||
changed raw records still refuse; this read result cannot authorize a save.
|
||||
|
||||
`PATCH /v1/sessions/{id}` renames and/or archives a saved session and returns a
|
||||
lifecycle receipt shaped like the thread patch receipt:
|
||||
|
||||
|
||||
@@ -505,6 +505,10 @@ owner 存储中的原始操作;响应丢失或来源历史后来增长不会
|
||||
它不会从可能已变化的来源重新构造原始意图。尚未准备好的目标保持待决;
|
||||
变化或无法验证的目标拒绝完成。使用同一个键重复恢复会观察到同一个已提交结果。
|
||||
|
||||
新的历史操作预留会保存原始文档和检查点的观察值。尚未提交的恢复必须匹配完整的
|
||||
预备文档及该操作自己的发布代次,且原始检查点保持不变。旧预留没有该原始观察值时,
|
||||
仍保持待决或拒绝恢复;读取当前文件不能补造过去的写入依据。
|
||||
|
||||
选定工作区来自已确认的 owner 或明确获准的请求。历史和旧回执不能提供权限、
|
||||
凭据、端点或另一个 owner。绑定存储缺失、变化、繁忙或不兼容时明确失败。
|
||||
规范目标缺失不会启动一个空的替代会话。
|
||||
@@ -917,6 +921,19 @@ bootstrap URL。该能力会创建一个 30 分钟的进程内
|
||||
会让列表视图变成无界读取。完整转录
|
||||
预览位于 TUI 会话选择器中,它只读取所选的那一个会话。
|
||||
|
||||
元数据检查最多读取每份常规会话文件的16 MiB。目录和生命周期路由超限时返回
|
||||
`413`,错误代码为 `session_metadata_too_large`,来源保持不变。存在无法读取的记录时,
|
||||
目录返回错误而不是不完整的成功列表;终端选择器保留可读行并报告检查警告。
|
||||
此限制针对元数据检查,不代表完整转录加载的上限。
|
||||
|
||||
保存文档或检查点的原生写入依据在准备快照或排队之前捕获。归档、修剪、替换或消费
|
||||
检查点之前,现有会话写入接纳范围内的两份物理记录都必须仍匹配;变化时拒绝并保留
|
||||
恢复记录。可选序列化元数据 `publication_id` 只标识发布代次,不授予写入权限。
|
||||
来源未知的检查点需要显式且持有写入接纳的恢复、挂接或提升,自动完成才可消费它。
|
||||
Runtime 前缀恢复验证与后续写入所需的完整文档条件保持独立。
|
||||
可选用量存储损坏时,不完整覆盖仍然可见,合法的恢复数据可在不带发布回执的情况下
|
||||
读取。已确认删除或原始记录变化时仍拒绝;这种读取结果不能授权保存。
|
||||
|
||||
`PATCH /v1/sessions/{id}` 重命名并/或归档一个已保存会话,并返回一个
|
||||
形态与线程 patch 回执相同的生命周期回执:
|
||||
|
||||
|
||||
Reference in new issue
Block a user