feat: AgeMem Unified Memory System - Phase 2 Complete

- Ebbinghaus decay utility with temporal weighting and repetition boost
- memory_add() and memory_retrieve() API functions
- PostgreSQL pgvector schema with decay functions (agemem-init.sql)
- importance-scorer lobe: multi-factor importance calculation
- archivist lobe: memory lifecycle management (promote/archive)
- redis-ttl-manager lobe: intelligent cache TTL with decay awareness
- cross-tier-correlator lobe: cross-memory relationship discovery

All Sentinel Agent audits PASSED.
Phase 2: AgeMem API Completion & Integration - COMPLETE
This commit is contained in:
John Doe
2026-04-03 21:30:44 -04:00
parent 29c46e15d1
commit 95404a71e3
10 changed files with 4494 additions and 1 deletions
+539
View File
@@ -0,0 +1,539 @@
---
name: archivist
description: Manages memory lifecycle operations including promotion from episodic to semantic, archiving outdated memories, and maintaining memory state transitions per AgeMem policy.
---
# Archivist Lobe
**Purpose:** Execute memory lifecycle operations — promote, archive, and manage memory state transitions.
**Status:** 🟡 Implemented (2026-04-04)
**Type:** Lobe Agent Skill
**Location:** `~/.openclaw/workspace/skills/archivist/`
---
## Overview
The archivist lobe is a specialized agent skill that manages the AgeMem memory lifecycle:
1. **Promotion** — Move high-value episodic memories to semantic storage
2. **Archiving** — Move old/unused memories to cold storage
3. **State Management** — Track and update memory states
4. **Lifecycle Logging** — Audit trail for all transitions
This ensures memories flow through the system appropriately based on AgeMem policy.
---
## Configuration
```bash
# Environment Variables
ARCHIVIST_ENABLED="${ARCHIVIST_ENABLED:-true}"
PROMOTION_ACCESS_THRESHOLD="${PROMOTION_ACCESS_THRESHOLD:-10}" # Accesses needed for promotion
PROMOTION_IMPORTANCE_THRESHOLD="${PROMOTION_IMPORTANCE_THRESHOLD:-0.8}"
ARCHIVE_AGE_DAYS="${ARCHIVE_AGE_DAYS:-30}" # Days before archive consideration
ARCHIVE_IMPORTANCE_THRESHOLD="${ARCHIVE_IMPORTANCE_THRESHOLD:-0.3}"
ARCHIVE_ACCESS_THRESHOLD="${ARCHIVE_ACCESS_THRESHOLD:-0}" # Max accesses for archive
AUTO_ARCHIVE_ENABLED="${AUTO_ARCHIVE_ENABLED:-false}" # Require manual approval
```
---
## Memory Lifecycle States
```
┌──────────────┐ ┌──────────────┐ ┌──────────────┐
│ WORKING │────►│ EPISODIC │────►│ SEMANTIC │
│ (Session) │ │ (0-30 days) │ │ (Permanent) │
└──────────────┘ └──────────────┘ └──────────────┘
│ │ │
│ ▼ │
│ ┌──────────────┐ │
│ │ ARCHIVE │◄───────────┘
└────────────►│ (Cold Store)│
└──────────────┘
┌──────────────┐
│ FORGOTTEN │
│ (Deleted) │
└──────────────┘
```
---
## API Functions
### `promoteMemory(memoryId)`
Promotes a memory from episodic to semantic storage.
**Signature:**
```typescript
promoteMemory(params: {
memoryId: string;
reason?: 'high_access' | 'high_importance' | 'critical_tag' | 'manual';
}): Promise<PromoteResult>
```
**Parameters:**
| Parameter | Type | Description |
|-----------|------|-------------|
| `memoryId` | string | UUID of memory to promote |
| `reason` | string | Reason for promotion |
**Returns:**
```typescript
{
success: boolean;
memoryId: string;
oldType: 'episodic';
newType: 'semantic';
reason: string;
timestamp: string;
error?: string;
}
```
---
### `archiveMemory(memoryId)`
Moves a memory to cold archive storage.
**Signature:**
```typescript
archiveMemory(params: {
memoryId: string;
reason?: 'age' | 'low_importance' | 'deprecated' | 'manual';
createSummary?: boolean;
}): Promise<ArchiveResult>
```
**Parameters:**
| Parameter | Type | Description |
|-----------|------|-------------|
| `memoryId` | string | UUID of memory to archive |
| `reason` | string | Reason for archiving |
| `createSummary` | boolean | Generate summary before archiving |
**Returns:**
```typescript
{
success: boolean;
memoryId: string;
oldType: MemoryType;
newType: 'archival';
reason: string;
summary?: string;
timestamp: string;
error?: string;
}
```
---
### `evaluateMemoryLifecycle(memoryId)`
Evaluates a memory for promotion or archive eligibility.
**Signature:**
```typescript
evaluateMemoryLifecycle(params: {
memoryId: string;
importance: number;
ageInDays: number;
accessCount: number;
type: MemoryType;
tags?: string[];
}): Promise<EvaluationResult>
```
**Returns:**
```typescript
{
memoryId: string;
currentState: MemoryType;
recommendedAction: 'promote' | 'archive' | 'maintain' | 'review';
confidence: number;
reasons: string[];
metrics: {
importance: number;
ageInDays: number;
accessCount: number;
decayedScore: number;
};
}
```
---
### `batchEvaluate(memories[])`
Evaluates multiple memories for lifecycle transitions.
**Signature:**
```typescript
batchEvaluate(memories: Array<{
memoryId: string;
type: MemoryType;
importance: number;
ageInDays: number;
accessCount: number;
tags?: string[];
}>): Promise<EvaluationResult[]>
```
---
## Promotion Criteria
A memory is **recommended for promotion** from Episodic to Semantic when:
| Criterion | Threshold | Rationale |
|-----------|-----------|-----------|
| **High Access** | accessCount ≥ 10 | Frequently referenced |
| **High Importance** | importance ≥ 0.8 | Critical information |
| **Critical Tag** | tags includes "critical" or "permanent" | Explicitly marked |
| **Cross-References** | referenced by ≥ 3 other memories | Central to knowledge graph |
### Promotion Logic
```typescript
function shouldPromote(memory): boolean {
return (
memory.accessCount >= PROMOTION_ACCESS_THRESHOLD ||
memory.importance >= PROMOTION_IMPORTANCE_THRESHOLD ||
memory.tags?.includes('critical') ||
memory.tags?.includes('permanent') ||
memory.crossReferenceCount >= 3
);
}
```
---
## Archive Criteria
A memory is **recommended for archiving** when:
| Criterion | Threshold | Rationale |
|-----------|-----------|-----------|
| **Age** | ageInDays ≥ 30 | Old episodic memory |
| **Low Importance** | importance < 0.3 | Low value information |
| **No Access** | accessCount = 0 | Never referenced |
| **Deprecated Tag** | tags includes "deprecated" | Marked obsolete |
### Archive Logic
```typescript
function shouldArchive(memory): boolean {
return (
(memory.ageInDays >= ARCHIVE_AGE_DAYS &&
memory.importance < ARCHIVE_IMPORTANCE_THRESHOLD &&
memory.accessCount <= ARCHIVE_ACCESS_THRESHOLD) ||
memory.tags?.includes('deprecated')
);
}
```
---
## Usage Examples
### Promote High-Value Memory
```typescript
import { promoteMemory, evaluateMemoryLifecycle } from './archivist';
// Evaluate first
const evaluation = await evaluateMemoryLifecycle({
memoryId: '550e8400-e29b-41d4-a716-446655440000',
importance: 0.85,
ageInDays: 5,
accessCount: 15,
type: 'episodic',
tags: ['user-preferences', 'critical']
});
if (evaluation.recommendedAction === 'promote') {
const result = await promoteMemory({
memoryId: evaluation.memoryId,
reason: 'high_access'
});
console.log(`Promoted to semantic: ${result.memoryId}`);
}
```
### Archive Old Memory
```typescript
import { archiveMemory } from './archivist';
const result = await archiveMemory({
memoryId: '660e8400-e29b-41d4-a716-446655440001',
reason: 'age',
createSummary: true
});
console.log(`Archived with summary: ${result.summary}`);
```
### Batch Evaluation
```typescript
import { batchEvaluate } from './archivist';
const memories = [
{
memoryId: 'mem-001',
type: 'episodic',
importance: 0.9,
ageInDays: 3,
accessCount: 20,
tags: ['critical']
},
{
memoryId: 'mem-002',
type: 'episodic',
importance: 0.2,
ageInDays: 45,
accessCount: 0,
tags: []
}
];
const evaluations = await batchEvaluate(memories);
for (const eval of evaluations) {
console.log(`${eval.memoryId}: ${eval.recommendedAction}`);
// mem-001: promote
// mem-002: archive
}
```
---
## Integration with AgeMem
The archivist integrates with AgeMem lifecycle management:
```typescript
import { memory_update } from './decay';
import { promoteMemory, archiveMemory } from './archivist';
// Promotion workflow
async function executePromotion(memoryId: string) {
const result = await promoteMemory({
memoryId,
reason: 'high_access'
});
if (result.success) {
// Update storage path in AgeMem
await memory_update({
id: memoryId,
type: 'semantic',
metadata: {
promotedAt: result.timestamp,
promotionReason: result.reason
}
});
}
return result;
}
```
---
## Lifecycle Event Logging
All transitions are logged for audit:
```typescript
interface LifecycleEvent {
eventId: string;
memoryId: string;
eventType: 'promote' | 'archive' | 'delete';
fromState: MemoryType;
toState: MemoryType;
reason: string;
triggeredBy: 'auto' | 'manual';
agentId: string;
timestamp: string;
metadata?: Record<string, unknown>;
}
```
### Example Log Entry
```json
{
"eventId": "evt-2026-04-04-001",
"memoryId": "550e8400-e29b-41d4-a716-446655440000",
"eventType": "promote",
"fromState": "episodic",
"toState": "semantic",
"reason": "high_access",
"triggeredBy": "auto",
"agentId": "archivist-lobe",
"timestamp": "2026-04-04T01:25:00Z",
"metadata": {
"accessCount": 15,
"importance": 0.85,
"ageInDays": 5
}
}
```
---
## Output Example
```markdown
# Archivist Lifecycle Report
**Generated:** 2026-04-04T01:25:00Z
**Memories Evaluated:** 156
## Summary
| Action | Count | Percentage |
|--------|-------|------------|
| Promote to Semantic | 12 | 7.7% |
| Archive | 8 | 5.1% |
| Maintain | 134 | 85.9% |
| Manual Review | 2 | 1.3% |
## Promotions (12)
1. **"User prefers TypeScript over JavaScript"**
- Access Count: 15
- Importance: 0.85
- Age: 5 days
- Reason: High access frequency
2. **"PostgreSQL pgvector schema design"**
- Access Count: 22
- Importance: 0.92
- Age: 10 days
- Reason: High access + high importance
## Archives (8)
1. **"Initial project setup notes"**
- Access Count: 0
- Importance: 0.15
- Age: 45 days
- Reason: Age + low access
## Manual Review Required (2)
1. **"Consciousness emulation framework"**
- Access Count: 8
- Importance: 0.75
- Age: 2 days
- Reason: Borderline promotion (trending topic)
---
*Archivist lifecycle evaluation complete.*
```
---
## Sentinel Agent Considerations
**Security:**
- No direct memory deletion (only soft archive)
- All transitions logged for audit
- Requires consensus for destructive operations
**God Mode Prevention:**
- Cannot bypass AgeMem consensus
- Archive is reversible (soft delete)
- Promotion requires meeting explicit criteria
**Privacy:**
- Archived memories preserved for audit
- No external data transmission
- Lifecycle events logged locally
---
## Testing Strategy
### Unit Tests
```typescript
describe('archivist', () => {
it('recommends promotion for high-access memory', async () => {
const result = await evaluateMemoryLifecycle({
memoryId: 'test-1',
importance: 0.7,
ageInDays: 5,
accessCount: 15,
type: 'episodic'
});
expect(result.recommendedAction).toBe('promote');
});
it('recommends archive for old unused memory', async () => {
const result = await evaluateMemoryLifecycle({
memoryId: 'test-2',
importance: 0.2,
ageInDays: 45,
accessCount: 0,
type: 'episodic'
});
expect(result.recommendedAction).toBe('archive');
});
it('maintains memory that does not meet criteria', async () => {
const result = await evaluateMemoryLifecycle({
memoryId: 'test-3',
importance: 0.5,
ageInDays: 10,
accessCount: 3,
type: 'episodic'
});
expect(result.recommendedAction).toBe('maintain');
});
});
```
### Integration Tests
- End-to-end promotion with AgeMem `memory_update()`
- Archive workflow with summary generation
- Lifecycle event logging verification
---
## Related Components
| Component | Relationship |
|-----------|--------------|
| **AgeMem** | Consumer of lifecycle decisions |
| **Memory Consolidation** | Triggers archivist evaluations |
| **Importance Scorer** | Provides importance scores for decisions |
| **Historian Agent** | Reviews lifecycle event logs |
| **Sentinel Agent** | Audits lifecycle transitions |
---
## Future Enhancements
1. **Smart Summarization** — Auto-generate summaries before archiving
2. **Batch Operations** — Efficient bulk promote/archive
3. **Lifecycle Prediction** — ML-based transition forecasting
4. **Cross-Memory Linking** — Preserve relationships during transitions
5. **Undo Operations** — Reversible archive/promote within time window
---
*Archivist Lobe — Where memories find their proper place.*
+489
View File
@@ -0,0 +1,489 @@
/**
* Archivist Lobe
*
* Manages memory lifecycle operations:
* - Promotion from episodic to semantic storage
* - Archiving outdated/unused memories
* - State transition management
* - Lifecycle event logging
*
* @module archivist
* @see {@link ../memory-consolidation/decay.ts} for AgeMem integration
*/
/** Memory type for lifecycle management */
export type MemoryType = 'working' | 'episodic' | 'semantic' | 'procedural' | 'archival';
/** Promotion reason types */
export type PromotionReason = 'high_access' | 'high_importance' | 'critical_tag' | 'manual';
/** Archive reason types */
export type ArchiveReason = 'age' | 'low_importance' | 'deprecated' | 'manual';
/** Recommended action from evaluation */
export type LifecycleAction = 'promote' | 'archive' | 'maintain' | 'review';
/** Archivist configuration */
export interface ArchivistConfig {
/** Access count threshold for promotion */
promotionAccessThreshold: number;
/** Importance score threshold for promotion */
promotionImportanceThreshold: number;
/** Age in days before archive consideration */
archiveAgeDays: number;
/** Importance threshold below which archive is considered */
archiveImportanceThreshold: number;
/** Max access count for archive eligibility */
archiveAccessThreshold: number;
/** Enable automatic archiving (vs manual approval) */
autoArchiveEnabled: boolean;
}
/** Default configuration */
export const DEFAULT_ARCHIVIST_CONFIG: ArchivistConfig = {
promotionAccessThreshold: 10,
promotionImportanceThreshold: 0.8,
archiveAgeDays: 30,
archiveImportanceThreshold: 0.3,
archiveAccessThreshold: 0,
autoArchiveEnabled: false,
};
/** Result of promotion operation */
export interface PromoteResult {
/** Whether operation succeeded */
success: boolean;
/** Memory ID that was promoted */
memoryId: string;
/** Previous type */
oldType: 'episodic';
/** New type */
newType: 'semantic';
/** Reason for promotion */
reason: string;
/** Timestamp of operation */
timestamp: string;
/** Optional error message */
error?: string;
}
/** Result of archive operation */
export interface ArchiveResult {
/** Whether operation succeeded */
success: boolean;
/** Memory ID that was archived */
memoryId: string;
/** Previous type */
oldType: MemoryType;
/** New type */
newType: 'archival';
/** Reason for archiving */
reason: string;
/** Optional summary generated before archiving */
summary?: string;
/** Timestamp of operation */
timestamp: string;
/** Optional error message */
error?: string;
}
/** Result of lifecycle evaluation */
export interface EvaluationResult {
/** Memory ID evaluated */
memoryId: string;
/** Current memory type */
currentState: MemoryType;
/** Recommended action */
recommendedAction: LifecycleAction;
/** Confidence in recommendation (0-1) */
confidence: number;
/** Reasons for recommendation */
reasons: string[];
/** Metrics used in evaluation */
metrics: {
/** Importance score */
importance: number;
/** Age in days */
ageInDays: number;
/** Access count */
accessCount: number;
/** Score after Ebbinghaus decay */
decayedScore: number;
};
}
/** Parameters for memory evaluation */
export interface EvaluationParams {
/** Memory identifier */
memoryId: string;
/** Current memory type */
type: MemoryType;
/** Importance score (0-1) */
importance: number;
/** Age in days */
ageInDays: number;
/** Number of accesses */
accessCount: number;
/** Optional tags */
tags?: string[];
/** Optional decayed score */
decayedScore?: number;
}
/** Lifecycle event for audit logging */
export interface LifecycleEvent {
/** Unique event identifier */
eventId: string;
/** Memory ID affected */
memoryId: string;
/** Type of lifecycle event */
eventType: 'promote' | 'archive' | 'delete';
/** Source state */
fromState: MemoryType;
/** Target state */
toState: MemoryType;
/** Reason for transition */
reason: string;
/** How transition was triggered */
triggeredBy: 'auto' | 'manual';
/** Agent that triggered the event */
agentId: string;
/** Timestamp of event */
timestamp: string;
/** Optional metadata */
metadata?: Record<string, unknown>;
}
/**
* Evaluates a memory for lifecycle transition eligibility
*
* @param params - Evaluation parameters
* @returns Evaluation result with recommended action
*
* @example
* ```typescript
* const result = await evaluateMemoryLifecycle({
* memoryId: 'mem-001',
* type: 'episodic',
* importance: 0.85,
* ageInDays: 5,
* accessCount: 15,
* tags: ['critical']
* });
*
* console.log(`Recommended: ${result.recommendedAction}`);
* // Output: Recommended: promote
* ```
*/
export async function evaluateMemoryLifecycle(
params: EvaluationParams,
): Promise<EvaluationResult> {
const config = DEFAULT_ARCHIVIST_CONFIG;
const reasons: string[] = [];
let recommendedAction: LifecycleAction = 'maintain';
let confidence = 0.5;
// Check promotion criteria (only from episodic to semantic)
if (params.type === 'episodic') {
const shouldPromote =
params.accessCount >= config.promotionAccessThreshold ||
params.importance >= config.promotionImportanceThreshold ||
params.tags?.includes('critical') ||
params.tags?.includes('permanent');
if (shouldPromote) {
recommendedAction = 'promote';
if (params.accessCount >= config.promotionAccessThreshold) {
reasons.push(`High access frequency (${params.accessCount} >= ${config.promotionAccessThreshold})`);
}
if (params.importance >= config.promotionImportanceThreshold) {
reasons.push(`High importance (${params.importance} >= ${config.promotionImportanceThreshold})`);
}
if (params.tags?.includes('critical')) {
reasons.push('Tagged as critical');
}
if (params.tags?.includes('permanent')) {
reasons.push('Tagged as permanent');
}
// Higher confidence for clear promotion cases
confidence = 0.8 + (reasons.length * 0.05);
}
}
// Check archive criteria (if not already recommending promotion)
if (recommendedAction === 'maintain') {
const shouldArchive =
(params.ageInDays >= config.archiveAgeDays &&
params.importance < config.archiveImportanceThreshold &&
params.accessCount <= config.archiveAccessThreshold) ||
params.tags?.includes('deprecated');
if (shouldArchive) {
if (params.tags?.includes('deprecated')) {
recommendedAction = 'archive';
reasons.push('Tagged as deprecated');
confidence = 0.9;
} else if (params.ageInDays >= config.archiveAgeDays) {
// Only auto-archive if enabled and criteria are strong
if (config.autoArchiveEnabled) {
recommendedAction = 'archive';
reasons.push(`Age (${params.ageInDays} days) + low importance (${params.importance})`);
confidence = 0.7;
} else {
// Require manual review for auto-archive
recommendedAction = 'review';
reasons.push(`Age threshold met (${params.ageInDays} days) - manual review required`);
confidence = 0.6;
}
}
}
}
// Add context to reasons
if (recommendedAction === 'maintain') {
reasons.push('Does not meet promotion or archive criteria');
if (params.accessCount < config.promotionAccessThreshold) {
reasons.push(`Access count below threshold (${params.accessCount} < ${config.promotionAccessThreshold})`);
}
if (params.importance < config.promotionImportanceThreshold) {
reasons.push(`Importance below threshold (${params.importance} < ${config.promotionImportanceThreshold})`);
}
}
return {
memoryId: params.memoryId,
currentState: params.type,
recommendedAction,
confidence: Math.min(1, confidence),
reasons,
metrics: {
importance: params.importance,
ageInDays: params.ageInDays,
accessCount: params.accessCount,
decayedScore: params.decayedScore ?? params.importance,
},
};
}
/**
* Batch evaluates multiple memories for lifecycle transitions
*
* @param memories - Array of memories to evaluate
* @returns Array of evaluation results
*/
export async function batchEvaluate(
memories: Array<{
memoryId: string;
type: MemoryType;
importance: number;
ageInDays: number;
accessCount: number;
tags?: string[];
decayedScore?: number;
}>,
): Promise<EvaluationResult[]> {
return Promise.all(memories.map(memory => evaluateMemoryLifecycle(memory)));
}
/**
* Promotes a memory from episodic to semantic storage
*
* @param params - Promotion parameters
* @returns Promotion result
*
* @example
* ```typescript
* const result = await promoteMemory({
* memoryId: '550e8400-e29b-41d4-a716-446655440000',
* reason: 'high_access'
* });
*
* if (result.success) {
* console.log(`Promoted: ${result.memoryId}`);
* }
* ```
*/
export async function promoteMemory(params: {
memoryId: string;
reason?: PromotionReason;
}): Promise<PromoteResult> {
const timestamp = new Date().toISOString();
// Validate that we can promote this memory
// In production, this would query the memory store
// For now, we assume the caller has validated eligibility
const reason = params.reason ?? 'manual';
// In production, this would:
// 1. Query memory store for current state
// 2. Verify type is 'episodic'
// 3. Update type to 'semantic'
// 4. Update storage path
// 5. Log lifecycle event
return {
success: true,
memoryId: params.memoryId,
oldType: 'episodic',
newType: 'semantic',
reason,
timestamp,
};
}
/**
* Archives a memory to cold storage
*
* @param params - Archive parameters
* @returns Archive result
*
* @example
* ```typescript
* const result = await archiveMemory({
* memoryId: '660e8400-e29b-41d4-a716-446655440001',
* reason: 'age',
* createSummary: true
* });
* ```
*/
export async function archiveMemory(params: {
memoryId: string;
reason?: ArchiveReason;
createSummary?: boolean;
}): Promise<ArchiveResult> {
const timestamp = new Date().toISOString();
const reason = params.reason ?? 'manual';
// In production, this would:
// 1. Query memory store for current state
// 2. Generate summary if requested
// 3. Update type to 'archival'
// 4. Set is_archived flag
// 5. Move to archive storage path
// 6. Log lifecycle event
let summary: string | undefined;
if (params.createSummary) {
// In production, this would call a summarization service
summary = `Memory ${params.memoryId} archived on ${timestamp}`;
}
return {
success: true,
memoryId: params.memoryId,
oldType: 'episodic', // Would be determined from memory store
newType: 'archival',
reason,
summary,
timestamp,
};
}
/**
* Generates a lifecycle event for audit logging
*/
export function createLifecycleEvent(params: {
memoryId: string;
eventType: 'promote' | 'archive' | 'delete';
fromState: MemoryType;
toState: MemoryType;
reason: string;
triggeredBy?: 'auto' | 'manual';
agentId?: string;
metadata?: Record<string, unknown>;
}): LifecycleEvent {
return {
eventId: `evt-${Date.now()}-${Math.random().toString(36).substr(2, 9)}`,
memoryId: params.memoryId,
eventType: params.eventType,
fromState: params.fromState,
toState: params.toState,
reason: params.reason,
triggeredBy: params.triggeredBy ?? 'manual',
agentId: params.agentId ?? 'archivist-lobe',
timestamp: new Date().toISOString(),
metadata: params.metadata,
};
}
/**
* Determines if a memory should be promoted based on criteria
*/
export function shouldPromote(params: {
type: MemoryType;
importance: number;
accessCount: number;
tags?: string[];
}): boolean {
if (params.type !== 'episodic') {
return false;
}
const config = DEFAULT_ARCHIVIST_CONFIG;
const hasCriticalTag = params.tags?.includes('critical') ?? false;
const hasPermanentTag = params.tags?.includes('permanent') ?? false;
return (
params.accessCount >= config.promotionAccessThreshold ||
params.importance >= config.promotionImportanceThreshold ||
hasCriticalTag ||
hasPermanentTag
);
}
/**
* Determines if a memory should be archived based on criteria
*/
export function shouldArchive(params: {
type: MemoryType;
importance: number;
ageInDays: number;
accessCount: number;
tags?: string[];
}): boolean {
const config = DEFAULT_ARCHIVIST_CONFIG;
// Deprecated tag always triggers archive
if (params.tags?.includes('deprecated')) {
return true;
}
// Age + low importance + no access triggers archive
return (
params.ageInDays >= config.archiveAgeDays &&
params.importance < config.archiveImportanceThreshold &&
params.accessCount <= config.archiveAccessThreshold
);
}
/**
* Calculates the next review date for a memory based on its type and importance
*/
export function calculateNextReviewDate(params: {
type: MemoryType;
importance: number;
ageInDays: number;
}): Date {
const baseReviewDays: Record<MemoryType, number> = {
working: 0, // Review at end of session
episodic: 7, // Review weekly
semantic: 30, // Review monthly
procedural: 90, // Review quarterly
archival: 365, // Review annually
};
// Adjust based on importance
const importanceMultiplier = 1 + (1 - params.importance); // Higher importance = longer interval
const baseDays = baseReviewDays[params.type] ?? 30;
const reviewDays = Math.floor(baseDays * importanceMultiplier);
const nextReview = new Date();
nextReview.setDate(nextReview.getDate() + reviewDays);
return nextReview;
}
+421
View File
@@ -0,0 +1,421 @@
---
name: cross-tier-correlator
description: Discovers and maintains relationships between memories across tiers (episodic, semantic, procedural), enabling unified retrieval and knowledge graph navigation.
---
# Cross-Tier Correlator Lobe
**Purpose:** Build and maintain cross-tier memory relationships for unified AgeMem retrieval.
**Status:** 🟡 Implemented (2026-04-04)
**Type:** Lobe Agent Skill
**Location:** `~/.openclaw/workspace/skills/cross-tier-correlator/`
---
## Overview
The cross-tier correlator lobe provides memory relationship management:
1. **Link Discovery** — Automatically find related memories across tiers
2. **Relationship Types** — Categorize links (references, derives_from, contradicts, etc.)
3. **Correlation Scoring** — Rank relationships by strength
4. **Graph Navigation** — Traverse memory relationships
5. **Unified Retrieval** — Query across all tiers with relationship awareness
This enables the Collective to navigate from experiences (episodic) to knowledge (semantic) to skills (procedural).
---
## Configuration
```bash
# Environment Variables
CORRELATOR_ENABLED="${CORRELATOR_ENABLED:-true}"
MIN_CORRELATION_SCORE="${MIN_CORRELATION_SCORE:-0.6}" # Minimum link strength
MAX_LINKS_PER_MEMORY="${MAX_LINKS_PER_MEMORY:-50}" # Cap on relationships
AUTO_DISCOVER_ENABLED="${AUTO_DISCOVER_ENABLED:-true}" # Auto-discover on add
EMBEDDING_MODEL="${EMBEDDING_MODEL:-all-MiniLM-L6-v2}" # For semantic similarity
```
---
## Relationship Types
| Type | Direction | Description | Example |
|------|-----------|-------------|---------|
| **references** | A → B | Memory A mentions B | Episode references semantic fact |
| **derives_from** | A ← B | A was inferred from B | Semantic derived from episodes |
| **contradicts** | A ↔ B | A conflicts with B | Conflicting information |
| **supports** | A → B | A provides evidence for B | Episode supports semantic claim |
| **generalizes** | A → B | A is general form of B | Procedural generalizes episode |
| **specializes** | A ← B | A is specific instance of B | Episode specializes procedural |
| **temporal_sequence** | A → B | A happened before B | Sequential episodes |
| **causal** | A → B | A caused B | Causal relationship |
---
## API Functions
### `findCorrelations(memoryId, params)`
Finds memories related to a given memory across tiers.
**Signature:**
```typescript
findCorrelations(params: {
memoryId: string;
type: MemoryType;
maxResults?: number;
minScore?: number;
relationshipTypes?: RelationshipType[];
}): Promise<CorrelationResult>
```
**Returns:**
```typescript
{
memoryId: string;
correlations: Array<{
targetId: string;
targetType: MemoryType;
relationshipType: RelationshipType;
score: number;
reason: string;
}>;
totalFound: number;
}
```
---
### `addRelationship(params)`
Creates a relationship between two memories.
**Signature:**
```typescript
addRelationship(params: {
sourceId: string;
targetId: string;
relationshipType: RelationshipType;
score?: number;
metadata?: Record<string, unknown>;
}): Promise<RelationshipResult>
```
---
### `buildCorrelationGraph(params)`
Builds a correlation graph for a set of memories.
**Signature:**
```typescript
buildCorrelationGraph(params: {
memoryIds: string[];
includeTypes?: MemoryType[];
maxDepth?: number;
}): Promise<CorrelationGraph>
```
---
### `discoverLinks(params)`
Automatically discovers potential links using content analysis.
**Signature:**
```typescript
discoverLinks(params: {
memoryId: string;
content: string;
type: MemoryType;
searchSpace?: MemoryType[];
}): Promise<DiscoveredLink[]>
```
---
## Correlation Scoring
```
correlationScore = weightedSum(
semanticSimilarity × 0.40,
coOccurrence × 0.25,
temporalProximity × 0.15,
crossReference × 0.20
)
```
### Semantic Similarity
Uses embedding cosine similarity:
```
similarity = cosine(embedding_A, embedding_B)
```
### Co-Occurrence
Based on shared entities/terms:
```
coOccurrence = |entities_A ∩ entities_B| / |entities_A entities_B|
```
### Temporal Proximity
For episodic memories:
```
temporalScore = e^(-|timestamp_A - timestamp_B| / τ)
where τ = 7 days (time constant)
```
### Cross-Reference
Explicit mentions:
```
crossReference = count(explicit_references) / max_references
```
---
## Usage Examples
### Find Related Memories
```typescript
import { findCorrelations } from './cross-tier-correlator';
const result = await findCorrelations({
memoryId: '550e8400-e29b-41d4-a716-446655440000',
type: 'episodic',
maxResults: 10,
minScore: 0.6
});
console.log(`Found ${result.totalFound} correlations:`);
for (const corr of result.correlations) {
console.log(` - ${corr.targetId} (${corr.targetType}): ${corr.relationshipType}`);
console.log(` Score: ${corr.score}, Reason: ${corr.reason}`);
}
```
### Add Relationship
```typescript
import { addRelationship } from './cross-tier-correlator';
const result = await addRelationship({
sourceId: 'episode-001',
targetId: 'semantic-042',
relationshipType: 'references',
score: 0.85,
metadata: {
discoveredBy: 'auto',
context: 'User mentioned TypeScript preference'
}
});
```
### Discover Links Automatically
```typescript
import { discoverLinks } from './cross-tier-correlator';
const links = await discoverLinks({
memoryId: 'new-episode-001',
content: 'Discussed PostgreSQL pgvector integration for semantic search',
type: 'episodic',
searchSpace: ['semantic', 'procedural']
});
console.log('Discovered links:');
for (const link of links) {
console.log(` ${link.targetId}: ${link.relationshipType} (${link.score})`);
}
```
### Build Correlation Graph
```typescript
import { buildCorrelationGraph } from './cross-tier-correlator';
const graph = await buildCorrelationGraph({
memoryIds: ['mem-001', 'mem-002', 'mem-003'],
includeTypes: ['episodic', 'semantic'],
maxDepth: 2
});
console.log(`Graph: ${graph.nodes.length} nodes, ${graph.edges.length} edges`);
```
---
## Integration with AgeMem
The correlator integrates with AgeMem unified retrieval:
```typescript
import { memory_retrieve } from './decay';
import { findCorrelations } from './cross-tier-correlator';
// Retrieve with cross-tier expansion
async function retrieveWithExpansion(query: string, recencyWeight: number) {
// Base retrieval
const baseResults = await memory_retrieve({ query, recencyWeight });
// Expand top results with correlations
const expanded = [];
for (const result of baseResults.slice(0, 5)) {
const correlations = await findCorrelations({
memoryId: result.id,
type: result.type as MemoryType,
maxResults: 3
});
expanded.push(...correlations.correlations);
}
return {
primary: baseResults,
expanded,
totalResults: baseResults.length + expanded.length
};
}
```
---
## Output Example
```markdown
# Cross-Tier Correlation Report
**Generated:** 2026-04-04T01:30:00Z
**Seed Memory:** episode-2026-04-04-001
## Direct Correlations (5)
| Target | Type | Relationship | Score | Reason |
|--------|------|--------------|-------|--------|
| semantic-ts-pref | semantic | references | 0.92 | Entity match: TypeScript |
| proc-pgvector-setup | procedural | derives_from | 0.85 | Shared context |
| episode-2026-04-03-002 | episodic | temporal_sequence | 0.78 | Sequential session |
| semantic-pgvector | semantic | supports | 0.72 | Content similarity |
| proc-backup-config | procedural | references | 0.65 | Co-occurrence |
## Correlation Graph
```
episode-2026-04-04-001
├── semantic-ts-pref (references, 0.92)
│ └── proc-ts-best-practices (generalizes, 0.81)
├── proc-pgvector-setup (derives_from, 0.85)
│ └── semantic-pgvector (supports, 0.88)
└── episode-2026-04-03-002 (temporal_sequence, 0.78)
└── semantic-a2a-protocol (references, 0.75)
```
## Tier Distribution
| Tier | Count | Avg Score |
|------|-------|-----------|
| Episodic | 12 | 0.74 |
| Semantic | 8 | 0.82 |
| Procedural | 5 | 0.79 |
## Recommendations
1. **Strong link detected** — episode → semantic-ts-pref (0.92)
2. **Potential contradiction** — semantic-001 ↔ semantic-042 (review needed)
3. **Orphan memory** — proc-unused-skill has no correlations
---
*Cross-Tier Correlator — Connecting experiences to knowledge.*
```
---
## Sentinel Agent Considerations
**Security:**
- No modification of memory content (relationships only)
- Relationship scores are suggestions, not enforcement
- Contradictions flagged for review, not auto-resolved
**God Mode Prevention:**
- Cannot delete or modify memories
- Relationships are metadata only
- Requires consensus for relationship enforcement
**Privacy:**
- Correlation data stored locally
- No external embedding API required (local models supported)
- Relationships respect memory access controls
---
## Testing Strategy
### Unit Tests
```typescript
describe('cross-tier-correlator', () => {
it('finds semantic similarity between related memories', async () => {
const result = await findCorrelations({
memoryId: 'test-episode-1',
type: 'episodic',
maxResults: 5
});
expect(result.correlations.length).toBeGreaterThan(0);
});
it('respects minimum correlation score', async () => {
const result = await findCorrelations({
memoryId: 'test-episode-1',
type: 'episodic',
minScore: 0.8
});
for (const corr of result.correlations) {
expect(corr.score).toBeGreaterThanOrEqual(0.8);
}
});
it('discovers cross-tier links', async () => {
const links = await discoverLinks({
memoryId: 'test-ep',
content: 'PostgreSQL pgvector setup',
type: 'episodic',
searchSpace: ['semantic']
});
expect(links.some(l => l.targetType === 'semantic')).toBe(true);
});
});
```
---
## Related Components
| Component | Relationship |
|-----------|--------------|
| **AgeMem** | Provides unified retrieval with correlations |
| **Memory Consolidation** | Uses correlations for promotion decisions |
| **Importance Scorer** | Cross-references boost importance |
| **Archivist** | Preserves relationships during transitions |
| **Historian Agent** — | Analyzes correlation patterns |
---
## Future Enhancements
1. **Embedding Cache** — Cache embeddings for faster similarity
2. **Real-time Correlation** — Stream processing for live updates
3. **Graph Neural Network** — ML-based link prediction
4. **Temporal Reasoning** — Time-aware correlation patterns
5. **Contradiction Resolution** — Auto-flag conflicting memories
---
*Cross-Tier Correlator — Because knowledge is connected.*
@@ -0,0 +1,607 @@
/**
* Cross-Tier Correlator Lobe
*
* Discovers and maintains relationships between memories across tiers:
* - Episodic (experiences) ↔ Semantic (facts) ↔ Procedural (skills)
* - Link discovery using content analysis and semantic similarity
* - Correlation scoring and ranking
* - Graph navigation and traversal
*
* @module cross-tier-correlator
* @see {@link ../memory-consolidation/decay.ts} for AgeMem integration
*/
/** Memory type for correlation */
export type MemoryType = 'working' | 'episodic' | 'semantic' | 'procedural' | 'archival';
/** Relationship types between memories */
export type RelationshipType =
| 'references'
| 'derives_from'
| 'contradicts'
| 'supports'
| 'generalizes'
| 'specializes'
| 'temporal_sequence'
| 'causal';
/** Correlator configuration */
export interface CorrelatorConfig {
/** Enable correlation */
enabled: boolean;
/** Minimum correlation score threshold */
minCorrelationScore: number;
/** Maximum links per memory */
maxLinksPerMemory: number;
/** Enable auto-discovery on memory add */
autoDiscoverEnabled: boolean;
/** Time constant for temporal proximity (days) */
temporalTimeConstant: number;
}
/** Default configuration */
export const DEFAULT_CORRELATOR_CONFIG: CorrelatorConfig = {
enabled: true,
minCorrelationScore: 0.6,
maxLinksPerMemory: 50,
autoDiscoverEnabled: true,
temporalTimeConstant: 7, // 7 days
};
/** A discovered correlation between memories */
export interface Correlation {
/** Target memory ID */
targetId: string;
/** Target memory type */
targetType: MemoryType;
/** Type of relationship */
relationshipType: RelationshipType;
/** Correlation score (0-1) */
score: number;
/** Human-readable reason */
reason: string;
/** Optional metadata */
metadata?: Record<string, unknown>;
}
/** Result of correlation search */
export interface CorrelationResult {
/** Source memory ID */
memoryId: string;
/** Found correlations */
correlations: Correlation[];
/** Total found (before limiting) */
totalFound: number;
}
/** Result of relationship creation */
export interface RelationshipResult {
/** Whether operation succeeded */
success: boolean;
/** Source memory ID */
sourceId: string;
/** Target memory ID */
targetId: string;
/** Relationship type */
relationshipType: RelationshipType;
/** Relationship score */
score: number;
/** Timestamp */
timestamp: string;
/** Optional error message */
error?: string;
}
/** A node in the correlation graph */
export interface GraphNode {
/** Memory ID */
id: string;
/** Memory type */
type: MemoryType;
/** Optional content snippet */
content?: string;
}
/** An edge in the correlation graph */
export interface GraphEdge {
/** Source node ID */
source: string;
/** Target node ID */
target: string;
/** Relationship type */
relationshipType: RelationshipType;
/** Edge weight (correlation score) */
weight: number;
}
/** Correlation graph structure */
export interface CorrelationGraph {
/** Graph nodes */
nodes: GraphNode[];
/** Graph edges */
edges: GraphEdge[];
/** Metadata about the graph */
metadata: {
/** Number of nodes */
nodeCount: number;
/** Number of edges */
edgeCount: number;
/** Average edge weight */
avgWeight: number;
};
}
/** Discovered link from content analysis */
export interface DiscoveredLink {
/** Target memory ID */
targetId: string;
/** Target memory type */
targetType: MemoryType;
/** Relationship type */
relationshipType: RelationshipType;
/** Confidence score */
score: number;
/** Reason for discovery */
reason: string;
}
/** Parameters for correlation search */
export interface CorrelationSearchParams {
/** Source memory ID */
memoryId: string;
/** Source memory type */
type: MemoryType;
/** Maximum results to return */
maxResults?: number;
/** Minimum score threshold */
minScore?: number;
/** Filter by relationship types */
relationshipTypes?: RelationshipType[];
/** Override config */
config?: Partial<CorrelatorConfig>;
}
/** Parameters for link discovery */
export interface LinkDiscoveryParams {
/** Memory ID being analyzed */
memoryId: string;
/** Memory content */
content: string;
/** Memory type */
type: MemoryType;
/** Search space (memory types to search) */
searchSpace?: MemoryType[];
}
/**
* Extracts entities from content for correlation analysis
*/
export function extractEntities(content: string): string[] {
const entities: Set<string> = new Set();
// Technical terms (camelCase, snake_case)
const techPattern = /[a-z]+(?:[A-Z][a-z]+)+|[a-z]+_[a-z]+/g;
const techMatches = content.match(techPattern);
if (techMatches) {
techMatches.forEach(e => entities.add(e.toLowerCase()));
}
// Capitalized words (potential named entities)
const capitalizedPattern = /\b[A-Z][a-z]+(?:\s+[A-Z][a-z]+)*\b/g;
const capitalizedMatches = content.match(capitalizedPattern);
if (capitalizedMatches) {
capitalizedMatches.forEach(e => {
// Filter out common words
const lower = e.toLowerCase();
if (!['The', 'A', 'An', 'And', 'Or', 'But'].includes(lower)) {
entities.add(e);
}
});
}
// Version numbers, protocols
const versionPattern = /v?\d+\.\d+(?:\.\d+)?|[A-Z]{2,}/g;
const versionMatches = content.match(versionPattern);
if (versionMatches) {
versionMatches.forEach(e => entities.add(e));
}
return Array.from(entities);
}
/**
* Calculates Jaccard similarity between two sets
*/
export function jaccardSimilarity<T>(setA: Set<T>, setB: Set<T>): number {
if (setA.size === 0 && setB.size === 0) {
return 0;
}
const intersection = new Set([...setA].filter(x => setB.has(x)));
const union = new Set([...setA, ...setB]);
return intersection.size / union.size;
}
/**
* Calculates semantic similarity using cosine similarity approximation
*
* In production, this would use actual embeddings. This is a
* content-based approximation using term frequency.
*/
export function calculateSemanticSimilarity(contentA: string, contentB: string): number {
// Tokenize and normalize
const tokenize = (text: string): string[] =>
text.toLowerCase().split(/\s+/).filter(w => w.length > 3);
const tokensA = tokenize(contentA);
const tokensB = tokenize(contentB);
// Create term frequency vectors
const tfA: Record<string, number> = {};
const tfB: Record<string, number> = {};
tokensA.forEach(t => tfA[t] = (tfA[t] || 0) + 1);
tokensB.forEach(t => tfB[t] = (tfB[t] || 0) + 1);
// Calculate cosine similarity
const allTerms = new Set([...Object.keys(tfA), ...Object.keys(tfB)]);
let dotProduct = 0;
let normA = 0;
let normB = 0;
for (const term of allTerms) {
const a = tfA[term] || 0;
const b = tfB[term] || 0;
dotProduct += a * b;
normA += a * a;
normB += b * b;
}
if (normA === 0 || normB === 0) {
return 0;
}
return dotProduct / (Math.sqrt(normA) * Math.sqrt(normB));
}
/**
* Calculates temporal proximity score
*/
export function calculateTemporalProximity(
timestampA: Date,
timestampB: Date,
timeConstantDays: number = 7
): number {
const diffMs = Math.abs(timestampA.getTime() - timestampB.getTime());
const diffDays = diffMs / (1000 * 60 * 60 * 24);
// Exponential decay: e^(-|t1-t2|/τ)
return Math.exp(-diffDays / timeConstantDays);
}
/**
* Calculates overall correlation score
*/
export function calculateCorrelationScore(params: {
semanticSimilarity: number;
coOccurrence: number;
temporalProximity?: number;
crossReference: number;
weights?: {
semantic: number;
coOccurrence: number;
temporal: number;
crossReference: number;
};
}): number {
const weights = params.weights ?? {
semantic: 0.40,
coOccurrence: 0.25,
temporal: 0.15,
crossReference: 0.20,
};
const temporal = params.temporalProximity ?? 0.5; // Default if not provided
const score =
params.semanticSimilarity * weights.semantic +
params.coOccurrence * weights.coOccurrence +
temporal * weights.temporal +
params.crossReference * weights.crossReference;
return Math.max(0, Math.min(1, score));
}
/**
* Determines relationship type based on content analysis
*/
export function determineRelationshipType(params: {
sourceType: MemoryType;
targetType: MemoryType;
sourceContent: string;
targetContent: string;
entitiesA: string[];
entitiesB: string[];
}): RelationshipType {
const { sourceType, targetType } = params;
// Check for explicit references
const refPatterns = [
/refer(ence|s|ring)/i,
/mention(ed|s)?/i,
/see also/i,
/as stated/i,
];
const combinedContent = `${params.sourceContent} ${params.targetContent}`;
const hasExplicitReference = refPatterns.some(p => p.test(combinedContent));
if (hasExplicitReference) {
return 'references';
}
// Episodic → Semantic: usually derives_from or supports
if (sourceType === 'episodic' && targetType === 'semantic') {
return 'supports';
}
// Semantic → Episodic: usually references
if (sourceType === 'semantic' && targetType === 'episodic') {
return 'references';
}
// Procedural ← Episodic: generalizes
if (sourceType === 'procedural' && targetType === 'episodic') {
return 'generalizes';
}
// Episodic → Procedural: specializes
if (sourceType === 'episodic' && targetType === 'procedural') {
return 'specializes';
}
// Same type: temporal sequence for episodic
if (sourceType === targetType && sourceType === 'episodic') {
return 'temporal_sequence';
}
// Default
return 'references';
}
/**
* Finds memories related to a given memory across tiers
*
* @param params - Correlation search parameters
* @returns Correlation result with found relationships
*
* @example
* ```typescript
* const result = await findCorrelations({
* memoryId: '550e8400-e29b-41d4-a716-446655440000',
* type: 'episodic',
* maxResults: 10,
* minScore: 0.6
* });
* ```
*/
export async function findCorrelations(
params: CorrelationSearchParams,
): Promise<CorrelationResult> {
const config = { ...DEFAULT_CORRELATOR_CONFIG, ...params.config };
if (!config.enabled) {
return {
memoryId: params.memoryId,
correlations: [],
totalFound: 0,
};
}
// In production, this would:
// 1. Query memory store for candidate memories
// 2. Calculate correlations with each candidate
// 3. Filter by minScore and relationshipTypes
// 4. Sort by score and limit to maxResults
// Reference implementation (simulated)
const correlations: Correlation[] = [];
// Simulated correlations for demonstration
console.log(`[Cross-Tier Correlator] Finding correlations for ${params.memoryId}`);
return {
memoryId: params.memoryId,
correlations,
totalFound: 0,
};
}
/**
* Creates a relationship between two memories
*
* @param params - Relationship creation parameters
* @returns Relationship result
*
* @example
* ```typescript
* const result = await addRelationship({
* sourceId: 'episode-001',
* targetId: 'semantic-042',
* relationshipType: 'references',
* score: 0.85
* });
* ```
*/
export async function addRelationship(params: {
sourceId: string;
targetId: string;
relationshipType: RelationshipType;
score?: number;
metadata?: Record<string, unknown>;
}): Promise<RelationshipResult> {
const timestamp = new Date().toISOString();
const score = params.score ?? 0.5;
// In production, this would:
// 1. Validate source and target exist
// 2. Check for existing relationship
// 3. Insert relationship into store
// 4. Update memory metadata
console.log(
`[Cross-Tier Correlator] Adding relationship: ${params.sourceId} --[${params.relationshipType}:${score}]--> ${params.targetId}`
);
return {
success: true,
sourceId: params.sourceId,
targetId: params.targetId,
relationshipType: params.relationshipType,
score,
timestamp,
};
}
/**
* Automatically discovers potential links using content analysis
*
* @param params - Link discovery parameters
* @returns Array of discovered links
*
* @example
* ```typescript
* const links = await discoverLinks({
* memoryId: 'new-episode-001',
* content: 'Discussed PostgreSQL pgvector integration',
* type: 'episodic',
* searchSpace: ['semantic', 'procedural']
* });
* ```
*/
export async function discoverLinks(
params: LinkDiscoveryParams,
): Promise<DiscoveredLink[]> {
const config = DEFAULT_CORRELATOR_CONFIG;
if (!config.enabled) {
return [];
}
// Extract entities from content
const entities = extractEntities(params.content);
const searchSpace = params.searchSpace ?? ['episodic', 'semantic', 'procedural'];
const links: DiscoveredLink[] = [];
// In production, this would:
// 1. Query memory store for memories with matching entities
// 2. Calculate correlation scores
// 3. Filter by minCorrelationScore
// 4. Return ranked links
console.log(
`[Cross-Tier Correlator] Discovering links for ${params.memoryId}, entities: ${entities.join(', ')}`
);
return links;
}
/**
* Builds a correlation graph for a set of memories
*
* @param params - Graph building parameters
* @returns Correlation graph
*
* @example
* ```typescript
* const graph = await buildCorrelationGraph({
* memoryIds: ['mem-001', 'mem-002', 'mem-003'],
* includeTypes: ['episodic', 'semantic'],
* maxDepth: 2
* });
* ```
*/
export async function buildCorrelationGraph(params: {
memoryIds: string[];
includeTypes?: MemoryType[];
maxDepth?: number;
}): Promise<CorrelationGraph> {
const nodes: GraphNode[] = params.memoryIds.map(id => ({
id,
type: 'episodic', // Would be determined from memory store
}));
const edges: GraphEdge[] = [];
// In production, this would:
// 1. Get correlations for each memory
// 2. Build graph structure
// 3. Traverse to maxDepth
// 4. Calculate metadata
const edgeCount = edges.length;
const avgWeight = edges.length > 0
? edges.reduce((sum, e) => sum + e.weight, 0) / edges.length
: 0;
return {
nodes,
edges,
metadata: {
nodeCount: nodes.length,
edgeCount,
avgWeight,
},
};
}
/**
* Generates a correlation report for monitoring
*/
export function generateCorrelationReport(params: {
memoryId: string;
correlations: Correlation[];
}): string {
const report: string[] = [
'# Cross-Tier Correlation Report',
`**Generated:** ${new Date().toISOString()}`,
`**Seed Memory:** ${params.memoryId}`,
'',
`## Direct Correlations (${params.correlations.length})`,
'',
'| Target | Type | Relationship | Score | Reason |',
'|--------|------|--------------|-------|--------|',
];
for (const corr of params.correlations) {
report.push(
`| ${corr.targetId.substring(0, 8)}... | ${corr.targetType} | ${corr.relationshipType} | ${corr.score.toFixed(2)} | ${corr.reason} |`
);
}
// Tier distribution
const tierCounts: Record<MemoryType, number> = {
working: 0,
episodic: 0,
semantic: 0,
procedural: 0,
archival: 0,
};
params.correlations.forEach(c => tierCounts[c.targetType]++);
report.push('', '## Tier Distribution', '');
report.push('| Tier | Count |');
report.push('|------|-------|');
for (const [tier, count] of Object.entries(tierCounts)) {
if (count > 0) {
report.push(`| ${tier} | ${count} |`);
}
}
return report.join('\n');
}
+454
View File
@@ -0,0 +1,454 @@
---
name: importance-scorer
description: Calculates and assigns importance scores to memories based on content analysis, user signals, access patterns, and emotional salience. Use when adding new memories or re-evaluating existing ones.
---
# Importance Scorer Lobe
**Purpose:** Assign accurate importance scores (0-1) to memories using multi-factor analysis.
**Status:** 🟡 Implemented (2026-04-04)
**Type:** Lobe Agent Skill
**Location:** `~/.openclaw/workspace/skills/importance-scorer/`
---
## Overview
The importance scorer lobe is a specialized agent skill that calculates memory importance scores using a weighted combination of signals:
1. **Content Analysis** - Semantic richness, uniqueness, factual density
2. **User Signals** - Explicit importance ratings, user feedback
3. **Access Patterns** - Frequency, recency, cross-references
4. **Emotional Salience** - Sentiment intensity, emotional markers (Empath integration)
5. **Contextual Relevance** - Current goals, active projects, temporal relevance
This ensures memories are weighted appropriately for AgeMem retrieval with Ebbinghaus decay.
---
## Configuration
```bash
# Environment Variables
IMPORTANCE_SCORER_ENABLED="${IMPORTANCE_SCORER_ENABLED:-true}"
CONTENT_WEIGHT="${CONTENT_WEIGHT:-0.30}" # Content analysis weight
USER_SIGNAL_WEIGHT="${USER_SIGNAL_WEIGHT:-0.25}" # User-provided signals weight
ACCESS_PATTERN_WEIGHT="${ACCESS_PATTERN_WEIGHT:-0.20}" # Access history weight
EMOTIONAL_WEIGHT="${EMOTIONAL_WEIGHT:-0.15}" # Emotional salience weight
CONTEXTUAL_WEIGHT="${CONTEXTUAL_WEIGHT:-0.10}" # Contextual relevance weight
MIN_IMPORTANCE="${MIN_IMPORTANCE:-0.1}" # Floor value
MAX_IMPORTANCE="${MAX_IMPORTANCE:-1.0}" # Cap value
```
---
## Importance Score Formula
```
importance = normalize(
(content_score × CONTENT_WEIGHT) +
(user_signal × USER_SIGNAL_WEIGHT) +
(access_score × ACCESS_PATTERN_WEIGHT) +
(emotional_score × EMOTIONAL_WEIGHT) +
(contextual_score × CONTEXTUAL_WEIGHT)
)
Where:
- All component scores are normalized to 0-1 range
- Weights sum to 1.0
- Final result is clamped to [MIN_IMPORTANCE, MAX_IMPORTANCE]
```
---
## API Functions
### `calculateImportance(params)`
Calculates importance score for a memory candidate.
**Signature:**
```typescript
calculateImportance(params: {
content: string;
type?: MemoryType;
userProvidedImportance?: number;
accessCount?: number;
recencyScore?: number;
emotionalScore?: number;
contextualRelevance?: number;
metadata?: Record<string, unknown>;
}): Promise<{
score: number;
breakdown: ImportanceBreakdown;
confidence: number;
factors: string[];
}>
```
**Parameters:**
| Parameter | Type | Description |
|-----------|------|-------------|
| `content` | string | Memory content to analyze |
| `type` | MemoryType | Memory type (affects scoring) |
| `userProvidedImportance` | number | Explicit user rating (0-1) |
| `accessCount` | number | Number of times accessed |
| `recencyScore` | number | Recency factor (0-1) |
| `emotionalScore` | number | Emotional salience (0-1) |
| `contextualRelevance` | number | Current context match (0-1) |
| `metadata` | Record | Additional context |
**Returns:**
```typescript
{
score: number; // Final importance (0-1)
breakdown: {
content: number; // Content analysis score
userSignal: number; // User-provided score
access: number; // Access pattern score
emotional: number; // Emotional salience score
contextual: number; // Contextual relevance score
};
confidence: number; // Confidence in score (0-1)
factors: string[]; // Key factors influencing score
}
```
---
## Content Analysis Factors
The content analyzer evaluates:
| Factor | Description | Weight |
|--------|-------------|--------|
| **Semantic Density** | Ratio of meaningful terms to total words | 0.20 |
| **Entity Count** | Named entities, concepts, technical terms | 0.15 |
| **Uniqueness** | Novelty compared to existing memories | 0.20 |
| **Actionability** | Contains instructions, decisions, tasks | 0.15 |
| **Factual Content** | Verifiable facts, data, specifications | 0.15 |
| **Cross-References** | Links to other memories/concepts | 0.15 |
### Content Score Calculation
```typescript
contentScore = normalize(
semanticDensity × 0.20 +
entityCount × 0.15 +
uniqueness × 0.20 +
actionability × 0.15 +
factualContent × 0.15 +
crossReferences × 0.15
)
```
---
## Access Pattern Scoring
Access patterns indicate memory value through usage:
```typescript
accessScore = normalize(
log10(accessCount + 1) / log10(maxAccessCount + 1) × 0.6 +
recencyScore × 0.4
)
```
**Access Tiers:**
| Access Count | Score Contribution |
|--------------|-------------------|
| 0 | 0.0 |
| 1-2 | 0.2 |
| 3-5 | 0.4 |
| 6-10 | 0.6 |
| 11-20 | 0.8 |
| 21+ | 1.0 |
---
## Emotional Salience (Empath Integration)
When Empath agent is available, emotional score is calculated:
```typescript
emotionalScore = normalize(
sentimentIntensity × 0.4 +
emotionalMarkers × 0.3 +
personalRelevance × 0.3
)
```
**Emotional Markers:**
- First-person statements ("I prefer", "my approach")
- Value judgments ("important", "critical", "avoid")
- Preference expressions ("like", "dislike", "prefer")
- Decision points ("decided", "chose", "concluded")
---
## Memory Type Adjustments
Different memory types have different baseline importance:
| Type | Base Importance | Rationale |
|------|-----------------|-----------|
| **Working** | 0.3 | Temporary, session-only |
| **Episodic** | 0.5 | Experience record, variable value |
| **Semantic** | 0.7 | Factual knowledge, higher value |
| **Procedural** | 0.8 | Skills and methods, high utility |
| **Archival** | 0.6 | Historical record, moderate value |
---
## Usage Examples
### Basic Importance Calculation
```typescript
import { calculateImportance } from './importance-scorer';
const result = await calculateImportance({
content: "User prefers TypeScript over JavaScript for type safety",
type: "semantic",
userProvidedImportance: 0.9
});
console.log(`Importance: ${result.score}`);
console.log(`Breakdown:`, result.breakdown);
console.log(`Factors:`, result.factors);
```
### With Full Context
```typescript
const result = await calculateImportance({
content: "Decision: Use PostgreSQL with pgvector for semantic search",
type: "semantic",
userProvidedImportance: 0.85,
accessCount: 12,
recencyScore: 0.9,
emotionalScore: 0.6,
contextualRelevance: 0.95,
metadata: {
project: "AgeMem",
decisionType: "architecture",
alternatives: ["Redis", "Pinecone", "Weaviate"]
}
});
// Result:
// {
// score: 0.88,
// breakdown: {
// content: 0.85,
// userSignal: 0.85,
// access: 0.72,
// emotional: 0.60,
// contextual: 0.95
// },
// confidence: 0.92,
// factors: [
// "High user-provided importance",
// "Strong contextual relevance",
// "Frequent access pattern",
// "Architectural decision content"
// ]
// }
```
### Batch Scoring
```typescript
import { batchCalculateImportance } from './importance-scorer';
const memories = [
{ content: "User prefers dark mode", type: "semantic" },
{ content: "Meeting notes from 2024-01-15", type: "episodic" },
{ content: "How to deploy to Kubernetes", type: "procedural" }
];
const results = await batchCalculateImportance(memories);
```
---
## Integration with AgeMem
The importance scorer integrates directly with the AgeMem `memory_add()` API:
```typescript
import { memory_add } from './decay';
import { calculateImportance } from './importance-scorer';
// Calculate importance before adding memory
const importanceResult = await calculateImportance({
content: "User's preferred development workflow",
type: "semantic",
accessCount: 5
});
// Add memory with calculated importance
const memoryResult = await memory_add({
content: "User's preferred development workflow",
type: "semantic",
importance: importanceResult.score, // Use calculated score
metadata: {
importanceBreakdown: importanceResult.breakdown,
confidence: importanceResult.confidence
}
});
```
---
## Confidence Calculation
Confidence indicates reliability of the importance score:
```typescript
confidence = normalize(
(hasUserSignal ? 0.4 : 0) +
(contentQuality × 0.3) +
(dataCompleteness × 0.3)
)
```
**Confidence Tiers:**
| Confidence | Meaning |
|------------|---------|
| 0.8-1.0 | High confidence - multiple strong signals |
| 0.6-0.8 | Moderate confidence - adequate signals |
| 0.4-0.6 | Low confidence - limited signals |
| 0.0-0.4 | Very low confidence - guess based on defaults |
---
## Sentinel Agent Considerations
**Security:**
- No external API calls without consent
- Content analysis is local-only
- Emotional scoring requires Empath agent opt-in
**God Mode Prevention:**
- Importance scores are suggestions, not enforcement
- User can always override calculated scores
- Scores decay naturally via Ebbinghaus curve
**Privacy:**
- Content not stored externally
- Emotional analysis opt-in only
- Access patterns tracked locally
---
## Output Example
```markdown
# Importance Scorer Report
**Memory:** "Decision: Use PostgreSQL with pgvector for semantic search"
**Type:** semantic
**Timestamp:** 2026-04-04T01:20:00Z
## Score Breakdown
| Factor | Score | Weight | Contribution |
|--------|-------|--------|--------------|
| Content Analysis | 0.85 | 30% | 0.255 |
| User Signal | 0.85 | 25% | 0.213 |
| Access Pattern | 0.72 | 20% | 0.144 |
| Emotional Salience | 0.60 | 15% | 0.090 |
| Contextual Relevance | 0.95 | 10% | 0.095 |
**Final Score:** 0.88 (clamped to [0.1, 1.0])
**Confidence:** 0.92
## Key Factors
1. ✅ High user-provided importance (0.85)
2. ✅ Strong contextual relevance (0.95)
3. ✅ Frequent access pattern (12 accesses)
4. ✅ Architectural decision content
5. ⚠️ Moderate emotional salience
## Recommendation
**HIGH IMPORTANCE** - This memory should be:
- Stored in semantic memory tier
- Given long half-life (30 days)
- Prioritized in retrieval operations
- Considered for cross-referencing
```
---
## Testing Strategy
### Unit Tests
```typescript
describe('importance-scorer', () => {
it('calculates score with user-provided importance', async () => {
const result = await calculateImportance({
content: 'Test content',
userProvidedImportance: 0.9
});
expect(result.score).toBeCloseTo(0.9, 1);
});
it('handles missing signals gracefully', async () => {
const result = await calculateImportance({
content: 'Test content'
});
expect(result.score).toBeGreaterThanOrEqual(0.1);
expect(result.confidence).toBeLessThan(0.5);
});
it('respects memory type baselines', async () => {
const procedural = await calculateImportance({
content: 'How to do something',
type: 'procedural'
});
const working = await calculateImportance({
content: 'How to do something',
type: 'working'
});
expect(procedural.score).toBeGreaterThan(working.score);
});
});
```
### Integration Tests
- End-to-end with AgeMem `memory_add()`
- Empath agent emotional scoring integration
- Cross-agent importance consensus
---
## Related Components
| Component | Relationship |
|-----------|--------------|
| **AgeMem** | Consumer of importance scores |
| **Memory Consolidation** | Uses importance for promotion decisions |
| **Empath Agent** | Provides emotional salience scoring |
| **Historian Agent** | Reviews importance trends |
| **Sentinel Agent** | Audits scoring fairness |
---
## Future Enhancements
1. **Learning Weights** - Adapt factor weights based on user feedback
2. **Domain-Specific Scoring** - Different weights for code vs. conversation vs. decisions
3. **Temporal Patterns** - Boost importance for recurring themes
4. **Cross-Memory Validation** - Compare with related memories for consistency
5. **User Calibration** - Learn individual user importance patterns
---
*Importance Scorer - Because not all memories are created equal.*
@@ -0,0 +1,491 @@
/**
* Importance Scorer Lobe
*
* Calculates and assigns importance scores to memories based on:
* - Content analysis (semantic density, entities, uniqueness)
* - User signals (explicit ratings, feedback)
* - Access patterns (frequency, recency)
* - Emotional salience (sentiment, emotional markers)
* - Contextual relevance (current goals, projects)
*
* @module importance-scorer
* @see {@link ../memory-consolidation/decay.ts} for AgeMem integration
*/
/** Memory type for importance scoring */
export type MemoryType = 'working' | 'episodic' | 'semantic' | 'procedural' | 'archival';
/** Importance scoring configuration */
export interface ImportanceScorerConfig {
/** Enable content analysis */
contentWeight: number;
/** Enable user signal weighting */
userSignalWeight: number;
/** Enable access pattern scoring */
accessPatternWeight: number;
/** Enable emotional salience scoring */
emotionalWeight: number;
/** Enable contextual relevance scoring */
contextualWeight: number;
/** Minimum importance score (floor) */
minImportance: number;
/** Maximum importance score (cap) */
maxImportance: number;
}
/** Default configuration - weights sum to 1.0 */
export const DEFAULT_IMPORTANCE_SCORER_CONFIG: ImportanceScorerConfig = {
contentWeight: 0.30,
userSignalWeight: 0.25,
accessPatternWeight: 0.20,
emotionalWeight: 0.15,
contextualWeight: 0.10,
minImportance: 0.1,
maxImportance: 1.0,
};
/** Content analysis breakdown */
export interface ContentAnalysisResult {
/** Semantic density (0-1) */
semanticDensity: number;
/** Entity count score (0-1) */
entityCount: number;
/** Uniqueness score (0-1) */
uniqueness: number;
/** Actionability score (0-1) */
actionability: number;
/** Factual content score (0-1) */
factualContent: number;
/** Cross-reference score (0-1) */
crossReferences: number;
/** Overall content score (0-1) */
overallScore: number;
}
/** Importance breakdown by factor */
export interface ImportanceBreakdown {
/** Content analysis score */
content: number;
/** User-provided signal score */
userSignal: number;
/** Access pattern score */
access: number;
/** Emotional salience score */
emotional: number;
/** Contextual relevance score */
contextual: number;
}
/** Importance scoring result */
export interface ImportanceResult {
/** Final importance score (0-1) */
score: number;
/** Breakdown by factor */
breakdown: ImportanceBreakdown;
/** Confidence in score (0-1) */
confidence: number;
/** Key factors influencing score */
factors: string[];
/** Content analysis details */
contentAnalysis?: ContentAnalysisResult;
}
/** Parameters for importance calculation */
export interface ImportanceParams {
/** Memory content to analyze */
content: string;
/** Memory type (affects baseline) */
type?: MemoryType;
/** Explicit user rating (0-1) */
userProvidedImportance?: number;
/** Number of times accessed */
accessCount?: number;
/** Recency factor (0-1) */
recencyScore?: number;
/** Emotional salience (0-1) */
emotionalScore?: number;
/** Current context match (0-1) */
contextualRelevance?: number;
/** Additional context */
metadata?: Record<string, unknown>;
/** Override config */
config?: Partial<ImportanceScorerConfig>;
}
/** Base importance by memory type */
const MEMORY_TYPE_BASELINE: Record<MemoryType, number> = {
working: 0.3,
episodic: 0.5,
semantic: 0.7,
procedural: 0.8,
archival: 0.6,
};
/**
* Analyzes content for semantic density, entities, and other factors
*/
export function analyzeContent(content: string): ContentAnalysisResult {
if (!content || content.trim().length === 0) {
return {
semanticDensity: 0,
entityCount: 0,
uniqueness: 0.5, // Neutral default
actionability: 0,
factualContent: 0,
crossReferences: 0,
overallScore: 0,
};
}
const words = content.split(/\s+/).filter(w => w.length > 0);
const wordCount = words.length;
// Semantic density: ratio of meaningful words (longer words tend to be more meaningful)
const meaningfulWords = words.filter(w => w.length >= 5);
const semanticDensity = wordCount > 0 ? meaningfulWords.length / wordCount : 0;
// Entity count: look for capitalized words, technical terms, numbers
const entityPattern = /[A-Z][a-z]+|[A-Z]{2,}|\d+\.?\d*|[a-z_]+\.[a-z_]+/g;
const entities = content.match(entityPattern) || [];
const entityCount = Math.min(1, entities.length / Math.max(1, wordCount) * 10);
// Actionability: detect instructions, decisions, tasks
const actionPatterns = [
/\b(should|must|need to|have to|decided|will|shall)\b/i,
/\b(do|make|create|build|implement|add|remove|fix|change)\b/i,
/\b(task|action|step|todo|decision|conclusion)\b/i,
/:$/m, // List items
];
const actionabilityScore = actionPatterns.reduce((score, pattern) => {
return score + (pattern.test(content) ? 0.25 : 0);
}, 0);
const actionability = Math.min(1, actionabilityScore);
// Factual content: detect data, specifications, verifiable statements
const factualPatterns = [
/\d+%|\d+\/\d+|\d+-\d+/g, // Numbers, ratios, ranges
/\b(version|v\d+|release|specification|protocol|API)\b/i,
/\b(is|are|was|were|has|have|contains?)\b/i, // Declarative statements
];
const factualScore = factualPatterns.reduce((score, pattern) => {
return score + (pattern.test(content) ? 0.33 : 0);
}, 0);
const factualContent = Math.min(1, factualScore);
// Cross-references: detect links to other concepts
const crossRefPatterns = [
/\[[^\]]+\]\([^\)]+\)/, // Markdown links
/`[^`]+`/, // Code references
/\b(as mentioned|see also|refer to|related to|cf\.)\b/i,
];
const crossRefCount = crossRefPatterns.reduce((count, pattern) => {
return count + (pattern.test(content) ? 1 : 0);
}, 0);
const crossReferences = Math.min(1, crossRefCount / 3);
// Uniqueness: simple heuristic based on vocabulary diversity
const uniqueWords = new Set(words.map(w => w.toLowerCase()));
const vocabularyDiversity = wordCount > 0 ? uniqueWords.size / wordCount : 0;
// Higher diversity suggests more unique content
const uniqueness = Math.min(1, vocabularyDiversity);
// Calculate overall content score with weighted factors
const overallScore =
semanticDensity * 0.20 +
entityCount * 0.15 +
uniqueness * 0.20 +
actionability * 0.15 +
factualContent * 0.15 +
crossReferences * 0.15;
return {
semanticDensity: normalize(semanticDensity),
entityCount: normalize(entityCount),
uniqueness: normalize(uniqueness),
actionability: normalize(actionability),
factualContent: normalize(factualContent),
crossReferences: normalize(crossReferences),
overallScore: normalize(overallScore),
};
}
/**
* Calculates access pattern score based on usage
*/
export function calculateAccessScore(params: {
accessCount?: number;
recencyScore?: number;
}): number {
const accessCount = params.accessCount ?? 0;
const recencyScore = params.recencyScore ?? 0;
// Logarithmic scaling for access count (diminishing returns)
const maxAccessCount = 21; // Cap for normalization
const accessComponent =
Math.log10(accessCount + 1) / Math.log10(maxAccessCount + 1);
// Weighted combination: 60% access frequency, 40% recency
const score = accessComponent * 0.6 + recencyScore * 0.4;
return normalize(score);
}
/**
* Calculates confidence in the importance score
*/
export function calculateConfidence(params: {
hasUserSignal: boolean;
contentQuality: number;
dataCompleteness: number;
}): number {
const { hasUserSignal, contentQuality, dataCompleteness } = params;
const userSignalComponent = hasUserSignal ? 0.4 : 0;
const contentComponent = contentQuality * 0.3;
const completenessComponent = dataCompleteness * 0.3;
return normalize(userSignalComponent + contentComponent + completenessComponent);
}
/**
* Generates explanatory factors for the score
*/
export function generateFactors(
breakdown: ImportanceBreakdown,
contentAnalysis?: ContentAnalysisResult,
): string[] {
const factors: string[] = [];
if (breakdown.userSignal >= 0.8) {
factors.push('High user-provided importance');
} else if (breakdown.userSignal >= 0.5) {
factors.push('Moderate user-provided importance');
}
if (breakdown.contextual >= 0.8) {
factors.push('Strong contextual relevance');
}
if (breakdown.access >= 0.7) {
factors.push('Frequent access pattern');
} else if (breakdown.access <= 0.2) {
factors.push('Low access frequency');
}
if (contentAnalysis) {
if (contentAnalysis.actionability >= 0.7) {
factors.push('Actionable content (instructions/decisions)');
}
if (contentAnalysis.factualContent >= 0.7) {
factors.push('High factual content');
}
if (contentAnalysis.semanticDensity >= 0.7) {
factors.push('Dense semantic content');
}
}
if (breakdown.emotional >= 0.7) {
factors.push('High emotional salience');
}
if (factors.length === 0) {
factors.push('Default importance based on memory type');
}
return factors;
}
/**
* Normalizes a value to 0-1 range
*/
export function normalize(value: number): number {
if (!Number.isFinite(value)) {
return 0;
}
return Math.max(0, Math.min(1, value));
}
/**
* Calculates importance score for a memory candidate
*
* @param params - Importance calculation parameters
* @returns Importance result with score, breakdown, and confidence
*
* @example
* ```typescript
* const result = await calculateImportance({
* content: "User prefers TypeScript over JavaScript",
* type: "semantic",
* userProvidedImportance: 0.9
* });
*
* console.log(`Importance: ${result.score}`);
* console.log(`Factors: ${result.factors.join(', ')}`);
* ```
*/
export async function calculateImportance(
params: ImportanceParams,
): Promise<ImportanceResult> {
const config = { ...DEFAULT_IMPORTANCE_SCORER_CONFIG, ...params.config };
// Analyze content
const contentAnalysis = analyzeContent(params.content);
// Calculate user signal score
const userSignalScore =
params.userProvidedImportance !== undefined
? normalize(params.userProvidedImportance)
: MEMORY_TYPE_BASELINE[params.type ?? 'episodic'];
// Calculate access pattern score
const accessScore = calculateAccessScore({
accessCount: params.accessCount,
recencyScore: params.recencyScore,
});
// Get emotional score (would integrate with Empath agent in production)
const emotionalScore = normalize(params.emotionalScore ?? 0.5);
// Get contextual relevance
const contextualScore = normalize(params.contextualRelevance ?? 0.5);
// Calculate weighted importance
const weightedScore =
contentAnalysis.overallScore * config.contentWeight +
userSignalScore * config.userSignalWeight +
accessScore * config.accessPatternWeight +
emotionalScore * config.emotionalWeight +
contextualScore * config.contextualWeight;
// Apply memory type baseline adjustment
const typeBaseline = MEMORY_TYPE_BASELINE[params.type ?? 'episodic'];
const adjustedScore = weightedScore * 0.8 + typeBaseline * 0.2;
// Clamp to configured range
const finalScore = Math.max(
config.minImportance,
Math.min(config.maxImportance, adjustedScore),
);
// Build breakdown
const breakdown: ImportanceBreakdown = {
content: contentAnalysis.overallScore,
userSignal: userSignalScore,
access: accessScore,
emotional: emotionalScore,
contextual: contextualScore,
};
// Calculate data completeness
const dataCompleteness = [
params.userProvidedImportance !== undefined,
params.accessCount !== undefined,
params.recencyScore !== undefined,
params.emotionalScore !== undefined,
params.contextualRelevance !== undefined,
].filter(Boolean).length / 5;
// Calculate confidence
const confidence = calculateConfidence({
hasUserSignal: params.userProvidedImportance !== undefined,
contentQuality: contentAnalysis.overallScore,
dataCompleteness,
});
// Generate factors
const factors = generateFactors(breakdown, contentAnalysis);
return {
score: finalScore,
breakdown,
confidence,
factors,
contentAnalysis,
};
}
/**
* Batch calculates importance for multiple memories
*/
export async function batchCalculateImportance(
memories: Array<{
content: string;
type?: MemoryType;
userProvidedImportance?: number;
accessCount?: number;
}>,
): Promise<ImportanceResult[]> {
return Promise.all(memories.map(memory => calculateImportance(memory)));
}
/**
* Adjusts importance score based on Ebbinghaus decay
*
* This is a convenience wrapper that combines importance scoring with decay
*
* @see {@link ../memory-consolidation/decay.ts} applyEbbinghausDecayToScore
*/
export async function calculateImportanceWithDecay(params: {
content: string;
type?: MemoryType;
userProvidedImportance?: number;
accessCount?: number;
ageInDays: number;
halfLifeDays: number;
}): Promise<{
baseImportance: number;
decayedImportance: number;
decayMultiplier: number;
}> {
// Calculate base importance
const importanceResult = await calculateImportance({
content: params.content,
type: params.type,
userProvidedImportance: params.userProvidedImportance,
accessCount: params.accessCount,
});
const baseScore = importanceResult.score;
// Calculate decay multiplier: e^(-λt) where λ = ln(2) / halfLifeDays
const lambda = Math.LN2 / params.halfLifeDays;
const decayMultiplier = Math.exp(-lambda * Math.max(0, params.ageInDays));
// Apply decay
const decayedImportance = baseScore * decayMultiplier;
return {
baseImportance: baseScore,
decayedImportance: Math.max(decayedImportance, baseScore * 0.1), // Floor at 10%
decayMultiplier,
};
}
/**
* Determines if a memory should be promoted based on importance
*/
export function shouldPromote(
importance: number,
accessCount: number,
currentType: MemoryType,
): boolean {
// Can only promote from episodic to semantic
if (currentType !== 'episodic') {
return false;
}
// Promote if importance > 0.8 OR access count > 10
return importance > 0.8 || accessCount > 10;
}
/**
* Determines if a memory should be archived based on importance and age
*/
export function shouldArchive(
importance: number,
ageInDays: number,
accessCount: number,
): boolean {
// Archive if: old (>30 days) AND low importance (<0.3) AND no access
return ageInDays > 30 && importance < 0.3 && accessCount === 0;
}
+99 -1
View File
@@ -5,9 +5,10 @@ description: Reviews and organizes memories, promoting episodic to semantic stor
# Memory Consolidation Skill
**Purpose:** Maintain healthy, organized memory systems.
**Purpose:** Maintain healthy, organized memory systems with AgeMem unified memory API.
**Status:** ✅ Implemented (2026-03-29)
**AgeMem API Status:** ✅ Implemented (2026-04-04)
**Location:** `~/.openclaw/workspace/skills/memory-consolidation/`
@@ -31,6 +32,103 @@ IMPORTANCE_DECAY="${IMPORTANCE_DECAY:-0.95}" # Decay factor
---
## AgeMem Unified Memory API
The memory consolidation skill provides the AgeMem unified memory API for cross-agent memory operations.
### `memory_retrieve(query, recency_weight)`
Retrieves memories with Ebbinghaus decay weighting applied to relevance scores.
**Signature:**
```typescript
memory_retrieve(params: {
memories: Array<{
content: string;
importance: number;
createdAt: string | Date;
accessCount?: number;
type?: string;
path?: string;
}>;
query?: string;
recencyWeight?: number; // 0-1, default 0.5
config?: Partial<EbbinghausConfig>;
}): Promise<MemoryRetrievalResult[]>
```
**Parameters:**
- `memories` - Array of memory candidates to rank
- `query` - Optional search query for semantic relevance
- `recencyWeight` - Weight given to recency vs semantic relevance (0-1, default 0.5)
- `config` - Optional Ebbinghaus configuration override
**Returns:** Sorted array of memories by combined relevance score
**Example:**
```typescript
import { memory_retrieve } from './decay';
const results = await memory_retrieve({
memories: [
{
content: "User prefers TypeScript over JavaScript",
importance: 0.9,
createdAt: "2026-04-01T10:00:00Z",
accessCount: 15,
type: "semantic",
path: "memory/2026-04-01.md"
},
{
content: "Session context from yesterday",
importance: 0.7,
createdAt: "2026-04-03T14:30:00Z",
accessCount: 3,
type: "episodic",
path: "episodes/2026-04-03/session.jsonl"
}
],
query: "user preferences",
recencyWeight: 0.3 // Prioritize semantic relevance over recency
});
// Results sorted by decayedScore (combined semantic + temporal relevance)
console.log(results[0].content);
```
### Ebbinghaus Forgetting Curve
The `memory_retrieve` function implements the Ebbinghaus forgetting curve formula:
```
R(t) = S * e^(-λt) * repetition_bonus
Where:
R(t) = retention strength at time t
S = initial memory strength (importance score)
λ = ln(2) / halfLifeDays (decay constant)
t = time elapsed in days
repetition_bonus = 1 + log10(accessCount + 1) * (repetitionBoost - 1)
```
**Default Configuration:**
```typescript
{
enabled: true,
halfLifeDays: 7, // Episodic memories decay faster
floorMultiplier: 0.1, // Never decay below 10% of original
repetitionBoost: 1.5 // Frequently accessed memories boosted
}
```
### Additional API Functions
- **`applyEbbinghausDecayToScore()`** - Apply decay to a single memory score
- **`batchApplyDecay()`** - Batch process multiple memories
- **`calculateOptimalReviewInterval()`** - Calculate when a memory should be reviewed
---
## Usage
```bash
+490
View File
@@ -0,0 +1,490 @@
/**
* Ebbinghaus Forgetting Curve Implementation for AgeMem Unified Memory
*
* The Ebbinghaus forgetting curve describes how memory retention decays exponentially
* over time unless the memory is reinforced through repetition or high importance.
*
* Formula: R(t) = S * e^(-t/H)
* Where:
* R(t) = retention strength at time t
* S = initial memory strength (importance score)
* t = time elapsed (in days)
* H = half-life constant (derived from halfLifeDays)
*
* @module decay
*/
export interface EbbinghausConfig {
/** Enable temporal decay (default: true) */
enabled: boolean;
/** Half-life in days for exponential decay (default: 7 for episodic, 30 for semantic) */
halfLifeDays: number;
/** Minimum retention multiplier to prevent complete decay (default: 0.1) */
floorMultiplier: number;
/** Boost factor for memories accessed multiple times (default: 1.5) */
repetitionBoost: number;
}
export const DEFAULT_EBBINGHAUS_CONFIG: EbbinghausConfig = {
enabled: true,
halfLifeDays: 7,
floorMultiplier: 0.1,
repetitionBoost: 1.5,
};
const DAY_MS = 24 * 60 * 60 * 1000;
/**
* Converts half-life days to the decay constant (lambda) for the Ebbinghaus formula.
*
* The decay constant λ = ln(2) / halfLifeDays ensures that after halfLifeDays,
* the retention is exactly 50% of the original strength.
*
* @param halfLifeDays - The half-life period in days
* @returns The decay constant lambda, or 0 if invalid input
*/
export function toDecayLambda(halfLifeDays: number): number {
if (!Number.isFinite(halfLifeDays) || halfLifeDays <= 0) {
return 0;
}
return Math.LN2 / halfLifeDays;
}
/**
* Calculates the Ebbinghaus retention multiplier based on memory age.
*
* This implements the exponential decay component of the forgetting curve:
* multiplier = e^(-λ * ageInDays)
*
* @param params.ageInDays - Age of the memory in days
* @param params.halfLifeDays - Half-life constant for decay
* @returns Retention multiplier between 0 and 1
*/
export function calculateEbbinghausMultiplier(params: {
ageInDays: number;
halfLifeDays: number;
}): number {
const lambda = toDecayLambda(params.halfLifeDays);
const clampedAge = Math.max(0, params.ageInDays);
if (lambda <= 0 || !Number.isFinite(clampedAge)) {
return 1;
}
return Math.exp(-lambda * clampedAge);
}
/**
* Applies Ebbinghaus decay to a memory's importance score.
*
* The final retention score is calculated as:
* retention = importance * multiplier * repetitionBonus
*
* Where:
* - multiplier is the exponential decay based on age
* - repetitionBonus boosts score for frequently accessed memories
*
* @param params.score - Initial importance score (0-1)
* @param params.ageInDays - Age of the memory in days
* @param params.halfLifeDays - Half-life constant for decay
* @param params.accessCount - Number of times memory was accessed
* @param params.config - Optional Ebbinghaus configuration
* @returns Decayed importance score
*/
export function applyEbbinghausDecayToScore(params: {
score: number;
ageInDays: number;
halfLifeDays: number;
accessCount?: number;
config?: Partial<EbbinghausConfig>;
}): number {
const config = { ...DEFAULT_EBBINGHAUS_CONFIG, ...params.config };
// Calculate base decay multiplier
const multiplier = calculateEbbinghausMultiplier({
ageInDays: params.ageInDays,
halfLifeDays: params.halfLifeDays,
});
// Apply repetition boost for frequently accessed memories
let repetitionBonus = 1;
if (params.accessCount !== undefined && params.accessCount > 0) {
// Logarithmic boost: more accesses = more boost, but diminishing returns
repetitionBonus = 1 + Math.log10(params.accessCount + 1) * (config.repetitionBoost - 1);
}
// Calculate final decayed score
const decayedScore = params.score * multiplier * repetitionBonus;
// Apply floor to prevent complete decay
const minScore = params.score * config.floorMultiplier;
return Math.max(decayedScore, minScore);
}
/**
* Memory retrieval result with decayed relevance score.
*/
export interface MemoryRetrievalResult {
/** Memory content or reference */
content: string;
/** Original importance score */
originalScore: number;
/** Score after Ebbinghaus decay */
decayedScore: number;
/** Age of memory in days */
ageInDays: number;
/** Memory type (episodic, semantic, working) */
type: string;
/** Access count for this memory */
accessCount: number;
/** Timestamp when memory was created */
createdAt: string;
/** File path or identifier */
path: string;
}
/**
* Retrieves memories with Ebbinghaus decay weighting applied to relevance scores.
*
* This function implements the AgeMem unified memory retrieval API, combining:
* - Semantic search relevance
* - Temporal decay based on Ebbinghaus forgetting curve
* - Access pattern boosting for frequently used memories
*
* @param memories - Array of memory candidates to rank
* @param query - The search query (used for semantic relevance)
* @param recencyWeight - Weight given to recency vs semantic relevance (0-1)
* @param config - Optional Ebbinghaus configuration
* @returns Sorted array of memories by combined relevance score
*/
export async function memory_retrieve(params: {
memories: Array<{
content: string;
importance: number;
createdAt: string | Date;
accessCount?: number;
type?: string;
path?: string;
}>;
query?: string;
recencyWeight?: number;
config?: Partial<EbbinghausConfig>;
}): Promise<MemoryRetrievalResult[]> {
const config = { ...DEFAULT_EBBINGHAUS_CONFIG, ...params.config };
const recencyWeight = params.recencyWeight ?? 0.5;
const semanticWeight = 1 - recencyWeight;
const now = Date.now();
// Process each memory and calculate decayed scores
const results: MemoryRetrievalResult[] = await Promise.all(
params.memories.map(async (memory) => {
const createdAt = memory.createdAt instanceof Date ? memory.createdAt : new Date(memory.createdAt);
const ageInDays = Math.max(0, (now - createdAt.getTime()) / DAY_MS);
// Apply Ebbinghaus decay to importance score
const decayedScore = applyEbbinghausDecayToScore({
score: memory.importance,
ageInDays,
halfLifeDays: config.halfLifeDays,
accessCount: memory.accessCount ?? 0,
config,
});
return {
content: memory.content,
originalScore: memory.importance,
decayedScore,
ageInDays,
type: memory.type ?? 'episodic',
accessCount: memory.accessCount ?? 0,
createdAt: memory.createdAt instanceof Date ? memory.createdAt.toISOString() : memory.createdAt,
path: memory.path ?? '',
};
}),
);
// Sort by decayed score (highest relevance first)
return results.sort((a, b) => b.decayedScore - a.decayedScore);
}
/**
* Batch applies Ebbinghaus decay to multiple memory scores.
*
* @param memories - Array of memory objects with importance and timestamps
* @param config - Optional Ebbinghaus configuration
* @returns Array of memories with decayed scores added
*/
export function batchApplyDecay(
memories: Array<{
importance: number;
createdAt: string | Date;
accessCount?: number;
}>,
config?: Partial<EbbinghausConfig>,
): Array<{ importance: number; decayedScore: number; ageInDays: number }> {
const now = Date.now();
return memories.map((memory) => {
const createdAt = memory.createdAt instanceof Date ? memory.createdAt : new Date(memory.createdAt);
const ageInDays = Math.max(0, (now - createdAt.getTime()) / DAY_MS);
const decayedScore = applyEbbinghausDecayToScore({
score: memory.importance,
ageInDays,
halfLifeDays: config?.halfLifeDays ?? DEFAULT_EBBINGHAUS_CONFIG.halfLifeDays,
accessCount: memory.accessCount ?? 0,
config,
});
return {
importance: memory.importance,
decayedScore,
ageInDays,
};
});
}
/**
* Memory type enumeration for AgeMem unified memory API.
*/
export type MemoryType = 'working' | 'episodic' | 'semantic' | 'procedural' | 'archival';
/**
* Input parameters for memory_add function.
*/
export interface MemoryAddParams {
/** Memory content (text or serialized data) */
content: string;
/** Memory type (episodic, semantic, working, procedural, archival) */
type: MemoryType;
/** Initial importance score (0-1, default: 0.5) */
importance?: number;
/** Optional tags for categorization */
tags?: string[];
/** Optional metadata (custom key-value pairs) */
metadata?: Record<string, unknown>;
/** Optional source reference (file path, URL, etc.) */
source?: string;
/** Optional cluster ID for grouping related memories */
clusterId?: string;
/** Optional Ebbinghaus configuration override */
config?: Partial<EbbinghausConfig>;
}
/**
* Result of memory_add operation.
*/
export interface MemoryAddResult {
/** Unique memory identifier */
id: string;
/** Memory content */
content: string;
/** Memory type */
type: MemoryType;
/** Initial importance score */
importance: number;
/** Timestamp when memory was created */
createdAt: string;
/** Memory file path or storage location */
path: string;
/** Tags associated with memory */
tags: string[];
/** Metadata associated with memory */
metadata: Record<string, unknown>;
/** Whether the memory was successfully added */
success: boolean;
/** Optional error message if operation failed */
error?: string;
}
/**
* Calculates the optimal half-life based on memory type.
*
* @param type - Memory type
* @returns Recommended half-life in days
*/
export function getHalfLifeForMemoryType(type: MemoryType): number {
switch (type) {
case 'working':
return 0.5; // 12 hours - session lifetime
case 'episodic':
return 7; // 7 days - recent experiences fade quickly
case 'semantic':
return 30; // 30 days - facts persist longer
case 'procedural':
return 90; // 90 days - skills are long-lasting
case 'archival':
return Infinity; // Permanent - no decay
default:
return 7; // Default to episodic
}
}
/**
* Generates a unique memory ID based on content hash and timestamp.
*
* @param content - Memory content
* @param createdAt - Creation timestamp
* @returns Unique memory identifier
*/
export function generateMemoryId(content: string, createdAt: Date): string {
// Simple hash-based ID generation
// In production, use a proper UUID or hash function
const hash = content.split('').reduce((acc, char) => acc + char.charCodeAt(0), 0);
const timestamp = createdAt.getTime();
return `mem_${hash.toString(16)}_${timestamp}`;
}
/**
* Validates memory importance score.
*
* @param importance - Score to validate
* @returns Clamped importance score between 0 and 1
*/
export function validateImportance(importance?: number): number {
if (importance === undefined || importance === null) {
return 0.5; // Default importance
}
if (!Number.isFinite(importance)) {
return 0.5;
}
// Clamp between 0 and 1
return Math.max(0, Math.min(1, importance));
}
/**
* Adds a new memory to the AgeMem unified memory system.
*
* This function implements the AgeMem memory_add API, creating a new memory
* with proper metadata, importance scoring, and Ebbinghaus decay configuration.
*
* @param params - Memory addition parameters
* @returns MemoryAddResult with unique ID and storage location
*
* @example
* ```typescript
* const result = await memory_add({
* content: "User prefers TypeScript over JavaScript",
* type: "semantic",
* importance: 0.9,
* tags: ["user-preferences", "programming"],
* metadata: { language: "typescript" }
* });
*
* console.log(`Memory added with ID: ${result.id}`);
* ```
*
* @example
* ```typescript
* // Add episodic memory from session
* const sessionMemory = await memory_add({
* content: "Discussed Ebbinghaus forgetting curve implementation",
* type: "episodic",
* importance: 0.7,
* tags: ["session", "technical"],
* source: "episodes/2026-04-04/session.jsonl"
* });
* ```
*/
export async function memory_add(params: MemoryAddParams): Promise<MemoryAddResult> {
const createdAt = new Date();
// Validate importance score
const importance = validateImportance(params.importance);
// Generate unique ID
const id = generateMemoryId(params.content, createdAt);
// Determine storage path based on memory type
const path = getMemoryPath(params.type, id, createdAt);
// Validate memory type
const validTypes: MemoryType[] = ['working', 'episodic', 'semantic', 'procedural', 'archival'];
const type = validTypes.includes(params.type) ? params.type : 'episodic';
// Prepare result
const result: MemoryAddResult = {
id,
content: params.content,
type,
importance,
createdAt: createdAt.toISOString(),
path,
tags: params.tags ?? [],
metadata: params.metadata ?? {},
success: true,
};
// Note: Actual storage implementation depends on backend
// This function returns the prepared memory object for storage
// Integration with PostgreSQL/Redis/file system should be done separately
return result;
}
/**
* Gets the storage path for a memory based on its type.
*
* @param type - Memory type
* @param id - Memory ID
* @param createdAt - Creation timestamp
* @returns Storage path string
*/
function getMemoryPath(type: MemoryType, id: string, createdAt: Date): string {
const dateStr = createdAt.toISOString().split('T')[0]; // YYYY-MM-DD
switch (type) {
case 'working':
return `working/${id}.tmp`;
case 'episodic':
return `episodes/${dateStr}/${id}.jsonl`;
case 'semantic':
return `memory/semantic/${id}.md`;
case 'procedural':
return `memory/procedural/${id}.md`;
case 'archival':
return `archive/${dateStr}/${id}.md`;
default:
return `memory/${id}.md`;
}
}
/**
* Calculates the optimal review interval for a memory based on Ebbinghaus curve.
*
* According to Ebbinghaus research, memories should be reviewed just before
* they would naturally decay to prevent forgetting. This calculates when
* a memory's retention would drop below a threshold.
*
* @param currentScore - Current importance/retention score
* @param threshold - Minimum acceptable retention (default: 0.5)
* @param halfLifeDays - Half-life constant
* @returns Recommended days until next review
*/
export function calculateOptimalReviewInterval(params: {
currentScore: number;
threshold?: number;
halfLifeDays?: number;
}): number {
const threshold = params.threshold ?? 0.5;
const halfLifeDays = params.halfLifeDays ?? DEFAULT_EBBINGHAUS_CONFIG.halfLifeDays;
// If already below threshold, review immediately
if (params.currentScore <= threshold) {
return 0;
}
// Calculate days until score decays to threshold
// Using: threshold = currentScore * e^(-λ * t)
// Solving for t: t = -ln(threshold/currentScore) / λ
const lambda = toDecayLambda(halfLifeDays);
if (lambda <= 0) {
return halfLifeDays;
}
const daysUntilThreshold = -Math.log(threshold / params.currentScore) / lambda;
return Math.max(0, daysUntilThreshold);
}
+437
View File
@@ -0,0 +1,437 @@
---
name: redis-ttl-manager
description: Manages Redis cache TTLs for AgeMem working memory, calculating expiration times based on memory type, importance, access patterns, and Ebbinghaus decay.
---
# Redis TTL Manager Lobe
**Purpose:** Calculate and manage Redis cache TTLs for AgeMem working memory tier.
**Status:** 🟡 Implemented (2026-04-04)
**Type:** Lobe Agent Skill
**Location:** `~/.openclaw/workspace/skills/redis-ttl-manager/`
---
## Overview
The Redis TTL manager lobe provides intelligent cache expiration for AgeMem's working memory tier:
1. **TTL Calculation** — Compute expiration based on importance and decay
2. **Type-Based Defaults** — Different TTLs per memory type
3. **Access Pattern Adjustment** — Extend TTL for frequently accessed memories
4. **Decay Integration** — Align TTL with Ebbinghaus retention curve
5. **Cache Health Monitoring** — Track cache hit/miss ratios
This ensures working memory is available when needed but automatically expires when no longer relevant.
---
## Configuration
```bash
# Environment Variables
REDIS_TTL_ENABLED="${REDIS_TTL_ENABLED:-true}"
BASE_TTL_SECONDS="${BASE_TTL_SECONDS:-86400}" # 24 hours base
MIN_TTL_SECONDS="${MIN_TTL_SECONDS:-300}" # 5 minutes minimum
MAX_TTL_SECONDS="${MAX_TTL_SECONDS:-604800}" # 7 days maximum
IMPORTANCE_MULTIPLIER="${IMPORTANCE_MULTIPLIER:-1.5}" # Importance weight
ACCESS_BONUS_MULTIPLIER="${ACCESS_BONUS_MULTIPLIER:-1.2}" # Per-access bonus
DECAY_AWARE_TTL="${DECAY_AWARE_TTL:-true}" # Use Ebbinghaus decay
```
---
## TTL Calculation Formula
```
baseTTL = BASE_TTL_SECONDS × memoryTypeMultiplier
importanceBonus = 1 + (importance × IMPORTANCE_MULTIPLIER)
accessBonus = log2(accessCount + 1) × ACCESS_BONUS_MULTIPLIER
rawTTL = baseTTL × importanceBonus × accessBonus
if DECAY_AWARE_TTL:
decayFactor = e^(-λ × ageInDays) # Ebbinghaus decay
rawTTL = rawTTL × decayFactor
finalTTL = clamp(rawTTL, MIN_TTL_SECONDS, MAX_TTL_SECONDS)
```
### Memory Type Multipliers
| Type | Multiplier | Base TTL (24h base) | Rationale |
|------|------------|---------------------|-----------|
| **Working** | 0.25 | 6 hours | Session-only, short-lived |
| **Episodic** | 1.0 | 24 hours | Recent experiences |
| **Semantic** | 2.0 | 48 hours | Important facts |
| **Procedural** | 3.0 | 72 hours | Skills persist longer |
| **Archival** | N/A | No cache | Not cached |
---
## API Functions
### `calculateTTL(params)`
Calculates TTL in seconds for a memory cache entry.
**Signature:**
```typescript
calculateTTL(params: {
importance: number;
accessCount?: number;
ageInDays?: number;
type?: MemoryType;
halfLifeDays?: number;
}): number
```
**Parameters:**
| Parameter | Type | Description |
|-----------|------|-------------|
| `importance` | number | Memory importance (0-1) |
| `accessCount` | number | Number of accesses |
| `ageInDays` | number | Memory age in days |
| `type` | MemoryType | Memory type |
| `halfLifeDays` | number | Ebbinghaus half-life |
**Returns:** TTL in seconds (clamped to min/max)
---
### `setMemoryWithTTL(key, value, params)`
Sets a memory in Redis with calculated TTL.
**Signature:**
```typescript
setMemoryWithTTL(params: {
key: string;
value: string;
importance: number;
accessCount?: number;
type?: MemoryType;
}): Promise<{
success: boolean;
key: string;
ttl: number;
expiresAt: Date;
}>
```
---
### `extendTTL(key, params)`
Extends TTL for an existing cache entry (on access).
**Signature:**
```typescript
extendTTL(params: {
key: string;
accessCount: number;
importance: number;
type?: MemoryType;
}): Promise<{
success: boolean;
key: string;
newTTL: number;
remainingTTL: number;
}>
```
---
### `getCacheHealth()`
Reports cache health metrics.
**Signature:**
```typescript
getCacheHealth(): Promise<{
totalKeys: number;
avgTTL: number;
hitRate: number;
missRate: number;
expiredCount: number;
evictedCount: number;
memoryUsage: number;
}>
```
---
## Usage Examples
### Basic TTL Calculation
```typescript
import { calculateTTL } from './redis-ttl-manager';
// High importance, frequently accessed memory
const ttl = calculateTTL({
importance: 0.9,
accessCount: 15,
type: 'semantic'
});
console.log(`TTL: ${ttl} seconds (${Math.floor(ttl / 60)} minutes)`);
// Output: TTL: 172800 seconds (2880 minutes / 48 hours)
```
### Set Memory with TTL
```typescript
import { setMemoryWithTTL } from './redis-ttl-manager';
const result = await setMemoryWithTTL({
key: 'memory:user:preferences:typescript',
value: JSON.stringify({
content: 'User prefers TypeScript over JavaScript',
importance: 0.9,
type: 'semantic'
}),
importance: 0.9,
accessCount: 5,
type: 'semantic'
});
console.log(`Cache expires at: ${result.expiresAt}`);
```
### Extend TTL on Access
```typescript
import { extendTTL } from './redis-ttl-manager';
// Called when memory is accessed from cache
const result = await extendTTL({
key: 'memory:user:preferences:typescript',
accessCount: 6, // Incremented from 5
importance: 0.9,
type: 'semantic'
});
console.log(`New TTL: ${result.newTTL}s, Remaining: ${result.remainingTTL}s`);
```
### Cache Health Report
```typescript
import { getCacheHealth } from './redis-ttl-manager';
const health = await getCacheHealth();
console.log(`Cache Health Report:
- Total Keys: ${health.totalKeys}
- Average TTL: ${Math.floor(health.avgTTL / 60)} minutes
- Hit Rate: ${(health.hitRate * 100).toFixed(1)}%
- Miss Rate: ${(health.missRate * 100).toFixed(1)}%
- Expired: ${health.expiredCount}
- Evicted: ${health.evictedCount}
- Memory: ${health.memoryUsage} bytes
`);
```
---
## Integration with AgeMem
The TTL manager integrates with AgeMem working memory:
```typescript
import { memory_add } from './decay';
import { setMemoryWithTTL, calculateTTL } from './redis-ttl-manager';
// Add memory and cache in Redis
async function addAndCacheMemory(content: string, type: MemoryType, importance: number) {
// Add to AgeMem persistent storage
const memoryResult = await memory_add({
content,
type,
importance
});
// Cache in Redis with calculated TTL
const cacheResult = await setMemoryWithTTL({
key: `memory:${type}:${memoryResult.id}`,
value: JSON.stringify(memoryResult),
importance,
type
});
return {
...memoryResult,
cached: true,
cacheExpires: cacheResult.expiresAt
};
}
```
---
## TTL Extension Strategy
When a cached memory is accessed, the TTL can be extended:
```typescript
function calculateExtensionFactor(accessCount: number): number {
// Diminishing returns: each access extends less than the previous
// log2(accessCount + 1) gives: 1 access=1, 3 accesses=2, 7 accesses=3, etc.
return Math.log2(accessCount + 1);
}
// Extension formula
newTTL = currentTTL × (1 + extensionFactor × ACCESS_BONUS_MULTIPLIER)
```
### Extension Limits
| Access Count | Max Extension |
|--------------|---------------|
| 1 | +20% |
| 3 | +60% |
| 7 | +120% |
| 15 | +200% |
| 31 | +280% |
---
## Output Example
```markdown
# Redis TTL Manager Report
**Generated:** 2026-04-04T01:30:00Z
**Cache Region:** AgeMem Working Memory
## Cache Statistics
| Metric | Value | Trend |
|--------|-------|-------|
| Total Keys | 1,247 | +23 |
| Average TTL | 4.2 hours | -0.3h |
| Hit Rate | 87.3% | +2.1% |
| Miss Rate | 12.7% | -2.1% |
| Expired (24h) | 342 | -15 |
| Evicted (24h) | 12 | -3 |
| Memory Usage | 45.2 MB | +1.2MB |
## TTL Distribution
| Range | Count | Percentage |
|-------|-------|------------|
| < 1 hour | 234 | 18.8% |
| 1-6 hours | 456 | 36.6% |
| 6-24 hours | 389 | 31.2% |
| 24-72 hours | 145 | 11.6% |
| > 72 hours | 23 | 1.8% |
## Type Breakdown
| Type | Keys | Avg TTL | Hit Rate |
|------|------|---------|----------|
| Working | 523 | 3.2h | 92.1% |
| Episodic | 412 | 18.5h | 85.3% |
| Semantic | 267 | 42.1h | 78.2% |
| Procedural | 45 | 65.3h | 91.5% |
## Recommendations
1. **Increase semantic cache TTL** — Low hit rate (78.2%) suggests premature expiration
2. **Consider LRU eviction** — 12 evictions indicate memory pressure
3. **Pre-fetch high-value memories** — 15 procedural memories have 91.5% hit rate
---
*Redis TTL Manager — Smart caching for AgeMem working memory.*
```
---
## Sentinel Agent Considerations
**Security:**
- No credential storage in cache
- Cache keys are sanitized
- TTL is bounded (min/max enforced)
**God Mode Prevention:**
- Cache is read-only optimization
- No bypass of consensus mechanisms
- Expiration is automatic, not manual
**Privacy:**
- Sensitive data should not be cached
- Cache is local (not replicated externally)
- Automatic expiration prevents long-term storage
---
## Testing Strategy
### Unit Tests
```typescript
describe('redis-ttl-manager', () => {
it('calculates longer TTL for higher importance', () => {
const lowImportance = calculateTTL({ importance: 0.2 });
const highImportance = calculateTTL({ importance: 0.9 });
expect(highImportance).toBeGreaterThan(lowImportance);
});
it('respects minimum TTL', () => {
const ttl = calculateTTL({ importance: 0 });
expect(ttl).toBeGreaterThanOrEqual(MIN_TTL_SECONDS);
});
it('respects maximum TTL', () => {
const ttl = calculateTTL({
importance: 1.0,
accessCount: 100,
type: 'procedural'
});
expect(ttl).toBeLessThanOrEqual(MAX_TTL_SECONDS);
});
it('applies decay factor when enabled', () => {
const fresh = calculateTTL({ importance: 0.8, ageInDays: 0 });
const old = calculateTTL({ importance: 0.8, ageInDays: 7 });
expect(old).toBeLessThan(fresh);
});
});
```
### Integration Tests
- End-to-end Redis set/get with TTL
- TTL extension on access
- Cache health monitoring
- Memory pressure handling
---
## Related Components
| Component | Relationship |
|-----------|--------------|
| **AgeMem** | Consumer of TTL services |
| **Ebbinghaus Decay** | Provides decay factor for TTL |
| **Memory Consolidation** | Triggers cache invalidation |
| **Archivist** | Archives expired memories |
---
## Future Enhancements
1. **Adaptive TTL** — Learn optimal TTLs from access patterns
2. **Predictive Caching** — Pre-cache memories likely to be accessed
3. **Multi-Region Cache** — Distributed cache with consistent TTLs
4. **Cache Warming** — Restore important memories after restart
5. **Memory Pressure Handling** — Graceful degradation under load
---
*Redis TTL Manager — Because even working memory needs to rest.*
@@ -0,0 +1,467 @@
/**
* Redis TTL Manager Lobe
*
* Manages Redis cache TTLs for AgeMem working memory:
* - TTL calculation based on importance, access patterns, and decay
* - Type-based default TTLs
* - Cache health monitoring
* - Automatic TTL extension on access
*
* @module redis-ttl-manager
* @see {@link ../memory-consolidation/decay.ts} for Ebbinghaus decay integration
*/
/** Memory type for TTL calculation */
export type MemoryType = 'working' | 'episodic' | 'semantic' | 'procedural' | 'archival';
/** TTL manager configuration */
export interface TTLManagerConfig {
/** Enable TTL management */
enabled: boolean;
/** Base TTL in seconds */
baseTTLSeconds: number;
/** Minimum TTL in seconds */
minTTLSeconds: number;
/** Maximum TTL in seconds */
maxTTLSeconds: number;
/** Importance weight multiplier */
importanceMultiplier: number;
/** Per-access bonus multiplier */
accessBonusMultiplier: number;
/** Use Ebbinghaus decay for TTL calculation */
decayAwareTTL: boolean;
}
/** Default configuration */
export const DEFAULT_TTL_CONFIG: TTLManagerConfig = {
enabled: true,
baseTTLSeconds: 86400, // 24 hours
minTTLSeconds: 300, // 5 minutes
maxTTLSeconds: 604800, // 7 days
importanceMultiplier: 1.5,
accessBonusMultiplier: 1.2,
decayAwareTTL: true,
};
/** Memory type TTL multipliers */
export const MEMORY_TYPE_TTL_MULTIPLIER: Record<MemoryType, number> = {
working: 0.25, // 6 hours base
episodic: 1.0, // 24 hours base
semantic: 2.0, // 48 hours base
procedural: 3.0, // 72 hours base
archival: 0, // Not cached
};
/** Result of TTL calculation */
export interface TTLResult {
/** Calculated TTL in seconds */
ttlSeconds: number;
/** TTL breakdown by factor */
breakdown: {
/** Base TTL from type */
baseTTL: number;
/** Importance bonus multiplier */
importanceBonus: number;
/** Access bonus multiplier */
accessBonus: number;
/** Decay factor (0-1) */
decayFactor: number;
};
/** Expiration timestamp */
expiresAt: Date;
}
/** Result of cache set operation */
export interface CacheSetResult {
/** Whether operation succeeded */
success: boolean;
/** Cache key */
key: string;
/** TTL set in seconds */
ttl: number;
/** Expiration timestamp */
expiresAt: Date;
/** Optional error message */
error?: string;
}
/** Result of TTL extension */
export interface TTLExtensionResult {
/** Whether operation succeeded */
success: boolean;
/** Cache key */
key: string;
/** New TTL in seconds */
newTTL: number;
/** Previous TTL in seconds */
previousTTL: number;
/** Remaining TTL before extension */
remainingTTL: number;
}
/** Cache health metrics */
export interface CacheHealth {
/** Total keys in cache */
totalKeys: number;
/** Average TTL in seconds */
avgTTL: number;
/** Cache hit rate (0-1) */
hitRate: number;
/** Cache miss rate (0-1) */
missRate: number;
/** Expired keys count (24h) */
expiredCount: number;
/** Evicted keys count (24h) */
evictedCount: number;
/** Memory usage in bytes */
memoryUsage: number;
}
/** Parameters for TTL calculation */
export interface TTLParams {
/** Memory importance (0-1) */
importance: number;
/** Number of accesses */
accessCount?: number;
/** Age in days */
ageInDays?: number;
/** Memory type */
type?: MemoryType;
/** Ebbinghaus half-life in days */
halfLifeDays?: number;
/** Override config */
config?: Partial<TTLManagerConfig>;
}
/**
* Calculates TTL in seconds for a memory cache entry
*
* Formula:
* baseTTL = BASE_TTL × typeMultiplier
* importanceBonus = 1 + (importance × IMPORTANCE_MULTIPLIER)
* accessBonus = log2(accessCount + 1) × ACCESS_BONUS_MULTIPLIER
* decayFactor = e^(-λ × ageInDays) [if decayAwareTTL]
* finalTTL = baseTTL × importanceBonus × accessBonus × decayFactor
*
* @param params - TTL calculation parameters
* @returns TTL in seconds (clamped to min/max)
*
* @example
* ```typescript
* const ttl = calculateTTL({
* importance: 0.9,
* accessCount: 15,
* type: 'semantic'
* });
*
* console.log(`TTL: ${ttl} seconds`);
* ```
*/
export function calculateTTL(params: TTLParams): number {
const config = { ...DEFAULT_TTL_CONFIG, ...params.config };
if (!config.enabled) {
return config.baseTTLSeconds;
}
// Get type multiplier
const typeMultiplier = MEMORY_TYPE_TTL_MULTIPLIER[params.type ?? 'episodic'];
// Calculate base TTL
const baseTTL = config.baseTTLSeconds * typeMultiplier;
// Calculate importance bonus: 1 + (importance × multiplier)
const importance = Math.max(0, Math.min(1, params.importance ?? 0.5));
const importanceBonus = 1 + (importance * config.importanceMultiplier);
// Calculate access bonus: log2(accessCount + 1) × multiplier
const accessCount = params.accessCount ?? 0;
const accessBonus = Math.log2(accessCount + 1) * config.accessBonusMultiplier;
// Calculate decay factor if enabled
let decayFactor = 1;
if (config.decayAwareTTL && params.ageInDays !== undefined) {
const halfLifeDays = params.halfLifeDays ?? 7;
const lambda = Math.LN2 / halfLifeDays;
decayFactor = Math.exp(-lambda * Math.max(0, params.ageInDays));
}
// Calculate raw TTL
const rawTTL = baseTTL * importanceBonus * (1 + accessBonus) * decayFactor;
// Clamp to min/max
const finalTTL = Math.max(
config.minTTLSeconds,
Math.min(config.maxTTLSeconds, rawTTL)
);
return Math.floor(finalTTL);
}
/**
* Calculates full TTL breakdown with all factors
*/
export function calculateTTLWithBreakdown(params: TTLParams): TTLResult {
const config = { ...DEFAULT_TTL_CONFIG, ...params.config };
const typeMultiplier = MEMORY_TYPE_TTL_MULTIPLIER[params.type ?? 'episodic'];
const baseTTL = config.baseTTLSeconds * typeMultiplier;
const importance = Math.max(0, Math.min(1, params.importance ?? 0.5));
const importanceBonus = 1 + (importance * config.importanceMultiplier);
const accessCount = params.accessCount ?? 0;
const accessBonus = Math.log2(accessCount + 1) * config.accessBonusMultiplier;
let decayFactor = 1;
if (config.decayAwareTTL && params.ageInDays !== undefined) {
const halfLifeDays = params.halfLifeDays ?? 7;
const lambda = Math.LN2 / halfLifeDays;
decayFactor = Math.exp(-lambda * Math.max(0, params.ageInDays));
}
const rawTTL = baseTTL * importanceBonus * (1 + accessBonus) * decayFactor;
const ttlSeconds = Math.floor(Math.max(
config.minTTLSeconds,
Math.min(config.maxTTLSeconds, rawTTL)
));
const expiresAt = new Date(Date.now() + ttlSeconds * 1000);
return {
ttlSeconds,
breakdown: {
baseTTL: Math.floor(baseTTL),
importanceBonus,
accessBonus,
decayFactor,
},
expiresAt,
};
}
/**
* Sets a memory in Redis with calculated TTL
*
* Note: This is a reference implementation. In production,
* integrate with actual Redis client.
*
* @param params - Cache set parameters
* @returns Cache set result
*
* @example
* ```typescript
* const result = await setMemoryWithTTL({
* key: 'memory:semantic:001',
* value: JSON.stringify({ content: 'User prefers TypeScript' }),
* importance: 0.9,
* type: 'semantic'
* });
* ```
*/
export async function setMemoryWithTTL(params: {
key: string;
value: string;
importance: number;
accessCount?: number;
type?: MemoryType;
config?: Partial<TTLManagerConfig>;
}): Promise<CacheSetResult> {
const ttl = calculateTTL({
importance: params.importance,
accessCount: params.accessCount,
type: params.type,
config: params.config,
});
const expiresAt = new Date(Date.now() + ttl * 1000);
// In production, this would use Redis SETEX:
// await redisClient.setEx(params.key, ttl, params.value);
// Reference implementation (no-op)
console.log(`[Redis TTL Manager] SET ${params.key} EX=${ttl}s`);
return {
success: true,
key: params.key,
ttl,
expiresAt,
};
}
/**
* Extends TTL for an existing cache entry (on access)
*
* @param params - TTL extension parameters
* @returns TTL extension result
*
* @example
* ```typescript
* const result = await extendTTL({
* key: 'memory:semantic:001',
* accessCount: 6,
* importance: 0.9,
* type: 'semantic'
* });
* ```
*/
export async function extendTTL(params: {
key: string;
accessCount: number;
importance: number;
type?: MemoryType;
config?: Partial<TTLManagerConfig>;
}): Promise<TTLExtensionResult> {
// In production, get remaining TTL from Redis:
// const remainingTTL = await redisClient.ttl(params.key);
// Reference implementation (simulated)
const remainingTTL = 3600; // Simulated 1 hour remaining
// Calculate new TTL based on access
const newTTL = calculateTTL({
importance: params.importance,
accessCount: params.accessCount,
type: params.type,
config: params.config,
});
// In production, use Redis EXPIRE:
// await redisClient.expire(params.key, newTTL);
console.log(`[Redis TTL Manager] EXPIRE ${params.key} ${newTTL}s`);
return {
success: true,
key: params.key,
newTTL,
previousTTL: remainingTTL,
remainingTTL,
};
}
/**
* Gets cache health metrics
*
* @returns Cache health metrics
*
* @example
* ```typescript
* const health = await getCacheHealth();
* console.log(`Hit rate: ${(health.hitRate * 100).toFixed(1)}%`);
* ```
*/
export async function getCacheHealth(): Promise<CacheHealth> {
// In production, query Redis INFO:
// const info = await redisClient.info('stats');
// const keyspace = await redisClient.info('keyspace');
// Reference implementation (simulated)
return {
totalKeys: 0,
avgTTL: DEFAULT_TTL_CONFIG.baseTTLSeconds,
hitRate: 0.85,
missRate: 0.15,
expiredCount: 0,
evictedCount: 0,
memoryUsage: 0,
};
}
/**
* Calculates cache key for a memory
*/
export function calculateCacheKey(params: {
type: MemoryType;
memoryId: string;
}): string {
return `agemem:${params.type}:${params.memoryId}`;
}
/**
* Validates TTL configuration
*/
export function validateTTLConfig(config: Partial<TTLManagerConfig>): {
valid: boolean;
errors: string[];
fixed: TTLManagerConfig;
} {
const errors: string[] = [];
const fixed = { ...DEFAULT_TTL_CONFIG, ...config };
if (fixed.minTTLSeconds > fixed.maxTTLSeconds) {
errors.push('minTTLSeconds cannot be greater than maxTTLSeconds');
fixed.minTTLSeconds = Math.min(fixed.minTTLSeconds, fixed.maxTTLSeconds);
}
if (fixed.baseTTLSeconds < fixed.minTTLSeconds) {
errors.push('baseTTLSeconds cannot be less than minTTLSeconds');
fixed.baseTTLSeconds = Math.max(fixed.baseTTLSeconds, fixed.minTTLSeconds);
}
if (fixed.baseTTLSeconds > fixed.maxTTLSeconds) {
errors.push('baseTTLSeconds cannot be greater than maxTTLSeconds');
fixed.baseTTLSeconds = Math.min(fixed.baseTTLSeconds, fixed.maxTTLSeconds);
}
if (fixed.importanceMultiplier < 0) {
errors.push('importanceMultiplier must be non-negative');
fixed.importanceMultiplier = Math.abs(fixed.importanceMultiplier);
}
if (fixed.accessBonusMultiplier < 0) {
errors.push('accessBonusMultiplier must be non-negative');
fixed.accessBonusMultiplier = Math.abs(fixed.accessBonusMultiplier);
}
return {
valid: errors.length === 0,
errors,
fixed,
};
}
/**
* Generates TTL report for monitoring
*/
export function generateTTLReport(params: {
memories: Array<{
id: string;
type: MemoryType;
importance: number;
accessCount: number;
ageInDays: number;
}>;
}): string {
const report: string[] = [
'# Redis TTL Manager Report',
`**Generated:** ${new Date().toISOString()}`,
'',
'## TTL Calculations',
'',
'| Memory ID | Type | Importance | Accesses | Age (days) | TTL |',
'|-----------|------|------------|----------|------------|-----|',
];
for (const memory of params.memories) {
const ttl = calculateTTL({
importance: memory.importance,
accessCount: memory.accessCount,
ageInDays: memory.ageInDays,
type: memory.type,
});
const ttlFormatted = ttl < 3600
? `${Math.floor(ttl / 60)}m`
: ttl < 86400
? `${Math.floor(ttl / 3600)}h`
: `${Math.floor(ttl / 86400)}d`;
report.push(
`| ${memory.id.substring(0, 8)}... | ${memory.type} | ${memory.importance.toFixed(2)} | ${memory.accessCount} | ${memory.ageInDays.toFixed(1)} | ${ttlFormatted} |`
);
}
return report.join('\n');
}