LLM Memory Extraction for AI Agents

Pensyve can turn completed conversation episodes into structured observation memories. These records preserve countable details and other episode-level facts that an AI agent should be able to retrieve later without rereading an entire transcript.

What Gets Extracted

The observation extractor receives the ordered messages from one episode and emits structured records tied to that episode. Each observation keeps:

• the extracted content
• the source episode ID
• the namespace that owns the episode
• the relevant entities and timestamps

Observation memories are stored separately from semantic, episodic, and procedural memories. They do not enter the normal Reciprocal Rank Fusion candidate pool. Instead, recall_grouped attaches them to the relevant top-ranked episode groups so an LLM reader gets the supporting details with the conversation that produced them.

Extraction Pipeline

episode messages
      |
      v
local OpenAI-compatible extractor
      |
      v
parse and validate structured observations
      |
      v
namespace-scoped observation storage
      |
      v
recall_grouped attaches observations to source episodes

Extraction runs after an episode closes. A timeout, unavailable model server, or malformed model response does not block the original episode from being stored; Pensyve logs the extraction failure and continues without observation records for that episode.

Configure the Local Extractor

The Rust gateway uses an OpenAI-compatible local endpoint, such as vLLM. Configure it with environment variables before starting pensyve-mcp-gateway:

export PENSYVE_EXTRACTOR_URL=http://localhost:8888/v1
export PENSYVE_EXTRACTOR_MODEL=qwen3.6-35b-a3b
export PENSYVE_NETWORK_POLICY=local-only
cargo run -p pensyve-mcp-gateway

Explicit Facts and Contradictions

remember() stores the fact you provide directly as semantic memory; it does not depend on the observation extractor. During recall, Pensyve separately compares returned semantic memories and reports conflicting facts in the contradictions array. That check is part of retrieval and does not require a separate Tier 2 switch.

Use explicit remember() calls for durable decisions and preferences. Use episode extraction when the agent needs structured observations from the full interaction context, especially for grouped-reader and longitudinal-memory workflows.