Skip to content

imjohnzakkam/memoryclaw

Folders and files

NameName
Last commit message
Last commit date

Latest commit

Β 

History

11 Commits
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 

Repository files navigation

MemoryClaw

MemoryClaw

A brain-inspired memory system for AI agents.
Transparent. Debuggable. No black boxes.

Quick Start β€’ How It Works β€’ Architecture β€’ CLI β€’ Configuration β€’ Contributing

CI License Stars Bun


Most AI memory systems are just vector search with extra steps. You can't read the memories, you can't debug why something was recalled, and you can't fix it when it breaks.

MemoryClaw takes a different approach. Every memory is a plain markdown file. Every retrieval is explainable β€” you can see exactly which keywords matched and why. Vector search exists only as a fallback for the ~20% of queries where keywords aren't enough.

"What did I discuss with Mahesh last week?"
  β†’ keyword extraction: [mahesh, discuss, week]
  β†’ alias expansion: [mahesh, mahes, discuss, talk, conversation, week]
  β†’ 3 episodes matched (tags: mahesh, meeting)
  β†’ 1 semantic fact found (Mahesh: works at Uber)
  β†’ injected into working memory (412 tokens)

No embeddings computed. No API calls. Pure file I/O. Fully transparent.


Why MemoryClaw?

Traditional RAG MemoryClaw
Storage Opaque vector embeddings Plain markdown files you can read and edit
Retrieval Black-box similarity search Keyword + tag matching with explainable scoring
Debuggability "Why was this recalled?" β€” Good luck Every match shows exactly which keywords hit
Cost Embedding API calls on every message Zero API calls in default mode
Privacy Often requires cloud APIs 100% local by default (Ollama)
Fallback N/A Vector search kicks in only when keywords fail

How It Works

MemoryClaw implements a four-tier memory hierarchy inspired by human cognition:

β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚                 WORKING MEMORY                   β”‚
β”‚        Active context for current task           β”‚
β”‚              (~400-800 tokens)                   β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
β”‚               EPISODIC MEMORY                    β”‚
β”‚     Compressed records of past interactions      β”‚
β”‚         Markdown + YAML frontmatter              β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
β”‚               SEMANTIC MEMORY                    β”‚
β”‚       Persistent facts & relationships           β”‚
β”‚     contacts.md, projects.md, preferences.md     β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
β”‚              PROCEDURAL MEMORY                   β”‚
β”‚    Reusable skills compiled from experience      β”‚
β”‚          Auto-generated, user-approved           β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

The Retrieval Pipeline

User query
  β”‚
  β”œβ”€β†’ Extract keywords ──→ Expand aliases/synonyms
  β”‚                              β”‚
  β”‚                    β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
  β”‚                    β”‚  Episode Search    β”‚
  β”‚                    β”‚  (tag + summary    β”‚
  β”‚                    β”‚   scoring)         β”‚
  β”‚                    β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
  β”‚                              β”‚
  β”‚              Results < threshold?
  β”‚                    β”‚              β”‚
  β”‚                   Yes             No
  β”‚                    β”‚              β”‚
  β”‚            Vector Fallback   Use keyword
  β”‚            (optional)        results
  β”‚                    β”‚              β”‚
  β”‚                    β””β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”˜
  β”‚                           β”‚
  β”‚                    Blend results
  β”‚                           β”‚
  β”‚                  Semantic fact lookup
  β”‚                           β”‚
  β”‚                  Inject into working memory
  β”‚                           β”‚
  └───────────────────→  LLM Call (~500 tokens)

Quick Start

Prerequisites

  • Bun runtime (v1.0+)
  • Ollama (optional, for consolidation + vector fallback)

Install

git clone https://github.com/imjohnzakkam/memoryclaw.git
cd memoryclaw
bun install

Run the CLI

# Search your memories
bun run src/cli.ts retrieve "meeting with John about project deadline"

# Audit recent memories
bun run src/cli.ts audit

# Process raw logs into episodes
bun run src/cli.ts consolidate

# Build the search index
bun run src/cli.ts index

Run Tests

bun test
# 102 tests across 14 files

Use as OpenClaw Plugin

// openclaw.plugin.json
{
  "name": "memoryclaw",
  "version": "0.1.0",
  "extensions": ["src/plugin.ts"]
}

MemoryClaw hooks into OpenClaw's lifecycle β€” logging interactions, retrieving context before LLM calls, and running consolidation as a background service.


Architecture

src/
β”œβ”€β”€ config.ts            # YAML config loader with defaults
β”œβ”€β”€ keywords.ts          # Keyword extraction + stop word removal
β”œβ”€β”€ aliases.ts           # Synonym/alias expansion
β”œβ”€β”€ episodic.ts          # Episode parsing + scored keyword search
β”œβ”€β”€ semantic.ts          # Semantic fact entity lookup
β”œβ”€β”€ vector.ts            # Vector fallback (Ollama / OpenAI-compatible)
β”œβ”€β”€ retrieve.ts          # Full retrieval pipeline orchestrator
β”œβ”€β”€ logger.ts            # Raw interaction logging
β”œβ”€β”€ llm.ts               # Chat completion abstraction
β”œβ”€β”€ consolidate.ts       # Logs β†’ episodes + facts (LLM-powered)
β”œβ”€β”€ semantic-writer.ts   # Semantic updates with dedup + conflict detection
β”œβ”€β”€ memories.ts          # User commands: audit, search, delete
β”œβ”€β”€ patterns.ts          # Repeated action pattern detection
β”œβ”€β”€ skill-compiler.ts    # Skill template generation + approval
β”œβ”€β”€ working-memory.ts    # Working memory manager + onBeforeLLM hook
β”œβ”€β”€ indexer.ts           # SQLite FTS5 inverted index
β”œβ”€β”€ cli.ts               # CLI entry point
β”œβ”€β”€ plugin.ts            # OpenClaw plugin integration
└── index.ts             # Public API

Data Directory

~/.openclaw/memoryclaw/
β”œβ”€β”€ episodes/            # Compressed interaction summaries
β”‚   └── 2025-04-08_14-32-10_meeting-with-john.md
β”œβ”€β”€ semantic/            # Persistent facts
β”‚   β”œβ”€β”€ contacts.md
β”‚   β”œβ”€β”€ projects.md
β”‚   └── _pending.md      # Low-confidence facts awaiting review
β”œβ”€β”€ skills/              # Auto-generated skill templates
β”œβ”€β”€ logs/                # Raw interaction logs
β”‚   └── processed/
β”œβ”€β”€ index/               # SQLite FTS5 search index
└── config.yaml

Episode File Format

Every memory is a plain markdown file with structured YAML frontmatter:

---
timestamp: 2025-04-08T14:32:10Z
tags: [email, projectX, deadline, john]
summary: "Discussed project deadline with John. Confirmed April 10th."
participants: [user, assistant]
confidence: high
---
**Details:**
- Recipient: John <john@example.com>
- Subject: Project X deadline confirmation
- Agreed on April 10th hard deadline

CLI

memoryclaw <command> [args]

Retrieval
  retrieve <query>            Search episodes + semantic facts
  search <query>              Keyword search across memories

Memory Management
  audit [limit]               Show recent episodes and pending facts
  delete <filename>           Delete an episode
  delete-fact <file> <key>    Remove a fact from a semantic file

Consolidation
  consolidate                 Process raw logs into episodes + facts

Skills
  patterns [threshold]        Detect repeated action patterns
  compile [threshold]         Generate draft skills from patterns
  skills                      List all skills (draft + approved)
  approve-skill <file>        Approve a draft skill
  reject-skill <file>         Reject a draft skill

Indexing
  index                       Build/rebuild SQLite FTS5 index
  index-search <query>        Search using the full-text index

Configuration

Create ~/.openclaw/memoryclaw/config.yaml:

memoryclaw:
  disableDefaultMemory: true
  path: ~/.openclaw/memoryclaw

  retrieval:
    primary: keyword
    minPrimaryResults: 2      # Fallback triggers below this
    fallback: none            # "none" or "vector"
    maxResults: 5
    blendStrategy: primary_first

  llm:
    provider: ollama          # "ollama" or "openai-compatible"
    baseUrl: http://localhost:11434
    model: llama3:8b
    embeddingModel: nomic-embed
    apiKey: ""                # Only needed for openai-compatible

  consolidation:
    interval: 60              # Minutes between consolidation runs
    skillThreshold: 7         # Min pattern occurrences for skill suggestion
    factValidation: true      # Schema-validate extracted facts
    pendingReview: true       # Low-confidence facts β†’ _pending.md

  semantic:
    files: [contacts.md, projects.md]

LLM Backend

MemoryClaw supports any OpenAI-compatible API. Use Ollama for fully local operation, or point it at any hosted endpoint:

# Local (default)
llm:
  provider: ollama
  baseUrl: http://localhost:11434
  model: llama3:8b

# Hosted / OpenAI-compatible
llm:
  provider: openai-compatible
  baseUrl: https://api.openai.com/v1
  model: gpt-4
  apiKey: sk-...

Design Principles

  1. Keyword-first retrieval. Transparent, explainable, zero-cost. Not another RAG wrapper.
  2. 80/20 rule. Keywords handle ~80% of queries. Vector fallback catches the rest.
  3. Files you can read. Every memory is a markdown file. grep your memories. Edit them in vim. Version them with git.
  4. Privacy by default. Ollama for local LLM. No cloud dependencies. Your memories stay on your machine.
  5. Honest about trade-offs. Keyword matching struggles with paraphrasing β€” that's why vector fallback exists. Auto-generated skills require approval β€” because auto-executing buggy code is worse than no automation.

Error Handling

MemoryClaw is designed to fail gracefully and transparently:

  • Confidence scoring β€” Every episode carries a confidence field (low/medium/high)
  • Fact staging β€” Low-confidence facts go to _pending.md, not canonical files
  • Conflict detection β€” Contradictory facts are flagged, never silently overwritten
  • Source tracing β€” Every fact links back to the originating log file
  • Raw log preservation β€” Original logs kept for re-processing if summarization improves

Tech Stack

Component Technology
Runtime Bun
Language TypeScript (strict mode, ES modules)
Storage Markdown + YAML frontmatter
Search Index SQLite FTS5 (via bun:sqlite)
LLM Configurable β€” Ollama / OpenAI-compatible
Frontmatter gray-matter
Testing Vitest (102 tests, 14 files)
Platform OpenClaw plugin

Contributing

Contributions are welcome! See CONTRIBUTING.md for guidelines.

# Clone and install
git clone https://github.com/imjohnzakkam/memoryclaw.git
cd memoryclaw
bun install

# Run tests
bun test

# Run a specific test file
bun test tests/keywords.test.ts

Roadmap

  • Core retrieval pipeline (keyword + tag scoring)
  • Synonym/alias expansion
  • Vector fallback (Ollama + OpenAI-compatible)
  • Raw interaction logging
  • Consolidation daemon (logs β†’ episodes + facts)
  • Semantic memory with dedup + conflict detection
  • Pattern detection + skill compilation
  • Working memory injection (onBeforeLLM hook)
  • SQLite FTS5 indexing
  • OpenClaw plugin integration
  • CLI for all operations
  • Web UI for memory browsing
  • Multi-agent memory sharing
  • Cross-device sync
  • Advanced pattern mining (PrefixSpan)

License

MIT Β© John Zakkam


Built with frustration at black-box AI memory systems.

About

A brain-inspired, hierarchical memory system as a plugin layer for OpenClaw

Resources

License

Contributing

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors