MemoDocumentation
EN

Data Layer & Persistence

All Memo data lives on your disk — nothing is stored remotely unless you enable cloud sync. The data layer uses SQLite for structured data and JSON/YAML for configuration.

Data Directory Structure

data/
├── memory/
│   └── memory.db              # RAG vector store (SQLite + sqlite-vec + FTS5)
├── sessions/
│   └── sessions.db            # Chat history, session metadata
├── models/
│   ├── downloaded/            # GGUF model files
│   └── model-cache.json       # Download catalog cache
├── providers.json             # External provider configurations (API keys encrypted)
├── orchestra.json             # Orchestra mode role assignments
├── permissions.json           # Agent tool permission policies
├── calendar/
│   └── calendar.db            # Events, reminders, recurrence rules
├── whatsapp/
│   ├── whatsapp.db            # Messages, contacts, chat metadata
│   └── media/                 # Cached profile photos and media
├── mood/
│   └── mood.db                # Emotion engine state
├── proactive/
│   └── proactive.db           # Intent history, habits, learning state
├── skills/                    # Installed skill SKILL.md files
│   └── <skill-name>/
│       └── SKILL.md
├── cloudsync/
│   └── sync-state.json        # Last sync timestamp, file hashes
├── config/
│   └── config.yaml            # Runtime configuration (copy of config/)
├── identity/
│   ├── system-prompt.yaml     # Active persona definition
│   └── incognito-prompt.yaml  # Incognito mode persona
├── machine.key                # Hardware-derived encryption key (0600)
└── backups/                   # Manual .memo archive exports

Storage Engines

SQLite + sqlite-vec

The primary storage engine for all relational and vector data:

-- Memory store schema
CREATE TABLE memories (
    id TEXT PRIMARY KEY,
    content TEXT NOT NULL,
    timestamp DATETIME NOT NULL,
    importance INTEGER DEFAULT 1,
    retrieval_count INTEGER DEFAULT 0,
    pinned INTEGER DEFAULT 0,
    soft_deleted INTEGER DEFAULT 0,
    deleted_at DATETIME
);

CREATE VIRTUAL TABLE memory_vec USING vec0(
    embedding FLOAT[768],
    chunk TEXT
);

CREATE VIRTUAL TABLE memory_fts USING fts5(
    content,
    tokenize='unicode61'
);

vec0 is sqlite-vec's virtual table for approximate nearest neighbor (ANN) search with quantization. It provides O(log n) vector similarity queries with cosine distance.

FTS5 provides keyword-level search with the unicode61 tokenizer for proper Unicode word boundary handling.

JSON Files

Configuration and lightweight state:

// providers.json
{
  "providers": {
    "openai": {
      "enabled": true,
      "api_key_encrypted": "AES256:GCM:base64...",
      "model": "gpt-4o",
      "context_window": 128000,
      "priority": 1
    },
    "claude": {
      "enabled": true,
      "api_key_encrypted": "AES256:GCM:base64...",
      "model": "claude-sonnet-4-20250514",
      "context_window": 200000,
      "priority": 2
    }
  }
}

API keys are encrypted with AES-256-GCM using a key derived from data/machine.key:

// internal/encryption/crypto.go
func Encrypt(plaintext []byte, key []byte) ([]byte, error) {
    // AES-256-GCM with random 12-byte nonce
    // Output format: nonce || ciphertext || tag
}

func Decrypt(ciphertext []byte, key []byte) ([]byte, error) {
    // Verify tag, decrypt in-place
}

YAML Configuration

# config/config.yaml
user_name: "Buğra"
assistant_name: "Memo"
language: "en"
theme: "pewter"
memory_enabled: true
web_search_mode: true
embedding_model: "nomic-embed-text-v1.5"
max_memory_results: 10
cloud_sync_interval: 50
reminder_lead_time_minutes: 15

Config writes use atomic temp-file + rename to prevent corruption on crash.

RAM Management

Memory Store Indexing

The RAG memory store uses a lazy loading approach:

  1. On startup, only metadata (IDs, timestamps, importance) is loaded into RAM
  2. Embedding vectors stay on disk and are searched via sqlite-vec's indexed virtual table
  3. Full-text search operates entirely within SQLite's FTS5 engine
  4. Only the top N results (default 10) are fully materialized in memory

A memory base of 10,000 entries consumes roughly:

  • ~20 MB on disk (embeddings, text, metadata)
  • ~2 MB in RAM (index structures, metadata cache)

Session Management

Chat sessions are paginated. Only the active session's recent messages (last 100) are held in memory. Older messages are fetched from SQLite on-demand when scrolling.

Model Storage

GGUF model files are stored as regular files on disk. They are memory-mapped by llama.cpp at inference time — only the layers being actively computed occupy RAM. Layers offloaded to GPU reside in VRAM.

Data Integrity

Atomic Writes

All configuration and data files use atomic write operations:

// Pattern used throughout the codebase
func AtomicWrite(path string, data []byte) error {
    tmp := path + ".tmp"
    if err := os.WriteFile(tmp, data, 0600); err != nil {
        return err
    }
    return os.Rename(tmp, path)
}

WAL Checkpoint

Before exports and cloud sync, a WAL checkpoint flushes pending writes:

db.Exec("PRAGMA wal_checkpoint(TRUNCATE)")

Import Safety

Memory imports use temp-file + atomic rename. The original database is never modified in-place during import. This prevents corruption if the import is interrupted.

Factory Reset Protection

data/machine.key is stored outside the data wipe path. A factory reset clears all user data but preserves the encryption key — preventing permanent loss of cloud backup access.

Write Serialization

SQLite writes are serialized through a channel-based queue:

// internal/database/db.go
type DB struct {
    Write chan func(*sql.DB) // Serialized write queue
    Read  *sql.DB            // Concurrent reads
}

func (db *DB) Init(path string) error {
    db.Read, _ = sql.Open("sqlite3", path)
    db.Read.Exec("PRAGMA journal_mode=WAL")
    db.Write = make(chan func(*sql.DB), 100)

    go func() {
        for fn := range db.Write {
            fn(db.Read)
        }
    }()
}

This ensures no two goroutines write simultaneously, preventing database is locked errors while allowing concurrent reads.

Backup & Export

.memo Archive Format

The .memo export is a ZIP file containing:

export.memo
├── memory.db          # RAG vector store
├── sessions.db        # Chat history
├── calendar.db        # Events and reminders
├── whatsapp.db        # Messages and contacts
├── mood.db            # Emotion engine state
├── proactive.db       # Learning history
├── providers.json.enc # Encrypted provider configs
├── orchestra.json     # Orchestra configuration
├── permissions.json   # Agent permissions
├── config.yaml        # App configuration
├── system-prompt.yaml # Identity definition
└── machine.key        # Encryption key (if exporting for same machine)

Google Drive Sync

Cloud backups encrypt each file individually with AES-256-GCM using PBKDF2 (600,000 iterations) key derivation from a user passphrase. Without a passphrase, the machine key is used for automatic keyless encryption.

Encrypted blobs are uploaded to a hidden memo-backup folder on Google Drive. The sync state file (sync-state.json) tracks which files have been synced and their checksums to avoid redundant uploads.