Technical Deep Dive

An under-the-hood look at Memo's architecture and core subsystems.

Architecture Overview

Memo follows a bridge pattern with two runtimes:

• Go backend: manages the llama.cpp process, SQLite persistence, vector store, API server, and agent engine.
• Flutter/Web frontend: provides the cross-platform UI served by the Go API.

The Go binary embeds the Flutter web build and serves it alongside the API. On desktop, the Flutter app wraps the Go binary as a sidecar process.

SQLite + vec0 Persistence

All data lives in a single SQLite database at ~/.memo/memo.db:

conversations      — message history and sessions
memories           — stored facts and embeddings
config             — key-value settings
sync_state         — cloud sync metadata

The vec0 extension powers the vector index. Embeddings are stored as BLOBs and queried with cosine similarity. A Go-native cosine distance fallback is used when the SQLite vec0 extension is unavailable, ensuring compatibility on all platforms.

Llama Process Lifecycle

The Go backend manages the llama.cpp server as a child process:

1. Start: The llama server binary is launched with the configured model and parameters.
2. Health Check: A /health endpoint is polled every 500ms until the server reports ready. Timeout is 30 seconds.
3. Port Management: Ports are assigned dynamically to avoid conflicts. A random available port is selected and passed to the llama server via --port.
4. GPU Detection: On startup, the system checks for CUDA/Metal availability to set GPU layer defaults.
5. Cleanup: On shutdown, the llama process receives SIGTERM and is awaited. A force kill follows after 10 seconds if unresponsive.

Multi-Worker Vector Search

To keep searches fast under load, embedding and similarity computation run across a pool of goroutines. Each worker holds its own copy of the model to avoid lock contention. Queries are distributed via a channel, and results merge into a single ranked list.

End-to-End Sync Strategy

Cloud sync uses an append-only encrypted log:

1. Each change is encrypted with AES-256-GCM using the user's passphrase before leaving the device.
2. The server stores only opaque ciphertext blobs in insertion order.
3. On sync, the client downloads new entries, decrypts them locally, and merges into the local database.
4. Conflict resolution is last-write-wins per message, clocked by a monotonically increasing sequence number.

External Provider System

Memo supports pluggable LLM providers via a unified interface:

type Provider interface {
    Chat(ctx, messages)  (response, error)
    Models()             ([]Model, error)
    ValidateConfig(cfg)  error
}

Built-in providers include OpenAI, Anthropic, Google AI, and Ollama. Each is registered in the provider registry and selected by setting api.base_url and the corresponding API key.

Agent Engine

The agent subsystem is built around a pipeline:

1. Tool Registry — A centralized registry of available tools (file system, shell, web, calendar) with JSON Schema descriptions for LLM function calling.
2. Permission Manager — Controls which tools an agent may invoke. Permissions are scoped per session with allow/deny lists.
3. Sandbox — Tool execution runs in a restricted environment. File access is limited to the workspace directory. Shell commands use timeouts. Network calls go through an allowlist.
4. Pipeline — The agent loop: receive message → plan (LLM decides tools) → execute (invoke tools) → observe (capture results) → respond.

Orchestra Mode

Orchestra coordinates multiple agents working together:

• Conductor Pattern: A central conductor agent receives the user's request, decomposes it into subtasks, and assigns them to specialist agents.
• Roles: Each agent has a defined role (researcher, coder, reviewer, planner) with a tailored system prompt and tool set.
• Progress Streaming: The conductor emits structured progress events (planning, delegating, agent_response, merging) that the UI renders in real time.