Over the past year, I noticed that the biggest blocker to experimenting with new AI products is that my workflows, memories, skills, connectors, and so on are locked into current products like the Claude, Codex, and OpenCode desktop apps. I think we’ve all been there: Claude knows how Project A works and is already built into it, while Codex knows how to archive Project B in TestFlight properly.
Much of the problem here is a lack of local-first infrastructure for creating a centralized hub where all your projects, workflows, preferences, and personal information live. The biggest improvement I made to my AI development workflow was not switching models, buying another max plan, or using someone’s Rube Goldberg AI agent. It was coming to terms with the fact that an agent is only as useful as the workspace it can understand.
Most development environments are built for one human with local memory. You know where the weird script lives. You remember why the database has three nearly identical tables. You can skim a file and silently discard the parts that do not matter. An agent cannot do that unless the workspace gives it durable, searchable context and a narrow way to retrieve exactly the code or note it needs.
That is what I built around QMD, jCodeMunch, and agent memory: a local workspace where context is not trapped in chat history, source files are retrieved by symbol instead of dumped whole, and decisions survive long enough to help the next session.
The Memory Layer
The foundation is a plain Markdown memory vault in iCloud: agent-memory. Each project gets its own folder. Durable facts live in MEMORY.md, session summaries go in sessions/, explicit decisions go in decisions/, and supporting notes go in notes/.
That structure matters because it keeps memory auditable. I can open the same files in a text editor, Obsidian, or a terminal. The agent can search them before touching a repo. Nothing important depends on a proprietary transcript view or a single assistant remembering the right thing at the right time.
QMD turns that vault into a local search engine. It indexes the Markdown collections, builds embeddings, and gives agents a way to ask, “What do we already know about this project?” before they start inventing answers. In practice, that changes the tone of the work. Instead of re-explaining the same constraints every session, I can make the retrieval step part of the operating contract.
The Code Layer
QMD answers the memory question. jCodeMunch answers the code question.
The rule is simple: do not read entire files when a symbol-level lookup will do. jCodeMunch indexes source code with tree-sitter, then lets the agent search for functions, classes, methods, constants, and references directly. jDocMunch does the same for documentation sections, and jDataMunch covers structured data. Together, they replace the old habit of shoving thousands of tokens into context just to find one function.
The payoff is partly cost, but the better reason is attention. A good coding session starts with the smallest true slice of the system. If the agent needs a Swift type, retrieve the type. If it needs a route handler, retrieve the handler. If the index is stale, refresh it before reasoning from it. The workspace becomes less like a pile of files and more like a map with coordinates.
The Contract
The tools only work because the workspace tells agents how to use them. That job belongs to AGENTS.md and the project-specific instructions around it.
At the start of a session, the agent checks whether the relevant repo is indexed. Before starting on a topic, it searches QMD for prior context. For code, it uses symbol and outline retrieval first. For documentation, it searches sections instead of reading whole documents. Direct file reads are still allowed when they are the right tool, especially before editing, but they are no longer the default reflex.
This sounds procedural, and it is. That is the point. AI tools get much better when preferences become executable habits instead of vibes. The workspace should not rely on the user saying “remember to search memory first” every time. The rule should already be there, waiting.
Machine-Specific Reality
The most humbling part of the setup was making it boring across machines.
My Mac mini and MacBook Air do not share the same home path. One uses /Users/username1234; the other uses /Users/usernames4321. QMD indexes and embeddings are local per machine, even though the Markdown memory vault syncs through iCloud. LaunchAgent schedules are machine-specific too, which means a working maintenance job on one Mac should not automatically imply that the other Mac should run the same schedule against the same shared vault.
Those details are not glamorous, but they are where reliability lives. The durable notes now record the exact binary paths, MCP commands, runner scripts, and verification commands for each machine. When something breaks, the agent can check the local facts instead of copying a path from the wrong computer and making the problem stranger.
The Maintenance Loop
Memory has to be maintained or it turns into another junk drawer. The workflow is intentionally plain: append session summaries when useful, promote durable facts into curated memory, run QMD update and embedding refreshes, and lint the memory so stale or contradictory notes do not silently accumulate.
Some of that maintenance can run on a schedule, but the important bit is the shape of the loop. Work creates evidence. Evidence becomes a session note. Repeated facts become curated memory. Curated memory becomes searchable context for the next agent. The system improves because the workspace records what actually happened, not because any one conversation is expected to last forever.
What Changed
The practical effect is that sessions start closer to the real problem. When I ask for help with Studio, the agent can learn that it is a native macOS document/workspace creative production app before it reaches for web-app assumptions. When I ask about QMD, it can check whether the current machine has the native MCP tools exposed before blaming the shell path. When I ask for a website change, it can refresh the Website index and inspect the existing writing structure before adding a page.
That is the real promise of this setup. Not perfect memory. Not magic autonomy. Just a workspace that makes the right context cheap to find, easy to verify, and durable enough to survive the end of a chat.
The quality of the models still matters; a frontier closed-source model is going to outperform a six-month-old DeepSeek model. But now easy, context-efficient access to memory is no longer the impediment. Once the workspace can tell an agent where it is, the agent can spend a lot more of its effort helping instead of getting its bearings inside a codebase.
The main tools behind this setup are available on GitHub:
https://github.com/jgravelle/jcodemunch-mcp
← Back to Writing