I let an AI assistant help me analyze tickets on the FileMaker solutions I maintain. The assistant reads the Database Design Report, the only complete map of a solution, and traces where fields are used, which scripts call which, what a button actually does. On my biggest system that map is more than 500 MB of UTF-16 XML, and the assistant would stream the whole thing again for every question it had, a fresh pass and a hand-written throwaway parser, dozens of times per ticket.

fmsonar fixes that. It parses the DDR once into a SQLite index (every field, script, layout object, and reference edge), and after that every question is a millisecond query. The assistant works through a CLI, with a ready-made skill that teaches Claude Code to use it. For humans there is a browser explorer at fmsonar.com that parses locally, so nothing is uploaded (watch the network tab, or go offline; it still works).

A nautical-style contour chart: one contact point emits a sonar ping among depth soundings. Caption inside the chart reads: every reference echoes back.

An index obviously makes AI analysis better. That was the whole reason to build it. So instead of assuming it, I tested it: take a real production ticket and analyze it twice with the same AI setup and the same prompt. One arm gets fmsonar; the other is restricted to raw-XML streaming. The arms run in isolated sessions, the test is registered before either arm starts, and every claim is verified against the DDR before it is credited. Compare depth, cost, and correctness.

fmsonar lost the early rounds. Each loss changed the tool, and those changes did more for it than my own feature ideas did. This post is the story of the losses.

The first loss: tunnel vision

The streaming arm found the controller script, the exact sending mechanism, and a latent bug (a global variable written by one script and read by nothing, anywhere). The fmsonar arm found none of it. It looked up the one script the ticket named, read it, answered the narrow question, and stopped.

What made this uncomfortable is that fmsonar’s queries were better. Streaming is so clunky that the analyst’s first move was a survey, “list every script in this domain”, which put the controller and the dispatcher on the table before any deep reading began. The index’s surgical lookup skipped all of that, and nothing in the setup pushed it to look wider. The clunky tool had been forcing a good habit by accident.

The fix was not a code change. I wrote the investigation method into the tool’s own documentation, where the AI reads it at query time: survey the script family before deep-reading one script, always climb to callers, sweep the globals a script writes, report real FileMaker ids. If depth is not going to come from friction anymore, it has to come from the written working method.

The second loss: a blind spot, and a worse problem underneath

The next round, the method worked. The fmsonar analysis surveyed, climbed, even caught itself following a wrong same-named script and corrected course. It still lost on the verdict, for a completely new reason: the defect lived in layout data, in what a button passes to its script and a visibility condition on a card, and I had never told the tool’s users (myself included) that layouts have readable bodies too. Everything the streaming arm dug out of twenty thousand lines of raw XML was sitting in the index the whole time, three queries away. Nobody ran those three queries.

The claim-by-claim verification surfaced the worse problem. The index being queried was silently stale: built one day before a parser improvement, it reported “98% healthy” while missing every Set Field write-target in the solution, all 8,622 of them. A confident answer from a stale index looks exactly like a confident answer from a fresh one.

Both became releases. Layout objects are now first-class queryable entities (button parameters, hide conditions, tooltips). A one-command investigate report runs the whole method per script. And every index now carries a parser-version stamp that warns loudly at query time when the index predates the parser.

The win, and the two results that mattered more

With completeness as the metric (a coverage-style ticket, “find every cached field that goes stale and every write site”, scored against a mechanically established ground truth), and with the test run as a 2x2 across two AI models to keep the model effect out of the tool conclusion, fmsonar won within both models while being the cheapest run in each pair: about a quarter less wall-clock and a third fewer tokens, with better recall. A follow-up round on a fresh ticket confirmed it: every consumer of the affected fields found, mutation sites at 8/9 recall with zero false positives, in ten minutes.

Two side results mattered more than the win.

The first: the model effect was bigger than the tool effect. The decisive mechanism in one ticket (a script that appears to flip a status flag but actually just opens a confirmation panel, the real change happening later in a different script) was caught by both arms on one model and missed by both arms on the other. Same tool, same index, same method. Tooling narrowed the cost; it did not close that comprehension gap. No release can fix that, so the docs now say it plainly instead: use the most capable model available for mechanism-hunting.

The second: the merged result beat every individual result. Each analysis contributed something verified that the others missed. One found a family of snapshot fields the ground-truth queries were structurally blind to, another noticed a write site was commented out, a third proved a source field was unstored, which quietly invalidated part of the first one’s proposed fix. For work that matters, two independent analyses plus a verify-and-merge pass beat one analysis run twice as long. This one did become a change: the investigation method in QUERIES.md, the recipe book that ships with the tool and that the AI reads at query time, gained a sixth step recommending exactly that pass for high-stakes work.

What I would tell anyone building AI tooling

  1. Test the tool against the thing it replaces, on real work. The losses were more valuable than the wins: each one became a nameable, shippable fix.
  2. Ship the investigation method inside the tool’s docs, where the AI reads it, not in a wiki nobody opens.
  3. Version-stamp your artifacts. A silently stale index passed every health check while missing 8,622 references.
  4. Measure the model and the tool separately. One of them mattered more, and it was not the tool.
  5. Independent runs merge better than long runs. Budget a verify-and-merge pass for high-stakes work.

Where the roadmap comes from now

The A/B rounds did not stop at launch. The open items on the roadmap are, verbatim, whatever the last round caught. Deletion turned out to be invisible to field-writer sweeps (cascade deletes live in relationship options, not script steps), so cascade-aware deletion tracking is the feature in progress. The same goes for everyday use: a user screenshot showed commented-out script steps being counted as live code across the whole tool. That shipped as v1.7.0, with dead code flagged, excluded from usage counts, and queryable on its own.

Try it

The tool all of this produced: one parse of your DDR into a queryable index, where-used answers in milliseconds (fields, scripts, layout objects, custom functions, relationships), an investigate command that runs the full method on any script, usage counts that separate live code from commented-out code, full-text search over everything the structure can’t see, and an index that warns you when it is stale.

Installing fmsonar, indexing a FileMaker Tasks solution, and asking where a field is used from the command line
From zero: install, index a DDR, and ask where a field is used. Recorded on the FileMaker Tasks starter, nothing client-specific.
Terminal output of the fmsonar investigate command, showing a script's callers, the fields it writes to, and its recursive call chain.
One command per script: callers, writes, and the recursive call chain, with commented-out steps excluded.
  • Browser, no install: fmsonar.com. Drop a DDR on it; parsing runs locally, nothing is uploaded.
  • CLI + AI skill: pipx install fmsonar, then fmsonar install-skill for Claude Code.
  • Source, MIT: github.com/oogi-io/fm-ddr-analyzer, including the query recipe book and the honest coverage notes.
  • Sibling tool: fmstyle, a deterministic formatter and linter for FileMaker calculations, built on the same principles: pure-stdlib Python, everything local, free and MIT.

It is free. If you feed it a DDR and something surprises you, let me know: open an issue on GitHub, or use the contact form for anything client-confidential.