One Shot at Understanding

The usual way to build “ask your data a question” is to send the question and the database schema to an AI model, ask it for a query, and run whatever comes back. It demos well. It also fails in a costly way: the query runs, returns numbers, and answers a question nobody asked.

A query with a syntax error costs nothing, because you see it straight away. A plausible but wrong number gets read out in a meeting.

Two separate problems

The version that works treats understanding the question and running the query as two different jobs.

The query engine has nothing to do with language. It takes a structured description of a query (which records, which fields, which conditions), checks every part of it against a list of what exists, and runs it. No AI is involved, it costs nothing to run, and it gives the same answer every time.

The AI has one job: turning the person’s sentence into that structured description. Not into results, and not into a database query, but into something a person could have written by hand.

Because of that split, the engine keeps answering when the AI service is down, and nothing else in the system depends on the AI.

Rule out the obvious mistakes

A common mistake is the AI naming a field that doesn’t exist. The usual fix is to tell it not to, in the prompt.

A better fix is to make it impossible. The AI can only choose from fields that exist, so it can’t invent one. Permissions work the same way: if it can only choose fields this person is allowed to see, it can’t pick one they aren’t.

That still leaves the hard case. The AI can choose a real field, the wrong time period or the wrong condition, and produce a valid query that answers a different question. No list of allowed fields prevents that, because nothing is malformed. The understanding is wrong.

Show what was understood

So instead of trying to prevent that mistake, the design makes it visible.

Every answer shows how the question was understood, as plain statements: this field, this condition, this value, this time period. Each one can be edited. Changing one re-runs the query engine without asking the AI again, so it’s instant and free.

That’s the core idea: the AI gets one try at understanding the question. The person can correct it as often as they like.

It matters that corrections are free. If each one cost an AI call, people would make fewer of them and settle for the first answer. Because the engine costs nothing to run, the interface can invite corrections.

What it costs

Two things.

First, the list of what can be queried is used in several places: the AI’s instructions, the limits on what it may choose, and the engine’s checks. If one of them changes and the others don’t, they disagree without any error. Tests have to check all of them against the real data model, because nothing else will.

Second, the interface does more work. Showing how a question was understood is harder to build than just showing an answer, and it looks less impressive. A system that says “here’s what I think you asked” admits it might be wrong. The version that only shows an answer looks more certain, but it’s just as uncertain. It hides it.