Skip to content
Subscribe
Rafael Cordeiro, colunista

The semantic layer, how an AI agent knows what company data means

Rafael Cordeiro · 3 min read
Share

Part 2 of 4 in the series Enterprise AI architecture. Previous part, AI Gateway. Next part, the knowledge layer.

In the previous part of this series, the AI Gateway came up as the control point of enterprise AI. But controlling how AI accesses your systems solves only part of the problem.

The next question is harder. When an agent accesses your company's data, how does it know what that data actually means?

That technical detail decides whether the answer can be trusted.

A fast, structured and wrong answer

Imagine asking an agent what net revenue was last quarter.

It runs the query. It joins a few tables. And it answers R$184.7 million.

The answer comes back fast, structured and with two decimal places. There is just one problem. It may be wrong.

Maybe it used the wrong column, made an incorrect join, included cancellations or read net revenue differently from the way Finance does.

And this is one of the risks that worry me most about AI in business. The more convincing the answer, the more dangerous a silent error can be.

The benchmarks show the drop

On Spider 1.0, methods based on GPT-4 reached 91.2% accuracy. On Spider 2.0, which is closer to a real enterprise environment, with large and complex databases, the best agent based on o1-preview solved only 21.3% of the tasks.

The bottleneck is context.

Editor's context. Spider 2.0 was introduced in November 2024 in a paper by Fangyu Lei and co-authors, accepted at the ICLR 2025 conference. It tests SQL generation in workflows built on real enterprise databases, far larger than those in Spider 1.0.

And real companies have something schemas don't explain. Context.

What counts as an active customer? When is revenue recognized? Which margin should be used? Which table is the official one?

The semantic layer translates data into business concepts

This is where the Semantic Layer comes in.

It translates technical data into business concepts. With it, you create governed, reusable definitions that every agent shares.

Editor's context. According to dbt Labs, the semantic layer organizes metrics, dimensions, entities and relationships into a structure defined in advance. Queries are then generated deterministically from those definitions, and the trade-off is coverage, because it can only answer what has been modeled.

In a benchmark dbt Labs published in April 2026, with 11 questions about a fictional insurance company, text-to-SQL got 64.5% right on the original tables. After minimal data modeling, the Semantic Layer reached 98.2% with Claude Sonnet 4.6 and 100% with GPT-5.3-Codex. With the same modeling, text-to-SQL scored 90.0% and 84.1%, respectively.

More important than the numbers is the point behind them, that a model can query data without understanding the business.

AI architecture evolves in layers

That is why AI architecture evolves in layers. There are four, from top to bottom.

  • AI Gateway controls access, models and policies.
  • Semantic Layer defines metrics and meaning.
  • Agents reason over the context.
  • Execution Layer carries out actions in the systems.

And then there are Evals and Observability, to answer four questions. Did the agent get it right? What data did it use? What rule did it apply? What impact did it have?

We are moving from "connecting AI to data" to building systems where agents make reliable decisions.

Who gets in, what it means and whether it can be trusted

The AI Gateway controls who gets in. The Semantic Layer defines what things mean. And governance determines whether we can trust what happens next.

To me, this is the real architecture of an AI-driven company.

In your company, does AI understand what the data means, or does it just access it?

In short

  • The AI Gateway controls who gets in.
  • The Semantic Layer defines what things mean.
  • And governance determines whether we can trust what happens next.
What has changed since August 13, 2026 · Editor's note
In August 2026, dbt Labs published a new round of the same benchmark, with the run history open to the public. On September 2, Microsoft made it possible for Copilot skills in SharePoint to be improved with tests and evaluations.

Further reading

Part 2 of 4 in the series Enterprise AI architecture. Previous part, AI Gateway. Next part, the knowledge layer.

Originally published on LinkedIn on August 13, 2026. See the original post. This version has been revised and expanded with context and sources. The views are those of the author, a co-founder of OperOS AI.

Read next