How Tellius Keeps Enterprise AI Affordable at Scale
AI agents drive inference costs to scale with usage/adoption. Every question an AI worker answers, every agentic app opened, every agentic workflow run overnight is a stack of model calls, and can be especially costly if frontier models read the schema and work everything out from scratch each time. This blog goes into the approaches Tellius uses to keep AI bills affordable at scale.
What running agents naively costs
Pointing frontier models at enterprise data for answers is a common AI starting point.
But for the frontier model to know your data, the schema must go in the prompt, which is costly as the model parses hundreds of warehouse tables to figure out which one is relevant for the question at hand. Wrong columns, bad joins, type mismatches, and other issues can result in model retries, with each retry resending the full context to the model driving up cost.
When a sales director asks for last week's revenue by region, the model reads the schema, works out which of those hundreds of tables hold revenue and how region joins to them, writes the SQL, and fixes whatever breaks on the first try. Ask the same question next Monday and the model, having kept nothing from the week before, does all of that again for the same token bill. A second rep who asks "how did each region do on revenue last week" triggers a third full run, because the model has no way to know that is the query it already wrote. Scheduled work has the same problem on a timer. A nightly job that refreshes a pipeline summary, or an internal app a hundred reps open every morning, re-reasons its full chain of steps on every run, even though the question and the schema are the same as yesterday and only the data underneath has moved.
Finally, agents doesn't just ask one question — it breaks a task into many, each with it’s own model calls. Agentic apps used by hundreds of reps run the risk of hitting that loop all day. Naive architecture makes every one of those calls expensive.
How Tellius keeps LLM costs low
Every agent action, app, and agentic workflow undergoes the same inner loop of turning a request into validated SQL/Python and running it. Tellius orders that loop so the cheap checks run first and most requests never reach the expensive model.
1. Semantic cache check. Tellius caches past questions and the SQL that answered them. When a new question comes in, Tellius checks this cache before calling any model. If a similar question is already there, a smaller model edits the stored SQL to fit the new wording instead of writing a query from scratch. If the match is very close, above 0.9 similarity, the query also skips validation. A brand-new deployment has an empty cache and gets none of this saving until people start asking questions and accepting answers.
2. Compact schema assembly. On a cache miss, the SQL/Python generator needs to know the schema. Instead of the raw database, it gets the Business View's metadata as one block. That block lists the visible columns and the joins that are allowed between them. It includes the definition of each computed metric and the business terms that map to specific columns, plus any phrase or query learnings the team has added. If the generator needs more detail on a particular column, it fetches that through a tool call. The prompt ends up far smaller than the full-schema dump in the previous section.
3. Bounded generation. The generator writes SQL/Python against that block. If the warehouse throws an error on the query, or the validation step flags a problem, the generator tries again with the error attached. This retry loop is where a do-it-yourself build spends most of its tokens, because each retry re-sends the whole prompt. Tellius limits the retries so that after it stops it tells the user it could not produce a query it trusts. A retry cap sounds like it would mean more failed answers. In practice, most queries never retry at all, with ~90% pass validation on the first attempt. The ones still failing after several rounds are usually questions the data model cannot answer the way they were asked, because the metric is ambiguous or the column isn't there. More retries don't fix that, they just cost more.
4. Validation gate. Not every query gets validated. A small, fast model reads the question and the generated SQL and estimates how likely the SQL is to be wrong. Simple, low-risk queries skip validation and run. The rest go through a validation pass, which is a separate LLM call. First it compares the SQL to the question. It checks that the metric is the one the user asked for, that the time window and grain are right, and that no filter was added that the user didn't mention. Then it checks the SQL against the Business View's rules. Computed metrics have to be used exactly as defined. Only the allowed joins can appear. Row-level policies have to be intact, and each metric has to use its own date column. Customer-specific rules get injected here as well. On a Nielsen weekly dataset, for example, "this month" has to mean the latest four weeks, and distribution percentages can't be summed across product hierarchy levels. The validator does not check syntax, since the warehouse rejects bad syntax on its own. It also passes SQL that bends a rule when the result is unaffected. Flagging either would trigger retries that change nothing.
5. Warehouse execution. The SQL runs on the warehouse. Tellius starts execution while validation is still finishing and shows a preview of the answer while the check completes. If the query errors out on execution, validation stops there. The LLM does not calculate anything at this stage. The warehouse computes the result against the governed schema, and the model's last job is to write up what came back. Execution is deterministic on any platform, since a warehouse returns the same result for the same SQL. Getting the same answer twice depends on running the same SQL twice. The cache does that for repeat questions, and stored SQL does it for agentic workflows. A build that regenerates the query on every ask can give two people two different numbers for the same question.
Model Gateways keep LLM costs proportional to the work
Not every step needs the same model. Pulling a date range out of "last quarter" is not the same job as planning a multi-step analysis or orchestrating an agent's next action. Tellius’s model gateway automatically routes each of roughly 35 task types — SQL generation, chart config, time-filter extraction, orchestration, summarization, and more — to a model chosen per deployment, with fallback from lighter to heavier tiers. Simple, high-volume deterministic steps run on light models, and the reasoning-tier models are reserved for the steps that need them.
Routing goes the other way when it has to. A request that involves a skills catalog escalates to the reasoning model, because smaller models aren't reliable there. It matches the model to the work, not the other way around.
Why LLM costs drop in Tellius as usage grows
Three caches sit in front of the pipeline, and each one saves more the longer the system runs.
- Semantic cache. Every question-and-SQL pair a user accepts is stored and matched against future questions. That pair is one the next user, or the next agent run, does not pay to generate. The cache starts empty on a new deployment. As an app or coworker gets used, it fills with the team's real questions, and the share of requests answered from the cache goes up.
- Prompt-prefix caching. Each tool-calling loop re-sends the same system prompt and Business View block on every call. The model providers cache that repeated prefix and bill it at a reduced rate instead of full price each time. Tellius turns this on by default where the provider supports it.
- Schema assembly. The compact Business View block from the previous section is built once and stored. Later requests against the same Business View reuse it instead of rebuilding it on every call.
An all-LLM build has none of these, so the hundredth request costs the same as the first. A Tellius deployment at month eighteen runs the same pipeline it ran at month one, but with eighteen months of accepted SQL and cached context in front of it. By then most requests are answered from the cache or with a cached prefix, and the average cost per request keeps falling as the cache fills.
Agentic workflows are created once and run cheaply
An agentic workflow has one creator and many consumers. The creator describes the job in plain language, Kaiya works out the steps and writes the SQL for each one, and the creator reviews the plan and puts it on a schedule. Token usage here is the same as for any question asked in chat. It happens once.
From then on the workflow runs on its schedule. Each run executes the stored SQL against live data, and the model is called at the end to write the summary. The reasoning is not redone. Fifty people can receive the briefing every morning, and the reasoning cost was paid once, by the creator, on the day it was built. In a matched test an agentic workflow cost a few dollars to create and about fifty cents per run, and that per-run cost is the summary call plus warehouse time. Run the same job through a chat-based tool and each of those fifty people asks the question themselves, so the full reasoning runs fifty times a day instead of once.
The stored SQL is regenerated only when the data model underneath it changes, such as a column moving or a metric being redefined. In a production system that is a rare, deliberate change, not something that happens on a schedule.
Apps spend tokens when they're built, not when they're used
Token usage for an App happens while it is being built. Someone describes the app they want, and Kaiya does the analysis and generates both the SQL and the app itself. Edits after that, such as asking Kaiya to fix a chart or add a filter, also use the model. Once the app is published, people opening and using it do not, however many of them there are.
When the data needs to be refreshed, the stored SQL runs again against the warehouse and the new results replace the old ones in the app. Nothing is re-analyzed. The refresh is a deterministic execution of queries that were already written.
An App also inherits the platform's governance. Row-level security, access policies, deployment and scaling are handled by Tellius, so someone on a commercial team can put an internal tool in front of a hundred colleagues without a data-engineering team behind it.
What it costs to get a correct answer
The number that matters isn't cost per token, or even cost per query — it's cost per answer you can use. An all-LLM build hides its real cost in the answers it gets wrong. A query that returns the wrong number has to be caught, re-asked, and re-run, and the same question asked twice can come back with two different numbers. You pay for every attempt, including the ones you throw away.
On the same workload Tellius reaches a usable answer in fewer model calls, because most requests are served from cache or skip validation, and the ones that do generate SQL run under a bounded loop rather than an open-ended correction spiral. First time through, the saving is modest — roughly a quarter against thirty to forty cents. The gap opens on the second ask and every one after: a repeated question comes back for pennies, and a saved agentic workflow runs its stored SQL and pays only for the summary step.
The all-LLM column pays close to full price on every ask, with no way to promise the answer matches the last one — the same question can return a different number. Tellius doesn't re-reason what it has already validated, so the thousandth run costs a fraction of the first and returns the same answer.
Where the saving shows up depends on how the deployment is set up. If Tellius runs against your own OpenAI or Anthropic contract, the smaller call count lands on that bill. If inference is included in Tellius credits, it is priced on the same pipeline. In both cases the cost is decided by the number and size of model calls per correct answer, which is what the pipeline above reduces.
When the model changes, the savings stay
Models turn over. GPT-6 Astra just shipped, the next Claude will ship. A build that wrapped its logic in prompts has to re-prompt, re-eval, and re-validate on every swap. The prompts were the product and they just expired. The routing map, Business Views, caches, and bounded pipelines Tellius constructed don't depend on which models runs underneath, so a swap carries the savings forward.
If you're evaluating this, measure cost per answered question, not cost per token. Count every call it takes to get an answer you'd put in front of a CFO, and compare that.
Get release updates delivered straight to your inbox.
No spam—we hate it as much as you do!
LLM cost per query is the total inference spend — every input and output token across every model call — needed to turn one request into an answer someone can act on. It counts retries, re-sent context, and validation passes, not just a single prompt. It's the number on your OpenAI or Anthropic bill.
Tellius keeps the model out of the work it does badly. A semantic cache reuses validated SQL, a compact Business View replaces full-schema dumps, per-task routing sends simple steps to lighter models, and execution runs on the warehouse. The result is fewer and smaller model calls per answered request.
A semantic cache stores validated question-to-SQL pairs and matches new questions by meaning, not exact wording. A close match reuses previously validated SQL instead of generating fresh, and a near-exact match can skip validation entirely, so repeated and similar requests cost a fraction of new ones.
Direct LLM-to-warehouse querying re-sends the full schema on every request, spends tokens correcting failed SQL, and reuses nothing between similar questions. A pipeline that curates context, caps retries, and executes on the warehouse answers the same request with far fewer model calls.
Tellius splits a request into roughly 35 task types — SQL generation, chart config, time-filter extraction, orchestration, summarization, and more — each assignable to a different model per deployment, with light-to-heavy fallback. Simple, high-volume steps run on light models, and reasoning-tier models handle the steps that need them.
No. The semantic cache stores question-to-SQL mappings, not results, so a cached query re-runs against live data every time. You skip the cost of regenerating the query while the numbers still reflect the current state of the warehouse.
An agentic workflow generates the SQL for each step once, when it is built, and stores it. Each scheduled run executes that stored SQL against live data and calls the model only to write the summary, so the build cost is amortized across every run and each run costs a fraction of it. In a matched test an agentic workflow cost a few dollars to create and about fifty cents per run. The stored SQL is regenerated only when the underlying data model changes.
Yes. You describe the workflow or app in plain language, and Tellius generates and stores the steps. The App inherits the platform's governance — row-level security, access policies, deployment, and scaling — so what a business user builds is enterprise-grade without a data-engineering team standing behind each one.
Very little changes. The routing map, Business Views, caches, and pipeline don't depend on which model runs underneath, so swapping in a newer model carries the existing savings forward. A build wrapped in prompts, by contrast, re-prompts and re-validates on every model change.
It depends on the deployment. Tellius can run against your own OpenAI or Anthropic contract, in which case inference is billed to you at your provider's rates. It can also run on Tellius-provided inference, which is metered in credits. In both cases the pipeline described above reduces the number and size of model calls per correct answer, and that is what keeps the cost down.
Measure cost per answered question, not cost per token. Count every model call it takes to get an answer you'd put in front of a CFO — retries, re-sent context, validation passes — and compare that. A low per-token rate means little if the architecture burns ten calls per answer.
The best platform is one whose architecture minimizes model calls per answered request: a semantic cache that reuses validated SQL, curated context instead of full-schema dumps, per-task routing, and warehouse execution. Tellius is built around these controls.
How Agent Debugger Audits a Kaiya Analysis End to End
See how Kaiya’s Debug Agent provides an independent second opinion on AI-generated analysis by auditing the full analytical trace—not just the final answer. It reviews the data model, semantic layer, SQL and logic, skills, summaries, and recommendations to uncover errors that can otherwise look convincing. When it finds a problem, it determines whether to fix the individual analysis, the underlying skill, or the data model—helping enterprises make AI analytics more transparent, explainable, and trustworthy.
Why AI Workflows Give Different Answers Every Time — and How to Make Them Deterministic
One of the biggest challenges organizations face when deploying AI agents and workflows is inconsistency. The same question can generate different answers, recommendations, or actions depending on context, prompts, model behavior, and execution paths. This blog explains why AI systems are inherently probabilistic, where variability originates in modern agent architectures, and why prompt engineering alone cannot solve the problem. It explores how leading enterprises are introducing deterministic controls through semantic layers, governed business logic, structured workflows, context management, validation frameworks, and agent orchestration.