Introduction
they were killed because they worked inside the Valley of Choice in BI.
The Valley of Choice describes the right balance between the difficulty of a problem and the effort we are willing to put into solving it.
For example, an important business question about how revenue is generated, and why different versions of revenue have changed over time may warrant a manual audit. It can enable deep discovery driven by multiple SQL queries and document results and trace lineage.
This is High Effort, High Precision.
In contrast, simple questions for simple data sets like “How many subscribers did I get yesterday?” can be satisfied with low-effort, simple methods such as basic scripting to SQL, as seen in tools like Thinkspot.
In the middle are the dashboards. Dashboards provide a moderate-effort solution to a complex problem. They are capable of answering multiple questions and pivoting data to explain anomalies and forecast conditions. They don't help find out understanding, but it helped you deliver information you have already received.
If something is asked often enough, it qualifies for a dashboard, and quickly distills the confusing, one-off, manually-provided information into something simple and reproducible.
Agenttic Analytics offers a new way to find, and provide insights. That's text-to-SQL. This new approach means that hitherto difficult questions can be answered quickly.
Tools like Hex integrate memory into their platforms, which means that even complex queries, when combined with memory, allow Text-to-SQL to solve complex problems.
This changes the required amount of effort to complexity and will reshape the way we think about math. Let's see how.
Defining the Valley of Choice in BI
The Valley of Choice can be shown below.
It describes the trade-off between speed and accuracy determined by the nature of the need or understanding. Holding all things constant, the most important questions require a high level of precision, which also justifies the most time-consuming methods (High Effort).
It's like saying you can buy a delicious sandwich at your deli for $20 or go to 11 Madison Park for $500. Two different products, two different options, two different price points. Dashboards are like eating at a regular restaurant. $100 per person, but not necessary if you are an eligible bachelor in New York.
Now: Imagine there is a new restaurant. The New Restuarant is very special. Food is a wonderful thing; everything is organic. Excellent service. And a lot of celebrities go there. It is highly desirable. You can get a delicious 3-course meal for $40. We can call it Hugo's Diner.
In fact, it is part of a new trend. Hugo has a large restaurant empire. There is Hugo's Diner, but there is also Casa Hugo, Chez Hugo, Hugo's table, and even Hugo-San. Hugo-San offers the best omakase in town for $39.50.
What are you doing? Where do you eat? 11 Madison Park suddenly isn't so busy anymore. This is what happens with BI.
Simply put, the cost and speed of running a query using new age script to sql is incredibly low. It means there is no reason to do anything else, unless you really have to. Either way, you're definitely not moving down market.
Yes – sometimes you may have to do something manually. Just to confirm. And by the same token, nothing touches the day (data?) at night like Dorsia.
Shifting Valley: The Analytics bar just got higher
Consider that you don't need to work hard for the semantic layer to run and Text to SQL can solve, reliably, repeatedly, 95% of your queries.
A reviewer is almost identical to a senior reviewer. They will be asked some really painful questions. They are probably the people who designed the semantic layer in the first place.
The price of accuracy has become too high. To do something personally, it is better to ask the most important question.
What does this mean for analysts? Yes, it's the same as software engineers say. Now it means that the average people actually the usefulness has been very little – in a respectable way, of course.
An analyst who can't be a semantic architect will be useless in 12 months. They will be replaced by information agents. And if they persist, then CTOs are doing something wrong and should bring in Service Partners to help them implement their semantic layer.
The bar for performance in Analytics has already been set high. There is a new competitor on the block and his name is A. Gent. Like software engineering – the young are out, and the old are in.
What about the cost of tokens?
The cost of tokens is an interesting point. The cost of the tokens has generally decreased by 95% from the original models. Imaging costs continue to decline, and are expected to be reduced by another 95% in the next few years.
Apart from this, companies like Uber are busy with their debt obligations. After all, the credits are fun. Claude is addictive. Tokenmaxxing is encouraged.
Indeed, the price of boundary models it did not fall significantly. Claude's cost in 2023 was $32/million output tokens. Now it's still $25.
In fact, the use of tokens is expected to increase significantly as the systems become more complex – it is not clear when, if at all, the benefits will appear. For example, 1 million output tokens for Claude is about 25 dollars. Let's say running a few queries with an agent is 1,000 tokens. This means that the query cost is 0.025 of 2.5 cents.
10,000 people doing one of these a day 365 days a year is $91k. It's not bad.
If the cost of analysts was, say, 10 analysts at $100k each, then again the ROI starts to look good. It looks even better if the inference cost is reduced, which it should be.
I have a suspicion that the consideration (1,000 tokens for “understanding”) is too low. However, reducing observation costs and improving technology will continue to reduce it.
For example, let's say at first it takes an agent a long time to learn that when marketing says “show me revenue” it's a different set of questions when finance says “show me revenue”.
Once this is combined, and done over and over again, these should be small tokens. It should be a dashboard, or a continuous query, or maybe a ability of the agent. We don't need a million output tokens when people ask the same questions every week.
And usually, people ask the same questions as time goes by.
Therefore I am bullish on token costs. Although there is an unavoidably strange time in the market at the moment, when the cost of speculation is possible is very high, I believe the cost of speculation will go down. The number of tokens required to generate data reliably and repeatedly will also decrease as people build reliable and repeatable systems.
Which BI vendors are winning and losing?
There are several factors at play:
- Power of agent
- Market Position
- Size and Location
Agent Power (Agenttic Statistics)
It should go without saying that the power of the agent is important. Currently there are few companies with powerful analytics agents.
- Lightdash
- Omni
- Hex
- TextQL
These companies are all successful for different reasons. Lightdash has tight integration with the dbt semantic layer. Omni has his own, being an Ex-Looker, and he understands the space well. Hex has an amazing agent infrastructure and memory, while TextQL is just one of those things that has to be seen to be believed.
It's worth noting that databases have their own versions of this as well. Snowflake Cortex Intelligence and Databricks Genie are both products with a lot of investment behind them. If it's good enough, customers will use these.
Market Position
Not every BI vendor is in a position to care, or worry about these aspects. For example, Tableau will continue to generate revenue through Salesforce. Its success will be determined by Salesforce's adherence.
Standalone BI tools that don't have an edge are in trouble here. Sisens, Qlik, and Lookers have very little reason to continue to be chosen if they can't build a better agent analytics layer – and they don't have a big brother in the Boardroom to take care of them.
Companies like Sigma, which were closely related to companies like Snowflake, are now competing directly with them. So they are in a tricky position to see who can build the best AI Data Analyst.
Size and Location
Some companies may be too large in size and locations to be removed entirely. For example, both Tableau, Sigma, and Power BI have large areas that cannot be easily removed.
If dashboards serve their purpose, reliably, they will last. Remember – dashboards are available at Valley of Choice. This is because by clicking the will of a known, reliable dashboard still be less effort than creating a question, writing it in a dialog window, waiting, asking for graphs to be drawn, and so on.
This means that companies that have used dashboards are likely to keep them. However, expect more pricing pressure as agent BI forces user-based pricing models.
Conclusion – BI will integrate in 3 years
The logical conclusion is to add more.
AI means that building Dashboards becomes much easier.
BI Valley's shift from obscure to canny means the cost trade-off between agent analytics, dashboards, and manual labor has changed dramatically.
In fact, usage-based models are directly opposed to user-based pricing models, if not structurally then at least cost-wise. While companies may be willing to pay a small user fee for an agent BI tool like Hex, they certainly aren't willing to pay $5,000 for the right to create some Tableau Dashboards.
Read more about integration here -> The Ultimate Journey of the Modern Data Stack
All the while, companies will compete directly with the largest, most capitalized, and most research-heavy companies in the space: hyperscalers and data warehouse providers. There is no reason not to use tools like Snowflake Cortex Intelligence and Databricks Genie if they are good enough.
This means that the industry will continue to integrate BI and Analytics agents. Space will become an increasingly commoditized commodity unless independent retailers can continue to make better and better products faster than warehouses can hold.
At Orchestra, we build an Agentic Layer for developers. Follow us to find out more.
