What is a Data Agent? | About Data Science

0 2 4 minutes read

What is a Data Agent? | About Data Science

I have the opportunity to try new AI-powered analytics tools, including the Microsoft Fabric data agent. That's why I want to share what I've learned, explain what a data agent is, and highlight the difference between it and a “typical” AI agent.

So, without further ado, here is my definition of a data agent:

A data agent is a contact person.

For those of us in math, this means two long-held wishes may come true:

#1: Analysts spend very little time creating observations.
#2: Self-service data is getting closer to business users.

Let me expand on each of these points a bit more.

A few ideas, not a few details

I really appreciate a good report that can tell me “what's going on” with the metrics I'm currently interested in. But being trained in statistics, I know how reports can sometimes present metrics in a wrong light, leading business users to often ask analysts for KPI interpretations, often 10 minutes before important meetings.

And that's one of the reasons we often find ourselves in a vicious cycle of having dashboards that no one uses, and stakeholders who are always looking for a “number” given by an ad or in spreadsheets.

On the bright side, visualizations and spreadsheets aren't going anywhere, but presenting information has a new approach with the Fabric data agent.

Instead of wrapping queries with graphs, you can wrap them with prompts and commands paired with managed data ready for use in Fabric, i.e., a pool house, a warehouse, semantic Power BI models, a KQL database, or even an ontology. This means that the underlying data still needs to be prepared and modeled to answer business questions such as “What was your income this week compared to last week?”

However, from a design perspective, rather than creating a scoped virtual report to answer this business question, you now create a scoped data agent to provide this, and other sets of answers derived from the underlying data model.

More precisely, the input-output flow goes like this:
(1) the participant asks a question, (2) the agent, enabled by the Azure OpenAI Assistant API, interprets the question and “decides” which data sources are likely to have an answer based on the source schemas and agent commands, (3) generates the appropriate query (SQL, DAX, or KQL depending on the source type), (4) verifies' (4) under the source identity), (4) details, and (6) returns the result as a text or table, not (for now) as seen.

In general, the interaction with stakeholders and information about the data agent is a Q&A session over the selected dataset, and the visual drop-down can be replaced with follow-up questions, such as “Can you break down revenue by segment?“

With that, it is clear how the work of analysts no longer needs to be re-presented only by dashboards, which is the long-known tangible proof that the work of capturing business intelligence within data models has been delivered.

Now, let's talk about…

Self-service information, close to where business users “live”.

I've mentioned before that reports can sometimes distort metrics, but that's not the only reason why “If you build, they will come” rarely works for them even if the statistics are normal. The truth is, the knowledge barrier is often too high to understand basic semantic models and how to use BI tools to create visuals on top.

While this points to data literacy, which is a change management problem, it is true that the target business audience, which should be reporting consumers, often have too much on their plate to worry about learning BI tools for self-service analysis.

That's why it's important to bring data closer to where end users “live”, today they point to powerful AI tools like M365 Copilot.

By being able to expose data through data agents outside of Fabric, analysts can now focus on the analytical logic behind self-service data agents, and end users can access data from the same AI-enabled tools that support their other daily tasks, without the hassle of switching to another platform.

I should note that this is not the only way to integrate Fabric data agents into a workflow, and whether you're a developer or a consumer, it's good to know…

Difference between data and AI agent

Photo by Dynamic Wang on Unsplash

We've learned so far that a Fabric data agent is an analytics agent that focuses on accessing read-only, managed data, able to translate natural language commands into complex database queries that unlock data, even outside of the Fabric tenant.

On the other hand, an AI agent is defined as a system that allows large-scale language models (LLMs) to do things, don't just respond to prompts, on behalf of users or other programs by accessing tools and information.

That is, all the magic is in the AI agent setup, where you can use the Fabric data agent as a special tool or information source.

I will illustrate this with one simple example.

Imagine that an authorized user requests an AI agent from him “Create an email to the team summarizing last week's revenue by segment.” To perform this task, the AI agent will, among other things, need to prepare revenue information from the business database. Therefore, with the aim of reducing errors in revenue calculation, the developer will design an agent workflow to route the input data to the Fabric data agent. a toolwhich can handle the heavy lifting of determining a schema, writing a query, executing it, and returning accurate statistics. Finally, the AI agent will be using those statistics to complete its extensive navigation and compose the email.

So what is the difference between the two? That's an AI agent actionswhile the data agent reasons.

Thanks for reading.

If you found this post useful, feel free to share it with your network. 👏

Connect for more news In the middle✍️ again LinkedIn 🖇️.