Conversations in Collaboration: Peter Bailis with Google Cloud on Using AI Agents for Business Intelligence and AnalyticsConversations in Collaboration: Peter Bailis with Google Cloud on Using AI Agents for Business Intelligence and Analytics

Welcome to No Jitter’s Conversations in Collaboration, in which we speak with executives and thought leaders about the key trends across the full stack of enterprise communications technologies.

In this conversation, we spoke with Peter Bailis, VP of Engineering at Google Cloud, where he leads Looker and AI in Data and Analytics. Before joining Google, Peter was the CEO and Founder of Sisu Data, a Series C startup leveraging machine learning to accelerate data analysis. Peter also spent several years as a tenure-track professor in the Computer Science department at Stanford University, focusing on data analytics and systems for machine learning.

Source: Google Cloud

In this conversation, Bailis discussed Google Cloud’s “conversational agent” approach to making business intelligence and analytics more accessible to end users via AI agents.

NJ: Can you provide a bit of context for this discussion around ‘conversational agents’ and data analytics?

Peter Bailis (Bailis): I joined Google about a year ago to lead generative AI for our data analytics stack. We have, perhaps uniquely among cloud hyperscalers, [our own] core foundation model [family] with Gemini, data [tools] through BigQuery and our Google Cloud data stack, Looker – our business intelligence tool – and Workspace, our productivity suite.

The big punch line with that framing is that we have all this amazing user and company data – think about large enterprises that have stored their data in Google Cloud. We'll never train on that data, but it exists, and we have Gemini running first-party on top of that data – or we have the potential to do that, at least.

(Editor’s note: A follow-up email clarified this statement. Google does not train its foundation/base models on customer data. Customers can train custom Gemini models on their own data, but this is customer-controlled and opt-in, and customer data remains their data. Gemini runs as part of Google Cloud, so all use of Gemini in Google Cloud benefits from Google’s security and privacy controls. Moreover, when customers use Gemini on their Cloud Data, none of their data leaves Google Cloud.)

So, how do you go from what [we’ve] come to expect from the consumer chatbot experience to something that gets infused in everyone's day-to-day work?

That requires meeting users where they are. So not having a separate application but being part of where the work is happening in their collaboration tools and suites. It also requires a high degree of trust, so how do I come back and say: How is this calculated? How is it defined? How do I know that I can believe these results and keep them up to date as definitions change? And then, the final piece is just making sure that the results are able to answer the widest range of questions possible.

(Editor’s note: Bailis then showed a demo which consisted of the user asking the AI agent to provide summary information regarding the types of calls into a hypothetical call center. The AI agent interface looked like a standard ‘chat agent,’ as illustrated in the following animation.)

Source: Google Cloud

When we were making [AI agents] more useful to users in these ‘data plus AI experiences,’ that [involved] building agentic behaviors that look like coworkers. In this case, they look like a chat agent in my Google Chat, where I have the rest of my notifications and my email, etc., embedded.

What we've essentially done in [the call center demo] is built an agent using the existing data stored in BigQuery using the semantic models around the call center behavior in Looker and [then] gave the [AI] agent full access to those data and those data definitions in a secure way, but surface it to an end user who might have in a way that is just as simple as chatting with someone.

So, I can just ask [the AI agent] a question: What is the breakdown of call reasons by category? [The output] of this interaction is essentially something like I might expect from my [human] analyst.

In the background, the [AI] agent goes to Looker to go pull up those definitions and then it runs this analysis using these [data] definitions that are already present to run the analysis I requested, all without me having to leave my existing chat. So, it builds me a pivot table and I can dig into each of these different these different areas. Similarly, I can go beyond just simple descriptive questions and ask more advanced questions I might ask a data scientist like: What is the estimated call volume for next week?

Based on that conversation we have in that chat window, I can then have the agent go out and run another set of analyses using another set of tools. In this case, it builds me kind of a regression for how this is expected to continue to improve or change based on the data I have.

NJ: How are you using the word agent?

Bailis: In the early days of Gen AI, I had a chatbot or a large language model has been trained on a data [so it could] understand how to use that data to answer questions that it has access to. Here the LLM is acting on my behalf, and the actions that are being taken are multi step, and the agent is also deciding that sequence of steps autonomously.

In this case, when I say ‘estimate call volume,’ it is a conversational chat-based interaction, but what's happening under the hood is the LLM is basically deciding what this means in the context of my data sets. It is essentially pulling the data into an analysis environment. It is authoring the analysis itself – the Python code here – and then it summarizes that output and visualizes it. That's four or five steps that it is taking with [the] different tools it has access to in a way that it's never been trained on. This is not part of the base Gemini model. It is truly using runtime tools and runtime data and runtime data definitions to stitch it all together.

NJ: How does the user ensure that the answer the AI agents delivers is right?

Bailis: There are lots of interesting stories around models hallucinating – making stuff up, giving people discounts, etc. – but models have gotten much better [with] instruction following, so you're [less likely] to go off the rails in that way. But when you have a very specific question about giving customer or retention or loyalty or these very precise concepts, how do you actually get the [AI] agent to use the right definitions and leverage the trusted data definitions that are so important?

At Google we have those trusted data definitions, the data itself, as well as access to that data, so we can do quite a bit to increase the reliability [of responses.] So, when you ask for ‘call center volume,’ [the AI agent] uses the correct definition versus, as an extreme example, whatever the intern happened to write in their end of summer presentation that got stored somewhere in a database.

NJ: OK, so in the demo, you asked a general question of the agent, and it produced that graph. As a user, how do I know that the AI agent just didn't make that up?

Bailis: We have a couple ways to configure this, but at a high level when I set this up, I basically connect my data definitions to the [AI] agent, and that is done through the Looker semantic layer. The Looker semantic layer was built for reporting and [in it], I can define these metrics and build dashboards on top of them. When I configure this through the Looker application, you basically hook up the [AI] agent to those definitions, and if I ask something totally off the rails, the agent may, in fact, not respond, or say, ‘I don't have enough information.’ Or, ‘did you mean X, Y and Z?’

What's interesting about this from a ‘making it real’ perspective, is depending on the consumer of that experience, you may have higher or lower tolerance in terms of wanting to get the human involved to disambiguate.

Here’s an example of what I mean by that. One of the telcos we're working with is looking to put [our] conversational analytics agent in all of its store sales associates’ hands so that they can provide better service. [Maybe that associate] wants to ask a question about, for example, what device does this customer have? You don't want to expose the schema – the other tables in the database – you want to keep it very tightly scoped. So if [the AI agent] doesn't know how to answer a question, you basically just have it say, ‘I can't answer that question for you.’

On the other hand, if [the user] is on the data science team or data analytics team, it's useful [for them] to get that list of quick ‘I think you might have meant X, Y or Z.’

So you actually have these knobs as part of the [AI] agent configuration / agent instruction, where you can say: How precise do you need to be for a given user. One of the important breakthroughs we found from a product perspective is that, historically, everyone said ‘you connect data and voila, you get the right answers back.’ AI is not quite there yet.

By having a data analytics team be involved in a setup of that [AI] agent and then publish that agent with these configuration parameters – how precise you need to be, what is an end consumer's appetite for multi turn refinement – that's made it much easier to go and deploy with confidence, because it involves the subject matter expert in that [AI] agent creation. That’s similar to how today you have a subject matter create the dashboard and then many people consuming that [dashboard].

(Editor’s note: Bailis next demonstrated how the AI agent can handle both text and an image – i.e., multimodal reasoning.)

Google Gemini has a very long context window where I can put a ton of information in. With these same [AI] agents, I can do data analysis on different types of data – structured data, BigQuery, unstructured data, PDFs, you name it. In this case, I can add a picture to my conversation with the [AI] agent.

Into the chat app Bailis added a picture of a cable modem with Ethernet cables plugged in and then asked in text: Why do you think this modem isn't working?

The AI agent returned a schematic of how the cables should be properly plugged in along with a text providing a troubleshooting guide.

In this case, [the AI agent] went into the available unstructured data – the knowledge base – and pulled up an image from the Help manual and then provided some steps to debug the configuration.

This is powerful because if I have billing data, or call recordings, or really anything I have access to in terms of customer data or organizational data, I can use semantic search and some additional tooling on the [AI] agent side to generate answers.

(Editor’s note: The following video illustrates how the AI agent can handle multimodal inputs starting at timestamp ~1:46. Again, this is not the same example as Bailis demonstrated, but it does show it works in the chat interface.)

NJ: So is this solution targeted more toward structured, semi structured or unstructured data…or some combination? And how does that relate to where data is stored – warehouse, lake, etc.?

Bailis: The focus is overwhelmingly on data warehouse / lake house, because that’s where the data ends up getting aggregated. When build these custom first party experiences, like the [AI] agent I showed here, you can embed it inside of any software that a customer operates. That is usually governed in a lakehouse. But if you want to call Google Workspace or search Google Drive, there's ways to plug that in, as well.

(Editor’s note: Structured data refers to data contained in a spreadsheet or database; unstructured data refers to documents, presentation, images/audio/video; semi-structured data is a blend of both, e.g., email, HTML/JSON/XML. A data warehouse is a system for storing, analyzing and report on structured and semi-structured data from multiple sources such as CRM, etc., whereas a data lakehouse a “merges [data lakes and warehouses] to create a single structure that allows uses to access and leverage data for many different purposes.)

Historically, the core focus, and what we've seen from customers to date, is that they have a very specific use case – they’re basically giving people static reports they can't interact with. With this concept of a data agent, I now have the flexibility of building much more interactive, much richer, ‘question answering.’

The next, natural extension of this is: I want to be told when something changes. I have these same metrics I care about, every single day, every single week, [so I want the AI agent to] notify me when there's something [it] thinks I should be told.

NJ: You mentioned custom first-party software into which these AI agents were embedded. Is that the main approach for this product? And is it only available in Google products?

Bailis: We have first party experiences that are pretty deeply integrated. So, for example, inside of the [business intelligence] tool, you can basically start having these conversations and you don't have to leave your reporting environment. That's something we offer today. It's in public preview under the conversational analytics banner.

What we found, though, and especially if we think about enterprise-wide adoption, is that it’s incredibly valuable to make these [AI] agents accessible wherever users are already doing their work. I think it's like an activation challenge. What percentage of dashboards, reports, or pieces of information, are ever actually looked at? [Those are] important, but [they’re] not the same as having somebody you can just message and get a question answered. And you can build multiple [AI] agents depending on which subject matter experts you have. What we're seeing, as well, is that people are building [AI] agents that talk to other [AI] agents. I think that is really the future looking direction and we're very excited [about that].

Obviously, we'll make [our AI agents] work with the first-party Google products, but I recognize that people may be in products like Slack. They want to do things from their internal platforms where they're doing their work. Offering APIs and turnkey integrations for these [platforms] is important to activate that data even further.

NJ: Salesforce is pushing hard with Agentforce both in Slack and elsewhere – and that’s its own AI agent platform. And then you have other ‘assistants’ and ‘agents’ in all the other platforms – Teams, Zoom, Webex, ServiceNow, etc. How does this solution we’ve been talking about fit with those other ones?

Bailis: I think ‘agent-agent’ is the future, because you'll have domain specific experts. For example, if I want to ask something about my Zoom data, Zoom's agent is probably the best suited to have that indexed and ready to go. Our goal, and overarching philosophy, is to embrace an open approach and provide open APIs and [make it] easy to integrate and open the door for others to be able to leverage Google's data agents.

The thesis is simple: We think that with our integrated stack, and the work we've done, we'll have the most powerful way to ask questions about data that is stored in Google Cloud and is in people's lakehouses. With that approach, a lot of the Zoom data or Slack data…a lot of your sales data…ends up in in this lakehouse, as well.

Obviously, there's things like taking actions inside of Salesforce that we won't want to do, but when it comes to wanting an accurate, factual, grounded and potentially very powerful answer to a complex question that spans my entire business, we would be the best place for that. And we’ll make our APIs as open as possible so that it gets adopted to answer those questions for our users, wherever they happen to be.

The hardest part is always [figuring out] how to make it work for customers. This field of business intelligence is so funny because business intelligence [isn’t always] very intelligent; it doesn't [always] have much business impact. With AI, you can actually get to business level intelligence. And the questions [then become]: how do you really deliver on the promise and what does the workflow look like so you can really activate all these capabilities.