Bill Rogers: AI-Powered Assistants, Chat, and Search for Content Platforms – Episode 38

Podcast: Play in new window | Download

Subscribe: Apple Podcasts | Spotify | Android | TuneIn | RSS

Bill Rogers is an experienced AI entrepreneur whose latest venture, ai12z, gives web content platform owners tools to build digital assistants and chatbots and to run gen-AI-powered searches.

We talked about:

• his work at his latest startup, ai12z, which builds copilots designed to power content experiences
• his use of the term “copilot” as a generic AI capability, to distinguish it from branded uses of the word
• the two main capabilities of their copilot: question answering and ReAct (reasoning and action)
• his take on RAG architectures and how ReAct fits into them
• how integrating copilots into content and commerce architectures can guide users through complex interaction flows that are connected to third-party services
• how to ensure that users have confidence in AI systems and that the systems are technically secure
• the technical architecture that underlies their copilot platform
• how copilots help write queries to search utilities and other information and knowledge sources to help with tasks like complex product comparisons
• the variety of UIs their platform provides: search boxes, knowledge panels, etc.
• how interactions with copilots can inform an organization’s content planning
• the importance of including image AI in this kind of platform, to both better understand the content and create more robust ALT text

Bill’s bio

Bill Rogers is a visionary entrepreneur with a deep technologist background in AI and digital technologies. Recognized for significantly influencing the evolution of online experiences, Bill founded Ektron and served as its CEO. Under his leadership, Ektron emerged as a pioneering SaaS web content management platform, serving thousands of organizations globally. After Bill sold Ektron to Accel KKR, it merged with Episerver and became part of Optimizely. Bill then co-founded and led Orbita as its CEO, driving innovation in advanced conversational AI. Beyond these startups, Bill co-founded several other ventures and has had an expansive career in digital signal processing and robotics engineering. Bill holds a Bachelor of Science in Electrical Engineering from Boston University.

Connect with Bill online

• ai12z
• bill at ai12z dot com

Video

Here’s the video version of our conversation:

Podcast intro transcript

This is the Content and AI podcast, episode number 38. You wouldn’t try to operate an airliner without a copilot, and you shouldn’t operate a modern web architecture one function at a time either. That’s the case that Bill Rogers makes for his latest AI startup, ai12z. His company builds AI copilots – in the generic, non-branded sense of that term – that enable robust search and discovery, streamline complex tasks like mulitfaceted product comparisons, improve accessibility, and even help with content planning.

Interview transcript

Larry:
Hi, everyone. Welcome to episode number 38 of the Content and AI podcast. I am really delighted today to welcome to the show Bill Rogers. Bill is a longtime veteran in the content management and technology world. He founded a company called Ektron years ago, which was acquired by Episerver, which is now known as Optimizely. He ran a conversational AI platform long before ChatGPT came out called Orbita, and he’s currently the CEO and founder at ai12z. So, welcome, Bill. Tell the folks a little bit more about what you’re up to these days.

Bill:
Thank you, Larry. Yes. So, at ai12z, what we’re doing is we’re focused on building essentially a copilot, enabling websites and mobile applications, the ability to take advantage of AI to help drive experiences.

Larry:
Nice. And that’s a nice, succinct description of what you do, but a lot of websites have chatbots or things like that. How does a copilot… Well actually, first let me back up because copilot is an interesting term. I first became aware of it when GitHub did their coding assistant thing, and then Microsoft has a whole suite of branded products called Copilots. But we’re talking about a generic capability. Is that correct?

Bill:
That’s correct. I think the term copilot, Microsoft has used quite a bit, but it is a generic term. We actually like to refer to it as a website AI assistant. And if you think about it, in the days of Ektron, we had this phrase, “What do you want your website to do?” And now we are talking about, “What do you want your AI to do for your website?”

Larry:
Interesting. Human needs haven’t changed that much, but we have all these new capabilities. I guess what are one or two use cases that have jumped out early in your journey that are really helping people?

Bill:
So, when you think about, “What does copilot need to do?” So, one of the obvious things is this ability to be able to answer questions. And so when you talk about years back, when people were building chatbots, the challenge was creating the knowledge for that question and answering took a tremendous amount of work because you’d have to curate each piece of content that you’re going to answer a question with. You had to create an intent model. Just an awful lot of work.

Bill:
Today, we have a CMS connector, we ingest the data and we can answer any question that your content actually have. You don’t have to redo anything with your content in order to make it usable for question and answering. So, that’s the first step, just question and answer.

Bill:
Then there’s this concept of ReAct, which is reasoning and action. You enable these agents to do things. It can talk to backend systems like CRM systems or it could talk to any system that you have in your system. You just make a REST API available for it, and all of a sudden we can now use this data to create workflow to accomplish tasks that used to take an awful long time to go do and create, and it doesn’t need to be that way at all anymore.

Larry:
Yeah, I know a lot of conversational designers and I’ve watched them work in Voiceflow and tools like that and hand crafting all those query… all the questions and answers basically, and the intent discernment stuff that they do. There’s a lot to that. And so that ReAct, that sounds like a really intriguing… it’s like you can get your fingers into any other system that you have. And this kind of reminds me of a… Is this in the family of a RAG architecture where you’re…

Bill:
So, a RAG architecture would actually be just an agent to a ReAct system. So, let’s just describe RAG. To the users, RAG is a way for you to, instead of using the knowledge of the LLM, you are using the content of your own content and you’re answering… the LLM is answering questions based on that content. So, you have typically a vector database that when you ask the question, it gets the content and based on the content that it gets, the LLM will analyze that content and build a summary answer to it, actually very, very robust. And so that’s a core piece to it.

Bill:
What ReAct does is that there’s a large language model that does the reasoning. It thinks about what came in as a question and says, “Can I just answer that question or do I call one of my agents to help me answer the question?” And so, one of those agents can be ReAct… I mean, can be the RAG.

Bill:
So, why that becomes very exciting is that let’s say that you want to compare two products. Your RAG has the information about each product in the system. The reasoning engine knows if you said… We’ll use an example, sports example. If I said I wanted to compare the stats of Bobby Orr and Derek Sanderson, that’s very tough for RAG because that one compare question, are you going to find content in your system that actually does do the comparison? And you’re likely not. And so what will happen is that the reasoning engine says, “I’m going to go call the RAG for Bobby Orr, and then I’m going to call the RAG for what’s the stats of Derek Sanderson.”

Bill:
It gets the answer of those two information and then it combines the answers to do the comparison, and you get an amazing comparison around that concept. So then you take that step to the next level with a reasoning engine. And the reasoning engine, you tell them about all the tools that you have available to it: email, SMS, CRM, and the list goes on. Google Maps, Google Places. And you then say something to it like, let’s say you’re a hotel and you said, “What is the directions to the hotel from the airport?”

Bill:
And so the reasoning engine, from its system prompt, knows the address of where the hotel is and it knows where the nearest airport is, and it’ll actually call an agent called Google Maps and it passes to that, the address of the airport, address of the hotel and IT generates the Google map with the full map and the link so that you can actually… so you see all the directions just like you would in Google Maps, but you can click on it and now it’s in your mobile phone.

Bill:
So, you can see how a hotel can start looking at a reasoning engine as enabling all these third party services. Like if you said, “What are my activities?” Then the system is intelligent enough to say, “Oh, I have these eight activities, would you like to learn more?” And it gives you call to actions to learn more. And you then click on learn more and you see something about golf that you were interested in. It tells you about golf and you said, “Would you like to book a tee time?” You click book a tee time, a form has to come up to collect who are you and it collects your first and last name, your room number. And then it says, “Do you want to pick a date and a time?” So the time slots, when you pick a date, the slots are going to change. So now you pick all the information and then it might say, “Do you want to rent a club car?”

Bill:
And then it collects that data and it’ll analyze it, send you an email, register it with the system of record that this booking has occurred. And what’s amazing about it, Larry, is that there’s really no programming involved. It just knows that I have connections to the booking system so I know what dates and times are available. It knows about that because you collected from you the email, the user. It knows to go send an email message back to that user. And it, as a reasoning engine, figures out all that’s left to do.
You can help it align the system prompt, but besides writing the system prompt, there’s no programming to create that workflow. And that you had to do multiple steps actually for it to accomplish that task.

Larry:
Nice. A couple of things about that. One, maybe it’s the example you gave, but all of a sudden I’m picturing an old-fashioned switchboard operator and a hotel concierge kind of merged into one entity, and that’s the kind of thing that’s going on. Neither of whom needs to know all the details about the things they know about. So, this is like a routing agent, it almost sounds like.

Larry:
But I guess some of those things, like if you’re booking a thing that might require sharing credit card information or otherwise obligating you to some important decision, how do you ensure confidence and the accuracy of this thing so that you can do those kinds of consequential actions?

Bill:
So, the confidence piece of it, so we’ll get into the security in a second, but the confidence piece of it is that there’s actually AI that actually does QA testing. So, you go to this QA tester and you say… You actually will enter in the scenario that you’re trying to accomplish and you can execute that and you can execute as many times as you want. And what you see is that there’s actually a copilot that’s running that. So, the copilot actually analyzes the information that comes back because when you’re doing something like that, things change.

Bill:
So, things like the day of the week change, it could be… So, all of these things that could change means that you don’t have to write any… Traditionally, we’d have to write all of this decision-making logic to actually be able to handle all this. You don’t do any of that anymore. You’re actually using the copilot to test the copilot and evaluate, “Did it do the right thing or did something break in the process?”

Bill:
And so the reason why you build up all these different tests like that, and again, it’s just a human description. You’re not programming, you’re not making a decision, you’re just telling it, “Here’s what I want you to try to accomplish. Your goal is to book a golf tee time and go interact with the system to go do it.” And you might say, “Book it for two days from now.” And then it does the whole process. And at the end of the process, the copilot analyzes, “What was my thing that I was supposed to do and did I accomplish it?” And then you get the green light.

Bill:
See, as time goes on, you add more and more instructions to your system prompt, and you want to make sure that you didn’t add something that broke something in the past that was working. And so by having all these tests that can run against you system, you ensure the reliability of the system. That’s number one.

Bill:
The second thing that’s important is around security. And so in security, you need to follow the SOC 2 standards, HIPAA compliance, if you are working on a healthcare site. And those are things that from our long background in doing AI for healthcare and AI across know Ektron, that those are the things that we’re just building into the platform. We even are using AI as a way for our engineers as we’re coding. There’s a set of prompts that they need to go execute to actually evaluate our code to evaluate, “Are we meeting all of the security standards? Is there anything that you see in our code that actually could cause a problem that we should go think about and address?”

Bill:
And it’s very good at actually identifying stuff in your code that could be a mistake. So, it’s part of our code processing that not only do we do code reviews, but we’re actually using AI to do a code review of our code as well.

Larry:
Interesting. And it’s a combination of AI things because I’ve seen a number of articles lately about how just the pure LLM through kinds of code injections and weird prompts, you can do all kinds of security violations and horrible things. But tell me a little bit more about the architecture of the copilot. There’s the agents interacting with each other that are doing things, and then there’s… Are those all based on LLMs or are there knowledge representation things or other kinds of technology in the background?

Bill:
Yeah, so the agents could be an LLM, like the RAG actually has a multi-stage AI in it, in the RAG part of the agent. But a lot of the agents are just integrations with backend systems like Salesforce or Marketo or a marketing automation system, or it’s interacting with a third party system, that could be a ticketing system for if somebody’s doing a site that has… they sell tickets for the ballet or whatever, that it’s interacting with a backend system to know what that is. So, that’s not really AI. It’s really just exposing REST APIs to be able to interact with those, but it can be AI as well.

Larry:
Interesting. So, it just gets you to that API and then you’re just doing… Do these agents help write the queries or they’re going to an API? So, are there standards about that?

Bill:
Yeah, so you mentioned an interesting one around the concept of different types of… RAG is great for doing questions and answers. What it’s not really great at is when you are trying to say… Let’s say your organization that’s selling healthcare products like blood pressure devices. So, it’s not really great at saying if you say, “Tell me what’s the best blood pressure device.”

Bill:
Because now you have a lot of blood pressure devices, and RAG usually gets five or six documents from the vector database, and if you have 30 blood pressure devices, how does it analyze the 30 pieces of data from the blood pressure device? So, there’s different techniques of how you want to deal with structured data, especially when there’s lots of data out of that same relevance.

Bill:
Because what it is good at is if you said, “Go get these 30 items structurally,” but you’re creating the query to Elasticsearch that’s going to go filter and get the information that you’re looking for, you get back this JSON that represents the structure of those products and then it can analyze those products.

Bill:
But you might already have filtered out things by saying, “I want a blood pressure device that has memory.” And now all of a sudden the list has shrunk dramatically. The LLM knows how to talk to Elasticsearch to create the filter criteria. And so you do have the JSON REST APIs, but it is having the… it’s creating that query to go out to that system. So, that means you can do the same thing to a PIM system for e-commerce, you could do the same thing to other kinds of more advanced search, like the search that you were talking about earlier, where you have a graph interface and now you’re giving graph… So, the LLMs know things like SQL, it knows things like a graph query and what it can do, what it can’t do.

Bill:
But really, what you do is you tell it, “Here are the properties that you’re exposing.” You put a description of what it can do, you show it some examples, and then you start testing it. And as you do the testing, you sometimes have to add one or two more examples so that it’s intelligent about how it can use that agent to go call something. But then that’s where the power steps to really come in is that you’re not just using RAG, but you’re using the structure data that we’re used to when we’re in the content management world, where you might have structured information and you want to call that structure information and use that.

Larry:
Yeah, because I’m a content modeler at heart and that’s how I spend all day modeling this stuff for exactly these kind of uses. And I’ve seen a lot of architectures that the last stage is an LLM still making stuff up, but this sounds like this is just helping with the query and then you get the answer back, you share the answer, and is that kind of how it works?

Bill:
Yeah, but it can take it to a next whole level step. Imagine we talk about that product comparison. So, an agent can go call and say, “Go get me all of the blood pressure devices.” So, let’s say now all those blood pressure devices, that data is coming back. You can tell the system, “Don’t put the data through the LLM. Pass it right to the client, to the carousel.” And so that in the carousel you see the 30 blood pressure devices. And in that carousel, you have check boxes and so you can check, check, check, and then you click compare.

Bill:
So, now you’ve selected four items. Now it takes that four pieces of data, passes it to the LLM, and it does an amazing job of comparing products, if you give it the data. And so that’s a really good example of tying it all together.

Larry:
That’s really interesting. Is there on the fly structure that’s created as it goes out and grabs these 30 different blood pressure devices? Does it look and go, “Oh, there’s a bunch of these things that look similar,” and does it create… I’m imagining like a matrix or table of some kind or…

Bill:
Yeah, so if we were working with a device manufacturer and they’re selling products, you would tell in the system prompt, “These are the types of products that we sell.” So that you’re helping the reasoning engine know what is it trying to organize, what’s it trying to sort? Then you’re telling an agent that actually can go, when somebody asks these questions, “What can it do and what can it filter on?” And so it will be intelligent about it.

Bill:
You also can tell it about, “This is the organization that this is for.” And the LLM knows about every organization in the world. So, already the LLM in its reasoning head engine head, it already knows all of the kinds of products that company probably sells without giving it any more data around that. But what you want to do is you want to always get the data from the source of truth, which is the client’s data. You don’t want to rely on the LLM to give you the data, but you can rely on the LLM to know it knows about this company, and based on that, and you tell you it has this tools, it knows how to create the right kinds of queries to actually bring back data to the system.

Larry:
Interesting. So, it learns from other examples, I guess that’s the whole point of machine learning, of all those other similar entities out in the world, like how things might happen or be arranged or attributes they might have, that kind of stuff.

Bill:
That’s correct, yeah.

Larry:
Yeah, that’s fascinating. Yeah. And so how does this look in day-to-day? I think there’s kind of two kinds of people, I think, among my audience. People who do content design and sort of product development and the UI strings and that stuff around them, but also a lot of people who just do other general content stuff like content creation and working with websites. I think of that latter group might be interested in how these… So, do you just provide the technical underpinning to build these copilots or is this something that any one organization builds their own front end for it?

Bill:
So, we do have what we would call Web Controls, which are one type of control is a search control, so you can go, looks like a search toolbar, you can click on it and start asking it questions and you get an answer to exactly what you’re asking for, with the references back to where did they get the answer from.

Bill:
We also have a chatbot control that you would think about in the typical bottom right-hand or left-hand corner of your website is a chatbot control. We have another control called a knowledge box. So, the knowledge box is something that works in parallel to someone’s search. So, although what we show people immediately, they see the quality of these results compared to the search. Search is really a research project. You do a search, you get back all these results, you have to analyze it, you have to figure out which one you’re going to click on, you open up a multiple tabs.

Bill:
Where the results from the LLM gives you an answer that’s very precise in answering the question that the user asked for. But sometimes, politically, you’re not going to replace this search. So, you can use this knowledge box. Knowledge box is just like you would see used in Google where you do a Google search and this right-hand knowledge box comes up. That’s kind of what people will use our solution when they’re using the knowledge box.

Bill:
But that knowledge box can be used for other things. Like if somebody wanted to build their own product comparison, they can put their own UX to say, “I want to compare,” and you have a list of all the products, you can select the products and click compare. You might have a little text area that says, “What’s important to you?” The user types and what’s important to them, so that when it does the comparison, it actually is using the user’s input of what’s important and it will do the comparison and actually highlight what was important to the user, and this product has that added benefit because it meets this criteria that you have.

Larry:
Yeah, that’s an interesting one because I think a couple of things about that. One, I want to return to this search versus a copilot answer, but that last thing about the… Well, actually let’s go to that first because there’s this behavior, and both of these are about the behavioral shift that we see around using digital products now, that there’s that old… We’re all used to, not used to, but the paradigm that we all, most of us, grew up with is that collection of links when you do a search and you sort through those, and like you said, you end up with a bunch of open tabs versus getting an answer, which in the case of the scenario you’re describing, is tailored and fine-tuned based on your inputs into maybe a knowledge panel or something like that.

Larry:
Is there much evidence or research around the end quality of the user experience around that old, the “scent of information” around old-fashioned information retrieval kind of paradigms versus this new conversational answering paradigm? Does that make sense?

Bill:
Yep, it does. I think before, the older chatbots, you were trying to find an answer and stuff it into an answer for the user because you created every one of those answers. Whereas in this newer model, it’s using, if it’s RAG, it’s actually going out and finding what are all the most relevant documents to this question. And then all those documents are being analyzed by the LLM to actually do an answer that specifically answers the question that you asked, which is unlike when you see an answer from the older chat box, which was you get an answer and it’s like, “Well, that wasn’t quite why I asked, but I guess it’s close.”

Bill:
Whereas it is worded in ways… I was doing an example with NIH where we were doing, we adjusted their healthcare content and somebody would say, “Well, ask this question to it.” And you’d ask a question about type two diabetes, and it would come back with an answer and they were like, “I don’t think that even exists on the website.” And it was a reference to what page it came from. And you look through the page and you’re like, “Where is this? Where is this?” And it’s actually there in the content.

Bill:
But the trouble is that when you’re in that technique where you don’t even know where the page it’s coming from the answer, you’re doing research to try to read through and see if you can find an answer. And that’s the problem that people have with most websites when they’re going to visit, but they don’t spend much time analyzing your page and trying to figure out, “Do you answer the question with a better answer?”

Bill:
With this, the return on investment is you ask the question and it’s going to answer it based on the knowledge that you’ve put into the system. So, for content creators, it really is the first time that their content’s really being used really efficiently because it’s not going to answer questions that it doesn’t have content to it. It’ll come back and actually say, “I don’t know.” If you said, “What’s the capital of France?”

Bill:
It would come back and in a nice way basically say, “My job is… I’m an AI assistant for this healthcare company. I don’t really answer questions outside the scope of my range.” But really what it came back with was, “I don’t know.” And that, “I don’t know,” we log so that as content people, you can look and say, “Well, what are all the things that are not being answered?” And it actually will organize it to the most popular question that’s not being answered to the least popular question that’s not being answered. And that way you can actually see, “Do I have content coverage?”

Bill:
It’s a wonderful SEO tool because if we can’t answer the question, honestly, Google’s not going to answer the question either. They’re not going to find your website based on that question. But now you know people are asking that question, so you can go to your content management system, make an update, and it’ll automatically sync with our system.

Bill:
Or if you are in a situation where the person that manages essentially the copilot, they can go into our system and there’s a mini content management system there where they can add knowledge to it. And that’s built in there by experience because we know that there’s times that these groups… a content management group is not the same group as the people that do the call center and they’re looking for call deflection, and not all the content on the website is useful for call deflection, but they have the PDF documents that they train the call agents and so they just ingest the call agent PDFs and now it can answer those questions very, very well.

Larry:
Yeah, that message of finally your content is going to be used in a more efficient way. And you just opened up 10 other conversations I want to have around AI-assisted inventories and audits and gap analysis. But we’re coming up on time today, Bill. But before we wrap up, is there anything last, anything you want to make sure we get to before we close or revisit from the conversation?

Bill:
I think the only one thing that I think is extremely important is the concept of image AI. The fact that when you ingest this content, we’re ingesting the images. And then what we do is we use AI vision to analyze the image and we create a very robust ALT text. And then from that, when we answer the question, we say, “Where did the answer come from? Was there images on those source content?” And based on that source content, we have AI that will say, “Does the answer have a relevancy to any of these images?” And we show an image.

Bill:
So, you can imagine we ingested a manual for a automotive company for their manual for that car, and somebody says, “How do I change the stick shift?” And so it answers the question, but a picture is worth a thousand words. So, if you see the image of the stick shift and you’re asking how these change, from the description and the image, you get a much better response. And that’s a pretty important thing that is a combination now of images and answers to questions as well.

Larry:
That’s exciting too because I’ve been waiting for image stuff to come into the content world. So, this is one of the first concrete examples I’ve seen. So, thank you for that. Hey, one very last thing, Bill, if folks want to follow you or connect online, what’s the best way to connect?

Bill:
Ai12z.com or bill@ai12z.com.

Larry:
Great. I’ll put that in the show notes as well. Thank you so much, Bill. Fun conversation.

Bill:
Thanks, Larry. Nice talking with you.