Podcast: Play in new window | Download
Subscribe: Apple Podcasts | Google Podcasts | Spotify | Android | TuneIn | RSS
(AI-generated image)
Kurt Cagle has been reporting on and participating in the tech world for several decades. He’s never seen anything like the pace of change around AI.
His advice for staying ready to work with AI: stay nimble, pay attention to what’s going on, don’t get tied to any one technology, and always bear in mind that your work will have an impact.
We talked about:
- his interest in the intersection of AI and knowledge graphs
- the rapid and vast advancements in AI tech, especially recent developments at OpenAI [referring to the product announcements in early November, not the Sam Altman excitement later in the month]
- how building AI products in the current environment is “like building a luxury hotel on top of quicksand in an earthquake zone”
- the impact of open-source and component-based thinking in the AI ecosystem and how those dynamics are democratizing AI
- how a Disney character can help you understand LLMs
- how tokenization work
- the difference between how a query to a typical database works and how vectorization identifies things that are similar
- how knowledge graph technology can help solve some of the problems in the LLM space
- his advice for staying ready to work with AI: be nimble, pay attention to what’s going on, and don’t get tied to any one technology
Kurt’s bio
Kurt Cagle is managing editor of The Ontologist, Generative AI, and the Cagle Report, and is a thirty-year veteran in the knowledge management space with twenty-five books and several Fortune 500 clients and government agencies. He lives in Bellevue, Washington with his family and cats, where he likes watching the rain fall.
Connect with Kurt online
- kurt dot cagle at gmail dot com
Video
Here’s the video version of our conversation:
Podcast intro transcript
This is the Content and AI podcast, episode number 6. Developing software has always been challenging. AI takes things to a whole new level. Kurt Cagle says the current pace of development in the generative AI world is “like building a luxury hotel on top of quicksand in an earthquake zone.” Kurt has reported on and participated in many eras of technology advancement, but he’s never seen anything like this. His advice for thriving in uncertain times like these: stay nimble, pay attention to what’s going on, and don’t get tied to any one technology.
Interview transcript
Larry:
Hi, everyone. Welcome to episode number six of the Content and AI podcast. I’m really happy today to welcome to the show Kurt Cagle. Kurt, he’s the editor for The Ontologist. He’s a managing editor for the Cagle Report. He’s been a technologist for more than 35 years. He’s written 25 books. He’s been blogging for 20 years. He knows one or two things about AI, so welcome, Kurt. Tell the folks a little bit more about what you’re up to these days.
Kurt:
Thank you for having me on the show. I really appreciate it, Larry. I am busy in AI land and in the intersection of AI and knowledge graphs, and have been trying to find out where the two meet and otherwise contributed to one another. I do run a small consulting company. I have a Calendly account if someone wants to contact me directly for setting up an open office hour, and always looking for opportunities. Thank you very much.
Larry:
Yeah, cool. Well, and speaking of opportunities, we met through the knowledge graph community which I’ve been involved with for a few years now, and as the LLM explosion of the last year has come along, I’ve been like, “Where’s the connection between these?” And all of a sudden, the last, I don’t know, few weeks it almost seems… We’re recording this on November 7th, just for people’s reference because you need an anchor in this rapidly changing thing. This episode probably won’t air for a couple more weeks, but we’re recording this on November 7th and just either today or yesterday, OpenAI announced a bunch of new product features that include integrations of chat agents and knowledge graph stuff as well as some other stuff. Can you give us a quick overview of that big product announcement that just dropped?
Kurt:
Yeah, it was huge. Quite honestly, if you’re involved in the AI space and have already put something together into a product, the announcement of a couple days ago probably has completely upended any apple cart that you might actually have. There’s a whole lot that goes in. Better performance, much bigger context, which is essentially how much information you can send over the wire and how much you can actually refer back to. There is, and I’m pulling this up so if it looks like I’m looking off into the space here, I am, but a whole new Assistants API, new integration with Retrieval, what are called RIGs, and code interpreting, so you can actually define function calls. It now incorporates GPT-4 Vision so that you can effectively put up an image and say, “What is this?” Or even, “Here’s an invoice. Do something with it.” It has a new text to speech module, which is kind of cool. I’m still playing with that one. And then a lot of business-esque things, making it cheaper to use, cheaper to modify, cheaper in general.
Kurt:
I think a lot of people were going, “This is too expensive. We need to be able to pull things down.” And that was the bulk of it. There’s some other things, improvement with DALL·E, which is their image rendering API, and I’ve been having some fun just playing with that for producing surprisingly good images. But overall, what you’re seeing with this is essentially a shift away from, “Here’s a single product,” to, “Here’s something that is intended to be much more fully integrated.” There’s a new multimodal modality that is also associated with this that allows you to integrate what had been largely separate plugins into a single framework. So you can load in files. You can create files. You can really get access to any kind of information that you’re looking for and do things to manipulate it. It’s a really, really cool piece – and it’s also causing a lot of problems with other companies that are now going, “Oh, this is going to be so fun.”
Larry:
Well, it sounds like they’re trying to become the one-stop shopping solution for all of your… Is it mostly about generative AI? I was trying to listen to all this stuff you said. It sounds like it’s mostly generative AI stuff.
Kurt:
It’s mostly generative. Admittedly, most of what open AI is currently concentrating on is generative. There’s some agents and agent technology that’s coming in that is along the edge of that and gets more into, “How do you build companions?”, “How do you build avatars?”, “How do you integrate?” And to a significant extent, what it does is it raises the bar. There are others playing in this space like Hugging Face, like Google, like Amazon, and it was very definitely a shot across the bow that says, “Okay, you need to be able to match this at a minimum.”
Larry:
Yeah. So we’re going to be facing that for… One thing that’s occurred to me, and I’m almost embarrassed to admit this, in the two and a half weeks since I conceived of and started this podcast, I first thought of it as mostly sharing information so that people could use this technology better. But I’ve talked to a number of friends already just in the last few days who work at big enterprises and big tech companies who are doing product content work on these tools themselves. I can’t say which companies they’re at, but there are things like OpenAI… I haven’t talked to anybody at OpenAI, but I have talked to people at similar companies doing similar things. Do you have a feel for what it’s like to be a product person or designer, an engineer working on these teams that are creating these new AI products? All the stuff that went into this announcement we just covered?
Kurt:
Yeah, they’re not sleeping. Nobody is sleeping at this point, which is a little scary. Realistically, this industry tends to go in waves where you’ll have nothing and nothing and nothing. Then all of a sudden, you get a whole bunch of announcements at once and a paradigm shift that occurs in how things get done. The last major one that I can really think of was probably Hadoop and that, well, Hadoop and then the machine learning data learning or data science development. Before that, it was the rise of mobile phones. And in each of those cases, there was something of a mania aspect to it. People basically trying to get something up there, hoping it sticks, and hoping that someone else doesn’t come in. I cannot remember ever seeing it moving so fast that you’re literally having to wake up in the morning and spend an hour just keeping track of what happened overnight, and then take that and put it into sense and be able to play with it.
Kurt:
So it’s a challenge for a lot of people because it means that there’s a lot of people that are playing with it. A lot of people that are looking at it from a business perspective or a use case perspective that are fascinated by what they see, can see potential, but they’re also, in some respects, I think they’re afraid to necessarily push too fast simply because things seem to still be moving. As I put it, it’s like building a luxury hotel on top of quicksand in an earthquake zone. Things will change. It’s guaranteed that things will change, and when it does, you just hope that you have slightly more firm quicksand than your competitor down the road.
Larry:
Wow, that is some scary context for this, but I think people can relate to that. That’s how it’s felt, like we’re all quivering in the quicksand trying to figure out what the heck’s going on. One thing, we had a preliminary chat about this last week and I’m really glad we didn’t record then because now we got to break this news about the OpenAI announcement. But one of the things we talked about then, it’s addressed in this stuff we just reported, the need for orchestration around this stuff. There’s many different technologies and capabilities at play here. The top-level stuff that we’ve talked about, like the LLMs and the knowledge graph stuff working together, can you talk a little bit about, I don’t know, I don’t know how high concept you can go, about just how that orchestration could and should happen.
Kurt:
Well, I think what is happening right now is that what’s emerging among the different players is essentially a fairly broad agreement about the architecture, about how you build large language models, how you build community language models, which are, in my mind, much more interesting, but it’s generally usually smaller and more focused. And so because of that, you’re not at a point where any one given company is essentially driving everything. Instead, you’ve got about five or six different companies that are… I guess the best analogy would be you put half a dozen senior architects into a room and then handed off Red Bull to each one of them, and said, “Okay, whiteboard.” And that’s what’s happening. It’s like people are putting stuff up, they’re testing it, they’re trying to break things down in terms of, “Well, does this work here? Does this work there?” There’s two or three competitive strains, but it’s surprisingly, for the level of sophistication involved with these interfaces, it’s going pretty well.
Kurt:
I think one of the things that’s changed, and for me, it’s changed for the better, is we’ve gone from this notion of saying, “We have one single large monolithic model,” which was essentially the ChatGPT model, and there were variants, but they were variants largely by the same company, OpenAI. And that changed around the time that Meta got involved with this. And arguably, what Meta has been doing is pushing some of this back out into the open source community and because it goes into the open source community, it’s suddenly being seen by a lot of other people who are also coming to the table and saying, “Yeah, but what about this problem? Or what about this idea? How can we integrate that?” And as a consequence, it also means that, for those people who are trying to keep on top of it both to report and to build on, there’s a huge amount of experimentation going on right now that ordinarily would be a lot slower and would be a lot more measured in terms of how things are going.
Kurt:
That in turn means that… Pluses and minuses. On the minus side, there are still some huge issues, even technical issues that I don’t think have been fully addressed yet, have been fully solved yet. But a lot of it has, and the stuff that has really has moved into this mode of, “Let’s move away from the monolith and towards many, many, many smaller GPT-like components that can then communicate with other systems, with knowledge graphs, with documents, with other… Across traditional relational databases or NoSQL databases.” And that process, as it goes on, is democratizing AI. It’s putting it into the hands of enough people that it’s no one organization that is essentially doing things by fiat. Everything is at least tested within the broader community to be able to say, “Does this work? Is this feasible? Is this a bad idea or not?”
Larry:
That’s really interesting. I just want to say companies like Meta are not… Pros and cons of them, but one thing they’ve been good at, the React framework for website front ends, and they’ve open-sourced a lot of stuff like it. And now with the open sourcing of their LLaMA and there’s probably a whole bunch of stuff that they’ve opened-sourced, is that the open source that’s leading to the democratization or is it just the rooms full of engineers with whiteboards and cases of Red Bull everywhere that’s driving that?
Kurt:
Well, I think it’s a bit of both. I think part of it, I didn’t have the same attitude towards Meta as I do because I’d been involved early on with some of the things that were… Working on the metaverse side and the kind of things that were happening out of that was kind of cringe-worthy, to say the least. But in this particular case, I think that they realized that they could not necessarily compete in terms of technology or market presence. And so what they did was to say, “Well, let’s essentially go and release some of that into the broader community.” How big that community is, I honestly don’t think anybody knows. It’s still largely concentrated along the West Coast, Seattle, Portland, San Jose. And so that access seems to be really where a lot of the newer AI work is going on, which is a little bit different.
Kurt:
If you look at semantics, semantics is largely East Coast. It’s Boston and New York and that area, and it’s much more symbolic in terms of the underlying AI models. And that also means that, as a consequence, there’s still a “not invented here” attitude that hasn’t been completely resolved yet, but it’s getting there. It’s gotten to the point where people are saying, “Wait. There is already existing technology that’s doing what you’re trying to build something to do, and we don’t want another Hadoop. We don’t want another system where we start with a really cool idea and turn it into a really bad database.” A lot of people have learned their lesson after that one. So is it truly democratized? Yeah, I don’t know yet. I think that’s still a hard question to answer, but it’s more open now than I think a lot of other initiatives are in the IT space.
Larry:
That we’re at a similar point in their evolution? It feels more-
Kurt:
Yeah.
Larry:
Yeah. That’s good to hear. I could nerd out on the business and architectural part of this forever, but I was really taken by some of the stuff we talked about last week and I want to make sure we get in the… I characterized a part of our conversation last week as the Mad Libs part, the filling in the words. And I’m dying for you to share the story of Tangled, the Disney movie and the character in there and how that relates to LLMs, and then talk about an analogous thing that you talked about with knowledge graphs. I’m trying not to spoil your story here, but setting it up.
Kurt:
Well, I’ll spoil it on my own, I guess, but in this particular case, it’s a way of thinking about what’s happening. One of the problems that a lot of people look at when they look at LLMs, and I think this is slowly changing but not really fast enough, is they look at this and say, “Well, it’s basically a probabilistic way of dealing with language problems and there really is no intelligence there.” And the answer is not really… That’s not really the case. There’s actually some very significant thinking about the nature of how do we think and how does that translate into neural networks and what needs to happen to do this. And so when you look at the way that a typical LLM works, it essentially is a text processor. It tokenizes text, your content and says, “I’m going to take this information, and I’m going to turn it into a sequence of characters with occasional unknown points or undeclared points that we’ll call variables.” And those variables are holes that you have in this makeup templates.
Kurt:
And so a good example of this is that if you look at the movie Tangled, the first movie, there’s a sequence where Anna, the princess, is talking to Hans, which is the prince from another island or set of islands, putatively, Denmark. And her interest is essentially in finding connections and finding people. She’s lonely. She wants to fall in love, and a handsome guy comes along, and his own motivations are a little more shady. But in there, there’s this great assumption where they start into the libretto and the character Hans sings, “We finish each other’s,” and at that point, Anna comes back and says, “Sandwiches.” Now, that’s kind of the way that an LLM works. It looks along this string and says, “Okay, I’ve got a set of sequences or a sequence of tokens that I’m working with, and I’ve got a certain token in here that would normally say, ‘Sentences,'” because sentences is the way you would tend to think about that information.
Kurt:
But because there’s a small, very small probability or had been up until that point that someone would say something else, there’s essentially a chance that that could be something else, and that is what gets substituted into the thing. So there’s certain probabilities that you have for this thing to happen. Now in language, when we are talking about expressions of sequences, those chances can be thought of as these sequences basically have some kind of an identity that you can assign to, but they evolve. They change over time, and as they do, as something like Tangled comes along, people basically start hearing the libretto, start thinking, “Oh, that’s kind of cute. Everything is now ending with sandwiches instead of sentences,” and that probability begins to rise and it is that probability, that weight, that’s assigned to these variables and the tokenized sequences that you’re talking about that ultimately determine how the next set of sequences get interpreted.
Kurt:
So that essentially, that filling in of the blank, is essentially the way that language models work. It’s also sort of the same way that, in the semantics world and the knowledge graph world, the way that you query against a knowledge graphs is to do something very similar. You say, “I have unknowns, and those unknowns can effectively take many values, and when I make a call against that, I get the set back of all of the phrases and the associated values that come in for those variables that can then be used to determine, ‘Well, what concepts am I passing back to the user?'” Once I have that back, I can then take that information and I can say, “Here’s the context.” Gives me what’s happening at the time. “Here’s the variables. We’re going to then pass this on to a transformer, which will take this information, templatize it, and produce some output.”
Kurt:
Now, where things have been evolving so rapidly is that the context can then be used, which is the prompt essentially, or prompt plus a few things, is what can then be used to determine what kind of form that output comes from, what gets generated. And so it could be something that’s very simple. It could be something where you’re talking, “Here’s information that’s appropriate to a six year-old,” versus, “Here’s something that’s appropriate to a postgraduate student,” and the way that you can then pass that information in is to be able to pull from that context enough information to identify what templates you’re going to be able to utilize, and then using the same method to build out those templates and then pass them along to the end user.
Larry:
As you talk about this, I’m reminded that in both cases, in both an LLM, the data that that’s working with and in a knowledge graph, the data instances that populate a knowledge graph, they’re both tokenized or embedded. Actually, let me clarify some language around tokens and tokenization and embedding. Are those similar things? Because the way you hear them in media reports, you’d think they’re very similar, or how do they differ if they do?
Kurt:
They’re a little different in that a token is basically just a word or a phrase. It can be other things, but in general, it’s essentially a unit of meaning, and the tokens themselves essentially are then used to determine a vector, which is very big in terms of being able to say, “I’ve got 10,000 different tokens that are in this vector, and I want to be able to find other vectors that are close to that to be able to determine what other things are similar to that, to be able to then come back and say, ‘Here’s that set of information that we’re talking about.'” So it’s not just a matter of saying, “Do I have exact matches?” It’s if I was to create an information space in, say, 10,000 dimensions, where each dimension essentially is what’s called a feature set or a feature, each feature can have a value that essentially is defined through that mechanism to vary between a value of zero and one. And then the interpretation of that zero and one is something that’s then kept in the background.
Kurt:
These are actually, internally, they’re kept in what’s called a vector database, just like we’re talking about knowledge graphs and graph databases that hold information that way. And these are a little bit different than when we talk about a relational database. A graph database essentially describes graphs or connections of things to other things, and that graph can be very large. It can be some some kind of a sub-graph that says, “From this large set, let’s take smaller pieces that all satisfy a given query,” or it can be a vector in this case, which essentially is just that. It’s one very long array of values for each different feature that can then be used to calculate similarities.
Kurt:
And that’s where architecture in general is moving. It’s a mix of there’s some relational system for relating logically relating content, and there is a search system that’s looking towards clustering and similarity of concepts that way. Because of that, it in turn means that you’re trying to do the same thing in both cases. You’re looking for those things, that set of information that satisfies this and that can then be transformed to something useful, but it’s very different from typical relational databases or even NoSQL databases in that it’s not just a matter of saying, “Here’s a key. Pull back something.” It’s, “Here’s a really long complex key. Find things that are close to it.”
Larry:
Yeah, the thing that’s helped me-
Kurt:
And-
Larry:
Sorry, the thing that’s helped me understand that one best is the example that comes up a lot of you take high level royalty, subtract gender, add another gender, and you go from king to queen. Is that a good example?
Kurt:
Well, that’s how a semantic system works because there, you’re saying, “I have a concept,” and that concept has these constraints. In this case, a high level royalty and has a gender that satisfies male or female. With a vector database, on the other hand, what you’re doing is essentially specifying an information space, and when you pull information from that vector diagram, it really doesn’t look like anything that’s really useful. This is one of the problems that you run into with AI is that it’s very difficult looking at these vast sequences of essentially numbers from zero to one, and just enough of a labeling to be able to say, “This is this particular piece,” and so forth. But its purpose isn’t so much to be able to retrieve content as it is to calculate similarity.
Larry:
Got it.
Kurt:
How are things related to what?
Larry:
Yeah, I think what’s really interesting is there’s a couple things about this that are really super interesting. I think to my folks, is that, one, that leap from… Because I think, to the extent that content design and content engineers and stuff, we’ve been working mostly with relational databases over the years, which are really, to grossly oversimplify, just a bunch of tables, and we almost always prototype everything in a spreadsheet. So we’re on familiar terrain all the way through that whole thing. Now we’re jumping into 10,000 dimension vector space and it’s like, “Whoa.”
Larry:
So I guess I’m going to have to ask you back. We’re running up close to time, but I want to have you back at some point if you ever come up with a metaphor or some way to ground us in that to help especially non-technical people, designers, content people, product managers. Everybody’s going to be wrestling with that. The way you just described it totally makes sense, but I just feel like for the day-to-day of it, we’re going to need to figure out just our mental models, aligning our mental models with what’s actually going on in these crazy complex black boxes we’re dealing with.
Kurt:
Well, I think that this is one of the areas where the knowledge graph community is coming back and raising their hands and saying, “Hey guys, we’ve solved this particular problem, and if you let us play in your sandbox, maybe we can find something that solves this to be able to handle this so that it’s not just a matter of, ‘We’re going to reinvent everything even if it’s not as efficient.'” And this gets back to Hadoop again, where Hadoop by itself, nobody uses Hadoop anymore. At least very, very few people do.
Kurt:
But it really was responsible for a lot of what we now think of as SaaS, the cloud, because a lot of the concepts that we needed to be developed were people going back and saying, “Well, you have this and it’s okay, but we can do much better than that for this particular sub problem. And if we do that, then we don’t need all this complexity over here. We have what we have over here and it’s fine, but we do need to build out something to be able to do it.” So Amazon and AWS and Azure and all of those systems essentially evolved out of the realization that, “Yes, you do have to have people in different domains working with the same content.”
Larry:
Yeah. And that’s where it’s getting really exciting to me, and I’d love to circle back at some point to the whole orchestration thing, but hey, I need to wrap up now, but before we do, I want to make sure. Is there anything last that you want to make sure we share today, or that’s just on your mind about AI, especially as it pertains to content these days?
Kurt:
Yeah, it’s evolving. It’s three and a half million years ago, and a bunch of apes are looking up and the volcano is going off in the background and everyone is saying, “Oh, crap.” And because of that, because it’s evolving so quickly, I honestly don’t think that anybody has a very clear idea about what the space is going to look like a year from now. So the best thing you can do is remain nimble, keep aware of what’s going on. Don’t get too tied up in any one given technology, but try to stay as current as you can with what’s out there and understand that, ultimately, what you are doing will have an influence. I think for the DAM community, as an example, I had a conversation a couple nights ago specifically on how DAM is going to be influenced, digital asset management, is going to be influenced by AI, and it changes everything, but it also is something where if I had to say exactly how, I think my guess is as good as yours.
Larry:
Sorry. My brain just exploded a bit. I haven’t played a lot with Midjourney and DALL·E and those things, but I’m going to go play with them. And this may be the illustration for this episode of the podcast. Scared apes and a volcano, trying to figure out, and a crystal ball, peering into the future. I think the way you just set this up like that, we’re at the beginning of a long evolutionary process.
Kurt:
Don’t forget the big black rectangular prism floating in the background and the ape with the bone. If you’re going to do it-
Larry:
I’ll do it right.
Kurt:
Let’s get a little Stanley Kubrick into it just for the grins.
Larry:
That’s funny. I was just having… Stanley Kubrick came up earlier in the day for me. That’s weird that he come up twice in one day. But anyhow. Well, thank you so much, Kurt. Oh, hey, one very last thing. What’s the best way for folks… You mentioned earlier your Calendly thing, but if folks just want to follow you on LinkedIn or connect, what’s the best way to connect?
Kurt:
Yeah, I’m linkedin.com/in/kurtcagle. All one word. The Calendly account, I mentioned earlier. And then you can get me at kurt.cagle@gmail.com. Generally, LinkedIn is probably the best way of getting ahold of me, but I do have open office hours as well where I’ll take anyone coming in. Just connect with the Calendly and set up a time, and we’ll try anyway to get you on the calendar and we can talk.
Larry:
Fantastic. Well, thank you so much, Kurt. I really enjoyed the conversation.
Kurt:
Thank you very much, Larry.
