Podcast: Play in new window | Download
Subscribe: Apple Podcasts | Spotify | Android | TuneIn | RSS
May Habib is the CEO at Writer.com, a generative-AI platform that has been helping enterprises use AI since 2020.
Her company builds its own award-winning large language models and is pioneering approaches like “headless AI” to help employees across an enterprise use AI to be more creative and productive.
We talked about:
- her work as CEO at Writer.com, a “full-stack generative-AI platform,” for the past four years
- her decade-long work in the AI and NLP space, beginning with translation solutions
- her take on the “over-chat-ification” of AI products, the reliance on chat interfaces as opposed to other ways to access AI capabilities
- her prediction that 2024 will the “get real” year for AI
- the use of fine-tuning and/or RAG to connect learning models
- the inadequacies of vector databases for knowledge retrieval and their exploration of knowledge graphs to fill the gap
- a new role, the “AI ontologist”
- another new role, the “AI program director” which includes a mix of left- and right-brain thinking and technical skills
- some of the use cases for “headless” AI
- their approach to securing and protecting the various kinds of data used in their LLM
- how she sees the role of data scientists in AI
- their tactical approach to building knowledge graphs for specific business use cases
- their work at Writer on no-code and low-code tooling to help their customers build solutions and tooling on the platform
- new content job roles that are emerging as AI takes hold in enterprises
May’s bio
May Habib is CEO and co-founder of Writer, the only fully-integrated generative AI platform built for enterprises. Leading companies, including Vanguard, Intuit, L’Oreal, Accenture, Spotify, Uber, and more, choose Writer to help them deploy generative AI across their businesses, allowing them to automate and augment key operational activities and increase employee creativity and productivity. Writer’s family of large language models (LLMs) are state-of-the-art, topping leaderboards for natural language understanding and generation. The company’s security-first approach means that Writer’s large language models and generative AI platform are deployed inside an enterprise’s own computing infrastructure.
Launched in 2020, Writer has seen immense success with customer adoption, has grown revenues by 10x in the last two years, and has over 150% net revenue retention. May and the Writer team have successfully raised over $126M in funding from notable investors, including ICONIQ Growth, Balderton Capital, and Insight Partners.
May began her entrepreneurial journey as a teenager, and founded her first language startup, Qordoba, a localization software company, 10 years ago. May is an expert in AI-driven language generation, AI-related organizational change, and the evolving ways we use language online. She has been recognized for many different awards, including the recent 2023 Forbes AI 50 and Inc.’s 2023 Female Founder Award. She is a MELI Fellow with the Aspen Institute. She graduated from Harvard University and spends her time between San Francisco, where Writer is based, and London, where her two children live.
Connect with May online
- email may at writer dot com
Video
Here’s the video version of our conversation:
Podcast intro transcript
This is the Content and AI podcast, episode number 14. Over the past year and a half, innovative artificial intelligence startups have taken the tech and content worlds by storm. In her position as the CEO of the generative AI platfom Writer.com, May Habib has been right in the middle of the excitement, and out in front of it. Writer and their clients were deploying LLM-driven generative AI programs inside of large enterprises long before OpenAI’s ChatGPT 3 captured the headlines and launched the current wave of AI disruption.
Interview transcript
Larry:
Hey everyone. Welcome to episode number 14 of the Content and AI podcast. I am really delighted today to welcome to the show Me Habib. May is the CEO and co-founder at Writer, an app many of you’re familiar with. They’re just having a great year and I was excited to get her on the show towards the end of 2023 to talk about topping the MMLU leaderboard with the Palmyra, their LLM, closed the nice funding round. Sounds like things are going well at Writer, May.
May:
Oh, thanks Larry. It’s so nice to come back and chat with you. Yeah, we’ve had a great year, thank goodness. I’ve got our last all-hands of the year after this conversation, and so it was definitely nice to look back. We do these weekly updates to the whole company. I write them, and I went back and looked at week one and compared it to week 52, and then one’s like, “Oh, let’s go back a little further.” I went 2022, week 52, and then 2021, week 52, and yeah, it’s awesome to see things build and all the progress.
Larry:
Yeah. Well and one thing, and you’ve been part of that progress. ChatGPT, which is where the current kerfuffle is all about, that’s barely a year old, but Writer’s older, and Cordova was even older than that, right?
May:
Yeah, well we’ve been in the NLP space for a decade, me and Waseem, and starting in machine translation. I think we were able to come to the world of transformers with maybe two distinct advantages, I think, over the folks that are in the space now, OpenAI and others as well included in that.
May:
One is, we were very much less a technology and search for a problem. Because we saw so many content challenges in the enterprise that could be solved with AI, having come from translation. So, it allowed us to really take a solution-based, outcome-based approach to thinking about how to productize this cool technology versus not.
May:
Now, obviously a general-purpose AI-based chat has captured the imagination, and has been an incredible thing that the OpenAI team has introduced that we obviously didn’t think of, but in a lot of ways what it’s done is open up what people thought could be possible with AI, and it’s made room for solutions like ours to really explode, because we really serve that enterprise need, that very solution-specific application that is enterprise-ready, is secure. So anyway, it has been a fun road and a really fun four years with Writer.
Larry:
One of the implications, and you know as much about this stuff as anybody, in fact you’re right up there with OpenAI in terms of your accomplishments and the power, the service you offer. I’m curious, what are you like… And I think there’s a couple of things in this question and that I’m hoping to get out of this conversation. One is just the general state of the AI market. A lot of what you just said, I think it’s going to help people ground themselves and feel it. But I think one of my questions is, for example, is this just another SaaS app that the software in the background is an LLM, or will there be fundamentally different things you think that content folks have to consider as they go into both working with these tools and working on these tools?
May:
Yeah, I think maybe taking that question a couple of ways. One, the user-experience cut of the market, and then the where-are-dollars-being-spent cut of the market. I think it’ll allow you to see the gap that we see and that we feel, actually, looking at it in these two ways. I think from an end-user perspective, that cut of the market, there is this over-chatification of what AI can do, and everything is a fricking dialogue to get stuff out of AI, and it’s just so early, and the interfaces obviously shouldn’t all be chat UIs, but that’s kind of the case right now. Whether somebody gave you a Copilot license or you’re personally paying for ChatGPT Enterprise, I think most people aren’t getting the value they thought they would, given all of the headlines.
May:
That adoption gap isn’t because the capabilities aren’t there because we are building the capabilities. They are fricking crazy magical, and I think when we chatted last, I said probably something along the lines, if this was 18 months ago, I probably said, “Larry generative AI is like giving everybody an assistant and a chief of staff.” I mean, that’s not what it’s like anymore. It’s giving you the best version of yourself, 20-years expert into the future. There is so much, even in 18 months, so much that the models can do.
May:
Anyway, all to say that the end-user experience cut of the market is super under-optimized and today, despite all of the hubbub, I can’t go into my sales force and say, “I’m in London in January, who should I see of our deals that are closing in Q1?” So even the AI that’s supposed to get built into all of our systems of record, isn’t really doing the things that we want it to do. Folks who are trying to connect Copilot to their Microsoft data aren’t seeing the kind of answers they would like. And I think power users who have figured out how to get a lot of value from ChatGPT are, but your median user really isn’t. So, that’s that cut of the market.
May:
In terms of where there are real dollars being spent here, I think the enterprise is probably over-investing in the infrastructure and the utility model layer, and are trying to rebuild from scratch every use case, and there are a lot of things that are breaking about that experience. And the total cost of ownership, I think, isn’t making sense for a lot of companies. The accuracy and business impact of some of these pilots and POCs isn’t materializing.
May:
So, next year is going to be get-real year, which is exciting. I think we’ll see a lot more exciting end-user interfaces and experience that build on the toy making and piloting of tools this year. And then I think enterprises are going to be really looking for just more comprehensive solutions to filling generative AI needs internally.
Larry:
Yeah, a couple of follow-ups. First thing, to the example you gave about not being able to get what you want out of Salesforce, you know that the information you want is in there, you know that the data’s available. Similarly with the Copilot, and it’s theoretically integrated with all the Microsoft suites and tools and probably all their BI stuff and whatever else, and that’s not working. What’s the disconnect there? How is this going to be stitched together?
May:
Connecting the models to your data is done through either fine-tuning or retrieval-augmented generation, or a combination of both. The context windows on these models, including ours… We had our customer conference last week and we announced a 200,000 token window for Palmyra and 30 languages. So, lots of improvements. But even with those capabilities, fine-tuning for various use cases has a ton of benefits. Connecting the models to structured and unstructured data have a ton of applications, including these knowledge-retrieval and smart-enterprise-search types of applications. The standard vector-database based approach, where you’re basically creating a bunch of embeddings, storing them in a vector DB and then, depending on what the prompt is, sending back results to the LLM from the vector DB. The accuracy rates on enterprise scale and enterprise state data just really aren’t cutting the mustard. We think a lot of that has to do with sort of a missing knowledge graph layer.
May:
Nobody really wants to do ontologies of old, and so what we have done, is basically train a separate LLM to build knowledge graphs of folks’ data. That is what we call full-stack generative AI. So, you’re not having to recreate all of that yourself and rebuild it, and the benefits are both, you’re not calling out to a bunch of third-party services, and you get a platform that understands your data a lot better than throwing all of it into an embeddings model and hoping for the best. So what is old is new again, Larry, but t’s LLMs.
Larry:
Yeah. As you talk about that, one of the things you mentioned there is knowledge graphs, and I don’t know if you said the word ontology, but that’s what kind of organizes a knowledge graph.
May:
Yeah.
Larry:
And you said something the other day when we were pre-talking about this, about ontologies is in the future of content. Is that the practice that gives us a chance to organize the knowledge that better informs these tools?
May:
I think nobody wants to go back to manually-and-analog building ontologies. I do think folks with ontological expertise and content mapping and strategy, understanding and knowledge, are going to be really critical. We see them be really critical to helping get these digital-assistance and knowledge-retrieval applications to the level of quality that everybody really wants to see them at.
May:
So, we are using AI to actually build the relationships between entities. This is happening at such a massive scale that you need AI to do it, but I do think, for a lot of use cases, we are going to need help with folks who’ve got that expertise zeroing in on what data might actually be causing a bunch of confusion, or where are wrong answers getting traced back to. So, everyone is still so early in these kinds of applications, and there’s still very much a honeymoon period, I think. Getting to a hundred percent accuracy… We tend to ship things with our customers once 80% of answers, they’re at 90%-or-above accuracy and completion. But really, over time, working and improving these models, I don’t think is going to be about making the LLMs themselves bigger or smarter. It’s really going to be about better shaping the data that’s used for retrieval augmented generation.
Larry:
Right. You’re reminding me now, something we talked about, this was a year and a half ago when we talked, you said something about that you felt like you were helping to usher in a new type of content practitioner, this half data scientist, half editorial leader, a lot of what you just said. Is that a job description yet or how’s that unfolding?
May:
So, what I just talked about is a different job description. It’s like AI ontologist. So that’s what we’ll talk about in a year. What we were talking about last year, or a year and a half ago now, we actually had our first conference for those people, Larry, last week we called it the AI Leaders Forum. It is people whose job now is scaling up generative AI programs. It’s a lot of folks with this mix of left brain, right brain. They’re editorially minded, they know what excellent looks like, but they are executives with real stakeholder pull.
May:
They understand how to make stuff exciting, how to get adoption, how to showcase and evangelize generative AI use cases, how to apply generative AI principles to problems. What we really want, are people who are reinventing, re-architecting how work gets done, AI first. A lot of those early writer users have become those people at their companies. We had them all together in San Francisco last week. So it was really fun and yeah, it’s absolutely a job now. If you go to our blog, writer.com/blog, you’ll find, I think we call it AI program director job description, because it absolutely is a job.
Larry:
Probably a lot of overlap in that one with content operations, I’m going to guess or is there?
May:
Let me think about that. I do think there is some. Anna Maria from Salesforce is coming to mind. She’s got a lot of content ops experience. She plays this role at Salesforce, and she also has got a lot of tech stack rollout experience. So, really being able to track just new behavior change, and encourage it.
Larry:
Yeah. I wonder, is it more like… It reminds me, when you’re talking about the evangelism element and the need to sell it, it reminds me of design systems or other new headless content management systems where it’s not done after you architect and build it, you got to keep promoting it. Is that kind of what you were getting at with the evangelism?
May:
Yeah. Well, it’s interesting that you go to headless because I’ve actually borrowed that concept as we’ve described what Writer allows technology teams to do, right? Writer is a full-stack generative AI platform that allows companies to build dozens of different generative AI use cases and workflows on one platform, and you can take any one of those applications and run it in any of your clouds in your own virtual private cloud. So, I have come to describe that as headless AI apps. So yeah, the analogies with headless CMS are definitely there. What were you asking about headless?
Larry:
No actually, let me follow up on that a little bit, because that kind of follows on the whole, just the rise of decoupled architectures, like composable commerce and headless CMSs and all that. It totally makes sense the way you just said that, that AI capabilities like having a general-purpose or general purpose-ish AI platform as part of that mix of your enterprise architecture, that just seems like a no-brainer at this point. What do those APIs look like? Because content, I do structured content and content modeling for a living, so I know how that stuff looks, and I’ve seen other similar architectures. So the API calls, is that like question-answering, stuff or is it more than that?
May:
Yeah, what I mean by API calls, is the applications themselves are containerized. So, one app could be a sales enablement tool for your salespeople to basically query against all of the marketing material for ICP messaging and case studies. Anna at Commvault, the CMO there, she’s built a really cool app for her salespeople. That’s one app, that’s one use case. Another use case that people could have built in Writer, is an app that answers RFPs, that is trained on all the RFPs your company has answered before. That’s one application. So, you may want one in AWS and one in GCP for whatever reason, or that’s where the data is, et cetera, and that’s what we call headless.
Larry:
Got it. Okay. Yeah, that makes perfect… It’s like the capabilities of those LLMs can drive any number of… What’s the data look like under this? Does each application kind of bring its own data, or is there an emerging enterprise data strategy going on?
May:
Data, I have talked a lot about, I hate the word data as it relates to generative AI, because it’s not very specific. So, data could mean the training data inside of the foundation model and you don’t really, as a user or the buyer of AI products, even as an enterprise buyer of a very composable and auditable full-stack generative AI platform like Writer, you don’t have a lot of say in what goes in there. You can audit it, make sure there’s no copyrighted data, walk through the legality of how we built it. These are all things that our enterprise customers can do. But that’s like, ship has sailed months ago, on that.
May:
Then, from a data perspective, you’ve got data that you use for retrieval augmented generation, and so that might be structured or unstructured data that you are connecting to a particular use case or application that you have built. Data could also mean the examples or information you are uploading as part of a prompt, whether that is documents or just plain text. Data could be what the LLM outputs to your user. So, what we have seen be real state-of-the-art enterprise grade, is actually having a secure and protect approach to all of the different buckets of data that you are interacting with when you are using a generative AI application or introducing a generative AI application.
Larry:
Interesting. Those distinctions are really important, I think. As you outline them, each of those has a different intent and purpose and use in these systems. Is this where data science and data engineering come in? What percentage of the folks working on these kinds of things fall in those buckets like data science and data engineering?
May:
I think the data teams’ capabilities with respect to generative AI, really cut across a wide spectrum, Larry. There are folks who really get it and folks who are still trying to, quote, “Clean all of our data for AI.” I don’t think that they really understand necessarily what that means. And I think they’re probably under-consuming generative AI in favor of cleaning data. Because when you’re fine-tuning or producing examples for a lot of the, especially content-generation use cases, you don’t need that much data, which means you don’t need to spend months cleaning stuff. There is clean data, i.e. clean examples, sitting on business users desktops and in their inboxes and in their folders. So, it doesn’t rise to the occasion of this massive, large-scale data science project. We’ve seen incredible results, even at high-volume, no-more-human-in-the-loop type of use cases with 15, 20, 40 examples worth of training data.
May:
From a retrieval perspective that looks a lot different. There I really haven’t seen the data cleaning projects really do the kinds of things that you need to actually make retrieval easier. What would really freaking help? Tables not being broken up across two pages. No data science team is going to go do that. Models really struggle with that. The way that you actually understand how to pluck out an answer, a lot of enterprise data is in tables, especially the kind of stuff that agents need to be answering or looking at if you’re sort of like a level two, level three agent doing more higher-level work on these calls. So, there’s definitely a space, of course, for data science in generative AI. I think the function is likely to evolve to be generative AI first, versus adopting some of the techniques that served previous generations of AI.
Larry:
Got it. Interesting. Hey, I want to go back to something you said earlier too, about ontologies and building ontologies with gen AI. Because that echoes what I’m hearing. I go to The Knowledge Graph Conference every year and last year I went expecting-
May:
My God, I want to go to this conference, please invite me. I didn’t know there was a Knowledge Graph Conference. That sounds great.
Larry:
Oh God, you should speak. I’ll connect after this.
May:
Okay, thanks.
Larry:
Yeah. But I went last year, it happened in May, six months into the whole ChatGPT thing, and I went there thinking like, “Oh, thank God. Knowledge graphs are going to come to the rescue of these stupid stochastic parrots.” And instead all the excitement was about building ontologies with AI. And it sounds like you’re doing that. Can you tell me about how does that… Because in classic ontology engineering, you come in with a business intent and domain knowledge at the top, and then you look at your data instance and go to the middle and build ontology. But it sounds like the way you described this, I’m picturing more of a bottom-up approach. Is that accurate, or there must be some human input and insight in there as well.
May:
It’s very tactical. You are building a Q&A app for your salespeople in the field. So, what needs to go into that knowledge graph for that Writer use case, is everything you onboard those people with. And it might be like 80 PDFs and six different websites, internal websites, and then maybe some PowerPoints. That’s the data. In insurance, a knowledge graph could contain every policy document for millions of users, and that is a very different type of set of logic and work to get rag there to be really high quality and retrieve the right stuff and answer the questions the right way.
May:
So, the use cases are starting to bundle up and get repeatable, but we are learning alongside our customers and that’s what I think makes this space so fun and interesting, is how collaborative customers and tech vendors really have to be to build stuff that goes into production. We are building the capabilities, we are understanding what they do, we get to pattern match. But the real understanding, rubber hits the road, only when you’re actually looking at a set of questions that people have asked your app that we answered wrong, and that’s what sends you down the data rabbit hole and has you improving the knowledge graph so that we can answer those questions better and more accurately. So, I think that will become a data job, and I don’t think that job is getting done by data scientists today, if that’s your question.
Larry:
Yeah, no, exactly. And where that work happens. That kind of goes back to that thing you said a year and a half ago. That was one example of a new job. There’s a whole bunch of new work being…
May:
Yeah.
Larry:
Hey, that’s actually… This is something that I try to address in every episode, there is so much anxiety in the content world now about, “Oh God, AI is coming from my job,” and there seems like there’s some reason to think that, but I think that half of what we’ve talked about seems like examples of new kinds of jobs that are being created. Do some of those jobs exist at Writer now?
May:
Yeah, what a great question. Our stack looks like pre-sale solution architects then post-sale technical account managers, and they’re really doing the same job. We have built both a low-code and a no-code console to speed up the work of building these generative AI applications. Further up the stack, we have customer engineers and then AI engineers who all look at and see customer problems. So, it’s solving these problems at different levels of abstraction, is what that stack kind of does. Customer engineers are building building blocks that our solution architects and our technical account managers use to build specific customer solutions. Then the AI engineers and software engineers are taking as many of those as possible and abstracting them so that customers can use them themselves, and turning it into tooling, additional tooling that we all can use.
May:
So, the data work, I think is really happening alongside all of that. Some of it becomes building blocks. I’ll give you just silly examples, or not silly, but more trivial examples. Understanding tables in PDF separately from tables that we pull out of webpages. Or tools that make sure that something is a specific character count or a specific grade level. Those are all building blocks that can be applied by our customers and by us as we build these solutions. Those are the data work, are jobs being created that sort of tuck into our work right now, that our GSIs are doing for some of our customers, that our customers are doing for themselves.
May:
The annotation work, ugh, we haven’t even gotten to that. There are a lot of jobs here. I think what is fun is watching programs grow up around generative AI. A big part of what we were talking about last week, as I heard AI program managers talk to each other, is how they get more project management staffing for keeping up with all of the use cases and the rollout and the change management. I think, on the backend of that, really building those use cases out, I think in six months to a year, people will be asking, “How do we get more content people looking at the annotation data, looking at how we scale the building of these use cases, how we build the knowledge graphs that form, yes, AI ontologies, but choosing what data goes in there is going to be step one to having a really high quality AI graph. Right?
Larry:
Yeah. May, I can’t believe that we’re coming up on time already.
May:
Oh, wow.
Larry:
There’s so many rabbit holes that we… I’ve kept us out of some of them, but there’s still many I’d like to go down. But anyhow, so we’ll have to talk later in the year, I hope.
Larry:
But before we wrap today, is there anything last, anything that’s come up in the conversation that you want to revisit or that you just want to make sure we talked about before we wrap up?
May:
No, I think this was a great conversation, Larry, and we went really deep, and hopefully gave your listeners a lot to think about.
Larry:
Yeah, I think we definitely did. Well, thank you so much, May. Oh, one very last thing, if folks want to follow you or connect online, what’s the best way to reach you?
May:
Yeah, I’m may@writer is my email, and LinkedIn is where I’m most active.
Larry:
Okay, cool. I will put that in the show notes as well-
May:
Okay.
Larry:
… but good to have it in the recording.
May:
Awesome.
