Podcast: Play in new window | Download
Subscribe: Apple Podcasts | Spotify | Android | TuneIn | RSS
Figuring out how to best adopt new technology is difficult at any time for any organization. AI tech rachets up this challenge to new heights.
Wouter Sligter helps companies understand the capabilities and limitations of LLMs and related technologies to create trustworthy experience-delivery platforms.
Transparency is a key element in implementing solutions that evoke and support the authentic human experiences that underlie these systems.
We talked about:
- his background as a UX-focused designer and his shift to conversation and AI design
- the growing number of business use cases that his work supports as well as the growing palette of tech tools that he has to work with
- how he creates authentic and trustworthy experiences with LLMs and adjacent tech
- the benefits of RAG (Retrieval Augmented Generation)
- the growing number of platforms that support building AI experiences
- the huge failure rate of conversational AI implementations, and how better design might improve the success rate
- the importance of being genuinely customer-centric when implementing AI projects
- how his background in language and music helps his AI design work, in particular the benefits of “being comfortable with the uncomfortable”
- the importance of companies being transparent about their AI implementations
- how localization manifests in the AI world
- the growing acceptance of chatbots by consumers
- his advice to jump into AI now, beginning with due diligence about how you’ll implement it in your organization
Wouter’s bio
Wouter Sligter is a Senior Conversation Designer and Generative AI Engineer. He has been a committed team lead and has consulted for a large number of Conversational AI implementations, most notably in Finance, Healthcare and Logistics. He has an innovative mindset and a sharp sense for understanding user needs. Wouter always looks to improve the conversational user experience by following iterative design patterns and verifying outcomes through data analysis and user research. Both predictive NLU and generative LLMs and SLMs are part of Wouter’s toolkit.
Wouter has a background in ESL and IELTS teaching at language centres and universities in Vietnam. He has developed a strong awareness for language and cultural peculiarities, with native fluency in English and Dutch and good conversational skills in Vietnamese, German, and French.
Connect with Wouter online
Video
Here’s the video version of our conversation:
Podcast intro transcript
This is the Content and AI podcast, episode number 25. One of main reasons that people have taken so quickly to AI tools like ChatGPT is their conversational nature. People like talking to each other – and to computers. In human conversation, we’ve developed skills and instincts that help us determine the trustworthiness of the person we’re talking with. In tech-driven conversations, we often have reason to mistrust. Wouter Sligter helps companies build conversational systems that express the authentic humanity of their creators.
Interview transcript
Larry:
Hi everyone, welcome to episode number 25 of the Content and AI Podcast. I’m really delighted today to welcome to the show Wouter Sligter. I met him in Utrecht in the Netherlands. He’s in the co-working space we both work out of. There, he is a conversational AI consultant. He does conversation design and he’s a generative AI engineer. He has his own company called You and AI Welcome, Wouter. Tell the folks a little bit more about what you’re up to these days.
Wouter:
Hi Larry. Very good to be here. Thank you for inviting me. What am I up to? I think you mentioned the three things that I like most doing and that I do most often. I’ve come from being a self-employed freelance designer really, when in 2018, Facebook started with their chatbots on Messenger. I jumped in and quickly caught on and got a lot of clients worldwide, really building chatbots for them. At that time, I was mostly working on the content side with what you see is what you get kind of flow builders and slowly got pulled into the tech side as well.
Wouter:
I worked for enterprise as a consultant for a few years in the Netherlands, and then I decided last year to go back to being freelancer, and that eventually culminated in now having my own company, You and AI, with which I’m doing all kinds of outsourcing work from Vietnam. Of course, lately a lot of work is involving generative AI LLMs like RAG implementations and fine-tuning. In my bones I’m still a Uxer, so I’m always looking to build stuff that actually works for people rather than only playing around with tech that no one uses. There’s really my strong point, I think.
Larry:
I love the way you say that. I have many engineer friends, but they’re really prone to just building stuff because they can. We’re both designers and I love human-centered design and human-driven design decision making. One of the things you said in there, you kind of reminded me of your heritage because you come out of conversation design kind of UX and conversation design specifically within that. That field has evolved. All these new generative AI tools have a conversational or a chatty kind of interface, but you’ve been working with that kind of interface, but the bones underneath these interfaces are way different now. Five years ago, it was all NLP and kind of flow building tooling. Can you talk a little bit about the transition and your skillset and the demand for your kind of talent over the last five years?
Wouter:
Right. Yeah, so I think in the beginning, because most of my work was involving Facebook, there was a lot of demand for the marketing use case, the sales use case, like getting cold leads to convert and to some extent also customer service. Then when the enterprise-level companies jumped in, the customer service field became much bigger. I think now today, that’s still the major use case for most conversational AI. But now with the LLMs and all the generative AI functionality that we have, the possibilities have become so much bigger. There are so many more use cases that can successfully or let’s say an acceptable level of quality be implemented or be used.
Wouter:
Right now, I’m getting all kinds of stuff in, so it can be like fine-tuning for reading Excel sheets, fine-tuning for creating posts on LinkedIn, but also still the follow-up of let’s say the fall-backs on the traditional NLP bots where traditionally it would say, “Oh, sorry, I don’t know that.” We now often use LLMs to fill in those gaps and pull from the company website or company knowledge base to answer even those questions better than they could ever before.
Larry:
That’s right. You’re just reminding me, I kind of phrased it as an either/or an evolution in development, but we haven’t left the old stuff behind. Like you just said, you’re still the fallback if an LLM or another agent fails. You have just a bigger palette of conversational tools to work with, it sounds like. Is that accurate?
Wouter:
Yeah, definitely. Definitely. And that makes my job so interesting because we started with the rule-based stuff and then NLP came in and then we thought like, well, now it’s getting really interesting and a little bit difficult. Now we’re at a stage where we have these LLMs that produce or don’t produce the output that we expect with all kinds of hallucinations and technical challenges, which I think make my job so much more interesting, but also more challenging in a way because you need to explain to everyone what every bit of tech does and make sure that the clients who are actually using it understand why we’re using that tech so that they can also explain to their stakeholders why things work or why they don’t work.
Larry:
When we talked a couple of weeks ago… Oh, I’m sorry. You were going to say something?
Wouter:
Yeah, yeah, no, go ahead. I can keep talking for ages about this stuff. I’m actually trying-
Larry:
I’d love to circle back to something we talked about a couple of weeks ago when we were preparing for this. One of the implications of that, what you just said, this evolution of the tooling, you go from rules-based, like it’s all guardrails all the time in a system like that to NLP, which has intense understanding and all those utterance magic to these crazy hallucinating LLMs. I mean, I’m exaggerating of course there, but there’s been an evolution in that practice. One of the big things that comes up just every conversation and every conference and event I go to is the importance of trustworthiness and authentic, because these things are conversational.
Larry:
They sound like a human, but it’s not always authentic sounding. So there’s this sort of combination of things, at least I conflated them in my mind, this notion of authenticity and trustworthiness. Can you talk about how you instill those kinds of… How do you help people trust these experiences as you’re navigating them through?
Wouter:
There’s a lot of levels on that question. Let me just pick one first. I think that when a business, when an organization chooses to work with the kind of AI that we have now, then they need to decide if they’re comfortable with that level of risk that they’re allowing in their applications because we know that LLMs are not perfect. They do hallucinate even if we put the guardrails in place. Actually, you have to decide for each use case and each implementation which level of risk you are comfortable with as an organization. For an internal use case, it might be okay if 85% of the answers are correct, but for a customer-facing use case, you might want to see 90 or 95 or even 100% depending on the context. I think that’s one important thing to note.
Wouter:
With that extra level of quality really, of output quality, that does take a lot more development time because then you need to go through the data preparation, the fine-tuning, the rack, the guardrails to make sure at the end that the output is still within acceptable levels, which can be done many different ways, which just takes a whole lot more time than if you just put in a prompt to a generic LLM.
Larry:
As you’re saying that, I’m really curious about the tech stack that permits all this, and not just the tech stack, but the workflows as well. Like you mentioned, the quality of the data is obviously important, the fine-tuning of the models, and then things like RAG, that’s a whole… Maybe let’s focus on that because that’s in the news a lot lately. The RAG stands for Retrieval Augmented Generation. That’s the notion of adding authoritative knowledge sources to these LLM workflows. Can you talk a little bit about how that works?
Wouter:
Yeah, yeah, sure. Before I do that, I do want to stress, sorry, Larry, it was a good point, but I think I do want to stress that you don’t have to put all these bits of tech in. If you are a small business and you don’t have the means to go through that whole process of preparing your large data set, fine-tuning, et cetera, et cetera, then there are also ways to produce good use cases that don’t need that level of sophistication, I think. Apart from that, I think RAG is something that has proven to be very effective because what you’re really doing with RAG is you are kind of grounding a existing LLM, like for example, GPT4 or a ChatGPT or Llama, take any generic LLM and you’re grounding it with the data that is relevant for your use case. That could be, for example, a list of FAQs with the corresponding answers.
Wouter:
What happens then is that instead of answering from the generic data that the model has been trained on, it will fetch the information from the RAG knowledge base, and it will rephrase that data using the linguistic knowledge it has from its training data, but it will not change the meaning and the facts of that data.
Larry:
Got it. Hey, I’m really curious. The thing you just said too about the small business, it doesn’t have to be full on. I’m just reading all of the news every day and it’s all cool, like enterprise-scale, big stuff. It occurs to me that every small to medium-sized business in the world is also going to be concerned with this. Is there sort of an emerging conversation, design gen AI as a service kind of thing where you can get the benefits of those kinds of practices, but in a configurable, more consumer-like tool? Is that happening?
Wouter:
Well, yes. You’re seeing a lot of platforms pop up now, which make part of the journey easier. For example, there are the platforms who were traditionally the chatbot builders with the visual flow builders who have also introduced the LLMs through their platform. Then there’s now also platforms that make it very easy to do fine-tuning based on data that you already have. So instead of having to transform all that data to make it suitable for the fine-tuning process and then going through the coding, etc, etc, you don’t have to do that anymore. They will provide all that sort of out of the box.
Wouter:
There are also some very cool platforms that are actually testing the outcomes of your prompt over a variety of different LLMs. You just put in one prompt maybe with some variables, and then it will just run all of that at once. Immediately in the output you will see how long it took, how much it cost you, what the answer was, and you can also put in test parameters to see how high the output scores on your test parameters. Those kinds of platforms I think are useful except what you’re seeing is that they just solve one part of the problem, whereas most use cases need a combination of all these things.
Wouter:
Secondly, I also think that they are very tech focused at this time, and I have to say it again, in the end, you’re building it for the end user. If they don’t adopt the solution, then there was no point in building it in the first place, which we see all too often, unfortunately. That part is still something that is overlooked, understandably, but that I think and hope is going to pick up a lot in the next few years to make AI actually user-friendly, including that trustworthiness, what we were just talking about.
Larry:
Yeah, and as you were talking to this now, I was thinking that this is so common in new tech adoption cycles as people get enamored of the tech to the point where they kind of miss some of the business and design benefits that they are afforded there. I guess it’s just inevitable because the engineers are getting all the funding and doing all this stuff, and it’s sort of on you as a designer to tidy things up. Is that kind of how it feels sometimes?
Wouter:
That does happen, but I do try to get the idea in my client’s head as soon as possible that there is an important role there for the designers and that they have to cooperate. There was a time when I said that designers, like a senior designer, is supposed to get the same salary as a senior developer, and then people were laughing at me because that is so far from how it really is. But in the end, I think the number of implementations of conversational AI that has failed in the past years, it lies somewhere close to 80%, which is huge. I’m pretty sure that most of those failures stem from it not being adopted or not being made to fit with the actual demands by the users.
Larry:
Well, that may be. I’m thinking of two real valuable things you can bring to this conversation right now. One, how as a designer or a product person or somebody working with this to figure out how to not be in that 80% of failed projects? I guess, what makes for a successful implementation? Maybe that’s the way to phrase that.
Wouter:
Right. Well, I mean, I think it’s not just the designer, but it’s also the product manager and the whole culture in the company has to be customer-centric. A lot of companies say that and then they don’t always practice it. I think, just being a bit generalistic now, the tech and the content are kind of the two extremes in a typical AI team. For me, as a person who understand both worlds very well, it’s sometimes interesting to see how they misunderstand each other very easily and talk over each other’s heads, expecting the other person to go their way in the end, which of course is more often than not than it is the case. I think that for managers, there’s a big role there to make sure that the two skillsets are equally valued within the team, and that the decision can be made based on the soft research of a UXer rather than the binary findings of a developer.
Larry:
As you’re reminding me of this, it’s kind of a running observation in both of my podcasts that this is all about people ultimately. We’re serving people, we’re working with people to get these things done. I love the way you just said that too, that, “Well, of course the senior designer should be making the same as a senior engineer. It just makes sense and people laugh me out at the room.” But have you had any success? This is an ongoing thing in the content world, it’s just trying to get seat at the table, all those kinds of concerns about wielding appropriate influence in the organization. It sounds like you’re very attuned to that. The content that goes into these systems is equally as important as the technology that manipulates them. How do you get people to see that?
Wouter:
There’s a lot of experimentation in AI, and I think that’s something that has been true since the beginning, since I stepped in, and now maybe even more because there’s so much tech that we don’t completely understand yet, or that not the whole team understands yet, not the whole organization understands yet. You need to kind of gradually introduce it by running these experiments and then publishing it bit by bit, making people feel more comfortable with it bit by bit, both in and outside the organization.
Wouter:
I think that these experiments can also function as a great basis for your product development. So you start with something small, all maybe based on your initial UX research, or maybe it can be based on your business where the business value lies and maybe combined with hopefully with a user journey that makes sense. Then you kind of push that live to see how it works for people, how it responds, and then you do a conversation review and conversation-based iterations to make it better step by step.
Larry:
That start small, iterate, that’s very familiar to all of our design folks and especially the innovators. You never just dive right in. The other thing I was thinking about in terms of these people relationship things, I just wonder, you speak six languages. You grew up in the Netherlands where English is kind of almost a business language because of the international nature of things. Do you feel like that your language skills, and you’ve taught English and taught languages and learned a bunch of languages, do you feel like that… I guess this is a lot of language in all this, do you feel like that background helps you in your work?
Wouter:
Oh, yes. I love that you asked me that question, and yes, it does for sure. Maybe just as much as the languages is also the musical background of a lot of my family members, because both my parents were musicians. My sister is a musician, my aunt is a musician in classical music. That’s something that was so present in our home when I was a kid. If you’re talking about the opposite, I think of a binary calculation, then that’s probably the buildup of a classical piece of music. The dynamics are and the tempo, and I don’t know, it’s refined. I think that kind of refinement is something you also see in languages, which is why I’ve always been very easy to pick up on new languages.
Wouter:
That brings me then to the interesting observation that in languages, if you’re talking to someone from a different culture in a different language, then there is always a level of you’re out of your comfort zone. You’re not sure if you’re being understood and you’re not sure if you can understand them. Being comfortable with the uncomfortable has proven to be very helpful for me when working with AI because this level of uncomfortableness is there constantly, and it’s really not about trying to get it completely out because it’s probably never going to happen. It’s about understanding that why it is there and trying to limit it as much as possible by doing your due diligence.
Larry:
I love that image that it’s not about omitting the discomfort, it’s about being comfortable with being uncomfortable. That seems really useful with these things because, we’re both from the technical side where computers for the last 50 years have been these ultra reliable, predictable things, same information in same information out every time. Now we have these completely, like “stochastic parrots” as they’ve been called, models churning out content. But then on the content side, we’ve always had this kind of precision around language and how you say things and how you structure documents and other content artifacts.
Larry:
This kind of circles back in, I think, to the trustworthiness and authenticity. I guess I’m going to answer the question. I’m going to guess that the first step is to be comfortable in discomfort, but then what do you do from there?
Wouter:
I’m not advocating that we should all let go of all those guidelines or always allow for full level of uncomfortability, right? Obviously not. Like I said before, we should always be looking like what is the acceptable level of risk and the acceptable level of uncomfortableness, if that’s a word, for a particular use case. That might be different for a customer-facing product than for an internal product. I think most importantly, and I think you made the right link there with transparency and openness, because these models are going to make mistakes every now and then.
Wouter:
As much as we do our best to make it work perfectly, there’s going to be the odd hallucination, the odd outlier that we didn’t design for and didn’t expect. I think that the best way to deal with that as companies is to be completely transparent on these things, on how they’re using AI, why they’re using AI, maybe even some parts of their tech stack, but also maybe it has to be in their DNA to be an open company that is aligned with the values of the society that it’s in, because that is ultimately what’s going to make the difference between a customer or a user responding in anger to a mistake or responding in like, “Oh, well these guys, I trust them, so this is probably just a mistake.”
Larry:
You’re reminding me now too of the localization implications of this that I think you just alluded to in this culture, that kind of mistake is just unacceptable. In other places it might be like, “Yeah, whatever.” Have you had any experience with trying to localize the… How does localization as a practice manifest in this whole world of conversational AI?
Wouter:
Yeah, that’s a good question. I would say that there are different ways to approach that. Obviously with LLMs now, we can just have an LLM translate whatever we put in our home language and then trust that that translation will be okay, which is a danger, but it happens and we see it very regularly. Then there’s the gray area between the two extremes where you do use the auto-translation, but in cases with high involvement or high emotions, you build specific paths, a conversation path, to solve that in that culture to make sure that it doesn’t go wrong.
Wouter:
Then there’s the other extreme, which is you don’t use these other translating functions at all, but you just design a completely new experience for each culture and language, which may sound like a lot of work, but in the long end, if you see what kind of risk you’re taking by using auto-translation, it’s actually not necessarily a bad idea.
Larry:
Interesting. Yeah, because that comes up. Since I’ve moved to Europe, I’ve talked to so many people about localization, and that span, you just described it, everything from simple machine translation all the way out to full-blown transcreation. It sounds like the case for transcreation is getting stronger with the arrival of these tools.
Wouter:
Yeah, it’s become easier, and I think that there is… How can we say? I think because it’s been more generally accepted and used, people have become more comfortable with that level of uncertainty, and with it being there. If you look at back at five years ago when I started with chatbots, then it would be hard to find a person who embraced it. Everyone would say like, with “Well, this is a bit scary and I don’t like it. I prefer a human.” Usually in most research that I’ve seen recently is more than 50% that is actually very happy that the chatbot is there. So that’s shifting, and I think that brings opportunity for businesses to employ it without losing their user base.
Larry:
God, there’s so many opportunities, which reminds me of hey, we’re coming up close to time, Wouter. These conversations always go way too fast. I literally have 100 more questions, but we’ll save that for the next conversation. Before we wrap up, is there anything last, anything you want to make sure, if you want to revisit anything in the conversation or just anything you want to make sure we share before we leave?
Wouter:
Well, maybe for me, as having worked on so many different implementations in so many different countries and cultures, I am seeing that a lot of companies are struggling to understand and adopt AI successfully. If anything, then I hope that this conversation has made it a little bit clearer for the people who are listening that AI is not a total black box, but it requires a different way of looking at your applications and processes. I’m very eager to help people understand that problem and build use cases that actually bring value and work. I would like to give the advice to not wait, but jump in now by doing your due diligence, and that will set you up as a professional to build great things, including AI in future.
Larry:
Yeah. Nice. Well, thank you so much, Wouter. Oh, hey, one very last thing. What’s the best way, if folks wanted to stay in touch online, what’s the best way to connect or follow you?
Wouter:
I’m on LinkedIn, of course, and also my company website is on YouandAI.global, with G-L-O-B-A-L.
Larry:
Great. I’ll put those in the show notes as well. Well, thank you so much, Wouter. Great to talk again.
Wouter:
Yeah, thank you so much, Larry. It was great to be here. Cheers.
