Podcast: Play in new window | Download
Subscribe: Apple Podcasts | Spotify | Android | TuneIn | RSS
A few years ago, Dan Porder was teaching poetry to university students. Now he’s at IKEA training large language models to generate useful, usable content for user experiences.
He’s picked up new skills along the way, like Python programming, but much of his work still relies on well-established content and design crafts like content strategy and inclusive design.
We talked about:
- his role as a senior content designer at IKEA, where he focuses on AI
- some of his early experiments in composing and evaluating poetry
- his longstanding interest in AI and the development of his tech skills
- how content designers can leverage their skills to work in AI
- his perception that there is currently more opportunity than threat to content professionals in the AI world
- the make-up of the cross-functional teams he works with: data scientists, engineers, developers, content people, designers, subject matter experts
- how to brief and guide generative AI to get the outputs your users need
- how writing abilities prepare content designers to do prompt engineering
- the stack of data and technology that underlies AI and the orchestration mechanisms that connect them
- some of the tools he uses in his AI design practice
- the role of data in content design for generative AI
- the importance of staying aware of bias in training data and always wearing your inclusive design hat
- the role of explainability in AI ethics
- the importance of knowing how to ask data scientists and engineers questions that reveal as much as possible the inner workings of the “black box” in which AI content is generated
- his take on democratization opportunities that arise with the arrival of AI tech
Dan’s bio
Dan Porder is a Senior Content Designer and Content Engineer at IKEA. His recent work focuses on the intersection of AI, structured knowledge, and experience design. Outside of work, he runs an international writing community.
Connect with Dan online
Video
Here’s the video version of our conversation:
Podcast intro transcript
This is the Content and AI podcast, episode number 16. AI is quickly changing the way content designers work. New content duties are emerging that require fresh skills, but at the same time traditional skills like content strategy are becoming more important. In his work as a content designer at IKEA, Dan Porder has developed new skills, like Python programming, and has applied the writing skills he perfected as a poetry teacher as well as the inclusive design practices he developed earlier in his content design career.
Interview transcript
Larry:
Hey everyone. Welcome to episode number 16 of the Content and AI podcast. I’m really happy today to welcome to the show Dan Porder. Dan is a senior content designer at IKEA, where he’s currently focusing on AI stuff, and his title is content designer, but he is really more of a content architect. So welcome to the show, Dan. Tell me a little bit more about your AI and content adventures.
Dan:
Hey Larry. Thanks for having me on. Yeah, maybe I could just start by giving a little bit of background. I think at heart, despite what I’m doing now, I think of myself as a writer, and that’s been my life’s focus since I was young. Writing poetry, writing fiction. I did my bachelor’s in English literature and later did a masters, masters of fine arts, actually, in poetry. Some of it was more on a conceptual side, thinking of language as data. So there was some unusual experiments in the tech world even then for me. Using Google data to create poems. So imagining Google queries as a representation of the collective zeitgeist, and how can we leverage that data to create meaning in poetry? Or using NLP to find meaningful relationships in texts where you didn’t know they were there. But all of that then led me into copywriting, so like brand copywriting, product copywriting, ads, copywriting as creative direction.
Dan:
And then eventually back to the Google data, so SEO copywriting and SEO strategy. And I focused for a while on optimization, research, data analysis for SEO, some technical SEO. And then, yeah, my recent journey has been more in the design world. Content design, content strategy, user experience design. And I’d always been interested in AI and the question was always, how do you do that as a job? Particularly from the position I was coming from as a former student and teacher of poetry and writing. Of course, when ChatGPT came out, like for many people, the connection became clear to me and I started incorporating it immediately into all my work.
Dan:
I realized that I also needed to brush up on my coding skills, and particularly get more invested in Python. And I took some courses specifically on generative AI and machine learning for that purpose, just to make sure I was prepared. But now I think I’m leaning more into the world of knowledge, thinking about the data that we need for AI. The data structures that create meaning for these systems to ingest or to retrieve or to do with what they need. And in the case of generative AI, this is content. This is a task that requires a content designer, content strategist. It’s going to be primarily images, text, audio. So that’s what I’ve been up to lately. And yeah, I’m excited to talk to you about it.
Larry:
Well, that’s great. I got to say, it’s hard to imagine anyone better prepared for this stuff, because to go from playing with Google and poetry stuff, the notion of vectorized word embeddings was just like, “Oh, cool, that’s another way to do that.” I can almost picture this evolution going pretty smoothly for you. But a lot of content people are not as technically curious as you are, or haven’t had the same technical opportunities. And you have a lot of colleagues who are more like conventional content design kind of folks. Have you thought about how people who are less natively technically inclined can jump more into AI stuff?
Dan:
Yeah. I think it’s about leaning on their expertise, especially abstracting that expertise. So for a content designer who maybe imagines themselves more as a UX writer or comes from a copywriting background, it’s an understanding of information, of messaging, of what content works best for people in what scenarios. And that kind of knowledge, that’s less of the craft side and more of the wisdom of content, is incredibly valuable to data scientists and to engineers working on AI.
Dan:
This is some of the expertise that’s needed, is subject matter expertise, including on content. So generative AI consumes data and puts out data. That data is content. You need a content person to figure out what it will be, what the use case is, and what content you want these models to produce on the other end, either for a system or for an end user. So you’re giving up a bit of control on the craft side, but on the strategic side you’re actually, if you’re willing to have those conversations with the technical people, you are asserting control in a way.
Larry:
Right. That’s so interesting because, as you’re saying that, I’m picturing… it’s sort of like the way, a lot of these models, there’s attempts to capture subject matter expertise and incorporating that in there. But you also need that subject matter expertise to train the models. To write the prompts, do all the other stuff as well. Can you talk a little bit about that relationship between… and this gets at people’s concerns of AI replacing them, because if we capture all that subject matter expertise, then all of a sudden it’s like, “Oh, we don’t need content designers.” I personally don’t think that’s coming, but what do you think about that idea?
Dan:
Yeah. People have talked a lot about this. I think some of the concerns are overblown. Of course there’s a grain of truth in this. Theoretically, if we were to all give all of our best data, most of which is just in our minds as experts, so it doesn’t exist in the right data format, but if we were and we were to train models that somehow are still usable and not unwieldy as a result of that, you would start to replace people.
Dan:
That’s not what’s going on right now. That’s not the technical capabilities. Anyone who’s using these tools or working with them can see that. And also just the actual process of properly curating the data and testing and iterating on methods of fine-tuning and reward functions, and getting the right feedback from the right experts. That’s a lot of work. That’s a lot of resources, even for small use cases. So I don’t think that’s the worry. I think it’s more like an opportunity. This is an exciting opportunity to make your work more scalable faster. I think, especially from the content design perspective, also to be able to assert governance over content creation through the consistency of machines that doesn’t necessarily exist in people always.
Larry:
Right. And what you just said, I realized that my question was sort of like I’m projecting the alarm that I feel in a lot of circles. But I think more often the answers are like what you just said. It’s much more hopeful and optimistic in that, at every juncture, there’s going to be more need for our expertise that will, probably not for the next couple of decades, anyway, be codified in machines. So that kind of leads me back to one of the things that wanted to talk about a little earlier, actually. It’s just, done both conventional content design for regular, old digital products. And then now you’re working more on the AI side. Can you talk a little bit about the evolution of the practice as you go from one realm to another?
Dan:
Yeah. Well, I think, as we were just talking about, one thing to notice is the importance of cross-functional teams. So having not just the tech people in there, the data scientists and engineers and developers, but also the content people, also the designers, also the subject matter experts aligning with business goals, aligning with users. An obvious difference, though, for content people is that the content that is produced from a generative model is outside of your control, ultimately, or at least it might feel that way. So if you’re a word person, the exact token produced by an LLM, the word is not decided by you.
Dan:
And then therefore, to do content design, you essentially need to just go a couple layers deeper in the tech stack and manipulate things there. So you are essentially… under a traditional model you could imagine it as briefing the content creator or guiding the content creator, the content creator being the generative model. And briefing it with data, with prompts, with databases it can query and things it can retrieve from, APIs it can consume, which then influences or leads to the content outputs. But I think there’s also a lot of talk about using these methods to effectively control AI.
Dan:
And I think that’s beneficial. Of course we need controllability in order to do content strategy if it’s just producing any old thing, where is the content strategy? On the other hand, if you were to control a generative model completely, why are you using a generative model? That’s the magic of it, is the unexpected creativity it has. But I think day-to-day, a big difference is this cross-functionality. Being willing to dive into the technical or to interact with a technical person as your interface to the technology. Not being afraid to do prompt engineering, first of all. I think content designers are incredibly well positioned to do that as writers. But also not being afraid to learn and explore and imagine that now you have a really incredible new colleague who can do a lot of content creation for you. How will you guide them?
Larry:
Yep. Yeah, I think a friend of mine said, “We all have a dozen interns at our disposal now and permits us to do a lot more stuff.” But hey, I want to back up to something you said right at the start of this last question, about the stack. Diving deeper into the tech stack. Can you just outline kind of generically what that looks like? Because I think we all know, as web designers, from the surface level HTML to the underlying content management system to the writing that goes into putting that into the content management system, that kind of thing. Tell me, if you can describe the tech stack for a typical AI setup.
Dan:
Yeah, there’s a lot to this but we could go into the computing and hardware needs and all that. But I think what’s most meaningful is the… of course we’re going to have generative models as part of this. These might be LLMs like ChatGPT. They might be diffusion models, vision language models. There are various kinds. How did we get to that? There was data and they were trained. I think for a lot of companies, you’re not going to be the ones doing the training. You’re going to get a trained model and then you’re going to add a little more data on top of it through fine-tuning processes. Maybe use RLHF, so that’s human feedback fine-tuning processes, to improve the models to do what you want.
Dan:
And then there’s distinctions between use cases at this point. So for some use cases you have your model and you’re going to go do something with it. In other cases, maybe you want a few models working together. So they are maybe each fine-tuned to serve a different purpose, and they’re in communication with each other and then also perhaps with databases, including graph databases, or maybe they’re querying a knowledge graph. Or there might be other tools out there that they can use, so APIs that they can consume as they talk to each other.
Dan:
And then once we get closer to the experience layer, I think a lot of people, when they think about generative AI, they’re thinking about it as the interface. That’s what’s been presented to us with Chat GPT or Bing Chat or Bard, where you are talking to the LLM and it’s talking right back to you. That’s definitely a major use case. But there are also cases where the model or system of models will only exist on the back end, so it’s orchestrating a system. You could imagine a use case like search where the user types something in, then the LLM sees what the user typed in, they go and search through a database using their generative skills with semantic querying and this and that, and then it gives you results for a search. And you’ve actually never interacted with it.
Dan:
Or maybe if you wanted adaptive content on a webpage. Maybe what the model is receiving is user behavior information, and what it’s going and retrieving is content to match that, but you’re not interacting with it. And so in those cases, these models are acting more as orchestrators between different systems, which then eventually become an experience on the outside. And they’re incredibly powerful for that, too. They’re one tool in the toolkit that you can use in cases like that.
Larry:
Yeah. And what does that toolkit look like nowadays? I’m trying to think in conventional, we have Figma and Word docs and Google Docs and spreadsheets and stuff. Are there different tools that you find yourself using as you work with AI stuff?
Dan:
Well, for me personally, and I think for a lot of people, I should say, working in Python. Just coding is incredibly important. A lot of this is a practice of data science. But I think also, if you’re not familiar with coding languages, just I think playing around on the front end with AIs is incredibly useful. And also when we’re talking about curation of data sets, this is something, anyone who knows about content or knows where it might be in their organization or out there on the web, your tool is your web browser, probably, and being able to determine, “Okay, for this use case we want the model to know these things in these proportions. And we will reduce bias in the dataset by using this percentage of this and that. And here’s where you can find it. Here’s the database. Here’s the website. Here are the people you need to talk to, the subject matter experts.” Those are some of the skills.
Larry:
As you talk about that, I’m curious about the data that you’re sourcing now, because we think of data scientists and data like, I don’t know, supply chain people provisioning data from a variety of places. Are you actually getting involved as a content designer in saying, “Oh, we really should be sourcing data from here?” Or, “When I look at this data set I see that, and you need different metadata.” I don’t know. How tight are your interactions with the data?
Dan:
I think that’s something a content designer should have a say in, especially if we’re talking about AI that’s going to do something on behalf of a content designer, shaping an experience or shaping information and messaging. Then it does become relevant. And there’s also, some organizations are at a bigger advantage in this than others. These are old problems of, how good is your data? How much of it do you have? How well-managed is it? There’s also a lot of open source data, which is great. And there’s a whole interesting field now of synthetic data. So it’s essentially if the model… you can use one model to produce data that is then consumed by another model in its training. And that’s a controversial area but one way, I think, to think about that is your organization might have some sort of useful information that is essentially in the wrong format.
Dan:
So if you imagine that you were trying to train a conversational AI maybe, something like ChatGPT, but all you had was a bunch of heat maps. UX research just has a bunch of heat maps of users. And you’re like, “Oh, how do we train with this? This is not a conversation.” Can you then somehow create a prompt pipeline in which you use that data to instigate synthetic data, where it’s essentially reformatted into conversation by an LLM, and then you take that conversation of the heat map and train the model? That was just one example of how it’s not necessarily producing data out of thin air. It might be just a matter of using what you have and the resources that you have in a thoughtful way.
Larry:
Yeah, I’ve seen a couple of other examples of that as well. Noz Urbina talks about this methodology he uses around synthetic data for persona development. And I had a gig a few years ago. It was with a big enterprise that had a lot of… you didn’t want designers accessing the personally identifiable information about the users, so they created synthetic data for designers to work with in their designs that looked enough like the actual data that they could make reasonable designs with.
Dan:
Yeah.
Larry:
Yeah, it sounds like that’s prompting…
Dan:
Yeah. I think, yeah, design cases of being able to see scenarios that you weren’t expecting is valuable. Anything that is coming out of the LLM is synthetic data. So if you’re a copywriter and you’re trying to write a headline and can’t think of anything and you produce 10 options with ChatGPT, there is synthetic data that is useful to your use case. I do think we have to be a little careful when we’re talking about personas or synthetic interviews with users because the model itself is going to contain some amount of bias. And also people, I believe, are irreducibly unique. And we know that data coming out of an LLM is not very unique. So tread lightly, I would say.
Larry:
No, and you’re reminding me one of the early interviews on my other podcast. I talked with a person who’d worked on early data-driven persona development at Amazon and it was a complete failure. So that’s a good reminder of that.
Dan:
No, I think it’s super interesting, though.
Larry:
Yeah, but it’s an interesting topic. But hey, as we talk about the sourcing and stuff, and you just mentioned the issue of the notable lack of diversity in a lot of these data sets, is there anything within your purview and power as a designer? You can certainly point that out and say, “Hey, we really need better data that’s more representative of the population we’re serving.” Do you have any other insights about that?
Dan:
Yeah. I think that’s incredibly important, to put on an inclusive design hat in this work. But generative AI will always contain bias. Data always has a bias. The goal is to reduce harmful bias, and what that means exactly, there’s things that are universally harmful bias and then there are ones that might depend on your use case or your brand and its values and its context. So having a sophisticated understanding of that is certainly a first step but I think, from there, knowing which data… so it’s not just that we need inclusive data, but what would that mean? Where do we get that?
Dan:
If we don’t have it, then what do we do? If we’re producing synthetic data, how do we ensure that that’s not biased? Because you fall into this AIs producing things for AIs producing things for AIs, and it becomes harder to track. So explainability is another very important ethical consideration here that is both towards the bias, but also just un-black-boxing the thing as much as you can so that you don’t end up in unfortunate situations down the road. And there’s I think a lot of interesting research going on in that, too.
Larry:
Yeah. And as you say that, I’m reminded. I went to the European Chatbot & Conversational AI Summit last spring, and the big thing, came up in literally every presentation there, was the need for transparency and explainability to instill trust around these things.
Dan:
Yeah.
Larry:
But then subsequently, like every conversation I’ve had with data scientists or engineers or anybody over the last year, even they find the edges of that black box that they can’t penetrate.
Dan:
Sure.
Larry:
There’s always stuff underneath that is not-
Dan:
There are still things that are black-boxed. And I think, as the content person or as somebody who cares about knowledge and data in AI, it’s not necessarily about penetrating all the way into the black box yourself, but it’s about knowing the right questions maybe to ask these data scientists. Giving them angles to explore. So for example, if you’re not sure why your model is producing a certain type of answer given a certain type of input from the user, maybe you go to the data scientist and say, “Okay, given this query from the user, can you go do some investigating in the model and tell me, what tokens in the query is the model paying attention to?” They work based on attention to particular tokens and token relationships. That’s from that paper, Attention Is All You Need.
Larry:
That famous… yep.
Dan:
Yeah. But if you could answer that, then you start to understand, okay, I see. We personify. I see what the model is thinking about in this case. And then it makes a lot of sense from there, that it would lead to this answer. What if we did this with the prompting? Add extra tokens into its context that adjusts that. How does it adjust the attention with this particular query? Things like that. Or understanding how different layers of the transformer architecture are working at a given time. If you understand the system and how it works well enough, you can ask these questions at least. And if you have good enough data scientists, they can hopefully produce answers to them that are meaningful to you.
Larry:
And I just want to… because I think some people will never want to get as deep in the engineering as you are. But it seems like to me, from everything you just said, that just, as a good content designer, you’re going to point out these things. And your new collaborators, the engineers and the data scientists you’re working with, will be able to do the engineering and data acquisition and stuff that needs to be done to fix that.
Dan:
And I think they will be looking to you to ask the right questions. They will have to translate it perhaps into the technical terms, and you might have to translate, just the same way, your content insights and questions into their language. But I think it needs to be a balance and a collaboration. And I think there’s a lot of opportunity to offer this expertise where it’s missing. And I think technical people understand that that’s what they’re missing there, is this lens.
Larry:
This is reminding me of something I meant to ask you about earlier when you were talking about governance and tone of voice and that kind of stuff. That there are so many other people… there’s always been more people than just content folks doing content stuff. And so it seems like there’s an opportunity here for democratization of content practices of all kinds. Are there unique opportunities that come up with AI tech?
Dan:
For democratization?
Larry:
Yeah.
Dan:
Well, certainly we’ve seen democratization of content creation to some degree. It’s not going to be as good, and that’s where the human expertise remains incredibly important. But I think with writing skills, it’s also an accessibility tool in some ways. So if maybe English isn’t your first language, or maybe you’ve just never been a good writer. You didn’t, like me, get an MFA in poetry, using a tool like ChatGPT can open up a world to you through communication or through translation, both translation between languages and translation of sophisticated concepts into simpler ones and vice versa. So yeah, I do think it’s an incredible democratization that we should embrace with caution.
Larry:
Yeah. And you just reminded me. Dan, we’re coming up on time so we need to wrap up soon, but you just reminded me. There’s always a whole other thing I want to talk about, and you just opened the translation and localization rabbit hole. And we’ll have to go down that at some point but, hey, before we wrap up, is there anything last? Anything that you want to share before we sign off today?
Dan:
Yeah. Well, if there’s one thing I would emphasize at least, we touched on it a little bit, but this issue of bias and inclusivity I think is incredibly important. And I think it’s a question of inclusivity in content. So it requires ethicists, but also content people to keep their eye on the ball, to not cede the data work to people who aren’t imagining the data as content, which it is. And there are bigger things at stake in that conversation beyond just the democratization of writing or the selling of products, but actually treating fellow human beings fairly and right.
Larry:
Yeah, that’s my hope. There’s all this hope for machines to do all this stuff for us but I think we need to keep in mind that it’s always about people creating it, people consuming it. It’s all for our benefit.
Dan:
Yeah, definitely.
Larry:
Yeah. Oh, and one very last thing, Dan. If people want to stay in touch, what’s the best place to connect online?
Dan:
Yeah, you can find me on LinkedIn. Follow me there. I post from time to time about all of these topics. So yeah, I’d love to make some more friends.
Larry:
Excellent. Well, thank you so much, Dan. I really enjoyed the conversation.
Dan:
Thanks for having me on.
