Podcast: Play in new window | Download
Subscribe: Apple Podcasts | Spotify | Android | TuneIn | RSS
Andy Crestodina has been developing high-quality content for his business customers at Orbit Media for more than 20 years.
As they have incorporated AI into their workflows at the agency, Andy has discovered that the best use of these new tools is to improve the quality of their content and service offerings rather than simply doing more.
We talked about:
- his work as co-founder and CMO at Orbit Media
- how he uses AI to do audience research, develop personas, and address their needs through gap analysis
- his playbook for querying and validating information that AI generates for him: prompt, response, edit
- how they manage prompts at Orbit Media
- how their business operations are evolving to incorporate AI practices into their operations
- how he uses AI in his marketing analytics
- how the comprehensiveness of coverage that AI brings to his content helps with conversion
- the essential skills that persuasion copywriters need to develop to work effectively with AI
- his concern that some LLMs may be getting worse, not better
- the importance of setting aside a focus on how to be faster and instead focus on how to be better – to focus on quality over quantity of content
Andy’s bio
Andy Crestodina is the co-founder and Chief Marketing Officer of Orbit Media, an award-winning 50-person digital agency in Chicago. Over the past 23 years, Andy has provided digital marketing advice to 1000+ businesses.
Andy has written 500+ articles on content strategy, search engine optimization, visitor psychology, analytics and most recently, AI. These articles reach more than three million readers each year. He’s also the author of Content Chemistry: The Illustrated Handbook for Content Marketing.
Andy gives up to 100 webinars and presentations per year and is a frequent repeat speaker at many of the top national marketing conferences.
Connect with Andy online
Video
Here’s the video version of our conversation:
Podcast intro transcript
This is the Content and AI podcast, episode number 18. Creating content that gets found by Google and then persuades potential customers to act is a core competency for modern marketers. Andy Crestodina and his colleagues at Orbit Media, the agency he co-founded 20 years ago, have built websites for hundreds of businesses and created content for them that helps turn their prospects into customers. Andy uses AI extensively in his work now. His top finding? Focus on how AI can improve the quality of your work, not just your productivity.
Interview transcript
Larry:
Hey everyone. Welcome to episode number 18 of the Content and AI podcast. I am really happy today to welcome to the show Andy Crestodina. Andy is the co-founder and CMO at Orbit Media. It’s a Chicago based agency that does website development and a lot of other marketing stuff. So welcome, Andy. Tell the folks a little bit more about what you’re doing these days.
Andy:
Sure, Larry. Well, thanks for having me. 20 however many years ago, 2001, co-founded an agency. An agency totally focused on the website itself. So we build sites and we improve them forever after doing optimization work, both search and conversion optimization. It’s a 55-person firm that we grew strictly from organic and content marketing. So I’m someone that you might see if you go to an event like Content Marketing World, Social Media Marketing World, MozCon. I speak at a lot of search events and analytics events, but I’m an old-school content marketer who’s built an agency focused on websites.
Larry:
Nice. Yeah. We must have run into each other somewhere along the line because I ran in that world a lot back around that same time, the early 2000s. But one of the things that’s so interesting … So we’ve seen this evolution together and seen a lot of the … There’s always been a lot of drudgery associated with our work and a lot of intellectual work involved and you’re really excited about, and have experimented as much with, the new AI tools. Let’s start with the customer and the audience analysis stuff that you do. That was really interesting to me when I read about that. Can you talk a little bit about that?
Andy:
Sure. If you write a prompt that says draft a blog post of 2,000 words that talks about supply chain … You’re going to get something pretty boring. It’s going to be inherently undifferentiated. I joke that AI stands for average information. AI ate the internet. Literally the Common Crawl is 85% of the internet. We know that ChatGPT was trained on the Common Crawl. And it comes back and just gives you vanilla. It tastes like water. Of course it’s boring. It’s not for anybody. It’s generic. All you did was say, write me a blog post. So all of my most successful adventures in AI, and these are daily, begin by teaching it or training it on your target audience. Now, if you’ve got battle tested, ideal client profiles or marketing personas, you can upload them and it will read them.
Andy:
You can also write prompts that will do it. Create the persona of a job title within an industry, at a company size, in a geography with an objective and a challenge and then tell me their hopes and dreams, tell me their pain points, their frustrations, their fears, and their decision criteria for selecting a company like mine. It’s going to write you a persona. It will be incorrect. Of course, AI is not accurate. Don’t ever expect the AI to be accurate. Go fix it and prove it, validate. And now once you’ve improved that persona and you believe in it and it looks good, now ask it to write an article for that persona or to draft an outline or to write a headline or to write a social post or to research a keyword or to suggest an influencer, or to do whatever it is you want to do. It’s going to be far, far better results if you began that conversation with AI by teaching it who you’re talking to. It’s absurd for marketers to believe that they’re going to get any good response without focusing on the audience. It’s weird, right? I think it’s weird.
Larry:
It is. And it’s like that thing that anybody … I’m sure you’ve done a lot of presenting over the years. You always analyze your audience and figure out what they want first. What you’re saying reminds me that so much of the fuss the last year plus since ChatGPT-3 was introduced, has been about the generative capabilities of AI. We were talking a little bit before we went on the air and you were like, “Nah, it’s not about efficiency and time saving. It’s more about better stuff.” And like you just said, it sounds like you’re getting … How do you feel about the personas that you’re developing now compared to what you’re doing before you had ChatGPT to query?
Andy:
Well, if I’m on a conversation with a client or a prospect or a friend and they say, “Hey, check out this thing. Is this good?” I’ve done digital strategy forever and I’ve been part of the planning process for more than a thousand projects and I’m an SEO and a conversion person and persuasion copywriting nerd. So people are frequently asking me to evaluate something they made. But it’s really hard for me to quickly understand their audience. So instead, if I just begin with a persona prompt and together with the person I’m talking to, we get to where we believe, yeah, that is my buyer. Yeah, that looks like them. That feels right. Okay, good. Now I’m going to copy and paste in that thing that you wanted me to review and I’m going to have the AI tell me what it’s missing.
Andy:
AI powered, persona driven gap analysis on any page on your website. It will immediately tell you you failed to meet your audience’s information needs, or you did not address an important objection, or there’s a critical unanswered question with your persona after reading this copy. Those are things it’s very hard for a human brain to do. To look at something and say, what’s not there. Human brains are simply not good at doing that. AI is amazing at doing that, but only if you train it on the audience first. At that point, now everyone has a sense for it. And do we agree? Do we just automatically assume it’s correct? No. Here I joke, AI stands for another input. Earlier I jokingly said, AI stands for average information, which is what it does. If you just ask for general prompts, you’re lazy prompting, it’s going to give you back average information.
Andy:
But now I’m actually using it as a mini-consultant or research assistant or once I trained it on the persona, giving it a piece of copywriting and it’s giving me another perspective. Do I have to take it? No. Is it useful? Maybe. But it was a fast exercise that put me in the mindset of my audience and it’s got me looking at a key piece of content, a sales piece, a service page or a product page or a sales page. That exercise, sure, it is fast. I don’t love it because it’s fast. I love it because it’s going to help me generate more leads. But that exercise is 15 minutes, less, and the improvement that you might make from that … Dammit. I forgot to mention this important thing. Will be a durable improvement. Go fix your homepage and it’ll be a better homepage for the next 10,000 visitors you’ll have over the lifespan of that page. To me, this has been the most successful use of AI. Starts with the persona, give it a piece of copy, have it do gap analysis, and then just take it or leave it. But you have an opportunity now to do better, you could say, just conversion copywriting.
Larry:
Yeah. You’re reminding me now that the common and ubiquitous modern affliction is attention when none of us have it anymore. We’re incapable of paying attention for long periods of time. But these machines are just like, once they know what they’re looking for, they’re just on the job. They’re not picking up their phone and looking at it. So their attention to detail, I get that. One thing I want to do … And I want to preface this by saying that I do pretty much what you do in a lot of my persona development work these days. But I think of people like Debbie Levitt. I don’t know if you know her. The UX expert who talks a lot about researching personas and other kinds of user research. And I have friends in the service design community who make similar critiques that you really need to talk to human beings too.
Andy:
Oh yeah.
Larry:
Tell me about the balance because I’m sure you still do. But what’s the balance nowadays between … Or what’s the interplay between the humans and the AI gadgets that you’re working with?
Andy:
Well, prompt, response, edit. That’s the playbook. So you’re going to ask it for something, it’s going to give you something back, and then you’re going to look at that and you’re going to improve it. We never just assume that it’s always accurate. Ardath Albee is another persona expert, and after I started working with this mini framework we just described, I wrote to Ardath. I’m like, “Ardath, I’m going to share my screen and I want you to poke holes in this mini-process here.” I ran through it and she said, “Andy, I don’t know if that’s right. Is that your persona? Is that your competitor’s persona? I don’t trust this.” And that is the point. Do not trust AI. It’s a crazy idea. Just take it for granted. Why would it know?
Andy:
So everything it does … If you have it write headlines, if you have it write drafts, if you have it create images, if you have it write social posts or email copy or YouTube descriptions or transcribing podcasts, whatever it is, you have to look at it. You’re the editor. It would be very risky to assume that it’s going to be correct. So yeah, that interplay, that persona needs to be reviewed and improved upon by someone who’s on the front lines. If you’re not doing that, you may still find insights and your expectations may not be high. Doing quick gap analysis, it’ll probably be fine. But if you do have a really, really good persona built into AI, your opportunities are many. You can run your sales groups against it. You can ask it why it doesn’t buy. You can ask it how much money it would spend. You can ask it who else is involved in the buying decision? You can do lots of sales playbook role playing with your synthetic audience member to your synthetic prospect, but everything is garbage in, garbage out. It’s all about the quality of the input.
Larry:
As you’re talking about that, I’ve thought a lot about this lately. You mentioned the playbook. That you have the things you routinely do to do this. And a lot of that I’m assuming is prompts that you’ve written or custom GPTs or things like that. How do you document and … It occurred to me the other day that prompts are just another kind of content, and I’m a content management nerd, and I’m like, how are we going to manage our prompts? Tell me how you’re managing the information you’re working with.
Andy:
Yeah. This has been about a year and from the very beginning, I was just looking at it thinking we all need … Where’s the shared prompt library AI startup? And there are some. OpenAI. If you’re using ChatGPT, OpenAI is filling in the blank here gradually. There’s teams mode. You can archive these now. But what a lot of people are doing in the interim is just using a Google Doc. I have a folder in Google Drive or G Suite. Doesn’t matter what you use. Just a shared doc. We call ours the Codex. A fun term. I sometimes think of it as a spell book because it turns anybody into a sorcerer. Some of these things feel like magic. But there are levels of prompts in there. Some that we have vetted and know to be useful. Others are still experimental.
Andy:
Every now and then someone posts some interesting prompts, which I store or keep. But I don’t encourage anyone else to use them until I’ve played with them enough myself and improved upon them myself. Constantly editing those. So you’ll have a content manager who has the authority to improve or edit prompts. Other people may just be going and grabbing them there. But no, you need some kind of document. Larry, I’m sure you’ve got a ton of ideas, right? Notion, Evernote, G Doc, whatever. But yeah, the shared prompt library is a key feature for collaboration, and it gets to this other topic of like, are people going to turn it inside out? Are you going to share all these with the world? Are these going to become private and confidential? You can copyright your prompts. It’s interesting.
Larry:
You just mentioned collaboration. And now I’m picturing we have a bunch of different … A friend has likened these AI gadgets to we all have 10,000 interns now. And we have bigger, more diverse teams. And like you said, they don’t forget stuff. They’re on the case all the time. But you still have your human. You’ve got 55 people working there at Orbit Media. Tell me, what does the office look like, the interactions and how are you welcoming your AI colleagues?
Andy:
Well, there is a little working group of people that discuss this. The managers have time and agenda items where people are sharing things on their team. But I think you can separate AI use cases into two groups. There are things that will fundamentally change standard operating procedures. This is how we used to do it. Here’s how we’re doing it now. We got another input. We’re going to validate this copy with AI. We’re going to run these pages through AI. It’s a double check on something and that becomes an official task that you use every time. But there’s a million little things in AI, as we all know, that you use just to write a better email or to trigger some ideas.
Andy:
The smart manager in the context of innovation encourages their team to do experimentation, encourages cross-functional collaboration conversation. And then when you know something to be effective, that’s when you need to train. Companies that are managed very closely, it’s difficult for them to just say, here’s a tool, go play. But actually it’s an important phase at this stage. These are the early days. This is breakfast. We all just woke up and we’re all trying something new. So I’d encourage everyone to play around. I would pony up the 20 bucks for ChatGPT Plus or whichever tool you like. But don’t spend the 500 bucks a month until you’ve really vetted a process and documented a new SOP and decided that’s going to be how we do it every time. Maybe we need an enterprise-level tool.
Larry:
As you’re talking about that, I realize that you’re operationalizing this. It’s just sort of an addition to your ongoing regular content operation. But when you mentioned training too, that kind of gets into ensconcing it in culture and how people operate. Have you got to the point, do you have multiple training programs at this point to bring people up to speed?
Andy:
Not really. We’re using it for certain projects, and I don’t have enough data yet to confirm that it’s producing higher performing copy. We’re not using AI to write anything for clients. We’re using AI to validate copy and provide another input. To give us insights into did this copy miss anything? So it’s very useful to bring to a meeting and to say, “AI device its own metric and this page got a 95.” Whatever. You could do all these kinds of things. But after we know that this is going to work, these client projects feel like pilot programs. But no, it’ll become standard. Of course, you run that. It’s like spell check. Why wouldn’t you do that? And it won’t seem weird, and no one’s going to be intimidated. It’s just part of your day. Of course, you run it through there and that’s how we create things that are more comprehensive and higher performing.
Larry:
And when you’re talking about operationalizing these new SOPs and stuff, it kind of hints at, there’s a lot of analysis and insight discovery and then like you said, the validation of things, whether they work or not. Are you using AI for that stuff as well?
Andy:
For analysis?
Larry:
Yeah, for analysis. Yeah.
Andy:
I do that personally and I do it often, but we have a tiny micro marketing team. It’s like me and one other person. But hey, I want to know what’s working on LinkedIn. I put all the data I could get from LinkedIn into a spreadsheet, tag the topics of all these posts, uploaded it to AI, asked to edifying correlations between topics and new followers or engagement or impressions. Export all your data from YouTube Studio. Ask it to find correlations between the day you publish or the length of the piece and the engagement or new subscribers. Export a data from GA-4, from your email service provider or whatever.
Andy:
I would need a data scientist or days of work to try to find the correlation between … I mean, if you upload all of your GA-4 data and have AI combine it with your email service provider data, you can have AI produce for you a full funnel report that shows deliverability, open rate, click-through rate. Now I’m on the website. Bounce rate, engagement rate, conversion rate, average engagement time. Probably no one listening to this has ever seen a full funnel email marketing report that goes from the delivery of the email all the way through to the thank you page conversion. AI made it for me. How did I do that? I just uploaded the spreadsheets and kept prompting it until I was confident in the data and then had it draw a chart. It draws charts. It’s making visuals. A heat map. Make a heat map matrix showing me the correlation between different elements of content and its performance against these six metrics. Beautiful.
Larry:
Right. I haven’t talked to people doing this. Because that’s the new multimodal capabilities of the newer GPTs that they can create images from text prompts and vice versa. That’s really interesting. But you said that there’s just a couple of you, it sounds like at Orbit working with that at this point. I’m also wondering, are there vendors who are starting to do that kind of thing?
Andy:
Yeah. Yeah. AI does a beautiful job of visualizing data. I call these “long shot prompts.” Just for fun, you can export a spreadsheet from your favorite tool. Probably clean it up a little bit first. I think it AI can clean it up, but you’ll get better results if you clean it up a bit yourself. Upload it to AI and try this prompt. Create for me an unusual visualization showing an unexpected insight. It’s going to do something cool. It’s going to show you something weird. It might be useful. If it’s not, so what. You wasted three minutes of your life. But it’ll give you a sense for what’s possible. No, AI draws charts. It does so quite well. The heat map matrix. Make a heat map matrix of the degree to which this webpage copy meets or does not meet the information needs of the persona I trained you on. It’ll make a heat map matrix showing little colored boxes. Oh, you didn’t talk about this. Or give it you and four competitors. Create for me a heat map matrix showing the extent to which me and all these websites meet or does not meet the persona’s information needs. So yeah, no, those are useful, especially if you’re going to a meeting. Charts are persuasive. The internet’s visual. People are persuaded by visuals.
Larry:
And the way you described that, that’s just another … It’s like you have a whole bunch of new gap analysis tools, like gaps between the content your customers need. It’s just getting more and more fine-grained all the time. Is that kind of how it feels? That you’re more comprehensively serving folks?
Andy:
Yeah. But what you’re mostly looking for if you’re doing conversion copywriting is a big miss. It’s like I give a presentation where I just picked a sample industry like spacecraft launch services and looked at this page and which of my personas most important information needs are not addressed. It’s like, well, this page didn’t mention insurance or risk analysis. Oops. The commercial satellite operator cares about risk. So if the page doesn’t mention that it’s less likely to convert. Every unmet information need hurts your conversion rates. That’s the game, right? That’s the job. They came for information. No one came to see how pretty it is. They came to get questions answered. That’s our philosophy for conversion. Clarity and evidence. Which is another fun prompt. You can actually ask the AI to give you a list of all of the unsupported claims. Marketing claims can be supported by testimonials, reviews, years in business, data statistics, case studies, success stories, awards, certifications, memberships. Give me a list of all the unsupported marketing claims on this page, and it will give you a list of all the claims you made that you did not support with evidence. So that’s answer, evidence, answer, evidence, call to action. That’s still the formula for conversion. AI is just a very useful tool at helping you get to a higher converting page faster.
Larry:
And as you mentioned that, I can picture when you’re the human doing the writing, you’re trying to dredge up an example like that and you just prompt ChatGPT that list you just said, awards and all those other things, and it’s going to come up with probably 10 times as many as you might have on your own. Again, back to that, the tasks that the humans can’t do. Well, what are you finding is the key human capability? What do you think the last thing that AIs will do for us is, I guess?
Andy:
It’s the perfect question, Larry. And when you think about it, what I just named, I just verbally spoke it, but that prompt of giving the AI the list of all those, that actually is the skill. If you’re a persuasion copywriter, you’re going to write better prompts to get persuasion copywriting out of AI than a junior generalist marketer. The skill is understanding what to ask the AI for. It’s like analyze this URL and give me ideas for how to add urgency to calls to action. Some people don’t know that they should be looking for opportunities to add urgency to call to action. So what we still need is digital strategists who can identify what the success criteria are, what the inputs are, and then use AI to find the gaps in those inputs. The detailed prompts I use for SEO, a normal person wouldn’t write those. If you’d done 25 years of SEO, you would.
Andy:
I’m writing prompts that probably most people wouldn’t write. They’re sometimes 200 words. But that’s the skill actually is knowing … It’s like a skilled manager working with a junior person. It’s the same thing. The skill of the manager determines the output of that person. The skill of the prompter determines the output of the AI. So knowing how to prompt it, I think that’s … AI can help you write better prompts and you can keep asking it, but there are some brilliant people out there who are writing prompts and getting things from AI that … Talking to AI about cognitive biases. Not everyone talks about cognitive biases. They’re critically important, but only the more veteran senior digital strategists think in that way so those are the people that write better prompts.
Larry:
Interesting. And as you talk about that, I’m picturing the way you work with a mentee or a junior person. So your prompt writing skills are like your management skills for managing this new army of interns that you have.
Andy:
Yeah. Same thing. And the senior manager doesn’t expect to get much output from a 12-word instruction. Why do people expect to get a good response out of a 12-word prompt? You have to keep training your mentee. So the way the best managers keep giving feedback. So you need to save these prompts. Keep telling it how to improve the persona. If you edited the draft, give it back to the AI, tell it to learn from your edits. You keep going back and forth, but the pro digital marketers who are also embracing change are getting exponentially greater results from AI than people who are just writing short, simple prompts without first training the AI on the audience.
Larry:
As you’re talking about that, you mentioned the prompt library and then how you interact with them and the training that goes into this, it’s all coming together. It’s this new management challenge that we all have. We’re about 15 months since ChatGPT-3 was introduced. Sometimes as we talk about this stuff we’ve just been talking about, I worry that these are learning models that are doing machine learning in the background to do this stuff. Have you found them starting to pick up any of these … Are there any slightly more advanced skills that the AIs are able to do since a year and three months ago when this all started?
Andy:
The changes to the AI … Actually, there’s some fear that the AI is getting worse, and there’s some evidence that it’s getting a little bit dumber, I’ve heard. I haven’t seen the research. But mostly the changes I see to AI are the safety AI elements. They’re putting in more guardrails. It’s like a few months ago, you could just say, draw me a picture in the style of Picasso and DALL-E through ChatGPT would do it. It doesn’t do that anymore. So it’s getting more conservative and more cautious, I think. I welcome that. That doesn’t bother me. But no, what we’re seeing now … We saw 2023 was a lot about the specific models getting very good, and now you’re going to see the models get combined or the multimodal models will get combined. So the way that we upload data, have it analyzed, visualize the data or the transcription tool is happy to have it make an avatar for you and show you’re facing the transcription. All these different things. I think the models are going to combine and that the browse feature will get far better, the research abilities, the APIs to existing tools, and then suddenly AI is going to be built into everything we do, and we’re probably going to … It’s like cloud or internet. It’s going to be standard.
Larry:
Right. It’ll be everywhere. Every form field in the world now has auto complete and there’ll be something analogous.
Andy:
Yeah, same. Good example. I agree.
Larry:
Hey, Andy, I can’t believe it. We’re coming up close to time already, but before we wrap up, is there anything last, anything that you want to revisit from the conversation or you just want to make sure we talk about before we wrap?
Andy:
Well, just to reiterate, our company builds websites and the context of that has been … It keeps our focus on quality. I think the people that are getting the best results from AI are literally budgeting time for research and experimentation and reading. They’re not getting productivity gains, at least not this year. That’s maybe later. Set aside the I want to be faster and focus on, I want to be better. Focus on the quality of your deliverables. If you do that, AI will save you time in the long run. It may not be right away, but that’s fine because our job on the web is to make something that’s 10X what our competitors are doing. That’s where you win. Make something that’s memorable. Make something that exceeds the expectation of the visitor. Make something that’s highly differentiated, and AI can be useful in that, but not of our whole goal is to save 10 minutes. I don’t see that as being the biggest benefit of AI.
Larry:
Well, yeah, and you’ve said a bunch of stuff today that points to the bigger benefits of just changing how you operate rather than trimming 10 minutes here and there from your work.
Andy:
Exactly.
Larry:
One very last thing, Andy. If folks want to connect with you or follow you online, what’s the best way to find you?
Andy:
Orbitmedia.com is where I post first. I write an article once every two weeks. I’ve done it for 15 years, and you can find my best there. Lots of articles about AI on the blog at orbitmedia.com. LinkedIn is my best social network. Anyone is welcome to connect with me on LinkedIn. Just skip the blue follow button if you see that. Click the three dots. You can go to the connect button. Feel free to connect with me there. And then we’re also active on YouTube, so Orbit Media also has a YouTube channel where there’s a lot of my best advice.
Larry:
Excellent. Well, I’ll put all that in the show notes as well. Well, thank you so much, Andy. I really enjoyed the conversation.
Andy:
Fun stuff. I did too, Larry. Thanks for having me on.
Larry:
My pleasure.
