We Can Definitely Get You First in ChatGPT ![rw-book-cover](https://wsrv.nl/?url=http%3A%2F%2Fwww.themshow.com%2Fi%2Fmoc.jpg&w=512&h=512) ## Metadata - Author: [[Marketing Over Coffee Marketing Podcast]] - Full Title: We Can Definitely Get You First in ChatGPT - Category: #podcasts - URL: https://share.snipd.com/episode/fdddc488-46d8-4897-bf24-f74919c0f1e0 ## Document Notes ## Highlights - **Chatbot Prompts Are Unobservable** - You cannot know what users type into ChatGPT because OpenAI doesn't release prompts and single-word changes alter outputs dramatically. - Trackable signals like AI overview presence and referral clicks exist, but conversational inputs remain opaque. Transcript: Christopher Penn And take this, even this with a grain of salt. Number one, no one can know what people are typing into chatbots like ChatGPT. OpenAI does not release that information, period, at all, except in like court-ordered cases, but no one can know. You cannot know what someone's typing in. And as we have seen with these tools in the past, even a one-word change in a prompt can deflect and change the results substantially. Of Ahrefs and SEMrush at the time, each shared, you know, the ChatGPT said that they were the top SEO tool. There was a one word difference in their two prompts. That was it. And they got different results that said that their company was on top. So you cannot know what someone's typing, particularly because these things are often conversational. So number one, anyone who says that they know what people type into ChatGPT or other similar services, they should, total, immediately disqualify them because they're lying. Second, you can know what searches prompt an AI overview in Google because that is a knowable thing, right? That is something that companies like Ahrefs do test. SEMrush tests that as well. They run thousands and thousands of keywords. They look to see if the AI overview box pops up, right? That's pretty straightforward. That is a knowable thing. The third thing, and this is where there is a lot of snake oil. If you look at the way that tools like ChatGPT and Gemini and Google AI mode and stuff work, they do a lot of what's called search grounding, which means that if they're not confident in Their answer, they kick off a web search. What does that mean? That means basic traditional SEO still works. And in fact, that's one of the best ways to feed these models is by doing a good job with the SEO you already do. So that when these tools start doing web searches, they're pulling from the right data sources. So if you suck at SEO, you're going to suck at the alphabet soup of whatever we're calling this thing. And there are a lot of folks in the search space who are relabeling their SEO services, whatever the acronym of choice is, adding a zero onto their prices and saying, yo, we can get you Optimized for AI. And because you can't know what people type in the chat, they can't prove or disprove that what they're doing is working or not. So there's definitely the cold-pressed snake oil version of that. But the bottom line is that people use these tools conversationally. You can't know what's in the conversations. The single best thing that every marketer should be doing to get you real data today is this. On your contact forms, ask people how they heard of you. If they never say chat GPT or Gemini or Claude, then you know you've got a problem because you do know people are using these tools. If people say, yeah, I've used, I asked to chat GPT, right? ([Time 0:01:04](https://share.snipd.com/snip/56dffc7d-21bf-40e9-b891-9d3ce3b81ab0)) - **SEO Still Feeds AI Answers** - Generative AI often performs search grounding and falls back to web search when unsure, so traditional SEO still matters. - Strong SEO improves chances that models pull your site as a trusted source during those web lookups. Transcript: Christopher Penn If you look at the way that tools like ChatGPT and Gemini and Google AI mode and stuff work, they do a lot of what's called search grounding, which means that if they're not confident in Their answer, they kick off a web search. What does that mean? That means basic traditional SEO still works. And in fact, that's one of the best ways to feed these models is by doing a good job with the SEO you already do. So that when these tools start doing web searches, they're pulling from the right data sources. So if you suck at SEO, you're going to suck at the ([Time 0:02:30](https://share.snipd.com/snip/7dda0ba1-14ec-4e71-8a1f-bf0be88da963)) - **Ask Users How They Heard Of You** - Add a free-form "How did you hear about us?" field on contact forms to capture if users reference ChatGPT, Gemini, or similar tools. - Analyze those free-text answers with AI to get real, actionable referral data instead of guessing from hype. Transcript: Christopher Penn The single best thing that every marketer should be doing to get you real data today is this. On your contact forms, ask people how they heard of you. If they never say chat GPT or Gemini or Claude, then you know you've got a problem because you do know people are using these tools. If people say, yeah, I've used, I asked to chat GPT, right? And do not have it be a dropdown box. Have it be a free form text field that people can write their answer in so that you get good information. John Wall And there's still, as far as I'm aware, there's nothing as far as like referral codes or whatever, like anybody that's linking out from all of these tools. You have no way to know if that's just a direct search or is there some tracking going on there? Christopher Penn There is there's referral domains. So you can see when someone clicks on a link in chat GPT dot com. If you've set up your Google Analytics, there's a free guide on the Trust Insights website, a PDF on how to set it up in Google Analytics for. So if they click out of Perplexity, they click out of Gemini, they click out of ChatGPT, you can see that traffic. You cannot see the conversation. You have no idea what was said, but you can see the click that came from ChatGPT arrive on your website. And, of course, you could see what page they landed on from that click. And that's one of the best click streams. But here's the thing. Most of the time, people don't do that. They will ask a question like, oh, I need a good marketing analytics firm. Who's good? And it'll come up with answers. And then what's the natural inclination that people do because they've done it for 25 years? They Google it. Go to browser, right? John Wall Okay. So they come up short on that. All right. So, well, you could at least though, get a picture of like, okay, which models are sending the most traffic. So you could get an idea at least of where it's coming from. And if it's not coming from anywhere, you realize that your problems are bigger than normal. You're doing your same SEO, which is not getting you anything, I guess. Christopher Penn Exactly. My big advice here is to make sure that you are asking people how they heard of you. And you are doing the work using AI of analyzing those freeform text responses. ([Time 0:03:39](https://share.snipd.com/snip/b3f93945-6f01-41a7-8fb4-c80119fb9c56)) - **Track Click Referrals From AI Tools** - Configure analytics to capture referral domains when users click out from AI tools like ChatGPT, Perplexity, or Gemini. - Use landing page data from those referrals to understand which content AI tools send users to. Transcript: Christopher Penn So you can see when someone clicks on a link in chat GPT dot com. If you've set up your Google Analytics, there's a free guide on the Trust Insights website, a PDF on how to set it up in Google Analytics for. So if they click out of Perplexity, they click out of Gemini, they click out of ChatGPT, you can see that traffic. You cannot see the conversation. You have no idea what was said, but you can see the click that came from ChatGPT arrive on your website. And, of course, you could see what page they landed on from that click. And that's one of the best click streams. ([Time 0:04:30](https://share.snipd.com/snip/a5f5c58e-a494-4acb-8bda-770f32710699)) - **LinkedIn Uses An LLM For Ranking** - LinkedIn replaced multiple ranking systems with a tuned LLM called 360 Brew that predicts user engagement in language-space. - The model ranks posts by likelihood a user will like or comment, making profile and post wording far more influential. Transcript: Christopher Penn So this is the second version of this paper this year. The first one was in May, which looked at all 14 of LinkedIn's major systems that create the news feed. And that was the first paper where we said, ignore the word the algorithm. There is no one algorithm. There's 14 separate interconnected systems. Seven of those systems have been roughly replaced by one big language model that in the paper that they document is called 360 Brew. And it is a large language model, generative AI, that has given a templated prompt with your data in it and repeatedly asked millions of times an hour, will this user like this post? Will they comment on this post? Will they be engaged with this post? It's trying to rank based on language whether or not something's going to appeal to you. This is a big change because what used to happen in the old system was the second pass ranker system was a series of very complex regression analysis to try and figure out whether the probability That you would like something. And now they move to a language model because once they weed out the obvious crap through the first pass ranker, which to my knowledge hasn't really changed a lot, that filters down to A set of candidates of the things that you're most likely should be put in front of you. Then it goes to the language model and says, rank order these things by whether the user is going to like them. It spits out results. That then goes back through the rest of the system, trust and safety and all that stuff to make sure it's not obviously a violation of the terms of service. And so this system, by the very nature of the language model is in development, they used Mixtrall 8x22, which is a well-known language model. It's their version, highly tuned, like they customize the hell out of this thing. And it tells us, because we know the architecture of the model, it tells us the things like recency do matter, but context. Context matters so much. The words that you use in your profile, the words that you use in your comments, the words that you use in your posts, not in abstract, but in the actual stuff that you write and post and Show up every day on LinkedIn, that is what the 360 brew model does at the heart of the new recommendation system. And so it is very, very powerful. It's pretty cool. And it also means the probability of you quote, hitting the algorithm is zero. It is now zero because it's a language model. A, it's already probabilistic in nature. But B, now with a language model under the hood, making the recommendations, it's going to be very, very difficult to get past the first pass ranker, get past the model, and then get past Trust and safety to come out on top. So that's what's changed. ([Time 0:07:15](https://share.snipd.com/snip/338ff133-ffcc-4290-a0ac-99480d17ced7)) - **Optimize Language On LinkedIn Daily** - Focus on consistent, contextual language in your LinkedIn profile, posts, and comments because the LLM ranks content by words and context. - Prioritize recency and relevant wording to increase the probability the model surfaces your content. Transcript: Christopher Penn And it tells us, because we know the architecture of the model, it tells us the things like recency do matter, but context. Context matters so much. The words that you use in your profile, the words that you use in your comments, the words that you use in your posts, not in abstract, but in the actual stuff that you write and post and Show up every day on LinkedIn, that is what the 360 brew model does at the heart of the new recommendation system. And so it is very, very powerful. ([Time 0:08:58](https://share.snipd.com/snip/85153609-fc27-48ef-bb26-c5006b4978d2)) - **AirPods Pro 3 Exceeded Expectations** - Christopher Penn reports Apple AirPods Pro 3 deliver significantly improved noise cancellation and comfort due to foam-encapsulated silicone tips. - He prefers them to Bose for comfort and praises eight-hour battery life and heart-rate features. Transcript: Christopher Penn They are incredible. I would say that Apple's claim that the noise cancellation is twice as good as the AirPods 2 is accurate, mostly because of the new tips. The ear tips are foam encapsulated in silicone. So the foam's not going to rot out or whatever because it's fully encapsulated. And then the silicone tips make it comfortable. The foam tips create a really, really solid seal. That's two things. Number one, it's more passive noise cancellation, which is good. And two, they stay in, which is even better. And I love them. I wear them literally from the moment I sit down at my desk to the moment I get up at my desk. And I'll put them on the charger at lunchtime while I'm eating. But the battery life is about eight hours on them. And it has the heart rate monitor in it. So if you want to do a workout and you didn't want to wear your Apple Watch or you didn't have it with you, you could still get your heart rate monitoring and stuff that way. But I love these things. I mean, they live up to their billing. I would say on a qualitative basis, and now this is a 50-year dude's subjective impression, I think the noise cancellation on these is as good or better than the Bose in your earbuds. ([Time 0:11:06](https://share.snipd.com/snip/4f98dc91-f5e7-44a1-b82e-653ca1eaa66d)) - **Models Evolve To Manage Context Better** - New model releases continue the arms race: DeepSeek cut costs with a new attention mechanism and Anthropic released Claude Sonnet 4.5 with 'context editing.' - Context editing compresses earlier chat history to maintain coherence and improve long-session performance. Transcript: Christopher Penn Ai models uh this week alone deep seek the chinese model maker released deep seek version 3.2 that supposedly has a brand new attention mechanism that allows them to keep or improve Accuracy while cutting costs in half. And they're already silly cheap. So now they're like stupid cheap. You will spend more on a day's worth of high intensity coding with DeepSeq. You will spend more on a large fry at McDonald's than you will on their API, which is pretty incredible. Now, granted, obviously, if you're using their API, it is subject to the data privacy rules of the People's Republic of China, aka none. So keep that in mind. But the big news in the last 24 hours, as time of recording this, is Anthropic released its newest model, Anthropic Claude Sonnet 4.5. This is a new model. It's their cheaper model. So it's not Opus, it's Sonnet, but it is the best coding model ever released according to their benchmarks, which of course, everyone, you know, take that with a grain of salt. But it incorporates a couple of very, very interesting features, one of which is called context editing. Every AI model, and we've all experienced this, within the same chat, within a single chat, if you keep a chat going long enough, it eventually loses cohesion, right? It starts getting incoherent, starts saying stupid stuff, forgets details. It's because the short-term memory of a chat gets full. Every chat has a short-term memory. Well, what Anthropic has done is they have enabled the model and the architecture around it to say, hey, you know what? My memory is getting full. I'm going to start compressing the first half of the chat, the stuff that we're no longer talking about. And, you know, either deleting stuff, deleting like, and this is specifically a lot of ways to coding, like deleting, you know results and test results and stuff and things like that. But selectively basically compressing the first half of the chat to free up room, which if you are of a certain age like me, where you've got a lot of gray hair, you probably remember back In the bad old days in the 1990s when you had to do stuff to your computer called defrag your hard drive, where you had to run utilities to rearrange and compress the memory and stuff. That's basically what this is. Everything old is new again, except now the AI knows to do it itself to make it more efficient and to improve the correctness of its answers. And as a result, Cloud 4.5, Asana 4.5 has been out for about 24 hours now, as of the time of this recording. And everyone who is using this saying this is a significant improvement over everything, including over OpenAI's new coding model, GPT-5 Codex, which came out, what, two weeks ago Now? It's just much smarter because it's being more efficient with its own memory. And so Anthropix has done a lot of really, really cool stuff under the hood with this one to make it work. And we've gone out to all of our clients and saying, hey, for your Claude subscriptions, now is the time to make sure everything is updated and stuff so that you're using the new model. It's really freaking smart, particularly for coding. It's still good in other areas, but the changes, the fact that they now have, they've upgraded their coding utilities. ([Time 0:17:50](https://share.snipd.com/snip/3c6dcbe3-3a29-49c4-bb07-d0ddb244e25b)) - **Use Claude 4.5 For Overnight Coding Tasks** - Upgrade to Claude Sonnet 4.5 for improved coding assistance and use supervisor utilities to run long background coding tasks. - Delegate repetitive or junior developer tasks to the model and verify test results when you return. Transcript: Christopher Penn And as a result, Cloud 4.5, Asana 4.5 has been out for about 24 hours now, as of the time of this recording. And everyone who is using this saying this is a significant improvement over everything, including over OpenAI's new coding model, GPT-5 Codex, which came out, what, two weeks ago Now? It's just much smarter because it's being more efficient with its own memory. And so Anthropix has done a lot of really, really cool stuff under the hood with this one to make it work. And we've gone out to all of our clients and saying, hey, for your Claude subscriptions, now is the time to make sure everything is updated and stuff so that you're using the new model. It's really freaking smart, particularly for coding. It's still good in other areas, but the changes, the fact that they now have, they've upgraded their coding utilities. Like I've got a supervisor utility running in the background that is like, I went to bed last night with a particularly thorny coding problem. I said, go work on this thing while I'm asleep. Here's all the permissions, all the things you're allowed to do. And when I woke up this morning and said, hey, I'm done, you know, test it out. I ran the test, the test come out clean, you test it and make sure it's clean. I tested it like, yep, it's done. ([Time 0:20:12](https://share.snipd.com/snip/0b7e86e0-5e14-4fa0-a3fa-0e71039ab749))