So What? How to Do Prompt Engineering  ## Metadata - Author: [[Christopher Penn]] - Full Title: So What? How to Do Prompt Engineering - Category: #articles - Document Note: --- tags: - #promptengineering - #marketinganalytics - #generativeai Author: Christopher Penn (with Katie Robbert and John Wall) Source: YouTube video (So What? How to do prompt engineering) Reference: https://www.youtube.com/watch?v=nLLg3pxB_0k date created: 2025-08-22 00:00 --- # Title: So What? How to do prompt engineering ## Table of Contents: - [📝 Summary](#-summary) - [📖 Main Content](#-main-content) - [🗒️ Notes](#️-notes) - [📖 References](#-references-1) - [🗣️ Paraphrase](#-paraphrase) - [📰 Newsletter](#-newsletter) - [✅ Action Items (Advocacy-aligned prompt)](#-action-items) - [🧪 Hypotheses](#-hypotheses) - [📊 Methodology](#-methodology) - [❓ Questions](#-questions) - [🔮 Future Research](#-future-research) - [💡 Implications](#-implications) - [🧠 Learning Styles Assessment](#-learning-styles-assessment) - [📊 Accessibility Assessment](#-accessibility-assessment) - [🌍 DEIB Assessment](#-deib-deib-assessment) - [✅ Logical validity](#-logical-validity) - [🔍 DARVO Analysis](#-darvo-analysis) - [🔍 SEO Evaluation](#-seo-evaluation) - [🎯 Conclusion](#-conclusion) ### References: - Trust Insights resources: newsletter, podcast, Slack community mentioned throughout, e.g., closing callouts at the end [96–100%]. - Podcast mention: “This Week in Machine Learning & AI” and Allen Institute discussion on NLP context [3–6%]. ## 📝 Summary: The episode demystifies prompt engineering for generative AI (ChatGPT, Stable Diffusion), emphasizing specificity, context, and iterative refinement. Christopher Penn explains large language models using the principle “a word is known by the company it keeps” and demonstrates how to move from “party tricks” to production by integrating prompts into workflows and APIs. The team shows practical use cases: social copy generation, reformatting transcripts, crafting sales emails from CRM notes, readability targeting, creating quizzes, and building a “master prompt” through iteration. They stress that prompts should read like instructions to a human assistant, include persona, purpose, format, constraints, tone, and success criteria; outputs still require human review. They caution that research-preview tools will become paid, the underlying models will change, and real ROI appears when integrated into production systems. ## 📖 Main Content: - Big idea: Prompt engineering is structured delegation. Treat the model like a junior teammate: define persona, purpose, audience, tone, constraints, and desired format; iterate; then consolidate into a reusable master prompt. - Conceptual foundation: - LLMs map associations: “a word is known by the company it keeps” [3–6%]. This underpins why more context yields better outputs. - Specificity controls probability space; repetition adds “weight” to desired attributes (e.g., “photorealistic, highly detailed, 8K” in image prompts) [49–53%]. - From hacks to production: - “Party trick” phase vs integrating via APIs for repeatability and scale [34–44%]. - Token economics: ~1,000 tokens ≈ $0.01; costs scale in production [40–44%]. - Practical patterns (live demos): - Social tweets: start with weak prompt; add persona, show memory use, add intent (listen vs subscribe), include URLs, and refine tone [12–23%]. - Instruction clarity: write as if delegating to a VA; set readability (e.g., fourth-grade), bullet formats, and number of items [46–52%]. - Quiz generation and weighting: using precise phrasing like “series of three multiple choice questions” to reduce ambiguity [49–53%]. - CRM win-back emails: transform call notes into tailored outreach; still requires refinement and guardrails to avoid awkward content or placeholders [72–82%]. - Transformation framework: take a transcript and generate TikTok/Instagram ideas, show reuse and cross-channel ideation [62–70%]. - Rewriting: polishing transcripts for grammar, clarity, tone; re-spinning bios for LinkedIn [84–92%]. - Governance and change: - Human-in-the-loop editing is essential; models are fallible like humans [82–85%]. - Model updates (e.g., GPT-4) change behavior; track and test prompts over time [85–90%]. - Research preview will become paid; plan migration to production tools [92–97%]. ### 🗒️ Notes - Foundations and myths [0–8%]: - LLMs aren’t sentient; they produce outputs from statistical associations. - The “company words keep” explains coherence and style adaptation. - Prompt engineering principles [8–20%]: - Specificity is everything; vague prompts yield generic outputs. - Use persona setup: “You are a social media manager…” to shape voice and outputs. - Iterative refinement: build to a master prompt with all constraints. - User story mapping [14–18%]: - As a [persona], I want [task], so that [outcome]. Use it to structure prompts. - Iteration and memory [16–23%]: - Chat-based memory supports incremental constraints (URL inclusion, intent). - Turn conversation into a consolidated, reusable prompt. - From novelty to utility [30–39%]: - Party tricks vs production: APIs, integration, and reproducibility matter. - Token costs small at human scale, significant at product scale. - Practical use cases [44–76%]: - Summarization with length/readability constraints. - Quiz question generation with precise instruction. - Campaign ideation for TikTok/Instagram based on long-form content. - CRM-driven win-back emails from call notes (needs polishing, placeholders). - Delegation analogy [53–60%]: - If unclear inputs yield poor work, it’s on the manager; same with prompts. - Engineering patterns [60–70%]: - Persona + channel constraints (Twitter/TikTok/Instagram). - “Reply ‘ready’…” to stage interactive sessions. - Pipe data from any source into the LLM via code for automation. - Editing and QA [80–90%]: - Always review: check tone, personalization, placeholders, claims. - Track model versioning to prevent silent behavior drift. - Closing cautions [92–100%]: - Research preview will end; plan for cost and integration. - Real gains happen when prompts become embedded in tools and workflows. ## 📖 References: - So What? How to do prompt engineering (YouTube): https://www.youtube.com/watch?v=nLLg3pxB_0k - Trust Insights resources: - Newsletter: https://www.trustinsights.ai/resources/data-in-the-headlights-newsletter/ - YouTube: https://www.trustinsights.ai/youtube - Podcast: https://www.trustinsights.ai/resources/in-ear-insights-the-trust-insights-podcast/ - Slack: https://www.trustinsights.ai/analyticsformarketers - Mentioned podcast: “This Week in Machine Learning & AI” (TWiML) ## 🗣️ Paraphrase: 1) Original idea: “A word is known by the company it keeps” [3–6%]. Paraphrase: In LLMs, a term’s meaning is inferred from the words that routinely surround it. By feeding in ample, relevant context, you steer the model to associations that match your target domain, producing more accurate, on-style outputs. 2) Original idea: From party tricks to production with APIs [34–44%]. Paraphrase: The chat interface is a great R&D sandbox, but real value emerges when you codify your refined prompts and wire them to data sources through APIs. That enables reproducible, scalable outcomes—and you can budget via token costs as usage grows. ## 📰 Newsletter: Learning web development is a lot like prompt engineering—you get the best results when you’re specific, iterative, and a little bit opinionated. This week, we’re diving into smarter content workflows with generative AI, fresh releases, and community wins. - Learn: Turn transcripts into TikTok scripts, blog summaries, and newsletter blurbs with prompts that define persona, audience, format, tone, and constraints. Think “junior teammate” instructions, not magic spells. - Updates: WordPress 6.x performance improvements continue to land across core and block themes. Keep an eye on Gutenberg’s design tools and Patterns API refinements. - Community: Big kudos to first-time contributors in Core and Plugins last week—onboarding sessions are rolling in Slack; jump in and say hi! - New Plugins: Check out block-based A/B testing utilities, enhanced Open Graph tools for social previews, and schema plugins to enrich your snippets. - Summary: In this episode:: -What is prompt engineering -Why prompts succeed or fail in systems like ChatGPT and Stable Diffusion -How to engineer prompts to maximize your success Sign up to our weekly #email newsletter: https://www.trustinsights.ai/resources/data-in-the-headlights-newsletter/ Subscribe to our YouTube channel for more #marketing and #analytics videos! https://www.trustinsights.ai/youtube Need help with your company's #data and #analytics? Let us know: https://www.trustinsights.ai Subscribe to the Trust Insights podcast: https://www.trustinsights.ai/resources/in-ear-insights-the-trust-insights-podcast/ Join our free private Slack community, Analytics for #Marketers: https://www.trustinsights.ai/analyticsformarketers - URL: https://www.youtube.com/watch?v=nLLg3pxB_0k&list=WL&index=9&ab_channel=ChristopherPenn [Music]foreign[Music]happy New Year happy Thursday uh welcome to so what the marketing analytics andinsights live show I'm Katie joined byChris and John New Year same us uhuh in today's episode we're going to becovering prompt engineering and so thehot topic for the past at least month orso I know Chris it makes you cringebecause you're gonna say for like thepast few years but people have only juststarted you know playing with it andunderstanding even wrapping their headsaround it are AI assisted content writers and so specifically we're goingto be talking about uh chat GPT andstable diffusion now I don't know aboutthe two of you but I'm seeing posts andthreads and you know people everywheresaying that they have figured it out andhere's the top 10 ways that you can makethis work for you in your business inyour writing and your marketing and youknow it's everyone thinks they have thesolution to it and you know what we'regoing to do today is talk about howindividuals can come up with their own prompts basically prompt engineering fortheir own personal use becausemy concern with these people who aresaying here's the top 10 ways to use itthere's no personalization there likewhat if none of those scenarios apply tothe business that I run I'm still sortof stuck so what we want to do today istalk throughthe correct way to approach promptengineering and John I know you're goingto have like your top five fails yes we have the listopen or close with that whatever youwant to dowe can probably we can probably uhwe'll figure it out I have some ideasall right so Chris this is somethingthat you know a lot about and have beenworking on actually for quite a longtime and have been experimenting with itso where where would you like to starttoday for this to be the most valuablelet's do a little myth busting firstokay and then uh dig into the mechanics so all these AI systems they are notsentient they're not self-aware theTerminator is not anytime soon these areall based on what are called largelanguage models and a large languagemodel is really simple to understandconceptuallygiven a enormous amount of text that hasbeen digested down and categorized youthen ask these models to createsomething from that textthere was a great expression I was listening to this week in machinelearning and AI the podcast which Ihighly recommend if you're a verytechnical personumand they were interviewing uh some folksfrom uh with the Allen Institute aboutthe evolution of natural languageprocessing on these models and the oneguest whose name I can't remember it andI apologize for thatumsaid a word is known by the company itkeepsa word is known by the company it keepsand this is the essence of largelanguage models if you were to scrape the entire trust insights website rightwhat words would be associated withKatie Robert that those two words rightbecause the chance of me words like apodcast insights live stream ChristopherPenn John Wall and so on and so forthbecause these words appear in proximityto each other in a block of textso whencompanies like open Ai and the AllenInstitute and Google and such buildthese large language models essentially what they're doingthey take a whole bunch of words putthem on a given topic put them all in ablenderum and and create a sort of word soupbut within every word and phrase aremathematical associations for the wordsand phrases that appear nearbythat's why these tools can generatethings like grammatically correctsentences we have not taught them therules of grammar but they knowstatistically from a probabilityperspective you're probably not going tosay happy new rutabaga right that's probably that's not a phrase you saynearly as often as happy New Year rightand so that construct is is appropriateit will understand based on frequencythink differentis is a phrase is supposed to use eventhough it's grammatically wrong right itApple's Apple intentionally did it wrongit's think differently right the advertversion but I think different was thename of the campaign because it's beenused so much and appears in so much ofthe text out in the public the tools understand that that is a thing so whenwe're working with these large languagemodels that expression a word is knownby the company it keeps guides How We Doprompt engineering the more words thatwe use in our prompts and the morespecific we are the better the resultswe get the less detail we provide theless fewer guard rails and associationsthat we provide in a prompt the worsethese things perform and if you're unfamiliar a prompt is nothing more thanjust putting a piece of text into one ofthese machines asking it to do somethingso this is the chat GPT interface openai's other interface the playground issimilar and a prompt is simply theinstructions that you want it to do so Ithink okay that's the the first placethat start is helping people understandwhat it is that you're doingwell and so you know what I'm hearingyou say is if you put in a prompt which is basically a set of instructions uh aprompt that is very vague that says youknow write a 500 word blog on marketingit's gonna do its best to come up withsomething but if you're more specific tosayI need to teach you know my users aboutthe best practices for you know digitalmarketing on small budgets you're goingto get a very different set of resultsand so really what you're saying Chrisis be as specific as possible when generating your set of instructions andin this case it's a prompt exactly andthat's the thing about a lot of thesethese tools is users don't realize howdetailed you can be so let's do a uhsome a couple of examples I'm going todoum you are a socialmedia manager for a podcast write fivepromotional tweets for your podcast and so this is a very simplestraightforward prompt and you can seelike it doesn't really know what to dothere's not a lot of detail there rightit says podcast name topic guest namegive it a listen on platform and so onand so forth this is not I I would ifyou were going to put this in the thefails category I think this is kind ofnot great so while it's thinking aboutthat let's rephrase that let's reframethat well and I think that it'sinteresting that you started with you are a social media manager for apodcast so you're already introducing anew kind of prompt that I don't knowthat users are thinking about you'retelling the systemthe Persona that they're supposed tohave when they're writing these promptsso that's step one it almost sounds likehey if you created a user story youmight have all the pieces that you needto create a proper prompt that's correct and so for those who aren't familiar orjust want to Refreshera user story is a simple statement withthree parts as a Persona in this case itwould be the social media manager I wanttowrite uh five promotional tweets for thepodcast so that people sign up for thepodcast so that would sort of becomeyour guideline for creating your promptsin this instanceand so Chris you're definitely uh getting more detailed with the promptthe second time aroundthat's right now what makes chat GPTinteresting compared to other versionsof the same model is that it has afeature that other versions of it don'thave which is memory when you have achat that you're gauging in it does havethe ability to remember what haspreviously happened so in this case Isaid the name of the podcast ismarketing over coffee right the host thepodcaster John Walton Christopher Pennpodcast has been on the air since 2007and airs weekly the podcast URL is marketingovercoffee.comum the podcast is a weekly conversationon marketing current events rewritethese tweets with these new details sonowthe question I have isdo you need to be setting an intentionthough with the purpose of gettingpeople to listen or with the purpose ofgetting people to subscribeyou absolutely can so again this isweird because you can should youyou should be as specific as possible so specificity in prompt engineering is theway to happiness right as it is withmany things in life so John in terms ofyour top five fails uh it sounds likeyou know write five promotional tweetsfor my podcast is probably the wrong wayto approach ityeah definitely you know in fact let meI'll just throw this out so the ideawith this was that there are actuallyqueries you can make that are so horriblethe chat GPT just won't even refuse usto do it so number five write 500 tweetsencouraging entrepreneurs to increasetheir Hustlenumber four generate blog posts of VCScongratulating themselves for theirmorning routines and vision boardsnumber three make a convincing argumentwhy we need more Tick-Tock influencersoh my God goodnumber two write a children's book aboutconceding elections with class and dignity by a Republicanand number one create a podcast aboutbranding from c-level agency Executivesthose are the five worst uh prompts thatyou could throw out there that refusesto take so that was my my big brainstormthis morning in the shower to throwthose out so I hope those are fantasticbut I I think you make a good point Johnbecause I feel like that's themisunderstanding of how a system likethis is going to be used people aregoing to try to use it that way when you really need to be more thoughtful aboutthe approachum so thank you for sharing that Johnthat was fantasticexactly do you see that that's thatiterative approach all the time though Imean have you actually had times whereyou're going five seven eight times tocontinue tweaking the thing ohabsolutely and in fact you should rightpart of the the benefit of a system thathas some memory is that you can continuerefining it so now I've said hey be sureto include the URL on each tweet rightthe Tweet should encourage the user to listen inum and now it's including the URL intherethere's still some placeholders andstuff in here but it's it is gettingbetter and so this is one of the thingsthat's really nice about a system likethis is that you can get it to toimprove its results and then what you dois you go back and you consolidate allthe different prompts that you've put inand create sort of a master prompt thathas all those details that specificityup front so you can and should start with a user story but if you forgot oryou just are messing around you can gothrough it and iterate until you get aresult that you're pretty happy with andthen you save that that prompt somewherein a notebook and you know a text filewhateverand you ultimately have itum available to reuse when you need itto now the nice thing about these toolsis you know we're just talking aboutplain text here so this is this is verystraightforward stuff and there's a lotof use cases that people have done some fun stuff withum they've you know there's ones likeyou know pretend I'm a your personalchef I'll give you my my uh dietarypreferences uh you're gonnagive me some healthy dinner ideas andthings like that that's a kind of a funone but from a practical perspectivethere's a lot of things that we can dowith this software that I think havemore utility so let me give you anexample of one well before you move onto the Practical utility I want to askthe Devil's Advocate question here so you know John you would ask you know doyou go through like five six seven eightrounds of iteration with the prompt youknowwhat do you say to those who are goingto push back and say well by the timeyou got you know the refined prompt andthe responses you want I just could havewritten the thing in the first placeit's the same as why would you user instead of excel because you wantreproducibility you want it to be ableto be reused so because this is the gpt3 language modelunder underneath what you eventually endup doing is use these playgrounds usethese tools to sort of iterate be youryour R and D lab and then once you areready to be to really um build you knowbuild something with it you go in andyou start writing code right you startwriting code that can it can that cantake this data and turn it intosomething that is you would put into production so um I'll give you anexample for the longest timeum at trust insights we have beencreating uh keyword forecasts rightthese keyword forecasts uh I said wegive a uh our software uh list ofkeywords search volumes and past searchhistory and we say hey let's build aforecast for the next 12 weeks of thecontent that uh is the Search terms thatare likely to Trend and we've we've donethat we've used it for ourselves we'veused it to hand off to clients and stuff like that and it's it's been I would sayreasonably well received but we've notalways gotten people to do as much withit as we want including ourselves rightincluding ourselves so we startedplaying with the open AI system and thenstarted incorporating that into our codeand so now instead of just giving you alist of keywords uh each week uh for ayear we have said okay what if we startgiving you the actual blog topics rightgiven our keyword list maybe you're likeuh you know Katie something you've saidon our podcast is ah I don't know what to do with this keyword I mean this islike the 16th time I've seen thiskeyword well here's a different way ofapproaching this where you now have uh aa starting point to create some newcontentand this is the automation part of AIand this is this is a criticallyimportant the tools like chat GPT andstuff are fun they're a lot of fun youcan spend a lot of time in fact wetalked to folks on our analytics formarketer slack about how have you beenintegrating this and people are still atthe I call it the party trick stage like hey we made this thing and it did thisthing it was fun but to graduate fromparty tricks to production requires nowthinking how do I want to use this thingand bring it to life in a meaningful wayand I think that that's where you knowand as sort of as I mentioned at the topof the episode I'm seeing so many peopleposting the you know top 10 ways to getchat GPT to work for you and it stillfeels very hacky very party tricky because it doesn't it doesn't have thatrepeatability it still feels like thatone and doneand it's not very instructional it'sjust more of ideas like yeah I wouldlove to have you know AI write blogposts for me I could put that out as atop 10 way to use this thing but I'mstill not telling people how to make itrepeatable I'm still not telling peoplehow to actually train it to do the thingI'm just saying you could do that youyou could also switch from tying yourshoes to using Velcro it's also a thing that you could doelastic shoelaces I'm telling youelastic shoelaces this is the way to goin a nursing homeand that's becausethese tools as they are now are intendedfor humans to use right and they'reintended as one-offs they are intendedfor people to to get a sense of what'spossible but these tools are not theproduction uses of these tools andthere's a big gap between prototype and production these are prototypes and evenin the you know the more sophisticatedplayground this is still a prototyperight this is still just a a webinterface to the model this is not whatyou need to do to take it fromum you know what it is into somethingthat actually does the work behind thescenesso that's that's where the Gap is andthat's why it still feels like up youknow party tricks and and fun things and yes a lot of people are prognosticatingright now oh this is going to changeeverything yes it will but it not inthis form it because at some point youhave to take into production they haveto make it be able to scale and you knowhaving humans copy pasting all day longis not a particularly scalable way ofdoing things particularly when the toolsdo have things like apis that allow youto do thisoh they do have an API I was going toassume that they didn't have that yet ohGod yes um that they've had apis sincethe very beginning because that's how they make their money right if you lookat the building on the back endum an API call I think is a thousandtokens is is a pennyum so when you write a prompt in factlet's do this let's do something hereum let me takean episode of one of my shows and I'mgonna say I want tosummarize in a bolt Point list to the key points upthis episodenowI'm going to give it a maximum tokenlength of a thousand that's about 800words give or take uh tokens areslightly less than a wordwhen I hit go it's going to create thisthing it's going to start spinning thesethings outand every time it's generating a wordnow it is incurring a very small costright so this probably used it looks like he's about 300 tokens so it cost uhour account a third of a penny togenerate thatif this goes into a piece of softwarelike a word processor or something likethat where now hundreds of users orthousands of users using itthat third of a penny ramps up real fastand that's how open AI makes its moneyright you know you're having tenthousand twenty thousand thirty thousanddollars worth of compute timebecause they've taken it and put it intoproduction so as individual marketers these tools are actually one of the mostcost effective tools for doing stufflike this at the scale at human scaleonce you take it into production scalethat's when it gets expensive but againbecause you're probably using softwaredevelopers and stuff you already knowthat it's like any other API calls youknow sending stuff to AWS sending stuffto Google Cloud you know how that worksbut for marketing to take a leap forwardit's got to be integrated to the toolswe use the these these you know one anddone interfaces are good r d facilitiesso let's go back to the question ofprompt engineering and so you knowobviously people are still you knowdoing the party tricks playing with ityou know create a you know Samuraiversion of me riding on the back of aunicorn in space like yeah that is athing I don't know how that's useful uhpersonally wouldn't be useful to me butso what is how do we approach writing prompts because ultimately if I'munderstanding you correctly that's goingto be the key to making this work foryou in a repeatable way is having asolid structure which I do think wouldstart with a user story of why you'redoing it in the first place and what youwant the outcome to be so that whenyou're getting the text back you can sayyes it did the thing or no it didn't dothe thingpurpose is is the first part and you having that purpose be very clear isessential the second part is what is thelanguage that you would use if you weredirecting a human being to do it rightso uh or what is language you would useif you were instructing a human being todo it so if I said summarize this that'snot super helpful right I mean yeah youcan summarize it but if I handed this toa virtual assistant said summarize thisin a bullet point list okay cool what amI summarizing summarize the key points of this episode in a bullet point listokay that's a little bit more clearright that's that's pretty clear as towhat it is you want me to doand thenif I said summarizesumat a fourth grade readability level nowI'm starting to think about the peoplewho are involved right who are thepeople that would be using this outputright umand now we have obviously a verydifferent result because this is nowscaled down to a different level ofreadability so you've got to think aboutwho the people are that are going to beusing the outputs and and what they'regoing to be using the outputs for youthink about how is the output going tobe used so here's uman example let's see write a series ofthree multiple choice quiz questionsfrom this textquestion for you about age readabilitylevel yes so you just wrote write aseries of three multiple choice quizquestions would you get the same resultif you just said write three multiplechoice questionsmaybe it depends on though it dependsagain a word is told by the company itkeeps so in this case I use thatterminology because there's a conceptcalled waiting and the weight is how many times you use a word or relatedword in the prompt gives it additionalweight so you'll see this most withimage prompts for example with Dolly 2or stable diffusion or mid-airing wemight saydo a rendition of Katie Robert riding ona horse in a samurai outfit in outerspace highly realistic photorealistichighly detailed 8K resolution those lastfour words would give additional weightto the image to say this is it should bea photo should be photorealistic shouldbe very detailed as opposed to saying watercolor painting in the style ofHenri Matisse right that would be verydifferent a different set of Weightsthat the model would use to make adecision about how to render it withwith this by saying I want a series ofthree questions okaysometimes the model because we areambiguous writers is unclear what we'reasking so if we use words and phrasesthat are very specific we can get betterperformance out of the modelso you know what's interesting is thisis very much akin to delegation and so one of the questions that I get a lot asa manager ishow do I trust my team like I don't Idon't know how to delegate how do I getall this stuff off my plate and my firstpiece of advice to them is to setappropriate expectations of what youwant them to do so Chris to your pointof you know write a summary and thenthey give it to you back and they didn'tproduce what you want well guess whatthat's your fault not theirs because youwere not clear enough about what it was that you wanted so write a summary at afourth grade level including five bulletpoints telling me what someone's gonnalearn from listening to this episodethat's a very clear set of instructionsand then if the person or in this casethe machine doesn't give you back whatyou asked for that's when you can startto change things but until you are clearabout those expectations it's still yourfault not the machine or the persons in that case and so I see this as verysimilar toif you think about it like delegating tosomeone and you are very clear aboutwhat you want from them then you'regoing to get that thing backexactly that's exactly it it is it isthe probably the most underpaid virtualassistant you can ask for[Laughter]delegating a taskexactly prompt engineering is is the delegation of tasksumand when you think now about thedifferent tasks that you might havesomebody dothis is where the tools start to becomeusefulright so if I take the same thing andI'm going to go into the into the chatone because I like to have the thememory a bit I'm going to give it somedirections you are a social mediamanager responsible for the promotion ofcontent to users on Twitter Tick Tock andInstagramdo you have a a cap in terms ofcharacters four thousand okayfour thousandumyou will be creating social mediacontent to promotea piece of larger content buy ready when you are ready to beginokayall of the promotion is to encouragepeople to consume the content bywatching or reading it a piece ofcontent is this transcript so it's interesting it almost appears asthough you're having a conversation withit this time around like you've totallyswitched up you know the promptengineering the way in which you've beentalking about it so you ended your firstprompt with reply ready when you areready to begin versus here's the here'sthe purpose of the thingexactly and again that's one of thethings that makes this interface to themodel interesting is that it has theability to behave in in a moreconversational way so for people who maybe are not clear about how toengineer good props this may be thething so here's the here is the contentall rightso again this is from a YouTube videoso as it's pulling that up so to John'squestion about apis do youcan you connect this interface to different data sourcesor are you sort of stuck with here's thedata sources we've connected open AI togood luckno so think about it the data sources isessentially just being it's a pipe sowhatever pipe you put into the pipe uhis one way and then and then you knowwhatever however it is you want tochoose to get stuff out of it so forexample uh when we're doing it withGoogle Trends and Google and sent Rush data and ahrefs data uh you're justconnecting the code from from one systemto another uh let's take a look here atsee if I can open this up in VisualStudio code just to give you a sense ofwhat that looks like so inour code we just have a single functionhere that sendsthe text of our choice into the systemand Returns the results this is the thisis the code equivalent of typing in that window right of copypasting stuff except nowI can connect you know any data source Icould pull data out of a database Icould pull scrape data from the web Icould do any number of things to putdata into a text prompt feed it to theAPI and then get the response back andthat's how that's how we do it for ourstuff so you could integrate this into acall system if you're having real-timevoice transcripts you could integrate itintoum social media monitoring software you could even take it off you know liveclosed captions off of a video if youwanted to I'd be feeding the data in butno matter what it is you're doing that'syou just get the data in so now we'vegotten this um in based on thetranscriptwrite five promotional tweets for thecontentso and then this goes back to could younot have just posted the transcript andsay right for five promotional tweetsbased on this transcriptI could have but I enjoyed giving it extra contextum you know I could have put all that inone big prompt but in this case I didn'tbecause I would have had to have thoughtabout that beforehand and I and andagain this is one of the things that alot of people are thinking of is sodifferent about this tool is that youdon't need to have that you you can havea conversation to your point as if itwere an employee of yours and say okayhere's here's these these sweets okaygreat that looks interesting uh give me some video ideas for tick tock aboutwhat kind of videos I should make fortick tock to promotethis content now this is all text basedit can't do anything anything exceptgenerate text but it's goingto go ahead and give me some conceptualideas for the types of tick tock videosI could make so if you are one of thosefolks who's like uh I don't know what to do with Tick Tock right well ask amachine right ask a machine uhand here we are here we have behind thescenes create a series of tick tockvideos that give viewers a behind thescenes look at the process of writingcreating content highlighting uniqueimperfections of human written textwhen we think aboutwe have talked for a long time about theumtrans media content framework the ideathat you can take a piece of me likevideo you know split into all these different formats now the next evolutionof that is the transformation uhframework where you say Okay given thiscontent asking machines just like youwould a person what else can I do withthiswhat are some things I could do so wehave some Tick Tock video ideas hereum I would say uh what are some insInstagramalbum ideas to promote this content so I guess you know back to my questionabout you know apis and usabilityum you know could someone like Johnwho's responsible for businessdevelopment hook up a system like thisto their CRM and say give me a list ofyou knowand whatever the qualifications are likeyou know contacts who haven't you knowparticipated in anything or engaged inanything and write them a brief email to help them get re-engaged like is that ause case for something like thisabsolutely againwould you ask it is it a language taskif the answer is yes then this is theappropriate tool for it would you ask ahuman employee to do that the answer isyes is it repetitive work the answer isyes those three criteria make this agood candidate to do that well yes thewriting the email is the language taskbut the looking for accounts that are no longer engagedyes that's a language task in terms ofmy instruction but that's more of adatabasepiece is that something that a systemlike this could doI would not trust a system like this todo that because that's too vague what Iwould do is the example from the code Iwas showing earlier you would have apiece of code that would go through yourCRM and say you know make a list of onlyaccounts that are aged past X days right and then you look at the criteria inthem based on that criteria you couldthen have it create some content solet's start a new thing uh you are a topperforming sales executive we're talkingabout you John yeah even a list oflapsed accountsand you will take existing call notesand use them to craft aemail to each accounttrying to win them back so that's sortof our our table setting nowumLet's uh let's put together uh somethingthat is uh not under NDA so account nameuhAcme mechanicscontacton alllast contact eight April 1 2021all notesJohn said he didn't have the budgetright now probably lyingAcme sold a billion dollars in widgetslast orderfollow up with an email now let's see what it does with thisright a persuasive sales email based onthis informationwhat's interesting is uh you knowinstead of at responding to yourquestion earlier it's sort of uhfor lack of better term it's sort oflike machine splain to you how to dealwith your sales problem exactlyit was like no that's not what I askedyou to do I asked you to write the damnthing and so now I have to bemansplained on the left and machinesslain on the right gosh look at thebright side though it's machinesplitting to me so it's it's Equitableeverybody gets machinesexactlyumokay so it's created a response right itsays Dear John I hope this finds youwell I recently spoke to you on April 1st now to be fairit did say John I think you're lyingyeah they call them outbut it does sayumbut now we can also see this needs somerefinement sure refine this email thecompanyourcompanyname is trust insightswe sellanalytics and management consulting servicesto tell John on our Google analytics foraudit and implementationsee how it does now with a bit ofrefinement it still calls them outoh I love thatdear liarI know you said you don't have the moneybutmoney but that hugeexactly so to your question about is this a valid use case yes again is ittext yes is it language yes is it arepetitive task yeswe would just need to to be veryspecific in our prompts and our promptengineering to provide all theinformation needed so we can still seehere this one is not quite right yetbecause it says sincerely your name youwould not want this email going out uhright no and I think that that'sactually you know the number one pieceof advice is that you still need toreview and edit all of the things that come out of a system like this like ifyou are just taking it and Publishing itgoing livethat's that's a mistake because thesemachines are flawedjust like humansexactly an interesting thing though soif you you know if you've done this witha certain prompt a hundred times andevery time you know you've finally tunedit to the point where it's always givingyou what you want like you stillcouldn't go live with it right I meanone is you don't know how it could respond to a new query and then howoften does the Corpus get updated toolike when would this change underneathyou without you knowing it so at leastin the case of you know the open AImodels they actually have do publishwhen they're changing the underlying ummodel itself so uh this year I believesometime in q1 they're going to bereleasing gpt4 which is going to be thethe largest model of its kind it's about8X larger than the gp3 model which meansbetter performance which means more natural use of language and things likethat so now we can see we're finally ata point now we're okay this is actuallycoming from somebody right it's not yourname it's coming from Katieum and we we told it all in addition tothat new fact to change the tone goingto be warm conversational andprofessional right and so it said warmlyexactly so let's see I want to see if itdoesn't poke them in the eye yeah anywayit's still way nicer than I would havebeenold hostile professional tone ah there I amDear Mr wall Mr wall that's awesomeI'm gonna see if it ends with up yours[Laughter]ohuntil he calls him on his billion dollarof widgets again mm-hmmI mean it just it just won't let go ofthat I suggestions sincerely oh yeah and then you throw thetitle out there sincerely is like wowgood afternoonyes soum again these are some of the thingsone of the use cases that I use thesetools the most for is meeting notes andaction items so I will take thetranscript of a call put it in here andsay give me a list of the action itemsfrom this call becauseyeah forget stuff and reading through an unedited transcript is better thanlistening re-listening to the same 45minute call again but you still missstuff because your eyes kind of glazeover about a third of the way throughthe machine when it spits out your 10bullet points great okay copy paste thatgoes straight into your task managementsystem and now you're Off to the Racesum that's a very practical use otherthings you can do with it is having itdo a lot of rewrites um there are a lotof things that frankly these systems aregood at creating content but they aregreat at reprocessing existing content and I think that's one of the thingsthat's that's most overlooked so let'sstart a new oneyou are asocial media marketing expertwise in actingpatchy content for audiences onlineyou write the followingbiography in the first person appropriate for a LinkedIn bioso Katie I just pasted your bio from thetrust insights website and now it'sgoing to rephrase this into somethingthat will be good for your LinkedIn bioso for folks who are like uh I don'tknow what to put for my LinkedIn biotake the data that you've already gotand and re-spin itthank you so there's a a niceyou know decent LinkedIn bioum let's say rewrite this in a morecasual fun yet still professional tonehey there hey there that's me hey therethicker guns hey everybody oh boyit's not all work and no play for me I'ma Google language certified professionaloh yeah that's great and if that wasn'tenough I have a master's degree exactlybut so this is again this is my word forit this is all just languagemanipulation it's taking the data thatit's been given and re-spinning it andso that's where it's really good at um one of the things that we've seensome examples of this already butum let's takethis hang on hereI'm going to take this transcript fromso let's doI'm just gonna go with a shorter versionnowrewrite this transcriptumthey have Quantum Computing there likewe joke about that every week like noone can explain it I'm gonna have to go back and run thatokay grammar syntaxspelling punctuationandagain same exact thing but we've seen atthe bottom of our blog posts you know wepaste in the unedited transcripts fromour our live streams and things likethisif we wanted to get that contentRewritten to be more clear uh more readable but still preserve the languagewe're using this is a fantastic tool fordoing that um for when we're drivingaround and one of the things I like todo is I like to have a little voicerecorder with me either on my phonestuff like this and just foam with themouth I do this when I'm driving uh tolike conferences and stuff and if I haveideaswe know that sounds like word salad halfthe time and yeah Katie you've even saidI do not want to listen to you know 60Minutes of you rambling nope I can feed to this and say okay give me fourparagraphs or summarize my main pointsand now I've taken that and turned tosomething useful so there are any numberof applications again if you would hirea virtual assistant to do it if youwould a direct an employee to do it it'sa text-based language-based task that isrepetitivethese tools are good candidates that ifyou engineer your prompts well as youwere giving the way you were giving themto an employee that was a contractorjust started that daywith all the details they need you can make these tools very successful for youso I will share a quick anecdote of whatI fear is going to happen moving forwarduh so what's going to happen is Chris isgoing to start speaking to me and Johnas if we were the uh prompt boxum and I say this because this hashappened before so many many moons agowhen Chris and I worked uh elsewhere wewere traveling and Chris had gotten soused tospeaking into his phone to respond to emails that he started to speak to me asif I were the phone taking the dictationincludingstop new paragraph and I was like heydude I'm right here in front of you likeyou don't have to be like that and so Ican imagine that as people get more inthe weeds with prompt engineering itwill change the way that we communicatehopefully for the better maybe morespecifically maybe more directlywho knowsexactly the other thing I'll caution people is that right now chat GPT is ina research preview it will not beforever it will at some point come witha cost so you should be prepared to dealwith that you know when it can but morethan anything use it as a testing groundso that you can integrate it intoproduction level tools that will let itscale far beyond what one person typingprompts in the machine can do and that'sthat's where the real value is going tobe unlocked these things it's uh we areseeing the human interface to a veryvery powerful toolgreed John final thoughts that's all good just remember it's builton the average human's ability to writeso when you put in you're going to getbackall right thanks for tuning in folks andwe will catch you all next weekthanks for watching today be sure tosubscribe to our show wherever you'rewatching it for more resources and tolearn more check out the trust insightspodcast at trustinsights.ai TI podcastat our weekly email newsletter at trustinsights.ai newslettergot questions about what you saw intoday's episode join our free analyticsfor marketers slack group attrustinsights.ai analytics for marketerssee you next time[Music], So What? How to Do Prompt Engineering, Christopher Penn, 01gpecvheb3d40n1mfx20nrbt8.