Jev Is Not an LLM  🔗 **Source:** [Jev Is Not an LLM](https://www.youtube.com/watch?v=PNbbMi2h9VA) > [!note]+ Document Notes > tags: [system-one-models, structured-decisioning, inference-cost, web-scraping, vendor-benchmarks] > author: Firecrawl > source: Firecrawl (YouTube) > reference: https://www.youtube.com/watch?v=PNbbMi2h9VA > date created: 2026-09-21 00:00 > > > [!info] Generated analysis — not the author's words. Sections below are machine-written > > from the captured source. Only text inside `> [!quote]` blocks is verbatim from the author. > > Do not quote or attribute any other line to the named author. > > # 🏷️ Title: Jev Is Not an LLM (And That's Why It Wins) > <!-- deep-read:sections start --> ## 🧭 Table of contents - [📝 Summary](#-summary) - [📖 Main content](#-main-content) - [🗒️ Notes](#️-notes) - [📚 References](#-references) - [🗣️ Paraphrase](#️-paraphrase) - [📰 Newsletter](#-newsletter) - [✅ Action items](#-action-items) - [🧪 Hypotheses](#-hypotheses) - [📊 Methodology](#-methodology) - [❓ Questions](#-questions) - [🔮 Future research](#-future-research) - [💡 Implications](#-implications) - [🧠 Learning styles assessment](#-learning-styles-assessment) - [📊 Accessibility assessment](#-accessibility-assessment) - [🌍 DEIB assessment](#-deib-assessment) - [✅ Logical validity](#-logical-validity) - [🛡️ DARVO analysis](#️-darvo-analysis) - [🎯 Conclusion](#-conclusion) - [🔄 Refactor appendix](#-refactor-appendix--jev-is-not-an-llm-and-thats-why-it-wins) ## 📝 Summary - **What happened.** Firecrawl published a 5:41 walkthrough of Jev, a model from Type Safe AI that returns typed decisions with probabilities instead of generated text. The auto-transcript renders the product name as "Jeff" and the vendor as "Type Safe" / "Typesafe" throughout; every quote below preserves that spelling. - **The claim.** Jev is pitched as averaging 200x faster and 400x cheaper than "every frontier model," with input tokens priced below both a model the transcript calls "Claude 3 Sonnet 5" and one it calls "GPT-5 6 Luna," and output tokens priced at zero because Jev emits no tokens. - **The mechanism.** You pass state (instructions or a data row) plus questions with predefined answers. Jev returns the selected answers with confidence numbers. Three question types are named: choice, score, and a yes/no type the transcript transcribes as "null." - **Why it matters.** If classification, routing, and ranking move off token-generating models onto a typed classifier, the cost model for agent scaffolding changes — the expensive model only runs when prose is the actual deliverable. - **Key stakeholders.** Type Safe AI (vendor of Jev), Firecrawl (publisher of the video, and the scraping layer in the demo), Anthropic and OpenAI as the named cost comparisons, and developers currently paying frontier prices for structured output. - **Risks and tradeoffs.** This is vendor-adjacent marketing: the headline 200x/400x figures arrive with no benchmark name, no test set, no hardware baseline, and no methodology. The video ends with a Firecrawl signup code for 500 credits. Treat the numbers as claims, not measurements. - **Bottom line.** The architectural argument — scoring a fixed answer set costs far less than generating one — is sound and worth testing. The specific multipliers are unsourced in this source. ## 📖 Main content **The core argument.** The video frames Jev against Daniel Kahneman's two-system model of cognition: System 1 is fast, automatic, and intuitive; System 2 is slow and effortful. Jev is positioned as the System 1 component of an AI stack. It does not write. It picks. The narrator's framing is that today's stack uses a System 2 tool (a generative LLM) for System 1 work (choosing among known options), and pays generation prices to do it. **The interface.** Three pieces go in: state, questions, and a predefined answer set per question. What comes back is typed answers with attached probabilities. Three question types are named: - **choice** — select one option from a defined set - **score** — rate content against descriptive levels - the yes/no type, which the transcript renders as **"null"**. Given the described behavior ("when the answer is yes or no, returning probabilities and not a verdict"), this is near-certainly an ASR mangling of a boolean type name. Verify against Type Safe's docs before citing the name. **The demos.** Two are shown. First, a toy: state text "What a breathtaking sunrise," a choice question "What color is the sky?" over a color list. Jev returns salmon at 0.89 confidence. Adding a time-of-day question yields dawn. Changing the state text to describe stars between clouds flips the answers to indigo and night. The point being made is that the type system is doing the constraining while language understanding is doing the work — the same input to a generative model would need an output schema library (Zod is named) to reach the same shape. Second, a practical one: finding a specific book on a site with no search bar. Firecrawl scrapes the page and hands the category list to Jev; Jev ranks categories by likelihood; Firecrawl scrapes the winners; Jev picks the match. The run ranks music first (the book is a musical), finds nothing, falls through to non-fiction, and walks 53 pages before hitting the target. The narrator notes Firecrawl's map feature would have supplied the sitemap directly and calls the demo a showcase for sites without sitemaps — a useful piece of self-disclosure that a less careful vendor video would have omitted. **The objection the video raises against itself.** Game AI has had constrained, defined-output decisioning for roughly 30 years. The answer given is that game AI requires a developer to hand-author every rule, while Jev infers the choice from state it has never seen. That is a real distinction — the difference between a decision table and a learned classifier — though the video doesn't name the tradeoff that comes with it: a hand-written rule is auditable and deterministic, and a probability is neither. **Contested and unverifiable in this source.** The 200x/400x figures carry no benchmark, no baseline hardware, and no task suite. The cost comparison is shown on screen as a chart, so the analysis can't verify the numbers from the transcript alone. The Claude comparison concedes that Claude also returns salmon, with the stated differences being confidence level and token cost — meaning the head-to-head shows a tie on correctness for the single case demonstrated. "Completely free" output tokens is a pricing statement, not a compute-cost statement; a model that emits no tokens has nothing to bill for, which is a category difference rather than a discount. ## 🗒️ Notes #### ⏱️ [00:00:00](https://www.youtube.com/watch?v=PNbbMi2h9VA?t=0) The headline claim and the pedigree - Attribution: built by "one of the inventors of chat GPT" and "the mind behind the popular post trading technique RLHF" — transcript garble for post-training. No name is given. - Applications cited: self-driving a Tesla in real time, natural-language database search, playing four characters in Smash Bros faster than a human. > [!quote] [00:00:00] > Jeff by Type Safe AI is on average 200 times faster and 400 times cheaper than every frontier model, but it wasn't designed to generate text. #### ⏱️ [00:00:39](https://www.youtube.com/watch?v=PNbbMi2h9VA?t=39) System 1 framing and the API shape - Kahneman's two-system split is the organizing metaphor. Jev occupies System 1. > [!quote] [00:00:39] > You hand it some state like instructions or a row of data, a list of questions with multiple predefined answers, and Jeff hands back the [music] typed answers with probabilities. > [!quote] [00:00:39] > Jeff has [music] three question types: choice to select an option from a defined set, score for rating content against descriptive levels, and null when the answer is yes or no, returning probabilities and not a verdict. #### ⏱️ [00:01:20](https://www.youtube.com/watch?v=PNbbMi2h9VA?t=80) The sunrise demo - State text: "What a breathtaking sunrise." Question type: choice. Question: "What color is the sky?" > [!quote] [00:01:20] > So, if I run the code, Jeff comes back with a choice of salmon and a 0.89 confidence number. #### ⏱️ [00:01:56](https://www.youtube.com/watch?v=PNbbMi2h9VA?t=116) State change flips both answers - Changing the text to stars between the clouds yields indigo and night. - Suggested composition pattern: branch on the decision, or hand it to an LLM for a human-readable response. Objects can be attached to each criterion for context and clearer option boundaries. #### ⏱️ [00:02:38](https://www.youtube.com/watch?v=PNbbMi2h9VA?t=158) The cost comparison - Claude with a Zod-defined type returns salmon too, at lower confidence, and bills output tokens. - Named comparisons: "Claude 3 Sonnet 5" (input tokens a fraction of Jev's cost advantage) and "GPT-5 6 Luna" (Jev cheaper on input). Both names look ASR-mangled; confirm the real model versions before reuse. > [!quote] [00:02:38] > And remember, with other models, output tokens cost money, but with Jeff, it's completely free. #### ⏱️ [00:03:19](https://www.youtube.com/watch?v=PNbbMi2h9VA?t=199) Firecrawl + Jev book-finding loop - Loop: Firecrawl scrapes → Jev ranks categories → Firecrawl scrapes the ranked pages → Jev matches against the goal. #### ⏱️ [00:03:59](https://www.youtube.com/watch?v=PNbbMi2h9VA?t=239) The run, and the honest caveat - Music ranked first, no hit; fell through to non-fiction. > [!quote] [00:03:59] > it goes through all 53 pages until it finds the right one over here, which for some reason was in the non-fiction section. > [!quote] [00:03:19] > But, what's so revolutionary about Jeff? Haven't computer games had this sort of technology for about 30 years? #### ⏱️ [00:04:35](https://www.youtube.com/watch?v=PNbbMi2h9VA?t=275) Why this isn't game AI - Game AI has defined outputs, but a developer authors every rule in advance. - Concession: offline AI will always beat Jev on latency, because Jev makes network calls. > [!quote] [00:04:35] > With Jeff, you don't have to write any of that. You just show it what's going on and it picks the best option. #### ⏱️ [00:05:12](https://www.youtube.com/watch?v=PNbbMi2h9VA?t=312) Close and promo - Jev described as the first System 1 model from Typesafe. - Firecrawl signup code offers 500 extra credits. Next video teased: playing Doom inside Claude Code using a feature called Claude Mods. ## 📚 References - [Jev Is Not an LLM (And That's Why It Wins) — Firecrawl](https://www.youtube.com/watch?v=PNbbMi2h9VA) — the source video, 5:41, uploaded 2026-09-21. - [Firecrawl](https://firecrawl.dev) — publisher; supplies the scrape and map features used in the demo. - Type Safe AI — vendor of Jev. No URL is given in the transcript; find the official docs before citing the question-type names, which the auto-transcript garbles. - *Thinking, Fast and Slow*, Daniel Kahneman — the System 1 / System 2 source the video builds its framing on. Named on screen, not linked in the transcript. - [Zod](https://zod.dev) — the TypeScript schema library named as the structured-output workaround for generative models. - RLHF (reinforcement learning from human feedback) — the post-training technique the transcript renders as "post trading technique RLHF." ## 🗣️ Paraphrase Firecrawl argues that most of what developers currently spend frontier-model money on isn't writing — it's deciding. Routing a request, ranking a list, scoring a document, answering yes or no. Those are closed-set problems, and a generative model solves them the expensive way: it writes the answer out, token by token, then you parse it back into a type you already knew you wanted. Jev inverts that. You declare the answer set up front, hand it state, and get back the selection with a confidence number. Nothing is generated, so nothing is billed as output. The video's demonstration is deliberately small — sky color from a sentence about a sunrise — then scales into a scrape-and-decide loop that walks a bookstore site with no search bar, 53 pages deep, until it lands on the book. The narrator pre-empts the obvious objection (game AI has done constrained decisions since the 1990s) with a real answer: nobody wrote Jev's rules. The numbers attached to all of this, though, arrive without a benchmark behind them. ## 📰 Newsletter **The cheapest token is the one you never generate.** Firecrawl's 6-minute look at Jev, a "System 1" model from Type Safe AI, is worth your time even if you never touch the product — because the architectural point applies directly to WordPress work. Jev doesn't write text. You hand it state and a fixed list of answers, and it hands back the pick plus a confidence number. The pitch is 200x faster and 400x cheaper than frontier models on average, with output tokens at zero, though the video shows no benchmark for those figures. Where this lands for us: plugin-directory triage, support-ticket routing, Openverse moderation queues, Tide result classification, and "which handbook page answers this" lookups are all closed-set decisions. We're currently paying generation prices for them. A typed classifier that returns probabilities instead of prose is a different cost curve — and the probability is the part that matters, because it gives you a threshold for when to escalate to a human. Worth noting: the demo also shows the honest limitation. The book-finding run brute-forced 53 pages, and the narrator says outright that Firecrawl's map feature would have skipped that entirely. Good demos admit their shortcuts. Links: [the video](https://www.youtube.com/watch?v=PNbbMi2h9VA), [Firecrawl](https://firecrawl.dev), [Zod](https://zod.dev). ## ✅ Action items #### DevRel - Developer advocacy: build a two-model demo — typed classifier for routing, generative model for the final prose — and publish the token accounting per request, not a percentage. - Education: run one office-hours session on "when your LLM call is actually a classification call," using the sunrise example as the warm-up. - Developer experience: add a decision-vs-generation section to any agent quickstart, with a worked example of each. - Developer marketing: do not repeat 200x/400x in any Pantheon or community material without a benchmark citation. Cite the mechanism (no output tokens) instead. - Community management: add "vendor claim, no benchmark" as a standing note in link-sharing channels so numbers don't propagate unattributed. - Internal/external: ask Type Safe AI's docs team directly for the real question-type names and a published eval before anyone builds against it. #### OSPO - Policy and governance: decide whether a closed-weights, network-only classifier is acceptable in contribution tooling, and write the rule down before someone ships it. - License compliance: check Jev's terms for training-on-input clauses before any internal data touches it. - Security management: a classifier in a scraping loop is a prompt-injection surface — scraped page content becomes model state. Add that path to the threat model. - Software inventory: if Jev enters any pipeline, it's an SBOM entry with a network dependency and an availability SLO, not a library. - Developer support: publish guidance on confidence thresholds — a 0.89 is not a guarantee, and code that treats it as one is the failure mode. - Community engagement: track whether other System 1 vendors appear; a single-vendor category is a lock-in risk. - Strategic alignment: set a KPI for inference spend per resolved decision, so a swap like this is measurable rather than vibes. #### Community - Maintainers: pilot typed classification on issue triage in one repo, measure agreement with human labels for 30 days before expanding. - Contributors: document the escalation rule — what confidence level routes to a person — so contributors aren't guessing. - Sponsors: if a vendor funds a classification pilot, publish the cost figures both ways, before and after. - Ecosystem: run one cross-project test of the same classifier on the same issue corpus, so results are comparable instead of anecdotal. #### WordPress - Core: no action. This is tooling, not core code. - Core Performance: none directly; the latency claim is network-bound and irrelevant to PHP request budgets. - Core AI: add "closed-set classification" as a distinct pattern in any responsible-AI guardrail doc, separate from generation. The evaluation criteria differ — accuracy and calibration, not fluency. - **Core Program Team:** flag the terminology risk in cross-team notes — "System 1 model" is a vendor coinage, not an established category. - Mobile: none. - Playground: a browser-based demo comparing classification and generation on the same input would teach the distinction in one page. - Community: none. - Training: draft a short lesson on choosing between classification and generation. The sunrise-to-stars flip is a clean teaching example. - Marketing: none. - Support: evaluate typed classification for ticket routing, measured against current human routing accuracy. - Design: none. - Documentation: add a plain-language glossary entry for System 1 / System 2 framing if it enters WordPress AI docs, noting Kahneman as the origin. - Photos: closed-set tagging is a candidate use; test on a moderation sample first. - TV: caption-quality scoring against descriptive levels maps onto the score question type. - Polyglots: none. - Accessibility: none from this source. - Meta: none. - CLI: none. - Hosting: none. - Tide: score-type classification could rank scan findings by severity; run it against known-labeled results first. - Openverse: moderation queue classification is the strongest fit named here. Pilot with a human-review threshold. - Themes: review-queue prioritization is a closed-set ranking problem; worth a measured test. - Plugins: same as Themes, with a human gate on anything security-adjacent. #### The WPOCC - Programs: none directly. If The WP Open Community Collective funds maintainer tooling, inference cost per decision is a reasonable line item to track. - Projects: none from this source. - Funding experiments: if a microgrant covers AI-assisted triage, require published before/after cost and accuracy numbers as a condition. ### FAIR - Technical independence: Jev is a hosted, network-only model. Any FAIR-adjacent tooling that depends on it inherits a single-vendor availability dependency — the opposite of the independence goal. Document that before adoption. - Policies and covenants: if classification enters metadata or provenance checks, require the decision and its confidence be logged and auditable, not just the outcome. - Productized governance artifacts: add a "model dependency" row to operator scorecards covering hosted classifiers, not just hosted LLMs. ## 🧪 Hypotheses 1. For closed-set classification tasks, a typed classifier matches a frontier LLM's label accuracy within 2 percentage points while costing at least 10x less per decision — a far weaker claim than 400x, and the one actually testable from this video. 2. End-to-end latency advantage collapses below 5x once network round-trip is included, because the video concedes Jev makes network calls and offline AI beats it. 3. Confidence numbers from Jev are better calibrated than LLM-reported confidence on the same closed-set task, measured by Brier score. 4. Splitting an agent into classifier-for-routing plus LLM-for-prose reduces total cost per completed task by more than 50% versus LLM-only, on a workload where fewer than 20% of steps produce user-visible text. 5. The 53-page brute-force walk in the demo indicates the category-ranking step adds no value over sitemap traversal when a sitemap exists — i.e., the classifier's contribution is only measurable on sitemap-less sites. 6. Vendor-stated multipliers of this magnitude (200x, 400x) fail to reproduce within a factor of 5 under independent testing on a published benchmark. ## 📊 Methodology - **Design for H1 and H3.** Assemble a 500-item labeled set from an existing corpus — WordPress support tickets with known routing outcomes works. Run the same items through Jev's choice type and through a frontier model with a Zod-constrained schema. Report accuracy, macro-F1, and Brier score for calibration. Record cost per 1,000 decisions from provider invoices, not estimates. - **Design for H2.** Measure wall-clock time from request issue to parsed result, p50 and p95, from three network locations. Separate model time from transit time so the comparison isn't an ISP test. - **Design for H4.** Instrument one real agent workflow twice — once LLM-only, once split. Count completed tasks, not requests, and divide spend by completions. - **Design for H5.** Reproduce the book-finding demo on 20 sites, 10 with sitemaps and 10 without. Compare pages fetched under Jev-ranked traversal versus sitemap traversal versus naive breadth-first. - **Design for H6.** Pull Type Safe AI's published benchmark, if one exists. If none exists, that absence is the finding, and any replication must state which task it chose and why. - **Controls throughout.** Fix prompt and state text across arms. Run three seeds. Pre-register the threshold for "matches accuracy" before looking at results. ## ❓ Questions - What benchmark produced 200x and 400x, and against which frontier models at which context lengths? - Is the yes/no question type actually called "null," or is that an ASR error? The described semantics (probabilities, not a verdict) suggest a boolean. - What are the real model names behind "Claude 3 Sonnet 5" and "GPT-5 6 Luna" in the cost chart? - If output tokens are free, what is billed — input tokens only, requests, or a seat? - How is confidence calibrated? Is 0.89 a softmax value or a calibrated probability, and against what validation set? - What happens when the correct answer isn't in the option set? Does Jev return a low-confidence wrong pick, or signal out-of-set? - Is Jev fine-tunable on a domain option set, or is the option list purely inference-time? - What are the rate limits and availability guarantees for real-time uses like the Tesla and Smash Bros examples? - Does input state get retained or used for training? ## 🔮 Future research - **Calibration under distribution shift.** Closed-set classifiers degrade when the input distribution moves. Nobody has published how a hosted System 1 model behaves when the option set is stable but the state text drifts. - **Prompt injection through scraped state.** The Firecrawl loop feeds scraped page content directly into model state. A page that contains text shaped like instructions is an unexplored attack surface for typed classifiers specifically — the literature covers generative models far better. - **Hybrid cost modeling.** There's no standard accounting for split classifier/generator pipelines. A shared cost-per-completed-task metric would make claims like this comparable across vendors. - **The category question.** Is "System 1 model" a real architectural category or a marketing frame for a well-served classifier endpoint? Independent architectural analysis would settle it. - **Option-set design as a discipline.** The video hints at it — attaching objects to criteria for "clearer boundaries between options" — but option-set construction is unstudied as a skill, the way prompt engineering was in 2023. ## 💡 Implications - **Practice.** The default agent architecture changes: classify first, generate only when prose ships to a human. That's a design rule you can apply today regardless of whether Jev specifically is the right tool. - **Policy.** Probability outputs need documented escalation thresholds. A system that acts on 0.51 confidence without a human gate is a governance failure, and the video gives no guidance on where the line sits. - **Security.** Scraped content entering model state is injectable input. Any team copying the Firecrawl+Jev loop inherits that surface and should treat scraped text as untrusted. - **Education.** The System 1 / System 2 framing is a genuinely good teaching device for a distinction learners struggle with. Credit Kahneman, and note the framing is analogy rather than architecture. - **Funding models.** Zero-cost output tokens shift inference spend toward input volume. Organizations budgeting AI by token count need a different meter. - **Vendor literacy.** A 6-minute video from a company with a product placement, quoting 200x and 400x with no benchmark and closing with a signup code, is a training case in how to read AI marketing. That's a reusable artifact for any community education program. ## 🧠 Learning styles assessment - **Reading level estimate.** Flesch-Kincaid Grade Level on the transcript body: approximately grade 9–10. Flesch Reading Ease lands near 60 — plain conversational English, but domain terms (RLHF, frontier model, Zod, System 1) push effective comprehension to an intermediate-developer audience. #### 👀 Visual learners - Present: live code editing on screen, a side-by-side cost comparison chart, the site being scraped in real time. - Improve: no architecture diagram of the state → questions → typed answers flow, which is the single idea that most needs one. Add a before/after pipeline diagram and label the cost chart axes verbally for viewers who can't read them. #### 🔊 Aural learners - Present: full narration throughout; music stings mark section transitions. - Improve: the music bed overlaps speech at least three times — the transcript captures [music] mid-sentence twice. Duck the music under speech. A 90-second audio-only summary would serve podcast listeners. #### 🗣️ Verbal learners - Present: the three question types are named and defined in one sentence each. The Kahneman analogy does real explanatory work. - Improve: RLHF, frontier model, and Zod go undefined. One clause each would cost 10 seconds. #### 🧩 Kinesthetic learners - Present: two runnable demos, shown executing. - Improve: no repository link, no copy-paste snippet, no quickstart in the transcript. The viewer can watch but not do. #### 🧠 Logical learners - Present: a clear premise-to-conclusion structure, and a self-raised counterargument (game AI) with a stated rebuttal. - Improve: the 200x/400x figures have no derivation. Show the arithmetic behind one comparison and the logical learner can extend it. #### 🧑‍🤝‍🧑 Social learners - Present: YouTube comments, and a next-video callout that builds a series. - Improve: no Discord or forum invitation in the transcript, no discussion prompt. #### 🧘‍♀️ Solitary learners - Present: short enough to watch and rewatch; self-contained. - Improve: no reflection prompt or self-check. "Which of your current LLM calls is really a classification?" would convert the whole video into an exercise. #### 🌿 Naturalistic learners - Present: nothing explicit. The sunrise and stars examples are natural-world text, but incidentally. - Improve: a 400x cost reduction implies a compute-per-decision story with an energy dimension. Naming the energy-per-inference angle would land with this group and is a real claim the vendor could substantiate. ## 📊 Accessibility assessment #### Screen reader semantics - Present: YouTube supplies the page landmarks and player controls; the auto-generated transcript is machine-readable. - Improve: on-screen code and the cost chart carry information the narration never speaks. WCAG 2.2 AA 1.1.1 (Non-text Content) and 1.2.5 (Audio Description, Prerecorded) — neither is met for the chart. #### Keyboard navigation - Present: the YouTube player is keyboard operable by default, with visible focus. - Improve: nothing in the video adds barriers. 2.1.1 and 2.4.7 pass via the platform, not via the content. #### Contrast and readability - Present: unverifiable from a transcript. Editor code on a dark theme is the likely presentation. - Improve: small editor font at default YouTube resolution risks failing 1.4.3 (Contrast Minimum) and is unreadable on mobile. Zoom into code regions during demos. #### Text resizing and reflow - Present: the player scales; captions resize independently. - Improve: burned-in on-screen text doesn't resize. 1.4.4 (Resize Text) and 1.4.10 (Reflow) fail for any text baked into the video frame. #### Media and captions - Present: an auto-generated transcript exists, which is how this analysis was produced. - Improve: the auto-captions garble the product name ("Jeff" for Jev), the vendor, the technique ("post trading" for post-training), and both comparison model names. That's a 1.2.2 (Captions, Prerecorded) failure in substance — captions exist but are inaccurate on every proper noun. Human-corrected captions are the fix, and they're cheap for 5:41 of audio. #### Forms and interactions - Present: none in the video. The promo code is spoken and shown. - Improve: speak the signup code character by character, or put it in the description as selectable text. Not applicable to WCAG in-video, but 3.3.2 (Labels or Instructions) applies to the landing page it points to. #### Standards reference - Primary gaps against WCAG 2.2 AA: **1.2.2** (inaccurate captions), **1.2.5** (no audio description for on-screen code and charts), **1.1.1** (chart data conveyed visually only), **1.4.4 / 1.4.10** (burned-in text doesn't reflow). The captions failure is the highest-impact and the easiest to fix. ## 🌍 DEIB assessment #### Cultural breadth - Present: examples are Tesla, Smash Bros, and a Hamilton book — all US-centric consumer and entertainment references. - Improve: swap one example for a non-Western or non-consumer use case. Multilingual classification would also demonstrate the language-understanding claim more convincingly than a single English sentence. #### Inclusive language - Present: no exclusionary terms in the transcript. Language is direct and second-person. - Improve: "mind-blowing," "crazy," and "mind" as ability-framing appear at 00:04:35 and 00:00:00. Low severity, easily swapped for concrete descriptors — which also satisfies the every-claim-carries-a-fact standard. #### Neurodiversity support - Present: strong chunking. Each segment does one thing, and the structure is predictable: claim, demo, objection, answer. - Improve: the music stings that overlap speech are a processing obstacle. Chapter markers would let viewers navigate by topic rather than scrubbing. #### Representation - Present: one narrator, unnamed and uncredited in the transcript. No other voices. - Improve: no independent or third-party voice appears anywhere. A single skeptical outside comment would improve both representation and credibility. #### Psychological safety - Present: the tone is enthusiastic but not dismissive of alternatives. The game-AI objection is treated as legitimate, and the sitemap caveat is volunteered. - Improve: superlatives ("revolutionary," "mind-blowing," "amazing") without benchmarks can make a viewer who doesn't see the magic feel they've missed something. Stating the actual measured scope would lower that pressure. ## ✅ Logical validity #### Strong reasoning - **Deductive, 00:02:38.** If output tokens are billed and Jev produces no output tokens, then Jev incurs no output-token cost. Valid, and the conclusion follows from the premise without needing a benchmark. This is the strongest argument in the video. To clarify further: state what *is* billed. - **Analogical, 00:00:39.** The Kahneman System 1 / System 2 mapping is apt for the interface being described — fast, automatic, closed-set on one side; slow, generative, open-ended on the other. Clarify by naming it as an interface analogy, not an architectural claim, since nothing about Jev's internals is disclosed. - **Anticipatory counterargument, 00:03:19 to 00:04:35.** Raising the game-AI objection and answering it with the hand-authored-rules distinction is genuine intellectual work, and the distinction holds. Strengthen it by naming the cost of the swap: learned decisions aren't auditable the way rule tables are. - **Empirical demonstration, 00:01:56.** Changing state text from a sunrise to stars flips both sky color and time of day. That's a controlled single-variable change — the closest thing to evidence in the video, and it supports the narrow claim (language understanding drives the choice) it's used for. - **Self-limiting disclosure, 00:03:59 and 00:04:35.** Admitting that Firecrawl's map feature would have skipped the 53-page walk, and that offline AI beats Jev on latency, are concessions against interest. They raise the credibility of the surrounding claims. #### Fallacies - **Unsupported quantification (bare assertion), 00:00:00.** "on average 200 times faster and 400 times cheaper than every frontier model" carries no benchmark, no task set, no model list, and no date. "On average" across what? "Every frontier model" is an unbounded universal. *Fix:* name the benchmark, the models and versions tested, the hardware, and the date, and publish the harness. Absent that, state the mechanism ("no output tokens are generated") and drop the multiplier. - **Appeal to authority, 00:00:00.** "Built by one of the inventors of chat GPT and the mind behind the popular post trading technique RLHF" — no name is given, and neither credential bears on whether the model is accurate at classification. *Fix:* name the person and link the prior work, then let the benchmark carry the argument. - **Hasty generalization, 00:02:38.** A single prompt (sky color) returning salmon from both Jev and Claude, with Claude at lower confidence, is presented as a general comparison. One case, one task, one option set. *Fix:* run 100+ items and report accuracy and calibration, not one confidence number. - **Equivocation on "free," 00:02:38.** "with Jeff, it's completely free" conflates zero output-token *price* with zero *cost*. Input tokens are still billed, and the video's own chart shows input-token pricing. *Fix:* say "output tokens aren't billed because none are produced; you pay for input." - **Suppressed evidence / undisclosed interest, 00:03:19 and 00:05:12.** Firecrawl is both the publisher and half the demo stack, and the video closes with a Firecrawl signup code for 500 extra credits. The commercial relationship to the tool being praised is never stated as a disclosure. *Fix:* a one-line disclosure at the top. - **Argument by impressive example, 00:00:00.** Self-driving a Tesla, database search, and four-character Smash Bros play are cited with no demonstration, source, or link — none of the three is shown in the video. *Fix:* link each claim to a demo or drop it. - **Ambiguity from uncorrected captions, throughout.** Not a fallacy on the author's part, but "null" as a question-type name and "Claude 3 Sonnet 5" / "GPT-5 6 Luna" as model names are unusable as cited facts. *Fix:* corrected captions. #### Tone of voice - Enthusiastic, fast, and demo-first, in the standard developer-YouTube register. Second person and contractions throughout ("I can write code like this," "you don't have to write any of that"). The register shifts to genuine surprise at 00:04:35 — "what's even crazy is that we're having this comparison in the first place." Superlatives cluster at the open and close, which is exactly where the unsupported numbers and the promo code sit; the middle third, where the demos run, is measurably more careful and includes the self-limiting caveats. That pattern — marketing at the edges, engineering in the middle — is the most useful thing to know about how to read this source. ## 🛡️ DARVO analysis Not applicable to this source. DARVO (Deny, Attack, Reverse Victim and Offender) describes a response pattern to accusation or criticism. This is a product walkthrough with no accusation, no accuser, and no defensive posture — the nearest thing to a challenge is a rhetorical question the narrator raises against his own product at 00:03:19 and then answers substantively rather than deflecting. No denial, no attack on critics, no victim reversal. For completeness across the three prongs: **Deny** — nothing is denied; limitations are volunteered (sitemap shortcut, offline latency). **Attack** — competitors are compared on price, not disparaged. **Reverse victim/offender** — no such move appears. ## 🎯 Conclusion The durable idea here is architectural and doesn't depend on Jev: when the answer set is known in advance, generating the answer is the wrong mechanism. A model that scores a closed option set and returns probabilities skips the token generation you were paying for and hands back a number you can threshold on. That argument is valid on its own terms at 00:02:38, and the sunrise-to-stars demo at 00:01:56 shows the language understanding survives the constraint. The specific numbers do not survive scrutiny in this source. 200x faster and 400x cheaper than "every frontier model" arrives with no benchmark, no model list, and no date; the head-to-head against Claude is a single prompt where both models return the same answer. Firecrawl publishes the video, supplies half the demo stack, and closes with its own signup code, and none of that is disclosed. **Next steps for readers:** audit your current LLM calls and count how many produce a value from a known set rather than prose for a human. That count is the size of the opportunity, and you can measure it today without adopting anything. If the count is meaningful, run the 500-item labeled comparison in the Methodology section before trusting any vendor multiplier — including this one. And get the real product name, vendor URL, and question-type names from Type Safe AI's own documentation, because the auto-captions on this video mangle all three. ## 🔄 Refactor Appendix — Jev Is Not an LLM (And That's Why It Wins) > Density: 2–5 Literature atoms (H3 pad), 0–3 Permanent candidates (H4 pad). > Emitted: 4 Literature, 2 Permanent. Resonance filter applied — the Tesla/Smash Bros > application claims were dropped as unevidenced. > > Konik claim-header rules: present tense, declarative, ≤10 words, searchable, unique. > > Atomicity: one declarative sentence per atom. "and"/"also" → split. > > Tag pool: #bmf · #addendum · #xref · #articleseed · #storystem · #share · #aphorisms ### 📚 Literature Split Pad #### [H1] Closed-set answers cost nothing to emit > [!quote] [00:02:38] > And remember, with other models, output tokens cost money, but with Jeff, it's completely free. **Context & use**: - 00:02:38, immediately after the Claude/Zod comparison returns the same answer at lower confidence. - This is the one deductively valid argument in the video. It holds independent of any benchmark, which is exactly why it's the claim worth keeping. **Tags**: #xref #articleseed <!-- atomicity: 1 idea ✓ --> #### [H2] Typed answers with probabilities replace parsed prose > [!quote] [00:00:39] > You hand it some state like instructions or a row of data, a list of questions with multiple predefined answers, and Jeff hands back the [music] typed answers with probabilities. **Context & use**: - 00:00:39, the API description right after the Kahneman framing. - The probability is the operationally important half. It's what lets a pipeline set an escalation threshold instead of blindly trusting a label. **Tags**: #xref <!-- atomicity: 1 idea ✓ --> #### [H3] Learned choice beats hand-authored rules on setup cost > [!quote] [00:04:35] > With Jeff, you don't have to write any of that. You just show it what's going on and it picks the best option. **Context & use**: - 00:04:35, answering the self-raised objection that game AI has done constrained decisions for ~30 years. - The honest counterweight the video omits: a rule table is auditable and deterministic; a learned pick is neither. Record both sides together or the atom misleads. **Tags**: #xref <!-- atomicity: 1 idea ✓ --> #### [H4] Vendor multipliers arrive without benchmarks > [!quote] [00:00:00] > Jeff by Type Safe AI is on average 200 times faster and 400 times cheaper than every frontier model, but it wasn't designed to generate text. **Context & use**: - 00:00:00, the opening claim, never substantiated anywhere in the remaining 5:41. - Keep as a teaching specimen for AI-marketing literacy: two precise-sounding numbers, a universal quantifier ("every frontier model"), and no method. **Tags**: #xref #share #articleseed <!-- atomicity: 1 idea ✓ --> ### 🧠 Atomic Split Pad #### [P1] Classify before you generate **Claim** (one sentence, Courtney's voice): When the set of valid answers is known before the call, a classifier is the correct tool and a generative model is an expensive substitute. **Evidence pointer**: [[#H1]] · [[#H2]] **Scope & conditions**: Holds when the option set is enumerable and stable. Breaks when the answer set is open, when the output itself is the deliverable, or when the option set changes faster than you can maintain it. Also breaks if out-of-set inputs are common and the classifier has no way to signal "none of these." **Links**: - [[AI cost architecture]] — because extends - [[Structured output with schema validation]] — because contrasts on where the constraint is enforced — schema library at parse time versus model at inference time - [[Support ticket triage]] — because applies-in closed-set routing <!-- atomicity: 1 idea ✓ --> #### [P2] A precise number without a method is a marketing claim **Claim** (one sentence, Courtney's voice): Numeric specificity in vendor material signals confidence, not measurement, unless the benchmark, baseline, and date travel with the number. **Evidence pointer**: [[#H4]] **Scope & conditions**: Applies to any comparative performance claim. Does not apply to mechanism statements, which can be verified from architecture alone — "no output tokens are generated" is checkable; "400x cheaper" is not. The tell is a universal quantifier over an unbounded comparison class. **Links**: - [[Reading vendor benchmarks]] — because extends - [[Every claim carries a fact]] — because applies-in writing standards - [[Self-limiting disclosure raises credibility]] — because contrasts on direction of interest — this video does both, superlatives at the edges and concessions in the middle <!-- atomicity: 1 idea ✓ --> ### ✏️ Fleeting Queue - [F1] Confirm the real product name, vendor, and question-type names from Type Safe AI's docs — auto-captions render "Jeff," "Type Safe," and a type called "null" that is probably a boolean — https://www.youtube.com/watch?v=PNbbMi2h9VA #fleeting #inbox - [F2] Identify the actual models behind "Claude 3 Sonnet 5" and "GPT-5 6 Luna" in the 00:02:38 cost chart before citing the comparison anywhere #fleeting #inbox - [F3] Open question: does scraped page content entering classifier state create a prompt-injection path distinct from the generative case? The Firecrawl loop at 00:03:19 is the concrete instance #fleeting #inbox - [F4] Count how many current LLM calls in my own tooling return a value from a known set rather than prose — that count sizes the opportunity #fleeting #inbox - [F5] Article seed: "Marketing at the edges, engineering in the middle" — the tonal pattern where unsupported superlatives cluster at open and close while the demo section volunteers caveats #fleeting #inbox #articleseed - [F6] Kahneman's System 1 / System 2 is becoming a product-category label. Track whether other vendors adopt "System 1 model" as a category name #fleeting #inbox ### → Things 3 (max 2 actions) - [ ] Verify Jev's real product name, vendor URL, and question-type names against Type Safe AI's own documentation before citing this video anywhere — research, 15 min #things - [ ] Audit current agent tooling for LLM calls that return a value from a known option set, and log the count — AI cost review #things <!-- deep-read:sections end --> 📄 **Full text:** [[Full Document Contents/Articles/Jev Is Not an LLM.md|Jev Is Not an LLM by Firecrawl]] ## Highlights - [00:00:00] A new model has just been released that's changing the way we use AI. Built by one of the inventors of chat GPT and the mind behind the popular post trading technique RLHF, Jeff by Type Safe AI is on average 200 times faster and 400 times cheaper than every frontier model, but it wasn't designed to generate text. No chat, no code, no sentences. Despite this, people have used it to self-drive a Tesla in real time, search through a database with natural language, and play four characters in Smash Bros quicker than a human. How? Well, Jeff is what Type Safe call a system one model inspired by this book from Daniel [00:00:39] Kahneman that splits human thinking into two systems. System one for fast, intuitive, and automatic decisions, and system two for slow and effortful decisions. Jeff is built to be system one. You hand it some state like instructions or a row of data, a list of questions with multiple predefined answers, and Jeff hands back the [music] typed answers with probabilities. And that's essentially it. Jeff has [music] three question types: choice to select an option from a defined set, score for rating content against descriptive levels, and null when the answer is yes or no, returning probabilities and not a verdict. I'll tell you later on why Jeff ([View Highlight](https://read.readwise.io/read/01m48tbkmk2rx9g13nr5vge72j)) - Note: [H2] Typed answers with probabilities replace parsed prose - Tags: atom/literature/h2 - [00:02:38] also returns salmon with a lower confidence level. But, and here's the interesting part, it actually cost us output tokens to generate this output. And remember, with other models, output tokens cost money, but with Jeff, it's completely free. Comparing Jeff with Claude 3 Sonnet 5, we can see input tokens from Jeff are a fraction of the cost of Sonnet and output tokens on Jeff are completely free. And even with GPT-5 6 Luna, Jeff has much cheaper input tokens. So, this can be used to do some really cool things. Here's another example. I need to find the Hamilton book on this site, but there's no search bar. So, instead of me going through [00:03:19] each section one by one or the most likely section to find the books, I can use Jeff and Firecrawl to do so. So, Firecrawl will do an initial scrape of the page and hands the categories to Jeff. Jeff will pick the most likely categories that the book will be in, and then Firecrawl will scrape those pages, give them to Jeff, and Jeff will find the right book based on our criteria. So, in our code, I've given Jeff a goal. Firecrawl is going to scrape those pages, and Jeff is going to take those pages and use it to make the right decision based on the goal we set earlier. So, running this code, Jeff has ranked music the highest since Hamilton ([View Highlight](https://read.readwise.io/read/01m48tb86vtt0kw4bcw1dxjk45)) - Note: [H1] Closed-set answers cost nothing to emit - Tags: atom/literature/h1 - [00:03:59] is a musical, Firecrawl has scraped all the books for music and it hasn't found it, so it's gone to non-fiction and it's taken all the pages from non-fiction and it goes through all 53 pages until it finds the right one over here, which for some reason was in the non-fiction section. Now, I know Firecrawl has a map feature to give Jeff all the pages that already exist from the sitemap and it could choose the best one, but this is a cool demo to show for sites that don't have a sitemap. But, what's so revolutionary about Jeff? Haven't computer games had this sort of technology for about 30 years? Well, yes and no, because although game AI also [00:04:35] has defined outputs like Jeff, a developer has to write the logic and every rule in advance. With Jeff, you don't have to write any of that. You just show it what's going on and it picks the best option. So, although offline AI will always be faster than Jeff, which has to make calls to the network, what's even crazy is that we're having this comparison in the first place. The fact that Jeff is fast enough to play these games in real time and even multiple games in parallel is mind-blowing. So, that's a quick overview of Jeff, the first system one model from Typesafe. I know it seems simple from the outset, but it opens up ([View Highlight](https://read.readwise.io/read/01m48tc0vvkxzcb5w2c1z06b2a)) - Note: [H3] Learned choice beats hand-authored rules on setup cost - Tags: atom/literature/h3