Markdown Is a Terrible Language - CourtneyR.dev's 2nd Brain

Markdown Is a Terrible Language ![rw-book-cover](https://img.youtube.com/vi/DWcqbPm_Rn4/maxresdefault.jpg) 🔗 **Source:** [Markdown Is a Terrible Language](https://www.youtube.com/watch?v=DWcqbPm_Rn4) > [!note]+ Document Notes > tags: #markdown #markup-languages #web-development #parsing #programming-languages #syntax #html > author: Theo - t3․gg > source: YouTube > reference: https://www.youtube.com/watch?v=DWcqbPm_Rn4 > date created: 2024-12-19 14:22 > > # 🏷️ Title: Markdown is a terrible language >  ## 🧰 Action Inbox - [H1] Evaluate markdown alternatives for documentation systems — current parsing vulnerabilities #web-development - [H2] Audit existing markdown implementations for ReDoS vulnerabilities — CVE examples shown #security - [H3] Consider build system approaches for markup languages — compile-time hooks #architecture ## 🧭 Table of Contents - [[#📝 Summary]] - [[#📖 Main Content]] - [[#🗒️ Notes]] - [[#📚 References]] - [[#🗣️ Paraphrase]] - [[#📰 Newsletter]] - [[#✅ Action Items]] - [[#🧪 Hypotheses]] - [[#📊 Methodology]] - [[#❓ Questions]] - [[#🔮 Future Research]] - [[#💡 Implications]] - [[#🧠 Learning Styles Assessment]] - [[#📊 Accessibility Assessment]] - [[#🌍 DEIB Assessment]] - [[#✅ Logical Validity]] - [[#🛡️ DARVO Analysis]] - [[#🎯 Conclusion]] ### 🔗 References - [Why the heck are we still using Markdown?](https://blog.brockmckegney.com/posts/why-are-we-using-markdown/) (Original article) - [RWX CI Platform](https://swade.link/rwx) (Sponsor) ## 📝 Summary - Theo critiques Markdown's fundamental design flaws while acknowledging his historical love for the language - The core problems include ambiguous syntax, security vulnerabilities (ReDoS), and scope creep beyond its original purpose - Markdown has evolved from simple markup to requiring full HTML parsing and complex compilation - Multiple syntaxes can produce identical output, making parsing exceptionally difficult - The language suffers from legacy decisions dating back to email/Usenet conventions from the 2000s - Bottom line: Markdown succeeded as simple markup but fails when used as a pseudo-programming language ## 📖 Main Content Theo examines a critical analysis of Markdown, explaining how a language he once loved enough to only submit Markdown resumes has fundamental flaws. The discussion covers how Markdown's flexibility created multiple ways to express the same formatting (bold, italic, headers), leading to parsing nightmares. Security vulnerabilities like ReDoS (Regular Expression Denial of Service) affect major implementations due to complex regex patterns. The inline HTML feature, while powerful, breaks the simplicity promise and creates XSS attack vectors. Modern usage requires features like footnotes, math typesetting, and custom styling that transform simple transliterators into full compilers with context-sensitive grammars. The author argues for purpose-built tools with proper build systems rather than continuing to extend Markdown beyond its capabilities. ## 🗒️ Notes #### ⏱️ [00:00](https://youtu.be/DWcqbPm_Rn4?t=0) — [H1] Theo's historical dedication to Markdown - Key idea: Refused job applications unless he could submit Markdown resume - Evidence: "I literally didn't apply to jobs at companies where I couldn't submit my Markdown resume" #### ⏱️ [01:42](https://youtu.be/DWcqbPm_Rn4?t=102) — [H2] Markdown scope creep problem - Key idea: New file extensions embedding other content signals overextension - Evidence: "Once we got to the point where we were making new file extensions to embed other types of content into our Markdown, we were probably over the line" #### ⏱️ [03:21](https://youtu.be/DWcqbPm_Rn4?t=201) — [H3] RWX sponsor segment - Key idea: Local CI testing without commits using cloud infrastructure - Evidence: "All you have to do is run their CLI to trigger a run without having to commit the code" #### ⏱️ [05:45](https://youtu.be/DWcqbPm_Rn4?t=345) — [H1] Multiple syntaxes produce identical output - Key idea: Two completely different markdown syntaxes can render identically - Evidence: "There is almost no syntax overlap between these other than the block quote at the end" #### ⏱️ [07:23](https://youtu.be/DWcqbPm_Rn4?t=443) — [H2] Bold/italic syntax complexity - Key idea: Multiple valid ways to write bold create parsing nightmares - Evidence: "You can write a bold in different ways. You have the double star syntax, double underscores, or the B tag" #### ⏱️ [09:15](https://youtu.be/DWcqbPm_Rn4?t=555) — [H1] ReDoS vulnerability example - Key idea: Regular expression denial of service affects Markdown parsers - Evidence: "This 6.9 severity level CVE for Markdown it. This is a legit DoS vulnerability" #### ⏱️ [11:32](https://youtu.be/DWcqbPm_Rn4?t=692) — [H2] Inline HTML problems - Key idea: Markdown parsers must also handle HTML parsing complexity - Evidence: "The real issue is that to ship a Markdown parser, you also need to ship a friendly HTML parser" #### ⏱️ [14:45](https://youtu.be/DWcqbPm_Rn4?t=885) — [H3] Feature creep requirements - Key idea: Modern markdown needs far more than simple markup - Evidence: "Basic LaTeX support with TikZ support with the ability to install packages, PlantUML, mermaid, custom styling" #### ⏱️ [16:21](https://youtu.be/DWcqbPm_Rn4?t=981) — [H2] Security attack surface expansion - Key idea: Plugin hooks and execution engines create XSS vulnerabilities - Evidence: "Every time we allow inline HTML, plugin hooks, or embedded execution engines, we expand the attack surface" #### ⏱️ [18:45](https://youtu.be/DWcqbPm_Rn4?t=1125) — [H3] Legacy syntax decisions - Key idea: 2000s email conventions create modern parsing problems - Evidence: "Markdown was inspired by pre-existing conventions for marking up plain text in email and Usenet posts" #### ⏱️ [21:32](https://youtu.be/DWcqbPm_Rn4?t=1292) — [H1] Context-sensitive grammar problem - Key idea: Footnotes make parsing dependent on document-wide context - Evidence: "Reference style links and footnotes require global definition resolution" #### ⏱️ [23:15](https://youtu.be/DWcqbPm_Rn4?t=1395) — [H2] From transliterator to compiler - Key idea: Modern features transform simple mapping into full compilation - Evidence: "Vanilla markdown needs a transliterator, which is literally a one-to-one mapping function. But modern markdown has to support stuff like footnotes, which takes this simple transliterator into a full-blown compiler" #### ⏱️ [26:45](https://youtu.be/DWcqbPm_Rn4?t=1605) — [H1] Proposed solution approach - Key idea: Purpose-built tools with proper build systems instead of extending Markdown - Evidence: "What we need is a custom-built tool, not a Frankenstein's monster that we dare to call a language" #### ⏱️ [29:32](https://youtu.be/DWcqbPm_Rn4?t=1772) — [H3] Historical criticism acknowledgment - Key idea: Jeff Atwood called out Markdown problems in 2012 - Evidence: "John Gruber's original markdown.pl is one of the worst small programs I've ever read, completely riddled with outright bugs" ## 📚 References - [Original article by Brock McKegney](https://blog.brockmckegney.com/posts/why-are-we-using-markdown/) - [RWX CI Platform](https://swade.link/rwx) - [CommonMark Specification](https://commonmark.org/) - [Jeff Atwood's 2012 Markdown critique](https://blog.codinghorror.com/) ## 🗣️ Paraphrase - Markdown's flexibility creates parsing nightmares because multiple syntaxes can produce identical output - Security vulnerabilities like ReDoS affect major implementations due to complex regular expressions - Modern feature requirements transform simple markup into complex compilation with context-sensitive grammars ## 📰 Newsletter Theo dives deep into why Markdown might be holding us back as developers. While acknowledging his former devotion to Markdown (he used to only apply to jobs that accepted Markdown resumes!), he explores how the language's flexibility has become its weakness. Multiple syntaxes for the same output, security vulnerabilities, and the complexity of modern requirements have transformed simple markup into compilation nightmares. Key takeaway: Consider whether your documentation and content tools are truly serving your needs, or if you're working around fundamental limitations. Resources: [Learn WordPress](https://learn.wordpress.org/), [Developer Docs](https://developer.wordpress.org/), [WP-CLI](https://wp-cli.org/) ## ✅ Action Items #### DevRel - Developer experience: Evaluate documentation formats beyond Markdown for complex technical content - Education: Create guidelines for safe Markdown usage patterns in community documentation - Developer marketing: Consider accessibility implications of markup language choices #### OSPO - Security Management: Audit Markdown parsers for ReDoS and XSS vulnerabilities in documentation systems - Software Inventory: Document markup language dependencies and their security implications - Developer Support: Provide alternatives to Markdown for complex documentation needs #### Community - Contributors: Establish safe Markdown patterns for community-generated content - Maintainers: Consider build systems for documentation that require advanced features #### WordPress - Core: Evaluate Gutenberg block parsing lessons applicable to markup language design - Documentation: Assess DevHub/HelpHub Markdown usage for security and maintainability - Support: Create guidelines for safe markup in user-generated content #### The WPCC - Programs: Develop best practices for documentation formats in funded projects ## 🧪 Hypotheses - Context-sensitive grammars in markup languages lead to exponentially more parsing vulnerabilities - Simple transliterator approaches are more secure than full compilation for basic content needs - Build-time processing can eliminate many runtime security issues in markup languages - Purpose-built tools outperform extended general-purpose languages for specific domains ### 📊 Methodology - Compare CVE counts between simple and complex markup language implementations - Measure parsing performance and memory usage across different syntax approaches - Test attack surface reduction with build-time vs runtime processing ### ❓ Questions - Could compile-time markup processing eliminate most XSS and ReDoS vulnerabilities? - What would a security-first markup language design look like? - How do we balance simplicity with the advanced features users demand? ### 🔮 Future Research - Security analysis of alternative markup languages and their parsers - User experience studies comparing different documentation formats - Performance benchmarks of various markup compilation approaches ### 💡 Implications - Practice: Reconsider Markdown for complex documentation projects requiring advanced features - Security: Audit existing Markdown implementations for regex-based vulnerabilities - Education: Teach developers about markup language security implications - Policy: Establish guidelines for safe markup language usage in production systems ### 🧠 Learning Styles Assessment Reading level estimate: Grade 12-14 (technical content) #### 👀 Visual learners - Present: Code syntax examples, vulnerability demonstrations - Improve: Syntax comparison diagrams, parser flow charts #### 🔊 Aural learners - Present: Video format with clear narration and examples - Improve: Audio-described code examples #### 🗣️ Verbal learners - Present: Analogies (C++ comparison), technical term definitions - Improve: More real-world examples and use cases #### 🧩 Kinesthetic learners - Present: Interactive parsing examples, hands-on vulnerability demonstrations - Improve: Sandbox environments for testing different markup approaches #### 🧠 Logical learners - Present: Clear problem progression, formal language theory references - Improve: Decision trees for markup language selection #### 🧑‍🤝‍🧑 Social learners - Present: Community discussion context, historical perspectives - Improve: Discussion prompts about team documentation choices #### 🧘‍♀️ Solitary learners - Present: Detailed technical analysis, independent research references - Improve: Self-assessment checklist for markup security #### 🌿 Naturalistic learners - Present: Evolution analogy of markup languages - Improve: Ecosystem thinking about tool dependencies ### 📊 Accessibility Assessment #### Screen reader semantics - Present: Clear heading structure, code block labels - Improve: More descriptive alt text for code examples #### Keyboard navigation - Present: Standard video controls, clear focus indicators - Improve: Timestamped navigation shortcuts #### Contrast and readability - Present: High contrast code examples - Improve: Ensure syntax highlighting meets contrast requirements #### Text resizing and reflow - Present: Responsive layout supports zoom - Improve: Test code block readability at 200% zoom #### Media and captions - Present: Full transcript provided - Improve: Code example audio descriptions #### Forms and interactions - Present: Standard video interface - Improve: Accessible video speed controls ### 🌍 DEIB Assessment #### Cultural breadth - Present: English-focused technical discussion - Improve: Examples from international markdown usage patterns #### Inclusive language - Present: Gender-neutral technical terms - Improve: Avoid casual profanity in technical discussions #### Neurodiversity support - Present: Clear section structure, consistent formatting - Improve: Content warnings for rapid code switching #### Representation - Present: Technical merit focus - Improve: Acknowledge diverse developer tool preferences #### Psychological safety - Present: Respectful critique of technology choices - Improve: Emphasize learning over blame for tool decisions ### ✅ Logical Validity #### Strong reasoning - Deductive strength: Security vulnerabilities logically follow from parsing complexity - Inductive strength: Multiple examples support pattern of Markdown problems - Causal reasoning: Feature creep directly causes parsing complexity - Analogical strength: C++ comparison effectively illustrates design problems #### Fallacies - Appeal to authority: Relying on Jeff Atwood's criticism without independent analysis - False dichotomy: Presenting limited alternatives without exploring hybrid approaches #### Tone of voice - Overall tone: Conversational but technically informed, balancing humor with serious critique ### 🛡️ DARVO Analysis - Deny: No denial patterns detected - Attack: No personal attacks, focuses on technical merit - Reverse victim/offender: No victim/offender dynamics present - Emotional cues and rhetoric: Frustration expressed constructively toward technical problems ## 🎯 Conclusion Markdown's evolution from simple markup to complex pseudo-programming language has created fundamental security, parsing, and maintenance problems. While acknowledging Markdown's historical importance and continued utility for basic use cases, the analysis suggests that modern requirements demand purpose-built tools with proper build systems rather than continuing to extend Markdown. The key insight is recognizing when a tool has exceeded its intended scope and being willing to adopt alternatives that better match current needs. ## 🧱 Refactor Appendix ### 📚 Literature Split Pad Purpose: Stage CEE units that become Literature notes. Split level: H3 ### [H1] Multiple syntaxes create parsing complexity > [!quote] Markdown is a terrible language by Theo - t3․gg via YouTube. Published 2024. Accessed 2024-12-19. > "There is almost no syntax overlap between these other than the block quote at the end." Demonstrates how Markdown's flexibility becomes a liability for reliable parsing. [view highlight](https://youtu.be/DWcqbPm_Rn4?t=345#H1) #xref #articleseed ID:rw_markdown_syntax ### [H2] ReDoS vulnerabilities affect major implementations > [!quote] Markdown is a terrible language > "This 6.9 severity level CVE for Markdown it. This is a legit DoS vulnerability." Shows concrete security impact of complex regex patterns in --- ## Full Transcript [00:00] I've been a huge fan and advocate of [00:01] Markdown for as long as I can remember, [00:03] even to the point where it hurt my [00:04] career. I literally didn't apply to jobs [00:07] at companies where I couldn't submit my [00:08] Markdown resume as my official resume [00:11] for the company. I had my entire resume [00:13] on my GitHub, and if I had to or I was [00:15] really interested, I would just submit a [00:17] PDF with a single link to the Markdown [00:19] file on my GitHub. That's how dedicated [00:21] I was. I just didn't think all of these [00:23] fancy ways of presenting information [00:25] mattered when you had a good, simple, [00:27] reliable language for it specifically. I [00:30] will always be incredibly thankful to [00:32] John Gruber and Aaron Swartz for [00:33] creating something as magical as [00:35] Markdown, having such a simple, reliable [00:38] way to just write content and have it [00:41] come out and render so well. It still [00:44] has a bit of a magical feel to it. As [00:46] someone who tried really hard to make [00:48] LaTeX work in college, Markdown, once it [00:50] clicked for me, just kind of became how [00:52] I write everything to this day, which is [00:54] why this article immediately grabbed my [00:56] attention. Why the heck are we still [00:59] using Markdown? As great as Markdown is, [01:02] I am more than happy to agree we have [01:03] went too far with it. Once we got to the [01:05] point where we were making new file [01:07] extensions to embed other types of [01:09] content into our Markdown, we were [01:11] probably over the line. And now that [01:13] we're in a world where we're using LLMs [01:15] and agents and all of these things, and [01:17] Markdown is the method we communicate [01:18] between them, it's getting even more [01:20] questionable. I have a lot of different [01:22] things I want to talk about with [01:23] Markdown in the near future, things like [01:25] how Markdown is not the right primitive [01:26] for memory or how Markdown is actually [01:28] pretty good as a programming scripting [01:30] language in specific scenarios, which [01:32] we'll definitely talk a lot about in the [01:33] future. But today I want to question the [01:35] fundamentals. Why are we still using [01:37] Markdown? What makes it so great? And [01:40] what would a better solution for the [01:42] stuff we do today look like? I've barely [01:44] even started and I can already imagine [01:46] all of the Obsidian comments in my head, [01:48] and if I know y'all love anything more [01:50] than Obsidian, it's probably today's [01:51] sponsor. I have a question for you. [01:53] What's the most times you've committed [01:54] code that didn't work just trying to fix [01:56] CI? I've probably stacked up to 15 [01:58] commits before just watching CI break [02:00] again and again. And all I wanted was [02:02] the ability to run it locally without [02:03] having to make more commits, ideally in [02:05] a way that doesn't even force me to [02:06] leave my editor or my agent. Today's [02:08] sponsor RWX. I can tell you about how [02:10] fast they are, how great they are to [02:11] work with agents, or how they run things [02:13] in parallel so you don't have to wait [02:14] for a bunch of stuff that's blocking [02:16] when it doesn't have to be. There's a [02:17] lot of reasons to use RWX, but I don't [02:19] want to talk about any of that because [02:20] I'm in love with the CLI. Once you write [02:22] the YAML file for your CI, all you have [02:24] to do is run their CLI to trigger a run [02:26] without having to commit the code. So, [02:28] instead of committing, pushing, and [02:29] waiting for the whole thing to run, you [02:31] can just run a CLI command locally, it [02:33] will then spin up a cloud instance that [02:34] can still take advantage of all of the [02:36] caching infra and all the other things [02:38] they do right, and immediately get your [02:39] feedback as to whether or not it worked. [02:41] This should be enough to sell you. But [02:42] when you combine it with their agent [02:44] skill, suddenly it's like, oh, yeah. [02:47] Seriously though, Run Loops are such a [02:48] great solution to so many different [02:50] problems. If you were working on a hard [02:52] feature that you can't test, go write [02:53] the test, commit it, and then tell the [02:55] agent, make sure CI passes by running [02:57] the RWX run command, and then when it [02:59] passes, commit it and push. This is such [03:01] a good workflow, and I'm amazed nothing [03:02] else has it. And this is just the tip of [03:04] the iceberg. The more you dig in, the [03:05] more you'll appreciate RWX's unique way [03:07] of fixing CI so that you will move [03:09] faster instead of slower. Sign up for [03:11] the CI that makes your agents faster at [03:13] swade.link/rwx. [03:14] So, why the heck are we still using [03:16] Markdown? There are a few things in life [03:18] that bring me much joy and heat at the [03:20] same time, like a chocolate that hurts [03:22] when eaten, or Markdown. Seriously, why? [03:25] Half of the time we aren't even using [03:26] the full language. Speaking of which, [03:28] this is a bold opening. HTML is the best [03:31] programming language. I know you've [03:32] heard people say that the only [03:33] programming language they know is HTML, [03:35] and I know we both roll their eyes in [03:37] discontent trying to get our programming [03:39] language papers out of assembled decks [03:40] on of papers on how HTML is only a [03:43] markup language and not a programming [03:44] language. I mean, yes, we're on the [03:46] right, but that guy probably has [03:48] something that we don't have. [03:50] A life. Okay, already owned and the [03:53] article's barely started. I I will have [03:55] you know that my girlfriend loves when I [03:58] talk about HTML and Markdown. [04:00] But to each their own. [04:01] Note, when the author is talking about [04:02] Markdown, he is specifically talking [04:04] about Common Mark unless stated [04:05] otherwise, because it is the unambiguous [04:08] syntax specification. I love the [04:09] project. I really appreciate their [04:11] efforts on making this language a bit [04:12] more grounded. It's not the [04:14] specification that's broken, it's the [04:15] language itself. Oh boy. [04:17] The good. Markdown is a minimal language [04:20] used for typesetting trivial documents. [04:21] It needed to do one simple job, get a [04:23] Markdown file and output an HTML file. [04:26] Its index is as legible as it gets, and [04:28] it's easy to write even with no assists. [04:30] Like the C language, you can see the [04:31] output that will be created. Bold is [04:33] always open, closed B tags at the end, [04:36] and the italics are the same. Learning [04:38] curve is simply nonexistent if you're [04:39] just a casual user. Just one look at the [04:41] cheat sheet and you're ready. But now [04:43] the bad. We don't know what we want. Do [04:46] we want UI? Do we want a programming [04:47] language? We don't know. The only reason [04:50] feature creep exists is because of [04:51] unclear specifications. You want a [04:54] minimal, easily legible markup language, [04:56] you have Markdown. Simple as that, [04:58] right? Well, here are two different [05:00] syntaxes to generate the exact same [05:03] output. You have the single hash for [05:05] title, hello, stars for doing an italics [05:09] here for I am, and then unambiguous, [05:11] which you want bold, and then grammar in [05:13] a block quote. This is roughly how I [05:15] would write this. I would have used [05:16] double stars for unambiguous, but I am [05:19] what I am. Actually, actually, I would [05:20] have used double stars for unambiguous [05:22] and underscores for that, so I guess I'm [05:23] closer to here. But I would never have [05:25] done the header syntax like this. God. [05:28] Yeah. There is almost no syntax overlap [05:31] between these other than the block quote [05:33] at the end. Woof. Oh boy, we're about to [05:35] have fun. Hope you have the two eyeballs [05:37] enough to see that Markdown is not what [05:39] you asked for. These produce two [05:40] identical outputs, and this is just the [05:42] tip of the iceberg. It has so many poor [05:44] decisions baked in that if you try to [05:46] use it, it will actively fight against [05:47] you the moment you think you know what [05:49] you're doing. Let's start with bold, [05:50] italic, and bold italic. In Markdown, [05:52] you can write a bold in different ways. [05:54] You have the double star syntax, double [05:56] underscores, or the B tag. And these are [05:58] just some of the ways a valid bold can [05:59] be written. And these are for Common [06:00] Mark. If you're using something which [06:02] isn't marketing itself as Common Mark [06:03] compliant, you can very well encounter [06:05] valid stuff that produces the same [06:07] input. Like this, or this, or yeah, God, [06:10] yeah. We have had to deal with so much [06:12] [ __ ] like this for T3 Chat, you guys [06:14] have no idea. It is so absurdly [06:18] difficult to handle all the different [06:20] weird cases for Markdown rendering, but [06:22] I will also personally say the edge [06:24] cases for LaTeX are even worse. So, [06:27] yeah. [06:28] >> [snorts] [06:28] >> And don't even get him started on [06:30] layered ones like this. Oh God, I don't [06:31] even want to try to parse this. Okay, we [06:34] got the two star here, then this one is [06:36] an italic. So, this whole section is [06:37] bold here, this one is italicized. This [06:40] is italicized. [06:42] A is bolded here, not italicized, and [06:45] this is re-italicized. Of is italicized. [06:49] This is italics, and this is doubled the [06:52] italics for the led here, I think. That [06:55] hurt. [06:57] Or this. Oh. [06:59] No. No, please no. [07:02] I'm not even going to try on that one. [07:04] This thing is actually so peak that we [07:06] have a class of parser vulnerabilities [07:08] called ReDoS, regular expression denial [07:10] of service, that are affecting this. [07:12] Like this 6.9 severity level CVE for [07:15] Markdown it. This is a legit DoS [07:18] vulnerability. [07:20] You just put a bunch of stars in a link [07:21] and it can't process it. Beautiful. [07:25] Beautiful. [07:26] Yeah, this string here takes 65,553 [07:31] steps. That will [ __ ] over anything. [07:33] Great, super cool. We love that. [07:37] So, how much worse can it possibly get [07:39] here? Exhibit B, ASM was a good idea, [07:42] but this? In old languages where [07:43] compilers were producing optimal code [07:45] like a river in a desert, inline [07:47] assembly helped them write [07:48] performance-critical code with ease and [07:50] the cost of the compiler engineers' [07:52] blood, sweat, tears, and the birth of [07:54] their firstborn son. Yeah, writing your [07:56] inline assembly to get perfect code is [07:59] hellish. It's one of the reasons that a [08:00] project like FFmpeg is so tough, but [08:02] also so important. Inline ASM allowed [08:04] stuff like SIMD operations before the [08:06] compiler supported them. You want an [08:07] overview of early SIMD generation [08:09] failures, you can take a look at those [08:10] here. Oh God, that's the last thing I [08:12] want to look at. Now, let's take this [08:14] wonderful idea and bolt it directly into [08:16] the most bloated, single-threaded, [08:17] sandboxed environment expecting a simple [08:19] and easy way to write documents. And [08:21] that's how inline HTML inside Markdown [08:23] was born. Oh boy. Inline HTML allows you [08:27] to do stuff like this. I've had so many [08:29] problems with inline HTML and Markdown [08:31] personally that I am even more on this [08:33] author's side now. [08:35] Yeah. I am an insert simple programmer [08:38] doing span class fancy text elegant [08:41] close span programming. Div class [08:42] animation. And here is my portfolio. [08:45] Close div. Isn't this just simple? Isn't [08:47] this just neat? The main reason why [08:49] correct Markdown parsing is [08:51] exceptionally hard isn't because [08:52] Markdown syntax is so hard to [08:54] comprehend, it's only 1/10 of the issue. [08:56] The real issue is that to ship a [08:58] Markdown parser, you also need to ship a [09:00] friendly HTML parser. If you're using [09:01] HTML inside of Markdown, why not use [09:03] HTML from the start? Fair. I actually [09:06] remember now, friend of the channel, who [09:09] might even be here right now, Jacob [09:10] Evans, tried really hard a while back, [09:14] and it looks like he has given it up [09:15] since, but he tried super hard to have a [09:18] single Markdown file for his GitHub [09:20] profile and for his homepage by serving [09:23] the Markdown as HTML, because there's [09:25] enough overlap that hypothetically, you [09:28] could write a valid HTML homepage and [09:30] have it render as your profile as well [09:33] without code between the two. Literally [09:35] the same exact bytes for both. And the [09:38] results were hellish. He tried really [09:39] hard to make it work. [09:41] But man, it was very fun watching him [09:43] try. Markdown in and of itself isn't [09:45] powerful enough to satisfy the simple [09:47] monkey brain developers like the author, [09:49] who's only satisfied when the site looks [09:50] good enough. Good enough in this case [09:52] means it needs to have at least basic [09:54] LaTeX support with TikZ support with the [09:56] ability to install packages, PlantUML, [09:58] mermaid, custom styling, custom [10:00] shortcodes, tagging and taxonomy, proper [10:03] footnotes, BibTeX support. I don't want [10:05] a simple job from the simple tool, too. [10:08] To nail a painting to the wall, I need a [10:09] hammer. In this case, the hammer is [10:11] markdown. But what if I wanted to paint [10:13] too? I'll break the canvas the moment I [10:14] try to paint it with a hammer. Breaking [10:16] the canvas also means a whole lot of [10:17] CVEs, primarily around cross-site [10:19] scripting vulnerabilities. This has me [10:21] thinking. As we think, I want to take a [10:22] look at my blog and see how [ __ ] my [10:25] markdown is. I haven't thought about [10:26] this in a while. So we already have a [10:28] helmet here. That's great. [10:31] I have an iframe embedded cuz it's a [10:33] YouTube video. So in the first 13 lines [10:36] in the second paragraph here, I'm [10:38] already doing weird HTML [ __ ] Then the [10:40] rest is fine. Oh, here's a great one. I [10:42] needed to have an image tag that was [10:45] also a link. So I put an image HTML tag [10:49] inside of brackets to attach the link. [10:52] Isn't this wonderful? [10:54] I had to check two posts to find some [10:56] cursed [ __ ] More cursed HTML [10:58] inlined. Man, I used to be so good at [11:00] writing. It's a shame that the syntax is [11:02] awful. Yeah. Get the idea. It As soon as [11:04] you want to use markdown for more [11:06] interesting stuff, you end up writing [11:09] some cursed stuff. Every time we allow [11:11] inline HTML, plugin hooks, or embedded [11:13] execution engines, we expand the attack [11:15] surface. The result is predictable. [11:17] Recurring cross-site scripting [11:18] vulnerabilities across major markdown [11:20] implementations and much, much more. And [11:22] this will continue to rise with the vibe [11:23] [ __ ] parsers that are hitting the [11:25] market. Yep. I've seen a lot of attempts [11:27] to rethink how we parse markdown because [11:29] all of the agents spit out markdown. We [11:31] want to make things that are better, [11:32] faster, more reliable, and all of them [11:34] are just full of exploits, edge cases, [11:36] vulnerabilities, and more. Apparently, [11:37] Obsidian markdown is even more cursed. [11:39] I'm sure I will love that. Speaking of [11:41] things I love, we're now in exhibit C, [11:42] the old and obscure syntax. Markdown, [11:45] like all the good tech we have around [11:46] the World Wide Web, was made in the good [11:48] old 2000s. Markdown was inspired by [11:50] pre-existing conventions for marking up [11:52] plain text in email and Usenet posts. [11:54] Mind you, before 2000, most serious mail [11:56] looked like this. Do you see this quote [11:58] syntax putting the colon at the start of [12:00] every line? [12:02] Oh god, this ending in the signature. [12:04] Woo. Do you see the beauty of it? The [12:06] quote syntax, the vertical separation [12:08] with pipes. This is what markdown was [12:09] needed for, which it would fail horribly [12:11] at because in markdown you can't divide [12:12] the screen into n parts without inline [12:14] HTML black magic. But because of this [12:16] legacy, you have two different ways of [12:18] writing headings, the normal way and the [12:20] ATX way. You have two different ways and [12:22] more than two CVEs of writing bold and [12:24] italic. You have two different ways of [12:25] writing horizontal rules, which collide [12:27] with one of them, colliding with the set [12:29] text header syntax. You have two [12:30] different ways of writing unordered [12:31] lists. You have an ordered list, which [12:33] doesn't care about how you ordered them. [12:35] And you have a footnote syntax, which [12:37] moves the entire grammar to [12:38] context-dependent grammar. And I [12:40] remember the first time I saw that like [12:42] the right way to do lists in markdown, [12:44] the correct thing is to put zeros [12:45] because then your markdown parser can [12:47] number them correctly and if you change [12:48] the order of things, you don't have to [12:49] go redo all the numbers. Ugh, so much [12:52] [ __ ] Oh, oh, I didn't see this [12:55] line. This language is literally the C++ [12:58] of markup languages. Oh man. Now that [13:01] I've seen this, I can't unsee this. That [13:03] is entirely correct. This might be my [13:06] favorite programming video ever other [13:08] than maybe the OOP is garbage video. [13:12] It's 2 hours long and I had to stop and [13:15] pause and go back to it later because [13:17] unlike most 2-hour-long videos, I [13:19] couldn't watch that one in the [13:20] background cuz it was too good. So I [13:22] actually sat and like locked in and just [13:24] stared at my phone for 2 hours. My [13:26] roommate at the time asked if I was [13:27] okay. I was like, "I'm more than okay. [13:29] You're going to have to watch this [13:30] after." This video is so good. Highly [13:33] recommend it. And knowing that, having [13:35] that still in my head, C++ in markdown, [13:39] yeah, there's a lot of overlap. Nearly [13:41] everything can be done in two different [13:42] ways, which some of them might allow for [13:43] cross-site scripting and somehow leak [13:45] memory in HTML. It's used everywhere, [13:47] but it also fails everywhere equally. [13:49] And now we're at exhibit D. You can't [13:51] simply just parse it, you also have to [13:53] use it. Parsing is the easy thing. [13:55] If you remember, I told you that the [13:57] footnote syntax moves this grammar up to [13:59] a context-sensitive one. Let me [14:01] elucidate you. Okay, so if we have this [14:03] syntax with the carrot for a citation, [14:06] this becomes test bracket one there. But [14:09] if we actually handle a citation at the [14:11] bottom, now the markdown changes cuz [14:14] this is now a link instead. Previously, [14:17] it just put this in, now this thing [14:19] further on changes how this part here is [14:22] parsed. Oh boy. Actually though, [14:24] footnotes aren't supported in [14:25] CommonMark. They have links. The only [14:27] notable difference is that link syntax [14:29] means you can put only one word after [14:31] the definition as opposed to a shameful [14:33] explanation of why bananas on pizza is a [14:35] good idea. Reference style links and [14:37] footnotes require global definition [14:38] resolution. A token's meaning depends on [14:41] declarations elsewhere in the document. [14:43] That breaks purely context-free parsing [14:44] assumptions. Ergo, the updates to a CSG [14:47] from a CFG. I had some intuition that [14:50] markdown as as we use it was a [14:52] context-sensitive grammar, but god damn, [14:54] this is a really, really damning example [14:57] of that. [14:58] Woo. [14:59] All this a lot of words for saying, "If [15:01] you want a simple language, it should [15:02] say simple." [15:03] Rendering is the hard thing. I think [15:05] this is the most controversial part [15:06] about the author's rant because nobody [15:08] wants to admit that what we use and need [15:10] are two very different things. Vanilla [15:12] markdown needs a transliterator, which [15:14] is literally a one-to-one mapping [15:15] function. You see bold and then HTML [15:18] bold comes out. But modern markdown has [15:20] to support stuff like footnotes, which [15:22] takes this simple transliterator into a [15:24] full-blown compiler. And after you made [15:26] this step, here's the ladder that you'll [15:27] climb. I need a personal knowledge [15:29] management system. Oh boy, the Obsidian [15:31] people are going to love this. [15:33] Requirement, I'll need footnotes. Cool. [15:35] We're now at a CSG grammar. Now we've [15:37] built a compiler. Oh, you want custom [15:39] callouts? Hooks that bind HTML with [15:40] markdown? Now you have a dependency [15:42] graph. I'll need math. Well, you need [15:44] typesetting library integrations, too. [15:45] So your dependency graph just got way [15:47] more complex with execution engines as [15:49] well. And if you want custom styling, [15:51] custom CSS, you now have to deal with [15:52] CSS injections with file-based scoping [15:55] rules. This one is really fun. Having [15:57] CSS in one markdown file and making sure [15:59] it doesn't break things in others, good [16:01] luck, have fun. This is a great section. [16:03] I'm sure y'all will love. I swear what I [16:04] have to do in Obsidian work to make my [16:06] needs using markdown make it seem like [16:08] Notion made a sane decision copying Word [16:10] and Excel internally instead. [16:12] Yeah. The author thinks that Notion [16:15] seems sane because of how bad markdown [16:17] is. Do you understand how absurd that [16:19] is? Honestly, okay, here I'm going to [16:21] drop my hot take here. I like Notion's [16:23] use of markdown as hotkeys effectively, [16:27] where you can export markdown from [16:29] Notion, but the formatting of it's [16:31] broken and garbage. It's so bad. So like [16:33] don't do that. But when I'm typing in [16:35] Notion, the fact that I can just use [16:37] markdown syntax to change the element [16:39] without having to learn a whole bunch of [16:40] new hotkeys, that's really nice. Like if [16:42] I'm looking at the Notion doc for this [16:44] video, [16:45] I say sup and I decide this should [16:46] actually be a title. Hash space, now [16:48] it's a title. Oh, I want a smaller one, [16:50] two hash space, three hash space. [16:52] That's really nice. But that's a [16:54] different use case. That's not the [16:56] syntax we're using. That is a syntax or [16:59] a grammar that we're familiar with being [17:01] used effectively as a hotkey layer. Fun. [17:03] Uh Wabo just linked me async syntax in [17:07] markdown. This is JS though, but still, [17:11] this syntax for doing execution with [17:13] awaits in blockers [17:16] is so cursed. This is like view levels [17:19] of cursed, but in a markup language. [17:21] Async markdown, [17:22] not even once. [17:25] This This horrifies me. [17:27] I feel sick. Oh god, there's more? Oh, [17:29] this is DataView JS as text in Oh, no, I [17:33] want nothing to do with any of this. [17:35] Save me. Get me away. No, let's let's [17:37] see how we solve this according to the [17:39] author. You ask the guy with a majestic [17:41] beard with 30-plus years of experience [17:42] in 8086 assembly, the answer is plain [17:44] text. You ask the guy with MacBook and [17:46] OpenClaw running three different Mac [17:48] minis making a startup a day, the answer [17:50] is MDX. No, I Okay, I'm not an OpenClaw [17:53] guy with three Mac minis. I do have [17:55] three Mac minis in this apartment, but [17:56] only one is on. I even know MDX is not [17:58] the right answer here. You ask the guy [18:00] who codes in C for a living, the answer [18:01] is reStructuredText. [18:03] You ask the author, the answer is none. [18:05] All of these solutions are broken in [18:06] their own ways. Plain text is beautiful, [18:08] but I can't show it to somebody that [18:09] doesn't know what a null pointer [18:11] dereferences. reStructuredText is [18:13] wonderful if you only read it and never [18:14] write it. And MDX is so busy trying to [18:17] be HTML, it forgets it needs to be [18:18] legible. And the biggest problem in all [18:20] of them is that they don't have a build [18:22] system. We cut corners in the name of [18:24] speed. If they had a build system, a [18:26] sane, unambiguous, legible syntax [18:28] purpose-built for what it's needed to [18:29] be, I think all of the problems will be [18:31] fixed. Don't allow inline HTML, allow [18:34] for well-defined shortcodes and [18:35] functions to manipulate the text. Allow [18:37] for custom hooks to be executed before, [18:39] during, and after the compilation. Oh, [18:41] so you are in favor of async markdown. [18:43] And most importantly, define everything. [18:45] What we need is a custom-built tool, not [18:47] a Frankenstein's monster that we dare to [18:48] call a language. Not saying markdown or [18:50] markup should be a programming language, [18:52] I'm saying we are trying to use it like [18:53] one and because it lacks a formal [18:55] foundation, it's failing at both. I [18:57] genuinely think that a proper build [18:58] system around a saner markup language [19:00] with compile-time hook support can fix a [19:02] lot of the problems around this with the [19:04] right constraints. At this point, we [19:05] should just let go of markdown for good [19:07] and look for our answers elsewhere, [19:09] probably with a trivially parsable [19:10] grammar. He has a whole formal language [19:12] theory section here. Markup languages [19:15] are a standard text encoding system [19:16] consisting of a set of symbols inserted [19:18] in a text document to control its [19:20] structure, formatting, or the [19:21] relationship between its parts. And [19:23] programming languages are computer [19:25] programming languages, any of various [19:27] languages for expressing a set of [19:28] detailed instructions for digital [19:29] computer. [19:30] As you can clearly see, these official [19:32] definitions are not the way to go. So [19:33] he's going to use another separate one. [19:35] A language is a programming language if [19:36] it's Turing complete. Oh god, so [19:38] markdown is definitely a programming [19:39] language. I write JavaScript for a [19:41] living, so we're not going to get into [19:44] EBNF and context-free languages. [19:46] Articles in the description if you're [19:47] the type that cares about these depths [19:49] to Turing completeness things. God, none [19:52] of these words are in the Bible. The [19:53] thing separating a set of mathematical [19:55] axioms from provability to [19:56] non-provability is the want of recursion [19:59] self-reference, which manifests itself [20:01] as the common multiplication operation. [20:03] A trivial observation can have [20:05] non-trivial reactions. Yeah. Oh God, I [20:08] missed this. [20:09] Man, I forgot about this. This was in [20:11] 2012. Jeff Atwood wrote an article [20:14] begging Gruber to give up Markdown or at [20:17] least bless these attempts to make a [20:19] better standard. Oh man, this is [20:21] actually really good. I wish that this [20:22] happened. RIP. John Gruber's original [20:24] markdown.pl is one of the worst small [20:26] programs I've ever read, completely [20:27] riddled with outright bugs and [20:29] misfeatures that continually bite its [20:30] users in the ass. It's awful even by the [20:33] already low standards of handwritten [20:35] many past regex-based spaghetti parsers. [20:37] Nobody should be using this original [20:39] script and unfortunately many of the [20:40] other implementations out there are [20:41] direct transliterations that replicate [20:43] all of its absurd errors. Like where if [20:45] you mention the MD5 hash of another [20:46] token in the document, the hash will be [20:48] replaced with the token because it uses [20:49] that as inline escaping mechanisms. [20:51] Reddit hit with a cross-site scripting [20:53] virus that got through the filters [20:54] because of this. Jesus Christ. And [20:57] another Markdown is a disaster from last [20:59] year. [21:08] Wild. I did not know there was this [21:10] quiet cult of people fighting the chaos [21:13] that is Markdown, but it's really cool [21:15] to see and I have a lot of respect for [21:17] the author for taking the time to write [21:19] this. Thank you to Brock for writing [21:20] this. This was a fantastic read. I did [21:24] not realize how much I hated Markdown [21:26] and I use it every day. This article [21:29] very much pushed me over the edge to [21:31] more deeply appreciate how bad it is. [21:34] Sadly, this author does not have a [21:35] Twitter or anything linked so I can't [21:37] throw you at anything other than his [21:38] blog, but I would recommend checking it [21:40] out if you're nerdy about these things [21:41] and the original article has even more [21:43] fun details if you want to check it out [21:44] yourself. I got nothing else though. [21:47] I can't help but notice the irony of me [21:48] wearing this HTML shirt and I'm going to [21:50] resist the urge to go rewrite all of my [21:52] stuff in HTML or maybe a new markup [21:55] language I make up myself. Please don't [21:56] make me do this guys. T3 down is not the [21:58] solution, but [22:00] I am admitting [22:01] >> [sighs] [22:02] >> a little bit of temptation. [22:03] Until next time. [22:05] Peace nerds.  📄 **Full text:** ![[Full Document Contents/Articles/Markdown Is a Terrible Language.md|Markdown Is a Terrible Language by YouTube]] ## Highlights - will always be incredibly thankful to John Gruber and Aaron Swartz for creating something as magical as Markdown, having such a simple, reliable way to just write content and have it come out and render so well. It still has a bit of a magical feel to it. As someone who tried really hard to make LaTeX work in college, Markdown, once it clicked for me, just kind of became how I write everything to this day, which is why this article immediately grabbed my attention. Why the heck are we still using Markdown? As great as Markdown is, ([View Highlight](https://read.readwise.io/read/01kq7w81yzjcbg4y12vg4jr1bv)) - Tags: 👻 ai highlighted - a good workflow, and I'm amazed nothing else has it. And this is just the tip of the iceberg. The more you dig in, the more you'll appreciate RWX's unique way of fixing CI so that you will move faster instead of slower. Sign up for the CI that makes your agents faster at swade.link/rwx. So, why the heck are we still using Markdown? There are a few things in life that bring me much joy and heat at the same time, like a chocolate that hurts when eaten, or Markdown. Seriously, why? Half of the time we aren't even using the full language. Speaking of which, this is a bold opening. HTML is the best ([View Highlight](https://read.readwise.io/read/01kq7w81zdvgks2bf4zwvfw21h)) - Tags: 👻 ai highlighted - As you can clearly see, these official definitions are not the way to go. So he's going to use another separate one. A language is a programming language if it's Turing complete. Oh god, so markdown is definitely a programming language. I write JavaScript for a living, so we're not going to get into EBNF and context-free languages. Articles in the description if you're the type that cares about these depths to Turing completeness things. God, none of these words are in the Bible. The thing separating a set of mathematical axioms from provability to non-provability is the want of recursion self-reference, which manifests itself ([View Highlight](https://read.readwise.io/read/01kq7w821fjbhc266wq87bdq5s)) - Tags: 👻 ai highlighted