Github continues to Downfall
๐ **Source:** [Github continues to Downfall](https://www.youtube.com/watch?v=E3_95BZYIVs)
> [!note]+ Document Notes
> tags: [github-actions, microsoft, ci-cd]
> author: ThePrimeTime
> source: YouTube
> reference: https://www.youtube.com/watch?v=E3_95BZYIVs
> date created: 2024-12-19 14:30
>
> # ๐ท๏ธ Title: Github continues to Downfall
>
<!-- deep-read:sections start -->
## ๐งฐ Action Inbox
- [H1] Review GitHub Actions runner code for inefficient sleep implementations โ 00:00-05:00 #github-actions
- [H2] Monitor CI costs for zombie processes eating CPU cycles โ 08:30-12:00 #cost-optimization
- [H3] Consider alternative CI platforms like Codeberg for critical projects โ 01:30-02:00 #platform-migration
## ๐งญ Table of Contents
- [๐ Summary](#-summary)
- [๐ Main Content](#-main-content)
- [๐๏ธ Notes](#๏ธ-notes)
- [๐ References](#-references)
- [๐ฃ๏ธ Paraphrase](#๏ธ-paraphrase)
- [๐ฐ Newsletter](#-newsletter)
- [โ
Action Items](#-action-items)
- [๐งช Hypotheses](#-hypotheses)
- [๐ Methodology](#-methodology)
- [โ Questions](#-questions)
- [๐ฎ Future Research](#-future-research)
- [๐ก Implications](#-implications)
- [๐ง Learning Styles Assessment](#-learning-styles-assessment)
- [๐ Accessibility Assessment](#-accessibility-assessment)
- [๐ DEIB Assessment](#-deib-assessment)
- [โ
Logical Validity](#-logical-validity)
- [๐ก๏ธ DARVO Analysis](#-darvo-analysis)
- [๐ฏ Conclusion](#-conclusion)
### ๐ References
- [GitHub Actions Runner Repository](https://github.com/actions/runner)
- [Zig Project Migration to Codeberg](https://codeberg.org)
## ๐ Summary
- GitHub Actions runner contains a catastrophic bug in sleep functionality that causes infinite loops and wastes CPU resources
- The bug has existed for years, costing companies thousands in unnecessary compute charges
- Zig project migrated away from GitHub to Codeberg due to these persistent issues
- Microsoft/GitHub appears negligent in addressing known critical bugs despite community fixes
- The problematic code stems from Windows programmers writing bash without understanding Unix sleep utilities
## ๐ Main Content
The video exposes a critical bug in GitHub Actions runners that has persisted for years, causing infinite loops and excessive CPU usage. The issue centers around a flawed sleep implementation that uses busy-waiting instead of proper system sleep calls.
The problematic code evolved from a Stack Overflow solution for sleeping in Windows batch files, using ping commands as a timing mechanism. Over time, this became a bash function that polls the `$SECONDS` variable in a while loop, but with a fatal flaw: if the loop execution takes longer than expected, it can miss the exact target time and run forever.
One documented case shows a runner consuming 5,135 hours of compute time (costing ~$2,400) due to this bug. Despite community members providing fixes, GitHub has been slow to merge solutions, leading some major projects like Zig to abandon the platform entirely.
## ๐๏ธ Notes
#### โฑ๏ธ [00:00](https://youtu.be/E3_95BZYIVs?t=0) โ [H1] Introduction to GitHub's code quality issues
- Key idea: Criticism of GitHub Actions runner code that costs companies real money
- Evidence: "It looks so bad it looks sinister... it's actually costing real companies real money"
#### โฑ๏ธ [01:30](https://youtu.be/E3_95BZYIVs?t=90) โ [H2] Zig project abandons GitHub
- Key idea: Major project migrated to Codeberg due to GitHub Actions bugs
- Evidence: "Actions has inexcusable bugs while being completely neglected"
#### โฑ๏ธ [02:45](https://youtu.be/E3_95BZYIVs?t=165) โ [H3] The problematic four lines of code
- Key idea: Four lines of bash code described as "utterly deranged" by legendary programmer
- Evidence: "It is also utterly deranged. I have never written four lines of code and have had anybody call those four lines of code utterly deranged"
#### โฑ๏ธ [03:30](https://youtu.be/E3_95BZYIVs?t=210) โ [H2] Historical evolution of the sleep function
- Key idea: Code originated from Windows programmers adapting Stack Overflow batch file solutions
- Evidence: "This is Windows programmers programming bash... a solution from a Stack Overflow question 16 years ago"
#### โฑ๏ธ [05:15](https://youtu.be/E3_95BZYIVs?t=315) โ [H3] Original safe_sleep implementations
- Key idea: Early versions had multiple fallback mechanisms including CPU-eating echo loops
- Evidence: "It echoes sleep into what appears to be a version of dev null 5,000 times"
#### โฑ๏ธ [07:00](https://youtu.be/E3_95BZYIVs?t=420) โ [H1] The infinite loop bug mechanism
- Key idea: $SECONDS variable polling can miss exact timing and run forever
- Evidence: "If you don't perfectly land on 5,000, what ends up happening? You run forever"
#### โฑ๏ธ [08:30](https://youtu.be/E3_95BZYIVs?t=510) โ [H2] CPU consumption impact
- Key idea: Bug consumes entire CPU cores while waiting, reducing available compute
- Evidence: "You could end up running half of your available compute waiting"
#### โฑ๏ธ [09:45](https://youtu.be/E3_95BZYIVs?t=585) โ [H3] Timeline of the bug introduction
- Key idea: Problematic code was introduced on February 22nd, 2022
- Evidence: "Somehow on February 22nd, 2022... we got rid of the old safe sleep"
#### โฑ๏ธ [10:30](https://youtu.be/E3_95BZYIVs?t=630) โ [H2] Real-world cost example
- Key idea: Single runner consumed 5,135 hours costing approximately $2,400
- Evidence: "That single dead sleeping process... cost about $2400"
#### โฑ๏ธ [11:15](https://youtu.be/E3_95BZYIVs?t=675) โ [H3] Community fix rejection
- Key idea: GitHub closed community fix after one month, took 1.5 years to merge
- Evidence: "About a month later, it was automatically closed"
#### โฑ๏ธ [12:30](https://youtu.be/E3_95BZYIVs?t=750) โ [H2] Additional code quality issues
- Key idea: Other bugs exist with equally problematic code patterns
- Evidence: "Failed to hash files... zero addressing of it"
#### โฑ๏ธ [13:45](https://youtu.be/E3_95BZYIVs?t=825) โ [H3] Refactoring that introduced more bugs
- Key idea: Recent refactoring made code more complex and introduced new issues
- Evidence: "The refactor actually just made it more complicated and introduced more bugs"
#### โฑ๏ธ [14:30](https://youtu.be/E3_95BZYIVs?t=870) โ [H1] Microsoft's apparent negligence
- Key idea: Multi-billion dollar platform shows lack of care for critical infrastructure
- Evidence: "This is like a many billion dollar business and they just don't care at all"
## ๐ References
- [GitHub Actions Runner Source Code](https://github.com/actions/runner)
- [Stack Overflow Batch File Sleep Solutions](https://stackoverflow.com)
- [Zig Project on Codeberg](https://codeberg.org/ziglang)
## ๐ฃ๏ธ Paraphrase
- GitHub's Actions runner contains a sleep function that can enter infinite loops, wasting CPU resources and costing users thousands of dollars in unnecessary compute charges
- The problematic code originated from Windows developers adapting outdated Stack Overflow solutions for Unix environments without proper understanding of system utilities
- Despite community members identifying and providing fixes for these critical bugs, GitHub/Microsoft has shown negligence in addressing them promptly, leading major projects to abandon the platform
## ๐ฐ Newsletter
**GitHub Actions Bug Costing Developers Thousands in Wasted Compute**
A critical bug in GitHub Actions runners has been silently draining CI budgets for years. The issue? A flawed sleep implementation that can enter infinite loops, consuming entire CPU cores while accomplishing nothing.
The problematic code uses busy-waiting instead of proper system sleep calls, occasionally missing timing windows and running forever. One documented case consumed over 5,000 hours of compute time, costing approximately $2,400 in unnecessary charges.
**Key Takeaway:** Review your GitHub Actions usage for unexpectedly long-running jobs and consider monitoring tools to catch zombie processes.
**What You Can Do:**
- Audit recent Action runs for unusually long execution times
- Consider alternative CI platforms for critical workflows
- Implement monitoring alerts for runaway processes
This highlights the importance of proper code review and testing, even for basic utility functions. When building CI/CD pipelines, always implement proper timeouts and monitoring.
Resources: [GitHub Actions Documentation](https://docs.github.com/en/actions), [CI/CD Best Practices](https://github.blog), [Alternative CI Platforms](https://about.gitlab.com/stages-devops-lifecycle/continuous-integration/)
## โ
Action Items
#### DevRel
- Developer advocacy: Create content warning about GitHub Actions sleep bugs, demonstrate proper CI monitoring
- Education: Workshop on CI cost optimization and zombie process detection
- Developer experience: Document alternative CI platforms and migration guides
- Developer marketing: Share cost-saving tips for CI/CD workflows
- Community management: Collect feedback on CI platform reliability experiences
#### OSPO
- Policy & Governance: Establish CI platform evaluation criteria and backup strategies
- License Compliance: Review CI platform terms for cost liability clauses
- Security Management: Audit CI runners for resource exhaustion vulnerabilities
- Software Inventory: Catalog CI dependencies and platform lock-in risks
- Developer Support: Training on CI debugging and cost monitoring tools
- Community Engagement: Coordinate with CI platform vendors on reliability improvements
- Strategic Alignment: KPIs for CI cost efficiency and reliability metrics
#### Community
- Maintainers: Provide templates for CI monitoring and cost alerts
- Contributors: Onboarding includes CI best practices and budget awareness
- Sponsors: Transparent reporting on CI costs and optimization efforts
- Ecosystem: Cross-project coordination on CI platform experiences
#### WordPress
- Core: Audit GitHub Actions workflows for potential zombie processes
- Core Performance: Monitor CI resource usage and execution times
- Community: Share CI cost optimization strategies with meetup organizers
- Training: Lesson plans on CI/CD platform selection and monitoring
- Documentation: DevHub guidance on CI best practices and cost management
- Meta: .org infrastructure CI reliability and cost monitoring
## ๐งช Hypotheses
- GitHub's market dominance reduces incentive to fix non-critical infrastructure bugs quickly
- Windows-centric development culture at Microsoft contributes to Unix/Linux platform bugs
- CI platform costs are often hidden from developers, reducing pressure for efficiency optimization
- Large-scale infrastructure bugs persist longer in proprietary platforms than open-source alternatives
### ๐ Methodology
- Comparative analysis of bug resolution times across different CI platforms
- Cost analysis of zombie processes across different runner configurations
- Survey of developer awareness regarding CI cost attribution and monitoring
- Code quality assessment comparing Microsoft's open-source vs proprietary projects
### โ Questions
- How many other critical infrastructure bugs exist in GitHub Actions that remain undiagnosed?
- What percentage of GitHub Actions costs are attributed to inefficient or buggy runner code?
- How do bug resolution times compare between GitHub's free and enterprise tiers?
### ๐ฎ Future Research
- Comprehensive audit of GitHub Actions runner codebase for similar efficiency issues
- Analysis of CI platform reliability and cost-effectiveness across the industry
- Investigation of Microsoft's internal code review processes for open-source infrastructure
### ๐ก Implications
- Practice: Implement CI monitoring and alerting to catch runaway processes early
- Policy: Establish CI platform evaluation criteria including reliability and support responsiveness
- Security: Resource exhaustion attacks via CI platforms may be more viable than previously considered
- Education: Developer training must include CI cost awareness and monitoring techniques
- Funding: CI costs may represent hidden technical debt that compounds over time without proper oversight
### ๐ง Learning Styles Assessment
Reading level estimate: 10th Grade (Flesch-Kincaid)
#### ๐ Visual learners
- Present: Code examples, before/after comparisons, cost calculations
- Improve: Architecture diagrams showing runner resource consumption, visual timeline of bug evolution
#### ๐ Aural learners
- Present: Video format with narrated code explanation
- Improve: Podcast version focusing on lessons learned, audio summary of key takeaways
#### ๐ฃ๏ธ Verbal learners
- Present: Clear analogies (comparing to real-world cost scenarios), defined technical terms
- Improve: Story-driven explanation of how bugs evolve in large codebases
#### ๐งฉ Kinesthetic learners
- Present: Code examples that can be tested and modified
- Improve: Hands-on lab for reproducing the bug, checklist for auditing CI workflows
#### ๐ง Logical learners
- Present: Timeline of bug introduction and fixes, cost calculations, cause-and-effect analysis
- Improve: Decision tree for CI platform selection, systematic debugging methodology
#### ๐งโ๐คโ๐ง Social learners
- Present: Community discussion around platform migration decisions
- Improve: Forum for sharing CI horror stories and solutions, peer review of CI configurations
#### ๐งโโ๏ธ Solitary learners
- Present: Self-contained code analysis and examples
- Improve: Self-assessment worksheet for CI efficiency, personal CI cost tracking template
#### ๐ฟ Naturalistic learners
- Present: Connection between code efficiency and environmental impact of compute waste
- Improve: Lifecycle analysis of CI resource consumption, sustainability implications of inefficient code
### ๐ Accessibility Assessment
#### Screen reader semantics
- Present: Clear heading hierarchy, code blocks properly labeled
- Improve: Alt text for code screenshots, better semantic markup for code explanations
#### Keyboard navigation
- Present: Video player supports keyboard controls
- Improve: Timestamped navigation links, skip-to-code functionality
#### Contrast and readability
- Present: High contrast code examples in video
- Improve: Transcript with proper formatting, adjustable text size for code blocks
#### Text resizing and reflow
- Present: YouTube player supports zoom
- Improve: Responsive transcript layout, code examples that reflow properly
#### Media and captions
- Present: Auto-generated captions available
- Improve: Professional captions with technical term accuracy, downloadable transcript
#### Forms and interactions
- Present: Standard YouTube interface
- Improve: Custom controls with better labeling, accessible sharing options
#### Standards reference
- Video content meets WCAG 2.2 AA for media with transcript availability
### ๐ DEIB Assessment
#### Cultural breadth
- Present: Technical content applicable globally
- Improve: Examples from diverse CI usage scenarios, international cost perspectives
#### Inclusive language
- Present: Generally neutral technical language
- Improve: Avoid dismissive language about programming skill levels, more supportive tone
#### Neurodiversity support
- Present: Clear structure with timestamped sections
- Improve: Content warnings for rapid code switching, summary sections for key points
#### Representation
- Present: Focus on technical merit rather than personal characteristics
- Improve: Acknowledge diverse backgrounds in development teams, credit community contributors
#### Psychological safety
- Present: Criticism directed at corporate practices rather than individuals
- Improve: Emphasize learning opportunities rather than just criticism, acknowledge complexity of large system maintenance
### โ
Logical Validity
#### Strong reasoning
- Deductive strength: Cost calculations based on documented compute rates and measured usage
- Inductive strength: Pattern recognition from historical code evolution and bug introduction
- Causal reasoning: Clear connection between code behavior and resource consumption
- Analogical strength: Comparison to other infrastructure reliability issues
#### Fallacies
- Ad hominem: Some attacks on Microsoft's competence rather than focusing purely on technical issues
- Hasty generalization: Extrapolating from one bug to overall platform quality without comprehensive analysis
#### Tone of voice
- Overall tone: Humorous but critical, occasionally inflammatory when discussing corporate responsibility
### ๐ก๏ธ DARVO Analysis
- Deny: No significant denial patterns detected
- Attack: Criticism directed at corporate practices rather than individuals making accusations
- Reverse victim/offender: No victim/offender reversal present
- Emotional cues and rhetoric: Uses humor and frustration to highlight technical incompetence, appropriate given context
## ๐ฏ
---
## Full Transcript
[00:00] Now, I do not like making fun of<br>[00:01] people's code, okay? We've all been<br>[00:03] there. We've all written code under<br>[00:05] duress, shall we say, and it's not<br>[00:07] impressive. You know, caught absolute<br>[00:10] cinema. I've been caught on many a PR<br>[00:13] trying to sneak in some just truly<br>[00:14] heinous lines of code. So, therefore,<br>[00:16] I'm not going to make fun of some random<br>[00:19] person's code, except when that code<br>[00:21] belongs to one of the world's largest<br>[00:22] corporations. And for years, people have<br>[00:25] been begging them to change the code.<br>[00:27] And they have effectively done nothing<br>[00:29] ever, including addressing or merging<br>[00:31] fixes in which is actually costing real<br>[00:34] companies real money. It looks so bad it<br>[00:38] looks sinister. Now, I'm not saying that<br>[00:40] they're intentionally keeping this code<br>[00:42] bad to make money. Okay? I'm not I would<br>[00:45] never suggest that. What I am saying is<br>[00:48] that it just looks really bad. Okay? It<br>[00:49] looks real bad. But first, I'm at<br>[00:52] 977,000<br>[00:54] subs. Okay, I don't want to say this.<br>[00:56] Quit making me say this. Just press the<br>[00:57] button. I am less than 23,000 subs away<br>[01:00] from making it to the gold. Okay, I get<br>[01:02] the gold plaque. Just make it happen.<br>[01:04] Make my dreams come true. The code<br>[01:05] that's making everybody upset is, of<br>[01:07] course, GitHub action runners. Now,<br>[01:09] apparently, this has just been limping<br>[01:12] for almost a decade now. So many bugs.<br>[01:15] And the bug that we're going to be<br>[01:16] looking at today was so egregious that<br>[01:19] Zigg has decided to leave GitHub<br>[01:22] altogether because of it and moved to<br>[01:24] Codeberg. In fact, they say right here,<br>[01:26] most importantly, actions has<br>[01:28] inexcusable bugs while being completely<br>[01:30] neglected. After the CEO said to embrace<br>[01:33] AI or get out, it seems the lackis at<br>[01:35] Microsoft took the hint because GitHub<br>[01:36] actions started vibe scheduling,<br>[01:38] choosing jobs to run seemingly at random<br>[01:40] combined with other bugs and inability<br>[01:42] to manually intervene. This causes our<br>[01:44] CI system to get so backed up that not<br>[01:47] even master branch commits can get<br>[01:49] checked. Okay, so what is the big<br>[01:51] problem? What is actually causing this<br>[01:52] huge gigantic just feathers being<br>[01:55] ruffled thoroughly? Well, it comes down<br>[01:57] to these simple four lines of code right<br>[02:00] here. Now, these four lines of code, you<br>[02:02] may not like this, but this is not what<br>[02:03] peak engineering looks like. Okay, this<br>[02:05] is some disgusting code. In fact,<br>[02:08] Mattlad, one of the most legendary uh<br>[02:11] programmers of all time, I believe he<br>[02:13] works at antithesis. He says the<br>[02:14] following about that function. It has<br>[02:16] clear formatting. It's short. It uses<br>[02:18] fancy bash features. The second feature,<br>[02:20] it has the elegant colon as the while<br>[02:22] body. It really reads nice. It is also<br>[02:25] utterly deranged.<br>[02:28] I have never written four lines of code<br>[02:31] and have had anybody call those four<br>[02:33] lines of code utterly deranged. and I<br>[02:35] have written some deranged code. It is<br>[02:38] truly impressive. Before you can<br>[02:40] understand those four lines of code, we<br>[02:42] need to do a bit kind of a historical<br>[02:44] deep dive into Microsoft ActionRunner.<br>[02:46] Trust me, it's worth it cuz it's<br>[02:48] hilarious the transfer. So, it all<br>[02:49] starts with this. This is the first<br>[02:51] public commit known as the GitHub action<br>[02:54] runner. And in here, we find this<br>[02:56] beautiful piece of code. So, now what is<br>[02:58] going on here? As best as I can tell,<br>[03:00] this is Windows programmers programming<br>[03:03] bash. And the key feature is is this<br>[03:05] right here. It's actually a solution<br>[03:07] from a Stack Overflow question 16 years<br>[03:10] ago on how to sleep for 5 seconds in a<br>[03:13] batch file, a command file on Windows<br>[03:15] because timeout some sometimes just<br>[03:17] wasn't available and someone suggests<br>[03:19] using ping as a means to sleep for 5<br>[03:23] seconds. So let's look at this code. So<br>[03:25] apparently if it returns a four, the<br>[03:27] code needed to do some sort of sleep.<br>[03:29] Now for whatever reason they were on<br>[03:30] machines that did not have the sleep<br>[03:32] command. Now, I actually don't even<br>[03:33] believe this cuz there is extremely few<br>[03:35] edge cases from 2010 to 2020 where sleep<br>[03:38] was not available and likely Microsoft<br>[03:41] had none of them other than if you're<br>[03:43] using Windows during the weasel one<br>[03:45] days. That was kind of it. So, if you<br>[03:47] had sleep, it'd sleep for 5 seconds. If<br>[03:49] you didn't have sleep, but you had ping,<br>[03:51] it would sleep for 4 seconds cuz<br>[03:53] remember, ping sends off a packet every<br>[03:55] 1 second. So, with five packets being<br>[03:57] sent, that means at second zero you send<br>[03:59] one. Second one, you send your second.<br>[04:01] Second two, you send your third. Soda<br>[04:03] only sleeps for 4 seconds. But the truly<br>[04:05] deranged one is this right here. It<br>[04:06] echoes sleep into what appears to be a<br>[04:08] version of dev null 5,000 times.<br>[04:11] Assuming that echoing takes 1<br>[04:13] millisecond. Now, I showed you that<br>[04:15] because apparently that function was so<br>[04:17] useful, so dynamite that someone decided<br>[04:19] that they needed it as a top level<br>[04:21] function called safe sleep, which again<br>[04:24] calls sleep for 5 seconds, ping for 4<br>[04:26] seconds, or literally eats your CPU, and<br>[04:29] tries to run 5,000 times, echoing to<br>[04:32] devnull. This is some truly deranged<br>[04:34] code. But somehow this code gets worse<br>[04:36] because it finally gets to its final<br>[04:38] form, this one right here. And it<br>[04:40] remained this way for years, costing<br>[04:42] companies and people who pay for GitHub<br>[04:44] actions thousands upon tens of<br>[04:46] thousands, if not hundreds of thousands<br>[04:48] of dollars. This is single-handedly a<br>[04:50] line of code that has probably eaten<br>[04:52] more energy than AI itself. Okay, that<br>[04:55] was a bit of an exaggeration, but you<br>[04:57] but just just see. Okay, so to<br>[04:59] understand this code, first you got to<br>[05:01] understand what this does. Seconds is a<br>[05:03] special bash variable that increments as<br>[05:05] time goes on. So if I set it to zero, it<br>[05:08] will slowly become one, then two, then<br>[05:10] three as time goes on on the system. So<br>[05:12] it is a way to be able to sleep.<br>[05:15] Obviously, dollar one is the value<br>[05:17] that's passed into the script. And so<br>[05:19] what happens here is this is equivalent<br>[05:21] to doing effectively like a while date.<br>[05:23] Now minus start is greater than 5,000.<br>[05:26] To make it easier for you JS Andes out<br>[05:28] there, this is effectively what's going<br>[05:30] on. We take a start time and then we<br>[05:33] just slowly wait until that exceeds the<br>[05:35] amount of time being passed in. In this<br>[05:37] example, I just put 5,000, but in the<br>[05:39] case of this code, it's whatever is<br>[05:40] being passed in. Now, you're probably<br>[05:42] thinking, okay, so what's the big deal?<br>[05:44] What's the big problem? What if I change<br>[05:46] this code to look something like that?<br>[05:49] Would you be able to easily spot the<br>[05:51] problem then? Well, what happen if<br>[05:53] there's longer than a single millisecond<br>[05:55] here between the two to the point where<br>[05:58] it would just run forever? Right?<br>[06:00] Because if you don't perfectly land on<br>[06:02] 5,000, what ends up happening? You run<br>[06:04] forever. Well, this code did the exact<br>[06:06] same thing because you're on a machine<br>[06:08] that's running long and difficult jobs.<br>[06:10] And so what would end up happening? You<br>[06:12] could end up having more than 1 second<br>[06:14] wait in between this while loop actually<br>[06:16] executing. And the crazy part is is that<br>[06:19] if there is no wait and the while loop<br>[06:21] is running, it just eats an entire CPU<br>[06:23] just waiting for 5 seconds so that<br>[06:26] nothing else can use that CPU. Now,<br>[06:28] considering that the average size of a<br>[06:30] GitHub runner is only two CPUs, you<br>[06:32] could end up running half of your<br>[06:34] available compute waiting, which means<br>[06:37] that it would actually cause more of<br>[06:38] these to overrun over time. And that's<br>[06:41] exactly what it did. So, somehow on<br>[06:44] February 22nd, 2022, a lot of twos in<br>[06:47] that. Okay, I love how many twos there<br>[06:48] are. Somehow we got rid of the old safe<br>[06:50] sleep in which actually did at least<br>[06:53] sleep most of the time and used the<br>[06:56] system utility when available to have<br>[06:58] the brand new insane bugridden<br>[07:01] absolutely destroying people's CI<br>[07:03] environment code. I think the thing that<br>[07:05] hurts more is that in 2024 Mr. over here<br>[07:09] fixes it by simply making it do a less<br>[07:12] than or equal to operation as opposed to<br>[07:15] just simply having a not equals to. This<br>[07:18] would fix all the hanging forever<br>[07:19] because as he puts a picture right here,<br>[07:21] this single GitHub action runner ran for<br>[07:24] 5,135<br>[07:26] hours just running forever on one of his<br>[07:29] machines. Now, GitHub CI does cost based<br>[07:32] on CPU commit. The cheapest machine is8<br>[07:35] per CPU minute. That means that that<br>[07:37] single dead sleeping process which was<br>[07:39] burning the CPU at 100% cost about<br>[07:42] $2400.<br>[07:44] Absolutely<br>[07:46] beautiful. We're talking about cinema<br>[07:48] going on right now. Now, with this<br>[07:50] incredibly lovely fix that of course is<br>[07:52] going to be able to save the whole<br>[07:54] GitHub CI running thing, you you would<br>[07:56] assume that this would get merged in<br>[07:57] relatively quick, right? No, about a<br>[07:59] month later, it was automatically closed<br>[08:00] and then after this whole problem became<br>[08:03] an actual issue a year and a half later,<br>[08:07] they have decided, okay, it is good to<br>[08:09] merge in this gigantic fix and which<br>[08:11] fixes the whole zombie process of this<br>[08:13] forever running loop. Now, there's<br>[08:15] plenty of other issues. I did just want<br>[08:16] to highlight this for a quick second.<br>[08:17] Failed to hash files. It is due to this<br>[08:20] bug that is in this diff that was merged<br>[08:22] in two weeks ago and which has had zero<br>[08:24] addressing of it. But look at this code<br>[08:26] that they're dealing with. If for if<br>[08:30] like this code is disgusting so they<br>[08:32] refactored into something nice, right?<br>[08:34] Okay. So it does get own property names<br>[08:36] or it's a function which does for and if<br>[08:38] right here and then returns this own<br>[08:40] keys. Just absolutely insane code going<br>[08:43] on right here that returns another<br>[08:44] function. Then inside here does an if<br>[08:46] four and then an if again. It has the<br>[08:48] same code but multiple times. It is just<br>[08:50] absolutely horrifying code that somehow<br>[08:53] the refactor actually just made it more<br>[08:55] complicated and introduced more bugs.<br>[08:58] What is going on with GitHub actions?<br>[09:01] Who approves this type of code? This<br>[09:04] should not be approved. This is not<br>[09:05] okay. Anyways, I wanted to highlight<br>[09:07] some of this code because the fact that<br>[09:09] there has been over three years with<br>[09:11] this bug that has just been destroying<br>[09:14] CI jobs, causing so many problems in<br>[09:17] life, and costing thousands upon<br>[09:19] thousands of dollars for no reason other<br>[09:20] than a bad while loop is insane that it<br>[09:24] was not diagnosed earlier. GitHub<br>[09:26] actions are single-handedly<br>[09:29] one of the most insane platforms ever<br>[09:32] when you look at the code. And somehow<br>[09:34] this runs countless projects. You know,<br>[09:36] I think I've been a little too harsh on<br>[09:38] the old AI coding because when I look at<br>[09:40] code like this, it restores my faith in<br>[09:42] AI<br>[09:44] because there's just absolutely no way<br>[09:46] that an AI could be this creative to be<br>[09:49] able to create code this bad. Truly a<br>[09:52] work of art. The thing that kills me the<br>[09:54] most is the fact that this is like a<br>[09:56] many billion dollar business and they<br>[09:58] just don't care at all. Microsoft just<br>[10:01] does not care. And if they do care,<br>[10:03] they're really really selling us that<br>[10:05] they don't care. They're like acting<br>[10:06] like the cool kid that just doesn't<br>[10:07] care. I don't care about nothing. I'm<br>[10:10] just too cool. No, your product sucks.<br>[10:11] You should fix it. Okay. Literally,<br>[10:14] millions of people rely on this thing to<br>[10:15] be good. You should fix it. You should<br>[10:17] make that nice. That's all I have to<br>[10:18] say. It's another day that I get to make<br>[10:20] fun of Microsoft. So, it's a day I'm<br>[10:22] going to take advantage of. The name<br>[10:26] is I don't really like Microsoft, but<br>[10:28] also you should hit subscribe because<br>[10:29] we're almost at a million. Okay, I'm<br>[10:30] going to stop saying it afterwards. A<br>[10:32] gen. Hey, do you want to learn how to<br>[10:34] code? Do you want to become a better<br>[10:36] back-end engineer? Well, you got to<br>[10:37] check out boot.dev. Now, I personally<br>[10:39] have made a couple courses from them. I<br>[10:40] have live walkthroughs free available on<br>[10:43] YouTube of the whole course. Everything<br>[10:45] on boot.dev you can go through for free.<br>[10:48] But if you want the gamified experience,<br>[10:49] the tracking of your learning and all<br>[10:51] that, then you got to pay up the money.<br>[10:52] But hey, go check them out. It's<br>[10:54] awesome. Many content creators you know<br>[10:56] and you like make courses there.<br>[10:58] boot.dev/prime dev/prime for 25% off.
<!-- deep-read:sections end -->