PHP barely avoided disaster ![rw-book-cover](https://img.youtube.com/vi/dzQEe7u3k2I/maxresdefault.jpg) ๐Ÿ”— **Source:** [PHP barely avoided disaster](https://www.youtube.com/watch?v=dzQEe7u3k2I) > [!note]+ Document Notes > tags: [supply-chain-security, php-ecosystem, github-actions] > author: [MISSING] > source: YouTube > reference: https://www.youtube.com/watch?v=dzQEe7u3k2I > date created: {{date:YYYY-MM-DD}} {{time:HH:mm}} > > # ๐Ÿท๏ธ Title: PHP barely avoided disaster > <!-- deep-read:sections start --> ## ๐Ÿงญ Table of Contents - [๐Ÿ“ Summary](#-summary) - [๐Ÿ“– Main Content](#-main-content) - [๐Ÿ—’๏ธ Notes](#๏ธ-notes) - [๐Ÿ“š References](#-references) - [๐Ÿ—ฃ๏ธ Paraphrase](#๏ธ-paraphrase) - [๐Ÿ“ฐ Newsletter](#-newsletter) - [โœ… Action Items](#-action-items) - [๐Ÿงช Hypotheses](#-hypotheses) - [๐Ÿ“Š Methodology](#-methodology) - [โ“ Questions](#-questions) - [๐Ÿ”ฎ Future Research](#-future-research) - [๐Ÿ’ก Implications](#-implications) - [๐Ÿง  Learning Styles Assessment](#-learning-styles-assessment) - [๐Ÿ“Š Accessibility Assessment](#-accessibility-assessment) - [๐ŸŒ DEIB Assessment](#-deib-assessment) - [โœ… Logical Validity](#-logical-validity) - [๐Ÿ›ก๏ธ DARVO Analysis](#-darvo-analysis) - [๐ŸŽฏ Conclusion](#-conclusion) ## ๐Ÿ“ Summary - A 14-hour window exposed GitHub tokens in Composer logs, creating massive supply chain vulnerability for PHP ecosystem - Damien discovered the issue while debugging GitHub Actions, found tokens leaking in plain text due to new GitHub token format incompatible with Composer validation - The vulnerability could have enabled attackers to push malicious code to any PHP package, potentially affecting hundreds of thousands of installs daily - Fast response from maintainers and gradual rollout prevented exploitation, but highlighted critical gaps in cross-ecosystem security communication - Bottom line: PHP dodged a potential top-3 supply chain attack through luck and quick community response, exposing structural weaknesses in disclosure processes ## ๐Ÿ“– Main Content The PHP ecosystem narrowly avoided what could have been its largest supply chain attack in history. For 14 hours, any PHP project using the popular `shivammathur/setup-php` GitHub Action had its GitHub tokens exposed in plain text logs whenever `composer install` failed. ### The Technical Issue GitHub was rolling out a new token format for app installation tokens. Composer's 13-year-old token validation code couldn't handle the new format and threw an exception that included the invalid token in the error message. When these tokens appeared in GitHub Actions logs, they became publicly visible. The exposed tokens typically had write permissions, allowing attackers to: - Push commits and tags to repositories - Trigger automatic package releases through Packagist - Execute code on every developer machine and CI runner that pulled the poisoned packages ### Discovery and Response Damien, who maintains GitHub Actions for the Magento community, discovered the issue while debugging failing tests. After realizing the scope, he took both official and unofficial channels: - Filed security advisories with Composer and GitHub - Reached out directly to major framework maintainers through Discord, Slack, and personal connections - Asked Laravel, Magento, and other communities to pause their GitHub Actions ### Mitigating Factors Several factors prevented widespread exploitation: - **Composer failed fast**: Jobs crashed immediately, revoking tokens quickly - **Visible failures**: The bug was loud, appearing in CI logs across major PHP repositories - **Quick patches**: Composer maintainers released fixes within hours - **Gradual rollout**: Not all repositories were affected simultaneously - **Branch protection**: Many repositories had protections that would have blocked direct pushes to main ### Structural Problems Revealed The incident exposed a critical gap in cross-ecosystem security communication. There's no established protocol for reporting vulnerabilities that affect multiple vendors and thousands of projects simultaneously. Damien had to rely on personal networks and informal channels to reach framework maintainers quickly. ## ๐Ÿ—’๏ธ Notes #### โฑ๏ธ [00:00:00](https://www.youtube.com/watch?v=dzQEe7u3k2I?t=0) Opening - scope of near-disaster - PHP community barely avoided "absolute disaster" due to supply chain vulnerability in Composer - Happened week after previous package compromise incident #### โฑ๏ธ [00:01:30](https://www.youtube.com/watch?v=dzQEe7u3k2I?t=90) The vulnerability window - 14-hour exposure window for any PHP project on GitHub using `shivammathur/setup-php` action - GitHub tokens with write permissions publicly logged in GitHub Actions - Potential for largest supply chain breach in PHP history #### โฑ๏ธ [00:03:45](https://www.youtube.com/watch?v=dzQEe7u3k2I?t=225) Scale of potential impact - Laravel alone: ~300,000 installs per day, ~150,000 potentially affected during window - Automatic releases through Composer mean immediate propagation of malicious code #### โฑ๏ธ [00:05:20](https://www.youtube.com/watch?v=dzQEe7u3k2I?t=320) Discovery process - Damien maintains complex GitHub Actions for Magento community - Intermittent test failures led to discovering tokens in error messages - Initial assumption was personal scripting error #### โฑ๏ธ [00:08:15](https://www.youtube.com/watch?v=dzQEe7u3k2I?t=495) Root cause analysis with Claude - Claude AI helped identify the specific Composer code throwing exception with token - Traced to 13-year-old token validation code that couldn't handle new GitHub token format - GitHub was rolling out new format for app installation tokens #### โฑ๏ธ [00:11:30](https://www.youtube.com/watch?v=dzQEe7u3k2I?t=690) Emergency response actions - Disabled all actions in organization immediately - Contacted major PHP communities through Discord, Slack - Filed security advisories while simultaneously using informal channels - Reached out to Laravel team directly through connections #### โฑ๏ธ [00:15:45](https://www.youtube.com/watch?v=dzQEe7u3k2I?t=945) Why multi-channel disclosure was necessary - Traditional responsible disclosure too slow for ecosystem-wide threat - No shared security inbox for PHP ecosystem - Time pressure: every minute increased risk of exploitation #### โฑ๏ธ [00:18:30](https://www.youtube.com/watch?v=dzQEe7u3k2I?t=1110) Attack window mechanics - Token live for seconds between error message and job teardown - Proof of concept showed exploitation possible in under one second with sophisticated polling - Upload artifacts could extend token lifetime to many minutes #### โฑ๏ธ [00:22:00](https://www.youtube.com/watch?v=dzQEe7u3k2I?t=1320) Potential attack scenarios - Push access to repo = push access to packages via Packagist - Poisoned tags automatically published and distributed - Could have been top-3 supply chain incident across all ecosystems #### โฑ๏ธ [00:25:15](https://www.youtube.com/watch?v=dzQEe7u3k2I?t=1515) Mitigating factors - Composer's fast failure and hard crash - Loud, visible failures across major repositories - Quick response from maintainers - GitHub token scope limitations (repo-specific) - Branch protection rules blocking direct main pushes #### โฑ๏ธ [00:28:45](https://www.youtube.com/watch?v=dzQEe7u3k2I?t=1725) Recommended actions - Upgrade Composer via `composer self-update` - Tighten GitHub Actions permissions - Protect tags with GitHub's protection rules - Review logs and history for the vulnerability window #### โฑ๏ธ [00:32:00](https://www.youtube.com/watch?v=dzQEe7u3k2I?t=1920) Structural disclosure problems - No ecosystem-wide security communication protocol - Reliance on personal networks and informal channels - Need for "phone number for ecosystem-scale incidents" - AI making discovery-to-abuse window shorter #### โฑ๏ธ [00:35:30](https://www.youtube.com/watch?v=dzQEe7u3k2I?t=2130) Composer maintainer response - Jordi confirmed 13-year-old validation code was root cause - Client-side token validation no longer relevant in modern automated world - PHP Foundation working on improved disclosure processes ## ๐Ÿ“š References - [Original blog post by Damien](referenced in video but URL not provided) - [shivammathur/setup-php GitHub Action](https://github.com/shivammathur/setup-php) - [Composer project](https://getcomposer.org) - [Packagist package repository](https://packagist.org) ## ๐Ÿ—ฃ๏ธ Paraphrase A critical vulnerability in the PHP ecosystem was discovered when GitHub began rolling out new token formats that were incompatible with Composer's token validation code. For 14 hours, any PHP project using popular GitHub Actions would have their repository tokens exposed in plain text when Composer installations failed. This created an unprecedented opportunity for supply chain attacks, as malicious actors could have used these tokens to push infected code to major packages like Laravel, which would then be automatically distributed to hundreds of thousands of developers. The vulnerability was caught and patched quickly, but it revealed serious gaps in how the PHP community handles ecosystem-wide security incidents. ## ๐Ÿ“ฐ Newsletter ๐Ÿšจ **PHP's Close Call: A Supply Chain Security Wake-Up Call** Last week, the PHP community dodged what could have been the largest supply chain attack in its history. A vulnerability in Composer exposed GitHub tokens in CI logs, potentially allowing attackers to poison popular packages and execute code on hundreds of thousands of developer machines. **What happened:** GitHub's new token format broke Composer's 13-year-old validation code, causing tokens to leak in error messages during failed builds. **Why it matters for WordPress:** Many WordPress plugins and tools rely on Composer packages. A successful attack could have affected the entire ecosystem. **Key takeaways:** - Update Composer with `composer self-update` - Review your GitHub Actions permissions (use principle of least privilege) - Enable tag protection rules on critical repositories - Consider your dependency hygiene practices The incident highlights how interconnected our development tools have become. It also revealed gaps in cross-project security communication that the PHP Foundation is now working to address. **Learn more:** Check out the security settings in your GitHub repositories and consider which packages you truly need in your projects. A little dependency hygiene goes a long way! ๐Ÿ›ก๏ธ ## โœ… Action Items #### DevRel - Create educational content about supply chain security best practices for PHP/WordPress developers - Develop workshops on GitHub Actions security configuration and token management - Build examples and templates showing secure CI/CD patterns with minimal permissions - Track and measure developer adoption of security best practices through surveys and metrics #### OSPO - Establish incident response procedures for ecosystem-wide vulnerabilities affecting multiple vendors - Create scanning and monitoring for exposed tokens in public CI logs across organization repositories - Develop cross-project communication channels for security incident coordination - Implement automated SBOM generation and dependency vulnerability tracking #### Community - Establish security working group with representatives from major PHP frameworks and tools - Create shared security disclosure protocols for ecosystem-wide incidents - Develop community-wide security education initiatives and resources - Set up emergency communication channels for critical security incidents #### WordPress - **Core:** Review GitHub Actions configurations for token security and minimal permissions - **Security Team:** Assess WordPress.org infrastructure for similar token exposure vulnerabilities - **Plugin Review:** Develop guidelines for secure CI/CD practices in plugin submissions - **Documentation:** Create security guidance for plugin and theme developers using GitHub Actions - **Meta:** Audit WordPress.org build and deployment processes for token management best practices #### The WPCC - Fund security audits of critical WordPress ecosystem dependencies and build tools - Support development of shared security infrastructure for WordPress vendor coordination - Create grants for security tooling and vulnerability disclosure platform development ### FAIR - Establish vendor-neutral security coordination protocols with transparent funding and governance - Develop technical standards for secure package management and distribution - Create policy frameworks for responsible disclosure across multiple vendor ecosystems ## ๐Ÿงช Hypotheses - Cross-ecosystem security incidents will become more frequent as development tools become more interconnected - Traditional single-vendor responsible disclosure models are inadequate for modern supply chain vulnerabilities - Automated token scanning in CI/CD platforms could prevent similar incidents if implemented proactively - Community-driven security coordination may be more effective than vendor-led approaches for open source ecosystems - The window between vulnerability discovery and exploitation will continue to shrink due to AI-assisted attack automation ### ๐Ÿ“Š Methodology - **Historical analysis**: Compare incident response times across similar supply chain vulnerabilities in different ecosystems - **Network analysis**: Map communication pathways used during the PHP incident vs. official disclosure channels - **Simulation studies**: Test effectiveness of proposed cross-ecosystem security coordination protocols - **Survey research**: Assess developer awareness and adoption of supply chain security practices - **Quantitative analysis**: Measure token exposure frequency across popular CI/CD configurations ### โ“ Questions - How many similar token validation vulnerabilities exist in other package managers and build tools? - What would be the optimal structure for a cross-ecosystem security coordination body? - How can we balance speed of incident response with responsible disclosure principles? - What automated monitoring could detect supply chain attack attempts in real-time? - How do we establish trust and verification in emergency security communications? ### ๐Ÿ”ฎ Future Research - Development of AI-powered supply chain vulnerability detection systems - Cross-ecosystem incident response protocol standardization - Token lifecycle management best practices for CI/CD environments - Community-scale security coordination governance models - Real-time supply chain attack detection and prevention mechanisms ### ๐Ÿ’ก Implications - **Practice**: Immediate need for dependency hygiene audits and CI/CD security reviews across all projects - **Policy**: Requirements for coordinated disclosure protocols in open source foundation governance - **Security**: Investment in automated token scanning and supply chain monitoring infrastructure - **Education**: Integration of supply chain security into developer training curricula and certification programs - **Funding**: Support for shared security infrastructure and cross-project coordination platforms ### ๐Ÿง  Learning Styles Assessment Reading level: Flesch-Kincaid Grade 12 - technical content with specialized terminology but clear explanations #### ๐Ÿ‘€ Visual learners **Present**: Timeline of incident response, token exposure window diagrams, supply chain attack flow charts **Improve**: Architecture diagrams showing GitHub Actions โ†’ Composer โ†’ Packagist flow, before/after security configurations, visual incident response playbooks #### ๐Ÿ”Š Aural learners **Present**: Video format with clear narration and explanations **Improve**: Audio summary of key security recommendations, podcast interview with incident responders, narrated walkthroughs of security configurations #### ๐Ÿ—ฃ๏ธ Verbal learners **Present**: Clear definitions of technical terms, analogies comparing to physical supply chains **Improve**: Glossary of security terms, story-driven examples of attack scenarios, plain-language security guidance #### ๐Ÿงฉ Kinesthetic learners **Present**: Step-by-step GitHub Actions configuration examples **Improve**: Hands-on security audit checklists, interactive vulnerability assessment tools, practical labs for secure CI/CD setup #### ๐Ÿง  Logical learners **Present**: Timeline of events, causal analysis of vulnerability, decision tree for incident response **Improve**: Risk assessment matrices, quantified impact analysis, formal incident response procedures with clear decision points #### ๐Ÿง‘โ€๐Ÿคโ€๐Ÿง‘ Social learners **Present**: Community response coordination, Discord/Slack communication examples **Improve**: Collaborative security review guides, peer code review checklists, community incident response simulations #### ๐Ÿง˜โ€โ™€๏ธ Solitary learners **Present**: Self-assessment questions about personal repository security **Improve**: Individual security audit worksheets, self-paced security training modules, reflection prompts on dependency practices #### ๐ŸŒฟ Naturalistic learners **Present**: Ecosystem-wide impact analysis, environmental metaphors for supply chain health **Improve**: Sustainability thinking about dependency choices, ecosystem health metrics, lifecycle impact of security decisions ### ๐Ÿ“Š Accessibility Assessment #### Screen reader semantics **Present**: Clear headings and logical structure, timestamp links with descriptive text **Improve**: More descriptive alt text for technical diagrams, ARIA labels for complex interaction flows #### Keyboard navigation **Present**: Standard video player controls, text-based content navigation **Improve**: Keyboard shortcuts for timestamp navigation, skip links for different content sections #### Contrast and readability **Present**: High contrast video player interface, clear text formatting **Improve**: Ensure code examples meet contrast requirements, larger font options for technical content #### Text resizing and reflow **Present**: Responsive text layout in transcript **Improve**: Test technical diagrams at 200% zoom, ensure code blocks remain readable when enlarged #### Media and captions **Present**: Full transcript provided, clear audio narration **Improve**: Add audio descriptions for visual elements, downloadable transcript with timestamp markers #### Forms and interactions **Present**: Simple video player interactions **Improve**: Accessible feedback for any interactive security assessment tools, clear error messages ### ๐ŸŒ DEIB Assessment #### Cultural breadth **Present**: Global perspective on supply chain security impacts **Improve**: Include examples from different geographic regions, consider varied regulatory environments #### Inclusive language **Present**: Technical jargon explained in context **Improve**: Plain-language summaries of key concepts, avoid assuming prior security knowledge #### Neurodiversity support **Present**: Well-structured content with clear headings **Improve**: Consistent formatting patterns, predictable information architecture, summary boxes for key points #### Representation **Present**: Focus on technical merit and community response **Improve**: Highlight diverse contributors to security research, inclusive examples in documentation #### Psychological safety **Present**: Non-judgmental tone about security mistakes **Improve**: Emphasize learning opportunities, provide clear guidance without blame, supportive community response examples ### โœ… Logical Validity #### Strong reasoning - **Causal analysis**: Clear connection between GitHub token format change โ†’ Composer validation failure โ†’ token exposure (timestamp 8:15) - **Evidence-based assessment**: Concrete examples of affected repositories and potential impact scale (timestamp 3:45) - **Risk quantification**: Specific timeframe analysis and attack window mechanics (timestamp 18:30) #### Fallacies - **Appeal to consequences**: "This could have been the worst attack ever" - while true, focuses on fear rather than systematic prevention (timestamp 22:00) - **False dilemma**: Presents responsible disclosure vs. fast response as either/or choice rather than exploring hybrid approaches (timestamp 15:45) - **Survivorship bias**: Analysis focuses on why this incident was caught quickly, may miss incidents that weren't detected (throughout) #### Tone of voice Professional and educational with technical expertise. Examples: "Let's read through to see what actually happened" shows analytical approach; "That's pretty bad" acknowledges severity without sensationalizing. ### ๐Ÿ›ก๏ธ DARVO Analysis No DARVO patterns detected. The content focuses on technical analysis and community response rather than interpersonal conflict or blame assignment. The narrator acknowledges complexity of disclosure decisions and presents multiple perspectives fairly. ## ๐ŸŽฏ Conclusion The PHP ecosystem's narrow escape from a massive supply chain attack serves as a critical wake-up call about the interconnected nature of modern development tools. While quick community response and technical safeguards prevented disaster, the incident exposed fundamental gaps in how we handle cross-ecosystem security threats. The key lesson isn't just about updating Composer or securing GitHub tokensโ€”it's about building resilient communication channels and response protocols for an increasingly connected development landscape. For WordPress developers and the broader PHP community, this incident underscores the urgent need for better dependency hygiene, security awareness, and collaborative incident response capabilities. ## ๐Ÿ”„ Refactor Appendix โ€” PHP barely avoided disaster > Density: 2โ€“5 Literature atoms (H3), 0โ€“3 Permanent candidates (H4). > Defaults: 3 Literature, 1 Permanent. Resonance filter: if you can't > articulate in one line why a claim matters, drop it. ### ๐Ÿ“š Literature Atoms (H3) #### Supply chain vulnerabilities require ecosystem-wide disclosure protocols > [!quote] ID:rw_timestamp_1515 > The responsible path and the fast path diverged. The responsible path was file with the composer security advisory queue, file a hacker one against GitHub, wait. The fast path was open Discord, ping mod mail, open X, find someone who knows the people you need. **Context & use**: - Timestamp 15:45 in discussion of disclosure challenges - Demonstrates inadequacy of traditional single-vendor security reporting for cross-ecosystem threats **Tags**: #articleseed #security-protocols **Highlight**: [view highlight](https://www.youtube.com/watch?v=dzQEe7u3k2I?t=945) <!-- atomicity: 1 idea โœ“ --> #### Token validation should never expose credentials in error messages > [!quote] ID:rw_timestamp_0815 > Notice that the logged token wasn't a typical PAT. It had a GitHub app ID embedded in it. That small an ID suggests a very old GitHub app, most likely GitHub Actions bot. **Context & use**: - Timestamp 8:15 during root cause analysis - 13-year-old Composer code included rejected tokens in exception messages, enabling the vulnerability **Tags**: #xref #secure-coding **Highlight**: [view highlight](https://www.youtube.com/watch?v=dzQEe7u3k2I?t=495) <!-- atomicity: 1 idea โœ“ --> #### Package manager automation creates amplified attack surfaces > [!quote] ID:rw_timestamp_2200 > In the PHP ecosystem, push access to a repo is push access to packages. The moment you push version blah blah blah to vendor package, packages publishes it, and the next composer install anywhere in the world pulls your release. **Context & use**: - Timestamp 22:00 explaining attack mechanics - Shows how convenience features become security multipliers in supply chain attacks **Tags**: #xref #automation-risks **Highlight**: [view highlight](https://www.youtube.com/watch?v=dzQEe7u3k2I?t=1320) <!-- atomicity: 1 idea โœ“ --> ### ๐Ÿง  Permanent Candidates (H4) #### Community networks enable faster incident response than formal channels **Claim**: Informal professional networks consistently outperform official security disclosure channels for ecosystem-wide incidents requiring immediate coordination. **Evidence pointer**: [[#Supply chain vulnerabilities require ecosystem-wide disclosure protocols]] **Scope & conditions**: Applies when vulnerabilities affect multiple vendors simultaneously; breaks down when reporters lack established network connections **Links**: - [[Incident response protocols]] โ€” because contrasts formal vs. informal coordination - [[Security disclosure models]] โ€” because extends beyond single-vendor scenarios - [[Community governance]] โ€” because applies-in crisis coordination <!-- atomicity: 1 idea โœ“ --> ### โœ๏ธ Fleeting Queue - [F1] Research other package managers for similar token validation vulnerabilities โ€” #security-audit #fleeting - [F2] Document PHP Foundation's new disclosure protocols mentioned for Monday release โ€” https://www.youtube.com/watch?v=dzQEe7u3k2I #fleeting #follow-up ### โ†’ Things 3 (max 2 actions) - [ ] Audit personal GitHub repositories for exposed tokens in Actions logs from vulnerability window โ€” security review #things - [ ] Update all project Composer installations and review GitHub Actions permissions โ€” dependency hygiene #things --- ## Full Transcript [00:00] Yesterday, the PHP community barely<br>[00:02] avoided absolute [music]<br>[00:04] disaster. That's how this blog post here<br>[00:07] starts, explaining how a supply chain<br>[00:10] vulnerability was detected in in<br>[00:13] composer, and it had the possibility for<br>[00:15] things to go [music] very bad for PHP.<br>[00:17] It's funny how how this thing happens<br>[00:20] just a week or two after that previous<br>[00:22] package that got compromised. I made a<br>[00:26] video about that if you want to [music]<br>[00:27] watch it after this one. So, let's read<br>[00:29] through here to see what actually<br>[00:32] happened. [music] In short, for a<br>[00:34] 14-hour window yesterday,<br>[00:37] which is already a couple of days, yeah,<br>[00:39] last week basically, any PHP project<br>[00:42] that was hosted on GitHub ran composer<br>[00:44] install after shivammathur set a PHP in<br>[00:47] GitHub actions, which is basically all<br>[00:49] PHP projects that rely on PHP used this<br>[00:52] action here.<br>[00:54] That ran that workflow against its main<br>[00:56] branch and was triggered by on push, on<br>[00:59] pull request target, or on schedule.<br>[01:01] Other triggers are likely impacted, but<br>[01:03] these are the most common. When you had<br>[01:05] that situation, you had a chance that<br>[01:07] the GitHub token, typically write<br>[01:10] enabled, could would be publicly logged<br>[01:12] in GitHub actions log on every build.<br>[01:15] For those 14 hours, any malicious actor<br>[01:18] can uh monitoring one of those uh repos<br>[01:22] could have pushed [snorts] new tags and<br>[01:23] commits without anyone noticing. If<br>[01:26] properly leveraged, this would have been<br>[01:28] the largest supply chain breach the PHP<br>[01:32] ecosystem has ever seen. To make matters<br>[01:34] worse, we're not even done yet. The same<br>[01:37] change may roll out again in a week at<br>[01:40] GitHub's discretion, and there's a<br>[01:43] chance it bleeds beyond PHP in other<br>[01:46] ecosystems. That's pretty bad. Uh we're<br>[01:49] going to continue to read through the<br>[01:50] rest of the post as well, but what<br>[01:52] they're saying here is that for 14<br>[01:55] hours, if people were monitoring GitHub<br>[01:58] actions and actually got a hold of your<br>[02:00] GitHub token, they would have been able<br>[02:02] to make changes to any repository that<br>[02:05] leaked their token uh without anyone<br>[02:08] noticing, tag those changes, which would<br>[02:11] then automatically be released through<br>[02:13] composer, and anyone who pulled them in<br>[02:15] would get the affected release. Just<br>[02:18] imagine this happening with Laravel, for<br>[02:20] example, which has, I don't know, let's<br>[02:22] see, like an average of 300,000<br>[02:25] installs per day. So, for 14 hours,<br>[02:28] let's just say 150,000<br>[02:30] installs could have been affected by<br>[02:33] this. Could just pull in the affected<br>[02:35] version without anyone ever noticing.<br>[02:37] That's That's the the extent of where<br>[02:39] this<br>[02:40] uh vulnerability goes to. As far as<br>[02:42] people know,<br>[02:43] I don't think this has been exploited,<br>[02:46] but we'll read about it. The person who<br>[02:47] discovered Damien here uh maintains a<br>[02:49] set of GitHub actions for the Magento<br>[02:51] community. They exist to lower the<br>[02:53] barrier uh uh the barrier to entry into<br>[02:56] the Magento and Magento OS ecosystem, so<br>[02:59] that a company wanting to ship or store<br>[03:01] an extension can go get working with CI<br>[03:04] without much fuss. Yeah, that's that's<br>[03:05] what all these actions do, right? To<br>[03:08] just make it easier to to use GitHub<br>[03:10] actions or CI platforms. For me, that<br>[03:13] means maintaining a wide variety of<br>[03:14] GitHub actions that work across many<br>[03:16] versions of Magento and Magento OS. To<br>[03:18] handle that, I main uh maintain a fairly<br>[03:20] complex action called supported version,<br>[03:23] which holds the complexity matrix for<br>[03:25] every supported version of Magento and<br>[03:27] Magento OS.<br>[03:31] Yeah, it's it's like<br>[03:33] the more GitHub actions I can avoid,<br>[03:35] the better in my case, to be honest, but<br>[03:37] yeah, it's not my thing. Uh I have for<br>[03:40] Tempest, I'm so happy that that other<br>[03:42] people are helping me out with GitHub<br>[03:44] because anything DevOps {slash} CI<br>[03:47] related is is really like, nah, just<br>[03:49] give me PHP. Uh while I was working on<br>[03:51] this action for the Magento release<br>[03:53] yesterday evening, my test started<br>[03:55] failing intermittently after some small<br>[03:57] changes<br>[03:58] to this action. The failures didn't<br>[04:00] track with anything I changed. When I<br>[04:02] went to debug the failure, I saw the<br>[04:04] following. In base IO, which I think is<br>[04:08] is a composer class, we'll we'll see in<br>[04:10] a minute. Your GitHub OAuth token for<br>[04:13] GitHub contains invalid characters and<br>[04:15] then just prints the old token. I was<br>[04:17] surprised more than anything. My first<br>[04:19] instinct was that screwed something up.<br>[04:21] Bash scripting processing the composer<br>[04:23] auth I I feed into my actions. I've seen<br>[04:25] secrets in my logs before whenever a<br>[04:27] script clobbers a string with new lines,<br>[04:30] so this looked like my fault. It took me<br>[04:32] about an hour, I'm not a very fast<br>[04:34] person, to realize that my actions<br>[04:36] weren't at fault. Nothing my actions<br>[04:39] Nothing I changed could have triggered<br>[04:42] this outcome. Shortly thereafter, I<br>[04:45] concluded that something outside my<br>[04:46] actions was wrong and wrong in a way<br>[04:49] that probably wasn't unique to me. So, I<br>[04:51] asked Claude, as you do,<br>[04:53] giving it all the context I just<br>[04:56] uncovered. Claude pointed out two things<br>[04:58] I'd seen but not noticed. The logged<br>[05:01] token wasn't a typical PAT. This is like<br>[05:04] a personal access token. It had a GitHub<br>[05:06] app ID embedded in it, specifically this<br>[05:09] one. That small an ID suggests a very<br>[05:12] old GitHub app, most likely<br>[05:15] GitHub Actions bot. And then Shiva<br>[05:17] matter said a PHP automatically inserts<br>[05:19] that token into composer auth as the<br>[05:22] GitHub OAuth value. That makes sense. At<br>[05:24] this point, I still wasn't too<br>[05:26] concerned. I naively thought this was<br>[05:28] one a one-off bug in how tokens<br>[05:31] had been generated. I knew the GitHub<br>[05:34] Actions token expires when the job ends.<br>[05:37] I had looked at workflow token security<br>[05:39] on my repos before, and the leak was on<br>[05:41] a pull request, so the token's<br>[05:42] permissions were already fairly<br>[05:44] restricted. I didn't lose sleep over it.<br>[05:47] I merged my changes to my main branch<br>[05:49] from my pull request, fully expecting<br>[05:51] that this would recover and I could move<br>[05:54] about my day. Then my main branch CI<br>[05:57] failed on a completely different job and<br>[05:59] I became concerned. Earlier yesterday,<br>[06:02] uh 10 sec router was breached because<br>[06:04] their main branch's cache was points<br>[06:06] poisoned by a pull request. Yeah. I was<br>[06:09] very concerned that something like that<br>[06:11] uh was going on here. It was at this<br>[06:14] point with the read writes token visibly<br>[06:16] in my logs that I knew something was up<br>[06:18] and that I had to dig in and act<br>[06:20] quickly. I had Claude triage the the<br>[06:23] error message from the log in base IO<br>[06:25] here, figuring that it would do a better<br>[06:27] job at scouring Composer's project code<br>[06:30] for the offending line faster than I<br>[06:32] would. And it uncovered it immediately.<br>[06:34] And so this is the code that throws the<br>[06:36] exception and then actually includes the<br>[06:38] token in the exception message, which is<br>[06:41] probably not really a smart thing to do,<br>[06:43] but you know, who thinks about these<br>[06:45] things, right? Allowed characters for<br>[06:46] GitHub tokens are these plus dots, which<br>[06:49] are at some point used for GitHub app<br>[06:52] app integration tokens. So basically, if<br>[06:54] your Composer checks if your token<br>[06:57] doesn't match a given format, it will<br>[06:59] throw an exception because then it's an<br>[07:01] invalid token. But what it also does is<br>[07:04] print that invalid token in the<br>[07:06] exception message. It also uncovered,<br>[07:08] and huge props to Claude here,<br>[07:10] this would have taken me hours on my<br>[07:12] own, that<br>[07:14] that GitHub was rolling out a change to<br>[07:16] their token formats. Notice about<br>[07:19] upcoming new format for GitHub app<br>[07:22] installation tokens. So the token format<br>[07:24] has an installation tokens,<br>[07:26] which will be changing to uh<br>[07:29] yeah, another format. A change to the<br>[07:31] token format. At this point, I knew very<br>[07:33] distinctly that, which is my inner child<br>[07:35] telling me, and now you, that something<br>[07:37] is undeniably wrong. How I acted. Here's<br>[07:40] what I did in order. I disabled all<br>[07:42] actions in my organization, better safe<br>[07:44] than sorry. I pinged the major OS<br>[07:46] Discord and asked them to pause all<br>[07:48] GitHub actions immediately. I submitted<br>[07:50] a security advisory report to the<br>[07:52] composer project based on my quick<br>[07:53] analysis. I reached out to Jessica who<br>[07:56] would have previously connected with on<br>[07:58] on fake repository hoping she would<br>[08:00] quickly and quietly get me in touch with<br>[08:02] people at the large frameworks at risk.<br>[08:04] My request she reached out to Taylor<br>[08:06] and asked him to pause Laravel GitHub<br>[08:08] actions. I pinged ModMail users on<br>[08:11] Laravel's Discord. Shortly thereafter<br>[08:13] their actions were disabled. I'm not<br>[08:15] sure which channel did it. I called a<br>[08:17] colleague and started trawling GitHub<br>[08:20] repos to confirm I wasn't the only one<br>[08:22] affected. I pinged Barry on Slack<br>[08:24] informing him to disable his GitHub<br>[08:27] actions as well. As an aside, I'm a<br>[08:29] small fish compared to many of the<br>[08:30] well-known developers in the PHP<br>[08:32] ecosystem. I didn't know if anyone<br>[08:35] would take me seriously and I wanted to<br>[08:37] make sure no one was hurt simply because<br>[08:38] I wasn't loud enough. I fully understand<br>[08:40] responsible disclosure. I filed several<br>[08:43] HackerOne reports before and I've<br>[08:45] received bounties for some of them. But<br>[08:46] when the breadth of concern is this<br>[08:48] large, this painfully obvious, this<br>[08:51] critical, and had been 4 hours since I'd<br>[08:54] opened the advisory without a response,<br>[08:55] I felt I needed to take my own action<br>[08:58] through quiet fast channels to make sure<br>[09:00] the largest parts of the ecosystem were<br>[09:02] protected as quickly as possible. And<br>[09:04] what were the chances I could get get<br>[09:05] Niels or Jordi, these two are the<br>[09:08] maintainers of composer, on a call<br>[09:09] minutes after I submitted an advisory?<br>[09:12] First, what were the chances I could get<br>[09:13] a HackerOne report submitted and<br>[09:15] reviewed by GitHub in the same time<br>[09:17] frame? You have to understand when<br>[09:18] something like this happens, you don't<br>[09:20] want people to know about it because<br>[09:22] there might be bad people out there who<br>[09:24] will abuse it as soon as it's getting<br>[09:27] known. You want the right people to know<br>[09:29] about it who can fix it. In this case,<br>[09:31] that's composer who's the root of the<br>[09:33] issue. You want to make sure that the<br>[09:35] the big players like in Symphony and<br>[09:37] Laravel are are aware of it as well. But<br>[09:40] you really don't want to be out there in<br>[09:42] the open shouting there's a huge<br>[09:44] security issue because it it it's it's<br>[09:46] not good. In reality, likely no one was<br>[09:49] compromised. In reality, likely no one<br>[09:52] next week will be compromised when<br>[09:53] GitHub rerolls the change out. Composer<br>[09:56] has already issued patches to all major<br>[09:58] versions of Composer combined, even<br>[10:00] non-supported non-LTS versions. Ash even<br>[10:03] matter has has already bumped their main<br>[10:05] line to update to a secure version of<br>[10:07] Composer as well. That being said, the<br>[10:09] bulk of the reason that this wasn't as<br>[10:10] impactful is primarily due to three<br>[10:13] components. First, Composer validator<br>[10:17] fails fast. The exception bubbles up and<br>[10:20] the job dies. This causes the GitHub<br>[10:22] token to be revoked the moment the job<br>[10:25] exists. From the attacker's perspective,<br>[10:28] the token is live for the seconds<br>[10:30] between Symphony console writing the<br>[10:32] message to standard error and the runner<br>[10:35] tearing down the job. That window is<br>[10:37] real, but it's narrow. To be clear, it's<br>[10:39] not narrow enough to be unexploitable. I<br>[10:42] have a proof of concept which showcases<br>[10:45] direct writes to main using the stolen<br>[10:47] token in this very narrow window. With a<br>[10:49] sufficiently sophisticated polling<br>[10:51] setup, you can do this in under a<br>[10:54] second. The second reason is that the<br>[10:56] failure was loud. While I was running<br>[10:58] through GitHub trying to figure out<br>[11:00] whether I was the only person hit, I<br>[11:02] could see the failure landing in CI logs<br>[11:04] all across most of the top PHP repos,<br>[11:06] Laravel, Composer itself, WooCommerce,<br>[11:09] OpenEMR, Sylius, Laravel Debugbar, and<br>[11:11] many other smaller projects. Anyone<br>[11:13] watching their own CI saw it within<br>[11:16] minutes. If it was quiet, then it would<br>[11:18] have gone on far longer. Who reads the<br>[11:21] build logs of a passing job? The third<br>[11:23] reason is that Composer shipped patched<br>[11:25] releases and the advisory went out the<br>[11:27] next morning. GitHub paused the format<br>[11:29] rollout shortly afterward. The response<br>[11:31] time from notification to resolution was<br>[11:33] remarkable. Now, the potential impact.<br>[11:35] Imagine someone saw what I saw earlier<br>[11:37] than I did, and who decided to do<br>[11:39] something with it instead of reporting<br>[11:41] it. The primitive you have is GitHub<br>[11:43] token from a push event on a repo's<br>[11:45] default branch printed in plain text to<br>[11:47] world readable log live for some number<br>[11:50] of seconds before the job ends. The<br>[11:52] tokens permissions vary by repo, but for<br>[11:55] the long tail of older apps and most<br>[11:56] maintainer configured workflows, the<br>[11:58] default is still contents rights. The<br>[12:01] legacy permissive default. That's enough<br>[12:03] to push commits and tags to the repo it<br>[12:05] came from. Worse, the leak doesn't<br>[12:07] necessarily end precisely when the<br>[12:09] failing step ends. A pattern I have in<br>[12:12] my own workflows, and that a lot of<br>[12:14] Magento and broader PHP workflows have,<br>[12:17] looks like this. The intent is fine.<br>[12:20] When CI fails, freeze the workspace<br>[12:22] state into an artifact so a human can<br>[12:24] debug it later. In this bug's context,<br>[12:27] the step compounds the leak. The core<br>[12:29] concept is that it extends the life<br>[12:31] token window, and the upload runs after<br>[12:33] composer has crashed and printed the<br>[12:35] token to standard error, but before the<br>[12:37] runner can tear down uh the job and<br>[12:40] revoke the token. Until the upload<br>[12:42] finishes, the token is still accepted by<br>[12:44] the GitHub API. For a small sandbox,<br>[12:47] that's seconds. For a fat integration<br>[12:50] test fixture with database dumps, it's<br>[12:52] many minutes, yeah. And an attacker<br>[12:54] polling failed runs gets a longer window<br>[12:56] to act, not a shorter one. In the PHP<br>[12:58] ecosystem, push access to a repo is push<br>[13:01] access to packages, yeah. Packages pulls<br>[13:04] GitHub for new tags, that's true, yeah.<br>[13:06] The moment you push version blah blah<br>[13:08] blah to vendor package, packages<br>[13:10] publishes it, and the next composer<br>[13:12] install or composer update anywhere in<br>[13:13] the world pulls your release. Composer<br>[13:16] plugins execute code during install,<br>[13:18] although not entirely true, you have to<br>[13:20] accept this. That's That's what we saw<br>[13:22] in the in the previous video. But But<br>[13:24] still, it's a good point. So, a poisoned<br>[13:26] tag on a sufficiently popular package is<br>[13:29] a remote code execution on every<br>[13:31] developer's machine, CI runner, and<br>[13:33] production deploy that runs composer<br>[13:35] install between the tag landing and<br>[13:37] rollback. Look at the names on the list<br>[13:39] above. Composer itself was failing in CI<br>[13:42] during this window. So was Laravel, so<br>[13:44] was Sylius. A successful XZ and push<br>[13:47] against anyone of those is a top 50 in<br>[13:50] history supply chain incident. Against<br>[13:52] several of them simultaneously with the<br>[13:55] same primitive, it would be the largest<br>[13:56] one ever in PHP and probably top three<br>[13:59] against all ecosystems. The Magento<br>[14:01] angle, since it's the corner of the<br>[14:03] world I actually live in, Magento and<br>[14:05] Mejo as distributions are composer<br>[14:07] projects. Composer self update pulls<br>[14:09] from getcomposer.org, which redirects to<br>[14:12] GitHub releases. A poisoned composer<br>[14:15] binary would execute under every Magento<br>[14:17] store deploys pipeline in the next<br>[14:19] build. Most e-commerce platforms do not<br>[14:21] have the supply chain hygiene of, say, a<br>[14:24] rail shop. The blast radius would not<br>[14:26] have been bounded by who reads Hacker<br>[14:29] News. It is It is pretty true. And I<br>[14:31] mean, we've become pretty lazy because<br>[14:33] of the tools we're using. I talked about<br>[14:35] this in the previous video as well, like<br>[14:36] the dependency hygiene concern. It's<br>[14:38] convenient that we can just, you know,<br>[14:40] pull in stuff that was published to<br>[14:42] GitHub and just is available through<br>[14:44] composer and it's all automated. It's<br>[14:45] It's super convenient. And it has so<br>[14:49] much potential of doing so much harm.<br>[14:52] So, the mitigating factors, a short list<br>[14:54] of things that intentionally or<br>[14:55] otherwise kept this from going off.<br>[14:57] Composer fails fast and fails hard. I do<br>[15:00] remember when when running the the 100<br>[15:04] million row challenge, that one of the<br>[15:06] tokens got leaked there in a GitHub<br>[15:08] action while we were debugging some<br>[15:09] stuff. I actually got an email from<br>[15:11] GitHub telling me that my token leaked,<br>[15:13] but I guess that since these these are<br>[15:15] GitHub action specific tokens, maybe<br>[15:16] GitHub doesn't check on those and only<br>[15:18] checks for like real personal access<br>[15:20] tokens that were generated on on the<br>[15:23] account or organization level. But that<br>[15:25] that would have been helpful, I reckon.<br>[15:27] Maybe a good lesson here is that you<br>[15:29] should never ever print out tokens in<br>[15:31] exception messages, but you know, you<br>[15:33] live and you learn. GitHub token is repo<br>[15:35] scoped even after even a successful<br>[15:38] exploit against Laravel framework<br>[15:40] doesn't get you anywhere near Laravel<br>[15:41] Passport. Each repo's token can only act<br>[15:44] against a repo. Although for mono repos,<br>[15:47] you get a much wider<br>[15:50] scope. But yeah, stuff like Laravel<br>[15:52] Passport indeed is<br>[15:54] is a separate package. The 6-hour<br>[15:56] ceiling, GitHub hosted runners kept<br>[15:59] workflow tokens at 6 hours. Okay. Branch<br>[16:02] protection blocks the obvious move.<br>[16:04] Pushing to main directly on a protected<br>[16:06] branch fails. Yeah, and<br>[16:07] which is good. Don't know if that really<br>[16:09] works though because you can tag<br>[16:11] releases on other branches, which is the<br>[16:14] the exploit from that previous video.<br>[16:17] You can do that just fine and composer<br>[16:19] will pick it up even if it's on another<br>[16:20] branch. So I don't know if that that's<br>[16:22] really good mitigation here. The rollout<br>[16:25] was gradual. Not every repo was on the<br>[16:27] new token format at the same time.<br>[16:29] Really good. And composer maintainers<br>[16:31] moved Yeah, really fast. Exactly. Now<br>[16:33] that's really great. What a typical<br>[16:35] public PHP maintainer has to do, upgrade<br>[16:38] composer. And that's really that's<br>[16:40] really simple by the way.<br>[16:42] Just run a composer self-update. You can<br>[16:45] tighten permissions. Don't rely on<br>[16:47] legacy permissive defaults. Absolutely.<br>[16:49] Protect your tags. The blast radius of<br>[16:52] this leak is overwhelmingly an attacker<br>[16:54] pushing a tag and packages publishing<br>[16:56] it. Enable GitHub's tag protection rules<br>[16:58] or branch protection rules. Yeah, that<br>[17:00] is indeed a very good idea. Sweep your<br>[17:02] action logs for the window. Check your<br>[17:04] tag and commit history for the window.<br>[17:06] Mhm. [clears throat]<br>[17:07] Deploy the fixes before the next<br>[17:10] rollout. I think the the biggest fix<br>[17:12] here and it's it's chief a matter action<br>[17:15] updates composer. As far as I understand<br>[17:18] that already happened. So, when it comes<br>[17:20] to GitHub actions at least, anyone using<br>[17:22] shivamuthu<br>[17:23] should be fine as far as I understand.<br>[17:26] But of course it's it should do all of<br>[17:28] this as well. What GitHub has to do,<br>[17:30] harden the runner secret maskers against<br>[17:33] console reframing. Today it matches<br>[17:36] registered values as exact substrings.<br>[17:38] Yeah. Yeah, so<br>[17:39] GitHub tokens are masked in output but<br>[17:42] it it's apparently not not strict<br>[17:45] enough. Yeah, that is something GitHub<br>[17:46] should do indeed. Coordinate disclosure<br>[17:48] with the major package managers before<br>[17:50] the next rollout wave. Good point as<br>[17:52] well. Maybe we need a little more than<br>[17:54] just a small blog post. And automate<br>[17:57] testing of the common GitHub action<br>[17:58] setups for the top 50 programming<br>[18:01] languages. What other ecosystems have to<br>[18:03] do, grab your code base for hard coded<br>[18:05] GitHub token regexes. Yeah, good idea.<br>[18:08] Never interpolate a rejected credential<br>[18:10] into a user visible error. This really<br>[18:12] is important. Other credential paths the<br>[18:15] same way and reach [snorts] out to<br>[18:16] GitHub security ahead of the next<br>[18:18] rollout. A final note on disclosure. I<br>[18:20] want to spend a couple of paragraphs on<br>[18:22] how this felt from the reporters side,<br>[18:24] not as a complaint but as a description<br>[18:26] of a structural problem someone with<br>[18:27] power should think about. The model for<br>[18:29] responsible disclosure assumes there is<br>[18:32] one vendor on the other end with a<br>[18:34] security team, a queue and a person paid<br>[18:37] to read the queue. Most of the time<br>[18:38] that's exactly how it works. You write<br>[18:40] up the bug, you submit it, they triage,<br>[18:42] they patch, you go home. This one<br>[18:44] doesn't have that shape. The root cause<br>[18:46] was GitHub. The fix that mattered most<br>[18:48] quickly was at composer. The blast<br>[18:50] radius was every PHP framework<br>[18:52] maintainer on Earth and we have no<br>[18:54] shared security inbox. There was no way<br>[18:56] to reach all of them on a Tuesday<br>[18:58] evening in under an hour as someone<br>[19:00] they've never heard before. Every minute<br>[19:02] the leak went undetected was a minute an<br>[19:05] attacker with a scraper could have been<br>[19:07] turning failing builds into push to tag<br>[19:09] access. So, the responsible path and the<br>[19:12] fast path diverged. The responsible path<br>[19:15] was file with the composer security<br>[19:18] advisory queue, file a hacker one<br>[19:20] against GitHub, wait. The fast path was<br>[19:24] open Discord, ping mod mail, open X,<br>[19:26] find someone who knows the people you<br>[19:27] need, cold call a colleague and hope<br>[19:30] they're taking you seriously by people<br>[19:32] who have no reason to know who you are.<br>[19:34] I picked both paths. I'm still not sure<br>[19:37] if I was right. I'm fairly sure that<br>[19:39] traversing the fast path was less wrong<br>[19:41] than waiting for the advisory queue to<br>[19:43] process what it would have been. What<br>[19:45] bothers me is that someone less well<br>[19:47] connected than I am with the same bug at<br>[19:49] 10:00 p.m. on a Tuesday doesn't have a<br>[19:52] faster lever to pull than guess who<br>[19:54] knows the Taylor Otwell's of the PHP<br>[19:56] ecosystem.<br>[19:58] And that this is a pretty like important<br>[20:00] issue that he's raising. If you're<br>[20:02] reading this and you have power in this<br>[20:03] picture at GitHub, HackerOne, one of the<br>[20:06] major ecosystem maintainers, I would<br>[20:08] like for there to be a phone number, a<br>[20:10] pager, a shared cross-vendor for this is<br>[20:12] an ongoing ecosystem scale leak and I<br>[20:15] need an adult. I don't know what that<br>[20:17] looks like in detail, but submit a form<br>[20:19] and pray is not adequate for incidents<br>[20:21] that look like this one. And I don't<br>[20:23] think we should keep relying on the fact<br>[20:25] that the next reporter happens to have<br>[20:27] my network, albeit small as it is.<br>[20:29] Especially in the modern era of AI as<br>[20:31] the discovery to abuse window titans,<br>[20:34] time to response becomes more critical<br>[20:36] than ever before. I I actually think<br>[20:39] this is the most important part of of<br>[20:41] this whole blog post. We need to think<br>[20:42] about this. Very well put. Thank you,<br>[20:44] Damien, for the effort you put into it.<br>[20:46] And I'd like to actually see if someone<br>[20:49] from composer, etc.<br>[20:51] replied. I do wonder about the value of<br>[20:53] validating tokens client-side. Wouldn't<br>[20:55] it be enough to just let the API throw<br>[20:56] an error? And here is Jordi from<br>[20:58] composer saying, "Yeah, I tracked down<br>[21:00] the commit which was added this<br>[21:01] yesterday and it was 13 or 14 years old.<br>[21:04] I I a PR open already to delete the<br>[21:06] validation entirely from an upcoming<br>[21:08] release and I don't think it's relevant<br>[21:10] now that tokens are not really entered<br>[21:12] manually anymore. Back then you had to<br>[21:13] go and create it and type it in and if<br>[21:15] you messed up the copy paste, it was<br>[21:17] probably helpful to show you what was<br>[21:19] wrong, but today the landscape is very<br>[21:20] different. This is just like makes<br>[21:22] sense, right? But in hindsight, maybe<br>[21:25] not the best idea. Yeah, on your last<br>[21:26] point regarding the disclosure path,<br>[21:28] please watch out for a PHP foundation<br>[21:30] next Monday. We've already been working<br>[21:33] on a plan for this especially<br>[21:34] considering the changes AI has brought<br>[21:36] through these processes. I entirely<br>[21:39] agree with you on how tricky this is in<br>[21:41] situations where a huge amount of people<br>[21:43] and projects are affected and<br>[21:44] communication becomes extremely<br>[21:46] challenging. This Monday, by the way,<br>[21:47] that's today, the day I'm recording<br>[21:49] this, I will follow up on this as well.<br>[21:51] I think [music] there are some some cool<br>[21:53] things coming. The PHP community barely<br>[21:56] avoided an absolute disaster. We're<br>[21:58] lucky that that the person who caught<br>[22:00] this had good intentions and wanted to<br>[22:02] help out.<br>[22:03] >> [music]<br>[22:03] >> So, let's keep this as the second<br>[22:06] reminder now on this channel at least<br>[22:08] that PHP isn't immune for these kind of<br>[22:10] things [music]<br>[22:11] and that dependency hygiene really is a<br>[22:15] good thing to keep in mind. So, yeah,<br>[22:18] let me know your thoughts in the<br>[22:19] comments down below and I'll see you in<br>[22:21] the next one.<br>[22:22] Bye-bye. <!-- deep-read:sections end --> ## ๐Ÿ“บ Video ![](https://www.youtube.com/watch?v=dzQEe7u3k2I) ## Highlights - The responsible path and the fast path diverged. The responsible path was file with the composer security advisory queue, file a hacker one against GitHub, wait. The fast path was open Discord, ping mod mail, open X, find someone who knows the people you need. - Note: ๐Ÿ“š Supply chain vulnerabilities require ecosystem-wide disclosure protocols ยท - Timestamp 15:45 in discussion of disclosure challenges - Demonstrates inadequacy of traditional single-vendor security reporting for cross-ecosystem threats ยท #atom/literature/h1 - Notice that the logged token wasn't a typical PAT. It had a GitHub app ID embedded in it. That small an ID suggests a very old GitHub app, most likely GitHub Actions bot. - Note: ๐Ÿ“š Token validation should never expose credentials in error messages ยท - Timestamp 8:15 during root cause analysis - 13-year-old Composer code included rejected tokens in exception messages, enabling the vulnerability ยท #atom/literature/h2 - In the PHP ecosystem, push access to a repo is push access to packages. The moment you push version blah blah blah to vendor package, packages publishes it, and the next composer install anywhere in the world pulls your release. - Note: ๐Ÿ“š Package manager automation creates amplified attack surfaces ยท - Timestamp 22:00 explaining attack mechanics - Shows how convenience features become security multipliers in supply chain attacks ยท #atom/literature/h3 - Community networks enable faster incident response than formal channels - Note: [[Incident response protocols]] โ€” because contrasts formal vs. informal coordination ยท - [[Incident response protocols]] โ€” because contrasts formal vs. informal coordination ยท #atom/zettel/z1 - Research other package managers for similar token validation vulnerabilities โ€” #security-audit #fleeting - Note: โœ๏ธ Fleeting capture #atom/fleeting/f1 - Document PHP Foundation's new disclosure protocols mentioned for Monday release โ€” https://www.youtube.com/watch?v=dzQEe7u3k2I #fleeting #follow-up - Note: โœ๏ธ Fleeting capture #atom/fleeting/f2 - Presents responsible disclosure vs. fast response as either/or choice rather than exploring hybrid approaches (timestamp 15:45) - Note: ๐Ÿ” Fallacy: False Dilemma #fallacy/false-dilemma --- # ๐Ÿ“ Full Notes (Deep-Read Analysis) # ๐Ÿท๏ธ Title: PHP barely avoided disaster ## ๐Ÿงญ Table of Contents - [๐Ÿ“ Summary](#-summary) - [๐Ÿ“– Main Content](#-main-content) - [๐Ÿ—’๏ธ Notes](#๏ธ-notes) - [๐Ÿ“š References](#-references) - [๐Ÿ—ฃ๏ธ Paraphrase](#๏ธ-paraphrase) - [๐Ÿ“ฐ Newsletter](#-newsletter) - [โœ… Action Items](#-action-items) - [๐Ÿงช Hypotheses](#-hypotheses) - [๐Ÿ“Š Methodology](#-methodology) - [โ“ Questions](#-questions) - [๐Ÿ”ฎ Future Research](#-future-research) - [๐Ÿ’ก Implications](#-implications) - [๐Ÿง  Learning Styles Assessment](#-learning-styles-assessment) - [๐Ÿ“Š Accessibility Assessment](#-accessibility-assessment) - [๐ŸŒ DEIB Assessment](#-deib-assessment) - [โœ… Logical Validity](#-logical-validity) - [๐Ÿ›ก๏ธ DARVO Analysis](#-darvo-analysis) - [๐ŸŽฏ Conclusion](#-conclusion) ## ๐Ÿ“ Summary - A 14-hour window exposed GitHub tokens in Composer logs, creating massive supply chain vulnerability for PHP ecosystem - Damien discovered the issue while debugging GitHub Actions, found tokens leaking in plain text due to new GitHub token format incompatible with Composer validation - The vulnerability could have enabled attackers to push malicious code to any PHP package, potentially affecting hundreds of thousands of installs daily - Fast response from maintainers and gradual rollout prevented exploitation, but highlighted critical gaps in cross-ecosystem security communication - Bottom line: PHP dodged a potential top-3 supply chain attack through luck and quick community response, exposing structural weaknesses in disclosure processes ## ๐Ÿ“– Main Content The PHP ecosystem narrowly avoided what could have been its largest supply chain attack in history. For 14 hours, any PHP project using the popular `shivammathur/setup-php` GitHub Action had its GitHub tokens exposed in plain text logs whenever `composer install` failed. ### The Technical Issue GitHub was rolling out a new token format for app installation tokens. Composer's 13-year-old token validation code couldn't handle the new format and threw an exception that included the invalid token in the error message. When these tokens appeared in GitHub Actions logs, they became publicly visible. The exposed tokens typically had write permissions, allowing attackers to: - Push commits and tags to repositories - Trigger automatic package releases through Packagist - Execute code on every developer machine and CI runner that pulled the poisoned packages ### Discovery and Response Damien, who maintains GitHub Actions for the Magento community, discovered the issue while debugging failing tests. After realizing the scope, he took both official and unofficial channels: - Filed security advisories with Composer and GitHub - Reached out directly to major framework maintainers through Discord, Slack, and personal connections - Asked Laravel, Magento, and other communities to pause their GitHub Actions ### Mitigating Factors Several factors prevented widespread exploitation: - **Composer failed fast**: Jobs crashed immediately, revoking tokens quickly - **Visible failures**: The bug was loud, appearing in CI logs across major PHP repositories - **Quick patches**: Composer maintainers released fixes within hours - **Gradual rollout**: Not all repositories were affected simultaneously - **Branch protection**: Many repositories had protections that would have blocked direct pushes to main ### Structural Problems Revealed The incident exposed a critical gap in cross-ecosystem security communication. There's no established protocol for reporting vulnerabilities that affect multiple vendors and thousands of projects simultaneously. Damien had to rely on personal networks and informal channels to reach framework maintainers quickly. ## ๐Ÿ—’๏ธ Notes #### โฑ๏ธ [00:00:00](https://www.youtube.com/watch?v=dzQEe7u3k2I?t=0) Opening - scope of near-disaster - PHP community barely avoided "absolute disaster" due to supply chain vulnerability in Composer - Happened week after previous package compromise incident #### โฑ๏ธ [00:01:30](https://www.youtube.com/watch?v=dzQEe7u3k2I?t=90) The vulnerability window - 14-hour exposure window for any PHP project on GitHub using `shivammathur/setup-php` action - GitHub tokens with write permissions publicly logged in GitHub Actions - Potential for largest supply chain breach in PHP history #### โฑ๏ธ [00:03:45](https://www.youtube.com/watch?v=dzQEe7u3k2I?t=225) Scale of potential impact - Laravel alone: ~300,000 installs per day, ~150,000 potentially affected during window - Automatic releases through Composer mean immediate propagation of malicious code #### โฑ๏ธ [00:05:20](https://www.youtube.com/watch?v=dzQEe7u3k2I?t=320) Discovery process - Damien maintains complex GitHub Actions for Magento community - Intermittent test failures led to discovering tokens in error messages - Initial assumption was personal scripting error #### โฑ๏ธ [00:08:15](https://www.youtube.com/watch?v=dzQEe7u3k2I?t=495) Root cause analysis with Claude - Claude AI helped identify the specific Composer code throwing exception with token - Traced to 13-year-old token validation code that couldn't handle new GitHub token format - GitHub was rolling out new format for app installation tokens #### โฑ๏ธ [00:11:30](https://www.youtube.com/watch?v=dzQEe7u3k2I?t=690) Emergency response actions - Disabled all actions in organization immediately - Contacted major PHP communities through Discord, Slack - Filed security advisories while simultaneously using informal channels - Reached out to Laravel team directly through connections #### โฑ๏ธ [00:15:45](https://www.youtube.com/watch?v=dzQEe7u3k2I?t=945) Why multi-channel disclosure was necessary - Traditional responsible disclosure too slow for ecosystem-wide threat - No shared security inbox for PHP ecosystem - Time pressure: every minute increased risk of exploitation #### โฑ๏ธ [00:18:30](https://www.youtube.com/watch?v=dzQEe7u3k2I?t=1110) Attack window mechanics - Token live for seconds between error message and job teardown - Proof of concept showed exploitation possible in under one second with sophisticated polling - Upload artifacts could extend token lifetime to many minutes #### โฑ๏ธ [00:22:00](https://www.youtube.com/watch?v=dzQEe7u3k2I?t=1320) Potential attack scenarios - Push access to repo = push access to packages via Packagist - Poisoned tags automatically published and distributed - Could have been top-3 supply chain incident across all ecosystems #### โฑ๏ธ [00:25:15](https://www.youtube.com/watch?v=dzQEe7u3k2I?t=1515) Mitigating factors - Composer's fast failure and hard crash - Loud, visible failures across major repositories - Quick response from maintainers - GitHub token scope limitations (repo-specific) - Branch protection rules blocking direct main pushes #### โฑ๏ธ [00:28:45](https://www.youtube.com/watch?v=dzQEe7u3k2I?t=1725) Recommended actions - Upgrade Composer via `composer self-update` - Tighten GitHub Actions permissions - Protect tags with GitHub's protection rules - Review logs and history for the vulnerability window #### โฑ๏ธ [00:32:00](https://www.youtube.com/watch?v=dzQEe7u3k2I?t=1920) Structural disclosure problems - No ecosystem-wide security communication protocol - Reliance on personal networks and informal channels - Need for "phone number for ecosystem-scale incidents" - AI making discovery-to-abuse window shorter #### โฑ๏ธ [00:35:30](https://www.youtube.com/watch?v=dzQEe7u3k2I?t=2130) Composer maintainer response - Jordi confirmed 13-year-old validation code was root cause - Client-side token validation no longer relevant in modern automated world - PHP Foundation working on improved disclosure processes ## ๐Ÿ“š References - [Original blog post by Damien](referenced in video but URL not provided) - [shivammathur/setup-php GitHub Action](https://github.com/shivammathur/setup-php) - [Composer project](https://getcomposer.org) - [Packagist package repository](https://packagist.org) ## ๐Ÿ—ฃ๏ธ Paraphrase A critical vulnerability in the PHP ecosystem was discovered when GitHub began rolling out new token formats that were incompatible with Composer's token validation code. For 14 hours, any PHP project using popular GitHub Actions would have their repository tokens exposed in plain text when Composer installations failed. This created an unprecedented opportunity for supply chain attacks, as malicious actors could have used these tokens to push infected code to major packages like Laravel, which would then be automatically distributed to hundreds of thousands of developers. The vulnerability was caught and patched quickly, but it revealed serious gaps in how the PHP community handles ecosystem-wide security incidents. ## ๐Ÿ“ฐ Newsletter ๐Ÿšจ **PHP's Close Call: A Supply Chain Security Wake-Up Call** Last week, the PHP community dodged what could have been the largest supply chain attack in its history. A vulnerability in Composer exposed GitHub tokens in CI logs, potentially allowing attackers to poison popular packages and execute code on hundreds of thousands of developer machines. **What happened:** GitHub's new token format broke Composer's 13-year-old validation code, causing tokens to leak in error messages during failed builds. **Why it matters for WordPress:** Many WordPress plugins and tools rely on Composer packages. A successful attack could have affected the entire ecosystem. **Key takeaways:** - Update Composer with `composer self-update` - Review your GitHub Actions permissions (use principle of least privilege) - Enable tag protection rules on critical repositories - Consider your dependency hygiene practices The incident highlights how interconnected our development tools have become. It also revealed gaps in cross-project security communication that the PHP Foundation is now working to address. **Learn more:** Check out the security settings in your GitHub repositories and consider which packages you truly need in your projects. A little dependency hygiene goes a long way! ๐Ÿ›ก๏ธ ## โœ… Action Items #### DevRel - Create educational content about supply chain security best practices for PHP/WordPress developers - Develop workshops on GitHub Actions security configuration and token management - Build examples and templates showing secure CI/CD patterns with minimal permissions - Track and measure developer adoption of security best practices through surveys and metrics #### OSPO - Establish incident response procedures for ecosystem-wide vulnerabilities affecting multiple vendors - Create scanning and monitoring for exposed tokens in public CI logs across organization repositories - Develop cross-project communication channels for security incident coordination - Implement automated SBOM generation and dependency vulnerability tracking #### Community - Establish security working group with representatives from major PHP frameworks and tools - Create shared security disclosure protocols for ecosystem-wide incidents - Develop community-wide security education initiatives and resources - Set up emergency communication channels for critical security incidents #### WordPress - **Core:** Review GitHub Actions configurations for token security and minimal permissions - **Security Team:** Assess WordPress.org infrastructure for similar token exposure vulnerabilities - **Plugin Review:** Develop guidelines for secure CI/CD practices in plugin submissions - **Documentation:** Create security guidance for plugin and theme developers using GitHub Actions - **Meta:** Audit WordPress.org build and deployment processes for token management best practices #### The WPCC - Fund security audits of critical WordPress ecosystem dependencies and build tools - Support development of shared security infrastructure for WordPress vendor coordination - Create grants for security tooling and vulnerability disclosure platform development ### FAIR - Establish vendor-neutral security coordination protocols with transparent funding and governance - Develop technical standards for secure package management and distribution - Create policy frameworks for responsible disclosure across multiple vendor ecosystems ## ๐Ÿงช Hypotheses - Cross-ecosystem security incidents will become more frequent as development tools become more interconnected - Traditional single-vendor responsible disclosure models are inadequate for modern supply chain vulnerabilities - Automated token scanning in CI/CD platforms could prevent similar incidents if implemented proactively - Community-driven security coordination may be more effective than vendor-led approaches for open source ecosystems - The window between vulnerability discovery and exploitation will continue to shrink due to AI-assisted attack automation ### ๐Ÿ“Š Methodology - **Historical analysis**: Compare incident response times across similar supply chain vulnerabilities in different ecosystems - **Network analysis**: Map communication pathways used during the PHP incident vs. official disclosure channels - **Simulation studies**: Test effectiveness of proposed cross-ecosystem security coordination protocols - **Survey research**: Assess developer awareness and adoption of supply chain security practices - **Quantitative analysis**: Measure token exposure frequency across popular CI/CD configurations ### โ“ Questions - How many similar token validation vulnerabilities exist in other package managers and build tools? - What would be the optimal structure for a cross-ecosystem security coordination body? - How can we balance speed of incident response with responsible disclosure principles? - What automated monitoring could detect supply chain attack attempts in real-time? - How do we establish trust and verification in emergency security communications? ### ๐Ÿ”ฎ Future Research - Development of AI-powered supply chain vulnerability detection systems - Cross-ecosystem incident response protocol standardization - Token lifecycle management best practices for CI/CD environments - Community-scale security coordination governance models - Real-time supply chain attack detection and prevention mechanisms ### ๐Ÿ’ก Implications - **Practice**: Immediate need for dependency hygiene audits and CI/CD security reviews across all projects - **Policy**: Requirements for coordinated disclosure protocols in open source foundation governance - **Security**: Investment in automated token scanning and supply chain monitoring infrastructure - **Education**: Integration of supply chain security into developer training curricula and certification programs - **Funding**: Support for shared security infrastructure and cross-project coordination platforms ### ๐Ÿง  Learning Styles Assessment Reading level: Flesch-Kincaid Grade 12 - technical content with specialized terminology but clear explanations #### ๐Ÿ‘€ Visual learners **Present**: Timeline of incident response, token exposure window diagrams, supply chain attack flow charts **Improve**: Architecture diagrams showing GitHub Actions โ†’ Composer โ†’ Packagist flow, before/after security configurations, visual incident response playbooks #### ๐Ÿ”Š Aural learners **Present**: Video format with clear narration and explanations **Improve**: Audio summary of key security recommendations, podcast interview with incident responders, narrated walkthroughs of security configurations #### ๐Ÿ—ฃ๏ธ Verbal learners **Present**: Clear definitions of technical terms, analogies comparing to physical supply chains **Improve**: Glossary of security terms, story-driven examples of attack scenarios, plain-language security guidance #### ๐Ÿงฉ Kinesthetic learners **Present**: Step-by-step GitHub Actions configuration examples **Improve**: Hands-on security audit checklists, interactive vulnerability assessment tools, practical labs for secure CI/CD setup #### ๐Ÿง  Logical learners **Present**: Timeline of events, causal analysis of vulnerability, decision tree for incident response **Improve**: Risk assessment matrices, quantified impact analysis, formal incident response procedures with clear decision points #### ๐Ÿง‘โ€๐Ÿคโ€๐Ÿง‘ Social learners **Present**: Community response coordination, Discord/Slack communication examples **Improve**: Collaborative security review guides, peer code review checklists, community incident response simulations #### ๐Ÿง˜โ€โ™€๏ธ Solitary learners **Present**: Self-assessment questions about personal repository security **Improve**: Individual security audit worksheets, self-paced security training modules, reflection prompts on dependency practices #### ๐ŸŒฟ Naturalistic learners **Present**: Ecosystem-wide impact analysis, environmental metaphors for supply chain health **Improve**: Sustainability thinking about dependency choices, ecosystem health metrics, lifecycle impact of security decisions ### ๐Ÿ“Š Accessibility Assessment #### Screen reader semantics **Present**: Clear headings and logical structure, timestamp links with descriptive text **Improve**: More descriptive alt text for technical diagrams, ARIA labels for complex interaction flows #### Keyboard navigation **Present**: Standard video player controls, text-based content navigation **Improve**: Keyboard shortcuts for timestamp navigation, skip links for different content sections #### Contrast and readability **Present**: High contrast video player interface, clear text formatting **Improve**: Ensure code examples meet contrast requirements, larger font options for technical content #### Text resizing and reflow **Present**: Responsive text layout in transcript **Improve**: Test technical diagrams at 200% zoom, ensure code blocks remain readable when enlarged #### Media and captions **Present**: Full transcript provided, clear audio narration **Improve**: Add audio descriptions for visual elements, downloadable transcript with timestamp markers #### Forms and interactions **Present**: Simple video player interactions **Improve**: Accessible feedback for any interactive security assessment tools, clear error messages ### ๐ŸŒ DEIB Assessment #### Cultural breadth **Present**: Global perspective on supply chain security impacts **Improve**: Include examples from different geographic regions, consider varied regulatory environments #### Inclusive language **Present**: Technical jargon explained in context **Improve**: Plain-language summaries of key concepts, avoid assuming prior security knowledge #### Neurodiversity support **Present**: Well-structured content with clear headings **Improve**: Consistent formatting patterns, predictable information architecture, summary boxes for key points #### Representation **Present**: Focus on technical merit and community response **Improve**: Highlight diverse contributors to security research, inclusive examples in documentation #### Psychological safety **Present**: Non-judgmental tone about security mistakes **Improve**: Emphasize learning opportunities, provide clear guidance without blame, supportive community response examples ### โœ… Logical Validity #### Strong reasoning - **Causal analysis**: Clear connection between GitHub token format change โ†’ Composer validation failure โ†’ token exposure (timestamp 8:15) - **Evidence-based assessment**: Concrete examples of affected repositories and potential impact scale (timestamp 3:45) - **Risk quantification**: Specific timeframe analysis and attack window mechanics (timestamp 18:30) #### Fallacies - **Appeal to consequences**: "This could have been the worst attack ever" - while true, focuses on fear rather than systematic prevention (timestamp 22:00) - **False dilemma**: Presents responsible disclosure vs. fast response as either/or choice rather than exploring hybrid approaches (timestamp 15:45) - **Survivorship bias**: Analysis focuses on why this incident was caught quickly, may miss incidents that weren't detected (throughout) #### Tone of voice Professional and educational with technical expertise. Examples: "Let's read through to see what actually happened" shows analytical approach; "That's pretty bad" acknowledges severity without sensationalizing. ### ๐Ÿ›ก๏ธ DARVO Analysis No DARVO patterns detected. The content focuses on technical analysis and community response rather than interpersonal conflict or blame assignment. The narrator acknowledges complexity of disclosure decisions and presents multiple perspectives fairly. ## ๐ŸŽฏ Conclusion The PHP ecosystem's narrow escape from a massive supply chain attack serves as a critical wake-up call about the interconnected nature of modern development tools. While quick community response and technical safeguards prevented disaster, the incident exposed fundamental gaps in how we handle cross-ecosystem security threats. The key lesson isn't just about updating Composer or securing GitHub tokensโ€”it's about building resilient communication channels and response protocols for an increasingly connected development landscape. For WordPress developers and the broader PHP community, this incident underscores the urgent need for better dependency hygiene, security awareness, and collaborative incident response capabilities. ## ๐Ÿ”„ Refactor Appendix โ€” PHP barely avoided disaster > Density: 2โ€“5 Literature atoms (H3), 0โ€“3 Permanent candidates (H4). > Defaults: 3 Literature, 1 Permanent. Resonance filter: if you can't > articulate in one line why a claim matters, drop it. ### ๐Ÿ“š Literature Atoms (H3) #### Supply chain vulnerabilities require ecosystem-wide disclosure protocols > [!quote] ID:rw_timestamp_1515 > The responsible path and the fast path diverged. The responsible path was file with the composer security advisory queue, file a hacker one against GitHub, wait. The fast path was open Discord, ping mod mail, open X, find someone who knows the people you need. **Context & use**: - Timestamp 15:45 in discussion of disclosure challenges - Demonstrates inadequacy of traditional single-vendor security reporting for cross-ecosystem threats **Tags**: #articleseed #security-protocols **Highlight**: [view highlight](https://www.youtube.com/watch?v=dzQEe7u3k2I?t=945) <!-- atomicity: 1 idea โœ“ --> #### Token validation should never expose credentials in error messages > [!quote] ID:rw_timestamp_0815 > Notice that the logged token wasn't a typical PAT. It had a GitHub app ID embedded in it. That small an ID suggests a very old GitHub app, most likely GitHub Actions bot. **Context & use**: - Timestamp 8:15 during root cause analysis - 13-year-old Composer code included rejected tokens in exception messages, enabling the vulnerability **Tags**: #xref #secure-coding **Highlight**: [view highlight](https://www.youtube.com/watch?v=dzQEe7u3k2I?t=495) <!-- atomicity: 1 idea โœ“ --> #### Package manager automation creates amplified attack surfaces > [!quote] ID:rw_timestamp_2200 > In the PHP ecosystem, push access to a repo is push access to packages. The moment you push version blah blah blah to vendor package, packages publishes it, and the next composer install anywhere in the world pulls your release. **Context & use**: - Timestamp 22:00 explaining attack mechanics - Shows how convenience features become security multipliers in supply chain attacks **Tags**: #xref #automation-risks **Highlight**: [view highlight](https://www.youtube.com/watch?v=dzQEe7u3k2I?t=1320) <!-- atomicity: 1 idea โœ“ --> ### ๐Ÿง  Permanent Candidates (H4) #### Community networks enable faster incident response than formal channels **Claim**: Informal professional networks consistently outperform official security disclosure channels for ecosystem-wide incidents requiring immediate coordination. **Evidence pointer**: [[#Supply chain vulnerabilities require ecosystem-wide disclosure protocols]] **Scope & conditions**: Applies when vulnerabilities affect multiple vendors simultaneously; breaks down when reporters lack established network connections **Links**: - [[Incident response protocols]] โ€” because contrasts formal vs. informal coordination - [[Security disclosure models]] โ€” because extends beyond single-vendor scenarios - [[Community governance]] โ€” because applies-in crisis coordination <!-- atomicity: 1 idea โœ“ --> ### โœ๏ธ Fleeting Queue - [F1] Research other package managers for similar token validation vulnerabilities โ€” #security-audit #fleeting - [F2] Document PHP Foundation's new disclosure protocols mentioned for Monday release โ€” https://www.youtube.com/watch?v=dzQEe7u3k2I #fleeting #follow-up ### โ†’ Things 3 (max 2 actions) - [ ] Audit personal GitHub repositories for exposed tokens in Actions logs from vulnerability window โ€” security review #things - [ ] Update all project Composer installations and review GitHub Actions permissions โ€” dependency hygiene #things --- # ๐Ÿ“„ Full Transcript *Auto-generated from YouTube captions ยท timestamps roughly every 30 seconds ยท [watch on YouTube](https://www.youtube.com/watch?v=dzQEe7u3k2I)* **[00:00]** Yesterday, the PHP community barely avoided absolute [music] disaster. That's how this blog post here starts, explaining how a supply chain vulnerability was detected in in composer, and it had the possibility for things to go [music] very bad for PHP. It's funny how how this thing happens just a week or two after that previous package that got compromised. I made a video about that if you want to [music] watch it after this one. So, let's read through here to see what actually **[00:32]** happened. [music] In short, for a 14-hour window yesterday, which is already a couple of days, yeah, last week basically, any PHP project that was hosted on GitHub ran composer install after shivammathur set a PHP in GitHub actions, which is basically all PHP projects that rely on PHP used this action here. That ran that workflow against its main branch and was triggered by on push, on pull request target, or on schedule. Other triggers are likely impacted, but **[01:03]** these are the most common. When you had that situation, you had a chance that the GitHub token, typically write enabled, could would be publicly logged in GitHub actions log on every build. For those 14 hours, any malicious actor can uh monitoring one of those uh repos could have pushed [snorts] new tags and commits without anyone noticing. If properly leveraged, this would have been the largest supply chain breach the PHP ecosystem has ever seen. To make matters **[01:34]** worse, we're not even done yet. The same change may roll out again in a week at GitHub's discretion, and there's a chance it bleeds beyond PHP in other ecosystems. That's pretty bad. Uh we're going to continue to read through the rest of the post as well, but what they're saying here is that for 14 hours, if people were monitoring GitHub actions and actually got a hold of your GitHub token, they would have been able to make changes to any repository that **[02:05]** leaked their token uh without anyone noticing, tag those changes, which would then automatically be released through composer, and anyone who pulled them in would get the affected release. Just imagine this happening with Laravel, for example, which has, I don't know, let's see, like an average of 300,000 installs per day. So, for 14 hours, let's just say 150,000 installs could have been affected by this. Could just pull in the affected version without anyone ever noticing. **[02:37]** That's That's the the extent of where this uh vulnerability goes to. As far as people know, I don't think this has been exploited, but we'll read about it. The person who discovered Damien here uh maintains a set of GitHub actions for the Magento community. They exist to lower the barrier uh uh the barrier to entry into the Magento and Magento OS ecosystem, so that a company wanting to ship or store an extension can go get working with CI without much fuss. Yeah, that's that's what all these actions do, right? To **[03:08]** just make it easier to to use GitHub actions or CI platforms. For me, that means maintaining a wide variety of GitHub actions that work across many versions of Magento and Magento OS. To handle that, I main uh maintain a fairly complex action called supported version, which holds the complexity matrix for every supported version of Magento and Magento OS. Yeah, it's it's like the more GitHub actions I can avoid, the better in my case, to be honest, but yeah, it's not my thing. Uh I have for **[03:40]** Tempest, I'm so happy that that other people are helping me out with GitHub because anything DevOps {slash} CI related is is really like, nah, just give me PHP. Uh while I was working on this action for the Magento release yesterday evening, my test started failing intermittently after some small changes to this action. The failures didn't track with anything I changed. When I went to debug the failure, I saw the following. In base IO, which I think is is a composer class, we'll we'll see in a minute. Your GitHub OAuth token for **[04:13]** GitHub contains invalid characters and then just prints the old token. I was surprised more than anything. My first instinct was that screwed something up. Bash scripting processing the composer auth I I feed into my actions. I've seen secrets in my logs before whenever a script clobbers a string with new lines, so this looked like my fault. It took me about an hour, I'm not a very fast person, to realize that my actions weren't at fault. Nothing my actions Nothing I changed could have triggered this outcome. Shortly thereafter, I **[04:45]** concluded that something outside my actions was wrong and wrong in a way that probably wasn't unique to me. So, I asked Claude, as you do, giving it all the context I just uncovered. Claude pointed out two things I'd seen but not noticed. The logged token wasn't a typical PAT. This is like a personal access token. It had a GitHub app ID embedded in it, specifically this one. That small an ID suggests a very old GitHub app, most likely **[05:15]** GitHub Actions bot. And then Shiva matter said a PHP automatically inserts that token into composer auth as the GitHub OAuth value. That makes sense. At this point, I still wasn't too concerned. I naively thought this was one a one-off bug in how tokens had been generated. I knew the GitHub Actions token expires when the job ends. I had looked at workflow token security on my repos before, and the leak was on a pull request, so the token's permissions were already fairly restricted. I didn't lose sleep over it. **[05:47]** I merged my changes to my main branch from my pull request, fully expecting that this would recover and I could move about my day. Then my main branch CI failed on a completely different job and I became concerned. Earlier yesterday, uh 10 sec router was breached because their main branch's cache was points poisoned by a pull request. Yeah. I was very concerned that something like that uh was going on here. It was at this point with the read writes token visibly in my logs that I knew something was up **[06:18]** and that I had to dig in and act quickly. I had Claude triage the the error message from the log in base IO here, figuring that it would do a better job at scouring Composer's project code for the offending line faster than I would. And it uncovered it immediately. And so this is the code that throws the exception and then actually includes the token in the exception message, which is probably not really a smart thing to do, but you know, who thinks about these things, right? Allowed characters for GitHub tokens are these plus dots, which **[06:49]** are at some point used for GitHub app app integration tokens. So basically, if your Composer checks if your token doesn't match a given format, it will throw an exception because then it's an invalid token. But what it also does is print that invalid token in the exception message. It also uncovered, and huge props to Claude here, this would have taken me hours on my own, that that GitHub was rolling out a change to their token formats. Notice about upcoming new format for GitHub app **[07:22]** installation tokens. So the token format has an installation tokens, which will be changing to uh yeah, another format. A change to the token format. At this point, I knew very distinctly that, which is my inner child telling me, and now you, that something is undeniably wrong. How I acted. Here's what I did in order. I disabled all actions in my organization, better safe than sorry. I pinged the major OS Discord and asked them to pause all GitHub actions immediately. I submitted a security advisory report to the **[07:52]** composer project based on my quick analysis. I reached out to Jessica who would have previously connected with on on fake repository hoping she would quickly and quietly get me in touch with people at the large frameworks at risk. My request she reached out to Taylor and asked him to pause Laravel GitHub actions. I pinged ModMail users on Laravel's Discord. Shortly thereafter their actions were disabled. I'm not sure which channel did it. I called a colleague and started trawling GitHub repos to confirm I wasn't the only one **[08:22]** affected. I pinged Barry on Slack informing him to disable his GitHub actions as well. As an aside, I'm a small fish compared to many of the well-known developers in the PHP ecosystem. I didn't know if anyone would take me seriously and I wanted to make sure no one was hurt simply because I wasn't loud enough. I fully understand responsible disclosure. I filed several HackerOne reports before and I've received bounties for some of them. But when the breadth of concern is this large, this painfully obvious, this critical, and had been 4 hours since I'd **[08:54]** opened the advisory without a response, I felt I needed to take my own action through quiet fast channels to make sure the largest parts of the ecosystem were protected as quickly as possible. And what were the chances I could get get Niels or Jordi, these two are the maintainers of composer, on a call minutes after I submitted an advisory? First, what were the chances I could get a HackerOne report submitted and reviewed by GitHub in the same time frame? You have to understand when something like this happens, you don't want people to know about it because there might be bad people out there who **[09:24]** will abuse it as soon as it's getting known. You want the right people to know about it who can fix it. In this case, that's composer who's the root of the issue. You want to make sure that the the big players like in Symphony and Laravel are are aware of it as well. But you really don't want to be out there in the open shouting there's a huge security issue because it it it's it's not good. In reality, likely no one was compromised. In reality, likely no one next week will be compromised when GitHub rerolls the change out. Composer **[09:56]** has already issued patches to all major versions of Composer combined, even non-supported non-LTS versions. Ash even matter has has already bumped their main line to update to a secure version of Composer as well. That being said, the bulk of the reason that this wasn't as impactful is primarily due to three components. First, Composer validator fails fast. The exception bubbles up and the job dies. This causes the GitHub token to be revoked the moment the job exists. From the attacker's perspective, **[10:28]** the token is live for the seconds between Symphony console writing the message to standard error and the runner tearing down the job. That window is real, but it's narrow. To be clear, it's not narrow enough to be unexploitable. I have a proof of concept which showcases direct writes to main using the stolen token in this very narrow window. With a sufficiently sophisticated polling setup, you can do this in under a second. The second reason is that the failure was loud. While I was running **[10:58]** through GitHub trying to figure out whether I was the only person hit, I could see the failure landing in CI logs all across most of the top PHP repos, Laravel, Composer itself, WooCommerce, OpenEMR, Sylius, Laravel Debugbar, and many other smaller projects. Anyone watching their own CI saw it within minutes. If it was quiet, then it would have gone on far longer. Who reads the build logs of a passing job? The third reason is that Composer shipped patched releases and the advisory went out the next morning. GitHub paused the format **[11:29]** rollout shortly afterward. The response time from notification to resolution was remarkable. Now, the potential impact. Imagine someone saw what I saw earlier than I did, and who decided to do something with it instead of reporting it. The primitive you have is GitHub token from a push event on a repo's default branch printed in plain text to world readable log live for some number of seconds before the job ends. The tokens permissions vary by repo, but for the long tail of older apps and most maintainer configured workflows, the default is still contents rights. The **[12:01]** legacy permissive default. That's enough to push commits and tags to the repo it came from. Worse, the leak doesn't necessarily end precisely when the failing step ends. A pattern I have in my own workflows, and that a lot of Magento and broader PHP workflows have, looks like this. The intent is fine. When CI fails, freeze the workspace state into an artifact so a human can debug it later. In this bug's context, the step compounds the leak. The core concept is that it extends the life **[12:31]** token window, and the upload runs after composer has crashed and printed the token to standard error, but before the runner can tear down uh the job and revoke the token. Until the upload finishes, the token is still accepted by the GitHub API. For a small sandbox, that's seconds. For a fat integration test fixture with database dumps, it's many minutes, yeah. And an attacker polling failed runs gets a longer window to act, not a shorter one. In the PHP ecosystem, push access to a repo is push **[13:01]** access to packages, yeah. Packages pulls GitHub for new tags, that's true, yeah. The moment you push version blah blah blah to vendor package, packages publishes it, and the next composer install or composer update anywhere in the world pulls your release. Composer plugins execute code during install, although not entirely true, you have to accept this. That's That's what we saw in the in the previous video. But But still, it's a good point. So, a poisoned tag on a sufficiently popular package is a remote code execution on every developer's machine, CI runner, and **[13:33]** production deploy that runs composer install between the tag landing and rollback. Look at the names on the list above. Composer itself was failing in CI during this window. So was Laravel, so was Sylius. A successful XZ and push against anyone of those is a top 50 in history supply chain incident. Against several of them simultaneously with the same primitive, it would be the largest one ever in PHP and probably top three against all ecosystems. The Magento angle, since it's the corner of the world I actually live in, Magento and **[14:05]** Mejo as distributions are composer projects. Composer self update pulls from getcomposer.org, which redirects to GitHub releases. A poisoned composer binary would execute under every Magento store deploys pipeline in the next build. Most e-commerce platforms do not have the supply chain hygiene of, say, a rail shop. The blast radius would not have been bounded by who reads Hacker News. It is It is pretty true. And I mean, we've become pretty lazy because of the tools we're using. I talked about this in the previous video as well, like **[14:36]** the dependency hygiene concern. It's convenient that we can just, you know, pull in stuff that was published to GitHub and just is available through composer and it's all automated. It's It's super convenient. And it has so much potential of doing so much harm. So, the mitigating factors, a short list of things that intentionally or otherwise kept this from going off. Composer fails fast and fails hard. I do remember when when running the the 100 million row challenge, that one of the tokens got leaked there in a GitHub **[15:08]** action while we were debugging some stuff. I actually got an email from GitHub telling me that my token leaked, but I guess that since these these are GitHub action specific tokens, maybe GitHub doesn't check on those and only checks for like real personal access tokens that were generated on on the account or organization level. But that that would have been helpful, I reckon. Maybe a good lesson here is that you should never ever print out tokens in exception messages, but you know, you live and you learn. GitHub token is repo scoped even after even a successful **[15:38]** exploit against Laravel framework doesn't get you anywhere near Laravel Passport. Each repo's token can only act against a repo. Although for mono repos, you get a much wider scope. But yeah, stuff like Laravel Passport indeed is is a separate package. The 6-hour ceiling, GitHub hosted runners kept workflow tokens at 6 hours. Okay. Branch protection blocks the obvious move. Pushing to main directly on a protected branch fails. Yeah, and which is good. Don't know if that really **[16:09]** works though because you can tag releases on other branches, which is the the exploit from that previous video. You can do that just fine and composer will pick it up even if it's on another branch. So I don't know if that that's really good mitigation here. The rollout was gradual. Not every repo was on the new token format at the same time. Really good. And composer maintainers moved Yeah, really fast. Exactly. Now that's really great. What a typical public PHP maintainer has to do, upgrade composer. And that's really that's **[16:40]** really simple by the way. Just run a composer self-update. You can tighten permissions. Don't rely on legacy permissive defaults. Absolutely. Protect your tags. The blast radius of this leak is overwhelmingly an attacker pushing a tag and packages publishing it. Enable GitHub's tag protection rules or branch protection rules. Yeah, that is indeed a very good idea. Sweep your action logs for the window. Check your tag and commit history for the window. Mhm. [clears throat] Deploy the fixes before the next rollout. I think the the biggest fix **[17:12]** here and it's it's chief a matter action updates composer. As far as I understand that already happened. So, when it comes to GitHub actions at least, anyone using shivamuthu should be fine as far as I understand. But of course it's it should do all of this as well. What GitHub has to do, harden the runner secret maskers against console reframing. Today it matches registered values as exact substrings. Yeah. Yeah, so GitHub tokens are masked in output but **[17:42]** it it's apparently not not strict enough. Yeah, that is something GitHub should do indeed. Coordinate disclosure with the major package managers before the next rollout wave. Good point as well. Maybe we need a little more than just a small blog post. And automate testing of the common GitHub action setups for the top 50 programming languages. What other ecosystems have to do, grab your code base for hard coded GitHub token regexes. Yeah, good idea. Never interpolate a rejected credential into a user visible error. This really is important. Other credential paths the **[18:15]** same way and reach [snorts] out to GitHub security ahead of the next rollout. A final note on disclosure. I want to spend a couple of paragraphs on how this felt from the reporters side, not as a complaint but as a description of a structural problem someone with power should think about. The model for responsible disclosure assumes there is one vendor on the other end with a security team, a queue and a person paid to read the queue. Most of the time that's exactly how it works. You write up the bug, you submit it, they triage, they patch, you go home. This one doesn't have that shape. The root cause **[18:46]** was GitHub. The fix that mattered most quickly was at composer. The blast radius was every PHP framework maintainer on Earth and we have no shared security inbox. There was no way to reach all of them on a Tuesday evening in under an hour as someone they've never heard before. Every minute the leak went undetected was a minute an attacker with a scraper could have been turning failing builds into push to tag access. So, the responsible path and the fast path diverged. The responsible path was file with the composer security **[19:18]** advisory queue, file a hacker one against GitHub, wait. The fast path was open Discord, ping mod mail, open X, find someone who knows the people you need, cold call a colleague and hope they're taking you seriously by people who have no reason to know who you are. I picked both paths. I'm still not sure if I was right. I'm fairly sure that traversing the fast path was less wrong than waiting for the advisory queue to process what it would have been. What bothers me is that someone less well connected than I am with the same bug at **[19:49]** 10:00 p.m. on a Tuesday doesn't have a faster lever to pull than guess who knows the Taylor Otwell's of the PHP ecosystem. And that this is a pretty like important issue that he's raising. If you're reading this and you have power in this picture at GitHub, HackerOne, one of the major ecosystem maintainers, I would like for there to be a phone number, a pager, a shared cross-vendor for this is an ongoing ecosystem scale leak and I need an adult. I don't know what that looks like in detail, but submit a form and pray is not adequate for incidents **[20:21]** that look like this one. And I don't think we should keep relying on the fact that the next reporter happens to have my network, albeit small as it is. Especially in the modern era of AI as the discovery to abuse window titans, time to response becomes more critical than ever before. I I actually think this is the most important part of of this whole blog post. We need to think about this. Very well put. Thank you, Damien, for the effort you put into it. And I'd like to actually see if someone from composer, etc. replied. I do wonder about the value of **[20:53]** validating tokens client-side. Wouldn't it be enough to just let the API throw an error? And here is Jordi from composer saying, "Yeah, I tracked down the commit which was added this yesterday and it was 13 or 14 years old. I I a PR open already to delete the validation entirely from an upcoming release and I don't think it's relevant now that tokens are not really entered manually anymore. Back then you had to go and create it and type it in and if you messed up the copy paste, it was probably helpful to show you what was wrong, but today the landscape is very different. This is just like makes sense, right? But in hindsight, maybe **[21:25]** not the best idea. Yeah, on your last point regarding the disclosure path, please watch out for a PHP foundation next Monday. We've already been working on a plan for this especially considering the changes AI has brought through these processes. I entirely agree with you on how tricky this is in situations where a huge amount of people and projects are affected and communication becomes extremely challenging. This Monday, by the way, that's today, the day I'm recording this, I will follow up on this as well. I think [music] there are some some cool things coming. The PHP community barely **[21:56]** avoided an absolute disaster. We're lucky that that the person who caught this had good intentions and wanted to help out. >> [music] >> So, let's keep this as the second reminder now on this channel at least that PHP isn't immune for these kind of things [music] and that dependency hygiene really is a good thing to keep in mind. So, yeah, let me know your thoughts in the comments down below and I'll see you in the next one. Bye-bye.