Node.js Diagnostics WG Session  ## Metadata - Author: [[OpenJS Foundation]] - Full Title: Node.js Diagnostics WG Session - Category: #articles - Document Note: --- tags: [#nodejs, #diagnostics, #esm] - #nodejs - #diagnostics - #esm Author: Node.js Diagnostics Working Group Source: Node.js Diagnostics WG Session (YouTube) Reference: [Node.js Diagnostics WG Session](https://www.youtube.com/watch?v=PAAYyDN9SDE) date created: 2025-08-24 20:30 --- # Title: Node.js Diagnostics WG – ESM Challenges, Module Loading, and Observability ## Table of Contents: - [📝 Summary](#-summary) - [📖 Main Content](#-main-content) - [🗒️ Notes](#️-notes) - [📖 References](#-references) - [🗣️ Paraphrase](#-paraphrase) - [📰 Newsletter](#-newsletter) - [✅ Action Items](#-action-items) - [🧪 Hypotheses](#-hypotheses) - [📊 Methodology](#-methodology) - [❓ Questions](#-questions) - [🔮 Future Research](#-future-research) - [💡 Implications](#-implications) - [🧠 Learning Styles Assessment](#-learning-styles-assessment) - [📊 Accessibility Assessment](#-accessibility-assessment) - [🌍 DEIB Assessment](#-deib-assessment) - [✅ Logical Validity](#-logical-validity) - [🔍 DARVO Analysis](#-darvo-analysis) - [🔍 SEO Evaluation](#-seo-evaluation) - [🎯 Conclusion](#-conclusion) --- ## 📝 Summary: This Node.js Diagnostics WG session tackled the major challenges introduced by Node.js 20’s new ESM (ECMAScript Modules) loader. Key issues include the loss of visibility into module load times due to off-thread processing, lack of APIs for retrieving exports, and incompatibilities with APM tools such as OpenTelemetry. Proposed solutions include using Acorn for AST parsing, requesting V8 APIs for export visibility, and considering loader lifecycle hooks with Diagnostics Channel. The group also debated async local storage improvements, promise tracking, and how new Linux I/O ring optimizations affect observability. --- ## 📖 Main Content: The discussion highlighted the ecosystem disruption caused by Node.js 20’s new ESM loader and potential paths forward: - **ESM Loader Issues** Off-thread loading improves async compliance but obscures diagnostics, breaking tools like `import-in-the-middle`. - **Diagnostics Channel Limitations** While useful for passive observation, it lacks mutation capabilities needed by APMs for instrumentation. - **Potential Fixes** - Use **Acorn parsing** to extract exports, though clunky. - Advocate for **V8-level APIs** to expose exports natively. - Add **experimental flags** for major changes before LTS adoption. - **Async Local Storage** Suggested improvements could simplify async context management but risk backward compatibility issues. - **Promise Tracking** Enhancements allow tracing resolves/rejects, but ambiguity remains in lifecycle and GC behavior. - **File System + Performance** Linux’s IOUring speeds up small file reads but reduces observability of exact timings. ### 🗒️ Notes - [00:03](https://www.youtube.com/watch?v=PAAYyDN9SDE&t=3) ESM adoption breaks existing instrumentation (e.g., APMs). - [02:42](https://www.youtube.com/watch?v=PAAYyDN9SDE&t=162) Off-thread loaders make startup time diagnostics harder. - [07:27](https://www.youtube.com/watch?v=PAAYyDN9SDE&t=447) IOUring improves performance but hides granular file metrics. - [17:11](https://www.youtube.com/watch?v=PAAYyDN9SDE&t=1031) Need edge-case metrics for accurate diagnosis. - [23:56](https://www.youtube.com/watch?v=PAAYyDN9SDE&t=1436) Node 20 broke `import-in-the-middle`. Acorn parsing proposed. - [28:30](https://www.youtube.com/watch?v=PAAYyDN9SDE&t=1710) OpenTelemetry’s ESM support disrupted by Node 20. - [30:36](https://www.youtube.com/watch?v=PAAYyDN9SDE&t=1836) Proposal for Node.js core API to expose exports. - [34:41](https://www.youtube.com/watch?v=PAAYyDN9SDE&t=2081) Diagnostics Channel should ideally cover lifecycle hooks. - [41:29](https://www.youtube.com/watch?v=PAAYyDN9SDE&t=2489) Async local storage improvements debated. - [50:13](https://www.youtube.com/watch?v=PAAYyDN9SDE&t=3013) V8 continuation APIs may help diagnostics. --- ## 📖 References: - Node.js WG documentation on [Diagnostics Channel](https://nodejs.org/api/diagnostics_channel.html) - Acorn Parser [GitHub](https://github.com/acornjs/acorn) - OpenTelemetry + Node.js ESM updates - IOUring overview: [Linux Kernel Docs](https://kernel.dk/io_uring.pdf) --- ## 🗣️ Paraphrase: 1. **Original**: “Diagnostics Channel is passive, but we need mutation for instrumentation.” **Paraphrase**: While Diagnostics Channel was designed for observation, APMs require it to actively modify runtime state for tracing. 2. **Original**: “Node 20 broke import-in-the-middle; Acorn parsing might fix it.” **Paraphrase**: The latest Node.js release disrupted existing tools, forcing teams to explore AST-based parsing as a workaround. --- ## 📰 Newsletter: ⚡ **Node.js Diagnostics: ESM Shakeup in v20** The Node.js Diagnostics WG is wrestling with the fallout of Node.js 20’s ESM loader changes. Off-thread loading improves async behavior but breaks diagnostics tools, leaving APMs and OpenTelemetry scrambling. Proposals include Acorn-based parsing, new V8 APIs, and lifecycle hooks in Diagnostics Channel. 👉 If you maintain Node.js tooling, now’s the time to test your stack against v20 before LTS in October. 🔗 Watch the full session: [Node.js Diagnostics WG](https://www.youtube.com/watch?v=PAAYyDN9SDE) --- ## ✅ Action Items: - Advocate within WordPress and WPCC for **early ESM testing** in Node 20 before LTS. - Push for **diagnostics-friendly APIs** in Node.js core (aligns with OSPO advocacy for sustainable tooling). - Frame discussion as **governance + contributor health** issue: observability tools are critical infrastructure. - Leverage parallels for **WordPress plugin dev** (e.g., compatibility-breaking changes without migration paths). --- ## 🧪 Hypotheses: - If Node.js exposes an API for module exports, APM breakage will drop significantly. - IOUring will improve performance but reduce observability unless complemented with better metrics. - Async local storage redesign can replace async hooks for context propagation. ### 📊 Methodology: - Benchmark Node.js 20 vs. Node.js 18 across real workloads (APM instrumentation enabled). - Simulate high-dependency apps with both CJS and ESM to measure startup diagnostics loss. - Survey APM vendors on Node 20 migration blockers. ### ❓ Questions: - Should Node.js require experimental flags for disruptive changes before LTS? - Can APMs realistically adopt AST parsing for exports? - How can OSS communities push back on breaking changes that harm ecosystem observability? ### 🔮 Future Research: - Investigate adding a **unified loader diagnostics API**. - Explore whether V8 continuation context APIs could become official Node.js hooks. - Study long-term effects of IOUring on dev observability. ### 💡 Implications: - Without fixes, Node.js risks **ecosystem fragmentation** (tools stuck on pre-20). - Improved APIs could make Node.js a leader in **developer experience observability**. - Lessons can guide **WordPress governance** when introducing breaking changes. --- ## 🧠 Learning Styles Assessment: - **Visual**: No visuals ❌ → add flow diagrams of ESM load lifecycle. - **Aural**: Session recording ✅. - **Verbal**: Deep technical discussion ✅. - **Kinesthetic**: No interactive demos ❌. - **Logical**: Strong reasoning on module resolution ✅. - **Social**: Collaborative WG format ✅. - **Solitary**: Limited reflection ❌. - **Naturalistic**: No analogies ❌. --- ## 📊 Accessibility Assessment: - Transcript improves accessibility ✅. - Dense technical jargon = barrier ❌. - No visual aids ❌. - Suggest: **captions + simplified summary docs** for broader audiences. --- ## 🌍 DEIB Assessment: - **Cultural Relevance**: Western-centric, lacks global perspectives ❌. - **Inclusive Language**: Neutral ✅. - **Neurodiversity**: Heavy jargon may alienate ❌. - **Representation**: Technical voices only ❌. - **Psych Safety**: Tone was supportive, collaborative ✅. --- ## ✅ Logical Validity: - **Sound Reasoning**: Correctly links off-thread design to diagnostics opacity. - **Causal Reasoning**: ESM loader - Summary: The discussion revolves around the challenges faced in implementing esm (ECMAScript modules) in the JavaScript ecosystem. The use of off thread loading for file retrieval causes difficulty in measuring load times accurately, and the inability to log events to the diagnostic channel from the off-thread process. There is a plan to use Acorn to parse the modules and extract imports. The conversation also touches on the concurrency of file retrieval and processing and how it affects the main thread, especially in virtualized environments. - URL: https://www.youtube.com/watch?v=PAAYyDN9SDE&list=PLyspMSh4XhLO15jJD9Sj069fN18j12AbQ&index=13&ab_channel=OpenJSFoundation there's been a lot of things that arelikegets gets implemented in the ecosystemlike promises there's like ecosystemimplementations and thenlikethey they just kind of like went rightinto let's just like copy paste this andnot like let's learn from this and liketune this to something thatit's like more suitableyeahum okay so what are we gonna do about uhthe new esm litter are we gonna put inlike Diagnostics Channel something inthere so we can see because what was thename of the module where you couldyeah in part of the middle is brokenright nowuh so there's a threat about itum brand English um trying to figure like trying tofigure out what's going to be the bestway to make it workumitit sounds kind of ugly butumthe plan at the moment seems to beum we're just going to add Acorn andlike just parse the modules andlike pull all the Imports out that waywhat do you meanum so the way the import in the middleworks is like like we Lo we load the module find out like what all of theexports of it are and then we create acopy of that module thatum basically just re-exports that butthenum we register a whole bunch of likeSetter callback things that like our owninstrumentation can just call thesefunctions to mutate the like because itlike in esm if you change the referenceum it'll change what is exportedoh okay um it's like even after like you'vealready done the export like changingthe ref like that like variablereferencewill change the exportum yeah that's basically what we've beendoingum butthat worked before because you couldlike load like you could load a modulefrom within the loader but you can't dothat anymorewell yeah moving it off thread made itso the loader can't load modules yeah because that makes sense okayyeahum so whatI I didn't the patch was huge and kindof messed up so I didn't dive into ittoo deep but what about it is that likeare they just loading the file offthread because you can't load it intothe Heap off threadright so what off thread work is itactually doingbasically that it's resolve the graphload the files and hand them back to the main threadoh a student oh it's doing the wholegraph resolution off thread yeah ahaokayis there any chance we could put in Iguessminus the mutating state part right uhlike and we don't need to be able to dothat but what we would like to be able to do is just seewhen and what modules are loaded rightand there's not a Diagnostics Channelin there yeah right to be able to justsee what's being loadedyeah so I had a pull request that Iopened at one point that yeah I'm justgetting back toum yeah I have that bookmark yeahyeah yeah I Ihad like originally built that in like aparticular way and then like we decided like we wanted another thing I builttracing channel uh like we want to layit on top of that so like we can measurelike how long an import like takessomething things like that really wellwe can measure how long our requiretakes Imports because of all thethreading nonsense now we won't be ableto see anything actually useful fromthatum yeah that's actually kind of yeahthat's a total freaking pain becausethat's going to be impossible to measurebut it's still good if you're runninglike in a duck container with a single CPU that's still gonna have overheadthat affectsthe main main thread but you just won'tbe able to measure that because it'slike yeah they'll have to pause the mainthread in order to run all theprocessing off the main threadbut then it'll hand back so so the mainthread will report like well I wasn'tactually doing anythingbut that's just because all CPUresources were being used somewhere elseyeah yep yeah so with the like moving tooff thread loader yeah we lose like all visibility into the startup load Timestuff other than like we can see likeblack box like from starting the processuntilwe actually can run some code it tookthis long yeah but we don't know whathappened in that time was there anywas there actually many gains doing itoff thread because sincewell the the the core reason for it islike the esm team wants loaders to beused for common Js which means it has to be synchronousum but because it because like by thespec it's async they can't do it on thesame threadif they do it on another thread thoughlike it can be async on that thread andthen they just wait for it onthe main interop is the reason for withcommon JS that they won yeah okayI can't tell whether that's better orworse I I have opinions yeah I can see that onyour facewellyeah that's unfortunateis it because diagnostic Channel youcan't call you can't commit the eventsoff the threadnot as it works right now like I I couldmake a way to do itbut yeah itit's it's not a thing that that it cando inherently currently and and yeah like none of this esm stuff was runby like any of the apms to it was justlike yeah we're just gonna go anddo our thing and you know break theworld for everyone as they are creatingthe tree or structure whatever is calledof all the files that need to beimportedare theyasynchronously reading the fileslike basically they could have like say they read a file like okay we needthese five other files they don'tsynchronously read those five files theysay go out and fetch those five and thenlet me know when you're done bringingthem backyeah yeah it's async yeah sothat's a even bigger pain in the buttbecause that's being run on a differentthread in Libby V in order to do thefile system operation well now withlatest wbb it's going to be an uh IOUringso yeah it doesn't even use the samemechanism anyway it doesn't even use the thread pool anymore no it'll be V sothat'll be funyep yeah so like it like it's like it'llload like your top top level script andthen like basically parse out all theImports get like the like specifier foreach of these Imports and then likelike concurrently try to load like allof theseonce like any one of those load is gonnalike parse that one and then like tryand get all the specifiers out of thatand like it just keeps going down and down until it runs out of things that'sadded to this queue of stuff it'swaiting for and then it just takes thatmassive glob of crap and hands it backto the main thread to yeahthis tree of async stuff making moreracing stuff and then yeah here's my BigBlob of stuff at the endoisoyeah so yeah we can't we can't detect ill it's impossible to detect when thefile was redlike I mean down at the Libby V levelit'll be impossible to even tell whenone of those files is read from the filesystem because we don't know when usingthe new IOU ring it just happensmagically and then it's just deliveredto you yeahum I mean I guess we could hookthe other thread to to take note of whenit acknowledged the file was a wasreceived yeah but that's the best we could dowelllike like wish we can probably hook intolike in in the loader thread like itit knows like for like for each of theselike specifiers that it's requested athing for it knows when it gets theresponse for thatso like that is a signal that like thisindividual file is just trying to loadis donebut like loading and like all the parsing to trigger the next tree ofthings but yeah and I'm not sure ifum like if if this file loads more filesif like the outer file would be like ifit'll say this is done when it still hasstuff below it or if it'll be like ithas to like go all the way down thencome back up before the upper onea but I've had it I've had the esmmodule resolution path explained to me a few times and I keep forgetting whetherit's top down or bottom up or whateveryeahumit's all the stuff's being done then itpasses it back he included the timesin the data in the past five and thencalled the existing guidelineslike this I'm trying to remember itdepends on like when you send the eventthat's where the timehere's the time stamp I want to give youfor this today uh well Diagnostics Channel itselfdoesn't like have any concept of timestamps it leaves it up to the user tolike the like that the Handler triggerssynchronously from where the publish isso like the way you're supposed to useit is like you just set the timewhenever the like subscriber istriggeredum if you are like pulling a thing offthread like that like we could addtimestamps to it like there's nothing stopping us from adding timestamps inthese like weird casesyeah because yeah it seems like thatwould be the only way then is just torecord metrics off thread and have thatbe part of the metadata sent back to themain thread ofyepyep the theany measurement of how long it took theto read the filepointless because you can say I sent therequest to Libby V to retrieve the file at this timeand then I received the file at thistime butthat's not actually how long it tookthat's just how long it took us toretrieve it right with with requiressince all synchronous you could actuallytrack and see this this is how muchoverhead reading from the file systemtookthat is impossible it's only a bit like you get when it'savailableis that Delta not what's interestingyou're like actually becauseyou're more interested in talk Xlike you're trying to measure the effortThrough Timeversus so okayum this is one thing that bothers meabout using esm on locallyis that you know it kind of makes itmakes sense in the browser because youcan make requests to many servers all atthe same time on the file system you're only going to be retrieving onefile at a time anyway it's like whileyour hard drive is delivering one fileit can't be simultaneously deliveringanother it has to do one at a timeright soit was easy to see like oh okay if ifwe're leading loading a lot of my a lotof files and likethousands and thousands of files in abig projectthat could especially in a virtualizedenvironment that could have noticeable overhead on your startup timebut um but the problem with right now isdoing the off thread loading what willhappen is it will go off threadand that will be busy so the main threadwill just stall and have nothing to dofor a secondum then off threadit will need to be doing all this workand since there's only like say in avirtualized environment there's only oneCPU available the the that thread is consuming all theCPUbut that's no longer measurable by themain thread we can no longer say likeum like if you take a CPU output you'regoing to see high CPU usage but then ifyou take elu the event Loop utilizationlook at the main thread it's going toread zero it's like what the frick isgoing on right soit'll be much harder to detectwhy there's all that extra overheadbecause not only is it I assume parsing isn't very expensive is itdon't know okayyeah I I I I I I don't think it's tooexpensive like like it's like it's allhandled in V8 and like it skips all thelike regular syntax it just looksspecifically for those important for theImportsso I think it's fairly quick okaybut yeah I have seen a virtualizedenvironments where a lot of file systemoperations can lead to noticeableoverhead so it's like in your case you might want to bundle and just read onebig file right instead of reading athousand two thousand tiny filesumthat but that's not how much time itactually took to read from the filesystem well it itumso it will send out I need five filesright and then it will return when allfive files are available but the problemisit only registers one file at a timeright so it's like all right I read Ihave now this one file ready I'm nowgoing to process this fileokay I read the next file this one waswritten so technically all five areready at the same time all of them should have the same time stampbut they don't because you can onlyprocess each one in sequence and so thetimestamp for each file would bedifferentit might not actuallyfile but I'm wondering like without youactually care about it or do you careabout itnowwait a second five seconds before I gotit back so my overhead isa file [Music]well no because like if somethinghappens in between then you likenow since this is happening off threadit shouldn't be an issue fortunatelyit'd be even worse it was because likeif it was on the main thread you couldsay I need these files and then I'mgoing to do something in betweenand that's going to block the event Loopand then the file will be availablelater and so you actually have no ideawhen the file was retrieved fortunately that's not a problem here because it'sonly I'm assuming it's only handlingretrieving the files rightI'm just saying formy history in doing research on how toproperlyidentify the health of an event Loop andthe health of your application havingspecifics mattersandumread times for like lots of small filescan be different than larger files and just having those specific metrics canreally help identifyextreme cases like the edge cases andwhen you're reading metrics the casesyou care about are the edge cases ifyour application is fine you don't givea crap about the metrics rightso just in my experience having thattype of specificity was advantageous inhelping me deduce the problems that Iwas facingyou know there's there's kind of liketwo levels that matter there's the likelike the high level of user impact like like how much startup time did this costme yeah and then you want the deeperokay it cost me this amount of startuptime whyand then like that's the like you needto dig in and see like oh yeah like itwas like it was spending all this timelike waiting on the disk maybe or maybeuse like busy doing other stuff maybethe parsing was really expensive maybewho knows what like you want to knowlikewhat was it actually doing that took this amount of time and like the orderis doing it's likethat like the like this the structure oflike resolving that graph and like theorder and all those things like doesactually matterum and like itit can very much have like an influenceonlike what the performance islike being able to express that isvaluableit's like like like personally like I Ilike I think we should ideally be ableto like actually capture like a full Trace fromthe like startupum like we'd be able to see like fourfour eacha separate import we should be able tosee like started this time ended thistimeand just like the likelike ownership relation between all ofthemumand as far as I've looked and sorry thisis just a complaint about esm as far asI've looked there has been there is noperformance advantage using esm over common.js because of the limitation ofyour hard driveright your hard drive can only readsequentiallyso there's no advantage in that rightand so esm is trying to group everythingtogether as if they're Network requestsbut thenyou're trying you're getting all thatfrom something that can only give it toyou one at a time anyway so it in theend it always comes out to be about thesame in terms of how long it takes toget your application up and running um I am I haven't done benchmarks on thelatest on with that patch so I don'tknowyep yeah they're like there couldtechnically be like a performance uhAdvantage like if you're using DynamicImports all the timebecause like it's gonna put it offthread so like your main thread can keepdoing stuff oh wait and workers tries todo it off the main thread tooor off its own thread sorrylike you could have like an application performance Advantage but like not manypeople are going to be like Dynamicimporting most of their stuff yeahlike it's mostly just gonna be like astartupthing that like you've used importstatements all of this is gonna likeresolve and load before any of my codeeven runs so it's like not really athing that you can control other thanjust likedon't depend on so many things okayall right so short of it is I guess wecould keep track of some metrics it likeat least the names of the modules beingloaded which is really for us it'sreally important to know the names ofthe modules being loaded and like thepath that they were read fromumand then timing is kind of pointlessreallyokay well timings okay so timing's outwith esm yeah exactly yeah yeahthat just occurred to me right nowumthatyou know with esm it'd just be like oneblock we were done processing everythingat this time right yeah we're done nowthat'sthat's still an interesting number yepwhatI don't even know how this will bepossiblebut if it were possible to register with with come and guess it was possibletoumokay sorry about the coffee attackesm is very upset that I'm insulting itfighting back[Music] sorry about thatwe're coming Jess it was possible tokeep track of how long each module tookto loadyou could create like a hierarchy and belike okay this module has started beingprocessedand then once another one was requiredyou just add it and remove right so youcould keep track relatively of how longeach module took[Music]hypothetically that'd be possible with esm but then we'd have to add in toolingto keep track ofhow long it took to process each scriptbut that would be a freaking nightmarebut even be possibleyeah I'm not really sure I'd have to diginto more of like how the like offthread loading thing actually works wellthis wouldn't be the after loading thiswould be after the fact when it handsbackbecausewell just what like just yeah yeahbasically when it executes the script itself and loads it into the isolate howlong does that take but that would beafter it hands it back rightyeah I I think so yeah so that's prettytrivial to do with common Jazzbut it would require a ridiculous amountof tooling I think inesm because we'd have to keep track ofall different things and when it's beingloaded in umyeah we can track when the file systemis called yeaheyeah and one over yeah and but that'lljust be general information yeahbut then all right whateveryeah maybe let's just start with tryingto get the file names the module namesand the paths but so it sounds like you kind of have asolution using what you say acornwell we we haven't like tried to buildit yet but basically hypotheticalwe we opened an issue basically sayinglike yeah node 20 like broke everythingapms Import in the middle just doesn'twork on node 20 and catch the way it'sdesigned currentlyumand so there's basically some ideas on likehere are the possible ways we can fixthisthere's like you can go and do thisAcorn thing which is obnoxious but umpretty much the only thing that you cando in like at all reasonable amount oftimewellbasically all we actually need is likethe names of the Imports or the namelike the names of the exports to likewe basically need like we like replace the module itself with the like wrappermodule that like we've just generated abunch of code that is basically like weknow there's an export of this name solike register a function that'sum like it provides like a Setterfunction basically that like ourinstrumentation can say like I want toreplace this function with this thingand like when we do that it'll triggerThis and like just do like set thisexport name to this thing we just passed into this function and it'll do that atthat that outer context level likereplace that variable binding so theonly thing we actually need is just likeknow the name of the export to do thatbutwe don't actually do that currentlythere's there's no likeAPI to look up what the exports of amodule are or anything like thatcurrently we'll go like if there waslike that's all we'd needyeahum yeah I think it was pretty last secondyeahit was like after last second[Laughter]there was like the the like I think thedeadline was like the Saturday orsomething and then like on on Sunday orMonday or something they're like oh butwe also want to get this in because likeevery major release everyone there'salways like a big pile of people thatare like oh but actually can you add this thing too in this thing of thisthingum yeahbut very very much uh props to Raphaelfor dealing with that like mountain ofstuff but yeahif we let things in at the last minutewe're going to run into these thingsyeah yeah so now it's like we have to wejust need you need to make sure there'sa solution before 20 goes LTSbecause yep yep yeah as it is right now like we likewe can't work on node 20 at all yeahlike CJs we worked fine like if you useesmOS doesn't worknono it's just like fully like rip outeverything and replace it with thisthing andlike just let's just push this thingthrough at the very last minute with likemaybe there was some review I don't knowif there was enoughthe pr itself wasI was constantly confused by it becausewas it just me or like there was so manycommits like intermingled and seemedvery messy and I was having a hard timekeeping track of what was happening inthat PR threadokayyeah I didn't really get a chance tolike look at it closely enough yep yeah it's like I I was aware it wasan effort that they wanted to work on atsome pointI never really knew what sort oftimeline there was on any of it yeahalways how close it was to being LandingI don't know I don't think soI don't know how early that was run ornot like that like that might haveit might be another way to catch things being broken earlier yeah yeah it'dprobably be a good one to get in therebecause like like most apms are startingto adopt that now at least the ones thatare trying to do esm there's not toomany of yetyeah that's uh yeah like datadog uses itsome other APM that's I'd never heard ofand I forget their name he uses itum and open Telemetry uses itjust aswe we didn't catch this in 20. thiswould help us catch similar well and honestly breaking open Telemetry is abig issue because that's like well to tobe fair like open Telemetry is esmsupport like isn't really even releasedyetyeahthey were like in the in the process oflike preparing it and like had juststarted integrating it and then node 20comes out and like actually nope it'sbrokenoh man yeah I was surprised when I saw that PRland because in in my opinion every timeI checked out routinely check on itbecause it seemed like a really bigimportant oneand then one day I was just in I waslike wait what I didn't I didn'tremember seeing the rigorous process oflanding something of that sizebut I that's all in the past now I guesswe're just stuck trying to deal with theresultokaythat I'm I'm mostly just surprised that like that wasn't put behind a flag orsomething like like have like anexperimental flag for like let's try andsee if this works for like a major orsomething like nope that's just likefully rip out the whole the guts of allthis and replace it with new guts inthis major it's fineyeah yeah right before right beforegoing to an even release two yeahyeah if they had done this before 19 or21 it's like sweet that's why thesebranches exist but they know 20 is goingto be LTS yeahhuhokay so short of it is there is no goodplan on how to fix this issuepretty much oh okay yeah there'sthe acorn thing which we think will workbut basically like every module we seewill have we'll have to have like Acornparset and find the exports in it we'llhave to write a bunch of stuff touse Acorn however you use Acorn yeahfreaking and is there any chance we can get something landed in core to assistwith this I mean likelike core already has acorn in itum so we we could just like have core dothis and make an API for itum I think that's acceptable honestlyyeah like that that was that was kind ofmy suggestionumthere are people that think that theexistence of acorn in core is not athing that um we should be relying on or likebut it does and it does exist it doescurrently but in theory if like the onething that depends on it currentlychanges like it could be removed in thefuture maybewhat if we just create an abstracted APIthat because some aspects of acorn willneed to exist whether we use Acorndirectly or not the idea of it needs toexist yeahwellyep yeah it's like my my suggestion was yeah like just likewhat we need is an API to get like whatare the exports of this module yeah andit's likemy suggestion was like just make an APIthat that's all it does yeahbut like even even then likelike that would depend on Acorn it likevery very Loosely ties us to having likeparsing inlike in node core somewhereum and likebasically like because Acorn existsoutside of node core a bunch of peoplewere like well why can't you just use itoutside of node core thenno you don't get it I guessuhsowell yeah yeah we'll probably be addinglike Acorn as a dependency of input inthe middle and then doing a bunch ofparsing there to do that so theum wait I just thought of this V8 doesn't have an API to allow you to viewthe exports of a fileI feel like that I feel like V8 shouldallow you to be likeshow me the like it has to parse themalready right yeahit's already doing the work I think itdoes add like C plus plus level yeahum I would have to look closer at itokaybecause yeah if V8 is it has to beparsing out the exports or the Imports and soI'm curious what the effort would be tojust be likewhenever you parse one send me a stringjust let me know what it isright because that that string has to begenerated somewhere at some time we knowit yeah somaybe that would shut them up and belike lookone unique V8 API that lets us know thestring of the important okayyeah II think what we get from V8 like if there'slike the module object which is likekind of a black box but it has like theget namespace on it and that just givesyou like an objectit's like we can explore the objectslike I think that's what we'd have to dooh I need to research this more becausenow I'm just having more questions allrightso grab your situationdiagnostic Channel using that isn't going to be a solution I guessyeah yeah so like my My Hope like longerterm was to like yeah have haveDiagnostics channel for like all theloading life cycle things so likeeventually we can just not even care ifthings are on esmum but we're not there yet yeahwhich is kind of the problemlike if we were there already then CSMnonsense wouldn't matter yepyeah all rightwell poopheyyep yeah like we'llprobably need to like continue havingthat for like even if we like addlikelike loader events with DiagnosticsChannelum it will probably still needto like have that in imported in the middle like over for likeback like version backwardscompatibility certainly but also likenot sure yet if like like I I want toadd like loader like loader hooksum or like loader eventsumwith Diagnostics Channel but there'sbeen a bit of pushback onum welllike I I also relatively agree that likediagnostic channels kind of intended tobe just like passive Observer like you're not meant to mutate thingsbut like we actually need to be able tooh you need to be able okay we need weneed to patch things for likeinstrumentation purposes okayum which like isn't like when I builddiagnostic Channel like that was not theintense we like we can make it work thatwayum a little bit of a hack but it'spossible uh but like there'ssome pushback to that idea of like Expolike it like look like esm folks don't wantus exposing like a mutable copy of themodule basicallybecause we're not the freaking browserget over it yeah sorrystrong opinions about thatyeah okay sothey're trying toyou knowthey don'tit's like they don't seethe necessity of it in terms of like how do I put itthey're hand waving it away saying weknow people need apms we know peopleneed these metricsbut we don't want you touching our stuffit's like well that's not the way itworks butmyinterpretation of what I've seen is likethey they're very much viewingbasically like parsing an AST transformas like the one true way to manipulatecode holy Hannahuhwhatlike basically everyone I've talked toon esm is just like just Parson likemuted ASTlike no thanks but yeahlikeso many reasons nowall rightwell having having seen another APM tryto do theAST person thing before noyeah there has been a history of apmsattempting to do ASD parsing it was notsuccessfulthat's insane that they're actuallyexpecting thatprogramwellping me on any issues if you needsomebody to back you up because umwe don't have dissimilar issues on ourside eitheryeah so now go ahead and ping meI don'tdoesn't mind me to ruffle feathers ortell them that they're wrong so be happyto helpyeah Imy my experience so far has not has beenlike welltelling them they're wrong doesn'tseem to work too well well see having me around will make you seem like the morereasonable person to talk toso then they'll want to communicate withyou because it really would me just bebe likedo you not see the obvious reality ofthe situation people rely on metrics tokeep their processes running and nowyou're not allowing thatyou you broke everyoneyou clearly are showing that you don'tcare about the project otherwise youwould help people do what they need to do you instead are more you know sorry Iwill not rant here I will rant an issueokayokay well yeah if you have if youlet me know if you have if you have anidea what you could do in coreum we have yet to move our stuff toversion 20 yet we usually wait till it'sclose to LTS butshould we try and do this do it earlierwith this okay all right then we'll we'll get on moving all of our code overto version 20. to see how it affects usand communicate with you andI'll let you know what we do becauselike we also have we don't mutate anyobjects but we do have to monitor awhole bunch of stuff at the module leveland I'll let you know what we find whenwe're migrating our codemanyep yeah thelike moving stuff off thread things likeit definitely seems like a very substantial change that should have gonethrough like a bit more communicationwith the rest of the project I thinkyeah oh yesand outside vendors yeahI think that was a very large changeveryumokayI feel like if I keep talking about thisI'm just gonna actually start ranting soI think I should stopfeel feels like a conversation that needs to happen with beers in handyeah totallyis that was that the main point of thismeeting was to talk about thatum as yeah that that breaking the worldand yeah the asynchical storage okayit's the ideaumif there's any other Diagnostics thingspeople want to I really think we should just Chuck inthe change to async Local storagewithout testing it right before anothermajor version see how that goes yep yeahthat sounds like a great planno that'd be great because I mean thatshouldyeah I can see where that would removethe overheadand that'd be awesome becauseI don't know if we could turn that inokay API only tracked the promises that I didmake a task of you right or somethinglike that how is he going to try the anyotherasynchronous source that is not theproblem yeah so it it flows throughpromises automatically but like the theway it works internally is just like atsome point you you get the value stored off band so much like in this case itgets stored on the like the promise taskand then when the prompts goes to resumethen it sets the current thing to thatagain so we just need to do the samewith async graph is like when theinternet happensinvolvedno we manually set it rightyeah yeah so like any sort of async taskthat would happen you grab like what isthe current state the V8 is holdingthen you hold that out of band somewhererelated to your task and then when your continuation would happen thenyou just restore that as the currentvalue it honestly sounds very similar towhat async rap is doing in terms ofkeeping track of the different IDs andmanual setting themand hey at least we have already knowall the code points where it needs to beput since async rap already has them allthere yeah yeah okay yeah yeah thethe the way that it yeah yeah like Isaid before I forget which way around itwas but like there'sumI I believe likeyeah the like like this new change wouldtrack the resolvenot the creation of the promise that'swhat we do right now or yeah like Correctional continuationumwhichlike that that means thatum like you you can doum like deferable patternum where like you can like pull theresolve and reject out and like storethat in an object and like hand thepromise object somewhere else and thenthe hand of these like resolver thingssomewhere else and that will like linkback to where it actually resolved fromokay um so that that way makes more sense andthen likelike essentially like it let like itlets youlike track back from there and then inthe case where you want like the otherway around like you can use likelike there's like ancient resource bindthing was used for likelike that that was like that thatbasically existed for like the like userland queuing or like any any sort oflike like that like thiscontinuation is likeokay yeah like the continuation likeit's running in a different place thanlikewhere like the actual likething that's tragic isum like it's similar sort of thing hereokay that likeyeah like the actual like continuationis attached like over here and thetrigger is happening over here so samesort of thing we can justlike use that to bind the two togetherif we need to oh well ohthat was my understanding fromI'm just saying about it anyway coolwell if it works that well I feel likewe could replacethe AC async rap mechanic completelyyeah possiblyyeah well I guess we'll have to see oncewe start using it yeahwe'll still need like Ace and Cooksaround for like other like things other than contactmanagement sorest of using crap we'll need tocontinue to exist for that butumyeah that's trueyeah I can probably like simplify abunch of things though and likeI I think like part of the problem withum like well it isn't cooked like thatbeing the core underneath like I thinklocal storageum and like async local storage kind ofbeing like well like it's it's already consideredstable and like it's a thing like a lotof stuff depends on has kind of made itdifficult to like make any changes daysand Cooksohit's like if there's any likelike refactor we want to do to it forlike performance reasons or things likethat if like any of the events are likeout of order at all or like like anytiming changes whatsoever doeseverything explodes oh that'sthat's a problem because when I wroteasync hooks the whole point was that you wouldn't be dependent on that so if ifyes that makes that makes perfect sensebecause async hooks no API was evermeant to be dependent on ordering andtiming with async hooks so yeahunfortunately a lot were including howlike async local started with it wellthat'd be great if they could migrate itoff to something faster and more stableyeah that'd be awesomeum well no a singular Cold Storage itself was not implemented is theall the other stuff that like that was acontinuation ofthe was it uh content continuation localstorage thing is calledand like all that whole family of thingslike the things trying to do like theproviding the context management API ontop of that yeah like it it had the likeit always did the like cleanup lifecycle as the likeit deletes this when the story happens um which dude like there's a bunch ofcases where like it was ambiguous likewhenthe destruction having like promiseslike happens someday maybe if thepromise is ever garbage collectedpotentiallywhich is not a great way to do thingsum yeah the there's like certain caseswhereumlike venablesumit would like produce and and knit andthen another in it and then the before and after oh really it actually do liketwo minutes because like there's there'slikethere's an init from like the Venablebeing converted to a promise and thenthe actual use of that promisethat's freaking hilarious okay I didn'trealize that happenedum and there wasumI'm trying to remember what the case wasthere's like a case where like it likenormal case was like with the promise trainingum you'd get like if you do like a likethen and then like inside of that likepromise resolve or something like thatlike goes down to the next thing in thechain it would do like the before andafter for that thenand then like the result would bebetween thereum but there was like certain caseswhere the result would actually happenafter the afterOh weirdyeah okay I think it's like promise dot like not promised at all but likepromise that right race and like theotheroh different variations but likeit would actually like create thepromise that all this stuff wasum being likeI don't know whatever handled in thebackgroundafter it's done doing the thing it wasweird okayinterestingwell cool I'm looking for it does thatAPI exist currently in V8 uh yep yeah thethe embedder continuing it's like inbetter continuation context data orsomething I thinkum yeah it exists in in V8 it's actuallybeen there for a while now oh okayum like maybe a year and a half twoyearsyeah yeah there's yeah there's aprototype module oh yeah okay yeahcoolawesome let's just figure something outbefore 20 goes LTS otherwise we're allscrewed yeaheverything's on fire forever yeah rightwhen does 20 oil TS three monthsyeah you're right October yepokay so may yeah we shouldyeah we areall right I'll be in contact with youabout that to tell you what we learnedwhen we migrate our codeokayuh fingers crossed it's not like evenmore on fire than I think it isI'll let you know it just it's an evenbigger change since Libby V has changedthe way they read files from the filesystem on Linux and every other likethey use basically they just use pole on every kernelon every platform except recently now onLinux they now use u-ring which iscompletely differentsoit's nice because it's faster but at thesame time it's more abstract so it'sharder to see into what's happeningyou get a populationyeah I guess that's true yepyeah I mean oh yeah it's a lot fasteryeahyepyeah so good news there onLinux it can read small files eighttimes fastersobutmaybe it'll just be so fast that wedon't need to care about you get someload time anymore yeah rightso fast people will just add like ahundred thousand more dependencies thatwill be what happens yeahandyeah it keeps it keeps us employedis someone gonna add the like everythingmodule back to npmoh yeah I forgot that was a thing wasn'titpretty sure it got banned and that's understandableI mean the size of that package JsonanywayI I think it's still on GitHubis it reallyI kind of want to install itnoand we can use it as a GitHub dependencyand still take down npm anyway oh that would be great, Node.js Diagnostics WG Session, OpenJS Foundation, 01h3j60md1yb56gpnhndpgx1qg.