Discovering Linux Kernel Subsystems in Use ![rw-book-cover](https://img.youtube.com/vi/q3ZsHQ-wvb0/maxresdefault.jpg) ## Metadata - Author: [[YouTube]] - Full Title: Discovering Linux Kernel Subsystems in Use - Category: #articles - Document Note: --- tags: [linux-kernel, embedded-systems, workload-analysis] author: Shuah Khan source: YouTube reference: https://www.youtube.com/watch?v=q3ZsHQ-wvb0 date created: 2024-12-19 14:30 --- # 🏷️ Title: Discovering Linux Kernel Subsystems in Use ## 🧰 Action Inbox - [H1] Implement system information gathering at three points: before, during, and after workload execution — establish baseline metrics #testing - [H2] Set up event tracing for workload analysis using system-wide tracing capabilities — understand subsystem usage patterns #tracing - [H3] Use strace for fine-grained command analysis to identify specific system calls and resource usage — supplement high-level analysis #debugging ## 🧭 Table of Contents - [📝 Summary](#-summary) - [📖 Main Content](#-main-content) - [🗒️ Notes](#️-notes) - [📚 References](#-references) - [🗣️ Paraphrase](#️-paraphrase) - [📰 Newsletter](#-newsletter) - [✅ Action Items](#-action-items) - [🧪 Hypotheses](#-hypotheses) - [📊 Methodology](#-methodology) - [❓ Questions](#-questions) - [🔮 Future Research](#-future-research) - [💡 Implications](#-implications) - [🧠 Learning Styles Assessment](#-learning-styles-assessment) - [📊 Accessibility Assessment](#-accessibility-assessment) - [🌍 DEIB Assessment](#-deib-assessment) - [✅ Logical Validity](#-logical-validity) - [🛡️ DARVO Analysis](#-darvo-analysis) - [🎯 Conclusion](#-conclusion) ### 🔗 References - [Linux Kernel Documentation](https://kernel.org/doc/) - [OpenAPS Documentation](https://openaps.readthedocs.io/) ## 📝 Summary - Shuah Khan presents methods for discovering which Linux kernel subsystems are actively used by workloads - The approach combines system information gathering, event tracing, and fine-grained command analysis - Primary use case demonstrated through OpenAPS (Open Artificial Pancreas System) running on Raspberry Pi - Enables proper system sizing, kernel configuration, tuning, and regression testing for embedded and safety-critical applications - Bottom line: Understanding workload-kernel interactions is essential for automotive, medical, and other safety-critical embedded systems ## 📖 Main Content The presentation outlines a systematic approach to understanding how applications interact with the Linux kernel. This is particularly important for embedded systems where resource constraints and safety requirements demand precise understanding of system behavior. The methodology involves three phases of data collection: establishing baselines before workload execution, monitoring during execution, and validating cleanup after completion. This approach reveals both resource usage patterns and potential issues with applications that fail to properly clean up after themselves. The speaker demonstrates practical applications using OpenAPS, an open-source artificial pancreas system that manages insulin delivery for diabetic patients. This real-world example illustrates how the methodology applies to safety-critical embedded systems where understanding kernel interactions is crucial for regulatory compliance and system reliability. ## 🗒️ Notes #### ⏱️ [00:00](https://www.youtube.com/watch?v=q3ZsHQ-wvb0?t=0) — [H1] Introduction and Problem Definition - Key idea: Understanding workload requirements is critical for embedded system design - Evidence: "sizing and memory and storage for a workload...especially when we are talking about Automotive talking about medical" #### ⏱️ [02:15](https://www.youtube.com/watch?v=q3ZsHQ-wvb0?t=135) — [H2] Why Understanding Workloads Matters - Key idea: Four key reasons drive workload analysis in embedded systems - Evidence: "sizing...configure and tune the kernel...configure and tune the system...identify tests that we care about to avoid regressions" #### ⏱️ [03:45](https://www.youtube.com/watch?v=q3ZsHQ-wvb0?t=225) — [H1] Three-Phase Data Collection Strategy - Key idea: Information gathering occurs at three critical points in workload lifecycle - Evidence: "before starting the workload while the workload is running and after stopping the workload" #### ⏱️ [04:30](https://www.youtube.com/watch?v=q3ZsHQ-wvb0?t=270) — [H2] System Information Categories - Key idea: Five categories of system information provide comprehensive workload understanding - Evidence: "supported system calls...features...system parameters...scheduler...modules kernel modules" #### ⏱️ [06:00](https://www.youtube.com/watch?v=q3ZsHQ-wvb0?t=360) — [H3] Tools for System Call Analysis - Key idea: Multiple tools enable comprehensive system call discovery and analysis - Evidence: "Ausyscall which is part of rdd tool set...checksyscalls script is part of the kernel" #### ⏱️ [08:15](https://www.youtube.com/watch?v=q3ZsHQ-wvb0?t=495) — [H2] Workload Behavior Assessment - Key idea: Post-execution analysis reveals workload quality and cleanup behavior - Evidence: "good workload good well-behaving workload would probably set the system back to another state" #### ⏱️ [10:30](https://www.youtube.com/watch?v=q3ZsHQ-wvb0?t=630) — [H3] Practical Tool Examples - Key idea: Command-line tools provide specific insights into system capabilities and usage - Evidence: "you can list the features you can also check...stack protector...memory leaks" #### ⏱️ [13:45](https://www.youtube.com/watch?v=q3ZsHQ-wvb0?t=825) — [H2] Change Detection Through Differential Analysis - Key idea: Comparing system states reveals workload footprint and subsystem usage - Evidence: "we find differences so we can figure out what exactly is the workload footprint" #### ⏱️ [15:20](https://www.youtube.com/watch?v=q3ZsHQ-wvb0?t=920) — [H1] Event Tracing for Workload Analysis - Key idea: System-wide event tracing captures comprehensive workload behavior beyond traditional debugging - Evidence: "tracing is not for just debugging...using tracing in this context for a very different purpose" #### ⏱️ [17:30](https://www.youtube.com/watch?v=q3ZsHQ-wvb0?t=1050) — [H2] Mapping Events to Kernel Subsystems - Key idea: System calls and functions in traces map directly to specific kernel subsystems - Evidence: "if you see get timer calls...then we know that the timer subsystem is being used" #### ⏱️ [19:45](https://www.youtube.com/watch?v=q3ZsHQ-wvb0?t=1185) — [H3] OpenAPS Case Study Results - Key idea: Real-world analysis of OpenAPS reveals specific module and subsystem usage patterns - Evidence: "we have seen a couple...three modules it does load and then unload" #### ⏱️ [21:15](https://www.youtube.com/watch?v=q3ZsHQ-wvb0?t=1275) — [H2] Fine-Grained Analysis with strace - Key idea: strace enables detailed analysis of individual command execution and failure cases - Evidence: "we want to understand run as stress run various commands...what kind of system calls this particular command is using" #### ⏱️ [24:30](https://www.youtube.com/watch?v=q3ZsHQ-wvb0?t=1470) — [H3] Command-Level Analysis Examples - Key idea: Individual OpenAPS commands reveal specific system interactions and failure modes - Evidence: "get insulin pump model...battery levels...pump history insulin temporary basal rates" #### ⏱️ [27:00](https://www.youtube.com/watch?v=q3ZsHQ-wvb0?t=1620) — [H3] Practical Failure Analysis - Key idea: strace output reveals specific failure modes and device interaction problems - Evidence: "it cannot connect to...the pump...trying to open Dev SPI Dev and it cannot open that" #### ⏱️ [30:15](https://www.youtube.com/watch?v=q3ZsHQ-wvb0?t=1815) — [H2] Runtime Context and Hardware Integration - Key idea: Complete system view shows hardware-software interactions through kernel subsystems - Evidence: "SPI Dev driver...gpio...continuous glucose monitor and night Scout" ## 📚 References - [Linux Kernel Event Tracing Documentation](https://www.kernel.org/doc/html/latest/trace/events.html) - [OpenAPS Documentation](https://openaps.readthedocs.io/) - [strace Manual](https://man7.org/linux/man-pages/man1/strace.1.html) - [Raspberry Pi GPIO Documentation](https://www.raspberrypi.org/documentation/usage/gpio/) ## 🗣️ Paraphrase - Understanding which kernel subsystems your workload uses is essential for proper system sizing, configuration, and regression testing in embedded environments - A three-phase approach (before, during, after workload execution) combined with differential analysis reveals workload behavior and resource usage patterns - Event tracing and strace provide complementary views: high-level system behavior and detailed command-level interactions respectively ## 📰 Newsletter **Kernel Subsystem Discovery for Embedded Workloads** Understanding how your applications interact with the Linux kernel is crucial for embedded systems, especially in safety-critical domains like automotive and medical devices. Shuah Khan's methodology provides a systematic approach to workload analysis that goes far beyond traditional debugging. The three-phase data collection strategy—gathering system information before, during, and after workload execution—reveals not just what resources your application uses, but also whether it behaves properly by cleaning up after itself. Combined with event tracing and fine-grained analysis tools like strace, this approach enables precise system sizing, optimal kernel configuration, and targeted regression testing. The OpenAPS case study demonstrates real-world application, showing how this methodology identifies critical subsystems like SPI drivers and GPIO interfaces that must be monitored for regressions in medical device deployments. Resources: [Kernel Tracing Guide](https://www.kernel.org/doc/html/latest/trace/), [Embedded Linux Wiki](https://elinux.org/), [strace Examples](https://man7.org/linux/man-pages/man1/strace.1.html) ## ✅ Action Items #### DevRel - Developer advocacy: Present workload analysis methodology at embedded systems conferences - Education: Create workshop curriculum for kernel subsystem discovery techniques - Developer experience: Document integration patterns for tracing tools in CI/CD pipelines - Developer marketing: Develop case studies showcasing methodology in different industries - Community management: Facilitate discussions on workload analysis best practices #### OSPO - Policy & Governance: Establish guidelines for kernel subsystem monitoring in safety-critical applications - License Compliance: Review tracing tool licensing for embedded deployment scenarios - Security Management: Assess security implications of system-wide tracing in production environments - Software Inventory: Track kernel subsystem dependencies revealed through workload analysis - Developer Support: Provide training on embedded Linux workload analysis techniques - Community Engagement: Support upstream contributions of workload analysis improvements - Strategic Alignment: Align embedded Linux initiatives with organizational safety and compliance goals #### Community - Maintainers: Support kernel documentation improvements for workload analysis - Contributors: Encourage contributions to tracing infrastructure and analysis tools - Sponsors: Fund development of improved workload analysis capabilities - Ecosystem: Coordinate cross-project efforts for embedded system analysis standards #### WordPress - Core: Apply workload analysis principles to WordPress performance optimization - Core Performance: Use tracing techniques to identify performance bottlenecks - Community: Share embedded WordPress deployment best practices - Documentation: Document WordPress resource usage patterns for embedded deployments ## 🧪 Hypotheses - Systematic workload analysis reduces embedded system resource over-provisioning by 20-30% - Three-phase data collection identifies 90% of critical kernel subsystems for regression testing - Fine-grained command analysis reveals failure modes not apparent in high-level system monitoring - Differential analysis between workload phases accurately identifies misbehaving applications ### 📊 Methodology - Controlled experiments comparing resource usage before/after workload analysis implementation - Regression testing effectiveness measurement across identified vs. non-identified subsystems - Comparative analysis of strace vs. system-wide tracing for different types of applications ### ❓ Questions - How does this methodology scale to multi-container or microservice architectures? - What additional tracing mechanisms would enhance automotive and medical device analysis? - How can this approach be automated for continuous integration pipelines? ### 🔮 Future Research - Integration with container runtime analysis for cloud-native embedded systems - Machine learning approaches to automatically classify subsystem criticality - Real-time workload analysis for adaptive resource management ### 💡 Implications - Practice: Embedded developers need systematic approaches to understand kernel interactions - Policy: Safety-critical systems require documented subsystem dependency analysis - Security: System-wide tracing raises privacy and security considerations in production - Education: Engineering curricula should include workload analysis methodologies - Funding: Investment needed in automated workload analysis tooling and infrastructure ### 🧠 Learning Styles Assessment Reading level estimate: 12th grade (technical presentation with domain-specific terminology) #### 👀 Visual learners - Present: System architecture diagrams, flowcharts, command output examples - Improve: Interactive subsystem dependency visualizations, timeline charts #### 🔊 Aural learners - Present: Conference presentation format with speaker explanations - Improve: Narrated command demonstrations, audio walkthrough guides #### 🗣️ Verbal learners - Present: Technical terminology definitions, analogies to software engineering processes - Improve: Glossary of embedded systems terms, comparison tables #### 🧩 Kinesthetic learners - Present: Hands-on OpenAPS analysis, command-line tool demonstrations - Improve: Lab exercises, interactive tracing workshops #### 🧠 Logical learners - Present: Three-phase methodology structure, systematic analysis approach - Improve: Decision trees for tool selection, cause-effect diagrams #### 🧑‍🤝‍🧑 Social learners - Present: Conference Q&A session, community discussion opportunities - Improve: Collaborative analysis projects, peer review processes #### 🧘‍♀️ Solitary learners - Present: Technical documentation references, individual analysis exercises - Improve: Self-assessment checklists, independent study guides #### 🌿 Naturalistic learners - Present: Real-world medical device example, practical applications - Improve: Environmental impact considerations, sustainability connections ### 📊 Accessibility Assessment #### Screen reader semantics - Present: Clear heading structure, technical content organization - Improve: Alt text for diagrams, ARIA labels for data presentations #### Keyboard navigation - Present: Standard video controls, text-based content - Improve: Skip links for technical sections, keyboard shortcuts documentation #### Contrast and readability - Present: High contrast slides, clear font choices - Improve: Ensure code examples meet contrast requirements #### Text resizing and reflow - Present: Scalable presentation format - Improve: Responsive documentation layout for technical content #### Media and captions - Present: Video presentation available - Improve: Detailed captions including technical terms, transcript with timestamps #### Forms and interactions - Present: Basic video interface - Improve: Interactive code examples with accessible controls ### 🌍 DEIB Assessment #### Cultural breadth - Present: Technical content applicable globally - Improve: Examples from different geographical embedded system deployments #### Inclusive language - Present: Professional technical terminology - Improve: Define acronyms, explain domain-specific concepts #### Neurodiversity support - Present: Structured presentation format, clear section breaks - Improve: Multiple content formats, processing time accommodations #### Representation - Present: Technical focus on methodology - Improve: Diverse use case examples, varied speaker representation #### Psychological safety - Present: Professional conference environment - Improve: Beginner-friendly entry points, question encouragement ### ✅ Logical Validity #### Strong reasoning - Deductive strength: Strong - methodology follows logically from embedded system requirements - Inductive strength: Good - OpenAPS case study supports general applicability - Causal reasoning: Clear - workload analysis directly enables better system design - Analogical strength: Effective - medical device example illustrates broader principles #### Fallacies - Appeal to authority: - Document Tags: [[testing]] [[tracing]] [[debugging]] - Summary: Discovering Linux Kernel Subsystems in Use - URL: https://www.youtube.com/watch?v=q3ZsHQ-wvb0 okay thank you everybody um my name is shua Khan um I am going to be talking about discounting Linux kernel Club systems in use um I think Kate mentioned earlier when she was doing the welcome I'm talking about um how we are doing this for open uh in open ABS case uh let's start like a talk about why is it important um understanding workload requirements why do we care the reason we care about this is um multiple really sizing and memory and storage for a workload we want to be able to especially when we are talking about Automotive talking about medical we want to be able to size the system um memory storage and all of those and then also configure and tune the kernel and that's important as well and configure and tune the system those two are two different things during the kernel enabling the right configuration options in the kernel that work for us say for example stack protection we want to be able to do that if that is important for the workload and the other things is configuring and tuning the system say our workload requires large number of files we want to make sure and processes maybe it kicks off several threads so we want to make sure that we have our system is configured when we have maximum load or when we have um when the system is in heavy use so we want to be able to tune that and then the lastly this is really important aspect is that we want to be able to identify tests that we care about to avoid regressions as the new releases come out kernel releases and then also be able to write tests if we think that the existing tests need to be either enhanced new test to be written so for two different things avoiding regressions as we keep taking new kernel bits um and and the platform too and then evaluating safety so then uh so first of all we need to understand what our workload uh requirements are and resource usages so we do that system information gathering at three different points one is before starting the workload while the workload is running and after stopping the workload so those are the three different points we take uh pulse of the system what do We Gather um these are the things that we want to gather supported system calls on the system um we we have a few tools that we can use to do that Asus call which is part of rdd tool set that can dump all of the system calls for a particular architecture and then also uh this the second script checks this calls script is part of the kernel that it can it can tell us the system called supported system calls on a architecture then the LA we also want to check the features supported features so we can do that using a script that is part of the kernel sources which is get feature get featured dot PL is get feed.vm and then the other thing is uh system can tell us information about system parameters the limits and so on and scheduler what kind of schedule reasoning priorities Etc and then lastly we'd also want to understand what modules kernel modules are loaded loaded in the system so we use LS mod for that and there are other uh obviously other things LS USB tells you USB ports available and then lspci tells you other things but for now we'll talk about these things and while the system so we talked about what we want to get before we start our workload uh that tells us Osmos like a baseline what is running currently and then we want to gather we start our workload and then start Gathering um system parameters and then loaded module currently loaded modules because some workloads the one that we played with extensively open APS it does go and load and unload modules so we want to see the differences in what is the workload um how how the workload is changing things is it loading new modules unloading some and so on and then also how it's changing the system parameters limits scheduler priority Etc and we let the workload run um for however long we want to run it we'll get into a little bit into how how we want to trace the system but um but right now we are talking about the when do we gather system information so after we stop the workload we probably run it for 30 minutes two hours it doesn't matter then we after stopping it we go and look at system parameters and load it uh currently loaded modules so this tells us a couple of things one is is the workload behaving um correctly so a good workload good well-behaving workload would probably set the system back to another state um is it doing that so and then also we find out what's happening is it loading modules and tons of modules and leaving them loaded or is it upping the system limits and leaving those um left I mean not cleaning up after itself Okay so with that I will walk you through a couple of what our output of these things looks like and how you can use I mentioned um Aus is called you can actually ask it to say tell me all of the timer related system calls that are supported so it can tell you um when I ran this on my system it tells you these are all things and then the number associated with this system call number system call numbers can change um from architecture to architecture so it is it is this information and the number associated with it is important to us and check system calls dot sh what that does is it tells us um the ones that are not implemented like you can see on my system when I ran it on my test system these are all the ones that are not supported or not implemented and it's it's a longer list and then now we switch into the other tool I mentioned the feature list tool so you can list the features you can also check and see this iron it on x86 it shows you what uh features are supported some of the important ones um like for example um BPF related ones there that is showing and then stack protector I am looking for specifically I'm saying here you can see I am looking for um get up I'm I'm looking at a feature list I'm looking for a stack protector what kind of Stack options are um does it have enabled and it shows me have stack projector on and and then what else is available and then memory leaks so you can get up for specific specific features like dma and so on so you can do um get up for specific things so and then LS mod if you look at the LSR tool that I mentioned before you can look for specific modules or have if you don't have that graph in there you can it'll dump every single module that's currently loaded and in addition to that it'll show you what does the what's the usage count of those like for example video back to Common has four users four other modules have um are holding reference to it so until all of those modules are unloaded you cannot unload video buff too common for example okay so what we do is uh we have gathered different uh information we gathered um all of the things that we talked about like for example the modules [Music] um modules configuration option especially the modules and system parameters so we look for changes in between two points before loading the workload before starting the workload and once we start we are continuously saving this information we find differences so we can figure out what exactly is the workload footprint um lack of better word let's let's use a different word what are the resources the workload uses and we can narrow the subsystems that way and say these are the subsystems we want to care about more when we are running regression tests then when the new releases come out are these we're watching those it's like our watch points okay so I will continue tracing um okay tracing is not for just debugging tracing is traditionally used for debugging and we are using tracing in this context for a very different purpose um when when I say tracing workload I'm talking about the even tracing feature available in the Linux kernel you can enable a system white tracing and say give me all of the events that you can possibly Trace generate trace for it and before starting the workload we enable system-wide tracing and we can disable the system by tracing after stopping the workload so we'll get a snap shot of all of the events that have happened after um while this workload is running so you will see the reference here you will see this is one that's um subsystem used by we have done a couple of uh blogs about this and then you can take a look at those for D in details on how uh we did the um gather traces so once you have that now we know that we we know the system calls um system calls that are supported on our system when we know the system called numbers associated with them and then in the traces we see the system calls um that are uh system calls that are used by the workload then we can go and look at identify functions and system calls usages and map them to Kernel subsystems like for example if you see get timer calls for example uh being called then we know that the timer subsystem is being used um that's one of the subsystems all workloads used that's um not a great example but if you see calls to setting priority for example or any kind of scheduler changes it might be doing priority levels on different processes and threat priorities then you will know some of the useful information that's coming up so far um this is all high level information we do not know specifically we do not know what the low level information is meaning fine grain information for example um we have done um analysis of overall open APS load we ran it we enable the system right tracing in its uh open APS in it in the crown Tab and then we went in and it has the init script so we enabled creating system-wide tracing in init script and then we let it run for a couple of hours and we stopped uh the workload and then we took this snapshot up this entire two hours worth of log and we analyzed system calls and modules it's loading and we have seen a couple a couple three modules it does load and then unload we have determined what exactly does it use and we have done these experiments on a Raspberry Pi running open APS workload however what we do not have is we wanted to also understand what happens say for example um a when individual open APS comments or run for example open APS might go and say user says tell me give me pump history what has happened with this insulin pipe pump in the last um last hour or two or whatever and then also how much insulin is left in that pump those are the ones it will be lost if we have just this large gain system view um we do not we won't be able to isolate when for uh the tracing or resource usage it happens just for those individual command runs and then this um and when errors happen also so for this we used a different tool asterisk estrace what estrace does is again this is also heavily used for debugging so far but we wanted to use this for a use this for a different purpose uh in this cases we want to understand run as stress run various commands with estrace and say hey what are we um seeing what kind of system calls this particular command is using and what kind of system um resources it's using the files it's opening for example and then also pay attention to failure cases as well so this a Blog um the link I put in here talks about more in detail um about all of that okay let's see um Esther's command you can trace system calls and signals and you can run it in multiple different modes uh one is the full mode where you can get um the full complete information um and you can run it in summary mode where it'll identify summarize the system calls and see invocations of those what we have done for this is we looked at open APS commands and we um pick the ones that would make sense to us get insulin pump model for example that would be something that uh um that would go out and then open APS will query the pump and then at the time and then battery levels and a few commands that we want we're interested in and more of that and you can look at this pump history insulin temporary temporary basal rates and then insulin this is how we wanted to kind of find out what happens when we run these commands what uh system um activity is associated with these command runs and this is a list of uh various commands in here and let's go to where we have um information about this I have some slides that show pump information so like I mentioned S3s Dash c will give us a summary of the summary of the actively and then the full if you remove this Dash C option it gives you a full Trace when we ran pump history command what we have seen is that um the output on this left side you can see Run pump history command so it prints all these this is the output from that command retrieving pump basically since when we ran this command um and it's telling us that it cannot connect to um a the pump this is the it's trying to open Dev SPI Dev and it cannot open that so and it's also saying pump is busy so this device is in use so this is a one of the busy paths we traced um and then what corresponds to it this is the right side what I'm showing is the analysis we did process startup course that has to happen management and then it goes and opens um systems file this associated with this device then it opens the pump device and then this these are the two it's trying to read um say these uh gpio active low and the direction these two are associated with this device and then it'll try to open them that's when it detected that it cannot open it and it's busy then it goes and closes the files and exits the program so you that's how we mapped um the subsystems with individual command runs so in addition to that this is a flowchart we developed a flowchart to it um that what happens so we have um it's uh anybody that's familiar with software engineering I mean this is a general software flow a flowchart that shows uh the command flow or a process flow it is showing here and that we try to open it we couldn't open it detect detecting device with the condition and print status from close files so that's what one of them is so and if you were to um if you were to look at there are two aspects to it we took the pros the command out output that we have gotten and we mapped it and then you can see that from this estrace output you will see everything else that has happened in that system during that time um it's you this corresponds to this output corresponds you can see the device in use right here so it corresponds to that and then it shows us um how many a percentage of time it's spent and how long each one of those system calls took so and it'll give us um activity on what has happened and you you probably saw mem map too in the previous slide and then we can corresponding on map and so you can get a feel for what is all happening while this is running any questions at this time before I switch into with go to the next slide okay just just go ahead and type here um just ask questions or type them in the chat um so this is a runtime context for what we have done Raspberry Pi system we ran it on Raspberry Pi and then during that time we have identified common um kernel functionality processes these are all um the subsystems that were invoked SBI driver is the one you noticed that uh when SBI Dev is being opened that's where SBI driver SBI Dev driver comes in and then you have to surface the driver files uh driver interfaces that are available that it exports and then you have a gpio and um over the gpio we are talking to insulin pump and then outside right outside you'll see continuous glucose monitor and night Scout night Scout is the device so that's this is the hardware part of it and then how Raspberry Pi Linux running on Raspberry Pi is managing that device so from looking at this a few things um we can already identify as uh important Linux kernel subsystems for us one is that we SBI Dev driver if there are any changes to the SBI Dev driver that we know that we have to go and look at our workload and see our workload is um there are no regressions and that our workflow is functioning well and then the gpio a subsystem for example so those are the kind of the high Riders besides the other things that um that could have regressions um the other areas are common to Process Management I mean if you can't start a process that you would already know some of the process management things I put them in a different bucket of a common functionality and whereas SBI driver as SBI Dev driver and interfaces and then gpio would be something that we cannot find our workload will not run if there are problems um or regressions in that area so I will leave you with these two blogs in here uh to read more about this this the bottom one we uploaded this to I mean we upstreamed this this is part of the Cardinal documentation now so you can see that we went and um went and documented in May upstreamed it so that it's available for people in the kernel um so um that's pretty much end of my presentation and if you have any questions go go for the questions sure I would have a question if you go one slide back you were there was the picture of the continuous glucose monitor and also the night Scout is their some interaction between the Raspberry Pi and the night Scout or the I guess ultimately glucose monitor they need some iteration is there some Wireless connectivity then so this means you would run depending on the commands you would see the different subsystems then yeah and is there a wired connection then from looks like there is a gpio and then there's a direct connection to the insulin pump I always thought it would my understanding was it was Wireless kind of thing so oh yeah I mean so yes um the rig connection is wireless connection so you will it that would be um that happens on um broadcast.com um ethernet interface network library interface so when I'm showing this I'm just showing the connections this is not the real connections but what I mean is these these arrows indicate input and output process so yes you're right it's a it's a wireless connection consider to add other tracing mechanisms tools to get a deeper knowledge or do you say we're standing here and who it goes to go for f trace or adding some other tools from the automotive work group we recently also discussed if we maybe add a system tap tap track on it to just get also another representation of it because I know it's used in automotive to get boot time optimization track down bugs and so on you can do more um I've trace for example we can use a function level tracing with that Chris and say um any specific functions we are interested in at the kernel level if we can identify that and system trap is another good one to be able to use that so for us uh what we are running into is we might not um do anything more with open APS unless we have access to a rig we are running into some issues with access having access to your rig so however we uh as well that is for medical working group right and then the other areas um that we can start looking at is um we are I I'm currently doing analysis for RT Linux real-time Linux um one in that one I might do be able to do more uh tracing um for with one of the during part of that analysis which we are planning to present Ilana and myself are doing that work and then we'll be representing that in in Prague so we are continuing it not on open APS so to answer your question completely yeah thanks a lot, Discovering Linux Kernel Subsystems in Use, YouTube, 01ka7psmmmn72g27r0bkdm43vc.