Scorecards Biweekly Sync  ## Metadata - Author: [[OpenSSF]] - Full Title: Scorecards Biweekly Sync - Category: #articles - Document Note: --- tags: - #scorecards - #openssf - #security Author: Scorecards Maintainers (Spencer, Adrian Markham, Jonathan Howard, et al.) Source: YouTube – "Scorecards Biweekly Sync (July 13, 2023)" Reference: https://www.youtube.com/watch?v=KVeDGB95Yno date created: 2025-08-25 22:45 --- # Title: Scorecards Biweekly Sync – July 13, 2023 ## Table of Contents: - [📝 Summary](#-summary) - [📖 Main Content](#-main-content) - [🗒️ Notes](#️-notes) - [📖 References](#-references) - [🗣️ Paraphrase](#-paraphrase) - [📰 Newsletter](#-newsletter) - [✅ Action Items](#-action-items) - [🧪 Hypotheses](#-hypotheses) - [📊 Methodology](#-methodology) - [❓ Questions](#-questions) - [🔮 Future Research](#-future-research) - [💡 Implications](#-implications) - [🧠 Learning Styles Assessment](#-learning-styles-assessment) - [📊 Accessibility Assessment](#-accessibility-assessment) - [🌍 DEIB Assessment](#-deib-assessment) - [✅ Logical Validity](#-logical-validity) - [🔍 DARVO Analysis](#-darvo-analysis) - [🔍 SEO Evaluation](#-seo-evaluation) - [🎯 Conclusion](#-conclusion) --- ## 📝 Summary: The July 13, 2023 Scorecards Biweekly Sync focused on **changes to CI/CD scoring checks, dependency update tools like Dependabot, and roadmap development**. The group debated how to handle repositories without GitHub workflows, how to detect and reward dependency update practices, and whether structured results should replace ambiguous "-1" scores. Additional topics included **community roadmap planning**, **time zone-friendly meeting rotations**, and **maintainer responsibilities**. --- ## 📖 Main Content: The session underscored the challenges of building **fair and representative security scoring** across diverse CI/CD systems. Key debates revolved around whether repositories without GitHub workflows should score negatively, neutrally, or positively, and how to interpret Dependabot usage when configuration files are absent. Broader concerns included ensuring **clarity in structured results**, encouraging automated dependency management for security, and building a **roadmap process inclusive of community input**. The meeting also touched on logistical needs such as global accessibility of meetings. --- ### 🗒️ Notes - [00:02](https://www.youtube.com/watch?v=KVeDGB95Yno&t=2) Discussion on scoring changes for GitHub workflows. - Some repos dropped from 10 → -1. - Maintainers debate whether negative scores are appropriate. - [14:12](https://www.youtube.com/watch?v=KVeDGB95Yno&t=852) Security assessment scope. - Need to consider *all* release components, not just GitHub Actions. - CI/CD pipelines beyond GitHub may be overlooked. - [20:26](https://www.youtube.com/watch?v=KVeDGB95Yno&t=1226) Dangerous CI/CD checks. - Future plans: expand to GitLab, CircleCI, Travis, etc. - Minus-one scoring explained as "inconclusive" placeholder. - [26:55](https://www.youtube.com/watch?v=KVeDGB95Yno&t=1615) Dependabot detection. - Relying on PRs is unreliable. - YAML files may confirm active usage. - Debate: require config files or allow PR-only evidence. - [32:50](https://www.youtube.com/watch?v=KVeDGB95Yno&t=1970) Automated dependency updates. - Save time, improve security posture. - Expand beyond security-only updates. - [39:12](https://www.youtube.com/watch?v=KVeDGB95Yno&t=2352) Scoring options. - Either require YAML or keep current PR-based detection. - Consider weighted scoring if evidence is weak. - [46:14](https://www.youtube.com/watch?v=KVeDGB95Yno&t=2774) Indicators of security hygiene. - Merge requests and config files as proof. - Empty files could game the system. - [52:19](https://www.youtube.com/watch?v=KVeDGB95Yno&t=3139) Roadmap building. - Collaboration with Stephen and Adrian Markham. - Aim: create a **community-driven roadmap**. - [58:25](https://www.youtube.com/watch?v=KVeDGB95Yno&t=3505) Scheduling issues. - Alternate EU/Pacific time zones. - Doodle poll suggested; Dan to follow up. --- ## 📖 References: - OpenSSF Scorecards project documentation - GitHub Dependabot and Renovate bot guides - Security best practices in CI/CD pipelines --- ## 🗣️ Paraphrase: 1. *Original*: “Negative one scores indicate inconclusive results when no workflows are found.” **Paraphrase**: A -1 isn’t literally negative; it means the system can’t judge due to missing workflows. 2. *Original*: “Automated dependency updates reduce security risk by removing manual oversight.” **Paraphrase**: Using tools like Dependabot automates patching, reducing reliance on developers constantly monitoring vulnerabilities. --- ## 📰 Newsletter: 🚀 **Scorecards Sync Update – July 2023** The OpenSSF Scorecards team debated how to fairly score repos without GitHub workflows. Should missing workflows mean a penalty, a pass, or just “inconclusive”? The group also tackled **Dependabot detection quirks**—does a single old PR count, or should config files be required? Takeaway: automation is key. Projects enabling automated updates (Dependabot, Renovate, etc.) are far better equipped to handle vulnerabilities. The maintainers also started planning a **community-driven roadmap** and are working toward **global-friendly meeting times**. 🔗 Learn more: [Scorecards Project](https://github.com/ossf/scorecard) --- ## ✅ Action Items: - Apply structured scoring to avoid ambiguous **-1 results**. - Propose **config-file-based detection** for Dependabot. - Encourage WordPress contributors to adopt **automated dependency update tools**. - Integrate lessons into **OSPO advocacy**: security automation = contributor sustainability. --- ## 🧪 Hypotheses: - H1: Projects using **automated update tools** have significantly fewer vulnerabilities. - H2: Clearer scoring semantics improve contributor trust in Scorecards. - H3: Structured results increase adoption across diverse CI/CD systems. ### 📊 Methodology: - Collect repo datasets (GitHub + GitLab). - Compare vulnerability reports for automated vs. manual update projects. - Run surveys with OSS maintainers on Scorecards trust. ### ❓ Questions: - Should **security updates** be scored separately from **general updates**? - How should OSS projects be judged when using **non-GitHub CI/CD**? - Can structured results support multi-platform CI/CD equally well? ### 🔮 Future Research: - Study developer behavior when adopting automated update tools. - Investigate biases in Scorecards scores across ecosystems. ### 💡 Implications: - Better scoring clarity → more adoption of Scorecards. - Supports arguments for **funding OSS maintenance** via automation. - Aligns with **OSPO director role** priorities: trust, transparency, security. --- ## 🧠 Learning Styles Assessment: - **Visual**: Minimal diagrams → add flowcharts of scoring logic. - **Aural**: Strong (meeting transcript + recording). - **Verbal**: Present (detailed discussions). - **Kinesthetic**: Weak → suggest exercises (simulate repo scoring). - **Logical**: High (structured scoring debates). - **Social**: Strong (community-driven roadmap). - **Solitary**: Limited → add reflection prompts. - **Naturalistic**: Absent → could connect OSS sustainability to ecosystems. Readability: ~12th grade level. --- ## 📊 Accessibility Assessment: - Transcript available → strong base. - Needs clearer **alt-text** and semantic structuring. - Meeting videos should ensure **caption accuracy**. --- ## 🌍 DEIB Assessment: - Inclusive participation (new contributors welcomed). - Needs broader cultural perspectives (discussion skewed to Western OSS). - Encourage neurodiversity-friendly notes (shorter sections, clearer headers). --- ## ✅ Logical Validity: - **Strengths**: - Causal reasoning: automated updates → fewer vulnerabilities. - Ethical reasoning: community-driven scoring = fairness. - **Weaknesses**: - False dilemma risk (either PRs or YAML files detect Dependabot). - Tone: Professional, technical, collaborative. --- ## 🔍 DARVO Analysis: - **Deny**: None. - **Attack**: None. - **Reverse V - Summary: Meeting notes: https://docs.google.com/document/d/1b6d3CVJLsl7YnTE7ZaZQHdkdYIvuOQ8rzAmvVdypOWM/ https://github.com/ossf/scorecard#connect-with-the-scorecards-community - URL: https://www.youtube.com/watch?v=KVeDGB95Yno&ab_channel=OpenSSF foreignly loud foreign a few more minutes for people to trickle in can everyone see the notes shared on my screen yep I can see it and just in case uh the new document has people typing suggestions um if you want to fix it you can join the same scorecards Dev Google group and it should give you editor access but if suggestions work then uh carry on and we'll accept them when we can it looks like everyone's doing it already but as you file in if you can put your nameon the list in the doc uh if you don't have it I can post thelink in the chatuh it should be at least viewable byeveryone these daysumyeah so that should be the docsthose that don't have the linkall right uhit's about three after so I will goahead and get started for after uh hi everyone I'm Spencer one of thescorecard maintainers and I'm thefacilitator for today's meetingum so I will be protecting this windowuh with theum the notes as we go through everythingumand we have a relatively small agendafor now we'll see if anything gets addedlaterumbut usually we start our agenda byinviting anyone that's new to the sync to introduce yourself say a few wordsum who you are what you're hoping to getout of the sink Etcum this is optionalbut if you want to now go ahead and muteand introduce yourselfI will and this is how long I can stayso I'm just going to introduce myselfand then run because I am here um andyou're getting to know the rest of theopen ssf Team my name is Adrian MarkhamI have just joined uh LF within the past three weeks as a technical projectmanager my first big assignment is tohelp get the scorecardproject up and running soum some of you have already poked andI'll continue to poke around and justtry to see what we can do to getum get the progress happening a littlebit faster if we canum let me know how I can help and whatneeds to be done all right thanks AdrianI can go my name is Jonathan Howard workfor Lockheed Martin uh I'm on the hopperproject and I did some work for uh wellon a hopper plug-in for the open ssfscorecardI'm here just for uh you know awarenessand exposurecool thanks Jonathan good to have you all right with that we can jump intothingsum so it doesn't look like we have anyannouncements I did miss last syncumI don't see it um scorecard vae 11 cameout a little bit ago[Music]umwhich you know had some things in thepatch notes but leads me into the first agenda itemum and that was sort of a change in howtwo of the checks workwhere sometime betweenV10 dot I think it was four and v11there was a change ina check that for gitlab made no senseumso gitlab didn't support the notion ofGitHub workloads understandablyand how GitHub token permissions work so there was this PR thatbasically changed it to say that if youdon't have any GitHub workflow filesthenwe're going to return a minus one scoreumfor repositories that were on GitHubthis meant that the score now droppedfrom a 10 to a negative oneumso not every repo has a GitHub orprofiles and there's some discussion inthe issue aboutyou know does it make sense to have a negative want does it make sense to havea 10umso I thought Drogo made a good pointum about how it impactsthe aggregate score the overall scoreonce you combine all the checks whensome of the checks areminus oneumso I think this issue is sort of simpleenough that we don't have to come to aconclusion today butjust sort of drawing attention to the issue if people feel like weighing inand then I'll be pinging some of themaintainersum I'm not sure how many others areon the call right nowum yeah so David in chat says that wesay the score is zero to ten so negativeone doesn't make sense traditionallywe've used it for when like we eitherdon't have enough information toscore a particular check or if there'ssome sort of error that prevents us fromdoing the analysis so this might be you knowumfor whatever reason the token you'reusing doesn't have permissions there'sdifferent tokens and sometimes some ofthem disagree with certain checks or youknow we're trying to score you on codereview but the onlyPRS that we see are from you know dependabout soyeah I mean negative one's unfortunatebut it's usually reserved for situationswherewe can't make a judgment umand I think even with workflows gone wecan make a judgmentumbut yeah I think that's the issue so ifyou have some comments feel free tomention something either in the issue orduring the meeting but that's prettymuch it for theitemumwhat are we trying to measure here I'mjust thinking so one of the projects Iwork on is k-native and k-native uses both GitHub actions and prowwhich is an external system running onkubernetesumif you're doing a security assessmentand you're just looking at the GitHubactions pieceyou're missing a whole bunch of otherstuff that's running in order to producea releaseand so you mightspuriously give ahigh score when actually there's someother stuff going on that you don't seethat's worse and so I'm just trying tofigure out what are we trying to do with this scoreyeah so traditionally scorecard onlysupported GitHub repos and it was commonfor GitHub repos to have these gotchaswhere token permissions are a littleunintuitive and the permissions are youknow overly broad or it's possible forumbasically untrusted injection so thecheck was originallyinterested in do your GitHub tokens or GitHubworkflows have these common GitHubproblemsumso that's part of the reason why for thegitlab stuff it returns minus oneum and I I just don't think it's beenexpanded ora parallel version has been introducedthrough GitHubwell but also you could be usingsomething other than GitHub actions foryour CIand it seems like what we're trying toassess is how trustworthy is your CI yes but alsoso like historically when it was justGitHub it was how trustworthy is yourGitHub actions because like people stillused Travis or Circle Ci or somethinglike that and I don't think we everlike that was to my understanding notthe part of the check because likeI I'd have to get input of someonethat's been on the project a little bit longer butI'm proceeding under thebasis that it was just to look at GitHubactionsyeah I'll support you there Spencer likethethe point is to check to see if you'redoing something wrong not to check ifyou're doing everything right so it'sit's really justfor the the things that scorecard knows about are you doing anything wrong soyeah somebody's using Travis ifsomebody's using whatever CI it wouldn'tgive you a bad score but it alsowouldn't give you you know and andthat's just the the extent ofscorecard's knowledgeum and thenit would you know as scorecard becomesmore and more developed it would maybelearn more and more about other CIS andbe able to detect uh improper usagethere as welldoes that make sense Evan foreign it does I just wonder ifhey we couldn't detect any workflowsumwhether we should be doing this as anegative points from a hundred system orlike a positive points from zero typesystemand so we can't give you any points foryour CI because we can't tell what it isyeah that's an app that's a greatquestion I think that's the topic ofthis discussion like what do we do okayyeah yeah and the way scorecard scores rightnow there's not sort of a great answerofwhat happens because like the aggregatescore is based on all of the checks andif we continue to say you have no GitHubworkflows so everything is a 10. youknow that sort of vacuously true thatwill sort of boost up your score but asjogo mentioned in the issue thread ifyou say you know you have no workflows I can't really give you a score based onthis then that inflates the weight ofall the other checks as wellso it's kind of pros and cons of bothapproachesyeah I was immediate reacting to the heywe tell everybody that we give scoreeverything zero through ten so it should be at the very least we should fulfillour promises and score it 0 through 10.umas far as the hey what do you do whenyou don't know I mean that's always beena challengeumand uh you know in in one additionalQuirk I should add is you know somesystems may use multiples I mean I willpoint out the best practices badge weuse GitHub actions for a couple thingsmost of our checks are on Circle CI um so I'm not even sure what scorecarddoes in that case sobut I think we very least need to havezero through ten and for not giving themtens because we don't know I mean zerobecause I don't I can't see that youhave a CI pipelineum I think it's defensible but then weneed to say hey we didn't detect itum you know you might have one we justdon'tsee one see I see a CI pipeline but you have a problem I can see it you know acertain score at that levelum and more generally I think we alwayswant to have the higher leveldescriptionin the scorecard the more Generaldescriptionand then within the scoring in moredetails hey you've got a pipeline it hasthese kinds of problems the scorebecause you know hey we support onepipeline now we'll support others laterum it should be possible for us to figure out approximately where our valueis going to end up just as we add moreand more of these cases thanksyeah um I may have to find a differentroom in a second butthis check isn't necessarily concernedwith do you have a CI pipeline that'smore with the CI tests check looks atthis is if you're using GitHub actionsis it configured appropriatelyyeah and just one comment uh David on the zero to ten thing just in case uhthere's a bit there might be amisunderstanding the minus one isn'tactually a minus oneexactly that's right exactlyyeahokay yeah I don't object to that it'sthe if you were actually averaging it inthat would be a problem but as asentinel that's fine um I I can uh uh so I I'm uh put somecomments in the issue as well but it'snot inconceivable that in the future ascorecard would supportum other CIA methodsum and it's definitely like open forCommunity contributions uh like we Idon't think all of us are experts incircle CI but if an expert in circle CIwants to come insay here's what a dangerous workflowlooks like at Circle Ci or I think some of the gitlab people who have moreexperience with that would be like thisis what a dangerous workflow looksdangerous pipeline looks like in gitlabthat's certainly with a scope of thescorecard and we we shouldum do thatand uh the the reason I believe that uha inconclusive result is uh suitable inthe case where we don't find workflowsis that we're we're just uh it's almostlike scorecard failed to do something orscorecard just not programmed to detect a particular kind of workflow so it'sjust like scorecard itself isinconclusive and just can't say anythingone way or another about whether therepo is falling good uh good practicesyepyeahI think that argument is a lot easiersorry Pedro real quickum if we had structured results now sortof thing whereif we could say this is the GitHub sortof check then returning minus one uhmakes sense I I think you know you bringup a good point that right nowthe check is dangerous workflow and wecan't evaluate it in the sense ofif this is a git lab or a GitHub projectum and I think for that reason returningminus one for a gitlab repo makes sensebut for a GitHub workflow if it doesn'thave any workflows I think you can stillscore it on zero to ten where none ofthe workflows are dangerous becausethere aren't any workflows yeah it sounds like you're giving them abonus though for not having yourworkflow which doesn't that sounds rightit doesn't sound rightI don't know if it's a bonus so much asthis portion of the score has alwaysbeen towards workflows and if you don'thappen to have any then you can't be marked down for ituh Pedrouh yeah I know and but uh just likegiveaway like raghav mentioned of likewe might include dangerous CI CD onother platforms uh the minus one becomesuh more explainablebecomes more defensible shall we say inin that regard becausethen it's a minus one now and then we add information on Travis uh in tomorrowand then the minus one becomes a realscoreum that is more natural shall we saythan you get a 10 out of 10and then uh we add Travis tomorrow andyour 10 out of 10 becomes a zero uh thatbecomes a bit that's a bit I might feela bit weirder shall we say in terms ofhey if you couldn't judge my dangerousworkflow maybe you shouldn't have beengiving you the points all along so I don't think anyone's going to becurrently complaining about getting a 10out of 10.all rightso I mean it sounds like there's some um votes both for and against I'd becurious to get umlaurent's opinionas you know we work towards structuredresults onif we just maintain the status quo fornowon tens and thenif structured results are you know sixmonths out does thatchange sort of the discussionumbut yes uh you know it there's some good discussion aboutif we're only checking for GitHubworkflows is that you know a point boostand do certain repos get Point boostsfor either choosing to or choosing nottoumbut yeah thank you for those that leftcomments in the issues umI I think it's hard to make I guess aconclusive decision withuh only two maintainers but thank you everyone forcommentsumso I'm happy to move on to the nextissue uh Pedroyeah so this issue here is the fact thatthe dependency update tool check uh whenit's looking at the for when it it candetect depend about by looking at PR'suh and they can all just check like the history of PR's in the project lookingfor depend about PRS and if there areany then it currently just assumes okaythis project has depend aboutand howevera project mighthave turned off and depend about becausemany projects get annoyed and dependabout and it happens relatively oftenthat'll project will just turn it offum and it's hard to detect when lookingat PRS if that has happened um one solution is to instead of lookingat the PRS or as well as looking at thePRS is uh looking if the project has thedependabot yaml file the settings filefor the pendabotbut that file isn't strictly requiredfor the pendabot to run you only needthat uh the depend about yaml fileif the projectwants theconsistent frequent dependabot version bump PRShowever a project might not want thatand only opt-in for what they call thedependabot security updates wheredependabot will send a PR to fixdependencies uh having knownvulnerabilityuh and if if you have a product onlyopts in for security updates it doesn'tneed a depend about the ammo file so thequestion that ends up happening here in this issue especially further furtherdownisuh should we should it be like should weperhaps change or depending thedependency update tool when looking atthe pendabot uh to look for the file ornot basically like should depend aboutonly count if the project has opted infor version updates or only for secureor is the fact that the project hasaccepted security updates sufficient but then also if that is sufficient whichmeans no file we can look atum is there a way that we could possiblydetermine that better than looking atPRS or something I don't knowyeahI mean I I guess I'm going to drop backthe first principles the the goal isn'tdepend about the goal is that if thereis a security vulnerability if there's avulnerability that they updated prompto right soum I mean so if they have that file thatwould suggest that maybe they're doingitum one other approach is looking at pastuh I'm looking at what what are thesimple indicatorsum obviously if they have depend aboutuhuh commits that have been merged inthat's a pretty good indicatorumyou know worse comes to worse and this would be suddenly a whole bunch ofanalysis you know if they haven't hadany updates based on aversion we canlook and see are there any knownvulnerabilities backwards in that shouldhave been merged in and not thechallenge is of course is that suddenlyis a lot more work even if you're onlylooking at lock filesum yeah the the current setup fordependency update tool is the second oneof looking at DRS and how mistaken itonly looks for like are you receiving the pendabot update dependent Bots forexample or renovate bot whatever are youreceiving uh those PRS I'm not sure ifit looks at whether they're merged ornot but just like is it uh do you haveit the issue of looking at PRS is thatif you don't have a file to know thatthe panda bot is running right nowsometimes it'll detect dependabotbecause of a depend about PR from threeyears ago and then the issue ishas the last depend about PR beenbecause this project doesn't have anyother vulnerabilities that or hasn't hadother vulnerabilities that depend aboutwhere they're fixed or have they turnedit offum so we don't have that signalunfortunately as far as I can tell andso the file would give us acheap way of figuring out that ofskipping that whole thought process butit also is forcing people to accept version updates instead of just securityupdates which is I think probably closerto the spirit of scorecardI I don't think it has to be an eitheror kind of thing where a project canjust sign up for depend about securityupdates like they're doing now and we doattempt to look for it inrecent PRS it's just if a project isn'thappy that you know hey I have dependaboutinstalled they can go out and create I'm curious if even like an emptydependabot.yaml Works where you know youdon't sign up for anythingbut I I think in the thread um I canchange my tab back to it I was sort ofsuggesting like a repository can add adependabot.yaml to say yes I am actuallyusing this sort of thingumso I I don't know iftoo much is required to be changed onthe scorecard side other for other than the issue you mentioned thatthis project three years ago used dependabout once and then turned it off sortof thingumso at least to me what makes sense issome sort of look back with a windowwhich will hopefully get rid of thosefalse positivesbut may introduce false negatives inwhich case the solutionwould be potentially adding an extrafilewhich kind of sucks and scorecard has thisproblem every now and then where wedon't detect everythingumand we're working towards userspecified remediations or like saying nono no this is why that's not accurateum but we're not there yet so in themeantimeI I think are you just seeingI guess I'm I'm saying is it acceptablefor a repository to include one if they want to get rid of a false negative andyou'reasking if it should be required oryeah like I mean like I'm asking like uhshould yeah like should scorecard changeto require the Dependable uh yamlanother thought that crossed my mind isit could be like uh five points forhaving the ammo five points for PRS uhfor having at least one PR uh or something I don't knowum yeah I mean this year like I camewith a question I didn't come here withan answerof what the scorecard position should bejust raising this issueI think that's why I opened the checkdoc assuming you know the docs areaccurate which they're not I guess thepurpose of the check is to see if you use a dependency update toolandIf part of this says out of datedependencies make you vulnerable toflaws and dependabot is giving yousecurity updatesthen to me as written requiring a filethat's not necessary for what the toolis trying to checkwouldn't be necessaryum if we want to add like a recommended like hey if you're using this butyou're not we're not picking it uphere's something you can do as a hint Ithink that worksumbut I'd be curious if someone feelsstrongly ifsaying you know the point of adependency update tool isn't just forsecurity updates but also blah blah blahI I let me um I certainly encourage updating in general because if you don'tupdate it gets harder later on whenthere is a security update but I'm alittle hesitant to press the you knowplease keep your everything updated atall timesuh because there are cases where youcan't updateum I'm thinking particularly of a casewhere uh there's some softwarecomponents that we're licensed to anon-open source software componentthere's a new version oh waitso there may be good reasons you're notusing the current version having nothingto do with securityforeignI don't think any of this is looking tosee if the PR's actually mergedum because like on Dependable you cansay ignore this version ignore this likeignore this dependencyokayall right so the question really does does become do we want to just encourageupdating all versions period or focus onthe security updates[Music]two minds on this I do think in generalit's wise to update I also do think thatyou want to prioritize the importantones whatever that means for the biggerprojects it's a little over Dependable could be a little overwhelming if youactually enable it for all versions I'mpretty sure we only enable it for uhuh the security vulnerabilities forexampleso best practicesyeah David I agree with you that there'sdefinitely good reasons for not mergingstuff in and and kind of where you'regoing earlierthe ideal state would be could we detectif that caused you to start taking onvulnerabilities because you weren't updatingthat's kind of more of where it startsto become a concern for security becauseI think what you just said is verycommon where you at least don't Automerge things that are beyond that patchversionum and you may choose to delay evenmanual merges of those minor and majorupdatesfor very good reasonsbut those Also may become unsupportedand then you do have a legitimatesecurity concern at some point rightright I guess we're on firm ground on askingpeople to you know make sure you've beenautomated you have automated enabling ofsecurity vulnerabilities and you knowencouraging you know push the button andupdate the question here is should webroaden that tojust you're out of date periodas a warning I think some of this would be helped ifGitHub had a way of querying if someonehad depend about security updatesenabled but it's just not somethingat least I don't think that you canquery with a normal tokenand I'm not sure if you can query itwith an admin one either butyeahthe idea of encouraging it because Ifeel like a lot of projectswouldn't go do that by default and it really is a huge Boon towards yoursecurity to not have that maintenanceburden of constantly watching for 50dependencies you have so I I really dothink it's a good rulein generalto enable an automated reviewyeah to enable an automatedautomated uh dependency update that ismerging things like I think that isa big security help for a lot of projects absolutelyabsolutelyyeah so the question is it comes back toit do we want to emphasize that only forthe security vulnerabilities or moregenerallybe nice if we had some announce likesomebody had done some analysis of theimpact of doing it either wayI don't know if it's uh if there's anysuch research I'm doing a quick Google Search rightnow this sounds like the sort ofquestion that somebody might haveansweredyeahI'm just picking up you knowGitHub information about how to set theconfiguration ah research gate on the use ofdependabot security pull-up requestsinteresting all right well all right I'm just trying to capturesome of our discussion in text umto sort of summarizeumsoit's like they're sort of two choiceswhereum either we require it or keep it as iswhich isa signal but not necessarily required umand then I don't think this was part ofPedro's question but are we trying toget I guess a decision onshould we bedisqualifyingthe current implementation that looksback indefinitely for a defend aboutcommit which I I think the answer is yesit just becomes very hard to say how farback do we look or what are we willingto accept sort of thing but [Music]umyeah sorry I'm just trying to thinkthrough the pros and cons hereforeign like a clunky way of doing it would beto notmake a decisionand instead say heyuh if you've got to depend about PR overthe last yearyou get a 10 out of 10.if the last one was two years ago three years agoand you don't have the dependent bodyammo file we don't know if this thing'sstill running but maybe it is so we'llgive you a couple of points we kind ofgive the if they don't have the file wegive them fewer points as time goes onum as we lose confidence that theyactually have depend about uh turned onlike it's a clunky it's actually it's clunky but it's not as crazy as itsounds giving them less points forsomething we have less confidence in[Music]umit's a little clunky but it has meritsuh you know because I guess the the easysolution is if they add this font thedepend about file then we know for sureyeah right that's the one well I guessthey could turn it off and have that yaml file in there couldn't they but Idon't think so actually if you have theyaml file like you can't turn it off I Iunderstand it just kind of keeps goingI think the question is what happenswith an empty file or a file without anecosystems because like in the file yousay this is an ecosystem for some of mypackagesand I guess what happens when you don'thave thatthat Iyeah I have no idea but I wouldn't besurprised if what it means is that it runs as like it runs the the scan whichmeans nothing but if you have the but ifyou have it active for version updateswhich is what you would with theDependable with the yaml file you alsohave it active for security uh updatesso it actually might be that having likea blank fileum also automatically puts you in for uhsecurity updates as well but haven'ttexted it so can't be sureforeign yeah so I mean really it sounds like ifwe have if you have a file we shouldcheck but our current under estimate isthat that probably means it runsyou know probably should go do thatresearch and then come back but let'sassume for the moment it does it seemslikelyso in that point the only real questionis what do we do if not and hey if there's depend about or some moreresults and we keep saying depend aboutbut obviously any other tool that doesdependency analysis and helps them keepup to date is what mattersso yeah either either we see evidence ofit in the in uh proposed changes orrequest merge requests or we see someconfiguration that suggests that in factit's it's going onyeah so I mean the things I depend aboutis a specific case because with renovate and the other ones they are likeconfig as code like renovate you canusually identify and actually you can'tactually I'm I'mI'm wrong right depend about itrenovate's actually activated on theaccount not on the uh that's true nevermind we do look for the um renovateconfig files that's the only way we haveevidence of using renovate right now forscorecard okay okay but you're stilllooking for it and again you're you'rejust looking for evidence right if youhave evidence hooray you get some points so we're just looking I mean this isjust no different from the otherCheckers it's you're looking forevidenceI mean same for gymnasium on the get labsideright I don't know if that's beinglooked for today but you could certainlylook for it at some pointyeah as I understand it Evan this is ayear using dependabot but for whateverreason we can't detect it because um you know it's not there's no evidencethat it's doing somethingyeahyou don't have it configured there's noevidence of a config file and there's noevidence of any people request mergerequestsfor from itat that point at that at that point itdoes make sense to give it a zero scorebecause we have no evidence thatsomething's going on I mean I'm just thinking aboutumcan I have a depend about yaml butactually have gone and slid the sliderto offwell it in general that we're not reallyassuming maliciousness yeahum I mean there's actually a muchsimpler way of disabling depend aboutand that's simply using a language thatit doesn't support that too yeah[Laughter]I can write about you know I can write a million lines in a language that isn'tsupported and yeah it doesn't matterwhat what I door just use a dependency tool that itdoesn't know aboutright which I mean that's a generalproblem right yeahyeah I mean there's all sorts of cornercases like if you're writing a librarythat has no dependencies too like whatdo you doumso before we get too deep into a can of worms I'm going to realize back a littlebit but it sounds likethe biggest question is how does thisbehave with an empty depend about emo umyou know we've brought that up a fewtimesum so I think a good Next Step would besomeoneum either Pedro or I could try to findtime between or before next meeting justsort of run a little experiment see whathappens um and then once we have a little moreinformation it might be easier to sortof make a decisionI think the the coming bound to thebrass tacks though the whole point is ifthere is a vulnerable dependency we wantthere to be automated reporting andideally automatedumrepair and the only indicators we haveare pull request merge requests and thepre and a yaml file which may have to bemore than empty depending on the results of your tests uh the empty dependencycase is actually easyput in the ammo fileit will detect that there's nothing todepend on and never report anythingyou're goldenand as silly as that sound and you get a10 and as silly as that sounds a lot ofprojects eventually managed to getthemselves a dependencywell it also shows that you've thoughtabout it that's right that's rightyou've thought about it you don't haveany dependencies today that's great you create the yaml you put a comment inhere I'm doing this for this reasonyep walk away walk awayand if you truly don't have anydependencies it's a one-time commitall right uh Spencer in the previous uh point ondangerous workflow token permissions uhyou said you wanted to get a Long'sopinion he is in the in the call now atleast I believe he is at least someonecalled so faryeahumLawrence currently muted butif you're listening Laura for context wewere talking about sort of structured results earlier inthe callwherethere's this notion of how do we judge arepository[Music]umor like a repo on dangerous workflow andtoken permissions wheneither so like there's two scenarios onewhere it's on an ecosystem that doesn'tsupport GitHub tokens and workflowsbecause it's not GitHubor it does your GitHub repo but you've done your CI CD on another system sortof thingumand it would be easy if we hadstructured results because we could youknow say not applicable or somethinglike that but in the current frameworklikedoes it make a difference where like agitlab repo gets a minus one a GitHubrepo gets a 10 but what if you have yourCI in a different systemso what's the question does it matterright now um are youuh did you get a chance probably not umto sort of see the issueno I haven't I understand the high levelproblem that ins in on some platform itdoesn't make sense to say that it passedbecause it doesn't use workloads forexample on GitHubyeah so I I think on gitlab no one'sarguing that you know if you have a gitLibrary po a score of minus one sort of makes sensethe question is like on GitHub reposwithout workflowsright minus one still makes sense oryou know because you have no repos youscore a 10 because there's no dangerousor workflows there's no Dangerous Onesit's just annoying becausehaving the negative ones does change thescoring the score yeah I think even for gitlab I'm not sure that minus one makessense because minus one meansinconclusive which which means I thinkwe we don't know I think what we'relooking for is like a new one that saysnot applicablewhich is basically ayeah so I guessI guess that's my first reaction aboutthe minus one maybe it's not a problemin practice when we compute the score Ihaven't thought enough about it and onGitHubyeah if you don't haveif you don't have workflow I would say it's the same it'snot applicable but it's not applicablein the sense that you passwhereas in the on the gitlab it's notapplicable because it makes no sense soI guess this this differentiationideally that would be able to capturebecause otherwise someone on git onGitHub that doesn'tyeah no I guess those this is what'sgoing through my mind as I uh addresshere so I think we need to consolidatethe return values that we have they're not good enough I guessyes yeah I I think you've left someholes in your enumerations forstructured results to sort of input moreum yeah yeah absolutely yeah for thatreason that I actually didn't know whatwe neededso uh yeah yeah I think it should weshould make sure that it works withstructured resultsI hope we canis there an action item that for me or is it just something for me to keep inmind for as I send PRS for thestructured resultsum I I I think it's something you knowbefore we move structured results to arelease kind of thing we should sayand I I think we'll see this as wemigrate more more checks umI I think the only action item for rightnowum is sort of getting a maintainermajority on If part of the scoring change should bereverted if this is just something thatwe're going to deal with until we havestructured resultsum I don't think that has to be right inthis meeting but if at some point youcan respond to this issue I think that'ssort of the action item and I'll youknow I'll tag Naveen as well butI think uh Markov and I have weighed inum yeah sounds good so just at a high level I don't think waiting forstructured results is necessary becauseanything that structured result does isreally just reusingit's reformatting the output but it'sthe scoring is happening independentlyof the structured resultsso anything thatany scoring we would change I thinkwouldn't affect the structured resultit's just that the scoring will happenon the structured result instead ofhappening on the Raw results today asit's passed that if that makes sense for those who are more familiar with thecodewe can take that offline offline Spencerif you want to discuss yeah I mean Ithinkthere's some interesting likemajor version this is how we changescorings like does it make sense tochange How We Do scoring significantlyin the middle of E4 like right now wehave zero to ten with a minus one ifwe're not applicable so at least in mymind as long as we're on V4 that's sortof what we're sticking to umdo that make sense to meall rightumum so Brian I saw that you added that ifyou want totalk about it you can otherwise I willjust wait a few seconds and then read ityeah I thought I'd just take an update that Steven put in slack it's alsogenerally my update building off of theroad mappingumI guess presentation or what I I talkedabout in our last meeting for just whatat least at Google we were committing toum and the feedback from the group tobuild a broader Community road mapum I'm gonna be uh chatting with Stephenand umAdrian Markham who uh if you haven't met yet is a is a new TPM for the openssfso if you remember from the previousmeeting Stephen volunteered to kind ofwalk through how some other projects uhbuilt up Community road maps with me andso we delayed that a little bit uh andit just happens thatum patreon is is looking at a verysimilar thing across a lot of differentprojects so the three of us are gonnajust have a discussion about what that can look like and then hopefully fornext meeting we can bring backumnot just kind of a summary of what wediscussed but alsoreally start that process of of buildingup a broader Community road map for theprojectcool thanks Brian yeah reallyinteresting to see Lauren did you want to talk about this Iunfortunately had to miss last meetingso I think you got a little more insightyes so I think last time we metum we discussed the uh the time zoneand whether we can alternate between aEU friendly and then a Pacificfriendly time zoneI think someone said they would share adoodle or or something equivalent I was I just wanted to to follow up because Iknow this thing justslip through the cracks and then we'regoing to forgetuh but just I was just curious if anyonehas updates onwhether something was shared on slackumor anythingand if not I don't I don't remember whowaswho said they would do something[Music] I think we can um tagumI'm looking at the thing from last uhDan we can ask him I would assume thatas the one that brought the issue hewould at leastuh it saidum okay I think I'm getting confused butyeah let's just ask Dan aboutif there was a doodle Zone in slack andthen we can follow up on ityeah thanks for catching thatwith that we are at the end of theagenda so barringme last minute questionsumall that's left is to pick a facilitator for the next sync which is currentlyJuly 27thumso if anyone wants to feel freeto volunteerumthey are otherwiseI'm sure we can force someone to do itnext weekone of the maintainers all rightuh thanks everyone for comingum discussing uhplease take a look at the issues andleave a comment ifanythingyou want to say wasn't represented themeeting or added to the notes um butotherwise see folks in two weeks andthanks everyone, Scorecards Biweekly Sync, OpenSSF, 01h65a739ey31vv5xrh35pax19.