Full Systematic Literature Review: PRISMA & PICO in 5 Steps  🔗 **Source:** [Full Systematic Literature Review: PRISMA & PICO in 5 Steps](https://www.youtube.com/watch?v=uur5MmjTe5I) > [!note]+ Document Notes > tags: [systematic-review, research-methodology, prisma, pico, evidence-synthesis, literature-review] > author: YouTube > source: https://www.youtube.com/watch?v=uur5MmjTe5I > reference: https://www.youtube.com/watch?v=uur5MmjTe5I > date created: 2026-06-23 00:00 > > # 🏷️ Title: Full Systematic Literature Review: PRISMA & PICO in 5 Steps > <!-- deep-read:sections start --> ## 🧭 Table of Contents - [📝 Summary](#-summary) - [📖 Main Content](#-main-content) - [🗒️ Notes](#️-notes) - [📚 References](#-references) - [🗣️ Paraphrase](#️-paraphrase) - [📰 Newsletter](#-newsletter) - [✅ Action Items](#-action-items) - [🧪 Hypotheses](#-hypotheses) - [📊 Methodology](#-methodology) - [❓ Questions](#-questions) - [🔮 Future Research](#-future-research) - [💡 Implications](#-implications) - [🧠 Learning Styles Assessment](#-learning-styles-assessment) - [📊 Accessibility Assessment](#-accessibility-assessment) - [🌍 DEIB Assessment](#-deib-assessment) - [✅ Logical Validity](#-logical-validity) - [🛡️ DARVO Analysis](#-darvo-analysis) - [🎯 Conclusion](#-conclusion) ### 🔗 References - [PROSPERO registry](https://www.crd.york.ac.uk/prospero/) - [Open Science Framework (OSF)](https://osf.io/) - [PRISMA statement](https://www.prisma-statement.org/) - [Rayyan screening tool](https://www.rayyan.ai/) ## 📝 Summary - A presenter walks through a five-step process for running a systematic literature review, anchored on three frameworks: PICO (how to ask the question), PROSPERO (how to register it), and PRISMA (how to report it). - It matters because systematic reviews carry weight in science precisely because they are pre-planned, transparent, and reproducible — not retrospective cherry-picking. A full review takes 11–16 months. - Key stakeholders: graduate researchers, health-sciences scholars, evidence-synthesis teams, and university research librarians. - Risks and tradeoffs: skipping registration invites suspicion of post-hoc bias; single-reviewer screening undermines credibility; ignoring risk-of-bias appraisal lets low-quality studies skew conclusions. - Bottom line: frame and register first, build a defensible Boolean search, screen blind with two reviewers, appraise quality, then report against the PRISMA checklist and flow diagram. ## 📖 Main Content - The core argument is that a credible systematic review is a sequence of pre-committed, documented decisions — not a literature scan dressed up after the fact. Each framework guards a different failure mode. - **PICO** turns a vague question into a searchable one. It stands for Population (who), Intervention (what you test), Comparison (against what), and Outcome (how you measure). Example given: "In adults with type 2 diabetes, does a structured exercise program compared with usual care improve blood sugar control?" For qualitative work, PICO shifts to **PICo** — Population, phenomenon of Interest, and Context. - **PROSPERO** is the prospective registry. You register the question, eligibility, search strategy, and analysis plan before screening starts; approval is free and takes days to a few weeks. Many health journals now require a PROSPERO ID at submission. PROSPERO accepts education, social care, criminology, and international development reviews only when there's a health-related outcome; otherwise use **OSF**. - **Boolean search building**: list keywords and synonyms per PICO concept, join synonyms within a concept using OR, then combine concepts with AND (each concept bracketed). Document every search string, date, and limit. Include gray literature (theses, conference papers, government reports, white papers). Get the strategy peer-reviewed by a research librarian. - **Two-reviewer blind screening**: two people independently screen titles/abstracts, then full text, without seeing each other's calls. Disagreements resolve by discussion or a third reviewer. Inter-rater agreement is measured with **Cohen's kappa** (>0.61 acceptable, >0.8 ideal). About 40% disagreement on full-text is described as normal. Tools named: Rayyan and a second screening tool. - **Appraise and extract**: assess each study's risk of bias with a fit-for-purpose tool, visualize it (the presenter names "Robis"/ROB-VIS-style figures), extract into a standardized form, and let the data — not preference — drive the synthesis choice. - **Report with PRISMA**: work through the PRISMA checklist, fill the PRISMA flow diagram with real numbers (records identified, duplicates removed, screened, excluded, assessed, included), state methods explicitly, and be honest about limitations. Flow-diagram generator tools exist to format this. - Contested/uncertain points: the presenter's tool names are spoken casually ("Robis," "Ryan") and likely refer to ROBIS/ROBVIS and Rayyan; the "80 million records" and "zero included" figures are illustrative placeholders from a sample diagram, not a real result. ### 🗒️ Notes #### ⏱️ [00:00:06](https://www.youtube.com/watch?v=uur5MmjTe5I?t=6) The three P's - PICO asks the question to the literature; PROSPERO registers the review; PRISMA defines what to report. - "There's this thing called PICO, and essentially this is how you ask a question to the literature." #### ⏱️ [00:00:32](https://www.youtube.com/watch?v=uur5MmjTe5I?t=32) Five-step overview - Frame and register → build a defensible search → screen with two reviewers → appraise quality and extract → report against PRISMA. - Registration is "often skipped" — it proves the plan wasn't retrospective. #### ⏱️ [00:00:57](https://www.youtube.com/watch?v=uur5MmjTe5I?t=57) Step 1 — register on PROSPERO - Free approval in days to weeks; a full review takes 11–16 months. - "You lock in the question, eligibility, search strategy, and analysis plan." - Most health journals want a PROSPERO ID at submission; non-health reviews use OSF. #### ⏱️ [00:01:56](https://www.youtube.com/watch?v=uur5MmjTe5I?t=116) Step 2 — build the PICO question - Population, Intervention, Comparison, Outcome → distilled searchable question. - Qualitative variant PICo: Population, phenomenon of Interest, Context (e.g., parents of autistic children implementing early interventions). #### ⏱️ [00:03:00](https://www.youtube.com/watch?v=uur5MmjTe5I?t=180) Boolean search strings - Synonyms joined by OR within each concept; concepts bracketed and joined by AND. - "Every search string that you create ... you need to make sure you document it somewhere." - Include gray literature; get the search peer-reviewed by a research librarian. #### ⏱️ [00:04:45](https://www.youtube.com/watch?v=uur5MmjTe5I?t=285) Step 3 — two blind reviewers - Independent title/abstract screening, then full text; resolve disputes by discussion or a third reviewer. - Cohen's kappa: >0.61 acceptable, >0.8 perfect; ~40% disagreement is normal. Tools: Rayyan. #### ⏱️ [00:05:57](https://www.youtube.com/watch?v=uur5MmjTe5I?t=357) Step 4 — appraise bias and extract - Assess each study's risk of bias with a fit-for-purpose tool; visualize with a bias figure. - Extract into a standardized form; let the data choose the synthesis method, not preference. #### ⏱️ [00:06:36](https://www.youtube.com/watch?v=uur5MmjTe5I?t=396) Step 5 — report with PRISMA - Use the PRISMA checklist and flow diagram; state methods explicitly; be honest about limitations. - "By being very clear and upfront with the limitations of your study, people are going to trust it much, much more." ## 📚 References - [PROSPERO international prospective register of systematic reviews](https://www.crd.york.ac.uk/prospero/) - [Open Science Framework](https://osf.io/) - [PRISMA statement and checklist](https://www.prisma-statement.org/) - [Rayyan systematic review screening](https://www.rayyan.ai/) - [Cochrane risk-of-bias tools (ROB 2 / ROBINS-I)](https://methods.cochrane.org/risk-bias) ## 🗣️ Paraphrase - A systematic review earns trust by being planned in advance: you frame a precise question, register it, then do the work — rather than fishing for a result you already want. - The mechanics are a sequence of guardrails: PICO sharpens the question, Boolean logic builds a search that surfaces the right literature, two independent reviewers keep screening honest, bias appraisal protects quality, and PRISMA makes the whole thing reportable and reproducible. ## 📰 Newsletter - Doing a research roundup before your next plugin or feature? The systematic-review playbook scales down nicely: frame a tight question (PICO), register your plan, search with documented Boolean strings, and screen with a second pair of eyes. It's the same discipline a good RFC or evidence-backed docs page deserves. Bonus: your friendly research librarian is an underused superpower — for code and for citations alike. Links worth bookmarking: [PRISMA](https://www.prisma-statement.org/), [PROSPERO](https://www.crd.york.ac.uk/prospero/), [Rayyan](https://www.rayyan.ai/). ## ✅ Action Items #### DevRel - Developer advocacy: model "pre-register your question" rigor when publishing benchmark or comparison posts. - Education: build a one-page PICO→Boolean→PRISMA cheat sheet for community researchers. - Developer experience: template a documented-search worksheet (string, database, date, limits). - Developer marketing: cite evidence-synthesis standards when making "best tool" claims. - Community management: recruit a second reviewer norm for any community research write-up. - Internal/external/cross-project: align on a shared evidence-quality bar before publishing comparisons. #### OSPO - Policy & Governance: adopt a registration-before-analysis norm for internal research reports. - License Compliance: not applicable to this source — the video covers research methodology, not licensing. - Security Management: not applicable to this source — no security content. - Software Inventory: not applicable to this source — no SBOM content. - Developer Support: offer a "how to run a defensible lit review" consultation hour. - Community Engagement: partner with university librarians for upstream research literacy. - Strategic Alignment: tie evidence-synthesis rigor to reporting KPIs. #### Community - Maintainers: require pre-registered questions for roadmap research. - Contributors: pair new contributors as second reviewers on research tasks. - Sponsors: fund transparent, registered evidence reviews. - Ecosystem: run cross-project evidence reviews with shared PRISMA reporting. #### WordPress - Core: not applicable to this source — methodology video, not WP Core work. - Core Performance: not applicable to this source. - Core AI: borrow risk-of-bias appraisal language for AI evaluation sets. - **Core Program Team:** adopt two-reviewer screening as a research ritual. - Mobile: not applicable to this source. - Playground: build a reproducible PICO/Boolean teaching module. - Community: host an evidence-synthesis workshop with a research librarian. - Training: add a "systematic review basics" learning path lesson. - Marketing: not applicable to this source. - Support: not applicable to this source. - Design: not applicable to this source. - Documentation: document a search-string logging pattern for research docs. - Photos: not applicable to this source. - TV: caption and index methodology talks for discoverability. - Polyglots: translate the PICO/PRISMA cheat sheet. - Accessibility: not applicable to this source. - Meta: not applicable to this source. - CLI: not applicable to this source. - Hosting: not applicable to this source. - Tide: borrow risk-of-bias transparency framing for scoring. - Openverse: not applicable to this source. - Themes: not applicable to this source. - Plugins: not applicable to this source. #### The WPCC - Programs: fund evidence-review fellowships using PRISMA reporting standards. - Projects: run governance-research sprints with registered questions. - Funding experiments: tie milestone payouts to transparent, reproducible reviews. ### FAIR - Technical independence: keep registration metadata and provenance local and verifiable. - Policies and covenants: adopt transparency covenants mirroring PROSPERO registration. - Productized governance artifacts: ship a review-protocol scorecard template. ## 🧪 Hypotheses - Pre-registering a review on PROSPERO reduces post-hoc outcome switching versus unregistered reviews. - Two-reviewer blind screening yields higher inter-rater agreement than single-reviewer screening on the same corpus. - Librarian-peer-reviewed Boolean searches retrieve more relevant studies than unreviewed searches. - Reviews reporting against the PRISMA checklist receive higher reviewer trust scores than those that don't. - Including gray literature changes the pooled effect estimate compared with published-only searches. ### 📊 Methodology - Study designs: retrospective cohort comparison of registered vs unregistered reviews; randomized methodology trial of one- vs two-reviewer screening; cross-over librarian-review experiment on search yield. - Data sources: PROSPERO/OSF registries, published reviews, screening-tool logs (Rayyan), PRISMA checklist adherence audits. - Analyses: Cohen's kappa for agreement, sensitivity/specificity of retrieval, effect-estimate deltas with/without gray literature. ### ❓ Questions - How much does mandatory registration actually shift published conclusions in practice? - Where do AI-assisted screening tools change kappa, and do they introduce systematic bias? - What's the marginal value of a third reviewer beyond the ~40% disagreement baseline? ### 🔮 Future Research - Quantify the effect of LLM-assisted search-string generation on retrieval precision and recall. - Study reporting-quality gains from PRISMA 2020 adoption versus the 2009 statement. - Measure how gray-literature inclusion affects bias and conclusions across disciplines. ### 💡 Implications - Practice: research teams should bake registration and dual screening into their default workflow. - Policy: journals and funders enforcing PROSPERO IDs raise the field's reproducibility floor. - Security: not applicable to this source — no security dimension. - Education: curricula should teach Boolean search and bias appraisal as core literacy. - Funding models: grants can require registered protocols and transparent limitations sections. ### 🧠 Learning Styles Assessment - Reading level estimate: Flesch-Kincaid Grade ~7–8 (conversational spoken transcript, short clauses, common vocabulary). #### 👀 Visual learners - Present: on-screen software demos, the PRISMA flow diagram, laser-pointer walkthroughs. - Improve: add a clean labeled PICO/Boolean diagram and a before/after search-string visual with captions. #### 🔊 Aural learners - Present: full spoken narration throughout. - Improve: offer a 2–5 minute audio recap and speaker notes. #### 🗣️ Verbal learners - Present: definitions and worked examples (diabetes PICO, autism PICo). - Improve: add a glossary (kappa, gray literature, risk of bias) and plain-language callouts. #### 🧩 Kinesthetic learners - Present: live tool walkthroughs (PROSPERO, OSF, Rayyan, flow-diagram maker). - Improve: add a downloadable step-by-step checklist and a sandbox search exercise. #### 🧠 Logical learners - Present: numbered five-step sequence, AND/OR Boolean logic, kappa thresholds. - Improve: add a decision tree (health vs non-health → PROSPERO vs OSF) and a RACI for reviewer roles. #### 🧑‍🤝‍🧑 Social learners - Present: emphasis on two-reviewer collaboration and librarian consultation. - Improve: add discussion prompts and a peer-review rubric for screening. #### 🧘‍♀️ Solitary learners - Present: self-paced framework the viewer can apply alone. - Improve: add reflection prompts and a self-check quiz on PICO components. #### 🌿 Naturalistic learners - Present: minimal — a casual aside about the presenter's shed. - Improve: tie evidence synthesis to lifecycle/ecosystem analogies (e.g., research provenance trails). ### 📊 Accessibility Assessment #### Screen reader semantics - Present: spoken structure with clear step numbering. - Improve: provide an HTML transcript with proper heading landmarks and alt text for diagrams. #### Keyboard navigation - Not applicable to this source — it's a video, not an interactive interface; applies to any companion web page. #### Contrast and readability - Present: laser-pointer highlighting on screen-shared tools. - Improve: ensure on-screen text and diagrams meet 4.5:1 contrast; avoid relying on pointer color alone. #### Text resizing and reflow - Not applicable to this source directly; ensure any companion checklist page reflows at 200% zoom. #### Media and captions - Present: auto-transcript available. - Improve: add human-verified captions, a downloadable transcript, and audio descriptions of diagrams. #### Forms and interactions - Not applicable to this source — no forms in the video itself. #### Standards reference - Map gaps to WCAG 2.2 AA: 1.2.2 Captions, 1.2.5 Audio Description, 1.4.3 Contrast, 1.1.1 Non-text Content. ### 🌍 DEIB Assessment #### Cultural breadth - Present: examples are generic (diabetes, autism) and not region-locked. - Improve: add multi-region case studies and note non-English database coverage. #### Inclusive language - Present: plain, friendly, mostly jargon-defined language. - Improve: define kappa and "gray literature" inline; expand a glossary. #### Neurodiversity support - Present: consistent numbered structure and chunked steps. - Improve: add frequent headings, icon+text labels, and predictable section patterns. #### Representation - Present: an autism early-intervention example centers a real population. - Improve: vary example domains and populations to broaden representation. #### Psychological safety - Present: encouraging tone ("you're going to kill it"); normalizes disagreement and imperfection. - Improve: note that a mild innuendo aside ("turned on by all of this stuff") could read as unprofessional; a content-light edit would broaden comfort. ### ✅ Logical Validity #### Strong reasoning - Causal: registration before screening → credibility, because it removes the suspicion of retrospective cherry-picking (Step 1). Strong because it ties a concrete action to a named failure mode. - Deductive: AND/OR Boolean composition reliably narrows by concept and broadens by synonym (Step 2/3). Strong and well-illustrated; could clarify with a Venn visual. - Analogical/inductive: kappa thresholds (>0.61, >0.8) generalize agreement quality from convention. Reasonable but presented as settled — citing the source convention would strengthen it. #### Fallacies - Appeal to authority (mild): "people will trust your review much more ... if you go through this" (Step 5) leans on trust/standards without evidence of the trust effect. Fix: cite reproducibility/reporting-quality studies. - Hasty generalization: "every review isn't perfect" and "about 40% disagreement is normal" are stated as universal without a source. Fix: attribute the 40% figure or frame it as typical-in-context. #### Tone of voice - Casual, encouraging, self-deprecating, and slightly irreverent (the shed/t-shirt aside, the "number one tool" joke, the librarian innuendo). Reads as an approachable educator addressing nervous grad students. ### 🛡️ DARVO Analysis - Deny: not applicable to this source — no accusation is being answered, so there's nothing denied or minimized. - Attack: not applicable to this source — no accuser or credibility target exists; the lone jab ("one of them's lying") is a lighthearted aside about two tools both claiming "number one," not a real attack. - Reverse victim/offender: not applicable to this source — there is no victim/offender dynamic in an instructional methodology video. ## 🎯 Conclusion - Bottom line: a trustworthy systematic review is pre-planned and transparent — frame and register the question (PICO + PROSPERO/OSF), build and document a Boolean search, screen blind with two reviewers, appraise bias, then report against PRISMA. - Next steps for readers: draft your PICO question, register on PROSPERO (or OSF if non-health), get your search peer-reviewed by a librarian, recruit a second reviewer, and download the PRISMA checklist and flow-diagram template before writing. ## 🔄 Refactor Appendix — Full Systematic Literature Review: PRISMA & PICO in 5 Steps > Density: 2–5 Literature atoms (H3), 0–3 Permanent candidates (H4). > Defaults: 3 Literature, 1 Permanent. Resonance filter: if you can't > articulate in one line why a claim matters, drop it. > > Konik claim-header rules: present tense, declarative, ≤10 words when > possible, searchable, unique, mirrors annotation focus if present. > > Atomicity: one declarative sentence per atom. "and"/"also" → split. > > Tag pool: #bmf · #addendum · #xref · #articleseed · #storystem · #share · #aphorisms ### 📚 Literature Atoms (H3) #### [H1] Pre-registration proves a review wasn't retrospective > [!quote] ID:rw_uur5MmjTe5I_h1 > "This tells people that it's not retrospective, but in fact you had a plan going in." **Context & use**: - [00:00:32](https://www.youtube.com/watch?v=uur5MmjTe5I?t=32) five-step overview. - Registration is the credibility anchor of evidence synthesis; mirrors the deep-read habit of committing a method before collecting. **Tags**: #xref **Highlight**: [view highlight](https://www.youtube.com/watch?v=uur5MmjTe5I?t=32) <!-- atomicity: 1 idea ✓ --> #### [H2] PICO turns a vague idea into a searchable query > [!quote] ID:rw_uur5MmjTe5I_h2 > "PICO stands for population, intervention, comparison, and outcome." **Context & use**: - [00:02:04](https://www.youtube.com/watch?v=uur5MmjTe5I?t=124) PICO breakdown. - A reusable question-framing scaffold; PICo (Context) covers qualitative work. **Tags**: #xref #aphorisms **Highlight**: [view highlight](https://www.youtube.com/watch?v=uur5MmjTe5I?t=124) <!-- atomicity: 1 idea ✓ --> #### [H3] Two blind reviewers keep screening honest > [!quote] ID:rw_uur5MmjTe5I_h3 > "Neither sees each other's calls, but we resolve disagreements by discussion or a third reviewer." **Context & use**: - [00:05:10](https://www.youtube.com/watch?v=uur5MmjTe5I?t=310) blind screening and Cohen's kappa. - Independent verification beats solo judgment — the maker/checker pattern applied to literature triage. **Tags**: #xref **Highlight**: [view highlight](https://www.youtube.com/watch?v=uur5MmjTe5I?t=310) <!-- atomicity: 1 idea ✓ --> ### 🧠 Permanent Candidates (H4) #### [P1] Pre-commitment is what separates synthesis from cherry-picking **Claim** (one sentence, Courtney's voice): Registering a question and method before you look at the evidence is what converts a literature scan into a trustworthy synthesis. **Evidence pointer**: [[#H1]] **Scope & conditions**: Holds for evidence-synthesis and decision research; breaks for exploratory or hypothesis-generating work where discovery is the goal. **Links**: - [[Maker-checker discipline]] — because extends - [[Confirmation bias]] — because contrasts on intent - [[Deep-read pipeline]] — because applies-in research note workflows <!-- atomicity: 1 idea ✓ --> ### ✏️ Fleeting Queue - [F1] Confirm tool names: "Robis"→ROBIS/ROBVIS, "Ryan"→Rayyan — verify before citing — https://www.youtube.com/watch?v=uur5MmjTe5I #fleeting #inbox - [F2] Does the deep-read/synthesis pass need an explicit "register the question" step like PROSPERO? #fleeting #inbox ### → Things 3 (max 2 actions) - [ ] Draft a one-page PICO→Boolean→PRISMA cheat sheet for the vault — research #things - [ ] Add a "second-reviewer" check to the synthesis-pass maker/checker loop — vault ops #things <!-- deep-read:sections end --> 📄 **Full text:** [[Full Document Contents/Articles/Full Systematic Literature Review PRISMA & PICO in 5 Steps.md|Full Systematic Literature Review: PRISMA & PICO in 5 Steps by youtube.com]] ## Highlights - and register it before screening. This tells people that it's not retrospective, but in fact you had a plan going in, and we'll talk about how ([View Highlight](https://read.readwise.io/read/01kvt8c9t3f2wendmvr7mc3her)) - Note: Registration proves a review wasn't retrospective. - Tags: atom/literature/h1 - a full systematic review, by the way, takes about 11 to 16 months. So, this is ([View Highlight](https://read.readwise.io/read/01kvt8cej0m0tv8fdxfaxmg5h1)) - Note: Systematic reviews are a months-long effort. - Tags: atom/literature/h3 - idea into question. So, PICO stands for population, intervention, comparison, and outcome. And don't worry, there's ([View Highlight](https://read.readwise.io/read/01kvt8cc3bgznbaqf83rp0fpgq)) - Note: PICO frames a searchable clinical question. - Tags: atom/literature/h2