How to catch a bad scientist 🔗 **Source:** [How to catch a bad scientist](https://www.youtube.com/watch?v=QTZ5_TYzk7A) > [!note]+ Document Notes > tags: [research-integrity, scientific-misconduct, academic-fraud] > author: The Sci-Fi > source: YouTube > reference: https://www.youtube.com/watch?v=QTZ5_TYzk7A > date created: 2024-12-19 10:30 > > # 🏷️ Title: How to catch a bad scientist > <!-- deep-read:sections start --> ## 🧰 Action Inbox - [H1] Monitor publication rates >10 papers/year for suspicious activity — 00:45 #research-integrity - [H2] Implement image duplication software across journals — 08:30 #publication-standards - [H3] Mandate raw data publication with all papers — 10:45 #data-transparency ## 🧭 Table of Contents - [📝 Summary](#-summary) - [📖 Main Content](#-main-content) - [🗒️ Notes](#️-notes) - [📚 References](#-references) - [🗣️ Paraphrase](#️-paraphrase) - [📰 Newsletter](#-newsletter) - [✅ Action Items](#-action-items) - [🧪 Hypotheses](#-hypotheses) - [📊 Methodology](#-methodology) - [❓ Questions](#-questions) - [🔮 Future Research](#-future-research) - [💡 Implications](#-implications) - [🧠 Learning Styles Assessment](#-learning-styles-assessment) - [📊 Accessibility Assessment](#-accessibility-assessment) - [🌍 DEIB Assessment](#-deib-assessment) - [✅ Logical Validity](#-logical-validity) - [🛡️ DARVO Analysis](#-darvo-analysis) - [🎯 Conclusion](#-conclusion) ### 🔗 References - [How to catch a bad scientist](https://www.youtube.com/watch?v=QTZ5_TYzk7A) ## 📝 Summary - Five methods to identify scientific misconduct, from basic red flags to sophisticated analysis - Publication rates >10 papers/year are suspicious, with some researchers publishing 2+ papers daily - Conflicts of interest, consistently surprising results, image manipulation, and data anomalies are key warning signs - Current detection infrastructure is inadequate, allowing many bad actors to escape consequences - Systematic implementation of detection tools and transparency requirements needed across academia ## 📖 Main Content This video synthesizes patterns from historical and recent cases of scientific misconduct to present five escalating methods for identifying problematic researchers. The approach ranges from obvious red flags like impossible publication rates to sophisticated data forensics requiring specialized expertise. The most accessible method involves monitoring publication volumes, where surveys indicate 10+ papers annually raises suspicions. However, some researchers are publishing over 200 papers in six months (1.8+ papers per working day), which is mathematically impossible for legitimate research. More nuanced indicators include undeclared conflicts of interest, consistently surprising results that defy statistical probability, image manipulation detectable through software analysis, and data anomalies requiring expert investigation. The video emphasizes that current detection infrastructure is insufficient to catch most offenders. ## 🗒️ Notes #### ⏱️ [00:00](https://www.youtube.com/watch?v=QTZ5_TYzk7A?t=0) — [H1] Introduction to catching bad scientists - Key idea: Synthesizing knowledge from multiple cases of scientific misconduct - Evidence: "Five ways to catch a bad scientist going from the easiest and most obvious red flags through to the more labor intensive" #### ⏱️ [00:45](https://www.youtube.com/watch?v=QTZ5_TYzk7A?t=45) — [H2] Publication rate as red flag indicator - Key idea: Excessive publication rates indicate impossible research productivity - Evidence: "10 plus Publications a year is already suspicious" based on follower surveys #### ⏱️ [01:30](https://www.youtube.com/watch?v=QTZ5_TYzk7A?t=90) — [H3] Extreme publication examples from 2024 - Key idea: Some researchers published over 200 papers in six months - Evidence: "Over 22 people with over 200 papers published already this year that works out to be over one paper per day" #### ⏱️ [02:15](https://www.youtube.com/watch?v=QTZ5_TYzk7A?t=135) — [H2] Mathematical impossibility of high publication rates - Key idea: Top publishers average 1.8+ papers per working day - Evidence: "These people are publishing around 1.8 papers per working day with this top person publishing over two papers per working day" #### ⏱️ [03:00](https://www.youtube.com/watch?v=QTZ5_TYzk7A?t=180) — [H2] Conflicts of interest as misconduct indicator - Key idea: Financial or professional conflicts often correlate with poor quality work - Evidence: Examples from PACE trial and Wakefield autism study with undisclosed patent conflicts #### ⏱️ [04:15](https://www.youtube.com/watch?v=QTZ5_TYzk7A?t=255) — [H3] Inadequate conflict disclosure systems - Key idea: Current checks and balances insufficient to catch undeclared conflicts - Evidence: "People can simply just not declare the conflict of interest and a lot of the time they get away with it" #### ⏱️ [04:45](https://www.youtube.com/watch?v=QTZ5_TYzk7A?t=285) — [H2] Consistently surprising results pattern - Key idea: Researchers producing repeatedly counterintuitive findings deserve scrutiny - Evidence: "Scientists and academics are incentivized to produce surprising results" for high-impact publications #### ⏱️ [06:00](https://www.youtube.com/watch?v=QTZ5_TYzk7A?t=360) — [H3] Jan Hendrik Schön case study - Key idea: Perfect theoretical results across multiple physics domains raised suspicions - Evidence: "Theoretically perfect in terms of the output... across many different domains of physics" #### ⏱️ [07:00](https://www.youtube.com/watch?v=QTZ5_TYzk7A?t=420) — [H2] Image manipulation in biological sciences - Key idea: Elizabeth Bik's work reveals widespread image manipulation - Evidence: "Western blot where people are trying to show their blots to be looking better than they really are" #### ⏱️ [08:30](https://www.youtube.com/watch?v=QTZ5_TYzk7A?t=510) — [H3] Software tools for image detection - Key idea: Image Twin and similar tools can detect duplications within and across papers - Evidence: "Two leaves are never identical... the patterns that you see there might be similar but they're not supposed to be identical" #### ⏱️ [09:15](https://www.youtube.com/watch?v=QTZ5_TYzk7A?t=555) — [H3] AI-generated content challenge - Key idea: AI image generators will make detection more difficult going forward - Evidence: "People won't only be duplicating images but generating totally new images that are fake" #### ⏱️ [10:00](https://www.youtube.com/watch?v=QTZ5_TYzk7A?t=600) — [H2] Data forensics investigation method - Key idea: Most labor-intensive but thorough approach to catching misconduct - Evidence: "Data colada has done against Franchesca Gino... going into a scientist data set analyzing the data" #### ⏱️ [10:45](https://www.youtube.com/watch?v=QTZ5_TYzk7A?t=645) — [H3] Data availability problem - Key idea: Most papers don't publish raw datasets despite claiming availability - Evidence: "Data available upon request which is the biggest lie in all of science" #### ⏱️ [11:30](https://www.youtube.com/watch?v=QTZ5_TYzk7A?t=690) — [H1] Call for systematic transparency - Key idea: All established journals should mandate raw data publication - Evidence: "Most important step that is needed to establish more transparency between the scientists and the rest of the scientific Community" ## 📚 References - [The Sci-Fi YouTube Channel](https://www.youtube.com/@TheSci-Fi) - [Publishing with Integrity X Account](https://x.com/PublishingInteg) - [Elizabeth Bick's Image Forensics Work](https://scienceintegritydigest.com/) - [Data Colada Research Integrity Blog](http://datacolada.org/) ## 🗣️ Paraphrase - Researchers publishing more than 10 papers annually should trigger automatic investigation, as this rate often indicates misconduct rather than exceptional productivity - Undisclosed financial or professional conflicts of interest frequently correlate with biased or fabricated research results, but current disclosure systems are inadequate - Scientists who consistently produce surprising or theoretically perfect results across multiple studies likely deserve scrutiny, as such patterns defy statistical probability ## 📰 Newsletter **Spotting Scientific Misconduct: Five Red Flags Every Researcher Should Know** Recent analysis reveals systematic patterns that can help identify problematic research before it spreads. From impossible publication rates (some researchers publishing 2+ papers daily!) to sophisticated data manipulation, these detection methods range from obvious red flags to specialized forensics. The most accessible warning sign? Publication volume. Community surveys suggest 10+ papers annually raises suspicions, yet current systems lack infrastructure to investigate systematically. Meanwhile, image manipulation software like Image Twin can detect duplicated blots and microscopy images that should be unique by nature. **Key takeaway:** The scientific community needs better detection tools and transparency requirements. Mandating raw data publication with every paper would be a crucial first step toward accountability. Resources: [Research Integrity Resources](https://ori.hhs.gov/), [Image Forensics Tools](https://imagetwin.com/), [Data Sharing Best Practices](https://www.nature.com/sdata/) ## ✅ Action Items #### DevRel - Developer advocacy: Create tools for automated publication rate monitoring across academic databases - Education: Develop training materials on research integrity for developer-researchers - Developer experience: Build APIs for accessing and analyzing publication metadata - Developer marketing: Promote open-source research integrity tools - Community management: Foster discussions about academic transparency in tech communities #### OSPO - Policy & Governance: Establish research data sharing policies for organization publications - License Compliance: Review academic software licensing for integrity tools - Security Management: Assess vulnerabilities in research data repositories - Software Inventory: Catalog research integrity and analysis tools - Developer Support: Train researchers on version control and reproducible research - Community Engagement: Support open-source research integrity initiatives - Strategic Alignment: Align research transparency with organizational values #### Community - Maintainers: Support integrity checking tools in scientific software projects - Contributors: Encourage participation in research transparency initiatives - Sponsors: Fund development of automated misconduct detection systems - Ecosystem: Coordinate standards for research data sharing and validation ## 🧪 Hypotheses - Automated publication rate monitoring would identify 80% of high-volume misconduct cases within existing academic databases - Mandatory raw data publication would reduce fabrication incidents by 40% within five years of implementation - AI-powered image analysis tools could detect 90% of current image manipulation techniques in biological sciences - Cross-institutional collaboration on research integrity would improve detection rates by 60% compared to isolated efforts ### 📊 Methodology - Longitudinal analysis of publication rates across disciplines to establish baseline thresholds for investigation triggers - Controlled studies comparing research quality before and after mandatory data sharing implementation - Machine learning training on known manipulation cases to develop automated detection algorithms ### ❓ Questions - How can academic institutions balance productivity incentives with research integrity requirements? - What technological solutions could scale research integrity monitoring to match publication volume growth? - How might blockchain or similar technologies ensure immutable research data provenance? ### 🔮 Future Research - Development of AI systems capable of detecting sophisticated data fabrication beyond simple duplication - Cross-cultural analysis of research integrity practices and misconduct patterns globally - Economic impact assessment of research misconduct on scientific progress and public trust ### 💡 Implications - Practice: Researchers need training in integrity tools and transparent practices - Policy: Academic institutions require systematic monitoring and investigation protocols - Security: Research databases need protection against manipulation and unauthorized access - Education: Graduate programs should include research integrity and detection methods - Funding: Grant agencies should mandate transparency tools and data sharing infrastructure ### 🧠 Learning Styles Assessment Reading level estimate: 9th Grade (Flesch-Kincaid) #### 👀 Visual learners - Present: Publication rate statistics, image manipulation examples - Improve: Flowcharts for detection processes, infographics of misconduct patterns #### 🔊 Aural learners - Present: Video format with clear narration available - Improve: Podcast version with extended discussion of case studies #### 🗣️ Verbal learners - Present: Clear definitions, analogies like "Parks and Rec" meme references - Improve: Glossary of research integrity terms, comparative explanations #### 🧩 Kinesthetic learners - Present: Step-by-step detection methods described - Improve: Interactive detection exercises, hands-on data analysis workshops #### 🧠 Logical learners - Present: Mathematical analysis of publication rates, systematic categorization - Improve: Decision trees for investigation priorities, quantitative thresholds #### 🧑‍🤝‍🧑 Social learners - Present: Community survey results, collaborative detection efforts - Improve: Group exercises in misconduct detection, peer review simulations #### 🧘‍♀️ Solitary learners - Present: Individual analysis methods, self-directed learning approach - Improve: Personal checklists for evaluating research integrity, reflection prompts #### 🌿 Naturalistic learners - Present: Biological examples of image manipulation, natural uniqueness analogies - Improve: Ecosystem metaphors for research communities, environmental impacts of misconduct ### 📊 Accessibility Assessment #### Screen reader semantics - Present: Clear headings, timestamps, descriptive content structure - Improve: Alt text for statistical tables, ARIA labels for interactive elements #### Keyboard navigation - Present: Video player supports keyboard controls - Improve: Skip links for long content sections, focus indicators #### Contrast and readability - Present: High contrast video text, clear narration - Improve: Transcript formatting with better visual hierarchy #### Text resizing and reflow - Present: Responsive video player, scalable text - Improve: Ensure 200% zoom accessibility for all interface elements #### Media and captions - Present: Audio narration available - Improve: Closed captions, downloadable transcript with timestamps #### Forms and interactions - Present: Simple video interface - Improve: Accessible controls for playback speed and quality #### Standards reference - Video player should meet WCAG 2.2 AA media accessibility guidelines ### 🌍 DEIB Assessment #### Cultural breadth - Present: Global perspective on research integrity issues - Improve: Examples from diverse academic institutions and countries #### Inclusive language - Present: Gender-neutral terms, accessible explanations - Improve: Plain language alternatives for technical terms #### Neurodiversity support - Present: Clear structure, consistent pacing - Improve: Content warnings for detailed misconduct discussions #### Representation - Present: Diverse range of misconduct cases discussed - Improve: Highlight integrity efforts by underrepresented researchers #### Psychological safety - Present: Focus on systemic issues rather than individual blame - Improve: Emphasize learning and improvement over punishment ### ✅ Logical Validity #### Strong reasoning - Deductive strength: Statistical analysis of publication rates provides solid foundation (01:30) - Inductive strength: Multiple case studies support pattern recognition (throughout) - Causal reasoning: Clear links between incentives and misconduct behaviors (04:45) - Analogical strength: Biological uniqueness analogy effectively explains image manipulation detection (08:30) #### Fallacies - Hasty generalization: Assumes all high-volume publishers are fraudulent without considering legitimate explanations like review articles or meta-analyses - False dichotomy: Presents detection vs. non-detection without discussing prevention strategies #### Tone of voice - Overall tone: Conversational and accessible while maintaining authority through specific examples and data ### 🛡️ DARVO Analysis - Deny: No evidence of denial tactics in content - Attack: No attacks on accusers present - Reverse victim/offender: No victim/offender role reversals identified - Emotional cues and rhetoric: Uses humor ("Parks and Rec" reference) and frustration ("biggest lie in science") appropriately to engage audience without manipulation ## 🎯 Conclusion The scientific community needs systematic implementation of these five detection methods, from automated publication monitoring to sophisticated data <!-- deep-read:sections end -->