How to catch a bad scientist  ## Metadata - Author: [[YouTube]] - Full Title: How to catch a bad scientist - Category: #articles - Document Note: --- tags: [research-integrity, scientific-misconduct, academic-fraud] author: The Sci-Fi source: YouTube reference: https://www.youtube.com/watch?v=QTZ5_TYzk7A date created: 2024-12-19 10:30 --- # 🏷️ Title: How to catch a bad scientist ## 🧰 Action Inbox - [H1] Monitor publication rates >10 papers/year for suspicious activity — 00:45 #research-integrity - [H2] Implement image duplication software across journals — 08:30 #publication-standards - [H3] Mandate raw data publication with all papers — 10:45 #data-transparency ## 🧭 Table of Contents - [📝 Summary](#-summary) - [📖 Main Content](#-main-content) - [🗒️ Notes](#️-notes) - [📚 References](#-references) - [🗣️ Paraphrase](#️-paraphrase) - [📰 Newsletter](#-newsletter) - [✅ Action Items](#-action-items) - [🧪 Hypotheses](#-hypotheses) - [📊 Methodology](#-methodology) - [❓ Questions](#-questions) - [🔮 Future Research](#-future-research) - [💡 Implications](#-implications) - [🧠 Learning Styles Assessment](#-learning-styles-assessment) - [📊 Accessibility Assessment](#-accessibility-assessment) - [🌍 DEIB Assessment](#-deib-assessment) - [✅ Logical Validity](#-logical-validity) - [🛡️ DARVO Analysis](#-darvo-analysis) - [🎯 Conclusion](#-conclusion) ### 🔗 References - [How to catch a bad scientist](https://www.youtube.com/watch?v=QTZ5_TYzk7A) ## 📝 Summary - Five methods to identify scientific misconduct, from basic red flags to sophisticated analysis - Publication rates >10 papers/year are suspicious, with some researchers publishing 2+ papers daily - Conflicts of interest, consistently surprising results, image manipulation, and data anomalies are key warning signs - Current detection infrastructure is inadequate, allowing many bad actors to escape consequences - Systematic implementation of detection tools and transparency requirements needed across academia ## 📖 Main Content This video synthesizes patterns from historical and recent cases of scientific misconduct to present five escalating methods for identifying problematic researchers. The approach ranges from obvious red flags like impossible publication rates to sophisticated data forensics requiring specialized expertise. The most accessible method involves monitoring publication volumes, where surveys indicate 10+ papers annually raises suspicions. However, some researchers are publishing over 200 papers in six months (1.8+ papers per working day), which is mathematically impossible for legitimate research. More nuanced indicators include undeclared conflicts of interest, consistently surprising results that defy statistical probability, image manipulation detectable through software analysis, and data anomalies requiring expert investigation. The video emphasizes that current detection infrastructure is insufficient to catch most offenders. ## 🗒️ Notes #### ⏱️ [00:00](https://www.youtube.com/watch?v=QTZ5_TYzk7A?t=0) — [H1] Introduction to catching bad scientists - Key idea: Synthesizing knowledge from multiple cases of scientific misconduct - Evidence: "Five ways to catch a bad scientist going from the easiest and most obvious red flags through to the more labor intensive" #### ⏱️ [00:45](https://www.youtube.com/watch?v=QTZ5_TYzk7A?t=45) — [H2] Publication rate as red flag indicator - Key idea: Excessive publication rates indicate impossible research productivity - Evidence: "10 plus Publications a year is already suspicious" based on follower surveys #### ⏱️ [01:30](https://www.youtube.com/watch?v=QTZ5_TYzk7A?t=90) — [H3] Extreme publication examples from 2024 - Key idea: Some researchers published over 200 papers in six months - Evidence: "Over 22 people with over 200 papers published already this year that works out to be over one paper per day" #### ⏱️ [02:15](https://www.youtube.com/watch?v=QTZ5_TYzk7A?t=135) — [H2] Mathematical impossibility of high publication rates - Key idea: Top publishers average 1.8+ papers per working day - Evidence: "These people are publishing around 1.8 papers per working day with this top person publishing over two papers per working day" #### ⏱️ [03:00](https://www.youtube.com/watch?v=QTZ5_TYzk7A?t=180) — [H2] Conflicts of interest as misconduct indicator - Key idea: Financial or professional conflicts often correlate with poor quality work - Evidence: Examples from PACE trial and Wakefield autism study with undisclosed patent conflicts #### ⏱️ [04:15](https://www.youtube.com/watch?v=QTZ5_TYzk7A?t=255) — [H3] Inadequate conflict disclosure systems - Key idea: Current checks and balances insufficient to catch undeclared conflicts - Evidence: "People can simply just not declare the conflict of interest and a lot of the time they get away with it" #### ⏱️ [04:45](https://www.youtube.com/watch?v=QTZ5_TYzk7A?t=285) — [H2] Consistently surprising results pattern - Key idea: Researchers producing repeatedly counterintuitive findings deserve scrutiny - Evidence: "Scientists and academics are incentivized to produce surprising results" for high-impact publications #### ⏱️ [06:00](https://www.youtube.com/watch?v=QTZ5_TYzk7A?t=360) — [H3] Jan Hendrik Schön case study - Key idea: Perfect theoretical results across multiple physics domains raised suspicions - Evidence: "Theoretically perfect in terms of the output... across many different domains of physics" #### ⏱️ [07:00](https://www.youtube.com/watch?v=QTZ5_TYzk7A?t=420) — [H2] Image manipulation in biological sciences - Key idea: Elizabeth Bik's work reveals widespread image manipulation - Evidence: "Western blot where people are trying to show their blots to be looking better than they really are" #### ⏱️ [08:30](https://www.youtube.com/watch?v=QTZ5_TYzk7A?t=510) — [H3] Software tools for image detection - Key idea: Image Twin and similar tools can detect duplications within and across papers - Evidence: "Two leaves are never identical... the patterns that you see there might be similar but they're not supposed to be identical" #### ⏱️ [09:15](https://www.youtube.com/watch?v=QTZ5_TYzk7A?t=555) — [H3] AI-generated content challenge - Key idea: AI image generators will make detection more difficult going forward - Evidence: "People won't only be duplicating images but generating totally new images that are fake" #### ⏱️ [10:00](https://www.youtube.com/watch?v=QTZ5_TYzk7A?t=600) — [H2] Data forensics investigation method - Key idea: Most labor-intensive but thorough approach to catching misconduct - Evidence: "Data colada has done against Franchesca Gino... going into a scientist data set analyzing the data" #### ⏱️ [10:45](https://www.youtube.com/watch?v=QTZ5_TYzk7A?t=645) — [H3] Data availability problem - Key idea: Most papers don't publish raw datasets despite claiming availability - Evidence: "Data available upon request which is the biggest lie in all of science" #### ⏱️ [11:30](https://www.youtube.com/watch?v=QTZ5_TYzk7A?t=690) — [H1] Call for systematic transparency - Key idea: All established journals should mandate raw data publication - Evidence: "Most important step that is needed to establish more transparency between the scientists and the rest of the scientific Community" ## 📚 References - [The Sci-Fi YouTube Channel](https://www.youtube.com/@TheSci-Fi) - [Publishing with Integrity X Account](https://x.com/PublishingInteg) - [Elizabeth Bick's Image Forensics Work](https://scienceintegritydigest.com/) - [Data Colada Research Integrity Blog](http://datacolada.org/) ## 🗣️ Paraphrase - Researchers publishing more than 10 papers annually should trigger automatic investigation, as this rate often indicates misconduct rather than exceptional productivity - Undisclosed financial or professional conflicts of interest frequently correlate with biased or fabricated research results, but current disclosure systems are inadequate - Scientists who consistently produce surprising or theoretically perfect results across multiple studies likely deserve scrutiny, as such patterns defy statistical probability ## 📰 Newsletter **Spotting Scientific Misconduct: Five Red Flags Every Researcher Should Know** Recent analysis reveals systematic patterns that can help identify problematic research before it spreads. From impossible publication rates (some researchers publishing 2+ papers daily!) to sophisticated data manipulation, these detection methods range from obvious red flags to specialized forensics. The most accessible warning sign? Publication volume. Community surveys suggest 10+ papers annually raises suspicions, yet current systems lack infrastructure to investigate systematically. Meanwhile, image manipulation software like Image Twin can detect duplicated blots and microscopy images that should be unique by nature. **Key takeaway:** The scientific community needs better detection tools and transparency requirements. Mandating raw data publication with every paper would be a crucial first step toward accountability. Resources: [Research Integrity Resources](https://ori.hhs.gov/), [Image Forensics Tools](https://imagetwin.com/), [Data Sharing Best Practices](https://www.nature.com/sdata/) ## ✅ Action Items #### DevRel - Developer advocacy: Create tools for automated publication rate monitoring across academic databases - Education: Develop training materials on research integrity for developer-researchers - Developer experience: Build APIs for accessing and analyzing publication metadata - Developer marketing: Promote open-source research integrity tools - Community management: Foster discussions about academic transparency in tech communities #### OSPO - Policy & Governance: Establish research data sharing policies for organization publications - License Compliance: Review academic software licensing for integrity tools - Security Management: Assess vulnerabilities in research data repositories - Software Inventory: Catalog research integrity and analysis tools - Developer Support: Train researchers on version control and reproducible research - Community Engagement: Support open-source research integrity initiatives - Strategic Alignment: Align research transparency with organizational values #### Community - Maintainers: Support integrity checking tools in scientific software projects - Contributors: Encourage participation in research transparency initiatives - Sponsors: Fund development of automated misconduct detection systems - Ecosystem: Coordinate standards for research data sharing and validation ## 🧪 Hypotheses - Automated publication rate monitoring would identify 80% of high-volume misconduct cases within existing academic databases - Mandatory raw data publication would reduce fabrication incidents by 40% within five years of implementation - AI-powered image analysis tools could detect 90% of current image manipulation techniques in biological sciences - Cross-institutional collaboration on research integrity would improve detection rates by 60% compared to isolated efforts ### 📊 Methodology - Longitudinal analysis of publication rates across disciplines to establish baseline thresholds for investigation triggers - Controlled studies comparing research quality before and after mandatory data sharing implementation - Machine learning training on known manipulation cases to develop automated detection algorithms ### ❓ Questions - How can academic institutions balance productivity incentives with research integrity requirements? - What technological solutions could scale research integrity monitoring to match publication volume growth? - How might blockchain or similar technologies ensure immutable research data provenance? ### 🔮 Future Research - Development of AI systems capable of detecting sophisticated data fabrication beyond simple duplication - Cross-cultural analysis of research integrity practices and misconduct patterns globally - Economic impact assessment of research misconduct on scientific progress and public trust ### 💡 Implications - Practice: Researchers need training in integrity tools and transparent practices - Policy: Academic institutions require systematic monitoring and investigation protocols - Security: Research databases need protection against manipulation and unauthorized access - Education: Graduate programs should include research integrity and detection methods - Funding: Grant agencies should mandate transparency tools and data sharing infrastructure ### 🧠 Learning Styles Assessment Reading level estimate: 9th Grade (Flesch-Kincaid) #### 👀 Visual learners - Present: Publication rate statistics, image manipulation examples - Improve: Flowcharts for detection processes, infographics of misconduct patterns #### 🔊 Aural learners - Present: Video format with clear narration available - Improve: Podcast version with extended discussion of case studies #### 🗣️ Verbal learners - Present: Clear definitions, analogies like "Parks and Rec" meme references - Improve: Glossary of research integrity terms, comparative explanations #### 🧩 Kinesthetic learners - Present: Step-by-step detection methods described - Improve: Interactive detection exercises, hands-on data analysis workshops #### 🧠 Logical learners - Present: Mathematical analysis of publication rates, systematic categorization - Improve: Decision trees for investigation priorities, quantitative thresholds #### 🧑‍🤝‍🧑 Social learners - Present: Community survey results, collaborative detection efforts - Improve: Group exercises in misconduct detection, peer review simulations #### 🧘‍♀️ Solitary learners - Present: Individual analysis methods, self-directed learning approach - Improve: Personal checklists for evaluating research integrity, reflection prompts #### 🌿 Naturalistic learners - Present: Biological examples of image manipulation, natural uniqueness analogies - Improve: Ecosystem metaphors for research communities, environmental impacts of misconduct ### 📊 Accessibility Assessment #### Screen reader semantics - Present: Clear headings, timestamps, descriptive content structure - Improve: Alt text for statistical tables, ARIA labels for interactive elements #### Keyboard navigation - Present: Video player supports keyboard controls - Improve: Skip links for long content sections, focus indicators #### Contrast and readability - Present: High contrast video text, clear narration - Improve: Transcript formatting with better visual hierarchy #### Text resizing and reflow - Present: Responsive video player, scalable text - Improve: Ensure 200% zoom accessibility for all interface elements #### Media and captions - Present: Audio narration available - Improve: Closed captions, downloadable transcript with timestamps #### Forms and interactions - Present: Simple video interface - Improve: Accessible controls for playback speed and quality #### Standards reference - Video player should meet WCAG 2.2 AA media accessibility guidelines ### 🌍 DEIB Assessment #### Cultural breadth - Present: Global perspective on research integrity issues - Improve: Examples from diverse academic institutions and countries #### Inclusive language - Present: Gender-neutral terms, accessible explanations - Improve: Plain language alternatives for technical terms #### Neurodiversity support - Present: Clear structure, consistent pacing - Improve: Content warnings for detailed misconduct discussions #### Representation - Present: Diverse range of misconduct cases discussed - Improve: Highlight integrity efforts by underrepresented researchers #### Psychological safety - Present: Focus on systemic issues rather than individual blame - Improve: Emphasize learning and improvement over punishment ### ✅ Logical Validity #### Strong reasoning - Deductive strength: Statistical analysis of publication rates provides solid foundation (01:30) - Inductive strength: Multiple case studies support pattern recognition (throughout) - Causal reasoning: Clear links between incentives and misconduct behaviors (04:45) - Analogical strength: Biological uniqueness analogy effectively explains image manipulation detection (08:30) #### Fallacies - Hasty generalization: Assumes all high-volume publishers are fraudulent without considering legitimate explanations like review articles or meta-analyses - False dichotomy: Presents detection vs. non-detection without discussing prevention strategies #### Tone of voice - Overall tone: Conversational and accessible while maintaining authority through specific examples and data ### 🛡️ DARVO Analysis - Deny: No evidence of denial tactics in content - Attack: No attacks on accusers present - Reverse victim/offender: No victim/offender role reversals identified - Emotional cues and rhetoric: Uses humor ("Parks and Rec" reference) and frustration ("biggest lie in science") appropriately to engage audience without manipulation ## 🎯 Conclusion The scientific community needs systematic implementation of these five detection methods, from automated publication monitoring to sophisticated data - Document Tags: [[data]] [[research]] [[publication]] - Summary: How to catch a bad scientist - URL: https://www.youtube.com/watch?v=QTZ5_TYzk7A over the last year on this channel we have discussed a lot of bad scientists some of them historical some of them more recent and so by covering all these different stories I have learned about many different ways to catch a bad scientist I thought I would synthesize all of that knowledge into this video today so in today's video I want to share with you five ways to catch a bad scientist going from the easiest and most obvious red flags through to the more labor intensive and tough stuff so let's start off off with the easiest one of them all which is publication rate so when I talk about publication rate I'm talking about how many papers can an academic publish in a given year and the reason why I was thinking about this is because this week I was tagged in a LinkedIn post by this guy called Teemo Mandela who surveyed his followers to see how many papers a scientist can publish in a year before it gets suspicious and the majority of his followers felt that just 10 plus Publications a year is already suspicious now I replicated that poll with you guys on this channel and you guys also agreed that in general 10 plus Publications a year is suspicious now I know a lot of you will be thinking well surely it depends on the field you know different fields have different rates of publication which is true you're absolutely right but just take a look at this table here this is an image posted by the X account publishing with Integrity which according to them at the time of filming 6 months into 2024 there have been over 22 people with over two 200 papers published already this year that works out to be over one paper per day and if you exclude weekends so only include working days that means that these people are publishing around 1.8 papers per working day with this top person publishing over two papers per working day now I don't care what field you're from but that rate of publication is absolutely impossible in my mind the people who are publishing papers at this rate it's like that Meme from Parks and wreck you know right to jail right to jail no trial right to jail right to jail right away no trial no no nothing and those are just the people publishing over 200 papers most of you guys felt that just 10 papers a year is already suspicious so all authors all academics who are publishing more than 10 papers a year should immediately be put under some kind of Investigation in my opinion but there just isn't the Manpower or the infrastructure of people doing this kind of work unfortunately so a lot of these people who are publishing an impossible amount of work are just going to get away with it scottf free okay the second easiest way to spot a bad scientist is to look for conflicts of interest in many of the doy studies that we've analyzed on this channel the authors had egregious conflicts of interest take for example the pace trial that we looked at a few weeks ago where the author's study was funded by the department of work and pensions which clearly had a conflict of interest in not wanting to give patients of me disability benefits which is exactly what the results of this study would impact or as another example look at that original autism vaccine study where Andrew Wakefield the author of that study had a direct conflict of interest because he held a patent for a rival vaccine a lot of the time conflicts of interest result in poor quality work and you can't always rely on the declared conflicts of interest that are published on people's papers from what I've seen there aren't enough checks and balances in that process to make sure that any true conflicts of interest are actually recorded on the published manuscript people can simply just not declare the conflict of interest and a lot of the time they get away with it okay but if they're not producing a ridiculous amount of papers and they have no declared conflicts of interest then how do you know who to look at well that brings us on to our third way of identifying a dodgy scientist which is to look at consistently surprising results scientists and academics are incentivized to produce surprising results from their research surprising and interesting results are a lot more likely to be published in high impact journals like nature or science if your paper gets published in a high impact Journal like that then you're also more likely to get a lot of press coverage press coverage might lead to more expectations on your work or just more work in general coming to you because of this research and so there's a lot of Fame and money to be gained from producing surprising results but the thing that makes results surprising is that they're rare and they're counterintuitive and occasionally these things do happen in science right by Nature they have to but if there's a researcher who somehow is able to consistently produce surprising results year after year paper after paper then you have to start looking at their portfolio and thinking is there something more dodgy going on here by its nature nobody should be able to consistently find surprising results from their work we saw an example of this from Yan Henrik sh one of the most notorious academic fraudsters in physics he was able to produce supposed lab results that were theoretically perfect in terms of the output that he was getting in the measures that he was getting which is obviously really surprising especially because so many people were trying to do the same thing for many years and couldn't and yet he was able to do it not only in one domain but across many different domains of physics when the scientists or academics results are too perfect and consistently so it's quite likely the case that it's too good to be true okay so number four is a little bit harder and that is image manipulation we particularly see this kind of thing happening in biological science the work of Elizabeth Bick which we've talked about extensively on this channel shows how common this practice is image manipulation happens with techniques like Western blot where people are trying to show their blots to be you know looking better than they really are or they use image manipulation when looking at like cross-sections of cells or cells under the microscope things like that the easiest way to catch these people is through software so Elizabeth Bick talked to me about different software like image twin and proof to to help me find these duplications and image twin has a database of images so you can sometimes find duplications of an image within one paper with another paper so everything in nature more or less uh you know making a generalization here is unique two leaves are never identical two rocks are always slightly different and so if you think about cells or tissues or or Western blots the the patterns that you see there might be similar but they're not supposed to be identical now of course with the new emergence of AI based image generators this is going to become a much harder challenge going forward because people won't only be duplicating images but generating totally new images that are fake as a first step I think that the duplication image software should just be standard practice across any Journal that has images as part of its publication but I would love to see new tools and software developed that can counter AI generated content and the fifth and final way that I want to talk to you about catching a bad scientist is through thorough investigation of somebody's research data this is by far the most labor intensive and most specialist way of catching somebody out when they're doing bad research this is the analysis that data colada has done against Franchesca Gino as well as a number of other authors that they've talked about on their block the way this is done is by going into a scientist data set analyzing the data in the way that they claim to have done in their paper and seeing if you can produce the same results also by looking at their data set you can look for anomalies in the way that the data is being processed or if certain numbers look out of place or unreasonable but it certainly does take someone's time and effort in order to find these things because you do need quite good specific knowledge in order to be able to counter this problem now I don't see why a software solution can't be developed that can at least do a first sweep of many different data sets to see if certain things look out of place and should be flagged that can be done in an automated way or at least a much faster way than someone mannually going through someone's data set that certainly seems like something which is possible now of course the other problem with this is that a lot of the time when papers are published the original data set doesn't get published along with it you'll often see papers published with this silly comment at the bottom saying data available upon request which is the biggest lie in all of science the data is not available if that's what it says in my opinion all journals or at least all the well-known well established journals should have a standard practice of every paper that they publish to have the original data set published along with it that is the first and most important step that is needed to in order to establish more transparency between the scientists and the rest of the scientific Community those are five ways to catch a bad scientist do you agree or disagree with me on any of these points are there any ways of catching a bad scientist that I missed in this video if so let me know in the comments below thank you guys so much for watching this channel subscribe if you haven't already we're almost at 100,000 subscribers and I would love to get there and I'll see you guys in the next one bye-bye [Applause], How to catch a bad scientist, YouTube, 01ka8k7we5a1b3k50416184c1v.