How to Edit Videos Properly With Claude and Remotion

How to Edit Videos Properly With Claude and Remotion ![rw-book-cover](https://img.youtube.com/vi/bDCLT4Je6WA/maxresdefault.jpg) 🔗 **Source:** [How to Edit Videos Properly With Claude and Remotion](https://www.youtube.com/watch?v=bDCLT4Je6WA) > [!note]+ Document Notes > tags: [ai-automation, video-editing, production-workflows] > author: [MISSING] > source: YouTube > reference: https://www.youtube.com/watch?v=bDCLT4Je6WA > date created: {{date:YYYY-MM-DD}} {{time:HH:mm}} > > # 🏷️ Title: How to Edit Videos Properly With Claude and Remotion (This Actually Works) >  ## 🧭 Table of Contents - [📝 Summary](#-summary) - [📖 Main Content](#-main-content) - [🗒️ Notes](#️-notes) - [📚 References](#-references) - [🗣️ Paraphrase](#️-paraphrase) - [📰 Newsletter](#-newsletter) - [✅ Action Items](#-action-items) - [🧪 Hypotheses](#-hypotheses) - [📊 Methodology](#-methodology) - [❓ Questions](#-questions) - [🔮 Future Research](#-future-research) - [💡 Implications](#-implications) - [🧠 Learning Styles Assessment](#-learning-styles-assessment) - [📊 Accessibility Assessment](#-accessibility-assessment) - [🌍 DEIB Assessment](#-deib-assessment) - [✅ Logical Validity](#-logical-validity) - [🛡️ DARVO Analysis](#-darvo-analysis) - [🎯 Conclusion](#-conclusion) ## 📝 Summary - A content creator demonstrates a production-grade AI video editing workflow using Claude and Remotion that has processed all their videos for 2 months. - The system achieved 15,000 views and 120,000 impressions in one week through fully automated editing. - The workflow involves five main stages: video ingestion, multi-modal analysis, intelligent clipping, storyboard creation, and automated rendering. - The system handles the complete pipeline from raw footage to finished video with motion graphics, thumbnails, and YouTube descriptions from a single prompt. - This represents a significant advancement from demo-only AI video editing tools to actual production deployment. ## 📖 Main Content The video reveals a comprehensive AI video editing system that breaks down the editing process into discrete, automated tasks. The workflow begins with raw video ingestion, where the system extracts audio for transcription using Deepgram, performs audio analysis with FFmpeg to detect volume levels and dead space, and conducts visual analysis using Gemini Pro to understand on-screen content. The clipping stage intelligently removes stuttering, dead space, and inferior retakes by combining transcript analysis with audio waveform data. The system maintains both 4K raw files and lower-resolution proxy files for API processing efficiency. A sophisticated storyboard creation process plans motion graphics placement throughout the video, drawing from a template library of pre-built assets and generating custom animations via Hera Video when needed. The system considers timing, content context, and brand consistency when selecting or creating visual elements. The final rendering stage uses Remotion to compile all elements—clipped video, motion graphics, B-roll, and effects—into a polished final product. The entire workflow operates through a structured folder system where each agent knows precisely where to find inputs and place outputs. ### 🗒️ Notes #### ⏱️ [00:00:00](https://www.youtube.com/watch?v=bDCLT4Je6WA?t=0) Production Claims and Results - System has edited all channel videos for 2 months - Last week generated 15,000 views and 120,000 impressions - High click-through rate achieved with AI-generated thumbnails #### ⏱️ [00:02:30](https://www.youtube.com/watch?v=bDCLT4Je6WA?t=150) Five-Stage Pipeline Overview - Stage 1: Raw video file ingestion and audio extraction - Stage 2: Triple analysis (transcription, audio levels, visual content) - Stage 3: Intelligent clipping (dead space, stutters, retake selection) - Stage 4: Storyboard creation with motion graphics planning - Stage 5: Asset compilation and Remotion rendering #### ⏱️ [00:04:15](https://www.youtube.com/watch?v=bDCLT4Je6WA?t=255) Technical Infrastructure - Deepgram API for transcription ($200 free credit) - FFmpeg for local audio analysis - Gemini Pro for visual analysis - Structured folder system for agent coordination - Proxy files (480p/720p) for API efficiency, 4K for final render #### ⏱️ [00:08:00](https://www.youtube.com/watch?v=bDCLT4Je6WA?t=480) Multi-Agent Architecture - Content strategist for competitive research and trending topics - Copywriter for script structure and diagrams - Publisher for cross-platform scheduling and copy - Video editor plugin with specialized sub-agents - Clearly defined input/output specifications for each agent #### ⏱️ [00:12:30](https://www.youtube.com/watch?v=bDCLT4Je6WA?t=750) Storyboard and Motion Graphics - JSON-based storyboard with timestamp precision - Template library for consistent brand assets - Hera Video integration for custom animation generation - HTML visualization option for pre-render approval - Intro/body separation for pacing differentiation #### ⏱️ [00:16:45](https://www.youtube.com/watch?v=bDCLT4Je6WA?t=1005) Production Deployment - Single command (FP) triggers entire pipeline - Remotion handles final compilation and rendering - System adaptable for short-form, VSLs, and advertising content - All motion graphics maintain brand color consistency ## 📚 References - [Deepgram API](https://deepgram.com) - Audio transcription service - [Remotion](https://remotion.dev) - Programmatic video creation - [Hera Video](https://hera.video) - AI motion graphics generation - [FFmpeg](https://ffmpeg.org) - Audio/video processing toolkit - [Gemini Pro](https://deepmind.google/technologies/gemini/) - Visual analysis ## 🗣️ Paraphrase This video demonstrates a sophisticated AI video editing system that moves beyond proof-of-concept demos to actual production use. The creator has automated their entire video editing workflow using a multi-stage pipeline that processes raw footage through intelligent analysis, clipping, storyboarding, and rendering phases. The system combines multiple AI services and tools—including Claude for orchestration, Deepgram for transcription, Gemini for visual analysis, and Remotion for rendering—to produce polished videos with motion graphics and professional pacing. The workflow is structured around a clear agent-based architecture where each component has defined inputs, transformations, and outputs, enabling full automation from a single command prompt. ## 📰 Newsletter 🎬 **AI Video Editing Reaches Production Maturity** A content creator has shared their battle-tested AI video editing workflow that's been processing their entire channel for two months. Unlike typical AI video demos, this system is genuinely deployed in production and generating real results—15K views and 120K impressions in just one week. The five-stage pipeline handles everything from raw footage ingestion to final render: intelligent clipping removes dead space and poor takes, AI-powered storyboarding plans motion graphics placement, and automated rendering produces polished videos with professional pacing. The system combines Deepgram transcription, Gemini visual analysis, and Remotion rendering in a multi-agent architecture that can process hours of content with a single command. This represents a significant milestone in AI content creation—moving from impressive demos to tools that creators actually rely on for their livelihood. For WordPress developers building content workflows, the structured agent approach and clear input/output specifications offer valuable patterns for automation systems. **Key takeaway**: Production-grade AI video editing is here, but it requires thoughtful system design, not just prompt engineering. ## ✅ Action Items #### DevRel - Research Remotion integration patterns for developer tutorial videos and documentation walkthroughs - Evaluate feasibility of automated video documentation for WordPress plugin demonstrations - Investigate multi-modal AI workflows for processing developer conference talks and workshops #### OSPO - Assess implications of AI-generated video content for open source project documentation standards - Review licensing considerations for AI-processed video content in open source projects #### Community - Explore automated video processing for WordCamp talks and community event recordings - Investigate accessibility improvements through automated captioning and transcript generation for community content #### WordPress - **Core**: Evaluate video processing capabilities for WordPress media library integration - **Training**: Research automated course content generation using similar multi-stage AI pipelines - **Marketing**: Assess AI video editing tools for WordPress promotional content and feature demonstrations #### The WPCC - Investigate funding opportunities for AI-assisted content creation tools for underresourced WordPress contributors - Explore AI video editing workshops for community capacity building ## 🧪 Hypotheses - Multi-stage AI video editing pipelines can achieve production quality comparable to human editors for structured content types - Agent-based architectures with clearly defined input/output specifications enable more reliable AI automation than single-prompt approaches - Combining specialized AI services (transcription, visual analysis, motion graphics) produces better results than general-purpose video editing AI - Proxy file workflows significantly improve processing speed and API cost efficiency for video analysis tasks - Automated storyboarding can maintain brand consistency more effectively than human editors across large content volumes ### 📊 Methodology - A/B testing comparing AI-edited vs human-edited video performance metrics (views, engagement, retention) - Cost analysis comparing AI workflow operational expenses to traditional video editing labor costs - Quality assessment studies using standardized video production criteria across different content types - User acceptance testing with content creators adopting similar multi-stage AI workflows - Technical performance benchmarking of different AI service combinations in video processing pipelines ### ❓ Questions - How does the system handle complex editing scenarios like multi-speaker interviews or live demonstrations? - What are the failure modes and quality control mechanisms when AI agents make poor editing decisions? - How much initial template and asset creation is required before the system becomes effective? - What content types and formats work best vs. worst with this automated approach? - How does the cost structure compare between AI automation and traditional video editing services? ### 🔮 Future Research - Integration of real-time AI editing for live streaming and virtual events - Cross-language video processing and automated localization workflows - AI-generated B-roll and stock footage creation for enhanced visual storytelling - Emotion and engagement prediction models for optimizing video pacing and retention - Collaborative AI editing systems where human editors work alongside AI agents for quality control ### 💡 Implications - **Practice**: Video content creators can achieve professional editing quality without technical editing expertise - **Policy**: Content platforms may need updated policies for AI-generated video disclosure and authenticity verification - **Security**: Automated video processing introduces new attack vectors through malicious media file uploads - **Education**: Video-based learning can scale dramatically with automated editing and production workflows - **Funding**: Traditional video production budgets may shift from labor to AI tooling and infrastructure costs ### 🧠 Learning Styles Assessment - **Reading level estimate**: Grade 12-14 (technical documentation style with specialized vocabulary) #### 👀 Visual learners - **Present**: Detailed folder structure screenshots, storyboard HTML visualization, Remotion interface demonstrations - **Improve**: Workflow diagrams showing agent interactions, before/after video comparisons, visual API integration maps #### 🔊 Aural learners - **Present**: Verbal walkthrough of entire process, spoken explanations of technical concepts - **Improve**: Audio-only version focusing on system architecture, podcast-style deep dive on implementation details #### 🗣️ Verbal learners - **Present**: Clear terminology definitions, analogies to traditional editing workflows, step-by-step explanations - **Improve**: Glossary of AI video editing terms, comparison frameworks between different approaches #### 🧩 Kinesthetic learners - **Present**: Live code demonstration, hands-on folder setup, real-time agent execution - **Improve**: Interactive setup checklist, downloadable starter templates, practice exercises with sample footage #### 🧠 Logical learners - **Present**: Sequential pipeline stages, clear input/output specifications, systematic agent architecture - **Improve**: Decision trees for choosing motion graphics, troubleshooting flowcharts, performance optimization matrices #### 🧑‍🤝‍🧑 Social learners - **Present**: Community building around AI Builders Hub, Q&A sessions, collaborative learning environment - **Improve**: Peer review processes for AI-edited content, collaborative template sharing, group troubleshooting sessions #### 🧘‍ Solitary learners - **Present**: Self-contained workflow documentation, independent setup process - **Improve**: Reflection prompts on workflow customization, self-assessment rubrics for output quality #### 🌿 Naturalistic learners - **Present**: Organic workflow evolution, adapting tools to content needs - **Improve**: Ecosystem thinking about AI tool integration, sustainable content production practices ### 📊 Accessibility Assessment #### Screen reader semantics - **Present**: Clear video structure with timestamp navigation - **Improve**: Detailed alt text for folder structure screenshots, transcript with action descriptions #### Keyboard navigation - **Present**: Standard video controls for playback navigation - **Improve**: Keyboard shortcuts for timestamp jumping, accessible timeline scrubbing #### Contrast and readability - **Present**: Clear visual distinction in folder hierarchies and code examples - **Improve**: High contrast mode for storyboard visualization, adjustable text size in documentation #### Text resizing and reflow - **Present**: Web-based content scales appropriately - **Improve**: Responsive design for technical diagrams, mobile-friendly workflow documentation #### Media and captions - **Present**: Auto-generated captions available - **Improve**: Professional transcript with technical term definitions, audio descriptions for visual elements #### Forms and interactions - **Present**: Standard video interface interactions - **Improve**: Accessible command line interface documentation, error handling guidance ### 🌍 DEIB Assessment #### Cultural breadth - **Present**: English-language content with universal technical concepts - **Improve**: Multi-language examples, culturally diverse use cases, global content creation perspectives #### Inclusive language - **Present**: Technical terminology with explanations, gender-neutral examples - **Improve**: Plain language summaries, expanded glossary, avoiding insider jargon #### Neurodiversity support - **Present**: Structured presentation with clear sections, predictable format - **Improve**: Content warnings for rapid visual changes, processing time estimates, cognitive load indicators #### Representation - **Present**: Single creator perspective on AI automation workflows - **Improve**: Diverse creator voices, varied content types, different production contexts #### Psychological safety - **Present**: Encouraging tone about AI accessibility, supportive community building - **Improve**: Failure mode discussion without judgment, alternative approach validation ### ✅ Logical Validity #### Strong reasoning - **Empirical evidence**: Provides specific metrics (15K views, 120K impressions) with timeframe context, demonstrating measurable outcomes rather than theoretical benefits - **Systematic methodology**: Breaks down complex video editing into discrete, analyzable stages with clear input/output specifications for each component - **Technical architecture**: Demonstrates understanding of production constraints through proxy file usage, API cost management, and agent coordination patterns #### Fallacies - **Hasty generalization**: Claims the system works for all content types based on one creator's experience with a specific format and style - **Appeal to novelty**: Implies AI automation is inherently better than traditional editing without comparing quality, cost, or time investment - **False dichotomy**: Presents AI editing vs manual editing as binary choice rather than exploring hybrid approaches or tool-assisted workflows #### Tone of voice - **Confident and authoritative**: Uses definitive statements ("this works", "production-grade") with technical credibility - **Educational and transparent**: Shares specific implementation details and actual code structure rather than high-level concepts - **Promotional undertone**: Balances genuine technical sharing with subtle marketing for AI Builders Hub community ### 🛡️ DARVO Analysis No DARVO patterns detected. The presenter acknowledges limitations of existing AI video editing demonstrations and positions their work as an improvement rather than attacking critics or portraying themselves as victims of unfair skepticism. ## 🎯 Conclusion This video represents a significant milestone in AI content creation by demonstrating a genuinely deployed production system rather than a proof-of-concept demo. The multi-stage pipeline approach, combining specialized AI services with structured agent coordination, offers a replicable model for creators seeking to automate video production workflows. The key insight is that production-grade AI video editing requires systematic engineering rather than clever prompting. Success depends on thoughtful system design, clear agent responsibilities, and robust file management—principles that extend beyond video editing to any complex AI automation workflow. For WordPress developers and open source contributors, this workflow architecture provides valuable patterns for building reliable AI-assisted content systems that can scale community documentation, training materials, and promotional content while maintaining quality standards. ## 🔄 Refactor Appendix — How to Edit Videos Properly With Claude and Remotion ### 📚 Literature Atoms (H3) #### Multi-stage AI pipelines achieve production quality through specialized agent coordination > [!quote] > You need to have a clearly defined what is the input, what's the transformation process in the middle, and then what is the output and where do I put it in the file structure, so the next agent knows where to get the information. **Context & use**: - Timestamp 00:11:30, discussing workflow architecture requirements - Applies to any complex AI automation system requiring reliable handoffs between specialized components **Tags**: #ai-automation #system-design #production-workflows  #### Proxy files enable efficient API processing for large media workflows > [!quote] > The raw files are typically in 4K. They're big files. Sending them to API services and stuff like that, they're too big. You want to create a proxy at 480 pixels or 720 pixels, which you can use to send to APIs. **Context & use**: - Timestamp 00:09:45, explaining technical infrastructure decisions - Critical for managing costs and processing speed in video analysis workflows **Tags**: #video-processing #api-optimization #media-workflows  #### Production AI systems require separation of intro and body content for optimal results > [!quote] > The reason for that is because intros typically for the first 30 seconds to a minute or so, a little bit higher paced, a little bit more motion graphics, slightly different structure to the actual body of a video. **Context & use**: - Timestamp 00:08:15, discussing content structure analysis - Demonstrates importance of context-aware processing in automated content creation **Tags**: #content-structure #ai-editing #pacing-optimization  ### 🧠 Permanent Candidates (H4) #### Production-grade AI automation requires systematic engineering over prompt creativity **Claim**: AI systems that reliably handle production workloads succeed through structured agent architectures and clear data flows rather than sophisticated prompting techniques. **Evidence pointer**: [[#Multi-stage AI pipelines achieve production quality through specialized agent coordination]] **Scope & conditions**: Applies to complex workflows with multiple processing stages; breaks down when tasks are simple enough for single-agent handling **Links**: - [[System design patterns]] — because extends - [[AI prompt engineering]] — because contrasts on implementation approach - [[DevOps for AI workflows]] — because applies-in deployment contexts  ### ✏️ Fleeting Queue - [F1] Research Hera Video integration patterns for motion graphics generation — https://hera.video #fleeting #video-tools - [F2] Investigate cost comparison between Deepgram vs other transcription APIs for video workflows — budget analysis #fleeting #api-costs - [F3] Explore Remotion template libraries for WordPress-specific content types — documentation videos #fleeting #wordpress-tools ### → Things 3 (max 2 actions) - [ ] Evaluate Remotion for WordPress plugin demonstration videos — technical feasibility study #things - [ ] Research multi-modal AI workflow patterns for developer documentation automation — architecture planning #things --- ## Full Transcript [00:00] You may have already watched videos that [00:02] teach you how to edit videos with Claude [00:04] and re-motion, but none of them are [00:06] actually being used in a production [00:07] environment. At this point in time, [00:09] they're really just there to get clicks [00:10] on YouTube. What I'm going to do in this [00:12] video is take you behind the scenes in [00:14] my AI workflow that's actually been [00:16] editing all the videos on my channel for [00:17] the last 2 months, and in the last week [00:19] alone has gotten me over 15,000 views [00:21] and over 120,000 impressions. If you [00:24] watch this video from start to finish, [00:26] you'll understand how to create your own [00:27] AI video editing workflow that takes an [00:29] unclipped long-form of any length, clips [00:32] the entire thing, adds motion graphics, [00:34] builds thumbnails, builds YouTube [00:36] descriptions with timestamps, and gives [00:38] you the entire package at the end with [00:40] one prompt only. But, you're also going [00:42] to understand the mechanics behind it [00:44] and how it actually works, so you can [00:46] tweak it to your own personal brand and [00:48] your own style. Let's get straight into [00:49] it. All right. Now, I'm going to take [00:51] you over this entire video generation [00:53] pipeline, and there is a little bit [00:55] involved with this, but if you actually [00:57] want to build a production-grade video [00:58] editing system, there is really no [01:00] skipping. And what we'll cover is we [01:02] will first talk about and I'll show you [01:03] my channel analytics and the proof. We [01:06] can take a look at it right here, so you [01:07] can see over the past 7 days that these [01:10] are the statistics that I've had over [01:11] the content that I've pushed to my [01:13] channel, and it has been entirely edited [01:15] by AI. So, we're talking about 15,000 [01:17] views, 120,000 impressions, a quite a [01:20] high click-through rate for AI-generated [01:22] thumbnails, and the average view [01:24] duration view generation. So, this [01:26] works. And how it works at a high level [01:28] before I jump in is we need to take the [01:30] video editing process and think about [01:32] what is actually What are the individual [01:34] tasks involved with video editing? And [01:36] so, the very first thing we do is we get [01:38] the raw video file and we drop it in a [01:39] folder, and we ask AI to ingest it. And [01:42] what it does here is is it extracts the [01:44] audio, and it sends that audio, and it [01:47] does three things. One, it transcribes [01:49] the audio to get all the text and the [01:51] timestamps when we're saying certain [01:52] things. Then, it also does an audio [01:54] analysis is the volume levels, so it [01:56] understands where to clip things, and [01:58] then it does uses Gemini to do a visual [02:00] analysis to actually see what's [02:02] happening on the screen in the video [02:04] throughout the entire video. So, that it [02:06] gets those three things, and it uses [02:08] those to clip the video. So, what we're [02:10] doing when we're editing is we're [02:11] clipping dead space, we're clipping me [02:13] stuttering in most cases, we're clipping [02:16] we're doing we're taking all that into [02:17] account, and we're basically clipping [02:19] the video. And if we if we basically [02:21] detect retakes, if you're filming [02:23] multiple intros, etc., it will basically [02:26] keep the best take and cut the others. [02:28] So, that's step three. With that clipped [02:30] video, it's then going ahead, and it's [02:31] building a storyboard. So, I'll show you [02:33] what this looks like in a little bit, [02:34] but essentially the storyboard is where [02:36] we're going to plan All right, Adam is [02:37] saying this here. Let's build motion [02:40] graphics, or whatever the case is, uh to [02:42] essentially build a storyboard so that [02:44] it can plan out when it goes to to [02:46] render. Um and it can also plan out when [02:49] what type of B-roll we're going to use, [02:50] uh what type of motion graphics, etc., [02:53] for specific words and concepts that I [02:54] might be talking about. So, we build the [02:56] storyboard, which is which is a big kind [02:57] of JSON file. Then, we basically build [02:59] the assets. So, we either get B-roll [03:01] from the existing video, or a library of [03:04] B-roll that I've built, and AI will [03:05] automatically select that, or we'll use [03:08] a service called Hera, or Hera video, to [03:10] generate motion graphics. [03:12] And then what will happen is we'll get [03:14] the storyboard, we'll add the motion [03:16] graphics that we found, and we'll use [03:17] Remotion to compile all of that [03:19] together. Now, what I'll also mention is [03:22] that I have a template library. So, if [03:23] you look at this, we've got all these [03:24] different motion graphics, and that's [03:27] provided to the storyboard agent, so it [03:29] knows, "Hey, look, like you can use all [03:31] of these different motion graphics at [03:33] different points in the video to do all [03:35] sorts of cool things." We've got text [03:36] reveals, and you can you can build this [03:39] to do all sorts of things, you know what [03:41] I mean? So, these are the kind of things [03:43] that we can source when we're building [03:45] our video, so that the at the end of [03:47] this, we get Remotion, we we render it [03:50] together with all these motion graphics [03:51] that we either have from our template [03:52] library or we're generating custom or [03:54] we're pulling from the main video [03:57] and our existing B-roll library to build [03:59] a full end-to-end video system. So, the [04:03] first step is to ingest the raw video [04:06] file and I'm going to show you exactly [04:08] how this looks directly in VS Code. So, [04:12] I'm going to look at my very last video [04:15] which was the real skill of AI [04:17] automation and in there I've got a [04:19] folder that looks like this. Now, I have [04:21] a number of different agents that do [04:22] different things. Like I have a [04:23] copywriter that writes scripts even [04:25] though in most cases I don't use them. [04:27] It builds general dot points that is [04:29] used for the rest of to build the actual [04:31] diagram that you're looking at right [04:33] here. So, I don't write read off a [04:35] script but I do use a general structure [04:37] to build this diagram. So, in there I [04:40] have the [04:41] copywriter that builds a diagram in the [04:43] script. I've got a strategist that [04:46] actually does the competitive research [04:48] and sees what topics are trending or [04:50] what people want to watch. I've got the [04:52] publisher that's actually going to [04:53] schedule this content across different [04:55] social media channels and build email [04:57] campaign copy and on all those kind of [04:59] things LinkedIn posts etc. And then I've [05:01] got the video editor. So, the first of [05:03] the video editor is to just drag and [05:04] drop the raw file directly into the [05:06] video ingest folder. And what happens so [05:09] if you're going to set this up yourself, [05:11] step one is to set up a similar folder [05:13] structure to what I have here. Now, if [05:15] we look at the agents you'll see I've [05:16] got the content plugin and the video [05:18] plugin. Now, the video plugin is purely [05:20] around video editing of how complex [05:22] video editing is and there's multiple [05:24] different steps. So, I made it its own [05:25] plugin. But my content plugin is the one [05:28] that has the content strategist, the [05:29] copywriter and everything else and then [05:32] the video editor plugin is purely just [05:33] for video editing. So, if you want to [05:35] set it up yourself, then have a think [05:38] about the different roles in your [05:39] content pipeline. You can copy me if you [05:41] like and have a content strategist, a [05:42] copywriter, creative director which is [05:44] for the thumbnails and editor which is [05:46] for basically a quality gate to make [05:48] sure your content is targeting right [05:49] audience, and a publisher. And then you [05:51] get your video editor agent. So, in [05:53] doing that, I have my agents here, and [05:56] then I have the actual output file. And [05:59] when I first create a new idea, so when [06:01] I run the content strategist for the [06:02] first time, just by basically running [06:04] content strategist, it will ask me [06:06] whether I want to create a new project. [06:07] And if I want to create a new project, [06:09] it's going to create this folder [06:10] structure straight away. And as I call [06:13] the different agents, it'll start to [06:14] push the output in the relevant agent [06:16] output folder. So, the first step is to [06:19] ingest a video. [06:21] And that's what I do first, and what [06:22] happens is it automatically triggers the [06:26] transcription and the visual analysis. [06:28] Now, the APIs that I use I use Deepgram [06:30] for [06:31] transcription. The reason for that is [06:33] you get $200 which is more than enough. [06:35] I don't think I've even used that $200 [06:37] yet, and I've been using this for [06:38] months. Heaps of free credit, so you can [06:40] pretty much run it for free. Gemini Pro [06:42] is what I use for the visual analysis, [06:45] and then the audio analysis is just done [06:46] with FFmpeg, which is completely free [06:48] and runs locally on your computer. So, [06:51] you'll see here I've run the content [06:52] strategist, it's asked if I want to [06:53] create a new project. If I type three, [06:55] it's going to take me through the [06:56] workflow to create a new project. Now, [06:58] what I'll do quickly is I'll show you [06:59] the video editor workflow, and I'm going [07:01] to trigger the long-form video editor [07:03] workflow because if we look in my video [07:06] editor agent, this doesn't just do [07:08] long-form, [snorts] it does short-form, [07:11] video sales letters, advertising, etc. [07:14] You can have all these different video [07:15] content types, and you can just tweak [07:18] the skills that are involved in the [07:20] workflows for each of them to be [07:21] specific to that. For example, the [07:23] timing shorter form, you might want to [07:25] have more transitions and those kind of [07:27] things, you'd have a slightly different [07:28] workflow for that than you would for [07:29] editing your long-form. So, if I trigger [07:31] the long-form skill, you'll see here [07:33] that we have the steps, and typically [07:36] what I run is the full pipeline, which [07:38] is what runs this entire thing on [07:39] autopilot. So, I type FP, and the first [07:42] thing it does is this ingest point. So, [07:44] what it will do is it will take a look, [07:45] it knows to look in the ingestion folder [07:47] for a specific It's It's read what the [07:49] active project is, which is the real [07:51] skill of AI automation. So, if I run FP, [07:54] it will ingest the file that is in the [07:56] video ingest folder. That's the first [07:58] step, and it's going to go on ahead and [08:01] trigger these three workflow steps. [08:04] Now, once I've run all of these analysis [08:07] through the audio analysis, the [08:08] transcription, and the visual analysis, [08:11] you'll get those three files outputted [08:14] in whatever format you like. So, I've [08:15] got this analysis folder, and you'll see [08:17] here I've got the full video, but then I [08:19] also split the intro from the body. The [08:21] reason for that is because intros [08:22] typically for the first 30 seconds to a [08:25] minute or so, a little bit higher paced, [08:27] a little bit more motion graphics, [08:28] slightly different structure to the [08:30] actual body of a video. So, what will [08:32] happen is we have the full video, and [08:33] we'll have the audio analysis files in a [08:35] number of different file formats. We'll [08:37] have the transcription, and we'll also [08:39] have the visual analysis in these files. [08:42] Now, they're they're structured I could [08:44] probably make this a little bit more [08:46] concise if I was being honest. Uh but [08:48] that with that, we're then going to [08:49] build what's called or what I call a [08:51] clip plan. So, we're going to take the [08:53] combination of those things, [08:55] and we're basically going to clip the [08:57] video. So, what we're going to do is [08:59] we're going to get AI to look at the [09:00] transcript and identify areas where I [09:02] might be study stuttering, or where I [09:05] might have taken gaps. Then it's going [09:07] to use the audio analysis, which is [09:09] getting the waveforms or the actual [09:11] volume levels to get much more specific [09:14] at exactly where to make those cuts. And [09:16] then it will also To be honest, I don't [09:18] use the visual analysis really, but it [09:20] will use the combination of those two [09:21] things to build a plan and say, "Hey, [09:23] cut these sections out of the video, uh [09:25] so that you can basically clip out all [09:27] of your retakes, uh any dead noise, all [09:30] those kind of things." You can clip the [09:31] video and make it a lot more concise. [09:33] So, that will be the next step that [09:35] automatically happens in the video [09:37] intake pipeline. You can kind of see in [09:39] here that it's clipping up and basically [09:42] clipping up different sections. The [09:44] other thing as well that I forgot to [09:45] mention is a proxy file. So, you'll see [09:46] we have raw and we have proxy. The raw [09:49] files are typically in 4K. They're big [09:51] files. Sending them to API services and [09:53] stuff like that, they're too big. You [09:55] want to create a proxy at 480 pixels or [09:57] 720 pixels, which you can use to send to [09:59] APIs. It's going to be a lot quicker to [10:01] work with cuz it's a much smaller file. [10:03] Then when you render, you can basically [10:04] swap the 4K video back in. So, that's [10:07] going to help when you're using like [10:09] Nano uh sorry, Gemini Pro Gemini Pro and [10:11] other API services without having to [10:13] upload 4K files, which is probably going [10:15] to cause you problems. So, that is step [10:18] four, which is getting a nice clipped up [10:20] raw video. Now, step five is where [10:22] things get interesting. So, now that we [10:24] have the clipped up video, we have the [10:25] transcript, we have the audio analysis, [10:27] we want to actually create a storyboard [10:29] so we can plan out what motion graphics [10:31] uh will look good in different sections [10:33] of the video. And not only that, but [10:36] we're also going to take a look at some [10:38] of our motion graphic templates and [10:39] we're going to build prompts for Hera [10:41] video, which is an AI motion design tool [10:44] that I've been using. So, we're going to [10:46] use the combination of those two things [10:48] to build a storyboard. And as you can [10:50] see, we have a storyboard photo folder. [10:52] So, as you can tell, one of the key [10:53] things we have here is we have inputs. [10:54] We know each of these steps is [10:56] outputting a certain type of file and [10:58] it's going into a specific folder so [11:00] that in order to build a workflow where [11:02] we can just click or write FP and [11:04] trigger this and it actually does the [11:05] entire thing, it needs to know there's [11:08] different there's multiple different [11:09] agents you can see here. There's [11:10] multiple different steps in the [11:12] workflow, but all of the outputs or all [11:14] the inputs are clearly defined so that [11:16] the agents know where to go to get the [11:18] input that the information that they [11:20] need or the files that cuz it's all [11:21] organized in folders. They know where to [11:23] put the output because we're creating [11:25] this entire file structure for every [11:26] single project. And that's one of the [11:28] key things if you're going to set up a [11:29] an agentic workflow that's going to [11:31] basically have multiple agents doing [11:33] different things in one work flow, you [11:35] need to have a clearly defined what is [11:37] the input, what's the transformation [11:39] process in the middle, and then what is [11:40] the output and where do I put it in the [11:42] file file structure, so the next agent [11:44] knows where to get the information. [11:47] So, what we've got here is the [11:48] storyboard. Now, if you want to, you can [11:49] actually make a HTML for your [11:51] storyboard. Now, this doesn't look [11:53] great. You can probably lay it out a [11:54] little bit better, but you'll see here [11:55] we've got the storyboard of the entire [11:57] video. And this is the visual timestamp [12:00] for the intro. So, we can basically go [12:01] through and take a look at the motion [12:04] graphic, which one it's going to use, [12:06] the transcript when I'm saying it, those [12:08] kind of things. I don't really use this, [12:10] to be honest. You can make it look a [12:11] hell of a lot nicer, but if you wanted [12:13] to build a visual storyboard in your [12:15] storyboard process to actually be able [12:17] to see what motion graphics will be used [12:19] at what point, you can easily have that [12:21] built into the workflow, so that you can [12:23] verify before you go and do the remotion [12:25] and actually build the video, you can [12:27] verify that the motion graphics look how [12:30] you intend. And if you want to tweak [12:32] them, again, you're just going to [12:33] basically You can get this to pause, [12:35] which mine actually does. It pauses at [12:36] the storyboard point and just confirms [12:38] that that's that that looks good. Uh it [12:41] can do all the motion graphic rendering [12:42] and everything, and it can present you [12:43] that HTML. And if you want to adjust [12:45] anything, you can adjust it there before [12:47] you then approve that to go ahead and [12:49] actually do the remotion rendering. [12:51] Certainly leads us to the next point. [12:52] Now, I found that B-roll and motion [12:56] graphics from here, I'm pretty happy to [12:58] have this in most cases, but the Hair of [13:00] Video motion graphics do They are that [13:02] next level where you can actually build [13:03] a prompt and you can provide a reference [13:05] image. So, if you want to have an [13:06] animation of, for example, of me typing [13:08] into a Claude desktop interface or doing [13:11] something a little bit more than just [13:12] these kind of static-looking motion [13:14] graphics, even though these are pretty [13:16] good, you can definitely build that in [13:17] and use Hair motion graphics in [13:19] combination with pre-existing B-roll and [13:22] remotion templates to get a really, [13:24] really polished video editor effect in [13:27] your AI video editor workflow. And the [13:29] last step is we're actually going to use [13:31] Remotion, which is what I'm using right [13:32] here, running locally on your local [13:34] server and installed to basically render [13:37] these videos. Now, it's going to be able [13:39] to compile everything into one spot, and [13:42] then once you have that built, you can [13:44] either just click render here [13:46] and basically render the video, or you [13:48] can again come here and basically type [13:51] in and say, "Hey, please render this [13:52] video. Render it at 4K," or whatever the [13:54] case is, because one thing you'll notice [13:56] is that in the video preview here, this [13:58] is in This is not 4K. So, you'd have to [14:01] update it to get it to render in 4K. But [14:04] that's basically going to render your [14:05] entire video [14:07] and output a video, no video editing [14:09] software required, all the motion [14:10] graphics sorted, and you can run this [14:12] entire thing on autopilot by yourself [14:14] and genuinely put an input in and get an [14:17] output, have it on brand, have motion [14:19] graphic templates that are all aligned [14:21] with your colors and everything like [14:22] that. And it looks really, really clean. [14:24] So, if you want a production-grade AI [14:27] video editing system, [14:29] this is the plugin for you. [14:32] And not only that, but you can then go [14:34] ahead and use this exact same system for [14:36] short-form content. Uh you can repurpose [14:38] it for video sales letters, all of those [14:40] kind of things. So, this is what an [14:41] actual production-grade AI video editing [14:44] workflow looks like, and it is being [14:46] used live by me on my channel. [14:48] If you want to learn how to build these [14:50] kind of systems and plugins or AI [14:53] operating systems for yourself, feel [14:54] free to jump into AI Builders Hub, where [14:56] I do Q&As every single week and help [14:59] teach the exact skills that we're [15:00] talking about in this video. Thank you [15:02] so much for watching, and I hope that [15:05] you use these skills to go ahead and [15:07] build your own AI video editing [15:08] workflows.  📄 **Full text:** ![[Full Document Contents/Articles/How to Edit Videos Properly With Claude and Remotion.md|How to Edit Videos Properly With Claude and Remotion by YouTube]] ## Highlights - video editing process and think about what is actually What are the individual tasks involved with video editing? And so, the very first thing we do is we get the raw video file and we drop it in a folder, and we ask AI to ingest it. And what it does here is is it extracts the audio, and it sends that audio, and it does three things. One, it transcribes the audio to get all the text and the timestamps when we're saying certain things. Then, it also does an audio analysis is the volume levels, so it understands where to clip things, and then it does uses Gemini to do a visual ([View Highlight](https://read.readwise.io/read/01ksn1ahe1fv1pams3b665x47d)) - Note: Breaking complex automation into discrete tasks enables reliable end-to-end workflows — decompose "edit video" into ingest → transcribe → audio analysis → visual analysis. - Tags: task-decomposition, automation-design, atom/literature - for transcription. The reason for that is you get $200 which is more than enough. I don't think I've even used that $200 yet, and I've been using this for months. Heaps of free credit, so you can pretty much run it for free. Gemini Pro is what I use for the visual analysis, and then the audio analysis is just done with FFmpeg, which is completely free and runs locally on your computer. So, you'll see here I've run the content strategist, it's asked if I want to create a new project. If I type three, it's going to take me through the workflow to create a new project. Now, what I'll do quickly is I'll show you the video editor workflow, and I'm going ([View Highlight](https://read.readwise.io/read/01ksn1ax0w6hh89jwtynkfdfra)) - Note: Production AI systems benefit from specialized tools per task — Gemini Pro for visual analysis, FFmpeg (free, local) for audio analysis — rather than one general-purpose model. - Tags: atom/literature, api-selection, cost-optimization - also split the intro from the body. The reason for that is because intros typically for the first 30 seconds to a minute or so, a little bit higher paced, a little bit more motion graphics, slightly different structure to the actual body of a video. So, what will happen is we have the full video, and ([View Highlight](https://read.readwise.io/read/01ksmzjvsaychwcps6xf2tm1es)) - Note: Structure: split intro from body — intros (first 30-60s) are higher-paced with more motion graphics. - Tags: video-editing, production-workflows, atom/literature/h2 - we have raw and we have proxy. The raw files are typically in 4K. They're big files. Sending them to API services and stuff like that, they're too big. You want to create a proxy at 480 pixels or 720 pixels, which you can use to send to APIs. It's going to be a lot quicker to work with cuz it's a much smaller file. ([View Highlight](https://read.readwise.io/read/01ksmzjqndsyms22d0ry8thpvn)) - Note: Practical tip: 4K raw is too big for APIs — make a 480/720 proxy for faster, cheaper agent processing. - Tags: video-editing, atom/literature/h2, ai-automation - basically have multiple agents doing different things in one work flow, you need to have a clearly defined what is the input, what's the transformation process in the middle, and then what is the output and where do I put it in the file file structure, so the next agent knows where to get the information. So, what we've got here is the storyboard. Now, if you want to, you can actually make a HTML for your storyboard. Now, this doesn't look great. You can probably lay it out a little bit better, but you'll see here we've got the storyboard of the entire video. And this is the visual timestamp for the intro. So, we can basically go ([View Highlight](https://read.readwise.io/read/01ksn1b7h2y02a13jxry8kc6gn)) - Note: Multi-agent workflows require explicit input/output contracts — each agent needs a clearly defined input, transformation, and output location so the next agent knows where to read. - Tags: atom/literature, workflow-design, automation-architecture - different things in one work flow, you need to have a clearly defined what is the input, what's the transformation process in the middle, and then what is the output and where do I put it in the file file structure, so the next agent ([View Highlight](https://read.readwise.io/read/01ksmzjm4npph5a89agyd2w1dg)) - Note: Workflow design: define input → transformation → output + where it lands in the file structure, so the next agent can pick it up. - Tags: atom/literature/h1, ai-automation, production-workflows