# Firecrawl + Convex > [!question] Lingering Questions > - If webhook signature verification fails on a delivery, or the Convex deployment is briefly unreachable, does Firecrawl retry that page, or is it silently dropped from the crawl? > - How does per-page mutation cost scale on Convex's pricing/quota model for crawls in the thousands-of-pages range, well beyond the 25-page demo? > - Does the component expose crawl cancellation, or is `startCrawl` strictly fire-and-forget once started? > [!tip] Article Ideas > - The "sandbox vs. actions" explanation is a reusable teaching moment for any serverless/BaaS platform with a similar function-privilege split — could be excerpted as a standalone concept clip. #articleseed > [!link] Cross Reference > - [[3. Resources/Permanent Notes/Outbound Integration Is Cheap While Inbound Concentrates the Auth Cost]] — adjacent territory: that note frames the general auth-cost asymmetry, this source is a concrete instance (Firecrawl's inbound webhook needs signature verification, the outbound search call doesn't) > - [[3. Resources/MOCs/AI & the Future of Knowledge Work MOC]] > [!quote] Vocabulary > - **Convex action** — the one Convex function type allowed to make outbound HTTP calls; queries and mutations run in a network-isolated sandbox and cannot. > - **Durable crawl** — Firecrawl's webhook-driven crawl pattern: the action starts a crawl and returns immediately, then Firecrawl POSTs each scraped page back to the Convex deployment as it completes, independent of whether the client app stays open. ## Bib capsule Firecrawl, "Firecrawl + Convex: Scrape the Web Straight Into Your Database," YouTube tutorial, August 14, 2026. https://www.youtube.com/watch?v=5XyirpOIO5M ## Author's thesis (Q1+Q2) The official Firecrawl Convex component lets a Convex app add web search, page scraping, and full-site crawling with results written directly into Convex tables. The presenter's central architectural claim: Convex's queries and mutations run in a network-isolated sandbox and cannot make outbound calls, so any Firecrawl call has to originate from a Convex action — and because crawls are long-running, they can't be handled by a function held open; instead the action starts the crawl and Firecrawl streams results back per-page via a signed webhook, giving a "durable" crawl that survives the client closing. ## Key arguments (≤3) 1. **Convex actions are the only sandbox exit.** Queries and mutations can't reach third-party APIs; only actions can, which is why the Firecrawl integration has to run through an action calling a component with its own tables. 2. **Long-running crawls need a webhook, not a held-open function.** Convex's serverless execution model is a poor fit for scraping hundreds of pages synchronously, so the action starts the crawl and returns immediately while Firecrawl calls back per page. 3. **The webhook endpoint is a public write surface that needs signature verification.** Skipping Firecrawl's webhook-signature check would let anyone POST arbitrary data into the app's database — the video's one explicitly flagged security risk. ## Claim atoms ### Convex actions are the only sandbox exit > [!quote] ID:rw_atom1 > these functions run inside a sandbox. So, how can they access the internet and do web searches? Where/why: Names the exact reason the integration needs an "action" rather than a query/mutation — the load-bearing architectural fact of the video. ### Durable crawls write to Convex via signed webhook > [!quote] ID:rw_atom2 > when each page is scraped, firecrawl posts to a HTTP endpoint in that deployment that verifies the signature and calls a mutation to update the database Where/why: Explains the "durable crawl" claim mechanically — per-page webhook plus signature check plus mutation, not a held-open connection or polling loop. ### Local Convex can't host Firecrawl's webhook > [!quote] ID:rw_atom3 > I'm using the cloud version of Convex, not the local one since they don't have public URLs Where/why: A concrete, easy-to-miss setup requirement — anyone following along on local Convex hits a silent wall on the crawl feature specifically, though search still works locally. ## Permanent candidates - **New**: the general pattern — serverless function limits force webhook-driven design for long external jobs (P1) — is not yet captured — promoted below as [[3. Resources/Permanent Notes/A Serverless Function That Cannot Wait Must Hand the Job to a Callback]]. ## Processing checklist - [x] Checked for existing Literature Note — none found - [x] Checked for existing Permanent Notes covering this source's claims — no direct match for the webhook-handoff pattern; promoted - [x] Connected into [[3. Resources/MOCs/AI & the Future of Knowledge Work MOC]] ## Links - [[3. Resources/MOCs/AI & the Future of Knowledge Work MOC]] — developer-tooling patterns for AI-adjacent data ingestion