--- title: "ScrapeNinja Integration" description: "ScrapeNinja is a web scraping API with a fast HTTP engine, a real browser engine, and a server-side Cheerio extractor. Workflows fetch pages, extract structured data with JavaScript extractors, capture screenshots, and grab AJAX responses." url: https://flowrunner.ai/integrations/scrapeninja date_modified: 2026-09-04T14:59:52-07:00 --- # ScrapeNinja [Developer Tools](https://flowrunner.ai/integrations/category/developer-infrastructure) ScrapeNinja is a web scraping API with a fast HTTP engine, a real browser engine, and a server-side Cheerio extractor. Workflows fetch pages, extract structured data with JavaScript extractors, capture screenshots, and grab AJAX responses. [Verified](https://flowrunner.ai/integrations/verified "What does verified mean?") · 7 actions · API key · available [ScrapeNinja website](https://scrapeninja.net/) · [Platform Documentation](https://scrapeninja.net/) · Capability data verified 2026-08-25 1. The catalog sync fires on a schedule with this run's URL list 2. Scrape fetches each page with the fast engine first, because the Chrome TLS fingerprint alone clears a lot of protection 3. Agent branches on \`looksBlocked\` and \`targetStatus\` rather than the HTTP status, since a block page arrives as a success 4. Extract Data runs the extractor in ScrapeNinja's cloud and returns fields instead of a page of HTML 5. Agent compares each record's field set against the shape the extractor was built to produce 6. Records that thinned out or shifted are collected with their source URLs and the sample they were compared to 7. An owner reads the extractor's output against the live page before it defines records at volume ## What This Integration Enables ScrapeNinja's argument is that most scraping tools give you the wrong artifact. A page of HTML sitting in a workflow variable is not data. It is a document that somebody now has to parse, usually in a second tool, usually with code that lives somewhere the workflow does not. ScrapeNinja moves the parse to its own side: an extractor is a small JavaScript function using the Cheerio parser that runs in ScrapeNinja's cloud and returns only the fields you asked for. The second thing worth knowing is that the engine is a decision, not a default. Scrape uses a conditioned HTTP client with a real Chrome TLS fingerprint and no JavaScript execution, which is fast, cheap, and enough for a surprising number of protected sites. Scrape With Browser runs actual Chrome and costs accordingly. The vendor's own guidance is to try the cheap one on every site first. FlowRunner agents can make that escalation automatically, because the response reports whether the fetch looks blocked rather than leaving a flow to guess from markup. What agents should not do unsupervised is decide that a new extractor is producing correct facts, and that is where the [human-in-the-loop](https://flowrunner.ai/concepts/human-in-the-loop) step sits. ### Without FlowRunner **HTML in a variable**: The scrape succeeds and the workflow now holds a page of markup that no later step can act on **One engine for every site**: Every fetch pays browser cost, including the majority of pages that never needed JavaScript **Blocks that look like data**: A Cloudflare interstitial returns HTTP 200, and the record built from it says the product costs nothing ### With FlowRunner **Fields, not markup**: A server-side extractor returns clean JSON, and the next step reads a price rather than a document **Cheap first, expensive on evidence**: The fast engine runs by default and the browser engine runs when the content is genuinely absent **Block detection the flow can branch on**: The response carries the upstream status and a block signal beside the body, so a branch needs no HTML parsing ## Use Case Scenarios ### A catalog sync that pays browser cost only when it has to A pricing workflow runs nightly against a competitor's product pages. The agent calls Scrape first. Where the response comes back with `looksBlocked` set or with a target status in the not-expected list, it retries through the browser engine rather than through the same route again, since retrying an identical request against a site that just refused it is a way to spend money confirming a refusal. Extract Data returns the fields, and the records land in [PostgreSQL](https://flowrunner.ai/integrations/postgresql) with a movement summary posting to [Slack](https://flowrunner.ai/integrations/slack). Setting Text Not Expected matters more than it sounds: ScrapeNinja carries some Cloudflare markers by default, but the block-page string your particular target uses is the one that decides whether tonight's run produces prices or produces interstitials. ### Reading the site's own API instead of its markup Many pages render from an internal JSON endpoint the site calls in the background. Capture AJAX Response loads the page in a real browser and returns the response of that call, matched on a substring of its URL. What comes back is the site's own structured data, which does not break when the front end is redesigned. For a listings site or a store that ships a new template every quarter, this is the difference between a scrape you maintain and a scrape you set up. The captured payload flows into [Google Sheets](https://flowrunner.ai/integrations/google-sheets) or into the model step that summarizes it. ### Content that is not in the page at all A partner's jobs board or booking widget often lives entirely inside an iframe, which means it is absent from the page's own HTML and invisible to any selector written against it. Scrape Iframe reads the named frame directly, and an extractor supplied alongside runs against the frame rather than the parent, which is almost always what was meant. It matches the iframe's `name` attribute rather than its id, src, or class, which is the detail that costs an afternoon when it is discovered by trial. ## Human-in-Loop Highlight Extract Data is deliberately loud about one failure and silent about the other. An extractor that errors raises rather than returning an empty object, because ScrapeNinja reports extractor errors inside a successful response body, and a silently empty extraction is the kind of thing that goes unnoticed for weeks. That covers the extractor that breaks. It does nothing for the extractor that keeps working and starts reading the wrong element after a redesign, which returns a full, plausible, confidently wrong record every night. So a new or edited extractor does not go straight to the URL list. The agent runs it against a named sample, and posts the actual output beside the actual page: "New extractor for the supplier catalog, run against 5 URLs. `price` resolved to 89.00, 129.00, 24.99, and empty on two. The previous extractor returned a value on all 5. The two empties are bundle pages where the price sits in a different node. Ship, or fix the selector first?" The extractor is code, it runs in somebody else's cloud, and everything downstream is about to treat its output as fact. Reading five records takes a person two minutes. Unwinding a month of wrong prices from a database does not. Agent processes routinely Detects exception requiring judgment Clear match Continues automatically Ambiguous Routes to human via preferred channel Human decides Agent resumes with decision ## Agent Capabilities 7 actions ### Fetch Engines 3 - **Scrape** Fetches a page with the fast engine: a real Chrome TLS fingerprint with no JavaScript execution. The one to try first on every site, because it clears a good deal of anti-scraping protection on fingerprint alone and costs a fraction of the browser engine. A block page still returns as a success, so the response also carries a block signal and the upstream status for the flow to branch on. - **Scrape With Browser** Fetches a page with real Chrome, executing JavaScript, waiting for elements, intercepting background requests, and taking screenshots. Slower and more expensive, so it is the escalation rather than the default. Blocking images and media cuts both time and bandwidth substantially and rarely changes what an extractor reads. - **Scrape With Browser V2** Fetches a page with the newer browser engine, which the vendor says performs better against Cloudflare, DataDome, Kasada, and PerimeterX. It is not a strict upgrade: it drops the iframe and background-request interception options the previous engine has, and it is available only through one of the two marketplaces ScrapeNinja is sold through. ### Extraction 1 - **Extract Data** Scrapes a page and returns only the extractor's JSON, discarding the HTML. The operation most workflows actually want, since fields are usable in a later step and a document is not. The extractor is required here, and an extraction that returns nothing raises rather than handing back an empty object. ### Content Other Fetches Miss 3 - **Capture AJAX Response** Loads a page in a real browser and captures the response of one of the site's own background requests, matched on a substring of the request URL. When a page renders from an internal JSON endpoint, reading that endpoint is far more durable than any selector, because a redesign of the markup does not touch it. - **Scrape Iframe** Loads a page in a real browser and returns the HTML of one named frame rather than the main document. The way to reach an embedded widget, such as a jobs board or a booking form, that lives entirely inside a frame and is therefore absent from the page's own HTML. It matches the frame's \`name\` attribute. - **Screenshot Page** Loads a page in a real browser, captures it, and stores the image in FlowRunner file storage. ScrapeNinja returns a screenshot as a URL rather than as bytes, so this fetches it and hands back a stored file that a later step can attach or upload. ## Frequently Asked Questions ### What can FlowRunner do with ScrapeNinja? FlowRunner agents can run Scrape, Scrape With Browser, and Scrape With Browser V2 in ScrapeNinja, plus 4 more actions. ### Does connecting ScrapeNinja to FlowRunner require OAuth? No. ScrapeNinja connects to FlowRunner with an API key, no OAuth flow required. ### Can ScrapeNinja trigger a FlowRunner workflow automatically? ScrapeNinja doesn't currently expose triggers in FlowRunner. It connects as an action step inside workflows started by another trigger. **Work at ScrapeNinja?** This integration exposes ScrapeNinja to AI agents on every FlowRunner plan, including through MCP, at no cost to you. [See what FlowRunner offers integration partners](https://flowrunner.ai/integrations/partners), including how to keep this page current. --- Markdown version of https://flowrunner.ai/integrations/scrapeninja. Site index: https://flowrunner.ai/llms.txt