Olostep

Olostep is a scraping, crawling, and search API built for AI pipelines. Agents scrape single pages with structured extraction, map and crawl whole sites, batch up to 100,000 URLs, search the web, get retrieval-backed answers, and monitor pages for changes.

Verified 35 actions API key available

Olostep website Platform Documentation Capability data verified 2026-08-25

A weekly market scan starts on schedule

Get Credit Balance confirms the team may still spend, since usage can be blocked while the balance reads healthy

Map Website returns every URL on the target site without scraping any of them

The page count is compared against the budget this scan was given

Start Batch scrapes the selected URLs asynchronously and returns their retrieve ids

Retrieve Content pulls each page in the format the downstream model expects

The extracted rows land in the warehouse and a digest posts to the team channel

A scan that would exceed its budget stops here for the owner to raise the budget or narrow the scope

What This Integration Enables

Olostep is a scraping, crawling, and search API built for pipelines that feed models rather than for people clicking through a builder. That shows up in its shape. A scrape can come back as Markdown for a model or HTML for a parser or JSON with fields already extracted. A batch handles URL counts no loop of single calls would ever match. A search can fetch the pages of its own results in the same call. And every product surface, a crawl page, a batch item, a standalone scrape, hands back a retrieve id that one operation reads, which is what lets you write the retrieval half of a pipeline once.

The other thing it means is that scale and spend are the same control. Every call costs credits, and the operations that keep running after your workflow ends, monitors, schedules, and agents, are standing commitments rather than requests. FlowRunner agents are good at the parts that reward machine speed here: mapping before crawling, correlating batch items by custom id, checking the failure count rather than the batch status, and reading the credit balance on a schedule. The parts that commit money get a person, and that is what the human-in-the-loop section is about.

Without FlowRunner

Scale and spend are the same dial, unlabeled A crawl configured generously bills for every page it reaches, read or not

Content scattered across product surfaces A crawl page, a batch item, and a one-off scrape each expose content differently

Standing jobs nobody re-approves A monitor or a schedule set up once keeps running and keeps costing after the project ends

With FlowRunner

The cheap read before the expensive one Agents map a site before crawling it, so the cost is known before it is incurred

One retrieval path Everything is addressed by retrieve id, so a pipeline is written once regardless of how the page was found

Standing commitments stay visible Monitors and schedules are listed, reviewed, and paused rather than quietly accumulating

Use Case Scenarios

Price and catalog monitoring that knows its own cost

Before a competitor scan runs, the agent calls Map Website on each target domain, which returns every URL without scraping any of them. That is both far cheaper than crawling and the only honest way to know what a crawl would have cost. The agent filters the map to the product paths, submits them with Start Batch and a custom id per URL, then polls Get Batch. It reads the failed item count rather than the status, because a batch reports complete with failures inside it and a status-only check treats a half empty result as a success. List Batch Items and Retrieve Content pull the pages, and the extracted rows append to BigQuery or Snowflake for the pricing review.
Research questions answered against the live web

A question arrives in Slack that needs current facts rather than an index from last quarter. The agent calls Get Answer with a JSON shape describing the fields it needs back, and Olostep searches, scrapes, and reads live pages to fill them in. Because the answer arrives as fields rather than prose, the workflow can branch on it. When the question is broader, the agent uses Search Web with scrape formats supplied, so each result comes back with its page content in one call rather than as a search followed by a batch. The finished brief lands in Notion with its sources.
Watching a page without watching a page

A team needs to know when a supplier's terms change, when a competitor's pricing page moves, or when a new company appears on a launch directory. Create Monitor takes that as a plain sentence and builds the pipeline from it. The agent registers the monitor, then reads List Monitor Events on a schedule rather than depending on a webhook arriving, which is also where a monitor that is quietly erroring shows up. During the review, List Monitors surfaces anything stuck provisioning or still watching a page that no longer exists, and Pause Monitor stops the spend without losing the baseline the way a delete would.

Human-in-Loop Highlight

Purchase Credit Top-Up charges the card already on file, and the vendor ships no confirmation step in front of it. There is also a state that makes this operation tempting to automate: Get Credit Balance can report a healthy number of credits while usage is not allowed, which is exactly the condition in which every scrape starts failing and the balance looks fine. An agent noticing a stalled pipeline is one call away from fixing it by spending money. So it does not. It posts what it found and leaves the decision where it belongs: "Batch 4471 is failing on every item. Usage is currently blocked. The balance reads 6,200 credits, of which 5,000 expire on the 14th. Olostep accepts four top-up amounts: 10,000, 20,000, 80,000 or 100,000. Which, or should the scan wait?" The operator picks. The agent had the diagnosis, the balance breakdown, and the exact options. It did not have permission to reach for the card.

Agent processes routinely

Detects exception requiring judgment

Clear match Continues automatically

Ambiguous Routes to human via preferred channel

Human decides

Agent resumes with decision

Agent Capabilities

35 actions

Scraping

Scrape URL Fetches one page as Markdown, HTML, plain text, structured JSON, a PDF, or a screenshot. Browser actions such as waiting, clicking, filling an input, or scrolling run before the capture, which is how a lazy loading list is reached.
Get Scrape Returns a previously created scrape with its content. There is no route that lists scrapes, so the id from the original call is worth keeping.
Retrieve Content Fetches the stored content for any scrape, crawl page, or batch item by its retrieve id, in whichever formats you ask for. The one operation that unifies every product surface, and the one the vendor now recommends over each product's own content fields.
List Countries Returns the country codes available for a given service and parser. It reports what one parser supports rather than a global list.

Crawling and Mapping

Start Crawl Crawls a site from a starting URL and scrapes every page it reaches. It returns immediately and keeps running, and it bills per page, so the include pattern is what keeps a faceted search from generating an unbounded page count.
Get Crawl Returns a crawl's status and page count. Poll it until the status reads completed.
List Crawl Pages Returns the pages a crawl found, with a cursor for paging. Take each page's retrieve id and read the content with Retrieve Content, since the content fields on this route are deprecated.
Map Website Returns every URL on a website without scraping any of them. Far cheaper than a crawl, and the right way to learn what a crawl would cost before committing to it.

Batches

Start Batch Scrapes a list of URLs asynchronously at a scale no loop of single calls will match. Give every item a custom id, because results come back in no guaranteed order.
Get Batch Returns a batch's progress. Read the failed item count as well as the status, because a batch reports complete with failures inside it.
List Batch Items Returns the items in a batch with their custom ids, URLs, and retrieve ids. It reports outcomes rather than page bodies.

Search and Answers

Search Web Runs a natural language web search and returns structured results. Supplying scrape formats brings each result back with its page content, which replaces a search followed by a batch.
Get Search Returns a previously created search by its id.
Get Answer Answers a question by searching, scraping, and reading the live web rather than a stale index. Supply a JSON shape and the answer arrives as fields a workflow can branch on.

Monitors

Create Monitor Sets up a recurring watch on a page or a query and alerts when it changes. The query is plain language, and Olostep builds the pipeline from it. It then runs on a schedule and spends credits every run.
List Monitors Returns your monitors with their status, query, and schedule. The review that catches one stuck provisioning or still watching a page that no longer exists.
Get Monitor Returns one monitor with its status, schedule, and generated specification.
Update Monitor Changes a monitor's query, schedule, or alerting. Changing the query rebuilds the pipeline and can reset the baseline, so the next run may report a difference that is really a new way of reading the page.
Pause Monitor Stops a monitor running without deleting it, which stops its credit consumption and keeps its baseline.
Resume Monitor Starts a paused monitor again. Its first run compares against the old snapshot, so a long pause arrives as one large change.
Delete Monitor Removes a monitor permanently along with its event history and baseline snapshot.
List Monitor Events Returns what a monitor has detected: each run, whether it saw a change, and what changed. The record to read in a workflow rather than waiting on a callback.

Schedules

Create Schedule Schedules any Olostep call to run on a cron expression or at a fixed time, taking the method, endpoint, and payload you would have sent. Every scheduled run costs credits exactly as a manual call would.
List Schedules Returns your scheduled calls, optionally including deleted ones so you can audit what used to be running.
Get Schedule Returns one scheduled call with its cron expression, payload, and next run time.
Delete Schedule Removes a scheduled call, stopping it and its credit consumption.

Files

Create File Starts a JSON file upload and returns a pre-signed URL to send the content to. This is the first of three steps, and a file that is never completed stays unusable.
Complete File Upload Marks an uploaded file as finished, which is what makes it usable by anything else.
Get File Returns a file's metadata and status, but not its content.
Get File Content Returns a time limited download URL for a file's content. The link expires, so fetch it rather than storing it.
List Files Returns your uploaded files, optionally narrowed to one purpose.
Delete File Removes an uploaded file. Anything still referencing it, such as a scheduled batch, starts failing.

Agents

Create Agent Creates a research agent from a plain language prompt that can search, scrape, crawl, write to a destination, and run on a schedule. It runs on its own afterwards and consumes credits every run, so it is a standing commitment rather than a request.

Billing

Get Credit Balance Returns your remaining credits, whether the team may still spend them, and the breakdown by purchase lot with expiry dates. The spending flag can be false while the balance is non-zero, which is the state where everything fails and the number looks fine.
Purchase Credit Top-Up Buys additional credits on the card already on file. It spends real money with no confirmation step, which is what this page's human gate exists for.

Frequently Asked Questions

What can FlowRunner do with Olostep?

FlowRunner agents can run Scrape URL, Get Scrape, and Retrieve Content in Olostep, plus 32 more actions.

Does connecting Olostep to FlowRunner require OAuth?

No. Olostep connects to FlowRunner with an API key, no OAuth flow required.

Can Olostep trigger a FlowRunner workflow automatically?

Olostep doesn't currently expose triggers in FlowRunner. It connects as an action step inside workflows started by another trigger.

Start building with Olostep

Free plan, no card required. Connect in minutes.

Start Building Free Talk to us