Apify LLM Scraper
Developer ToolsApify LLM Scraper runs the open-source Crawl4AI engine on Apify. Agents scrape a page or a whole site into Markdown, structured JSON by CSS or XPath selectors, or whatever an LLM prompt asks for, then collect the results when the run succeeds.
What This Integration Enables
The Apify LLM Scraper runs the open source Crawl4AI engine on Apify's infrastructure, and its whole shape is the pair of strategy choices it opens with. A crawl strategy decides how far past the start URLs to go, from a single page to a breadth first sweep to a best first crawl that ranks links against keywords so a large site can be read for one subject rather than exhaustively. An extraction strategy decides what comes back: Markdown, a JSON structure defined by CSS or XPath selectors, or whatever an LLM is instructed to pull out. Those two choices govern everything else, and several configuration objects only take effect under one of them.
That is what separates this connector from a page fetcher. It does not hand you a page, it hands you a record. Which is also the risk worth naming out loud: a selector that matched nothing and a model that hallucinated a plausible price both produce output that passes every structural check a workflow could run. A partly failed run still writes a dataset, and a run that hit its timeout mid crawl still returns pages. FlowRunner's answer is not more validation rules, it is human-in-the-loop at the one point where a person can actually tell: before the configuration that produced one questionable record is pointed at the whole list.
For turning a documentation site into clean Markdown for a retrieval pipeline, the Apify AI Website Crawler connector is the better tool. These two share an Apify account and a run model, and they answer different questions.
Without FlowRunner
With FlowRunner
Use Case Scenarios
-
Competitor pricing, with the extraction proved first
An analyst defines the fields a product page should yield: name, list price, currency, availability. The agent calls Scrape Pages against one known page with the JSON by CSS selector strategy and posts the record alongside the page's own Markdown. Once the analyst confirms the record is reading the price and not the crossed out anchor price, Start Scrape runs the same configuration across the list and On Scrape Succeeded delivers the dataset. Rows land in Google Sheets for the review and in BigQuery for the trend, and every row carries the run id that produced it.
-
Enriching inbound accounts from their own site
A new account arrives in HubSpot with nothing but a domain. The agent runs a best first crawl weighted toward the keywords that describe the segment, uses the LLM extraction strategy to pull the company's stated market, headcount signals, and product lines, and writes the summary back onto the record. Where the extraction returns nothing, the agent reads Get Scrape Run Log rather than reporting an empty company, because an empty result here is usually a content filter that pruned the page or a provider that refused the token, and those are different problems with different fixes.
-
A deep crawl that outran its own timeout
A research run over a large site fires On Scrape Timed Out, which this actor does often because its default run timeout is short relative to a deep crawl. The agent does not restart. It reads the dataset the run already wrote, calls Save Scrape Results To FlowRunner File so the partial output is preserved as one file rather than held in memory, and calls Resurrect Scrape Run with a longer timeout so the crawl continues into the same dataset instead of paying for the completed pages twice. The team sees one notification in Slack describing what was recovered.
Human-in-Loop Highlight
The vendor already built the gate on this connector and probably did not think of it that way. Scrape Pages runs synchronously against a page or a handful and hands the result straight back; Start Scrape points the same configuration at a whole site and spends accordingly. That split exists for convenience, and it is exactly the dry run a person needs, because an LLM extraction fails by producing a confident, well formed, wrong record. There is no status code for that. So the agent runs one page through Scrape Pages, puts the extracted record next to the source page it came from, and stops: "This configuration read a list price of 129 dollars from this page. The page also shows a crossed out 179 and a bundle price of 240. Is 129 the field you want, and should the same selectors run across the other 340 URLs?" The analyst answers, and only then does Start Scrape run. Everything after that point is the agent's: the crawl, the retries, the timeout recovery, the writes. What a person keeps is the judgment about whether the thing being extracted is the thing that was meant, made once, cheaply, before it is repeated hundreds of times and believed.
Agent Capabilities
17 actionsScraping
2- Scrape Pages Scrapes the given pages and returns the results, waiting for the run to finish. Apify holds a synchronous run open for a few minutes only, which makes this the right shape for a sample and the wrong shape for a deep crawl.
- Start Scrape Starts the scrape and returns immediately with a run id. The right shape for a deep crawl or an LLM extraction, both of which routinely outlast the synchronous limit.
Runs
6- Get Scrape Run Returns a run's status, timing, cost, and dataset id. Every outcome comes back as a healthy response, so the status field is what a flow branches on rather than the call succeeding.
- List Scrape Runs Returns this account's runs of the scraper. Old runs disappear once the plan's retention window passes.
- Get Last Scrape Run Returns the most recent run, optionally only one that ended in a given state. How a flow reaches the results of a run whose id was never stored.
- Abort Scrape Run Stops a run that is still going. The stop spending button for a crawl that turned out larger than expected, though pages already scraped are billed.
- Resurrect Scrape Run Restarts a failed, timed out, or aborted run and continues into the same dataset, keeping the pages already paid for. It cannot revive a run whose storage has passed retention.
- Get Scrape Run Log Returns the engine's own log as plain text. A content filter that pruned everything, a selector that matched nothing, and a model provider refusing the token all appear here and nowhere in the structured fields.
Results
5- Get Scrape Results Returns the pages a run scraped, from its dataset. Each result can carry raw HTML alongside the Markdown, so naming the fields a pipeline uses is what keeps the payload manageable.
- Get Dataset Items Returns a dataset's items by its own id rather than through a run. Used when a trigger payload or a stored dataset id is all a flow has.
- Save Scrape Results To FlowRunner File Downloads a run's results in the chosen format into FlowRunner's own file storage. The right shape for a deep crawl, and the practical way to produce CSV or XLSX.
- List Scrape Run Files Lists the keys in a run's key value store, which is where screenshots and PDFs go when the crawler configuration asked for them.
- Save Scrape Run File To FlowRunner File Downloads one record from a run's key value store, such as a captured screenshot or PDF, into FlowRunner's file storage.
Actor
2- Get Scrape Options Returns the scraper's published input schema with every option's type, default, and enumeration, plus the vendor's prefilled example for each configuration object. The authoritative reference for the engine's own inner fields.
- Get Scraper Info Returns the actor's record: version, pricing model, default run options, and statistics. Where the cost model is named before anything runs.
Account
2- Get Account Returns the Apify account the token belongs to, with its plan. The cheapest connection check available.
- Get Monthly Usage Returns this month's usage and cost broken down by what produced it, so a scheduled run can notice it is about to exhaust the plan before it does.
Triggers
4 triggersEvent Triggers
4- On Scrape Succeeded Fires when a scrape finishes successfully and carries the dataset id the results live in, which removes the polling loop a deep crawl or an LLM extraction otherwise needs. It fires for every run of this scraper on the account.
- On Scrape Failed Fires when a scrape fails. Results are written as pages complete, so the run's dataset may hold most of what was asked for and is worth reading before the run is treated as lost.
- On Scrape Aborted Fires when a run is stopped deliberately, in the Apify console or through Abort Scrape Run. A flow that retries on this one is fighting whoever pressed the button.
- On Scrape Timed Out Fires when a run hits its timeout, which matters more on this actor than most because its default run timeout is short and a deep crawl or an LLM extraction routinely exceeds it.
Frequently Asked Questions
What can FlowRunner do with Apify LLM Scraper?
FlowRunner agents can run Scrape Pages, Start Scrape, and Get Scrape Run in Apify LLM Scraper, plus 14 more actions.
Does connecting Apify LLM Scraper to FlowRunner require OAuth?
No. Apify LLM Scraper connects to FlowRunner with an API key, no OAuth flow required.
Can Apify LLM Scraper trigger a FlowRunner workflow automatically?
Yes. Apify LLM Scraper supports 4 triggers that can start a FlowRunner workflow automatically.
Other Developer Tools integrations
Start building with Apify LLM Scraper
Free plan, no card required. Connect in minutes.