ScrapingBee
Developer ToolsScrape JavaScript-heavy pages through ScrapingBee with premium proxies, country geolocation, screenshots, and structured extraction. Agents pull structured Google SERP data and check remaining credits before launching large scraping jobs.
What This Integration Enables
Scraping is an adversarial, metered activity, and most teams only plan for the adversarial half. The web fights back with JavaScript walls, geo-restrictions, and bot detection, and ScrapingBee's answer, headless rendering, premium and stealth proxies, country geolocation, is genuinely the right toolkit. But every escalation up that toolkit costs multiples in credits, which means the real operational question is not "can we get this page" but "is this page worth what it costs to get." FlowRunner turns that question into a workflow instead of a surprise on the invoice. - Extract HTML or structured data from JavaScript-heavy sites through CSS and XPath extract rules - Pull structured Google SERP data, organic results, ads, and related searches, for rank tracking and research - Route hard pages through premium or stealth proxies, geolocated by country, on explicit approval - Capture full-page or viewport screenshots as visual evidence alongside extracted data - Check remaining credits and concurrency before large jobs, and alert before the pool runs dry The result is scraping run like an operations function: budgeted, monitored, and escalated by a person when the [automation hits an exception](/concepts/automation-exceptions) it cannot price on its own.
Without FlowRunner
With FlowRunner
Use Case Scenarios
Rank tracking that runs itself
Every Monday, the agent calls Scrape Google for the keyword set, geolocated to each market the team sells in. Organic results come back structured: title, URL, description, position. Each row appends to a rankings sheet in [Google Sheets](/integrations/google-sheets), and the agent diffs against last week before posting a digest to [Slack](/integrations/slack) with new entrants and dropped pages called out. The team reads position changes over coffee instead of exporting SERP screenshots into a deck.
Competitor pages into structured intelligence
The agent calls Scrape URL against competitor pricing and product pages with JavaScript rendering on and extract rules mapping price, plan names, and feature lists into JSON. The structured payload goes to [Claude](/integrations/anthropic-ai) to classify what changed and whether it matters, and material changes land in the channel with the extracted evidence attached. A screenshot captured in the same call preserves what the page actually looked like, which settles the "are we sure" conversation before it starts.
The credit pool as a managed resource
On a daily schedule, the agent calls Get Usage and compares consumed credits against the billing period's ceiling. Crossing a threshold posts a warning to the team channel before the next large job runs, not after it fails. When a big one-off crawl is proposed, the agent estimates its credit cost from page count and rendering options, checks the remaining pool, and asks the owner to confirm the spend. The plan's concurrency limit feeds the flow too, so parallel jobs queue instead of tripping over each other.
Human-in-Loop Highlight
The expensive decision in this connector is escalation. A page that blocks a standard request can usually be beaten with stealth proxies, but stealth requests draw down the same finite credit pool many times faster than plain fetches, and a retry loop that escalates automatically will drain a month's budget on a handful of stubborn domains. So FlowRunner agents treat the escalation as a purchase that needs a signature. Blocked pages queue with the failure reason, the credit price of the stealth retry, and the pool's current balance from Get Usage. The owner approves the pages that matter and drops the ones that do not. The agent handles the fetching, rendering, and extraction at machine speed; the decision to spend disproportionate credits on one hard page belongs to the person who owns the budget.
Agent Capabilities
3 actionsScraping
2- Scrape URL Fetches a page through ScrapingBee's proxy network and returns the HTML, with options to render JavaScript in a headless browser, route through premium or stealth proxies, geolocate by country, wait for a CSS selector, block ads, extract structured data with CSS or XPath rules, capture a screenshot, or wrap the whole result in JSON. With extract rules enabled, the result arrives as structured data instead of markup.
- Scrape Google Runs a Google search through the Google Search API and returns structured SERP data: organic results with title, URL, description, and position, plus ads, related searches, and metadata. Supports country geolocation and a result limit.
Account
1- Get Usage Returns the account's plan limits and consumption for the billing period, including maximum credits, concurrency, and how much is used. The check that runs before every large job and feeds the escalation gate.
Frequently Asked Questions
What can FlowRunner do with ScrapingBee?
FlowRunner agents can run Scrape URL, Scrape Google, and Get Usage in ScrapingBee.
Does connecting ScrapingBee to FlowRunner require OAuth?
No. ScrapingBee connects to FlowRunner with an API key, no OAuth flow required.
Can ScrapingBee trigger a FlowRunner workflow automatically?
ScrapingBee doesn't currently expose triggers in FlowRunner. It connects as an action step inside workflows started by another trigger.
Start building with ScrapingBee
$100 in credits. No card required. Connect in minutes.