Oxylabs
Developer ToolsGive AI agents web data collection at scale through the Oxylabs Scraper API. Agents scrape any URL or dedicated targets like Google and Amazon with JavaScript rendering and structured parsing, and run large crawls as asynchronous batch jobs they poll to completion.
What This Integration Enables
Oxylabs is web data collection at industrial scale: a scraper API backed by a proxy network that handles JavaScript rendering, geo-localization, and the adversarial reality of scraping sites that do not want to be scraped. The dedicated sources are the difference in kind. Pointing google_search or amazon_product at a query returns structured JSON, parsed by scrapers maintained against those exact page layouts, which turns the most volatile part of scraping, the parsing, into someone else's full-time job instead of yours. FlowRunner agents run the full lifecycle. Quick lookups go through the synchronous endpoint; large crawls run as asynchronous batches the agent submits, polls, and collects without a human babysitting the queue. - Scrape (Realtime) answers short requests in one call, with rendering, parsing, and geo_location options - Submit Scrape Job (Async) handles slow or heavy targets that would blow past a synchronous timeout - Submit Batch Jobs fans one configuration across hundreds of keywords, ASINs, or URLs, each as its own trackable job - Job statuses progress through pending, processing, done, and faulted, so flows branch on outcomes instead of guessing - Batch scope and spend run through [human-in-the-loop](/concepts/human-in-the-loop) authorization before submission
Without FlowRunner
With FlowRunner
Use Case Scenarios
Marketplace prices watched at catalog scale
An e-commerce team tracks its full ASIN list. Nightly, the agent runs Submit Batch Jobs against amazon_product with parsing on, polls to completion, and writes price, availability, and rating per ASIN to [Google Sheets](/integrations/google-sheets). The diff against yesterday drives the morning digest in [Slack](/integrations/slack): who moved, who went out of stock, whose rating slid. Repricing decisions stay with the category manager, made from data that was collected while the team slept.
Search visibility tracked like a metric, not a ritual
Marketing tracks a keyword set across markets. The agent submits google_search jobs per keyword with geo_location set per market, parses the structured SERP results, and records where the company and its competitors rank. Week-over-week movement posts to the channel; a competitor suddenly outranking the company on a money keyword pages the SEO lead the same morning. The rank-tracking subscription becomes a workflow the team controls end to end.
JavaScript-heavy sources turned into structured records
Competitor dealer locators and careers pages render entirely client-side. The agent uses Scrape (Realtime) with Render enabled through the universal source, passes the resulting HTML to an extraction model for structured fields, and appends new locations and job postings to the market intelligence base in [Notion](/integrations/notion). Expansion signals, new cities, new sales roles, surface as weekly deltas a human analyst reviews, instead of quarterly surprises.
Human-in-Loop Highlight
Submit Batch Jobs is the commitment on this page: one call can queue thousands of jobs, each metered, each rendering and parsing on Oxylabs infrastructure, and once the batch is submitted the spend is spoken for. A loop that resubmits a batch on a malformed condition, or an agent that expands "tracked ASINs" from eight hundred to eighty thousand because an input sheet changed shape, commits real money at API speed. So batch submission is gated on scope. The agent assembles the run and presents it: "Nightly price sweep ready: 1,240 amazon_product jobs, parse enabled, no rendering. Yesterday's run was 1,238. Submit?" A batch that suddenly triples gets a question instead of a submission. The same review covers option changes that multiply cost, enabling JavaScript rendering across a whole batch is a different bill than parsing alone. Collection runs at whatever scale the job needs; the decision to run at that scale is made by someone who can read an invoice.
Agent Capabilities
5 actionsScraping
5- Scrape (Realtime) Submits a scraping job and returns results synchronously in one request. Takes a source, universal for any URL or a dedicated scraper such as google_search, amazon_search, or amazon_product, with optional JavaScript rendering, structured parsing on supported targets, geo_location, and user agent type. Best for short-running requests.
- Submit Scrape Job (Async) Submits a job through the Push-Pull endpoint, returning a job id with pending status immediately. Accepts the same source, rendering, and parsing options as the realtime action, plus an optional callback URL Oxylabs notifies on completion. The path for large, slow, or many-page targets.
- Get Job Status Retrieves an asynchronous job's status and metadata as it progresses through pending, processing, done, or faulted. The polling step; it does not return scraped content.
- Get Job Results Retrieves a completed job's output: raw HTML or structured JSON when parsing was enabled, with the final URL and HTTP status per scraped page. Returns full content only after status reports done.
- Submit Batch Jobs Submits many jobs in one request from shared parameters and a list of queries or URLs, each entry becoming its own trackable job id. The scale lever for keyword sets, ASIN lists, and URL inventories, and the operation this page's human gate sizes before submission.
Frequently Asked Questions
What can FlowRunner do with Oxylabs?
FlowRunner agents can run Scrape (Realtime), Submit Scrape Job (Async), and Get Job Status in Oxylabs, plus 2 more actions.
Does connecting Oxylabs to FlowRunner require OAuth?
No. Oxylabs connects to FlowRunner with basic authentication (a username and password), no OAuth flow required.
Can Oxylabs trigger a FlowRunner workflow automatically?
Oxylabs doesn't currently expose triggers in FlowRunner. It connects as an action step inside workflows started by another trigger.
Start building with Oxylabs
$100 in credits. No card required. Connect in minutes.