Lobstr.io
Developer ToolsLobstr.io runs a catalog of ready-made crawlers for sites like Google Maps, LinkedIn, and Amazon. Workflows configure squids, add tasks, start and schedule runs, collect results, and react when a run finishes or fails.
What This Integration Enables
Lobstr.io is not a scraping framework, and reading it as one is the fastest way to misuse it. You do not build crawlers here. Lobstr maintains a catalog of ready-made ones for the sources people actually want, and what you build is a squid: your configured instance of a crawler, carrying its parameters, its task list, its schedule, its delivery target, and any linked platform account. A run is one execution of a squid over its tasks. Nothing is collected until all three of those exist, which is why a squid created and left alone produces exactly nothing and looks broken.
The consequence is that this connector is less about extraction than about a standing commitment. A squid on a schedule runs with nobody watching, bills per record scraped, and can chain into a downstream squid automatically. FlowRunner agents drive the whole cycle: configure squids against a crawler's real parameter names, add or upload tasks, start and monitor runs, read the credit breakdown, retry a single failed task rather than the whole run, and collect results through the delivery channel or straight from the API. What FlowRunner adds that a cron job does not is that the money and the credentials in this loop reach a person. The human-in-the-loop gate on this page is aimed squarely at the one setting that lets an unattended system spend on its own authority.
Without FlowRunner
With FlowRunner
Use Case Scenarios
-
A territory list that refreshes itself
A sales team works a defined geography, and the source is a mapping search rather than a purchased file. Get Google Maps Locations returns the location values Lobstr's Maps crawlers actually accept, because an invented city name produces no results and no error. The agent adds those as tasks, starts the run, and waits for On Run Done. Download Run returns the whole set in one file rather than paging the results endpoint, which matters because that endpoint is rate limited to a couple of requests per second. Records are matched against HubSpot before anything is created, so the CRM gains new accounts rather than duplicates, and the run summary posts to the territory channel in Slack.
-
Enrichment that stays inside the guardrails
Some crawlers need a linked platform account, and Sync Account is how that link is made: it posts a platform's session cookies to Lobstr so the crawler can act as that user. Those cookies are that account's credentials, and platforms may suspend an account they detect being automated. So the flow uses a purpose-made account rather than a real seller's, sets Update Account Limits conservatively, and checks the linked account's health with Check Sync Status before every scheduled run rather than after a failure. Collected records feed a sequence in Instantly, and the daily cap on the account is the number the agent respects rather than the number it maximizes.
-
The run that finished and did not
Lobstr reports a run as DONE even when most of its tasks were skipped, delivery to a misconfigured sheet or bucket fails without failing the run, and a chain fires only when the source run reaches DONE, so a run that errored triggers nothing and says nothing. Every one of those looks like a quiet system rather than a broken one. The agent treats them as automation exceptions instead:
- Get Run Stats is read for the failed and skipped counts rather than the status
- On Run Error is wired up, so a broken upstream scrape does not present as a downstream squid that has gone quiet
- Test Delivery runs before any channel is relied on
- Retry Run Task re-runs the individual failures, which is far cheaper than re-running the squid
Results land in Google Sheets with the run's own statistics attached, so the person reading them knows what the number is out of.
Human-in-Loop Highlight
Set Auto Billing Reset is the one operation here that lets the system buy itself more capacity. Turned on, crossing a credit threshold starts a new paid billing cycle without anyone approving it. That is exactly what a production pipeline wants and exactly how an unattended runaway bills repeatedly, and Lobstr's rolling daily cap is a backstop rather than a decision. It pairs with a second behavior that hides the problem: a squid that runs out of credit pauses mid-run rather than failing, which looks like a stuck job rather than an empty wallet.
FlowRunner agents therefore treat On Run Paused as a question, never as a retry. The agent gathers the evidence first, calling Get Balance for the current position and Get Run Credits for where the last run's money went, since optional crawler functions like email extraction and business-detail enrichment bill separately and one of them usually dominates. Then it stops: "The Maps squid paused at 4,180 of 11,000 tasks with the balance exhausted. The last completed run cost 9,240 credits, of which 6,900 was the email-extraction function. Finishing this run needs roughly another 14,000 credits, and Auto Billing Reset is currently off. Fund the cycle, turn the enrichment function off and resume, or abort the run?" Abort Run is the stop-spending button and it stays available to the person answering. The agent scoped the work, priced it, and named the options. It did not open the wallet, and Rotate API Key stays a human operation for the same family of reasons: it invalidates the configured key immediately, including FlowRunner's own connection, and shows the replacement once.
Agent Capabilities
47 actionsAccount and billing
5- Get User Profile Reads the account the API key belongs to.
- Get Balance Reports the remaining credit. The cheap check before a large run, because running out pauses a run rather than failing it.
- Get Auto Billing Reset Reads whether crossing the credit threshold starts a new paid billing cycle automatically.
- Set Auto Billing Reset Turns automatic billing cycles on or off. The operation this page's human gate exists for, because on means the system can buy its own credit unattended.
- Rotate API Key Issues a new API key and invalidates the current one immediately, including this connection. The replacement appears once, in the response.
Crawlers
5- List Crawlers Lists Lobstr's ready-made crawlers. You do not create these, you configure an instance of one.
- Get Crawler Reads one crawler, including whether it requires a linked platform account.
- Get Crawler Parameters Returns the real parameter names a crawler accepts. Worth reading, because an unrecognized key is ignored rather than rejected and looks identical to a setting that did not take effect.
- Get Crawler Attributes Returns the output fields a crawler produces, which is what a downstream mapping should be built against.
- Get Google Maps Locations Returns the location values the Maps crawlers accept. A free-text place name is not accepted, and an invented city produces no results and no error.
Squids
6- List Squids Lists the configured squids in the account.
- Get Squid Reads one squid's parameters, schedule, delivery, and linked accounts.
- Create Squid Creates a configured instance of a crawler. A squid with no tasks and no run has collected nothing.
- Update Squid Changes a squid's settings and crawler-specific parameters.
- Empty Squid Clears a squid's collected data while keeping its setup, which is usually what starting over means.
- Delete Squid Deletes a squid along with its results, tasks, schedule, and delivery integrations.
Chaining
4- Get Squid Chain Reads the chain configured on a squid. A squid can have one outgoing chain, and chains are capped at a depth of three.
- Configure Squid Chain Sets up a downstream squid to follow this one. With autostart on, the downstream run begins by itself.
- Update Squid Chain Changes an existing chain.
- Delete Squid Chain Removes the chain, leaving both squids independent.
Tasks
6- List Tasks Lists the tasks configured on a squid.
- Get Task Reads a single task.
- Add Tasks Adds tasks to a squid, which is what gives a run something to work through.
- Upload Tasks Uploads a task list as a CSV, for a set too large to add one at a time.
- Check Upload Status Reports how an upload was processed. A mostly-rejected file still reports as finished, so read the errors rather than the status.
- Delete Task Removes a task from a squid.
Runs
10- Start Run Starts a run over the squid's tasks. It bills per record scraped, with no confirmation step of its own.
- List Runs Lists runs with their statuses.
- Get Run Reads a single run.
- Get Run Stats Returns the done, failed, and skipped counts. These are the fields that matter, because a run reports DONE with most of its tasks skipped.
- Get Run Credits Returns what a run actually cost, broken down by the optional crawler functions that billed separately. Where an unexpected bill explains itself.
- List Run Tasks Lists the individual tasks in a run with their outcomes.
- Retry Run Task Re-runs a single failed task, which is much cheaper than re-running the squid.
- Resurrect Run Restarts a run that ended without completing its task list.
- Abort Run Stops a run in progress. The stop-spending button.
- Download Run Returns the run's whole result set as one temporary file URL. The right way to collect a large export, since the results endpoint is tightly rate limited.
Results
1- Get Results Reads collected records directly through the API. Rate limited to a couple of requests per second, so page carefully or use Download Run.
Platform accounts
7- List Account Types Lists the platforms an account can be linked for.
- List Accounts Lists the linked platform accounts on the workspace.
- Get Account Reads one linked account, including its limits.
- Sync Account Posts a platform's session cookies to Lobstr so account-based crawlers can act as that user. Those cookies are that account's credentials, and the platform may suspend an account it detects being automated, so use a purpose-made one.
- Check Sync Status Reports whether a linked account is still usable, which is worth checking before a scheduled run rather than after a silent one.
- Update Account Limits Sets the daily caps on a linked account. The main thing standing between that account and a suspension.
- Delete Account Removes a linked platform account.
Delivery
3- Get Delivery Reads a squid's configured delivery channels.
- Configure Delivery Sets where results are sent: a sheet, a bucket, a webhook, or another supported channel. A squid has a single webhook slot, so writing webhook fields on a squid a trigger is watching replaces the trigger's callback URL and stops it firing.
- Test Delivery Checks a delivery channel before a run depends on it. A misconfigured target does not fail the run: the scrape succeeds, the credits are spent, and the data goes nowhere.
Triggers
4 triggersEvent Triggers
4- On Run Done Fires when a squid's run completes. The payload carries the run id and the squid rather than the records, so the next step is Download Run or Get Results. This is also the only status that fires a configured chain.
- On Run Error Fires when a run ends in error. Worth wiring up even when a chain exists, because an errored run triggers nothing downstream and says nothing, so a broken upstream scrape presents as a downstream squid that has simply gone quiet.
- On Run Started Fires when a run begins. Useful for recording that a scheduled squid actually woke up, which is the other half of noticing that one did not.
- On Run Paused Fires when a run pauses, which is what happens when a squid exhausts its credit mid-run rather than failing. The signal behind this page's human gate.
Frequently Asked Questions
What can FlowRunner do with Lobstr.io?
FlowRunner agents can run Get User Profile, Get Balance, and Rotate API Key in Lobstr.io, plus 44 more actions.
Does connecting Lobstr.io to FlowRunner require OAuth?
No. Lobstr.io connects to FlowRunner with an API key, no OAuth flow required.
Can Lobstr.io trigger a FlowRunner workflow automatically?
Yes. Lobstr.io supports 4 triggers that can start a FlowRunner workflow automatically.
Other Developer Tools integrations
Start building with Lobstr.io
Free plan, no card required. Connect in minutes.