CLI

Fetch live pages from your terminal or your agent's shell: one command, the same API, no code.

Install

Run it without installing

npx @agentscraper-cc/cli login                               # paste your API key (not echoed)
npx @agentscraper-cc/cli fetch https://example.com -o text   # the page as Markdown

Install it

npm install -g @agentscraper-cc/cli
agentscraper --help

Node.js 18 or later, no other dependencies. Open source (MIT): https://github.com/agentscraper-cc/cli on GitHub, @agentscraper-cc/cli on npm.

Fetch pages

fetch

agentscraper fetch https://example.com/a https://example.com/b      # up to 10 URLs; JSON with results[] and statuses[]
agentscraper fetch https://example.com -o text                      # just the content (Markdown, else text, else HTML)
agentscraper fetch https://example.com -o html --html               # the page HTML
agentscraper fetch https://example.com --country-code de --render   # from Germany, rendered in a real browser
agentscraper fetch --data @request.json                             # the whole request body from a file (- for stdin)
agentscraper account                                                # plan, credits and rate limit
agentscraper fetch --help                                           # every option, with the API's own descriptions

A fetch is the same request as POST https://agentscraper.cc/api/v1/contents (API reference), charged the same way. One that is still running after about 100 seconds is collected automatically with the same Idempotency-Key, so it is charged once.

API key

agentscraper login saves the key in ~/.config/agentscraper/config.json (mode 0600). In CI or an agent's environment, set AGENTSCRAPER_API_KEY or pass --api-key instead. The key is only ever sent to the API base URL (https://agentscraper.cc/api, or --base-url).

Always current

The commands and options come from the API's OpenAPI document, which the CLI downloads again after an hour. When the API gains a field or an endpoint, the option is there on your next run, without updating the CLI. Offline, it uses the last copy it downloaded, then the copy bundled with the package (agentscraper spec shows which). New versions are published to npm automatically whenever the API changes.

Output and exit codes

Option	Prints
`-o json` (default)	The API's JSON response
`-o text`	The content of each page: Markdown, else text, else HTML
`-o <field>`	One field of each result, e.g. `-o title`

Exit code	Meaning
0	OK
1	The API answered with an error (JSON on stderr, with its `requestId`), or every URL failed with `-o text`
2	Usage: unknown command or option, or a value the API does not accept
3	Network: no answer. The error carries the `Idempotency-Key` to collect the result without a new charge