Skip to content

docs: deprecate Scraper/Leads/Screenshots API sections, add Smart AI Proxy usage #8

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
x-ror wants to merge 2 commits into
base: main
Choose a base branch
from
Open

docs: deprecate Scraper/Leads/Screenshots API sections, add Smart AI Proxy usage #8

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
75984f5
docs: mark Scraper/Leads/Screenshots API sections deprecated, add Sma…
Sep 2, 2026
ea15165
docs: fix Smart AI Proxy example port and Crawling API link
Sep 2, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 23 additions & 0 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -153,6 +153,8 @@ The library reads `cb_status` from the API response first, then falls back to `p

## Scraper API usage

> ⚠️ **Deprecated.** The standalone Scraper API has been closed to new sign-ups since October 1, 2024. Existing integrations continue to work and no shutdown is scheduled, but new code should use the Crawling API with the `scraper` parameter instead (same scrapers, simpler endpoint, more parameters). The class below stays available for backward compatibility. See the [scrapers documentation](https://crawlbase.com/docs/scrapers).

Initialize the Scraper API using your normal token and call the `get` method.

```ruby
Expand Down Expand Up @@ -180,6 +182,8 @@ end

## Leads API usage

> ⚠️ **Deprecated.** The Leads API has been closed to new sign-ups since October 1, 2024. Existing integrations continue to work and no shutdown is scheduled. There is no direct replacement; for similar workflows use the Crawling API with the [`email-extractor`](https://crawlbase.com/docs/scrapers/email-extractor) scraper (any URL → emails) or the [`google-serp`](https://crawlbase.com/docs/scrapers/google-serp) scraper for domain-scoped contact discovery. The class below stays available for backward compatibility.

Initialize with your Leads API token and call the `get` method.

For more details on the implementation, please visit the [Leads API documentation](https://crawlbase.com/docs/leads-api).
Expand All @@ -203,6 +207,8 @@ If you have questions or need help using the library, please open an issue or [c

## Screenshots API usage

> ⚠️ **Deprecated.** The standalone Screenshots API has been closed to new sign-ups since November 1, 2024. Existing integrations continue to work and no shutdown is scheduled, but new code should use the Crawling API with the `screenshot=true` parameter — same JS-rendering pipeline, screenshot parameters on the standard endpoint. The class below stays available for backward compatibility. See the [Crawling API documentation](https://crawlbase.com/docs/crawling-api).

Initialize with your Screenshots API token and call the `get` method.

```ruby
Expand Down Expand Up @@ -255,6 +261,23 @@ end

Note that `screenshots_api.get(url, options)` method accepts an [options](https://crawlbase.com/docs/screenshots-api/parameters)

## Smart AI Proxy usage

The [Smart AI Proxy](https://crawlbase.com/docs/smart-proxy) is a standard rotating HTTP(S) proxy endpoint, so it needs no SDK: point any HTTP client at `smartproxy.crawlbase.com:8012` (HTTP) or `smartproxy.crawlbase.com:8013` (HTTPS) with your token as the proxy username and an empty password. Crawlbase handles proxy rotation, retries and anti-bot bypass on its side.

```ruby
require 'net/http'

uri = URI('https://httpbin.org/ip')
proxy = Net::HTTP::Proxy('smartproxy.crawlbase.com', 8012, 'YOUR_TOKEN', '')
http = proxy.new(uri.host, uri.port)
http.use_ssl = true
http.verify_mode = OpenSSL::SSL::VERIFY_NONE
puts http.get(uri.request_uri).body
```

Note: the proxy re-signs HTTPS traffic, so certificate verification must be disabled on the client (as in the example). The example uses the plain-HTTP proxy port `8012` because stdlib `Net::HTTP` cannot open a TLS connection to the proxy itself; the request to the target still goes through the encrypted tunnel. See the [Smart AI Proxy documentation](https://crawlbase.com/docs/smart-proxy) for all options.

## Storage API usage

Initialize the Storage API using your private token.
Expand Down