API reference
Internal scraping API for e-commerce and social platforms. All responses are JSON unless noted. Base URL below assumes a local run — substitute your deployment host.
Authentication
Every endpoint requires an X-API-Key header matching the server's
SERVER_API_KEY. Requests without a valid key get 401
before reaching any handler. Only this page and /health are open.
If the server starts without SERVER_API_KEY set it fails closed —
every guarded request is rejected.
Setup
export BASE='http://localhost:8000'
export KEY='<your SERVER_API_KEY>'
curl -H "X-API-Key: $KEY" "$BASE/browser/profiles"
Rejected
{"error": "Unauthorized", "message": "Invalid API key"}
Status codes
| Code | Meaning |
|---|---|
| 200 | Success. |
| 400 | Missing or invalid parameter. |
| 401 | Missing/invalid X-API-Key, or the server has no key configured. |
| 404 | Nothing found for the given input. |
| 409 | Requested browser profiles are all in use. |
| 500 | Unhandled server error. |
| 502 | Upstream platform fetch failed. |
Meta
Liveness probe. Returns the running environment.
No parameters.
Request
curl $BASE/health
Response
{"status": "active", "environment": "PRODUCTION"}
Scraping
Scrape a single product/profile URL. The platform client is picked from the URL's domain: Flipkart, Meesho, Amazon, IndiaMart, Shopsy, Facebook, Snapdeal, Instagram, Myntra, Pinterest, Ajio. An unknown domain returns {} with 200.
| Parameter | Type | Required | Description |
|---|---|---|---|
| url | string | required | Target URL. Query string takes priority over the JSON body. |
| screenshot | bool | optional | Capture a screenshot. Query string only. Default true. |
Request
curl -X POST '$BASE/scrape?screenshot=false' \
-H 'X-API-Key: $KEY' -H 'Content-Type: application/json' \
-d '{"url": "https://www.flipkart.com/.../p/itm..."}'
Response
{"title": "...", "price": "...", "screenshot": "https://s3/..."}
Fetch Flipkart seller details by product id or product URL. Needs at least one of pid / url.
| Parameter | Type | Required | Description |
|---|---|---|---|
| pid | string | optional | Flipkart product id. Required unless url is given. |
| url | string | optional | Product URL. Required unless pid is given. |
| sellerName | string | optional | Narrow the result to one seller. |
Request
curl -H 'X-API-Key: $KEY' '$BASE/seller-details?pid=ITMXXXXXXXX'
Response
{"sellerName": "...", "rating": "...", "address": "..."}
Find where a product is sold, from an image and/or a title. The image is matched by Google Lens visually — on pixels, not text — so it works on photos with no legible label. Needs at least one of url / title. Passing both is strongest: the image identifies the product and the title pins the variant (pack size, shade), which pixels cannot express. site tags hits on a registered shop and drives title-only search; registered sites: nykaa. One image per request. Note: image lookups drive a headed browser — expect ~10–20s, not milliseconds.
| Parameter | Type | Required | Description |
|---|---|---|---|
| url | string | optional | Image URL (alias image). Required unless title is given. One per request. |
| title | string | optional | Product title. Refines the image match; with no url it searches site directly and is then required to have one. |
| site | string|array | optional | Site(s), e.g. nykaa. Tags matching image hits; required for title-only lookups. |
| proxy | bool | optional | Route through the shared proxy pool. Default true. |
Request
curl -X POST '$BASE/product-lookup' \
-H 'X-API-Key: $KEY' -H 'Content-Type: application/json' \
-d '{"url": "https://cdn.example.com/listing.jpg", "title": "Colossal Kajal Super Black Pack of 2", "site": "nykaa"}'
Response
{"status": "success", "proxy": true, "failed": false, "sites": ["nykaa"], "input": "...", "query": "Colossal Kajal Super Black Pack of 2", "urls": [{"host": "www.nykaa.com", "url": "https://www.nykaa.com/.../p/33397"}, ...], "by_host": {"www.nykaa.com": ["..."], "www.amazon.in": ["..."]}, "matches": [{"site": "nykaa", "url": "..."}]}
Flipkart
Selenium-driven Flipkart search. Returns a CSV file download by default; pass json=true for a JSON body instead.
| Parameter | Type | Required | Description |
|---|---|---|---|
| query | string | required | Search term. Query string first, then JSON body. |
| page | int | optional | Pages to paginate. Default 25. |
| json | bool | optional | Return JSON instead of a CSV download. Default false. |
| proxy | bool | optional | Route through the configured proxy. Default false. |
| headless | bool | optional | Run the browser headless. Default true. |
| fullscreen | bool | optional | Maximise the browser window. Default false. |
| wait_until | int | optional | Extra wait in seconds. Default 0. |
Request
curl -H 'X-API-Key: $KEY' '$BASE/lid?query=running+shoes&page=5&json=true'
Response
{"result": 240, "data": [ ... ]}
Deep Flipkart search over the internal mobile API. Works around the ~1-2k deep-paging cap by replaying the query across price bands, brand facets and sort orders, then de-duping by pid. Always JSON, never a file. Always routed through the proxy pool (not client-configurable) — 400 if proxy env vars are unset.
| Parameter | Type | Required | Description |
|---|---|---|---|
| query | string | required | Search term. |
| strategy | string | optional | plain (default, shows the cap) | price | max. |
| fetch_details | bool | optional | Fetch full product details per row (much slower). Default false. |
| fetch_seller_count | bool | optional | Also fetch the true seller list per row via a dedicated API call (slower) — adds sellerCount and a full sellers array. Only applies when fetch_details is on. Default false. |
Request
curl -X POST '$BASE/flipkart/deep-search' \
-H 'X-API-Key: $KEY' -H 'Content-Type: application/json' \
-d '{"query": "men footwear", "strategy": "max", "fetch_details": true}'
Response
{"query": "men footwear", "strategy": "max", "count": 8431, "data": [ ... ]}
Parse one Flipkart product page: title, price, mrp, discount, rating, seller, images, productOrigin, breadcrumb — plus hero-image stats (pixel count, DPI, white-space %). The URL must be an https flipkart.com / flixcart.com address; anything else is rejected with 400 (SSRF guard).
| Parameter | Type | Required | Description |
|---|---|---|---|
| url | string | required | https flipkart.com product URL. |
| pincode | int | optional | Delivery pincode used for the lookup. Defaults to the module default. |
Request
curl -H 'X-API-Key: $KEY' '$BASE/flipkart/product-details?url=https://www.flipkart.com/.../p/itm...'
Response
{"url": "https://...", "data": {"title": "...", "productOrigin": "India", "imageDpi": 72}}
Browser pool
Open headed Chrome on signed-in profile(s) so an operator can log in manually. Blocks for the session duration. Returns 409 when fewer profiles are free than requested.
| Parameter | Type | Required | Description |
|---|---|---|---|
| profile | string | optional | Exact profile (e.g. profile_2). Without it, profiles are picked from the free pool. |
| open_count | int | optional | How many free profiles to open sequentially. Default 1. |
| timeout | int | optional | Seconds each session stays open. Must be 5–600. Default 60. |
| proxy_enabled | bool | optional | Launch behind the proxy. Default false. |
Request
curl -X POST '$BASE/browser/setup' \
-H 'X-API-Key: $KEY' -H 'Content-Type: application/json' \
-d '{"profile": "profile_2", "timeout": 120}'
Response
{"status": "success", "pool": "signed_in", "sessions": [{"session": 1, "profile": "profile_2", "success": true}]}
Live snapshot of the signed-in profile pool, read straight from the lock files.
No parameters.
Request
curl -H 'X-API-Key: $KEY' '$BASE/browser/profiles'
Response
{"in_use": 1, "profiles": [{"profile": "profile_1", "in_use": true}]}
Force-release profile locks left behind by a crashed session.
| Parameter | Type | Required | Description |
|---|---|---|---|
| profile | string | optional | Release just this profile. |
| stale_only | bool | optional | Only release locks whose owner is gone. Default true; false releases every lock. |
Request
curl -X POST '$BASE/browser/profiles/release' \
-H 'X-API-Key: $KEY' -H 'Content-Type: application/json' \
-d '{"stale_only": false}'
Response
{"status": "success", "released": ["profile_1"]}