Skip to content

AI products and agents, Retired

CrawlAPI

AI-powered web scraper (retired)

Describe in plain language what you need from a website; CrawlAPI scraped it and extracted the data. Now retired.

The AI/Web Crawler API demo running a scripted task on polestar.com: a step list on the left (click, wait, type) with the AI reasoning for each step, and the browser preview on the right as it opens the login page and fills in the email and password fields.
AI steps driving a real website

Stack

  • AI
  • NestJS
  • Node.js
  • LangChain
  • Message queue
  • React
  • TypeScript
Status
Retired
Type
AI products and agents

The problem

Writing a scraper usually means hand-coding selectors and logic for every site.

What it does

CrawlAPI was an AI-powered web scraper. You described the task in natural language, and the crawler scraped the site and extracted the data. It ran as a public beta and has since been retired; the demo below shows how it worked.

How it's built

A NestJS backend on Node.js with LangChain and a message queue, and a React front end, all in TypeScript.

Key features

  • Scraping tasks described in natural language
  • Data extraction from the pages it crawled
  • Ran as a public beta (now retired)

Screens

  • A CrawlAPI run near its end: the email and password are filled in on the Polestar login page and a red arrow marks the Inloggning (log in) button the AI located for the final Click step, with its reasoning in the step panel.
    AI locating the login button

Contact

Want something like this built?

Get in touchExperience