BystepBot

BystepBot: who we are and how to control it

BystepBot is Bystep's web reader. It only runs when a Bystep user asks for a public site to be dissected into a functional summary (features, flows, pricing, structure) as inspiration for their own product. Bystep does not copy your brand, text, images or look.

Updated 24 September 2026

How to recognise BystepBot

Every request carries the BystepBot token in its User-Agent and comes from the same IP address as bystep.dev.

HTML reading

Mozilla/5.0 (compatible; BystepBot/1.0; +https://bystep.dev/bot)

Isolated browser (screenshots, JavaScript pages)

Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/150.0.0.0 Safari/537.36 BystepBot/1.0 (+https://bystep.dev/bot)

What BystepBot does

Reads once, when a user asks: the entered page, then at most 12 public pages of the same site (picked from its navigation, sitemap and llms.txt).
Opens one page at a time with at least 0.5 seconds between requests, and never dissects the same site twice at once.
Honours robots.txt (RFC 9309) for every page, the first one included. Crawl-delay is honoured up to 5 seconds; when robots.txt errors or cannot be reached, BystepBot reads nothing.
Honours Content-Signal: when a site says ai-input=no, its content is not sent to an AI model and the analysis stops.
Opens at most 4 pages in an isolated browser for screenshots and pages that need JavaScript. Downloads, service workers and media are switched off.
Government sites are read in light mode: at most 3 pages, at least 3 seconds apart, no browser.

What it never does

Log in, create accounts or use anyone's cookies.
Fill in or submit forms, or press buy, sign-up or pay buttons.
Bypass captchas, paywalls, bot challenges or firewalls. As soon as such a page appears, BystepBot stops.
Read sign-in, checkout or payment pages, or the sites of banks, e-wallets and payment providers.
Change its User-Agent or IP address to get around a site's rules.
Train Bystep AI models on your content.

How to refuse BystepBot

The fastest way is robots.txt. The rule applies to the next analysis, no need to contact us.

robots.txt: refuse BystepBot completely

User-agent: BystepBot
Disallow: /

robots.txt: allow reading, refuse use as AI input

User-agent: *
Content-Signal: search=yes, ai-input=no, ai-train=no
Allow: /

With Content-Signal ai-input=no, BystepBot stops the analysis before any text is sent to an AI model.

Want your domain excluded entirely, or to report behaviour that does not match this page? Email hello@bystep.dev

What is kept

A rewritten functional summary (features, flows, pricing, structure, detected technology), only in the plan of the user who asked.
Emails, phone numbers, ID numbers and tokens are masked before any text is stored or sent to an AI.
Screenshots and the page text that was read are deleted automatically after 7 days.
Raw HTML is not stored. Masked text is only sent to the AI model provider Bystep uses for that analysis, and is never sold.

Contact: hello@bystep.dev