Skip to content
bold.

Legal

Bot Disclosure

In short

We run a small, polite crawler that covers public commercial and residential property listings to keep our acquisition score current. It honours robots.txt, never touches login-walled pages or personal data, and you can have your domain excluded by email.

What our crawler does

The engine behind Bold weighs the public property market so our acquisition score stays current. A small crawler fetches publicly available commercial and residential listings, the same pages anyone can open in a browser, and keeps our picture of the market up to date.

It fetches listing pages only. It never submits forms, never signs in, never posts or interacts with a site, and never downloads anything that is not linked from your public navigation.

How to identify it

Every request the crawler makes carries this user-agent string. You will see it in your logs and can apply rules to it. If a request claims to be us but does not match it exactly, it is not ours:

BoldEngineBot/1.0 (+https://boldpropertygroup.com.au/bot/)

Crawl rate

The crawler is deliberately light. It is rate-limited per hostname, fetches only a handful of pages per visit, and runs off-peak where we can. The aim is to stay invisible to your server, not to add load.

  • Rate-limited to one request at a time per hostname, with a pause between requests
  • A handful of pages per visit, not a full-site crawl
  • Scheduled off-peak where practical; visit cadence is weekly at most, and lighter for smaller sites

What we respect

We follow the standard rules for well-behaved crawlers.

  • robots.txt is honoured. A blanket or path-specific disallow stops the crawler immediately
  • Rate limits and crawl-delay directives are observed
  • We do not read login-walled or members-only areas, and we do not collect personal data beyond what is published openly on a listing

Opting out

If you would rather we did not visit your site, paste this into your robots.txt, or email info@boldpropertygroup.com.au with the domain name. We will add it to our permanent skip list within one business day:

User-agent: BoldEngineBot
Disallow: /