The own.miami crawler

Identification

Requests from Own Miami carry the user agent own.miami-bot/1.0 (+https://own.miami/bot). It fetches public pages that document Miami condo buildings — developer and association records, official datasets, news and event listings — to keep building profiles accurate. It never logs in to a site, never submits forms, and never reads content behind a paywall.

Pace and rules

• It reads robots.txt first and honours it, including crawl delays.
• At most one request every eight seconds to any host, usually far fewer; known pages are not re-fetched.
• Fetched text is retained for 14 days for extraction and then discarded; we store only short excerpts with a link to the source.

Opt out or ask a question

Add User-agent: own.miami-bot with Disallow: / to your robots.txt, or email [email protected] and we will exclude your host by hand. For how AI assistants may use our data in turn, see /for-ai.