The cmgraph crawler

We fetch under two named user agents, and never under a disguise:

cmgraph/1.0 (+https://cmgraph.com/bot)

Discovery: public directories, registries, and association member lists. Used to learn that companies exist — names, trading websites, addresses — never to extract capability claims.

cmgraph-research/0.1 (+https://cmgraph.com)

Verification: fetching a manufacturer’s own website to confirm its identity (legal name, address, registration number) and what it says about its capabilities, before anything about it is published.

• robots.txt is respected to the letter. We match the cmgraph token or the wildcard group, whichever your file addresses to us. If we are disallowed, we do not fetch — and we do not come back under a browser user agent instead.
• At most one request per second per host, and usually far less.
• No captcha bypass, ever. A site that asks not to be read by machines stays unread; we find public sources instead.
• Companies, not people. No LinkedIn scraping, no officer or shareholder records — cmgraph describes businesses, not individuals.
• What we use is cited. Facts on cmgraph carry the quoted source paragraph and a link to the page it came from. Attribution is how the product works, not a courtesy.

To allow our crawler explicitly, add to your robots.txt:

User-agent: cmgraph
Allow: /

To refuse, disallow us the same way — it is honoured without workarounds:

User-agent: cmgraph
Disallow: /

If your robots.txt disallows unnamed crawlers by default, we treat that as a “no” and won’t fetch — operators who would like us to read their site can add the allow rule above or simply email us permission.

Anything else — rate concerns, a removal request, or checking whether traffic in your logs was really ours — email hello@cmgraph.com with the timestamps and IP address and we’ll confirm either way.