Infrastructure
· 9 checks — DNS, redirects, IPv6, crawlability, URL variants, and domain intelligence rolled into one auditable list.BURL Variantswww/non-www, trailing slash, HTTP→HTTPSREVIEW
www / non-www
Inconsistent — duplicate content risk
HTTP → HTTPS
Consistent
BTLS Certificate Expiry & Recommendations85 days until leaf cert expires — 3 issues to addressREVIEW
Certificate validity
Recommended actions
- Enable HSTS: Strict-Transport-Security: max-age=31536000; includeSubDomains
- Enable DNSSEC on your domain for DNS spoofing protection
- Enable OCSP stapling on your TLS server to remove a CA roundtrip and protect user privacy
A+DNS Records2 A records, 7 ms lookupPASS
| A | 104.18.9.208, 104.18.8.208 |
| AAAA | 2606:4700::6812:9d0, 2606:4700::6812:8d0 |
| CNAME | — |
| NS | beth.ns.cloudflare.com, jeff.ns.cloudflare.com |
| MX | 1 aspmx.l.google.com 5 alt2.aspmx.l.google.com 5 alt1.aspmx.l.google.com 10 alt3.aspmx.l.google.com 10 alt4.aspmx.l.google.com |
| TXT | 9ab284e159654849be9c00070017a3fb E1349882DC MS=ms44993420 MS=ms52364032 "apps":[{"clientID":b45ea8246bc7eaa2f12cb0ce96400472} amazonses:XfUshZFMRKE2JOrdzaTumbNgGiHZr7LzUxdBoeKIEAw= apple-domain-verification=XvbXtDcmkb85AE4o atlassian-domain-verification=HAllRAExChJlug39hbBZ/pvbHZRtQbwOSE0/JrvkyleaHEATuR... brave-ledger-verification=1803baba7fea31b23ddf5e5787096f396921fc70d950b682d260c6... docusign=605eb30b-53bc-4b73-981f-ab33321b9ea6 google-site-verification=JN0iyQEOInofOqLkaOQ6UZowEq8JytLtmwNrXPLdDKI google-site-verification=YBuSFAYQOSRaH15OMod91YHH8-TZTm1spLBbmTTUAA0 google-site-verification=zBogziBDkhROg2rkPVaRIcKt9lLgTdqX9drZr3VxEPs keybase-site-verification=BZoMvvVxT6CEx5smtbJRJDLYYwsACxL77Rnj_Ss2Waw SPF v=spf1 mx include:sendgrid.net include:_spf.google.com include:mail.zendesk.com ... |
| CAA | Lookup not available with standard resolver |
CAA record lookup requires a specialized DNS resolver. This check will be available in a future update.
Informational: CAA (Certification Authority Authorization) records weren't checked in this scan.
ARedirect Chain1 redirect(s), 479 ms totalPASS
https://crunchbase.com
439 ms · HTTP/1.1
https://www.crunchbase.com/
41 ms · HTTP/1.1 FINAL
| # | URL | Status | Time | Protocol | Server |
|---|---|---|---|---|---|
| 1 | https://crunchbase.com | 301 | 439 ms | HTTP/1.1 | cloudflare |
| 2 | https://www.crunchbase.com/ | 200 | 41 ms | HTTP/1.1 | cloudflare |
See the visual redirect chain in the HTTP Probe tab →
A+IPv6 ReadinessIPv6 reachable (2 ms)PASS
ACrawlabilityrobots.txt present, no sitemapPASS
A sitemap helps search engines discover and index your pages more efficiently.
No sitemap.xml — Google relies on crawl-graph discovery alone, slowing indexing of deep or fresh URLs.
Learn more ▾ ▴
A sitemap accelerates Google's discovery of new and updated content. Most CMSes auto-generate one; static-site frameworks need a build-step plugin. Reference it from robots.txt and submit in Search Console to confirm Google can fetch it.
Source: sitemaps.org / Google Search Central
User-agent: *
# Allow API and JS paths to be requested by crawlers
Allow: /v4/md/applications/crunchbase
Allow: /*.js$
Disallow: /login
Disallow: /register
Disallow: /account
Disallow: /account/invite
Disallow: /reset-password
Disallow: /subscriptions
Disallow: /contribute
Disallow: /add-new
Disallow: /edit
Disallow: /edit/success
Disallow: /edit/review
Disallow: /buy
Allow: /buy/select-product
Disallow: /account-setup
Disallow: /verify
Disallow: /admin
Disallow: /v4
Disallow: /home
Disallow: /search
Disallow: /discover
Disallow: /textsearch
# AI and LLM Crawling
User-agent: CCBot
Disallow: /
User-agent: ChatGPT-User
Disallow: /
User-agent: OAI-SearchBot
Disallow: /
User-agent: GPTBot
Disallow: /
User-agent: Google-Extended
Disallow: /
User-agent: anthropic-ai
Disallow: /
User-agent: Omgilibot
Disallow: /
User-agent: Omgili
Disallow: /
User-agent: FacebookBot
Disallow: /
User-agent: Diffbot
Disallow: /
User-agent: Bytespider
Disallow: /
User-agent: ImagesiftBot
Disallow: /
User-agent: cohere-ai
Disallow: /
User-agent: Claude-Web
Disallow: /
User-agent: PerplexityBot
Disallow: /
Sitemap: https://www.crunchbase.com/www-sitemaps/sitemap-index.xml
No sitemap found
Adding a sitemap helps search engines discover your pages.
A+Domain Intelligencecrunchbase.com — via Gandi SAS, 19 years, 2 months oldPASS
306 days
May 17, 2027
85 days
Issued by Let's Encrypt
19 years, 2 months
Registered May 17, 2007
Not enabled
Protects against DNS spoofing
Unknown
2606:4700::6812:9d0
Gandi SAS
Expiry timeline
Recommended actions
- Enable DNSSEC to protect visitors from DNS spoofing
- Enable registrar lock (clientTransferProhibited) to block unauthorized domain transfers
DNSSEC protects against DNS spoofing attacks. While not required, enabling DNSSEC adds an additional layer of security. Contact your DNS provider to enable it.
Without DNSSEC, an attacker who can poison your DNS can hijack your domain — and SSL certs alone don't stop them.
Learn more ▾ ▴
DNSSEC adds cryptographic signatures to DNS records, preventing forged responses from poisoning resolver caches. Without it, an attacker who controls the network path can redirect your domain to a malicious server before any HTTPS handshake happens. Most modern registrars (Cloudflare, Google Domains, Route 53) enable it with one toggle.
Source: ICANN / RFC 4033
The domain can be transferred without an unlock step. Enable registrar lock (clientTransferProhibited) in your registrar's control panel to protect against unauthorized or accidental transfers.
Without registrar lock, an attacker who phishes your registrar credentials can transfer the domain in minutes — total brand hijack.
Learn more ▾ ▴
Registrar lock (clientTransferProhibited, clientUpdateProhibited, clientDeleteProhibited) requires extra verification before any transfer/update/delete. Every major registrar offers it free. Combined with 2FA on your registrar account, it's the strongest defense against domain hijacking.
Source: ICANN / domain-security best practice