Infrastructure
· 17 checks — DNS, redirects, IPv6, crawlability, URL variants, and domain intelligence rolled into one auditable list.DCrawlabilityActionrobots.txt present, no sitemapFIX
Disallow: / for all user-agents prevents search engines from indexing any page. This will remove the site from search results.
Disallow: / in robots.txt blocks every search crawler — the site becomes invisible in organic search.
Learn more ▾ ▴
Common deployment mistake: a staging robots.txt with `User-agent: * / Disallow: /` ships to prod. The site falls out of search results within days. Verify your robots.txt is the production-intended version. If this is intentional (private site), no action needed.
Source: Google Search Central
A sitemap helps search engines discover and index your pages more efficiently.
No sitemap.xml — Google relies on crawl-graph discovery alone, slowing indexing of deep or fresh URLs.
Learn more ▾ ▴
A sitemap accelerates Google's discovery of new and updated content. Most CMSes auto-generate one; static-site frameworks need a build-step plugin. Reference it from robots.txt and submit in Search Console to confirm Google can fetch it.
Source: sitemaps.org / Google Search Central
Add a 'Sitemap:' directive to robots.txt so search engines can discover your sitemap.
robots.txt omits Sitemap: directive — crawlers must fetch /sitemap.xml by convention; reliable but missing the explicit hint.
Source: sitemaps.org
# Pantheon's documentation on robots.txt: http://helpdesk.getpantheon.com/customer/portal/articles/1389628
User-agent: *
Disallow: /
No sitemap found
Adding a sitemap helps search engines discover your pages.
BDNS Records1 A records, 34 ms lookupREVIEW
| A | 185.178.196.4 |
| AAAA | 2a0a:6c80::4 |
| CNAME | fe.cfp2p.edge.pantheon.io |
| NS | — |
| MX | — |
| TXT | — |
| CAA | Lookup not available with standard resolver |
Multiple A records provide failover if one server goes down.
Single A record means a single point of failure — if that IP goes down, your site is unreachable until DNS TTL expires.
Learn more ▾ ▴
Add multiple A records for round-robin failover, or use a managed DNS provider with health-checked failover (Route 53, Cloudflare, NS1). Short TTL (60-300s) lets clients recover faster on outages.
Source: SRE practice / DNS architecture
A CNAME at the zone apex can break MX and NS records. Use ALIAS/ANAME or A records instead.
CNAME at the apex (example.com) breaks every other apex record (MX, TXT, NS) — DNS-protocol violation per RFC 1034.
Learn more ▾ ▴
RFC 1034 forbids CNAME alongside other records at the same name. Some DNS providers offer ALIAS / ANAME / flattened-CNAME records that work around this — use those instead. Otherwise apex-level CNAME breaks email (no MX), domain ownership verification (no TXT), and more.
Source: RFC 1034
SPF helps prevent email spoofing. Add a TXT record starting with 'v=spf1'.
Without SPF, receiving servers can't validate sending IPs — your domain is easier to spoof in phishing.
Learn more ▾ ▴
SPF complements DMARC. Both should be published. SPF records list authorized sending IPs (e.g., `v=spf1 include:_spf.google.com ~all` for Google Workspace). After publishing, verify in Google Postmaster Tools or mxtoolbox.
Source: RFC 7208 (SPF)
BDNSSECUnsigned (DNSSEC not deployed)REVIEW
BCAA RecordsNo CAA records (any CA may issue certificates)REVIEW
CReverse DNSAction0/2 IPs match cert SANREVIEW
BTLS Certificate Expiry & Recommendations83 days until leaf cert expires — 5 issues to addressREVIEW
Certificate validity
Recommended actions
- Extend HSTS max-age to at least 31536000 (1 year) to meet the preload list criteria
- Add includeSubDomains to the HSTS directive
- Add the preload directive and submit to hstspreload.org once max-age + includeSubDomains are in place
- Enable DNSSEC on your domain for DNS spoofing protection
- Enable OCSP stapling on your TLS server to remove a CA roundtrip and protect user privacy
BCDN & DeliveryCloudflareREVIEW
BCDN Cache ObservabilityNo CDN cache-status headers in the responseREVIEW
BOperational Status PageNo status page link detectedREVIEW
A+Subdomain TakeoverNo subdomain takeover risk detectedPASS
A+Multi-Resolver DNS SpeedMean 29ms across 3 resolvers (spread 14ms)PASS
A+Redirect Chain0 redirect(s), 66 ms totalPASS
https://dev-shivaninevrekar.pantheonsite...
66 ms · HTTP/1.1 FINAL
| # | URL | Status | Time | Protocol | Server |
|---|---|---|---|---|---|
| 1 | https://dev-shivaninevrekar.pantheonsite... | 403 | 66 ms | HTTP/1.1 | cloudflare |
A+IPv6 ReadinessIPv6 reachable (17 ms)PASS
A+URL Variantswww/non-www, trailing slash, HTTP→HTTPSPASS
www / non-www
HTTP → HTTPS
Consistent
A+HTTP Probe TimingTotal 108 ms — DNS, TCP, TLS, TTFB, content transfer breakdownPASS
Connection waterfall
A+Health Check EndpointAuth-protected health endpoint at https://dev-shivaninevrekar.pantheonsite.io/health (HTTP 403)PASS
Domain IntelligenceDomain intelligence data not availableINFO
RDAP and WHOIS lookup both failed