Infrastructure
· 17 checks — DNS, redirects, IPv6, crawlability, URL variants, and domain intelligence rolled into one auditable list.FCrawlabilityActionrobots.txt present, sitemap with 0 URLsFIX
Disallow: / for all user-agents prevents search engines from indexing any page. This will remove the site from search results.
Disallow: / in robots.txt blocks every search crawler — the site becomes invisible in organic search.
Learn more ▾ ▴
Common deployment mistake: a staging robots.txt with `User-agent: * / Disallow: /` ships to prod. The site falls out of search results within days. Verify your robots.txt is the production-intended version. If this is intentional (private site), no action needed.
Source: Google Search Central
Search engines may not be able to parse the sitemap. Fix XML validation errors.
An unparseable sitemap is silently ignored by Google — the URLs it advertises are never queued for crawl.
Learn more ▾ ▴
Google's sitemap parser is strict about XML validity. A single unescaped `&` or unclosed tag invalidates the whole file. Run your sitemap through a validator (Search Console's Sitemaps report flags it) and fix the offending entry. Most generators escape correctly; mistakes usually come from manually-written entries.
Source: sitemaps.org / Google Search Central
An empty sitemap provides no value. Add <url> entries for your pages.
An empty sitemap signals 'no content to index' to Google — actively harmful versus having no sitemap at all.
Learn more ▾ ▴
Google compares URLs in the sitemap against URLs it has crawled. An empty sitemap on a site with thousands of pages signals abandonment. Either populate it correctly (most CMSes auto-generate) or delete the file and let Google crawl normally.
Source: Google Search Central / sitemaps.org
Add a 'Sitemap:' directive to robots.txt so search engines can discover your sitemap.
robots.txt omits Sitemap: directive — crawlers must fetch /sitemap.xml by convention; reliable but missing the explicit hint.
Source: sitemaps.org
# Welcome to Reddit's robots.txt
# Reddit believes in an open internet, but not the misuse of public content.
# See https://support.reddithelp.com/hc/en-us/articles/26410290525844-Public-Content-Policy Reddit's Public Content Policy for access and use restrictions to Reddit content.
# See https://www.reddit.com/r/reddit4researchers/ for details on how Reddit continues to support research and non-commercial use.
# policy: https://support.reddithelp.com/hc/en-us/articles/26410290525844-Public-Content-Policy
User-agent: *
Disallow: /
BDNSSECUnsigned (DNSSEC not deployed)REVIEW
CReverse DNSAction0/8 IPs match cert SANREVIEW
BURL Variantswww/non-www, trailing slash, HTTP→HTTPSREVIEW
www / non-www
Inconsistent — duplicate content risk
HTTP → HTTPS
Consistent
BTLS Certificate Expiry & Recommendations147 days until leaf cert expires — 1 issues to addressREVIEW
Certificate validity
Recommended actions
- Enable DNSSEC on your domain for DNS spoofing protection
BCDN & DeliveryVarnishREVIEW
BCDN Cache ObservabilityNo CDN cache-status headers in the responseREVIEW
BOperational Status PageNo status page link detectedREVIEW
A+DNS Records4 A records, 68 ms lookupPASS
| A | 151.101.129.140, 151.101.1.140, 151.101.65.140, 151.101.193.140 |
| AAAA | 2a04:4e42::396, 2a04:4e42:200::396, 2a04:4e42:400::396, 2a04:4e42:600::396 |
| CNAME | — |
| NS | ns-1029.awsdns-00.org, ns-1887.awsdns-43.co.uk, ns-378.awsdns-47.com, ns-557.awsdns-05.net |
| MX | 1 aspmx.l.google.com 5 alt1.aspmx.l.google.com 5 alt2.aspmx.l.google.com 10 aspmx2.googlemail.com 10 aspmx3.googlemail.com |
| TXT | 00DV90000052YKT=1TBV90000000AsD 00Dbf000005OCRl=1TBbf0000000MJt 00Dbf000005OCTN=1TBbf0000000Mer 00Do0000000I6Pa=1TBcv0000000HQh 614ac4be-8664-4cea-8e29-f84d08ad875c MS=ms71041902 a5897185-49e1-4181-9065-51bd24737466 apple-domain-verification=qC3rSSKrh10DoMI7 atlassian-domain-verification=aGWDxGvt+oY3p7qTWt5v2uJDVJkoJAeHxwGqKmGQLMEsUXUJJe... atlassian-domain-verification=vBaV6PXyyu4OAPLiQFbxFMCboSTjoR/qxKJ2OlpI46ZEpZL/FV... box-domain-verification=95c33f4ee4b11d8827190dbc5371ca7df25b961019116e5565ce4aa3... cursor-domain-verification-f9fw38=WVB8yAMGTZXO7Je9DLuj1DIZB docusign=6ba0c5a9-5a5e-41f8-a7c8-8b4c6e35c10c google-site-verification=0uv13-wxlHK8FFKaUpgzyrVmL1YdNYW6v3PupLdw3JI google-site-verification=5Qzlkl8HJJSBKnMI7rQGDr9Sgiir4Ey6klx1rWQCJ2M google-site-verification=QO4VY1PRzsG-MxiQn3j0kxd_ZaFjcTk6MWBfKru0_5I google-site-verification=oh_YJE560y0e6FHP1RT7NIjyTlBhACNMvD2EgSss0sc google-site-verification=zZHozYbAJmSLOMG4OjQFoHiVqkxtdgvyBzsE7wUGFiw jamf-site-verification=EGK11tNR4hezQg0jZaIDpA jetbrains-domain-verification=bjn7o9fxbduga0omhaepeewtx onetrust-domain-verification=6b98ba3dd087405399bbf45b6cbdcd37 postman-domain-verification=ad4c89daaf83d06f70cb6e8737cb323b039aeaae9e0e288c7508... protonmail-verification=24cb91442caf54c8b820f4a0347292f143531210 stripe-verification=80aa7e2c9775fee2653f61b04b46da6fd227b56fecd9499e5f63cd0dcf56... stripe-verification=9bd70dd1884421b47f596fea301e14838c9825cdba5b209990968fdc6f80... twilio-domain-verification=5e37855d7c9445e967b31c5e0371ebb5 SPF v=spf1 include:amazonses.com include:_spf.google.com include:19922862.spf01.hubs... |
| CAA | Lookup not available with standard resolver |
A+Subdomain TakeoverNo subdomain takeover risk detectedPASS
ACAA Recordsissue: digicert.com, pki.googPASS
A+Multi-Resolver DNS SpeedMean 1ms across 3 resolvers (spread 1ms)PASS
ARedirect Chain1 redirect(s), 25 ms totalPASS
https://reddit.com
12 ms · HTTP/1.1
https://www.reddit.com/
14 ms · HTTP/1.1 FINAL
| # | URL | Status | Time | Protocol | Server |
|---|---|---|---|---|---|
| 1 | https://reddit.com | 301 | 12 ms | HTTP/1.1 | snooserv |
| 2 | https://www.reddit.com/ | 200 | 14 ms | HTTP/1.1 | snooserv |
See the visual redirect chain in the HTTP Probe tab →
A+IPv6 ReadinessIPv6 reachable (1 ms)PASS
A+Domain Intelligencereddit.com — via MarkMonitor Inc., 21 years, 4 months oldPASS
715 days
April 29, 2028
147 days
Issued by DigiCert Inc
21 years, 4 months
Registered April 29, 2005
Not enabled
Protects against DNS spoofing
Unknown
2a04:4e42::396
MarkMonitor Inc.
Expiry timeline
Recommended actions
- Enable DNSSEC to protect visitors from DNS spoofing
- Enable registrar lock (clientTransferProhibited) to block unauthorized domain transfers
DNSSEC protects against DNS spoofing attacks. While not required, enabling DNSSEC adds an additional layer of security. Contact your DNS provider to enable it.
Without DNSSEC, an attacker who can poison your DNS can hijack your domain — and SSL certs alone don't stop them.
Learn more ▾ ▴
DNSSEC adds cryptographic signatures to DNS records, preventing forged responses from poisoning resolver caches. Without it, an attacker who controls the network path can redirect your domain to a malicious server before any HTTPS handshake happens. Most modern registrars (Cloudflare, Google Domains, Route 53) enable it with one toggle.
Source: ICANN / RFC 4033
The domain can be transferred without an unlock step. Enable registrar lock (clientTransferProhibited) in your registrar's control panel to protect against unauthorized or accidental transfers.
Without registrar lock, an attacker who phishes your registrar credentials can transfer the domain in minutes — total brand hijack.
Learn more ▾ ▴
Registrar lock (clientTransferProhibited, clientUpdateProhibited, clientDeleteProhibited) requires extra verification before any transfer/update/delete. Every major registrar offers it free. Combined with 2FA on your registrar account, it's the strongest defense against domain hijacking.
Source: ICANN / domain-security best practice