Infrastructure
· 9 checks — DNS, redirects, IPv6, crawlability, URL variants, and domain intelligence rolled into one auditable list.BCrawlabilityrobots.txt present, no sitemapREVIEW
A sitemap helps search engines discover and index your pages more efficiently.
No sitemap.xml — Google relies on crawl-graph discovery alone, slowing indexing of deep or fresh URLs.
Learn more ▾ ▴
A sitemap accelerates Google's discovery of new and updated content. Most CMSes auto-generate one; static-site frameworks need a build-step plugin. Reference it from robots.txt and submit in Search Console to confirm Google can fetch it.
Source: sitemaps.org / Google Search Central
Add a 'Sitemap:' directive to robots.txt so search engines can discover your sitemap.
robots.txt omits Sitemap: directive — crawlers must fetch /sitemap.xml by convention; reliable but missing the explicit hint.
Source: sitemaps.org
User-agent: AhrefsBot
Disallow: /
User-agent: AddSearchBot
User-agent: AI2Bot
User-agent: Ai2Bot-Dolma
User-agent: aiHitBot
User-agent: Amazonbot
User-agent: Andibot
User-agent: anthropic-ai
User-agent: Applebot
User-agent: Applebot-Extended
User-agent: Awario
User-agent: bedrockbot
User-agent: bigsur.ai
User-agent: Brightbot 1.0
User-agent: Bytespider
User-agent: CCBot
User-agent: ChatGPT Agent
User-agent: ChatGPT-User
User-agent: Claude-SearchBot
User-agent: Claude-User
User-agent: Claude-Web
User-agent: ClaudeBot
User-agent: CloudVertexBot
User-agent: cohere-ai
User-agent: cohere-training-data-crawler
User-agent: Cotoyogi
User-agent: Crawlspace
User-agent: Datenbank Crawler
User-agent: Devin
User-agent: Diffbot
User-agent: DuckAssistBot
User-agent: Echobot Bot
User-agent: EchoboxBot
User-agent: FacebookBot
User-agent: facebookexternalhit
User-agent: Factset_spyderbot
User-agent: FirecrawlAgent
User-agent: FriendlyCrawler
User-agent: Gemini-Deep-Research
User-agent: Google-CloudVertexBot
User-agent: Google-Extended
User-agent: GoogleAgent-Mariner
User-agent: GoogleOther
User-agent: GoogleOther-Image
User-agent: GoogleOther-Video
User-agent: GPTBot
User-agent: iaskspider/2.0
User-agent: ICC-Crawler
User-agent: ImagesiftBot
User-agent: img2dataset
User-agent: ISSCyberRiskCrawler
User-agent: Kangaroo Bot
User-agent: LinerBot
User-agent: meta-externalagent
User-agent: Meta-ExternalAgent
User-agent: meta-externalfetcher
User-agent: Meta-ExternalFetcher
User-agent: MistralAI-User
User-agent: MistralAI-User/1.0
User-agent: MyCentralAIScraperBot
User-agent: netEstate Imprint Crawler
User-agent: NovaAct
User-agent: OAI-SearchBot
User-agent: omgili
User-agent: omgilibot
User-agent: OpenAI
User-agent: Operator
User-agent: PanguBot
User-agent: Panscient
User-agent: panscient.com
User-agent: Perplexity-User
User-agent: PerplexityBot
User-agent: PetalBot
User-agent: PhindBot
User-agent: Poseidon Research Crawler
User-agent: QualifiedBot
User-agent: QuillBot
User-agent: quillbot.com
User-agent: SBIntuitionsBot
User-agent: Scrapy
User-agent: SemrushBot-OCOB
User-agent: SemrushBot-SWA
User-agent: Sidetrade indexer bot
User-agent: Thinkbot
User-agent: TikTokSpider
User-agent: Timpibot
User-agent: VelenPublicWebCrawler
User-agent: WARDBot
User-agent: Webzio-Extended
User-agent: wpbot
User-agent: YaK
User-agent: YandexAdditional
User-agent: YandexAdditionalBot
User-agent: YouBot
Disallow: /
User-agent: *
Allow: /
No sitemap found
Adding a sitemap helps search engines discover your pages.
BURL Variantswww/non-www, trailing slash, HTTP→HTTPSREVIEW
www / non-www
Inconsistent — duplicate content risk
HTTP → HTTPS
Consistent
BTLS Certificate Expiry & Recommendations46 days until leaf cert expires — 4 issues to addressREVIEW
Certificate validity
Recommended actions
- Extend HSTS max-age to at least 31536000 (1 year) to meet the preload list criteria
- Add includeSubDomains to the HSTS directive
- Add the preload directive and submit to hstspreload.org once max-age + includeSubDomains are in place
- Enable OCSP stapling on your TLS server to remove a CA roundtrip and protect user privacy
A+DNS Records6 A records, 189 ms lookupPASS
| A | 146.75.109.91, 151.101.1.91, 151.101.65.91, 151.101.129.91, 151.101.193.91, 151.101.213.91 |
| AAAA | 2a04:4e42::347, 2a04:4e42:32::347, 2a04:4e42:8b::347, 2a04:4e42:200::347, 2a04:4e42:400::347, 2a04:4e42:600::347 |
| CNAME | — |
| NS | ns1.gentoo.org, ns2.gentoo.org, ns3.gentoo.org, ns4.gentoo.org |
| MX | 10 mail.gentoo.org |
| TXT | hibp-verify=dweb_tb9fks1uzgj4gl7mh1cvlazr have-i-been-pwned-verification=6b3f8fa57575612e2a08a985549020c9 google-site-verification=STcs6DxN7jSUEFJzJ7O9KzcsuHxX3Oq1jk_ua2xV64E google-site-verification=TghnZBoJySbGjLcdxG0LaQW4CEySKKUn-joa4u72Z2Q google-site-verification=WjowzRhWnytv0y7RnR9sAJFzAP5KDikCubfMt83FaCo SPF v=spf1 a:smtp.gentoo.org a:lists.gentoo.org a:liststest.gentoo.org include:_netb... |
| CAA | Lookup not available with standard resolver |
CAA record lookup requires a specialized DNS resolver. This check will be available in a future update.
Informational: CAA (Certification Authority Authorization) records weren't checked in this scan.
ARedirect Chain1 redirect(s), 966 ms totalPASS
https://gentoo.org
677 ms · HTTP/1.1
https://www.gentoo.org/
289 ms · HTTP/1.1 FINAL
| # | URL | Status | Time | Protocol | Server |
|---|---|---|---|---|---|
| 1 | https://gentoo.org | 301 | 677 ms | HTTP/1.1 | nginx |
| 2 | https://www.gentoo.org/ | 200 | 289 ms | HTTP/1.1 | nginx |
See the visual redirect chain in the HTTP Probe tab →
A+IPv6 ReadinessIPv6 reachable (1 ms)PASS
A+Domain Intelligencegentoo.org — via Amazon Registrar, Inc., 26 years, 11 months oldPASS
1174 days
October 4, 2029
46 days
Issued by Let's Encrypt
26 years, 11 months
Registered October 4, 1999
Enabled
Protects against DNS spoofing
Unknown
2a04:4e42::347
Amazon Registrar, Inc.
Expiry timeline
Recommended actions
- Enable registrar lock (clientTransferProhibited) to block unauthorized domain transfers
The domain can be transferred without an unlock step. Enable registrar lock (clientTransferProhibited) in your registrar's control panel to protect against unauthorized or accidental transfers.
Without registrar lock, an attacker who phishes your registrar credentials can transfer the domain in minutes — total brand hijack.
Learn more ▾ ▴
Registrar lock (clientTransferProhibited, clientUpdateProhibited, clientDeleteProhibited) requires extra verification before any transfer/update/delete. Every major registrar offers it free. Combined with 2FA on your registrar account, it's the strongest defense against domain hijacking.
Source: ICANN / domain-security best practice