Infrastructure
· 9 checks — DNS, redirects, IPv6, crawlability, URL variants, and domain intelligence rolled into one auditable list.BCrawlabilityrobots.txt present, no sitemapREVIEW
A sitemap helps search engines discover and index your pages more efficiently.
No sitemap.xml — Google relies on crawl-graph discovery alone, slowing indexing of deep or fresh URLs.
Learn more ▾ ▴
A sitemap accelerates Google's discovery of new and updated content. Most CMSes auto-generate one; static-site frameworks need a build-step plugin. Reference it from robots.txt and submit in Search Console to confirm Google can fetch it.
Source: sitemaps.org / Google Search Central
Add a 'Sitemap:' directive to robots.txt so search engines can discover your sitemap.
robots.txt omits Sitemap: directive — crawlers must fetch /sitemap.xml by convention; reliable but missing the explicit hint.
Source: sitemaps.org
#
# robots.txt for https://www.w3.org/
#
# $Id: robots.txt,v 1.104 2026/02/16 05:12:36 gerald Exp $
#
# the following settings apply to all bots
User-agent: *
# Blogs - WordPress
# https://wordpress.org/documentation/article/search-engine-optimization/#robots-txt-optimization
Disallow: /*/wp-admin/
Disallow: /*/wp-includes/
Disallow: /*/wp-content/plugins/
Disallow: /*/wp-content/cache/
Disallow: /*/wp-content/themes/
Disallow: /blog/?
Disallow: /blog/*/trackback/
Disallow: /blog/*/feed/
Disallow: /blog/*/comments/
Disallow: /blog/*/category/*/*
Disallow: /blog/*/*/trackback/
Disallow: /blog/*/*/feed/
Disallow: /blog/*/*/comments/
Disallow: /blog/*/*?
Disallow: /community/trackback/
Disallow: /community/feed/
Disallow: /community/comments/
Disallow: /community/category/*/*
Disallow: /community/*/trackback/
Disallow: /community/*/feed/
Disallow: /community/*/comments/
Disallow: /community/*/category/*/*
Disallow: /community/*?
Disallow: /Consortium/Offices/trackback/
Disallow: /Consortium/Offices/feed/
Disallow: /Consortium/Offices/comments/
Disallow: /Consortium/Offices/category/*/*
Disallow: /Consortium/Offices/*/trackback/
Disallow: /Consortium/Offices/*/feed/
Disallow: /Consortium/Offices/*/comments/
Disallow: /Consortium/Offices/*?
# Wikis - Mediawiki
# https://www.mediawiki.org/wiki/Manual:Robots.txt
Disallow: /wiki/index.php?
Disallow: /wiki/index.php/Help
Disallow: /wiki/index.php/MediaWiki
Disallow: /wiki/index.php/Special:
Disallow: /wiki/index.php/Template
Disallow: /wiki/Special:
Disallow: /wiki/Help:
Disallow: /wiki/Talk:
Disallow: /wiki/skins/
Disallow: /wiki/*/info
Disallow: /wiki/*/edit
Disallow: /wiki/*/history
Disallow: /*/wiki/index.php?
Disallow: /*/wiki/index.php/Help
Disallow: /*/wiki/index.php/MediaWiki
Disallow: /*/wiki/index.php/Special:
Disallow: /*/wiki/index.php/Template
Disallow: /*/wiki/Special:
Disallow: /*/wiki/Help:
Disallow: /*/wiki/Talk:
Disallow: /*/wiki/skins/
Disallow: /*/wiki/*/info
Disallow: /*/wiki/*/edit
Disallow: /*/wiki/*/history
# various other access-controlled or expensive areas
Disallow: /2004/ontaria/basic
Disallow: /Member/
Disallow: /Team/
Disallow: /Project/
Disallow: /Web/
Disallow: /Systems/
Disallow: /Out-Of-Date
Disallow: /2005/06/blog/
Disallow: /2004/08/W3CTalks
Disallow: /2007/11/Talks/search
Disallow: /People/all/
Disallow: /people/all/
Disallow: /People/domain/
Disallow: /people/domain/
Disallow: /RDF/Validator/ARPServlet
Disallow: /RDF/Validator/rdfval
Disallow: /2003/03/Translations/byLanguage
Disallow: /2003/03/Translations/byTechnology
Disallow: /2005/11/Translations/Query
Disallow: /2000/06/webdata/xslt
Disallow: /2000/09/webdata/xslt
Disallow: /2005/08/online_xslt/xslt
Disallow: /Search/Mail/Public/
Disallow: /2006/02/chartergen
Disallow: /2004/01/pp-impl
Disallow: /Consortium/supporters
Disallow: /2012/pyRdfa/extract
Disallow: /2012/pyRdfa/?
Disallow: /WAI/PF/comments/
Disallow: /WAI/events/
Disallow: /participate/conferences.xml
Disallow: /scripts/
Disallow: /2005/01/yacker/
Disallow: /2005/01/yacker?
Disallow: /2005/07/pubrules?
Disallow: /ns/hydra/console/?
Disallow: /2007/08/grddl/?
Disallow: /2009/07/webidl-check?
Disallow: /RDF/Validator/ARPServlet?
Disallow: /2000/06/webdata/xsv?
Disallow: /2000/09/webdata/xsv?
Disallow: /2007/10/htmldiff?
Disallow: /Style/CSS/members.be/
Disallow: /services
Disallow: /2021/05/view-gallery/
Disallow: /International/questions/qa-html-language-declarations/icons/
Disallow: /International/questions/qa-html-language-declarations/qa-html-language-declarations-data/icons/
Disallow: /*,*
Disallow: /WAI/beta/
Disallow: /WAI/ut1/
Disallow: /WAI/ut2/
Disallow: /WAI/ut3/
Disallow: /WAI/ut4/
Disallow: /WAI/drafts/
# Allow W3C Link checker to check any paths on this site
User-agent: W3C-checklink
Disallow:
No sitemap found
Adding a sitemap helps search engines discover your pages.
BURL Variantswww/non-www, trailing slash, HTTP→HTTPSREVIEW
www / non-www
Inconsistent — duplicate content risk
HTTP → HTTPS
Consistent
BTLS Certificate Expiry & Recommendations55 days until leaf cert expires — 3 issues to addressREVIEW
Certificate validity
Recommended actions
- Enable HSTS: Strict-Transport-Security: max-age=31536000; includeSubDomains
- Enable DNSSEC on your domain for DNS spoofing protection
- Enable OCSP stapling on your TLS server to remove a CA roundtrip and protect user privacy
BCDN & DeliveryCloudflareREVIEW
A+DNS Records2 A records, 33 ms lookupPASS
| A | 104.18.22.19, 104.18.23.19 |
| AAAA | 2606:4700::6812:1713, 2606:4700::6812:1613 |
| CNAME | — |
| NS | jamie.ns.cloudflare.com, pranab.ns.cloudflare.com |
| MX | 5 pan.w3.org 5 puck.w3.org |
| TXT | google-site-verification=cu9dn6ytlndLOV87MLxedTHxoKTRKk0dUf_jXK8OsTc SPF v=spf1 mx ip4:34.238.48.92 ip4:34.199.101.240 ip6:2600:1f18:7d7a:2700::/56 inclu... 853C844562 google-site-verification=CZkSL9bXgKDxRmeDnAQorwhaQy3Ji1vTn36bBn5rL-E ca3-d6556337888945edac9c159a068c46a2 google-site-verification=5TpwKtmzdpa5fntQgKHY96mTJ3FiSn9fGp07XKxFsMk google-site-verification=dS4IjfXgopxupUFDAqg3wk94IIZ2GuQ-jW4wiLm69Rg |
| CAA | Lookup not available with standard resolver |
CAA record lookup requires a specialized DNS resolver. This check will be available in a future update.
Informational: CAA (Certification Authority Authorization) records weren't checked in this scan.
ARedirect Chain1 redirect(s), 169 ms totalPASS
https://w3.org
64 ms · HTTP/1.1
https://www.w3.org/
105 ms · HTTP/1.1 FINAL
| # | URL | Status | Time | Protocol | Server |
|---|---|---|---|---|---|
| 1 | https://w3.org | 301 | 64 ms | HTTP/1.1 | cloudflare |
| 2 | https://www.w3.org/ | 200 | 105 ms | HTTP/1.1 | cloudflare |
See the visual redirect chain in the HTTP Probe tab →
A+IPv6 ReadinessIPv6 reachable (17 ms)PASS
A+Domain Intelligencew3.org — via Gandi SAS, 32 years, 2 months oldPASS
1053 days
July 5, 2029
55 days
Issued by Google Trust Services
32 years, 2 months
Registered July 6, 1994
Not enabled
Protects against DNS spoofing
Unknown
2606:4700::6812:1613
Gandi SAS
Expiry timeline
Recommended actions
- Enable DNSSEC to protect visitors from DNS spoofing
- Enable registrar lock (clientTransferProhibited) to block unauthorized domain transfers
DNSSEC protects against DNS spoofing attacks. While not required, enabling DNSSEC adds an additional layer of security. Contact your DNS provider to enable it.
Without DNSSEC, an attacker who can poison your DNS can hijack your domain — and SSL certs alone don't stop them.
Learn more ▾ ▴
DNSSEC adds cryptographic signatures to DNS records, preventing forged responses from poisoning resolver caches. Without it, an attacker who controls the network path can redirect your domain to a malicious server before any HTTPS handshake happens. Most modern registrars (Cloudflare, Google Domains, Route 53) enable it with one toggle.
Source: ICANN / RFC 4033
The domain can be transferred without an unlock step. Enable registrar lock (clientTransferProhibited) in your registrar's control panel to protect against unauthorized or accidental transfers.
Without registrar lock, an attacker who phishes your registrar credentials can transfer the domain in minutes — total brand hijack.
Learn more ▾ ▴
Registrar lock (clientTransferProhibited, clientUpdateProhibited, clientDeleteProhibited) requires extra verification before any transfer/update/delete. Every major registrar offers it free. Combined with 2FA on your registrar account, it's the strongest defense against domain hijacking.
Source: ICANN / domain-security best practice