Skip to content
https://20minutos.es

Infrastructure

· 9 checks — DNS, redirects, IPv6, crawlability, URL variants, and domain intelligence rolled into one auditable list.
SCORE
81
GRADE
B
FIX
0
REVIEW
4
PASS
4
INFO
1
Probed from New York, United Stated
301 Moved Permanently
Checks
9
4 PASS 4 REVIEW
C
IPv6 Readiness
Action
No IPv6 support
REVIEW
No IPv6 support
Info::
No IPv6 (AAAA) records found
IPv6 support is increasingly important for global accessibility. About 40% of internet users have IPv6 connectivity.
No IPv6 Support
About 40% of internet users have IPv6. Consider adding AAAA records.

IPv6 support is increasingly important for global accessibility. About 40% of internet users have IPv6 connectivity.

Why this matters

No AAAA records — same impact as 'no IPv6 (AAAA) records'; IPv6-preferring clients pay extra latency falling back to IPv4.

Source: Google IPv6 stats

B
Crawlability
robots.txt present, no sitemap
REVIEW
robots.txt present, no sitemap
Info::
robots.txt is present
Got: 5025 bytes
Info::
No sitemap.xml found
A sitemap helps search engines discover and index your pages more efficiently.
Info::
robots.txt does not reference a sitemap
Add a 'Sitemap:' directive to robots.txt so search engines can discover your sitemap.

A sitemap helps search engines discover and index your pages more efficiently.

Why this matters

No sitemap.xml — Google relies on crawl-graph discovery alone, slowing indexing of deep or fresh URLs.

Learn more ▾

A sitemap accelerates Google's discovery of new and updated content. Most CMSes auto-generate one; static-site frameworks need a build-step plugin. Reference it from robots.txt and submit in Search Console to confirm Google can fetch it.

Source: sitemaps.org / Google Search Central

Add a 'Sitemap:' directive to robots.txt so search engines can discover your sitemap.

Why this matters

robots.txt omits Sitemap: directive — crawlers must fetch /sitemap.xml by convention; reliable but missing the explicit hint.

Source: sitemaps.org

robots.txt 200 OK
Size 5025 B Sitemaps referenced 0 User-agents MSIECrawler, NetSeer crawler, SiteBot, admantx, TeleportPro, Crawl4AI, iaskspider, Kangaroo Bot, linko, DOC, SiteBot/0.1, WBSearchBot, cohere-ai, *, MJ12bot, discobot, IstellaBot, omgili, WRTNBot, rogerbot, BDCbot, Applebot-Extended, proximic, Moreoverbot, netEstate Imprint Crawler, netEstate NE Crawler, Diffbot, Yasni, Pixray-Seeker, AmazonBot, iAskBot, Maxthon, Yandex, Newscan, LCC, Spinn3r, NextGenSearchBot, VelenPublicWebCrawler, Download Ninja, FirecrawlAgent, webzio-extended, WebCopier, AddThis.com, laion-huggingface-processor, ZyBORG, JikeSpider, Zealbot, magpie-crawler, LexxeBot/1.0, Ezooms, seokicks, Bender, sogou, wotbox, ChatGLM-Spider, meta-externalagent, CyberAlert, Sosospider, psbot, PanguBot, grub-client, Exabot, Jyxobot, BLEXBot, Cotoyogi, WebZIP, cohere-training-data-crawler, SBIntuitionsBot, libwww, heritrix, discoverybot, AI2Bot, Ai2Bot-Dolma, Fetch, larbin, AmazonAdBot, UnisterBot, UbiCrawler, DeepSeekBot, WebStripper, HTTrack, wget, WeSEE:Search, Bytespider, CrystalSemanticsBot, NerdByNature.Bot, ApifyWebsiteContentCrawler, sitecheck.internetseer.com, k2spider, trovitBot, anthropic-ai, Datenbank Crawler, KunatoCrawler, Zao, CNCDialer, NPBot, ICC-Crawler, DotBot, 008, Teleport, TurnitinBot, Timpibot, Baiduspider, integromedb, SemrushBot-OCOB, BUbiNG, CCBot, ClaudeBot, Offline Explorer, Flamingo_SearchEngine, bl.uk_lddc_bot, ApifyBot, omgilibot, spbot, Spider, TavilyBot, WebReaper, GPTBot, SiteSnagger, FacebookBot Blocking No — crawling allowed
User-agent: *

Disallow: /view/
Disallow: /view/*
Disallow: /buscar
Disallow: /busqueda/
Disallow: /busqueda/*
Disallow: /imprimir/
Disallow: /mini20
Disallow: /mini20/
Disallow: /home
Disallow: /home/
Disallow: /img_validator/
Disallow: /aviso_comentario/
Disallow: /enviar_amigo/
Disallow: /usuarios/
Disallow: /proc/
Disallow: /iphoneapp
Disallow: /iphoneapp/
Disallow: /ajax
Disallow: /compartir
Disallow: /sso/
Disallow: /especial/especial-de-prueba/
Disallow: /especial/pruebas-comercial/
Disallow: /widgets/
Disallow: /codigo-descuento/
Disallow: /boletin/baja/
Disallow: /archivo/
Disallow: /archivo/*
Disallow: /*.woff2$
Disallow: /*.ttf$

# Agentes permitidos explicitamente
User-agent: proximic
Disallow:

# Agentes bloqueados por idioma

User-agent: Yandex
Disallow: /
User-agent: Baiduspider
Disallow: /

# Agentes nocivos

User-agent: 008
Disallow: /
User-agent: UnisterBot
Disallow: /
User-agent: JikeSpider
Disallow: /
User-agent: Zealbot
Disallow: /
User-agent: MSIECrawler
Disallow: /
User-agent: SiteSnagger
Disallow: /
User-agent: WebStripper
Disallow: /
User-agent: WebCopier
Disallow: /
User-agent: Fetch
Disallow: /
User-agent: Offline Explorer
Disallow: /
User-agent: Teleport
Disallow: /
User-agent: TeleportPro
Disallow: /
User-agent: WebZIP
Disallow: /
User-agent: linko
Disallow: /
User-agent: HTTrack
Disallow: /
User-agent: larbin
Disallow: /
User-agent: libwww
Disallow: /
User-agent: ZyBORG
Disallow: /
User-agent: UbiCrawler
Disallow: /
User-agent: DOC
Disallow: /
User-agent: Zao
Disallow: /
User-agent: sitecheck.internetseer.com
Disallow: /
User-agent: Zealbot
Disallow: /
User-agent: Download Ninja
Disallow: /
User-agent: Maxthon
Disallow: /
User-agent: CNCDialer
Disallow: /
User-agent: wget
Disallow: /
User-agent: grub-client
Disallow: /
User-agent: k2spider
Disallow: /
User-agent: NPBot
Disallow: /
User-agent: WebReaper
Disallow: /
User-agent: TurnitinBot
Disallow: /
User-agent: MJ12bot
Disallow: /
User-agent: magpie-crawler
Disallow: /
User-agent: Flamingo_SearchEngine
Disallow: /
User-agent: Exabot
Disallow: /
User-agent: Moreoverbot
Disallow: /
User-Agent: CyberAlert
Disallow: /
User-agent: Newscan
Disallow: /
User-agent: Spinn3r
Disallow: /
User-agent: LexxeBot/1.0
Disallow: /
User-agent: NextGenSearchBot
Disallow: /
User-agent: Sosospider
Disallow: /
User-agent: rogerbot
Disallow: /
User-agent: SiteBot/0.1
Disallow: /
User-agent: SiteBot
Disallow: /
User-agent: CrystalSemanticsBot
Disallow: /
User-agent: NetSeer crawler
Disallow: /
User-agent: trovitBot
Disallow: /
User-agent: DotBot
Disallow: /
User-agent: Ezooms
Disallow: /
User-agent: discobot
Disallow: /
User-agent: Jyxobot
Disallow: /
User-agent: sogou
Disallow: /
User-agent: heritrix
Disallow: /
User-agent: NerdByNature.Bot
Disallow: /
User-agent: psbot
Disallow: /
User-agent: WBSearchBot
Disallow: /
User-agent: AddThis.com
Disallow: /
User-agent: discoverybot
Disallow: /
User-agent: bl.uk_lddc_bot
Disallow: /
User-agent: IstellaBot
Disallow: /
User-agent: seokicks
Disallow: /
User-agent: Bender
Disallow: /
User-agent: wotbox
Disallow: /
User-agent: Yasni
Disallow: /
User-agent: netEstate NE Crawler
Disallow: /
User-agent: Pixray-Seeker
Disallow: /
User-agent: integromedb
Disallow: /
User-agent: BLEXBot
Disallow: /
User-agent: BDCbot
Disallow: /
User-agent: WeSEE:Search
Disallow: /
User-agent: admantx
Disallow: /
User-agent: spbot
Disallow: /
User-agent: BUbiNG
Disallow: /

# AI data scrappers for training LLMs
User-agent: anthropic-ai
Disallow: /
User-agent: AI2Bot
Disallow: /
User-agent: Ai2Bot-Dolma
Disallow: /
User-agent: AmazonBot
Disallow: /
User-agent: ApifyBot
Disallow: /
User-agent: ApifyWebsiteContentCrawler
Disallow: /
User-agent: Applebot-Extended
Disallow: /
User-agent: Bytespider
Disallow: /
User-agent: CCBot
Disallow: /
User-agent: ChatGLM-Spider
Disallow: /
User-agent: ClaudeBot
Disallow: /
User-agent: cohere-ai
Disallow: /
User-agent: cohere-training-data-crawler
Disallow: /
User-agent: Cotoyogi
Disallow: /
User-agent: Crawl4AI
Disallow: /
User-agent: Datenbank Crawler
Disallow: /
User-agent: DeepSeekBot
Disallow: /
User-agent: Diffbot
Disallow: /
User-agent: FacebookBot
Disallow: /
User-agent: FirecrawlAgent
Disallow: /
User-agent: GPTBot
Disallow: /
User-agent: iAskBot
Disallow: /
User-agent: iaskspider
Disallow: /
User-agent: ICC-Crawler
Disallow: /
User-agent: Kangaroo Bot
Disallow: /
User-agent: KunatoCrawler
Disallow: /
User-agent: laion-huggingface-processor
Disallow: /
User-agent: LCC
Disallow: /
User-agent: meta-externalagent
Disallow: /
User-agent: netEstate Imprint Crawler
Disallow: /
User-agent: omgili
Disallow: /
User-agent: omgilibot
Disallow: /
User-agent: PanguBot
Disallow: /
User-agent: SBIntuitionsBot
Disallow: /
User-agent: SemrushBot-OCOB
Disallow: /
User-agent: Spider
Disallow: /
User-agent: TavilyBot
Disallow: /
User-agent: Timpibot
Disallow: /
User-agent: VelenPublicWebCrawler
Disallow: /
User-agent: webzio-extended
Disallow: /
User-agent: WRTNBot
Disallow: /

# Amazon AdBot, ver #68519
User-agent: AmazonAdBot
Disallow:
sitemap.xml No sitemap found

No sitemap found

Adding a sitemap helps search engines discover your pages.

B
URL Variants
www/non-www, trailing slash, HTTP→HTTPS
REVIEW
www/non-www, trailing slash, HTTP→HTTPS
Critical::
Both www and non-www versions serve content
Got: Both variants return 200 Expected: One variant 301-redirects to the other
Info::
HTTP correctly 301-redirects to HTTPS

www / non-www

200https://www.20minutos.es/
200https://20minutos.es/

Inconsistent — duplicate content risk

HTTP → HTTPS

301http://20minutos.es/ → https://20minutos.es/

Consistent

B
TLS Certificate Expiry & Recommendations
128 days until leaf cert expires — 3 issues to address
REVIEW

Certificate validity

128
days left
0d 30d 60d 90d+

Recommended actions

  • Enable HSTS: Strict-Transport-Security: max-age=31536000; includeSubDomains
  • Enable DNSSEC on your domain for DNS spoofing protection
  • Enable OCSP stapling on your TLS server to remove a CA roundtrip and protect user privacy
A+
DNS Records
4 A records, 25 ms lookup
PASS
4 A records, 25 ms lookup
Info::
Resolves to 4 IPv4 address(es)
Got: 108.138.128.105, 108.138.128.50, 108.138.128.72, 108.138.128.77
Info::
No IPv6 (AAAA) records
Info::
4 nameserver(s) configured
Got: ns-1258.awsdns-29.org, ns-1906.awsdns-46.co.uk, ns-54.awsdns-06.com, ns-962.awsdns-56.net
Info::
1 mail exchanger(s) configured
Info::
CAA records not checked
CAA record lookup requires a specialized DNS resolver. This check will be available in a future update.
Info::
SPF record present in TXT
Info::
DNS resolution time: 25 ms
Got: 25 ms
A108.138.128.105, 108.138.128.50, 108.138.128.72, 108.138.128.77
AAAA—
CNAME—
NSns-1258.awsdns-29.org, ns-1906.awsdns-46.co.uk, ns-54.awsdns-06.com, ns-962.awsdns-56.net
MX
1 20minutos-es.mail.protection.outlook.com
TXT
_globalsign-domain-verification=oQMq_G3Amhi22taxA9iFT-dk_zHARxA27Z0jygBZzo
google-site-verification=canwC-LyLCNWqUt1lzY1geB89ujeljafn0dPL75177Q
SPF v=spf1 include:spf.protection.outlook.com include:amazonses.com include:_spf.goo...
CAALookup not available with standard resolver
Resolved in 25 ms

CAA record lookup requires a specialized DNS resolver. This check will be available in a future update.

Why this matters

Informational: CAA (Certification Authority Authorization) records weren't checked in this scan.

A
Redirect Chain
1 redirect(s), 231 ms total
PASS
1 redirect(s), 231 ms total
Info::
Single redirect
Got: https://20minutos.es → https://www.20minutos.es/ (301)
Info::
WWW normalization redirect

https://20minutos.es

201 ms · HTTP/1.1

301

https://www.20minutos.es/

30 ms · HTTP/1.1 FINAL

#URLStatusTimeProtocolServer
1https://20minutos.es301201 msHTTP/1.1
2https://www.20minutos.es/20030 msHTTP/1.1

See the visual redirect chain in the HTTP Probe tab →

A+
HTTP Probe Timing
Total 103 ms — DNS, TCP, TLS, TTFB, content transfer breakdown
PASS
DNS Lookup DNS Lookup — time to resolve the domain name to an IP address.
14 ms
TCP Connect TCP Connect — time to establish a TCP connection to the server.
2 ms
TLS Handshake TLS Handshake — time to complete the HTTPS encryption handshake.
5 ms
Time to First Byte Time to First Byte — how long the server takes to respond with the first byte of data.
103 ms
Total Time Total request time from DNS lookup through full response.
103 ms

Connection waterfall

DNS Lookup 14 ms TCP Connect 2 ms TLS Handshake 5 ms Server Processing 82 ms Content Transfer 0 ms
A
CDN & Delivery
AWS CloudFront (Miss from cloudfront)
PASS
AWS CloudFront (Miss from cloudfront)
Info::
Site is served via AWS CloudFront CDN (edge: JFK50-P4)
Got: x-amz-cf-id: phh_K6LK64gJ--KCubMlKOqbbxwXU3OXAMVv6PztriRoXxlSvSqo0g==
Info::
CDN cache status: Miss from cloudfront
CDN Detected: AWS CloudFront
Provider AWS CloudFront Cache Status Miss from cloudfront Evidence x-amz-cf-id: phh_K6LK64gJ--KCubMlKOqbbxwXU3OXAMVv6PztriRoXxlSvSqo0g==
Domain Intelligence
Domain intelligence data not available
INFO
Domain intelligence data not available

RDAP and WHOIS lookup both failed

All checks on this page are automated. Results are estimates - run targeted manual reviews when the score affects a release decision.

Send Feedback