Infrastructure
· 9 checks — DNS, redirects, IPv6, crawlability, URL variants, and domain intelligence rolled into one auditable list.FCrawlabilityActionrobots.txt present, sitemap with 0 URLsFIX
Disallow: / for all user-agents prevents search engines from indexing any page. This will remove the site from search results.
Disallow: / in robots.txt blocks every search crawler — the site becomes invisible in organic search.
Learn more ▾ ▴
Common deployment mistake: a staging robots.txt with `User-agent: * / Disallow: /` ships to prod. The site falls out of search results within days. Verify your robots.txt is the production-intended version. If this is intentional (private site), no action needed.
Source: Google Search Central
Search engines may not be able to parse the sitemap. Fix XML validation errors.
An unparseable sitemap is silently ignored by Google — the URLs it advertises are never queued for crawl.
Learn more ▾ ▴
Google's sitemap parser is strict about XML validity. A single unescaped `&` or unclosed tag invalidates the whole file. Run your sitemap through a validator (Search Console's Sitemaps report flags it) and fix the offending entry. Most generators escape correctly; mistakes usually come from manually-written entries.
Source: sitemaps.org / Google Search Central
An empty sitemap provides no value. Add <url> entries for your pages.
An empty sitemap signals 'no content to index' to Google — actively harmful versus having no sitemap at all.
Learn more ▾ ▴
Google compares URLs in the sitemap against URLs it has crawled. An empty sitemap on a site with thousands of pages signals abandonment. Either populate it correctly (most CMSes auto-generate) or delete the file and let Google crawl normally.
Source: Google Search Central / sitemaps.org
# Notice: Collection of data on Threads through automated means is
# prohibited unless you have express written permission from Threads
# and may only be conducted for the limited purpose contained in said
# permission.
# All authorized user-agents listed on this page must comply with Meta’s
# Automated Data Collection Terms available at:
# https://www.facebook.com/legal/automated_data_collection_terms
User-agent: Amazonbot
Disallow: /
User-agent: Applebot-Extended
Disallow: /
User-agent: Brightbot
Disallow: /
User-agent: ClaudeBot
Disallow: /
User-agent: Google-Extended
Disallow: /
User-agent: GPTBot
Disallow: /
User-agent: PerplexityBot
Disallow: /
User-agent: PetalBot
Disallow: /
User-agent: Scrapy
Disallow: /
User-agent: uptimerobot
Disallow: /
User-agent: viberbot
Disallow: /
User-agent: YaK
Disallow: /
User-agent: Yandex
Disallow: /
User-agent: Yeti
Disallow: /
User-agent: Applebot
Disallow: /accounts/
Disallow: /activity/
Disallow: /ajax/
Disallow: /client_error/
Disallow: /deck/
Disallow: /intent/
Disallow: /logging/
Disallow: /onboarding/
Disallow: /publicapi/
Disallow: /qp/batch_fetch_web/
Disallow: /query/
Disallow: /settings/
User-agent: Bingbot
Disallow: /*/c/
Disallow: /*/comments/
Disallow: /*/liked_by/
Disallow: /*/p/
Disallow: /accounts/
Disallow: /activity/
Disallow: /ajax/
Disallow: /client_error/
Disallow: /deck/
Disallow: /intent/
Disallow: /logging/
Disallow: /onboarding/
Disallow: /publicapi/
Disallow: /qp/batch_fetch_web/
Disallow: /query/
Disallow: /settings/
User-agent: Discordbot
Disallow: /accounts/
Disallow: /activity/
Disallow: /ajax/
Disallow: /client_error/
Disallow: /deck/
Disallow: /intent/
Disallow: /logging/
Disallow: /onboarding/
Disallow: /publicapi/
Disallow: /qp/batch_fetch_web/
Disallow: /query/
Disallow: /settings/
User-agent: DuckDuckBot
Disallow: /accounts/
Disallow: /activity/
Disallow: /ajax/
Disallow: /client_error/
Disallow: /deck/
Disallow: /intent/
Disallow: /logging/
Disallow: /onboarding/
Disallow: /publicapi/
Disallow: /qp/batch_fetch_web/
Disallow: /query/
Disallow: /settings/
User-agent: facebookexternalhit
Disallow: /accounts/
Disallow: /activity/
Disallow: /ajax/
Disallow: /client_error/
Disallow: /deck/
Disallow: /intent/
Disallow: /logging/
Disallow: /onboarding/
Disallow: /publicapi/
Disallow: /qp/batch_fetch_web/
Disallow: /query/
Disallow: /settings/
User-agent: Googlebot
Disallow: /*/c/
Disallow: /*/comments/
Disallow: /*/liked_by/
Disallow: /*/p/
Disallow: /accounts/
Disallow: /activity/
Disallow: /ajax/
Disallow: /client_error/
Disallow: /deck/
Disallow: /intent/
Disallow: /logging/
Disallow: /onboarding/
Disallow: /publicapi/
Disallow: /qp/batch_fetch_web/
Disallow: /query/
Disallow: /settings/
User-agent: Googlebot-Image
Disallow: /accounts/
Disallow: /activity/
Disallow: /ajax/
Disallow: /client_error/
Disallow: /deck/
Disallow: /intent/
Disallow: /logging/
Disallow: /onboarding/
Disallow: /publicapi/
Disallow: /qp/batch_fetch_web/
Disallow: /query/
Disallow: /settings/
User-agent: Googlebot-Video
Disallow: /accounts/
Disallow: /activity/
Disallow: /ajax/
Disallow: /client_error/
Disallow: /deck/
Disallow: /intent/
Disallow: /logging/
Disallow: /onboarding/
Disallow: /publicapi/
Disallow: /qp/batch_fetch_web/
Disallow: /query/
Disallow: /settings/
User-agent: Googlebot-News
Disallow: /accounts/
Disallow: /activity/
Disallow: /ajax/
Disallow: /client_error/
Disallow: /deck/
Disallow: /intent/
Disallow: /logging/
Disallow: /onboarding/
Disallow: /publicapi/
Disallow: /qp/batch_fetch_web/
Disallow: /query/
Disallow: /settings/
User-agent: Google-InspectionTool
Disallow: /accounts/
Disallow: /activity/
Disallow: /ajax/
Disallow: /client_error/
Disallow: /deck/
Disallow: /intent/
Disallow: /logging/
Disallow: /onboarding/
Disallow: /publicapi/
Disallow: /qp/batch_fetch_web/
Disallow: /query/
Disallow: /settings/
User-agent: LinkedInBot
Disallow: /accounts/
Disallow: /activity/
Disallow: /ajax/
Disallow: /client_error/
Disallow: /deck/
Disallow: /intent/
Disallow: /logging/
Disallow: /onboarding/
Disallow: /publicapi/
Disallow: /qp/batch_fetch_web/
Disallow: /query/
Disallow: /settings/
User-agent: msnbot
Disallow: /accounts/
Disallow: /activity/
Disallow: /ajax/
Disallow: /client_error/
Disallow: /deck/
Disallow: /intent/
Disallow: /logging/
Disallow: /onboarding/
Disallow: /publicapi/
Disallow: /qp/batch_fetch_web/
Disallow: /query/
Disallow: /settings/
User-agent: Pinterestbot
Disallow: /accounts/
Disallow: /activity/
Disallow: /ajax/
Disallow: /client_error/
Disallow: /deck/
Disallow: /intent/
Disallow: /logging/
Disallow: /onboarding/
Disallow: /publicapi/
Disallow: /qp/batch_fetch_web/
Disallow: /query/
Disallow: /settings/
User-agent: Screaming Frog SEO Spider
Disallow: /accounts/
Disallow: /activity/
Disallow: /ajax/
Disallow: /client_error/
Disallow: /deck/
Disallow: /intent/
Disallow: /logging/
Disallow: /onboarding/
Disallow: /publicapi/
Disallow: /qp/batch_fetch_web/
Disallow: /query/
Disallow: /settings/
User-agent: seznambot
Disallow: /accounts/
Disallow: /activity/
Disallow: /ajax/
Disallow: /client_error/
Disallow: /deck/
Disallow: /intent/
Disallow: /logging/
Disallow: /onboarding/
Disallow: /publicapi/
Disallow: /qp/batch_fetch_web/
Disallow: /query/
Disallow: /settings/
User-agent: Slurp
Disallow: /accounts/
Disallow: /activity/
Disallow: /ajax/
Disallow: /client_error/
Disallow: /deck/
Disallow: /intent/
Disallow: /logging/
Disallow: /onboarding/
Disallow: /publicapi/
Disallow: /qp/batch_fetch_web/
Disallow: /query/
Disallow: /settings/
User-agent: teoma
Disallow: /accounts/
Disallow: /activity/
Disallow: /ajax/
Disallow: /client_error/
Disallow: /deck/
Disallow: /intent/
Disallow: /logging/
Disallow: /onboarding/
Disallow: /publicapi/
Disallow: /qp/batch_fetch_web/
Disallow: /query/
Disallow: /settings/
User-agent: TelegramBot
Disallow: /accounts/
Disallow: /activity/
Disallow: /ajax/
Disallow: /client_error/
Disallow: /deck/
Disallow: /intent/
Disallow: /logging/
Disallow: /onboarding/
Disallow: /publicapi/
Disallow: /qp/batch_fetch_web/
Disallow: /query/
Disallow: /settings/
User-agent: Twitterbot
Disallow: /accounts/
Disallow: /activity/
Disallow: /ajax/
Disallow: /client_error/
Disallow: /deck/
Disallow: /intent/
Disallow: /logging/
Disallow: /onboarding/
Disallow: /publicapi/
Disallow: /qp/batch_fetch_web/
Disallow: /query/
Disallow: /settings/
User-agent: *
Disallow: /
Sitemap: https://www.threads.com/sitemap/threads_profiles_sitemap.xml.gz
Sitemap: https://www.threads.com/sitemap/threads_tags_hashtag_sitemap.xml.gz
Sitemap: https://www.threads.com/sitemap/threads_profiles_sitemap.xml.gz
Sitemap: https://www.threads.com/sitemap/threads_tags_gsc_bulk_keywords_sitemap.xml.gz
Sitemap: https://www.threads.com/sitemap/threads_tags_typeahead_sitemap.xml.gz
Sitemap: https://www.threads.com/sitemap/threads_tags_hashtag_sitemap.xml.gz
DTLS Certificate Expiry & RecommendationsAction7 days until leaf cert expires — 4 issues to addressFIX
Certificate validity
Recommended actions
- Renew certificate — 7 days remaining
- Submit your domain to hstspreload.org to be added to the Chrome preload list
- Enable DNSSEC on your domain for DNS spoofing protection
- Enable OCSP stapling on your TLS server to remove a CA roundtrip and protect user privacy
DCDN & DeliveryActionNo CDN detectedFIX
Consider using a CDN to improve global delivery speed and reduce origin load.
ADNS Records1 A records, 18 ms lookupPASS
| A | 57.144.222.192 |
| AAAA | 2a03:2880:f36f:c0:face:b00c:0:43fe |
| CNAME | threads.net |
| NS | b.ns.facebook.com, c.ns.facebook.com, a.ns.facebook.com, d.ns.facebook.com |
| MX | — |
| TXT | facebook-domain-verification=hiqltzmr7yrusmgfntnkxfgkb5u2tm j5sc6587dxzfj90qfmqnxnlq8c41hwnm google-site-verification=GpsmbpsLILYk3Kq9pLJXUil-TZ8NW1hrGKIt2whBGxE google-site-verification=Wby5jFTz1ZZYzgOO08qcjxlvP7cM_17A6w5HOijJsKc SPF v=spf1 a ~all |
| CAA | Lookup not available with standard resolver |
Multiple A records provide failover if one server goes down.
Single A record means a single point of failure — if that IP goes down, your site is unreachable until DNS TTL expires.
Learn more ▾ ▴
Add multiple A records for round-robin failover, or use a managed DNS provider with health-checked failover (Route 53, Cloudflare, NS1). Short TTL (60-300s) lets clients recover faster on outages.
Source: SRE practice / DNS architecture
A CNAME at the zone apex can break MX and NS records. Use ALIAS/ANAME or A records instead.
CNAME at the apex (example.com) breaks every other apex record (MX, TXT, NS) — DNS-protocol violation per RFC 1034.
Learn more ▾ ▴
RFC 1034 forbids CNAME alongside other records at the same name. Some DNS providers offer ALIAS / ANAME / flattened-CNAME records that work around this — use those instead. Otherwise apex-level CNAME breaks email (no MX), domain ownership verification (no TXT), and more.
Source: RFC 1034
CAA record lookup requires a specialized DNS resolver. This check will be available in a future update.
Informational: CAA (Certification Authority Authorization) records weren't checked in this scan.
ARedirect Chain1 redirect(s), 193 ms totalPASS
https://www.threads.net
36 ms · HTTP/1.1
https://www.threads.com/
157 ms · HTTP/1.1 FINAL
| # | URL | Status | Time | Protocol | Server |
|---|---|---|---|---|---|
| 1 | https://www.threads.net | 301 | 36 ms | HTTP/1.1 | |
| 2 | https://www.threads.com/ | 200 | 157 ms | HTTP/1.1 |
See the visual redirect chain in the HTTP Probe tab →
A+IPv6 ReadinessIPv6 reachable (2 ms)PASS
A+URL Variantswww/non-www, trailing slash, HTTP→HTTPSPASS
www / non-www
Preferred variant: www
HTTP → HTTPS
Consistent
ADomain Intelligencethreads.net — via RegistrarSEC LLC, 29 years, 3 months oldPASS
EXPIRED
May 27, 2026
7 days
Issued by DigiCert Inc
29 years, 3 months
Registered May 28, 1997
Not enabled
Protects against DNS spoofing
Unknown
2a03:2880:f36f:c0:face:b00c:0:43fe
RegistrarSEC LLC
Expiry timeline
Recommended actions
- Domain has EXPIRED — renew immediately to avoid total site outage
- Renew the TLS certificate or verify auto-renewal is working
- Enable DNSSEC to protect visitors from DNS spoofing
Consider enabling auto-renewal to prevent accidental expiration.
Domain expiry approaching — renew immediately and ensure auto-renew + alerting are configured.
Source: ICANN renewal policy
DNSSEC protects against DNS spoofing attacks. While not required, enabling DNSSEC adds an additional layer of security. Contact your DNS provider to enable it.
Without DNSSEC, an attacker who can poison your DNS can hijack your domain — and SSL certs alone don't stop them.
Learn more ▾ ▴
DNSSEC adds cryptographic signatures to DNS records, preventing forged responses from poisoning resolver caches. Without it, an attacker who controls the network path can redirect your domain to a malicious server before any HTTPS handshake happens. Most modern registrars (Cloudflare, Google Domains, Route 53) enable it with one toggle.
Source: ICANN / RFC 4033