Reference library
Standard
The specifications SEO is actually built on: robots exclusion, sitemaps, structured-data vocabularies, HTTP semantics. Dry, short, and far more precise than the folklore built on top of them.
19 references · 5 canonical · 19 free · Advanced / Intermediate / Beginner
| Title | Author / publisher | Year | Level | Cost |
|---|---|---|---|---|
| RFC 9309: Robots Exclusion Protocol ↗Canon The IETF Proposed Standard that finally formalised robots.txt after 25 years of de facto use, defining syntax, matching rules, caching behaviour and error handling. rfc-editor.org | M. Koster, G. Illyes, H. Zeller, L. Sassman | 2022 | Advanced | Free |
| Schema.org Vocabulary ↗Canon The shared structured data vocabulary maintained by the major search engines and a W3C community group. The authoritative type and property reference behind almost all rich results. schema.org | Schema.org (Google, Microsoft, Yahoo, Yandex) | , | Intermediate | Free |
| Sitemaps XML Protocol 0.9 ↗Canon The jointly supported sitemap specification from Google, Yahoo and Microsoft. Defines the XML schema, the 50,000 URL and 50 MB limits, sitemap index files and robots.txt autodiscovery. sitemaps.org | sitemaps.org | , | Intermediate | Free |
| The /llms.txt File ↗Canon Proposal for a markdown file at /llms.txt that gives large language models a curated, inference-time map of a site's most useful content. A community convention rather than a ratified standard; adoption by major AI providers remains limited. llmstxt.org | Jeremy Howard | 2024 | Intermediate | Free |
| Web Content Accessibility Guidelines (WCAG) 2.2 ↗Canon W3C Recommendation defining testable accessibility success criteria at levels A, AA and AAA. Overlaps heavily with SEO through semantic markup, alt text, heading structure and link text. w3.org | W3C | 2023 | Advanced | Free |
| ai.txt by Spawning ↗ Spawning's proposed ai.txt convention, a robots.txt-style file letting site owners state opt-in and opt-out preferences for AI training use of their media and text. site.spawning.ai | Spawning AI | , | Intermediate | Free |
| HTML Standard: Document Metadata ↗ The living HTML specification section defining head, title, base, link, meta and the standard metadata names, which is the normative basis for title tags and meta descriptions. html.spec.whatwg.org | WHATWG | , | Advanced | Free |
| HTML Standard: Links ↗ The living HTML specification section defining the a and link elements, the href attribute and every standard link relation including nofollow, sponsored, ugc, canonical and alternate. html.spec.whatwg.org | WHATWG | , | Advanced | Free |
| IAB Tech Lab Standards ↗ The technical standards library for digital advertising, including ads.txt, sellers.json, OpenRTB and transparency frameworks relevant to publisher monetisation alongside organic search. iabtechlab.com | IAB Tech Lab | , | Advanced | Free |
| IETF AI Preferences (aipref) Working Group ↗ IETF working group chartered to standardise a vocabulary and attachment mechanism for expressing preferences about automated processing of web content by AI systems. The formal successor track to ad hoc conventions such as ai.txt. datatracker.ietf.org | IETF | , | Advanced | Free |
| IndexNow Protocol Documentation ↗ Open protocol, backed by Microsoft Bing and Yandex, that lets sites push URL change notifications to participating search engines instead of waiting to be recrawled. indexnow.org | IndexNow | , | Intermediate | Free |
| JSON-LD 1.1 W3C Recommendation ↗ The W3C Recommendation defining JSON-LD, the serialization format Google recommends for structured data. Specifies @context, @type, @id, framing and expansion. w3.org | W3C | 2020 | Advanced | Free |
| RFC 3986: Uniform Resource Identifier (URI) Generic Syntax ↗ The specification governing URL structure, percent-encoding, relative reference resolution and normalization. The basis for correct canonical URL and parameter handling. rfc-editor.org | T. Berners-Lee, R. Fielding, L. Masinter | 2005 | Advanced | Free |
| RFC 6265: HTTP State Management Mechanism ↗ The cookie specification. Relevant to SEO through cookie-based personalization, consent gating and crawler behaviour, since Googlebot does not persist cookies between requests. rfc-editor.org | A. Barth | 2011 | Advanced | Free |
| RFC 8288: Web Linking ↗ Defines the Link HTTP header and the registry of link relation types, including canonical, alternate, prev and next. Obsoletes RFC 5988. rfc-editor.org | M. Nottingham | 2017 | Advanced | Free |
| RFC 9110: HTTP Semantics ↗ The current authoritative definition of HTTP methods, status codes, and header fields. The primary reference for correct use of 301, 302, 304, 404, 410 and 503 in SEO work. rfc-editor.org | R. Fielding, M. Nottingham, J. Reschke | 2022 | Advanced | Free |
| RFC 9111: HTTP Caching ↗ Defines Cache-Control, ETag, Last-Modified and conditional requests. Underpins crawl efficiency, CDN behaviour and correct handling of If-Modified-Since by crawlers. rfc-editor.org | R. Fielding, M. Nottingham, J. Reschke | 2022 | Advanced | Free |
| The Open Graph Protocol ↗ Specification for the og: meta properties that control how URLs are rendered when shared on social platforms and in many chat and AI interfaces. ogp.me | ogp.me | , | Beginner | Free |
| Web Content Accessibility Guidelines (WCAG) 2.1 ↗ The WCAG version referenced by most current accessibility legislation, including the European Accessibility Act and many public-sector procurement rules. w3.org | W3C | 2018 | Advanced | Free |