Restarting W3C QA

More details about this document
Identifier
https://csarven.ca/presentations/restarting-w3c-qa
Author
Sarven Capadisli
Published
Modified
License
CC BY 4.0
Language
English
Document Type
Slideshow
Topics
Audience

Restarting W3C QA

W3C TPAC 2025, Breakout, Kobe,

Sarven Capadisli Sarven Capadisli https://csarven.ca/#i @csarven

Knowledge Graph

View this presentation's knowledge graph.

Abstract

Working Groups are expected to produce deliverables such as reports, software, test suites, as per W3C Process.

The Success Criteria of WG charters outline how interoperability will be determined, e.g., at least two independent interoperable implementations of every feature defined in the specification, where interoperability can be verified by passing open test suites.

The W3C QA Activity published guidance and design considerations.


To make better specifications/standards, QA work needs to advance at W3C alongside other developments.

W3C QA Activity

Some of the output of the QA Working Group

QA Framework Primer and Usage Scenarios
[A] general orientation to the QA Framework
QA Handbook
A non-normative handbook about the process and operational aspects of certain quality assurance practices [..] focus on testability and test topics.
QA Specification Guidelines
[T]o help W3C editors write better specifications, by making a specification easier to interpret without ambiguity and clearer as to what is required in order to conform.
Variability in Specifications
[A]nalyzes how design decisions of a specification's conformance model may affect its implementability and the interoperability of its implementations.
A Method for Writing Testable Conformance Requirements
[A] method for writing, marking-up, and analyzing conformance requirements in technical specifications.
Test FAQ
[I]nformation about the purpose of testing, how to get started, and what the testing process involves.

Technical Report - Description

A typical technical report details:

  • Document details (e.g., title, contributors, dates, versions, rights and policies, test suite, feedback, errata, translations)
  • Abstract, status, introduction
  • Terminology, namespaces
  • Normative and informative content
  • Citations, examples, notes, diagrams, source of consensus
  • Conformance (key words, specification category, classes of products, interoperability)
  • Considerations (Security, Privacy, Accessibility, Internationalization, Self-Reviews)
  • Changelog
  • Acknowledgements (of individuals or communities that influenced the work)
  • References (normative, informative)
  • Appendix

Normative and Informative Content

Common terms from a controlled vocabulary often used in specifications to distinguish normative and informative content. Spec Terms incorporates these concepts.

Normative content
Mandatory rules, requirements, or behaviours that determine conformance.
Often signalled with key words such as “MUST” (“REQUIRED”, “SHALL”), “MUST NOT” (“SHALL NOT”), “SHOULD” (“RECOMMENDED”), and “MAY” (“OPTIONAL”) interpreted according to BCP 14 [RFC2119] [RFC8174].
Informative content
Explanations, examples, diagrams, notes, and guidelines.
Often includes key words such as “strongly encouraged”, “strongly discouraged”, “encouraged”, “discouraged”, “can", “cannot”, “could”, “could not”, “might”, and “might not”.

Interoperability

Which product classes defined by a specification are required to work together?

Are implementations actually interoperating? How can we assess and manage this in practice, including partial or optional features?

Who ensures interoperability as a specification evolves?

Authoring specifications

Any authoring tool can be used to author.

Documents must follow publication and transition rules.

Advance machine-readability of specifications to help with QA.

Accessible authoring tools (Authoring Tool Accessibility Guidelines) to create accessible content (Web Content Accessibility Guidelines).

Video of highlighting structured data in specifications in dokieli [, WebM]
Video of Conformance Requirements and Test Coverage in dokieli [, WebM]

What makes a good standard?