Parser Backend Audit

cargo xtask parser-audit compares mdwright's production pulldown-cmark backend with cmark-gfm, using the vendored GFM spec expected HTML as the primary oracle. The audit renders mdwright through the opt-in cmark-gfm render profile so renderer spelling drift is separated from parser-tree drift. It does not replace mdwright-document as the production parser boundary.

cmark-gfm is the primary oracle because crates/mdwright/tests/gfm-spec/spec.txt is vendored from cmark-gfm and the GFM ecosystem treats its rendered HTML as the reference. comrak is optional diagnostic evidence for rendered HTML and source-position behaviour; it is not a release gate unless a future audit shows it catches mdwright-relevant risks that cmark-gfm cannot expose.

Running

cargo xtask parser-audit \
  --case-set all \
  --output target/mdwright/parser-audit \
  --ensure-tools \
  --include-comrak

The command builds a pinned cmark-gfm under target/mdwright/tools/ when --ensure-tools is passed. To use an already-built binary explicitly, pass --cmark-gfm-bin <path>.

Reports are written to:

• target/mdwright/parser-audit/parser-audit.json
• target/mdwright/parser-audit/parser-audit.md

Examples marked disabled in the vendored GFM spec are still reported, but cmark-gfm binary drift from the expected HTML for those cases is not a command failure because the upstream spec does not treat the rendered checkbox spelling as a strict conformance assertion.

The audit also checks source-position envelopes for constructs mdwright uses as formatter/linter facts. It maps cmark-gfm data-sourcepos line/column ranges back to source bytes and compares them against mdwright document facts by construct kind. This is a risk gate, not exact AST equality: a difference is reported only when mdwright has no overlapping fact for a rewrite/lint-owned construct.

Status Values

• pulldown-html-mismatch: mdwright's pulldown-backed HTML differs from cmark-gfm expected HTML.
• mdwright-policy: mdwright intentionally differs from the cmark-gfm oracle for a documented parser policy.
• extension-gap: the compared parser does not implement the construct.
• sourcepos-risk: rendered output matches, but coordinate facts may affect formatter/lint safety.
• event-only: internal event/AST shape differs while rendered HTML and semantic signatures match.
• upstream-panic: parser panic or crash contained by mdwright-document.
• needs-mdwright-mitigation: upstream behaviour is unsafe for mdwright and still needs a fix.
• fixed: the difference should no longer appear; the audit fails if it does.

Classifications

Current gfm-spec audit snapshot with mdwright's cmark-gfm render profile:

Observed difference classes:

The cmark-gfm render profile is an HTML spelling profile. It fixes quote escaping, link-destination escaping, ordinary GFM table spelling, task-list checkbox spelling, and one newline-placement case where the parser already exposes enough structure. It does not change emphasis resolution or source-position semantics. Full cmark-gfm parser equivalence would require upstream pulldown changes, a maintained fork, or a backend switch.

Replacement Criteria

Do not replace pulldown-cmark based on event-shape differences alone. A replacement candidate must improve at least one release-relevant axis without regressing the others:

• fewer unclassified or policy-relevant HTML mismatches against cmark-gfm;
• safer behaviour on malformed/user input;
• stable byte/source coordinates sufficient for formatter rewrite ownership;
• extension coverage at least as good as the current document facts;
• acceptable runtime and dependency footprint.