← All work

31 / Skill A/B testing

Skill Bench

Alpha

Role
Builder
Scope
A rules parser for SKILL.md, a paired seeded generator, a rubric measured off the rendered page, paired bootstrap statistics, verdicts and a persisted leaderboard
Outcome
Real statistics over a procedural generator, not a model.
Twelve paired generations, skill off and on. Generated by a seeded procedure, not a model.

Does the skill actually change the output?

Every score shows what was measured on the rendered page.
A two-rule skill moves exactly one criterion. The zeros are exact.
A slow zoom over a still screenshot.

Results

  • 10 of 10seeds with a confirmed lift for frontend-design
  • 57tests passed across four suites
  • 7rubric criteria, measured off the rendered page