04 / EVIDENCE STUDY
A structural rubric for fictional project briefs
A reproducible eight-check run across six deliberately varied fictional briefs, with the fixtures, results and chart available for inspection.
A project brief can look complete while leaving a missing role, boundary, failure case or observable result hidden in polished prose. This study asks a narrow question: can eight fully disclosed structural checks detect those deliberately introduced gaps in a small set of fictional briefs?
The run evaluated six invented reading-list briefs. One contained every signal in the rubric, one contained only a positive path, and four removed or weakened one signal apiece. The fixtures, per-rule outcomes and chart all come from the same deterministic script.
Define checks that a reader can reproduce
The rubric awards one point for each of eight visible signals: a role more specific than the exact generic labels user, someone, person or people; a goal containing one token from a disclosed observable-action list; a constraints field containing a number or an explicit unknown; positive, negative and edge cases; complete Given, When and Then fields; and non-empty outcomes that avoid a disclosed vague-word list.
Those rules are intentionally mechanical. The observable-action list includes download, export, open, read, save, retain, remove, restore, share, sort, submit and view. The vague-word list includes quickly, easy, intuitive, seamless, correctly, properly, user-friendly, works and successful. The JSON record preserves the complete fixture text and every Boolean outcome.
A check can only report whether its defined signal is present. For example, a number in the constraints field passes the boundary-recorded check even if that number is a bad product decision. That narrowness is part of the method, not a claim that the rubric understands the brief.
Sources: Cucumber: Gherkin reference
Run six deliberately different fixtures
The complete mixed-case fixture contains all eight signals. The happy-path-only fixture omits negative and edge cases. The remaining four fixtures each alter one signal: a generic actor, an unrecorded boundary, a vague outcome or an empty Then field.
The generator loads the fixture file, refuses any record not marked fictional, verifies unique IDs and applies the same ordered rule list to every fixture. It derives the run ID from hashes of the fixture source, canonical rubric specification, evaluator source and resulting evaluations, so a change to any input, rule semantics, implementation or output changes the identifier.
| Fixture | Deliberate variation | Expected missed check |
|---|---|---|
| Complete mixed-case brief | No signal removed | None |
| Happy path only | Negative and edge cases removed | negative-case; edge-case |
| Generic role | Actor set to User | specific-role |
| Unrecorded boundary | No number or explicit unknown | boundary-recorded |
| Vague outcome | Outcome says works correctly | specific-outcomes |
| Unfinished step | One Then field left empty | complete-steps |
Inspect the actual run
Run d1acf8d76d0cc7c9 produced 48 Boolean results. The complete fixture scored 8/8. The happy-path-only fixture scored 6/8 because it lacked both a negative case and an edge case. Each single-gap fixture scored 7/8 and missed the check designed for that gap.
The chart is generated from the same result objects written to JSON and CSV. It is a view of rubric output, not a survey result, product benchmark or test report.
| Fixture | Score | Missed checks |
|---|---|---|
| Complete mixed-case brief | 8/8 | None |
| Happy path only | 6/8 | negative-case; edge-case |
| Generic role | 7/8 | specific-role |
| Unrecorded boundary | 7/8 | boundary-recorded |
| Vague outcome | 7/8 | specific-outcomes |
| Unfinished step | 7/8 | complete-steps |
Read the result at the right scale
Within this constructed fixture set, the rubric behaved as specified: every deliberately removed signal produced its corresponding failure and no other failure. That is a test of deterministic rule behavior on known inputs. It is not evidence that eight checks can evaluate arbitrary project briefs.
The checks do not evaluate whether a role reflects real users, whether a numeric limit has evidence behind it, whether an outcome is desirable, or whether the cases cover meaningful risk. Human review and product-specific evidence remain necessary for those decisions.
The useful artifact is the auditable separation between input, rule and output. A team can replace these fictional fixtures with its own controlled examples, change the rules openly and preserve a new version instead of treating the 8/8 score as a certificate.
Continue from evidence to a working brief
Use the Acceptance Criteria Builder to draft a goal, role, constraints and cases. Its exports remain marked draft-not-tested because writing an expected outcome does not execute a test.
Read Write a brief that can be checked for the project-brief method, Make the outcome something you can observe for clearer Then statements and Check the boundaries of a promise for negative and edge cases.
The Well Made Ideas home page collects the tool and guides. The publication profile identifies Awesome Patel as the Chicago publisher and points to the primary SaaS Mirror profile.