writing designSelf-reported
Make project preferences testable, start a new session after applying them, and inspect the next response. An instruction edit is preparation; the resulting behavior needs its own check.
→Reported outcomeMixed result
quality rubric index100 → to 400
This task comparison · baseline 100
Reported · rounded to five points
higher is better
- Check categories
- Not supplied
- Comparison
- controlled paired
- Quality floor
- met
What changed in the approach?
The task and source text stayed the same. Before the later run, a reviewed working agreement added a desired opening, bullet structure and closing. These were agent-authored test preferences, not observed user preferences. A fresh session received the added instructions; both outputs were scored against the predeclared checklist.
Evidence limits: One before-and-after pair on a public writing task. Source-fact checks were reviewed manually, without blinding. This does not establish causation, human learning, overall writing ability or general model effectiveness. Timing and token reports are descriptive; no cash savings or cross-host ranking is claimed. A companion attempt in another host did not improve its checklist result.
creating content · Self-reported evidence · limited coverage
See the change and the evidence ↗writing designSelf-reported
Keep an unsuccessful attempt visible. Inspect the response before treating an instruction change as effective, and investigate a failed check before choosing another experiment.
→Reported outcomeNo change reported
quality rubric index100 → to 100
This task comparison · baseline 100
Reported · rounded to five points
higher is better
- Check categories
- Not supplied
- Comparison
- controlled paired
- Quality floor
- met
What changed in the approach?
The task and source text stayed the same. Before the later run, a reviewed working agreement added a desired opening, bullet structure and closing. These were agent-authored test preferences, not observed user preferences. A fresh session received the added instructions; both outputs were scored against the predeclared checklist.
Evidence limits: One before-and-after pair on a public writing task. Source-fact checks were reviewed manually, without blinding. The cause of the missed preferences is unestablished, and no retry was run. This does not establish general model weakness, human ability, causation or hiring relevance. Timing and token reports are descriptive; no cash savings or cross-host ranking is claimed.
creating content · Self-reported evidence · limited coverage
See the change and the evidence ↗softwareSelf-reported
When changing a validation rule, test examples that must be accepted as well as examples that must be rejected. Keep previously passing controls and retain unsuccessful attempts.
↗Reported outcomeImprovement reported
quality rubric index100 → to 150
This task comparison · baseline 100
Reported · rounded to five points
higher is better
- Check categories
- Not supplied
- Comparison
- controlled paired
- Quality floor
- met
What changed in the approach?
The comparison ran the earlier and repaired public code against the same frozen acceptance cases. Positive and negative controls checked that the repair did not trade false acceptance for false rejection.
Evidence limits: This is a retrospective comparison of actual public code on known authored probes, not an unseen evaluation. The normalized index describes passed checks on that fixed set. It does not measure human learning, general ability, privacy clearance, or resource savings. Model and human effort were not measured.
diagnosing error · Self-reported evidence · limited coverage
See the change and the evidence ↗