Check the expected values in generated tests

Published by OpenAI Cookbook

This credits the original publisher. Better Loop membership or a shared assessment is not implied.

The public work

OpenAI publishes a multi-step test-writing notebook with a concrete Pig Latin function and generated pytest cases. Several printed expected values conflict with the function, making this a worked example of why generated tests themselves need review.

What to notice

Syntactic validity is not a reliable test oracle. Derive expected values from intended behavior before trusting generated assertions.

Keep the context

The notebook shows generated code, not a passing test run. It uses an older model and contains code and expectation inconsistencies; this is not a ready-to-run test generator or a validated accuracy result.

AI use: Documented in the source.

The displayed notebook uses gpt-3.5-turbo through the OpenAI API for explanation, planning and test generation.

A useful public example is not an assessment of a reader, a publisher or a Better Loop member.

Authored practice suggestion

Try the idea.
Check your own work.

Use material you are allowed to work with. This suggestion is preparation; it does not record a completed task or an improvement.

A prompt to adapt privately

Take a small public function. Explain its intended behavior, list normal and edge cases, then propose tests. For every expected value, show a short derivation independent of the generated test code. Flag disagreements between the implementation and the intended specification before changing either.

A check to adapt

Every expected value has a defensible derivation, and implementation bugs are distinguished from mistaken test expectations.

Open a related practice brief

Private practice needs no account. Your configured model provider may process the evidence you select.

See the full skill-to-sharing flow

Another way into the work.

Iterate on a tolerant XML parser

Ronacher published a tolerant XML parser after earlier parsing approaches failed. He describes directing the initial implementation, reviewing it, requesting further fixes and an extensive test suite, and releasing the resulting library.

Published byArmin Ronacher

Curated public exampleCode

Code & software

Expose clipboard formats before debugging paste

Willison built a browser clipboard viewer while investigating rich-text paste. His public write-up links the code and conversation; the conversation shows requests to display available formats and then make long output easier to inspect.

Published bySimon Willison