Proof, with its limits

How we check every kit

How the numbers in a kit are produced and checked, the record for each of the six kits, and what the checks cannot tell you.

A plan is only as good as its numbers. This is how ours are produced and checked, in the order it happens.

  1. STAGE 1

    One model. Each kit is written down once, as a list of parts whose sizes are formulas of the sizes you type. Code turns that one definition into the spreadsheet, the drawings, the cut list and the cutting diagram. If the drawn cutting diagram and the spreadsheet's cutting plan differ by a line, the release stops.

  2. STAGE 2

    A second program. A separate program, written from the same specification and sharing no code with the first, works out what every cell should say. LibreOffice recalculates the real spreadsheet at 56 sets of inputs and each answer is compared.

  3. STAGE 3

    A blind review. Reviewers read the kit's files cold and look for faults. They are AI models, not people. The six kits have not all had this stage in the same form; the record is further down.

What the automated test does

The test builds 56 sets of inputs for a kit:

  • the default size, once in millimetres and once in inches
  • 6 corner cases, with sizes at the smallest and the largest values the spreadsheet accepts
  • 30 random sizes inside the ranges
  • 3 with board lengths other than the default
  • 2 with the saw kerf set to zero
  • 13 that are wrong on purpose: 2 with a unit that is neither mm nor in, and 11 with a value outside its range or typed as text

Each set is written into the spreadsheet. LibreOffice recalculates it, and every cell the test knows about is read back and compared with the second program's answer: part lengths and widths, quantities, the number of boards and sheets, which part is cut from which board, the wording of every warning and of the Status line, the inch fractions, the hardware counts and the timber cost.

The 13 wrong sets have to be refused. For those, the Status line must read CHECK INPUTS.

Last, the test changes three of its own expected answers on purpose and confirms that the comparison reports exactly those three. A comparison that let everything through would fail at this step.

The record for each kit

KitChecksFailedRecalculated inReport
Chicken Run87,5860LibreOffice, 56 sets28 Oct 2026
Chicken Coop133,2560LibreOffice, 56 sets
Apple Numbers, 8 sets
20 Oct 2026
Chicken TractorNot re-tested: see below the table
Dog House52,3040LibreOffice, 56 sets
Apple Numbers, 8 sets
6 Oct 2026
Garage Workbench46,7350LibreOffice, 56 sets28 Oct 2026
Outdoor Cat Shelter84,5360LibreOffice, 56 sets12 Nov 2026

One row per kit, taken from that kit's own test report. A kit page shows the same record beside its spreadsheet notes.

Chicken Tractor. The file on sale differs from the one the last report covers. The test has not been run on it again, so we print no figures. Its page describes a spot check made on 14 Nov 2026.

A large count needs reading with care. One check is one comparison of one cell. In the Chicken Run report, 37,107 of the 87,586 checks only confirm that a cell does not show an error. The other 50,479 compare a number or a piece of text with the expected one.

What that test cannot see

The second program checks that the spreadsheet does what the kit's definition says. It cannot check the definition. If the definition describes a joint that cannot be built, and describes it the same way everywhere, both programs agree and the test passes.

We measured how much that matters. On 20 Oct 2026 we made 42 copies of kit definitions. In 26 of them we planted an error of that kind: one formula changed, and the built-in rule that restates it changed to match. Six more carried a plain error and ten were left untouched. Each copy went to an AI reviewer that was given the files and no hints.

Of the 26 planted errors:

  • 9were caught by the kits' own built-in rules, run at 400 random sizes. The other 17 passed.
  • 26were caught by the reviewers.
  • 24were caught by the reviewers if only their high-confidence findings are counted.

The reviewers raised false alarms as well. By the scoring rule we fixed before the test, 15 of their 47 findings counted as wrong. Among the 30 high-confidence findings, none did.

Four of those 15 were raised against copies we had not touched. We checked each by hand and all four were real faults. Three were in the Chicken Tractor, which was on sale at the time. They were fixed that day, and the corrected files replaced the old ones on Etsy on 21 Oct 2026.

That result is why a passing test is not treated as proof here.

The blind reviews on record

A reviewer starts with a kit's files and nothing else. It is an AI model, not a person and not an engineer. What it does well is read a design for contradictions: a screw row that misses the timber under it, build steps in the wrong order, a hinge drawn on the wrong side. Those three are real findings from the reviews listed here.

  • Chicken Run

    28 Oct 2026

    Two reviewers, 15 findings. Each was reproduced with numbers and fixed. The kit was then rebuilt and the automated test run again.

  • Chicken Coop

    20 Oct 2026

    One reviewer read an untouched copy in the test above and raised nothing. Three faults that came to light in that day's reviews were then fixed, and the kit was rebuilt and tested again. We have no record of a review since.

  • Chicken Tractor

    20 Oct 2026

    One reviewer raised three findings against an untouched copy in the test above. All three were real and were fixed that day, with the three faults it shared with the Chicken Coop. We have no record of a review since.

  • Dog House

    6 Oct and 20 Oct 2026

    On 6 Oct three reviewers found three faults that would have spoiled the build. All three were fixed that day. On 20 Oct one reviewer read an untouched copy in the test above and raised nothing.

  • Garage Workbench

    28 Oct 2026

    Two reviewers, 14 findings. Each was reproduced with numbers and fixed. The kit was then rebuilt and the automated test run again.

  • Outdoor Cat Shelter

    12 Nov 2026

    Three reviewers, 30 findings. All were dealt with in the kit except three: the wording of a title, two labels on the foam cutting diagram, and a note buyers do not see. A second check the same day raised three more points, which were also dealt with before the last test run.

A review can miss things too. On 20 Oct the reviewer who read the Chicken Coop raised nothing, and three faults in that kit were fixed the same day.

What we do not claim

  • No load, weight or strength rating. These are plans and calculators, not engineered products. Nothing on this page tests whether a build is strong enough for any use. Each kit page prints that kit's own safety note and limits in full.
  • Not a physical build. We have not built or photographed any of the six kits ourselves. Every check above is done on the files.
  • Not the design. The checks do not show that a design is the best way to build the thing, or that it suits your site, your timber or your animals.
  • Not the smallest board count. Our cutting plan is a good one, but nobody promises it uses the fewest boards possible. The test measures this for boards, using a second solver that finds the true minimum when the job is small enough for it. Where it found one, the plan used one board more in 4 of 42 board counts for the Chicken Run (22 more were too large for that solver), 4 of 86 for the Chicken Coop, 8 of 43 for the Dog House, 5 of 43 for the Garage Workbench and 2 of 86 for the Outdoor Cat Shelter. Sheet counts are not compared with a true minimum. The design note for the sheet method says its count is never too low but can be one sheet high. The spreadsheet prints a lower bound beside each count, a number no cutting plan can go under, so you can see how much room is left.
  • Pictures are not photographs. The step pictures are software renders drawn from each kit's part sizes. The photo-style pictures are AI-generated and are labelled where they appear.
  • Build steps. The written steps were drafted with AI help. They have been read by the reviewers above where a review is on record. We have not built from them ourselves.
  • Spreadsheet programs. A kit is "tested in" only the programs named in its row of the first table. We have no test on record for these six kits in Excel or Google Sheets, so we do not say they work there.

Before you cut

Whatever we check, the timber in your hands decides.

  • Read the Status line in the spreadsheet. Buy timber only when it reads OK.
  • Measure the boards and sheets you can really buy, and type those sizes in.
  • Check each cut part against the cut list before you assemble anything.

If a number looks wrong, message us on Etsy (opens in a new tab) with the kit, the sizes you typed and what you see.