How well can AI read construction drawings?

We ask the models 169 questions across 9 real drawing sets, like "What's the concrete spec for the F2 footings?" and "How many F2 footings are there?", and count how many they get right.

169
Questions
9
Real drawing sets
37
Raw models
22
Harness + model
82.1%
Best model alone
93.3%
Best in a harness

Inside a harness

# Harness Model Accuracy vs raw Details

Each pairing ran once, and repeat runs move by 1.1 to 5.4 points on their own.

The models on their own

# Model Lab Accuracy Cost Details

Cost is one run of all 169 questions.

The categories

What we asked them to do

The questions split into 9 kinds of work, and some are much harder than others. A model can read a schedule well and still not be able to count.

Reading the sheet

32 questions

Finding a fact the drawing states outright, in the title block, the general notes or the legend.

Schedules

18 questions

Pulling one value out of a table such as a door schedule, panel schedule or equipment list. Easy to get wrong when the table runs across half a sheet.

Counting

41 questions

Counting tagged items on a plan, like fittings or outlets. This is the core takeoff skill.

Element detail

25 questions

The size, type or material of a named element, like a beam size, a pipe material or a cable size.

Scale and measurement

22 questions

Distances, areas and volumes the model has to measure off the drawing at scale, because the number isn't printed anywhere.

Cross-referencing

14 questions

Answers that need more than one sheet, or tracing a single line diagram through.

Scope

7 questions

What's in and what's out: new or existing, who supplies it, and whether it's in this package or another one.

Sheet structure

7 questions

Reading the sheet itself: how many details or grid lines it has, and what revision it's at.

Knowing when to stop

3 questions

Questions the drawings don't answer, where the right answer is to say so.

We re-run this on every major release

Every new model goes through the same 169 questions. Join the community to get the results as they come in, and the workflows we build from them.

Get the results first →