Can AI do a quantity takeoff?

We hand the models a blank bill of quantities and have them fill it in from the drawings, across two drawing sets.

130

Lines to price

Models tested

Harnesses tested

Priceable bills

Inside a harness

#	Harness	Model	Correct	Lines	Declined	Counts	Measured	vs raw	Details
1	Cowork electrical + plumbing · both sets in one session	Claude Opus 5.5	53.1%	130	3	69%of 70	30%of 50	+24.6 pts API, same brief: 28.5%	›
Cowork · Claude Opus 5.5 by set 69 of 130 lines inside tolerance, 3 declined. Both sets in one session, with only the folder and the two blank bills. Performing arts centre electrical 70.8% 48 lines · raw 41.7%, +29.2 pts Supermarket plumbing 42.7% 82 lines · raw 20.7%, +21.9 pts
2	Gemini app electrical + plumbing · Task mode	Gemini 4 Argon	52.3%	130	2	73%of 70	22%of 51	+25.4 pts API, same brief: 26.9%	›
Gemini app · Gemini 4 Argon by set 68 of 130 lines inside tolerance, 2 declined. One session per set, with only the folder and the blank bill. Performing arts centre electrical 72.9% 48 lines · raw 31.3%, +41.7 pts Supermarket plumbing 40.2% 82 lines · raw 24.4%, +15.8 pts
3	Cowork electrical + plumbing · both sets in one session	Claude Sonnet 5.5	49.2%	130	3	69%of 70	20%of 50	+45.4 pts API, same brief: 3.9%	›
Cowork · Claude Sonnet 5.5 by set 64 of 130 lines inside tolerance, 3 declined. Both sets in one session, with only the folder and the two blank bills. Performing arts centre electrical 75.0% 48 lines · raw 6.3%, +68.8 pts Supermarket plumbing 34.2% 82 lines · raw 2.4%, +31.7 pts
4=	ChatGPT Work electrical + plumbing · both sets in one session · self-built tooling	GPT 6 Astra	46.2%	130	8	61%of 70	28%of 47	+7.7 pts API, same brief: 38.5%	›
ChatGPT Work · GPT 6 Astra by set 60 of 130 lines inside tolerance, 8 declined. Both sets in one session, with only the folder and the two blank bills. Performing arts centre electrical 60.4% 48 lines · raw 62.5%, −2.1 pts Supermarket plumbing 37.8% 82 lines · raw 24.4%, +13.4 pts
4=	ChatGPT Work electrical + plumbing · both sets in one session · self-built tooling	GPT 6.1 Sol	46.2%	130	16	63%of 68	30%of 40	+9.2 pts API, same brief: 36.9%	›
ChatGPT Work · GPT 6.1 Sol by set 60 of 130 lines inside tolerance, 16 declined. Both sets in one session, with only the folder and the two blank bills. Performing arts centre electrical 64.6% 48 lines · raw 56.3%, +8.3 pts Supermarket plumbing 35.4% 82 lines · raw 25.6%, +9.8 pts
6	Cowork electrical + plumbing · both sets in one session	Claude Fable 5.1	43.9%	130	5	67%of 70	8%of 48	no raw run on both sets	›
Cowork · Claude Fable 5.1 by set 57 of 130 lines inside tolerance, 5 declined. Both sets in one session, with only the folder and the two blank bills. Performing arts centre electrical 68.8% 48 lines · raw 29.2%, +39.6 pts Supermarket plumbing 29.3% 82 lines
7	Claude Code electrical + plumbing · 1 session each	Claude Opus 5	43.1%	130	2	67%of 70	6%of 51	+12.7 pts API, same brief: 30.4%	›
Claude Code · Claude Opus 5 by set 56 of 130 lines inside tolerance, 2 declined. One session per set, with only the folder and the blank bill. Performing arts centre electrical 66.7% 48 lines · raw 37.5%, +29.2 pts Supermarket plumbing 29.3% 82 lines · raw 26.2%, +3.0 pts
8	Cursor electrical + plumbing · 1 session each	Claude Opus 5	40.8%	130	24	66%of 68	6%of 31	+10.4 pts API, same brief: 30.4%	›
Cursor · Claude Opus 5 by set 53 of 130 lines inside tolerance, 24 declined. One session per set, with only the folder and the blank bill. Performing arts centre electrical 70.8% 48 lines · raw 37.5%, +33.3 pts Supermarket plumbing 23.2% 82 lines · raw 26.2%, −3.0 pts
9	ChatGPT Work electrical + plumbing · 1 session each	GPT 5.6 Sol	35.4%	130	—	55%of 71	4%of 52	+0.4 pts API, same brief: 35.0%	›
ChatGPT Work · GPT 5.6 Sol by set 46 of 130 lines inside tolerance. One session per set, with only the folder and the blank bill. Performing arts centre electrical 45.8% 48 lines · raw 52.1%, −6.3 pts Supermarket plumbing 29.3% 82 lines · raw 25.0%, +4.3 pts
10	Codex electrical + plumbing · 1 session each	GPT 5.6 Sol	32.3%	130	2	51%of 71	2%of 50	−2.7 pts API, same brief: 35.0%	›
Codex · GPT 5.6 Sol by set 42 of 130 lines inside tolerance, 2 declined. One session per set, with only the folder and the blank bill. Performing arts centre electrical 45.8% 48 lines · raw 52.1%, −6.3 pts Supermarket plumbing 24.4% 82 lines · raw 25.0%, −0.6 pts
11=	ChatGPT Work electrical + plumbing · both sets in one session	GPT 6 Sol	30.8%	130	62	59%of 58	33%of 3	+3.1 pts API, same brief: 27.7%	›
ChatGPT Work · GPT 6 Sol by set 40 of 130 lines inside tolerance, 62 declined. Both sets in one session, with only the folder and the two blank bills. Performing arts centre electrical 45.8% 48 lines · raw 41.7%, +4.2 pts Supermarket plumbing 21.9% 82 lines · raw 19.5%, +2.4 pts
11=	Cursor electrical + plumbing · 1 session each	Grok 4.6	30.8%	130	14	50%of 69	0%of 41	no raw run on both sets	›
Cursor · Grok 4.6 by set 40 of 130 lines inside tolerance, 14 declined. One session per set, with only the folder and the blank bill. Performing arts centre electrical 43.8% 48 lines · raw 25.0%, +18.8 pts Supermarket plumbing 23.2% 82 lines
13	Muse electrical + plumbing · both sets in one session	Muse Spark 1.3	16.2%	130	93	57%of 30	0%of 1	no raw run on both sets	›
Muse · Muse Spark 1.3 by set 21 of 130 lines inside tolerance, 93 declined. Both sets in one session, with only the folder and the two blank bills. Performing arts centre electrical 43.8% 48 lines Supermarket plumbing 0.0% 82 lines
14	Cowork electrical + plumbing · both sets in one session	Claude Haiku 5.5	14.6%	130	94	51%of 35	n/a	+7.7 pts API, same brief: 6.9%	›
Cowork · Claude Haiku 5.5 by set 19 of 130 lines inside tolerance, 94 declined. Both sets in one session, with only the folder and the two blank bills. Performing arts centre electrical 4.2% 48 lines · raw 6.3%, −2.1 pts Supermarket plumbing 20.7% 82 lines · raw 7.3%, +13.4 pts
15	ChatGPT Work electrical + plumbing · 1 session each	GPT 5.6 Terra	13.1%	130	96	50%of 34	n/a	−8.5 pts API, same brief: 21.5%	›
ChatGPT Work · GPT 5.6 Terra by set 17 of 130 lines inside tolerance, 96 declined. One session per set, with only the folder and the blank bill. Performing arts centre electrical 0.0% 48 lines · raw 27.1%, −27.1 pts Supermarket plumbing 20.7% 82 lines · raw 18.3%, +2.4 pts
16	ChatGPT Work electrical + plumbing · both sets in one session	GPT 6 Luna	0.0%	130	130	n/a	n/a	−29.2 pts API, same brief: 29.2%	›
ChatGPT Work · GPT 6 Luna by set 0 of 130 lines inside tolerance, 130 declined. Both sets in one session, with only the folder and the two blank bills. Performing arts centre electrical 0.0% 48 lines · raw 35.4%, −35.4 pts Supermarket plumbing 0.0% 82 lines · raw 25.6%, −25.6 pts

The models on their own

#	Model	Lab	Correct	Lines	Flagged	Counts	Measured	Details
1	GPT 6 Astra openai/gpt-6-astra	OpenAI	48.5%	130	70	66%of 71	19%of 52	›
GPT 6 Astra by set 63 of 130 lines inside tolerance, 70 flagged as estimates. When it could leave lines blank, it scored 38.5% and declined 51. Performing arts centre electrical 68.8% 48 lines Supermarket plumbing 36.6% 82 lines
2	GPT 6.1 Sol openai/gpt-6.1-sol	OpenAI	40.0%	130	61	58%of 71	10%of 52	›
GPT 6.1 Sol by set 52 of 130 lines inside tolerance, 61 flagged as estimates. When it could leave lines blank, it scored 36.9% and declined 38. Performing arts centre electrical 62.5% 48 lines Supermarket plumbing 26.8% 82 lines
3	Claude Opus 5.5 anthropic/claude-opus-5.5	Anthropic	36.9%	130	98	51%of 71	13%of 52	›
Claude Opus 5.5 by set 48 of 130 lines inside tolerance, 98 flagged as estimates. When it could leave lines blank, it scored 28.5% and declined 61. Performing arts centre electrical 43.8% 48 lines Supermarket plumbing 32.9% 82 lines
4	Claude Opus 4.7 anthropic/claude-opus-4.7	Anthropic	33.9%	130	93	51%of 71	6%of 52	›
Claude Opus 4.7 by set 44 of 130 lines inside tolerance, 93 flagged as estimates. When it could leave lines blank, it scored 20.0% and declined 88. Performing arts centre electrical 41.7% 48 lines Supermarket plumbing 29.3% 82 lines
5	GPT 5.6 Sol openai/gpt-5.6-sol	OpenAI	32.7%	130	63	50%of 71	2%of 52	›
GPT 5.6 Sol by set 43 of 130 lines inside tolerance, 63 flagged as estimates. When it could leave lines blank, it scored 35.0% and declined 3. Performing arts centre electrical 47.9% 48 lines Supermarket plumbing 23.8% 82 lines
6=	Claude Opus 5 anthropic/claude-opus-5	Anthropic	32.3%	130	94	47%of 71	7%of 52	›
Claude Opus 5 by set 42 of 130 lines inside tolerance, 94 flagged as estimates. When it could leave lines blank, it scored 30.4% and declined 8. Performing arts centre electrical 39.6% 48 lines Supermarket plumbing 28.1% 82 lines
6=	GPT 6 Luna openai/gpt-6-luna	OpenAI	32.3%	130	112	46%of 71	6%of 52	›
GPT 6 Luna by set 42 of 130 lines inside tolerance, 112 flagged as estimates. When it could leave lines blank, it scored 29.2% and declined 70. Performing arts centre electrical 39.6% 48 lines Supermarket plumbing 28.1% 82 lines
8=	Muse Wonder meta/muse-wonder	Meta	31.5%	130	108	46%of 71	6%of 52	›
Muse Wonder by set 41 of 130 lines inside tolerance, 108 flagged as estimates. When it could leave lines blank, it scored 24.6% and declined 65. Performing arts centre electrical 35.4% 48 lines Supermarket plumbing 29.3% 82 lines
8=	GPT 5.6 Terra openai/gpt-5.6-terra	OpenAI	31.5%	130	80	46%of 71	6%of 52	›
GPT 5.6 Terra by set 41 of 130 lines inside tolerance, 80 flagged as estimates. When it could leave lines blank, it scored 21.5% and declined 90. Performing arts centre electrical 35.4% 48 lines Supermarket plumbing 29.3% 82 lines
10	Gemini 3.6 Flash google/gemini-3.6-flash	Google	31.1%	130	—	46%of 71	6%of 52	›
Gemini 3.6 Flash by set 41 of 130 lines inside tolerance. When it could leave lines blank, it scored 27.7% and declined 0. Performing arts centre electrical 39.6% 48 lines Supermarket plumbing 26.2% 82 lines
11	Claude Fable 5 anthropic/claude-fable-5	Anthropic	30.8%	130	93	45%of 71	6%of 52	›
Claude Fable 5 by set 40 of 130 lines inside tolerance, 93 flagged as estimates. When it could leave lines blank, it scored 18.5% and declined 96. Performing arts centre electrical 35.4% 48 lines Supermarket plumbing 28.1% 82 lines
12	Claude Opus 4.8 anthropic/claude-opus-4.8	Anthropic	30.0%	130	107	46%of 71	2%of 52	›
Claude Opus 4.8 by set 39 of 130 lines inside tolerance, 107 flagged as estimates. When it could leave lines blank, it scored 20.8% and declined 96. Performing arts centre electrical 33.3% 48 lines Supermarket plumbing 28.1% 82 lines
13	Claude Haiku 4.5 anthropic/claude-haiku-4.5	Anthropic	29.2%	130	89	46%of 71	0%of 52	›
Claude Haiku 4.5 by set 38 of 130 lines inside tolerance, 89 flagged as estimates. When it could leave lines blank, it scored 13.9% and declined 94. Performing arts centre electrical 33.3% 48 lines Supermarket plumbing 26.8% 82 lines
14=	Claude Sonnet 5 anthropic/claude-sonnet-5	Anthropic	28.5%	130	98	45%of 71	0%of 52	›
Claude Sonnet 5 by set 37 of 130 lines inside tolerance, 98 flagged as estimates. When it could leave lines blank, it scored 11.5% and declined 112. Performing arts centre electrical 37.5% 48 lines Supermarket plumbing 23.2% 82 lines
14=	GPT 5.4 Mini openai/gpt-5.4-mini	OpenAI	28.5%	130	121	42%of 71	6%of 52	›
GPT 5.4 Mini by set 37 of 130 lines inside tolerance, 121 flagged as estimates. When it could leave lines blank, it scored 15.4% and declined 103. Performing arts centre electrical 27.1% 48 lines Supermarket plumbing 29.3% 82 lines
14=	GPT 6 Sol openai/gpt-6-sol	OpenAI	28.5%	130	46	44%of 71	2%of 52	›
GPT 6 Sol by set 37 of 130 lines inside tolerance, 46 flagged as estimates. When it could leave lines blank, it scored 27.7% and declined 67. Performing arts centre electrical 41.7% 48 lines Supermarket plumbing 20.7% 82 lines
17=	Gemini 3.1 Flash Lite google/gemini-3.1-flash-lite	Google	27.7%	130	1	45%of 71	0%of 52	›
Gemini 3.1 Flash Lite by set 36 of 130 lines inside tolerance, 1 flagged as estimates. When it could leave lines blank, it scored 23.1% and declined 53. Performing arts centre electrical 29.2% 48 lines Supermarket plumbing 26.8% 82 lines
17=	Gemini 3.7 Flash google/gemini-3.7-flash	Google	27.7%	130	—	44%of 71	0%of 52	›
Gemini 3.7 Flash by set 36 of 130 lines inside tolerance. When it could leave lines blank, it scored 27.7% and declined 0. Performing arts centre electrical 33.3% 48 lines Supermarket plumbing 24.4% 82 lines
19=	Claude Sonnet 5.5 anthropic/claude-sonnet-5.5	Anthropic	26.9%	130	105	44%of 71	2%of 52	›
Claude Sonnet 5.5 by set 35 of 130 lines inside tolerance, 105 flagged as estimates. When it could leave lines blank, it scored 3.9% and declined 119. Performing arts centre electrical 27.1% 48 lines Supermarket plumbing 26.8% 82 lines
19=	Gemini 2.5 Pro google/gemini-2.5-pro	Google	26.9%	130	13	39%of 71	4%of 52	›
Gemini 2.5 Pro by set 35 of 130 lines inside tolerance, 13 flagged as estimates. When it could leave lines blank, it scored 26.2% and declined 17. Performing arts centre electrical 25.0% 48 lines Supermarket plumbing 28.1% 82 lines
19=	Gemini 3.1 Pro Preview google/gemini-3.1-pro-preview	Google	26.9%	130	130	43%of 71	3%of 52	›
Gemini 3.1 Pro Preview by set 35 of 130 lines inside tolerance, 130 flagged as estimates. When it could leave lines blank, it scored 10.8% and declined 107. Performing arts centre electrical 27.1% 48 lines Supermarket plumbing 26.8% 82 lines
19=	Gemini 4 Argon google/gemini-4-argon	Google	26.9%	130	1	41%of 71	2%of 52	›
Gemini 4 Argon by set 35 of 130 lines inside tolerance, 1 flagged as estimates. When it could leave lines blank, it scored 26.9% and declined 1. Performing arts centre electrical 29.2% 48 lines Supermarket plumbing 25.6% 82 lines
23	GPT 5.5 openai/gpt-5.5	OpenAI	26.5%	130	55	39%of 71	4%of 52	›
GPT 5.5 by set 35 of 130 lines inside tolerance, 55 flagged as estimates. When it could leave lines blank, it scored 26.2% and declined 46. Performing arts centre electrical 29.2% 48 lines Supermarket plumbing 25.0% 82 lines
24	Mistral Large 2512 mistralai/mistral-large-2512	Mistral	26.2%	130	19	39%of 71	4%of 52	›
Mistral Large 2512 by set 34 of 130 lines inside tolerance, 19 flagged as estimates. When it could leave lines blank, it scored 19.2% and declined 70. Performing arts centre electrical 27.1% 48 lines Supermarket plumbing 25.6% 82 lines
25	Gemini 2.5 Flash google/gemini-2.5-flash	Google	25.4%	130	46	41%of 71	2%of 52	›
Gemini 2.5 Flash by set 33 of 130 lines inside tolerance, 46 flagged as estimates. When it could leave lines blank, it scored 23.8% and declined 21. Performing arts centre electrical 22.9% 48 lines Supermarket plumbing 26.8% 82 lines
26=	Claude Sonnet 4.5 anthropic/claude-sonnet-4.5	Anthropic	24.6%	130	21	38%of 71	4%of 52	›
Claude Sonnet 4.5 by set 32 of 130 lines inside tolerance, 21 flagged as estimates. When it could leave lines blank, it scored 20.0% and declined 61. Performing arts centre electrical 25.0% 48 lines Supermarket plumbing 24.4% 82 lines
26=	GPT 5.6 Luna openai/gpt-5.6-luna	OpenAI	24.6%	130	71	37%of 71	2%of 52	›
GPT 5.6 Luna by set 32 of 130 lines inside tolerance, 71 flagged as estimates. When it could leave lines blank, it scored 24.6% and declined 78. Performing arts centre electrical 31.3% 48 lines Supermarket plumbing 20.7% 82 lines
28=	Claude Haiku 5.5 anthropic/claude-haiku-5.5	Anthropic	22.3%	130	123	37%of 71	2%of 52	›
Claude Haiku 5.5 by set 29 of 130 lines inside tolerance, 123 flagged as estimates. When it could leave lines blank, it scored 6.9% and declined 119. Performing arts centre electrical 16.7% 48 lines Supermarket plumbing 25.6% 82 lines
28=	GPT 4.1 openai/gpt-4.1	OpenAI	22.3%	130	22	46%of 54	0%of 6	›
GPT 4.1 by set 29 of 130 lines inside tolerance, 22 flagged as estimates. When it could leave lines blank, it scored 24.6% and declined 55. Performing arts centre electrical 37.5% 48 lines Supermarket plumbing 13.4% 82 lines
30	Gemini 2.5 Flash Lite google/gemini-2.5-flash-lite	Google	16.9%	130	53	40%of 47	6%of 47	›
Gemini 2.5 Flash Lite by set 22 of 130 lines inside tolerance, 53 flagged as estimates. When it could leave lines blank, it scored 30.0% and declined 11. Performing arts centre electrical 6.3% 48 lines Supermarket plumbing 23.2% 82 lines
31	GPT 4o openai/gpt-4o	OpenAI	3.1%	130	16	9%of 35	0%of 23	›
GPT 4o by set 4 of 130 lines inside tolerance, 16 flagged as estimates. When it could leave lines blank, it scored 14.6% and declined 57. Performing arts centre electrical 8.3% 48 lines Supermarket plumbing 0.0% 82 lines
32	Mistral Medium 3.1 mistralai/mistral-medium-3.1	Mistral	2.3%	130	127	0%of 71	0%of 52	›
Mistral Medium 3.1 by set 3 of 130 lines inside tolerance, 127 flagged as estimates. When it could leave lines blank, it scored 0.0% and declined 130. Performing arts centre electrical 6.3% 48 lines Supermarket plumbing 0.0% 82 lines
33	GPT 5 Nano openai/gpt-5-nano	OpenAI	0.0%	130	130	0%of 71	0%of 52	›
GPT 5 Nano by set 0 of 130 lines inside tolerance, 130 flagged as estimates. When it could leave lines blank, it scored 0.0% and declined 130. Performing arts centre electrical 0.0% 48 lines Supermarket plumbing 0.0% 82 lines

Reading the columns

Flagged

Lines the model priced but marked as an estimate. Every line on the raw board needs a number, so this is where a model shows its doubt. A flagged line is scored like any other. The count shows how much of the bill the model itself would want checked.

The harness board still lets a session leave a line blank. Its Declined column counts those lines, and a declined line scores zero.

Counts

Lines you get by counting tagged items on a plan, like fixtures, fittings and outlets. The number has to match the estimator's exactly. Most of the bill is counts.

Measured

Lines you get by measuring off the drawing, such as pipe and cable runs by diameter, each a sum over dozens of branches. Scored inside ±5%. We chose that tolerance. It isn't a measured figure. Every model on this page still falls over here.

Read each rate with the count under it. On the raw board every line carries a number, so every model's rates cover the same lines. On the harness board a rate only covers the lines a session priced, so skipping the hard ones pushes the percentage up. The count under each rate is how many lines it covers.

Correct covers every scored quantity line, including the few read off a schedule that neither column above breaks out. Every model on the raw board did every set over the same 130 lines, and a model that can't do one of them is left off. Every number comes from the scorer we run internally, off the run records we keep. We don't publish the drawings. Full method and raw results: ContractorOS.

Can AI do a quantity takeoff?

Inside a harness

Cowork · Claude Opus 5.5 by set

Gemini app · Gemini 4 Argon by set

Cowork · Claude Sonnet 5.5 by set

ChatGPT Work · GPT 6 Astra by set

ChatGPT Work · GPT 6.1 Sol by set

Cowork · Claude Fable 5.1 by set

Claude Code · Claude Opus 5 by set

Cursor · Claude Opus 5 by set

ChatGPT Work · GPT 5.6 Sol by set

Codex · GPT 5.6 Sol by set

ChatGPT Work · GPT 6 Sol by set

Cursor · Grok 4.6 by set

Muse · Muse Spark 1.3 by set

Cowork · Claude Haiku 5.5 by set

ChatGPT Work · GPT 5.6 Terra by set

ChatGPT Work · GPT 6 Luna by set

The models on their own

GPT 6 Astra by set

GPT 6.1 Sol by set

Claude Opus 5.5 by set

Claude Opus 4.7 by set

GPT 5.6 Sol by set

Claude Opus 5 by set

GPT 6 Luna by set

Muse Wonder by set

GPT 5.6 Terra by set

Gemini 3.6 Flash by set

Claude Fable 5 by set

Claude Opus 4.8 by set

Claude Haiku 4.5 by set

Claude Sonnet 5 by set

GPT 5.4 Mini by set

GPT 6 Sol by set

Gemini 3.1 Flash Lite by set

Gemini 3.7 Flash by set

Claude Sonnet 5.5 by set

Gemini 2.5 Pro by set

Gemini 3.1 Pro Preview by set

Gemini 4 Argon by set

GPT 5.5 by set

Mistral Large 2512 by set

Gemini 2.5 Flash by set

Claude Sonnet 4.5 by set

GPT 5.6 Luna by set

Claude Haiku 5.5 by set

GPT 4.1 by set

Gemini 2.5 Flash Lite by set

GPT 4o by set

Mistral Medium 3.1 by set

GPT 5 Nano by set

Reading the columns

Flagged

Counts

Measured