Edison SinaniAI implementation architectBook a call

Benchmark

The HS6 classification benchmark

Accuracy for automated Harmonized System classification, measured on a fixed 1,480 case set, published one page per run with the method attached.

One page per run, every run against the same 1,480 cases. Every figure is computed from the run output. The weak slices are in the tables and they stay there.

What is measured

Every item that crosses a border carries a Harmonized System code. The first six digits are internationally standardised. The national tariff line that sets the duty rate, the trade agreement treatment and any permit requirement is built on top of those six. Get the six wrong and the rest of it is wrong too. That is a misdeclaration, and the exposure sits with the importer.

The nomenclature is legal text. It turns on distinctions that do not appear in the way a product is described on a commercial document: knitted against woven, whether a preparation is put up for retail sale, whether a machine's principal function is the one its name implies. The task is a translation between the language of trade and the language of the tariff schedule. Most of the difficulty sits in that gap, and the numbers below show it.

Why it is published

I went looking for a published accuracy figure in this space with a dataset, a scoring rule, a sample size and a date attached to it. I did not find one. If you are carrying the compliance risk, a bare percentage gives you nothing to work with.

So the set, the scoring rule, the confidence bands and the failures all get published, one page per run, at a URL that does not change. A run that goes backwards gets published with the regression visible. The series is the point, and it starts at one.

Results

One row per run. Accuracy on all counts every declined case as wrong. That is the number to read if you are carrying the risk.

RunDateSetnCoverageAcc. answeredAcc. allStatus
v1.0 · gpt-5.4-nanoAug 14, 2026HS6-2022 v1.01,48098.0%97.7%95.7%Published
v1.0 · gpt-5-miniAug 13, 2026HS6-2022 v1.01,48098.2%98.5%96.8%Published

Method, in short

The set

1,480 cases built from WCO HS 2022 nomenclature. Each case pairs an input description with the 6-digit subheading it resolves to. Half the set is worded close to the official text and half is reworded, because the reworded half is what a commercial document looks like.

What counts as correct

An exact match on all six digits. The right chapter with the wrong subheading scores zero. A two-digit near miss scores zero. A near miss is still a misdeclaration.

Confidence gating

The pipeline returns a confidence per field and declines below the gate. That produces two numbers: accuracy on the cases it answered, and accuracy across all cases with declines counted wrong. The gap between them is the cost of the gate.

Set versioning

The set carries a version. Runs on different set versions are not comparable, so the version sits in the results table.