Read invoices with AI: a five-field accuracy test

Five correct fields beat a tidy preview

AI extracts invoice data, but populated fields can still be wrong. For a small business, a neat preview is not the deciding factor. What matters is whether the supplier, invoice number, date, invoice total and tax amount match the original. Start with those five fields and a list of known correct values.

Our recommendation: begin with 40 invoices covering different layouts. That is a manageable starting point, not statistical proof. If corrections remain time-consuming, improve document intake or the existing accounting software first. Another AI service is not worth buying simply because it can recognize text.

The example here is Microsoft’s invoice model in Azure Document Intelligence. It returns structured data from PDFs and images. This test assesses that output. Our German guide to preparing incoming invoices with AI covers the subsequent approval workflow.

Step 1: Structured invoices bypass text recognition

The business separates data imports from image processing. First check whether an invoice already contains structured data. That includes hybrid formats combining a PDF view with embedded data. Import and validate the data rather than turning it into an image and asking a model to read the same values again.

Germany’s Federal Ministry of Finance distinguishes structured e-invoices from ordinary PDFs. Where a hybrid invoice contains conflicting values, the structured part is authoritative. A PDF is therefore not automatically a job for OCR, or optical character recognition.

For the remaining PDFs, JPGs and PNGs, Microsoft supports invoice extraction. Keep digital PDFs, scans, photographs and multi-page documents in separate test groups. Use fictional samples or documents approved internally for testing; clarify contractual terms, access and data processing before sending confidential originals to a cloud service.

Step 2: Set the expected values before running extraction

The checklist starts with the original, not the AI output. Record the expected values before starting the test. Have a second person check anything unclear. If a value is absent from the original, record “not present”; the model should leave the gap rather than fill it with a guess.

These five fields appear in the official invoice schema. Use this fictional record as a template, including the location of each value on the document:

• Supplier / VendorName: Example Trading Ltd; page 1. Check against the supplier register.
• Invoice number / InvoiceId: 00127-A; page 1. Preserve leading zeros and letters.
• Invoice date / InvoiceDate: 5 October 2026; page 1. Comparison format: 2026-10-05.
• Invoice total / InvoiceTotal: EUR 119.00; final page. Record currency separately.
• Tax amount / TotalTax: EUR 19.00; final page. An amount, not a tax rate.

Step 3: Include the final page in the test

The test deliberately includes difficult documents. Alongside readable PDFs, add a tilted photograph, a faint scan, a handwritten addition and an invoice with its total on page three. Track which field comes back wrong or empty for each variant. Taking a better photograph can be more effective than repeatedly prompting a model about an unreadable original.

The free F0 tier of Document Intelligence processes only the first two pages of PDFs and TIFFs. A missing total on page three would indicate an incomplete test, not poor recognition. Record the document’s page count, the processed range and the model version.

Handwriting is not categorically excluded: Microsoft’s Read OCR recognizes handwritten as well as printed text and underpins the invoice models. Whether a handwritten addition is assigned to the correct invoice field is a separate question. A readable scribble is not yet a tax amount.

Step 4: Two accuracy measures reveal different failures

The evaluation counts correct fields and fully correct invoices separately. Mark each result as correct, wrong or missing. Compare dates and amounts in a consistent format without altering invoice numbers. Store the original value, model output and correction side by side.

A constructed calculation: 40 invoices with five present reference fields produce 200 comparisons. Four incorrect fields on four different invoices mean 196 ÷ 200 = 98 percent field accuracy. Yet only 36 ÷ 40 = 90 percent of invoices have all five fields correct. We checked this arithmetic with Python; it is not a measured extraction result.

A confidence score is the model’s estimate, not acceptance of the output. Record mistakes with high scores too. Watch InvoiceTotal versus AmountDue: the Microsoft schema distinguishes new invoice charges from the outstanding balance, including earlier unpaid amounts. For the subsequent arithmetic and document checks, see AI invoice validation.

Step 5: Duplicate detection needs its own comparison

Duplicate detection runs separately from extraction. A digital fingerprint of the file catches identical uploads even when the filename changes. Rescanning the same invoice changes the file, however. The workflow also needs to compare extracted invoice data with the existing register.

Our starting rule: compare a verified supplier ID and invoice number, showing the date, currency and amount as additional signals. Missing or uncertain keys go to review. Amounts alone are insufficient because two invoices can have the same price. Matches flag a possible duplicate; they neither delete a document nor confirm a payment.

Copy these four cases into the acceptance checklist: upload an identical file again; rename that same file; rescan the same invoice; enter a different invoice number with the same amount. The first three should trigger a duplicate warning. The fourth should not trigger one solely because of the amount. Count missed duplicates and false warnings separately.

Include correction time in the cost calculation

The page rate is only one cost component. Microsoft bills Document Intelligence by analyzed pages; the pricing table for prebuilt models uses a unit of 1,000 pages. Calculate analyzed pages ÷ 1,000 × agreed rate. At 1,200 pages, that component is 1.2 × the rate, counting pages analyzed again.

Add setup, integration, storage and human checking. A planning example: reviewing and correcting 40 invoices at an average of 1.5 minutes each takes 60 minutes. At an assumed labor cost of EUR 36 per hour, that adds EUR 36. Compare this time with doing the same work without the new software, rather than with zero.

Set a limit before buying: do not move into regular operation while unfamiliar document types pass silently, duplicates are missed or correction work takes as long as the existing process. Our AI workflow consulting covers the scope of an integration.

Three questions determine the next step

The first test ends with a concrete record. For each document ID, log its type, page count, five expected values, five model values, error reason, duplicate flag and correction time. This reveals whether extra work comes from a supplier layout, scan quality or the wrong field assignment.

• Are five fields enough for accounting? No. They define the scope of this quality test. Line items, tax treatment, account allocation, payment data and approval can require additional checks.
• Which AI reads invoices best? The manufacturer documentation reviewed does not establish an independent winner. Compare options using the same documents and expected values.
• What should we do today? Create the checklist, fill in the sample record and choose 40 representative test invoices. Keep difficult document types in manual processing until their errors are understood.