How Accurate Is Aloan?
A measurement of AI spreading on tax returns and financial statements, on 36 borrowers whose every printed number we know.
Of the 288 tax returns, 286 came back with no wrong figure on any of three reads. Of the 168 financial statements, 166 did.
One number isn't enough
A credit officer asking "how accurate?" is asking four questions
- Did it read the number? The figure on the page is the figure in the spread.
- Will I know when it's wrong? A wrong figure the software can't see is worse than one it flags.
- Is it the same every time? The same file read twice gives the same answer.
- Does it change the decision? If something was misread, would the credit outcome have been different?
A single accuracy percentage, usually measured on clean documents, answers only the first and easiest of these. We measured all four and report them separately.
We also split the first question in two. A wrong figure is a plausible number that isn't what the page printed, so the underwriter has no reason to doubt it. A blank is a printed number that didn't come back, and the underwriter can see the empty cell. The two carry very different risk, so we never add them together. The headline is wrong figures.
We count only what Aloan reads. Anything we deliberately don't extract isn't counted for or against us.
How we measured
Synthetic borrowers, so the answer to every printed number is known
A generator writes three years of books for an invented company and its owners, files their taxes on the IRS form edition for each year, and prints financial statements from the same books. It records every printed number as it goes, and each return passes the generator's own consistency checks before it is printed.
The test has 36 companies: 12 kinds of business, each generated three times with different names, figures, owners, tax software and statement styles. Because every company is new, Aloan can't have learned the answers in advance.
What each test borrower comes with
Click any page to see it larger
12 kinds of businesseach generated three times with new names and figures
- ManufacturerS corp
- DistributorC corp
- ContractorC corp
- Services firmPartnership
- Restaurant and its landlordS corp and partnership
- Two-property real estatePartnership
- Medical practiceS corp, owner's Schedule C
- Distributor with a holding company and trustPartnerships and trust
- NonprofitForm 990
- FarmS corp
- Retailer, owner has a rental houseS corp, owner's Schedule E
- HotelPartnership
3 years of tax returnseach on that year's IRS form
- Business return
- Supporting statements
- K-1s to each owner
- Owners' personal returns
Financial statementsin six different forms
- CPA reviewed
- CPA compiled
- QuickBooks export
- Company-prepared
- Spreadsheet
- Interim, year to date
4 kinds of papereach still readable by a person
- Clean PDF
- Photocopy
- Phone photo, light
- Phone photo, heavy
Why the test files are hard to read
Real borrower files are messy, so we made the test files messy too. They include the preparer's full tax return with all its attachments, financial statements that don't always match the return, and owners who are paid through other companies. We know the correct value of every number on every page. Click any page to see it larger.
Line items are often on a separate page
Page 1 of a Form 1120-S, and the statement at the back of the same return
Form 1120-S, page 1
Statement page, page 13
Each tax program prints statements differently
The same statement from two other returns in the test. The return above says “See Statement 1.”
Return that says “STMT. 1”
Return that says “See Stmt. 1”
Prior-year numbers sit right next to current ones
A two-year comparison from a tax return, and a QuickBooks profit and loss
Tax return, two-year comparison
QuickBooks profit and loss
Expenses and depreciation aren't always where you expect
Financial statements from two companies in the test
Income statement with one line for all expenses
Expense schedule, four pages later
Income statement with no depreciation line
Cash flow statement, the only place depreciation appears
One borrower, five tax returns
A borrower owned through a holding company and a family trust. Each arrow is a K‑1.
The test also includes:
- Many kinds of returns: rental real estate (Form 8825), farms (Schedule F), sole proprietors (Schedule C), nonprofits (Form 990) and trusts (Form 1041).
- Each year on that year's IRS form, so the layout can change from one year to the next.
- Losses, shown in parentheses.
- Pages that repeat numbers from the forms, such as Section 199A statements and depreciation reports. These must not be counted twice.
- Balance sheets that carry over: each year's opening balances must equal the prior year's closing balances.
- Interim year-to-date statements, some as PDFs and some as spreadsheets.
In our QA files from real banks, about 11% of scanned pages are phone photos. We use that share to weight a typical bank file.
Did it read the number?
On clean files, 11 of 54,744 figures were wrong
Aloan read 32,433 figures from tax returns and 22,311 from financial statements. 11 were wrong and 17 came back blank. 3 of the 11 wrong figures failed a check, and the other 8 raised nothing.
Wrong figures and blanks, per 10,000 figures read
Tax returns and financial statements together, by kind of paper
| As Aloan received it | Paper | Figures read | Wrong | 1 in | Blank |
|---|---|---|---|---|---|
| Clean | 54,744 | 11 | 5,000 | 17 | |
| Photocopy | 17,234 | 5 | 3,400 | 13 | |
| Phone photo, light | 17,234 | 26 | 660 | 13 | |
| Phone photo, heavy | 17,234 | 20 | 860 | 27 | |
| A typical bank file (89% clean, 11% phone photo) | 3,000 | ||||
Each thumbnail is the same patch of one borrower's 1120, page 1, lines 1c to 5, from the file Aloan was given. The damaged copies are made from the clean PDF, and each level was tuned so a person can still read every figure. Select one to see it full size.
By document, 99.81% of tax-return figures and 99.91% of financial-statement figures were exact on clean files. 99.1% of statement lines landed in the same spread category as the answer key. Most of the rest are judgment calls an analyst makes either way, such as where a nonprofit's restricted net assets sit within equity.
Will I know when it's wrong?
Of 279 mistakes planted in the borrower's paper, 15 got through
Aloan ties every figure it can to another figure on the paper: totals against their lines, a K-1 against the Schedule K it came from, an owner's return against their K-1, this year's opening balance against last year's closing one, and a statement's subtotals against its printed totals. 9,377 of the 9,430 tax-return figures Aloan keeps (99.4%) take part in at least one such tie.
To test the checks, we changed one printed number in one document, the way people slip: two digits transposed, a digit dropped or doubled, a neighbouring figure copied. Every planted error was drawn fresh for this run, so none of the checks was tuned on it.
What happened to each planted mistake
One changed figure per document
Show the numbers
| Planted | Flagged | Corrected | Noted | Not flagged | |
|---|---|---|---|---|---|
| Tax returns | 135 | 126 | 3 | 4 | 2 |
| Financial statements | 144 | 110 | 15 | 6 | 13 |
| All | 279 | 236 | 18 | 10 | 15 |
Aloan doesn't compare a borrower's financial statements with their tax return. Credit policy decides which source governs, and the analyst reconciles the two. When we lowered revenue on a statement against the return, it was flagged in 1 of 18 cases.
Is it the same every time?
99.95% of figures came back the same way in all three reads
Each file was read three times, independently. Of the 18,248 figures on the clean files, 18,238 came back the same way every time.
Every DSCR, leverage and global cash flow figure landed on the same side of its line in all three reads.
Does it change the decision?
No re-read and no damaged copy moved a borrower across its line
A misread matters when it changes the credit outcome. We rebuilt three measures from every read, with the same calculations the underwriting worksheets use:
- DSCR: cash flow available for debt service over the year's debt service, against a 1.20x line, from the return and, where the borrower sent them, from the financial statements.
- Pro forma leverage: funded debt plus the request, less what it refinances, over EBITDA, against 3.0x. Lessors are excluded.
- Global cash flow: the business and its owners together, against 1.25x.
Each read is compared with the majority of the three clean reads. A flip is a read that lands on the other side of the line.
| Measure | Borrower-years | Clean re-reads that flipped | Damaged reads that flipped |
|---|---|---|---|
| DSCR | 99 | 0 of 297 | 0 of 297 |
| Leverage | 90 | 0 of 270 | 0 of 270 |
| Global cash flow | 99 | 0 of 297 | 0 of 297 |
The generator draws its deals close to their lines on purpose: clean DSCR runs from 0.93x to 1.71x, and 6 of the 99 years fail. A small misread is more likely to flip one of these deals than a typical file. DSCR from the financial statements was within 0.02x of the answer key's in all 99 years.
Compared with what?
On clean files, Aloan's error rate is a seventh of double data entry's
People make mistakes too. No study has published a per-figure error rate for bank analysts spreading tax returns, so the closest evidence comes from other fields where people copy numbers from paper.
Error rate per figure, published human studies and Aloan
Log scale: each gridline is ten times the one to its left
Show the numbers and sources
| Task | Source | Error rate |
|---|---|---|
| Abstraction from records: finding and interpreting each value | Garza et al., Int J Med Inform 2025 | 6.57% |
| Clinic staff typing meter readings into a record | Mays & Mathias, JAMIA 2019 | 3.7% |
| Single entry from a structured form | Garza et al. 2025 | 0.29% |
| Specialist keying operators | Klemmer & Lockhead, J Appl Psychol 1962 | 0.14–0.37% |
| Double entry: two people keying and reconciling | Garza et al. 2025 | 0.14% |
| Aloan, phone photos | This report | 0.13% |
| Aloan, clean files | This report | 0.02% |
People and software make different kinds of mistakes, which is why we report decisions as well as figures.
How a bank can measure its own rate. Pick about 40 recent deals. Have two analysts spread each one independently, and a third check every disagreement against the page. That gives a per-figure error rate to within about a quarter of a percentage point. Run the same deals through Aloan and score it against the same answers.
What to ask any vendor
These are the questions we'd ask of anyone's AI spreading, ours included:
- Accuracy of what, and out of how many? Figures, fields or documents? On which forms?
- Wrong or blank? How often is a figure wrong without anyone being able to see it?
- On what paper? Clean PDFs, or the photocopies and phone photos your bank actually receives?
- Will I know when it's wrong? Of the errors in the borrower's own paper, how many does it flag?
- Does it change the decision? How often would a misread have moved a DSCR across your line?
To see Aloan on your bank's files, book a 30-minute demo. How we built the test: the blog post.