Decide from the photo: does the receipt match the claim?

The step

When paperwork arrives as a photo, the usual pipeline is OCR, then rules or a prompt over the extracted text. It fails in predictable places.

• OCR first, decide second. The decision inherits every OCR mistake, and a blurred or partial photo produces confident garbage text.
• Extraction when you needed a judgement. Most steps are not “what is the total” but “does this support the claim”. Extracting ten fields to answer one yes/no is slow and brittle.
• A vision chat model. It describes the receipt in a paragraph. You still have to turn the paragraph into approve, reject or review, and it tells you nothing about how sure it was.

Omni reads the image as part of the situation and scores the options you give it, the same contract as text. Every question in the request sees every image.

What the request looks like

{
  "model": "saina-helm-2-omni-0.8b",
  "state": "Expense claim #2291: 'Team dinner, 4 people, 18 Sep,
EUR 186.40, Trattoria Nonna.' One photo attached.",
  "images": ["data:image/jpeg;base64,/9j/4AAQSkZJRg..."],
  "mode": "decision",
  "threshold": 0.75,
  "questions": {
    "doc_type": {
      "type": "single_choice",
      "question": "What is in the photo?",
      "options": { "receipt": "An itemized receipt or till slip", "card_slip": "A card terminal slip without items", "invoice": "An invoice", "other": "Not a payment document" }
    },
    "legible": { "type": "yes_no", "question": "Is the document legible enough to verify the amount and the merchant?" },
    "matches": { "type": "yes_no", "question": "Do the amount and merchant in the photo match the claim?" },
    "flags": {
      "type": "multi_choice",
      "question": "Which apply?",
      "options": { "alcohol": "Alcohol on the receipt", "partial": "Part of the document is cut off", "edited": "Signs of editing or a screenshot of a screenshot", "wrong_date": "Date does not match the claim" }
    }
  }
}

And what comes back:

{
  "answers": {
    "doc_type": {
      "type": "single_choice",
      "selection": "receipt",
      "reason": "accepted",
      "probabilities": {
        "receipt": 0.93,
        "card_slip": 0.05,
        "invoice": 0.01,
        "other": 0.01
      },
      "confidence": 0.91
    },
    "legible": {
      "type": "yes_no",
      "yes": 0.89,
      "no": 0.11,
      "confidence": 0.78,
      "selected": true,
      "reason": "accepted"
    },
    "matches": {
      "type": "yes_no",
      "yes": 0.81,
      "no": 0.19,
      "confidence": 0.62,
      "selected": true,
      "reason": "accepted"
    },
    "flags": {
      "type": "multi_choice",
      "selections": [
        "alcohol"
      ],
      "reason": "accepted",
      "memberships": {
        "alcohol": 0.84,
        "partial": 0.09,
        "edited": 0.04,
        "wrong_date": 0.07
      }
    }
  }
}

Illustrative answer: the fields are exact, the numbers show what a clean claim looks like. Try the receipt example in the playground, or upload one of your own.

Four decisions from one photo and one line of claim text. The policy flag (alcohol) is a tag, independent of the others, so it can be true alongside a matching receipt; your workflow decides what the policy says about that. Had the photo been unreadable, legible would have scored low and the claim would go straight back to the requester with a specific reason, before anyone spent time on it.

Putting it in place

Ask judgement questions, not extraction questions

“Does the amount match EUR 186.40?” is a question Omni can score. “What is the amount?” is extraction; use OCR for that if you need the value stored. Put the claim’s own numbers in the state so the model has something to compare against.

Several images, one request

Up to eight images ride along with the text. A two-page invoice, or a receipt plus the card slip, goes in one request and every question sees both. Images are counted as input tokens the way the model reads them (about one token per 32×32 pixels, after resizing to at most 1.6 megapixels); the API docs have the details.

Wire it in

Send the image as a base64 data URL in images. In n8n, use the HTTP Request node with the attachment from your mail trigger converted to a data URL; the Saina Helm community node is text-only today, so Omni requests go over HTTP. Its “Flag risky invoices for human review” template shows the text half of the same gate. Self-hosting: pip install 'saina[local,server,media]'; the weights are on Hugging Face under Apache-2.0, and the documents never leave your infrastructure. Files sent to the hosted API are not stored; see how data is handled.

What it won’t do

• It does not transcribe. For the line items as data, run OCR; use Omni to decide what to do with the document.
• It does not detect forgery. The edited tag catches obvious tells (a screenshot of a screenshot, mismatched crops); it is a flag for review, not proof.
• Ten-megabyte images, PNG, JPEG or WebP; URLs are not accepted, send the bytes.

Frequently asked questions

Read next