Skip to content
jeff.retrieval.extract-checkv1.4.0

Extracted value validation

Checks an extracted field and value against the source document. Identifies correct values, incorrect values, wrong-field errors, unsupported values and ambiguity.

Open in playground

Example

Context

document
INVOICE 7781 Issued: 14 October 2026 Due: 13 November 2026 Fenwick Home Energy, 12 Quay Street Aster 2 heat pump, supplied and installed: 8,400.00 VAT (5%): 420.00 Total due: 8,820.00

Input

field: total value: 420.00

Answer (JSON)
value wrong_field, confidence 0.991
Top options
  1. wrong_field 0.991
  2. ambiguous 0.003
  3. correct 0.003

The extracted figure is in the document but is the VAT, not the total, so the answer is wrong_field.

Open in playground

import os import jeff with jeff.Jeff.remote("https://api.jeffhub.ai", token=os.environ["JEFF_API_KEY"], timeout_s=30) as client:     decide = client.function("jeff.retrieval.extract-check.check", context={        "document": "INVOICE 7781\nIssued: 14 October 2026    Due: 13 November 2026\nFenwick Home Energy, 12 Quay Street\nAster 2 heat pump, supplied and installed: 8,400.00\nVAT (5%): 420.00\nTotal due: 8,820.00",    })     decision = decide({        "field": "total",        "value": "420.00",    })     print(decision.value, decision.confidence)

More examples

  • Total due in another formatanswer correct · confidence 0.98

    Context

    document
    INVOICE 7781 Issued: 14 October 2026 Due: 13 November 2026 Fenwick Home Energy, 12 Quay Street Aster 2 heat pump, supplied and installed: 8,400.00 VAT (5%): 420.00 Total due: 8,820.00

    Input

    field: total value: 8820

    Answer (JSON)
    value correct, confidence 0.98
    Top options
    1. correct 0.98
    2. ambiguous 0.008
    3. wrong_field 0.006

    The invoice's total due is 8,820.00, the value extracted written another way, so it is correct.

    Open in playground

    import os import jeff with jeff.Jeff.remote("https://api.jeffhub.ai", token=os.environ["JEFF_API_KEY"], timeout_s=30) as client:     decide = client.function("jeff.retrieval.extract-check.check", context={        "document": "INVOICE 7781\nIssued: 14 October 2026    Due: 13 November 2026\nFenwick Home Energy, 12 Quay Street\nAster 2 heat pump, supplied and installed: 8,400.00\nVAT (5%): 420.00\nTotal due: 8,820.00",    })     decision = decide({        "field": "total",        "value": "8820",    })     print(decision.value, decision.confidence)
  • Due date read wronglyanswer wrong_value · confidence 0.99

    Context

    document
    INVOICE 7781 Issued: 14 October 2026 Due: 13 November 2026 Fenwick Home Energy, 12 Quay Street Aster 2 heat pump, supplied and installed: 8,400.00 VAT (5%): 420.00 Total due: 8,820.00

    Input

    field: due_date value: 13 December 2026

    Answer (JSON)
    value wrong_value, confidence 0.99
    Top options
    1. wrong_value 0.99
    2. ambiguous 0.005
    3. not_in_document 0.002

    The invoice is due on 13 November and the extracted date is 13 December, so the answer is wrong_value.

    Open in playground

    import os import jeff with jeff.Jeff.remote("https://api.jeffhub.ai", token=os.environ["JEFF_API_KEY"], timeout_s=30) as client:     decide = client.function("jeff.retrieval.extract-check.check", context={        "document": "INVOICE 7781\nIssued: 14 October 2026    Due: 13 November 2026\nFenwick Home Energy, 12 Quay Street\nAster 2 heat pump, supplied and installed: 8,400.00\nVAT (5%): 420.00\nTotal due: 8,820.00",    })     decision = decide({        "field": "due_date",        "value": "13 December 2026",    })     print(decision.value, decision.confidence)
  • Purchase order number the invoice never givesanswer not_in_document · confidence 0.999

    Context

    document
    INVOICE 7781 Issued: 14 October 2026 Due: 13 November 2026 Fenwick Home Energy, 12 Quay Street Aster 2 heat pump, supplied and installed: 8,400.00 VAT (5%): 420.00 Total due: 8,820.00

    Input

    field: purchase_order_number value: PO-3108

    Answer (JSON)
    value not_in_document, confidence 0.999
    Top options
    1. not_in_document 0.999
    2. wrong_value 0
    3. ambiguous 0

    The invoice gives no purchase order number at all, so the answer is not_in_document.

    Open in playground

    import os import jeff with jeff.Jeff.remote("https://api.jeffhub.ai", token=os.environ["JEFF_API_KEY"], timeout_s=30) as client:     decide = client.function("jeff.retrieval.extract-check.check", context={        "document": "INVOICE 7781\nIssued: 14 October 2026    Due: 13 November 2026\nFenwick Home Energy, 12 Quay Street\nAster 2 heat pump, supplied and installed: 8,400.00\nVAT (5%): 420.00\nTotal due: 8,820.00",    })     decision = decide({        "field": "purchase_order_number",        "value": "PO-3108",    })     print(decision.value, decision.confidence)
  • Two different phone numbersanswer ambiguous · confidence 0.924

    Context

    document
    Fenwick Home Energy - service visit confirmation Customer: Alan Brierley, 7 Moor Lane Visit: Thursday 12 November 2026, morning Phone: 0161 496 0555 Engineer's notes: customer asks us to call on 0161 496 0777 on the day, not the number above.

    Input

    field: customer_phone value: 0161 496 0555

    Answer (JSON)
    value ambiguous, confidence 0.924
    Top options
    1. ambiguous 0.924
    2. wrong_field 0.067
    3. correct 0.004

    The visit confirmation gives two different phone numbers for the customer and the value extracted is one of them, so it is ambiguous.

    Open in playground

    import os import jeff with jeff.Jeff.remote("https://api.jeffhub.ai", token=os.environ["JEFF_API_KEY"], timeout_s=30) as client:     decide = client.function("jeff.retrieval.extract-check.check", context={        "document": "Fenwick Home Energy - service visit confirmation\nCustomer: Alan Brierley, 7 Moor Lane\nVisit: Thursday 12 November 2026, morning\nPhone: 0161 496 0555\nEngineer's notes: customer asks us to call on 0161 496 0777 on the day, not the number above.",    })     decision = decide({        "field": "customer_phone",        "value": "0161 496 0555",    })     print(decision.value, decision.confidence)

Use when

  • You need to validate fields extracted by another tool or model before using them.

Avoid when

  • You need to extract the value; this adapter only validates a value already supplied.

Context and input

FieldKindDescription
extracted
inputThe extracted value to check: an object with `field` (the field's name) and `value`.
document
contextThe document's text.

Functions

KeyInterfaceInstructionsLabels
checkclassifyIs the extracted value supported by the document? Find where the document states the extracted field and compare the values. A value written in another format still matches (dates written differently, amounts with or without currency or separators, a name in another order or without a company suffix). correct: the document states this value for this field. wrong_value: the document states this field with a different value, and the extracted value is not what it gives for another field. wrong_field: the extracted value appears in the document but for a different field (for example the tax amount given as the total, or the issue date given as the due date). not_in_document: the document does not state this field at all; do not work it out from other values. ambiguous: the document states this field more than once with different values, and the extracted value matches one of them.ambiguous, correct, not_in_document, wrong_field, wrong_value

How it was measured

Test accuracy: the share of test rows whose top answer was right.

  • 99.6% on the in-house test (2,000 rows). in-house: same generator as the training data
  • 98.8% on a second in-house test of unseen cases (2,000 rows). in-house: same generator as the training data

When confident

From the calibration set: at or above the threshold confidence, how often the answer is right, and how often the adapter is that confident. Below it, escalate.

FunctionRight when confidentConfident onThresholdCalibration rows
check99.1%100.0%0.482,000

Base: Jeff v1.4. Adapter trained: 25 October 2026.

Base

jeff-base v1.4

Call it

jeff.retrieval.extract-check.check, with placeholders for its input and context.

import os import jeff with jeff.Jeff.remote("https://api.jeffhub.ai", token=os.environ["JEFF_API_KEY"], timeout_s=30) as client:     decide = client.function("jeff.retrieval.extract-check.check", context={        "document": "<document>",    })     decision = decide("<extracted>")     print(decision.value, decision.confidence)

Download

Weights: mstrasser/jeff-adapter-extract-check (v1.4)

Licence: Apache-2.0