Skip to content
jeff.retrieval.passage-relevantv1.4.0

Passage relevance

Checks whether a passage answers a question about the specified subject and time period. Returns yes, partly or no.

Open in playground

Example

Context

source
warranty_terms: the warranty terms for each heat pump and boiler a home energy installer sells
question
How long is the warranty on an Aster 2 heat pump installed after March 2026?

Input

Aster 1 heat pumps carry a five-year parts and labour warranty from the date of installation, provided the unit is serviced every year by an approved engineer.

Answer (JSON)
value no, confidence 0.993
Top options
  1. no 0.993
  2. yes 0.006
  3. partly 0.001

The passage is about the Aster 1 while the question asks about the Aster 2, so the answer is no.

Open in playground

import os import jeff with jeff.Jeff.remote("https://api.jeffhub.ai", token=os.environ["JEFF_API_KEY"], timeout_s=30) as client:     decide = client.function("jeff.retrieval.passage-relevant.relevant", context={        "source": "warranty_terms: the warranty terms for each heat pump and boiler a home energy installer sells",        "question": "How long is the warranty on an Aster 2 heat pump installed after March 2026?",    })     decision = decide("Aster 1 heat pumps carry a five-year parts and labour warranty from the date of installation, provided the unit is serviced every year by an approved engineer.")     print(decision.value, decision.confidence)

More examples

  • Passage that gives the full answeranswer yes · confidence 0.713

    Context

    source
    warranty_terms: the warranty terms for each heat pump and boiler a home energy installer sells
    question
    How long is the warranty on an Aster 2 heat pump installed after March 2026, and what does it cover?

    Input

    For Aster 2 units installed on or after 1 March 2026 the warranty is seven years and covers parts and labour.

    Answer (JSON)
    value yes, confidence 0.713
    Top options
    1. yes 0.713
    2. partly 0.15
    3. no 0.136

    The passage gives both the length of the Aster 2 warranty and what it covers, so the answer is yes.

    Open in playground

    import os import jeff with jeff.Jeff.remote("https://api.jeffhub.ai", token=os.environ["JEFF_API_KEY"], timeout_s=30) as client:     decide = client.function("jeff.retrieval.passage-relevant.relevant", context={        "source": "warranty_terms: the warranty terms for each heat pump and boiler a home energy installer sells",        "question": "How long is the warranty on an Aster 2 heat pump installed after March 2026, and what does it cover?",    })     decision = decide("For Aster 2 units installed on or after 1 March 2026 the warranty is seven years and covers parts and labour.")     print(decision.value, decision.confidence)
  • Length of the warranty, but not what it coversanswer partly · confidence 0.823

    Context

    source
    warranty_terms: the warranty terms for each heat pump and boiler a home energy installer sells
    question
    How long is the warranty on an Aster 2 heat pump installed after March 2026, and what does it cover?

    Input

    For Aster 2 units installed on or after 1 March 2026 the warranty runs for seven years.

    Answer (JSON)
    value partly, confidence 0.823
    Top options
    1. partly 0.823
    2. yes 0.143
    3. no 0.035

    The passage gives the warranty's length but not what it covers, so the answer is partly.

    Open in playground

    import os import jeff with jeff.Jeff.remote("https://api.jeffhub.ai", token=os.environ["JEFF_API_KEY"], timeout_s=30) as client:     decide = client.function("jeff.retrieval.passage-relevant.relevant", context={        "source": "warranty_terms: the warranty terms for each heat pump and boiler a home energy installer sells",        "question": "How long is the warranty on an Aster 2 heat pump installed after March 2026, and what does it cover?",    })     decision = decide("For Aster 2 units installed on or after 1 March 2026 the warranty runs for seven years.")     print(decision.value, decision.confidence)
  • Right model, earlier installationsanswer no · confidence 0.956

    Context

    source
    warranty_terms: the warranty terms for each heat pump and boiler a home energy installer sells
    question
    How long is the warranty on an Aster 2 heat pump installed after March 2026?

    Input

    Aster 2 heat pumps installed before 1 March 2026 carry a five-year parts and labour warranty.

    Answer (JSON)
    value no, confidence 0.956
    Top options
    1. no 0.956
    2. yes 0.04
    3. partly 0.003

    The passage is about Aster 2 units installed before March 2026, the wrong period, so the answer is no.

    Open in playground

    import os import jeff with jeff.Jeff.remote("https://api.jeffhub.ai", token=os.environ["JEFF_API_KEY"], timeout_s=30) as client:     decide = client.function("jeff.retrieval.passage-relevant.relevant", context={        "source": "warranty_terms: the warranty terms for each heat pump and boiler a home energy installer sells",        "question": "How long is the warranty on an Aster 2 heat pump installed after March 2026?",    })     decision = decide("Aster 2 heat pumps installed before 1 March 2026 carry a five-year parts and labour warranty.")     print(decision.value, decision.confidence)
  • Boiler terms for a boiler questionanswer yes · confidence 0.997

    Context

    source
    warranty_terms: the warranty terms for each heat pump and boiler a home energy installer sells
    question
    How long is the standard warranty on the company's boilers?

    Input

    Our boilers come with a two-year warranty as standard, which can be extended to five years with an annual service plan.

    Answer (JSON)
    value yes, confidence 0.997
    Top options
    1. yes 0.997
    2. no 0.003
    3. partly 0

    The question asks for the boilers' standard warranty and the passage states it, so the answer is yes.

    Open in playground

    import os import jeff with jeff.Jeff.remote("https://api.jeffhub.ai", token=os.environ["JEFF_API_KEY"], timeout_s=30) as client:     decide = client.function("jeff.retrieval.passage-relevant.relevant", context={        "source": "warranty_terms: the warranty terms for each heat pump and boiler a home energy installer sells",        "question": "How long is the standard warranty on the company's boilers?",    })     decision = decide("Our boilers come with a two-year warranty as standard, which can be extended to five years with an annual service plan.")     print(decision.value, decision.confidence)

Use when

  • Search results may cover the right topic but the wrong company, product or year.

Avoid when

  • You need to compare several passages; use Passage selection and answer verification.

Context and input

FieldKindDescription
passage
inputThe retrieved passage's text.
source
contextWhere the passage comes from: the collection's name and a description of what it holds.
question
contextThe question to answer.

Functions

KeyInterfaceInstructionsLabels
relevantclassifyDoes the passage answer the question? Check who or what the passage is about (the named person, organisation, site, product or case), the period or date it covers, and every detail the question asks for. yes: the passage states every asked detail for the right subject and period (other content around it does not matter). partly: the passage is about the right subject and period and states at least one of the asked details but not all of them; for a question about several periods, it covers only some of them. no: the passage is about a different subject (even one with a similar name) or a different period, states none of the asked details (only related facts), or only says that the asked detail is not known or not decided yet.no, partly, yes

How it was measured

Test accuracy: the share of test rows whose top answer was right.

  • 83.9% on SQuAD 2.0, a public question-answering benchmark with human labels (2,000 rows). public benchmark
  • 99.5% on the in-house test (2,000 rows). in-house: same generator as the training data
  • 99.1% on the in-house test in domains never seen in training (2,000 rows). in-house: same generator as the training data

When confident

From the calibration set: at or above the threshold confidence, how often the answer is right, and how often the adapter is that confident. Below it, escalate.

FunctionRight when confidentConfident onThresholdCalibration rows
relevant99.5%100.0%0.522,000

Base: Jeff v1.4. Adapter trained: 25 October 2026.

Base

jeff-base v1.4

Call it

jeff.retrieval.passage-relevant.relevant, with placeholders for its input and context.

import os import jeff with jeff.Jeff.remote("https://api.jeffhub.ai", token=os.environ["JEFF_API_KEY"], timeout_s=30) as client:     decide = client.function("jeff.retrieval.passage-relevant.relevant", context={        "source": "<source>",        "question": "<question>",    })     decision = decide("<passage>")     print(decision.value, decision.confidence)

Download

Weights: mstrasser/jeff-adapter-passage-relevant (v1.4)

Licence: Apache-2.0