jeff.knowledge.knowledge-qav1.4.0
Multiple-choice general knowledge
Selects an answer to a multiple-choice question about science, common sense or general knowledge.
Example
Input
A patient taking warfarin is prescribed a course of antibiotics. Why should the patient's INR (a blood clotting test) be checked more often during the course?
Labels (the options)
- A
- Antibiotics neutralise warfarin in the stomach, so the dose stops working.
- B
- Some antibiotics can strengthen warfarin's effect and raise the risk of bleeding.
- C
- Antibiotics thicken the blood, so the patient is more likely to form clots.
- D
- Warfarin stops antibiotics from being absorbed, so the infection may not clear.
- Answer (JSON)
- value B, confidence 0.832
- Top options
- B 0.832
- C 0.078
- D 0.057
Some antibiotics strengthen warfarin's effect and raise the risk of bleeding, so the answer is option B.
import os import jeff with jeff.Jeff.remote("https://api.jeffhub.ai", token=os.environ["JEFF_API_KEY"], timeout_s=30) as client: decide = client.function("jeff.knowledge.knowledge-qa.question", labels={ "A": "Antibiotics neutralise warfarin in the stomach, so the dose stops working.", "B": "Some antibiotics can strengthen warfarin's effect and raise the risk of bleeding.", "C": "Antibiotics thicken the blood, so the patient is more likely to form clots.", "D": "Warfarin stops antibiotics from being absorbed, so the infection may not clear.", }) decision = decide("A patient taking warfarin is prescribed a course of antibiotics. Why should the patient's INR (a blood clotting test) be checked more often during the course?") print(decision.value, decision.confidence)
More examples
Why the sea is saltyanswer A · confidence 0.993
Input
Where does most of the salt in the oceans come from?
Labels (the options)
- A
- Rain and rivers wearing down rocks on land and carrying dissolved minerals to the sea.
- B
- Salt left behind by ships over thousands of years.
- C
- Fish producing salt as they breathe.
- D
- Salt falling from space in meteorites.
- Answer (JSON)
- value A, confidence 0.993
- Top options
- A 0.993
- B 0.003
- D 0.003
Rivers carry minerals worn from rocks on land into the sea, so the answer is option
A.import os import jeff with jeff.Jeff.remote("https://api.jeffhub.ai", token=os.environ["JEFF_API_KEY"], timeout_s=30) as client: decide = client.function("jeff.knowledge.knowledge-qa.question", labels={ "A": "Rain and rivers wearing down rocks on land and carrying dissolved minerals to the sea.", "B": "Salt left behind by ships over thousands of years.", "C": "Fish producing salt as they breathe.", "D": "Salt falling from space in meteorites.", }) decision = decide("Where does most of the salt in the oceans come from?") print(decision.value, decision.confidence)
Fill the blank in a sentenceanswer B · confidence 0.999
Input
The trophy would not fit in the suitcase because the _ was too small.
Labels (the options)
- A
- trophy
- B
- suitcase
- Answer (JSON)
- value B, confidence 0.999
- Top options
- B 0.999
- A 0.001
The trophy did not fit because the suitcase was too small, so the blank is option
B.import os import jeff with jeff.Jeff.remote("https://api.jeffhub.ai", token=os.environ["JEFF_API_KEY"], timeout_s=30) as client: decide = client.function("jeff.knowledge.knowledge-qa.blank", labels={ "A": "trophy", "B": "suitcase", }) decision = decide("The trophy would not fit in the suitcase because the _ was too small.") print(decision.value, decision.confidence)
Keeping a cut apple from browninganswer A · confidence 0.97
Input
Stop a cut apple from turning brown before lunch.
Labels (the options)
- A
- Brush the cut sides with lemon juice and keep them covered.
- B
- Leave the pieces on a sunny windowsill so they dry out.
- Answer (JSON)
- value A, confidence 0.97
- Top options
- A 0.97
- B 0.03
Lemon juice and a cover keep a cut apple from browning, so the answer is option
A.import os import jeff with jeff.Jeff.remote("https://api.jeffhub.ai", token=os.environ["JEFF_API_KEY"], timeout_s=30) as client: decide = client.function("jeff.knowledge.knowledge-qa.solution", labels={ "A": "Brush the cut sides with lemon juice and keep them covered.", "B": "Leave the pieces on a sunny windowsill so they dry out.", }) decision = decide("Stop a cut apple from turning brown before lunch.") print(decision.value, decision.confidence)
Why a friend brought soupanswer A · confidence 0.992
Context
- situation
- Ravi heard that his neighbour Mei had the flu, so he cooked a pot of soup and left it on her doorstep with a note.
Input
Why did Ravi leave the soup?
Labels (the options)
- A
- He wanted Mei to feel better while she was ill.
- B
- He wanted to get rid of food he did not like.
- C
- He was angry with Mei.
- Answer (JSON)
- value A, confidence 0.992
- Top options
- A 0.992
- B 0.005
- C 0.003
Ravi left soup for a neighbour with the flu to help her feel better, so the answer is option
A.import os import jeff with jeff.Jeff.remote("https://api.jeffhub.ai", token=os.environ["JEFF_API_KEY"], timeout_s=30) as client: decide = client.function("jeff.knowledge.knowledge-qa.social", context={ "situation": "Ravi heard that his neighbour Mei had the flu, so he cooked a pot of soup and left it on her doorstep with a note.", }, labels={ "A": "He wanted Mei to feel better while she was ill.", "B": "He wanted to get rid of food he did not like.", "C": "He was angry with Mei.", }) decision = decide("Why did Ravi leave the soup?") print(decision.value, decision.confidence)
Use when
- You have a multiple-choice question with a defined set of answer options.
Avoid when
- The answer depends on private documents or recent events; supply the relevant facts or use a retrieval adapter.
- You plan commercial use; the adapter is licensed CC BY-NC 4.0 (non-commercial use only), because its training data includes benchmarks licensed for non-commercial use only.
Context and input
| Field | Kind | Description |
|---|---|---|
question | input | A question or statement to complete, as plain text; the options are the possible answers (A, B, ...). |
Functions
| Key | Interface | Instructions | Labels |
|---|---|---|---|
| blank | classify | Which option correctly fills the blank (_) in the sentence? | A, B |
| Uses its own request shape, not the adapter's: input sentence (A sentence with one blank written as _, as plain text; the options are the two candidate fillers.). context: none. | |||
| question | classify | Choose the option that correctly answers the question or completes the statement. | from the caller |
| continuation | classify | Choose the ending that most plausibly continues the description of the activity. | A, B, C, D |
| Uses its own request shape, not the adapter's: input text (The start of a description of the activity, to be continued by one of the options.). context: activity. | |||
| passage | classify | Read the passage and choose the option that correctly answers the question or completes the statement about it. | A, B, C, D |
| Uses its own request shape, not the adapter's: input question (The question about the passage, or a statement to complete.). context: passage. | |||
| passage_text | classify | Read the passage and choose the option that correctly answers the question or completes the statement about it. | A, B, C, D |
| Uses its own request shape, not the adapter's: input passage_and_question (The passage followed by its question, as one plain text.). context: none. | |||
| social | classify | Read the situation and choose the option that best answers the question about the people in it. | A, B, C |
| Uses its own request shape, not the adapter's: input question (The question about the people in the situation.). context: situation. | |||
| solution | classify | Choose the solution that best achieves the goal. | A, B |
| Uses its own request shape, not the adapter's: input goal (A practical goal, as plain text; the options are two candidate solutions.). context: none. | |||
How it was measured
Test accuracy: the share of test rows whose top answer was right.
- 78.9% on held-out rows of public benchmarks: WinoGrande, ARC, OpenBookQA, QASC, CommonsenseQA, SciQ, HellaSwag, RACE, MMLU (auxiliary), Social IQa and PIQA (1,889 rows). public benchmark
When confident
From the calibration set: at or above the threshold confidence, how often the answer is right, and how often the adapter is that confident. Below it, escalate.
| Function | Right when confident | Confident on | Threshold | Calibration rows |
|---|---|---|---|---|
| blank | 95.8% | 100.0% | 0.52 | 165 |
| question | 90.0% | 87.9% | 0.50 | 876 |
| continuation | 90.0% | 27.8% | 0.73 | 180 |
| passage | 90.1% | 67.3% | 0.72 | 150 |
| passage_text | 90.0% | 66.7% | 0.51 | 195 |
| social | 90.0% | 51.9% | 0.82 | 135 |
| solution | 90.2% | 21.8% | 0.89 | 188 |
Caveats
- Part of the held-out rows were already in the base model's own pretraining data, which may make the score higher than on questions it has never seen.
Base: Jeff v1.4. Adapter trained: 25 October 2026.
Base
jeff-base v1.4
Call it
jeff.knowledge.knowledge-qa.blank, with placeholders for its input and context.
import os import jeff with jeff.Jeff.remote("https://api.jeffhub.ai", token=os.environ["JEFF_API_KEY"], timeout_s=30) as client: decide = client.function("jeff.knowledge.knowledge-qa.blank") decision = decide("<sentence>") print(decision.value, decision.confidence)
Download
Weights: mstrasser/jeff-adapter-knowledge-qa (v1.4)
Licence: CC-BY-NC-4.0