Getting started | JeffHub docs

Jeff is a small decision model. You describe a situation and list the options in plain words; Jeff returns a probability for each option from one forward pass. An adapter is a small add-on (tens of megabytes; see Results) that tunes Jeff for one job, such as naming the intent behind a customer message or the type of a contract clause. One server loads the base model once and any number of adapters beside it.

Adapter serving arrives with the next Jeff release. Until then, the commands on this page need the feat/lora branch of firelex/jeff.

1. Install the server

You need uv and a clone of the Jeff repository.

git clone https://github.com/firelex/jeff && cd jeff
uv sync --no-default-groups --extra lora                # CPU
uv sync --no-default-groups --extra lora --extra cuda   # NVIDIA GPU
uv sync --no-default-groups --extra lora --extra mac    # Apple silicon (MLX)

2. Download the base model

uv run --no-default-groups hf download mstrasser/Jeff-Qwen3.5-0.8B --local-dir checkpoints/jeff-0.8b

3. Download an adapter

Each adapter lives in its own folder inside one adapters folder. The folder's name is the name you use in requests.

uv run --no-default-groups hf download mstrasser/Jeff-Qwen3.5-0.8B-support-intents --local-dir adapters/support-intents

A one-line jeff hub add support-intents that does this for you is planned.

4. Start the server

JEFF_CHECKPOINT=checkpoints/jeff-0.8b JEFF_ADAPTERS=adapters/ PORT=8765 \
  uv run --no-default-groups jeff-serve

On a Mac, add JEFF_BACKEND=mlx for much faster answers.

5. Ask a question

Name the adapter as the model. Any other name the server does not know is refused.

curl -s localhost:8765/v1/systemone -H 'content-type: application/json' -d '{
  "model": "support-intents",
  "state": {"service": "Customer support chat of an online shop",
            "message": "I sent the jacket back two weeks ago and still have not seen the money."},
  "questions": {
    "intent": {"type": "choice",
               "instructions": "What does the customer want? Choose the request that best matches what the customer is asking for in their message.",
               "criteria": {"track_refund": "Check the status of a refund they are expecting.",
                            "get_refund": "Get their money back for a purchase.",
                            "track_order": "Find out where their order is or its current status."}}
  }
}'

Every adapter's page has a full example in Python, TypeScript and curl. Next, read how requests are built.