Skip to content

client: widen sync generate think type to accept string levels#697

Merged
ParthSareen merged 1 commit into
ollama:mainfrom
anxkhn:patch-3
Jul 15, 2026
Merged

client: widen sync generate think type to accept string levels#697
ParthSareen merged 1 commit into
ollama:mainfrom
anxkhn:patch-3

Conversation

@anxkhn

@anxkhn anxkhn commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

The think parameter accepts either a bool or the 'low'/'medium'/'high'
string levels, and the request model serializes all of them
(GenerateRequest.think in ollama/_types.py is
Optional[Union[bool, Literal['low', 'medium', 'high']]]).

PR #553 (commit aa4b476, "add support for high/medium/low think values")
widened this annotation across Client.chat, AsyncClient.generate, and
AsyncClient.chat, but missed the sync Client.generate overloads and
implementation, which stayed at Optional[bool].

As a result, calling the documented string levels through the sync generate path,
e.g. ollama.generate(model=..., prompt=..., think='high') (the module-level
generate is bound to the sync Client.generate), is flagged as a false type
error by static type checkers, even though it works at runtime and the equivalent
chat() and async generate() calls type-check cleanly.

What this changes

  • Widen the three sync Client.generate occurrences (the two @overload stubs
    and the implementation) from think: Optional[bool] = None to
    think: Optional[Union[bool, Literal['low', 'medium', 'high']]] = None, so
    they match Client.chat and AsyncClient.generate exactly. Union, Literal,
    and Optional are already imported; no new imports.
  • Add a signature-parity test (test_generate_think_annotation_matches_chat)
    asserting that Client.generate, Client.chat, and AsyncClient.generate
    share the same think annotation, guarding against this kind of drift in the
    future.

This is a pure type-annotation change: no runtime behavior changes, and the blast
radius is limited to static type checking of Client.generate / the module-level
generate.

Reproduction (before this PR)

import ollama

# Documented string think levels, flagged as a false type error on the sync path:
ollama.generate(model='gpt-oss', prompt='hi', think='high')

A type checker reports, on main:

error: Argument of type "Literal['high']" cannot be assigned to parameter "think"
of type "bool | None" in function "generate"
error: No overloads for "generate" match the provided arguments

The same call through chat() or async generate() type-checks cleanly today;
after this PR the sync generate() call does too.

Testing

  • uv run pytest ollama tests -q -> all tests pass (the CI scope; examples/
    is excluded as it needs a live server).
  • uvx ruff check . and uvx hatch fmt --check -> clean.
  • New test is a real regression guard: it fails on the current annotation
    (Optional[bool]) and passes after the widening.

The `think` parameter accepts either a bool or the 'low'/'medium'/'high'
string levels, and the request model (GenerateRequest.think) serializes all
of them. PR ollama#553 widened this annotation across Client.chat,
AsyncClient.generate, and AsyncClient.chat, but missed the sync
Client.generate overloads and implementation, which stayed at
Optional[bool].

As a result, ollama.generate(..., think='high') (bound to the sync
Client.generate) was flagged as a false type error by static type checkers,
even though it works at runtime and the equivalent chat() and async
generate() calls type-check cleanly.

Widen the three sync Client.generate occurrences to
Optional[Union[bool, Literal['low', 'medium', 'high']]] to match the other
methods, and add a signature-parity test guarding against future drift.

Signed-off-by: Anas Khan <83116240+anxkhn@users.noreply.github.com>

@vaderollama vaderollama left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM — fixes the annotation parity gap where sync Client.generate was left with Optional[bool] while Client.chat and AsyncClient.generate already accepted Union[bool, Literal['low','medium','high']]. Good regression test checking annotation equality across all three methods.

@ParthSareen — ready to merge.

@ParthSareen ParthSareen merged commit 3c5a26a into ollama:main Jul 15, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants