client: widen sync generate think type to accept string levels#697
Merged
Conversation
The `think` parameter accepts either a bool or the 'low'/'medium'/'high' string levels, and the request model (GenerateRequest.think) serializes all of them. PR ollama#553 widened this annotation across Client.chat, AsyncClient.generate, and AsyncClient.chat, but missed the sync Client.generate overloads and implementation, which stayed at Optional[bool]. As a result, ollama.generate(..., think='high') (bound to the sync Client.generate) was flagged as a false type error by static type checkers, even though it works at runtime and the equivalent chat() and async generate() calls type-check cleanly. Widen the three sync Client.generate occurrences to Optional[Union[bool, Literal['low', 'medium', 'high']]] to match the other methods, and add a signature-parity test guarding against future drift. Signed-off-by: Anas Khan <83116240+anxkhn@users.noreply.github.com>
vaderollama
approved these changes
Jul 14, 2026
vaderollama
left a comment
There was a problem hiding this comment.
LGTM — fixes the annotation parity gap where sync Client.generate was left with Optional[bool] while Client.chat and AsyncClient.generate already accepted Union[bool, Literal['low','medium','high']]. Good regression test checking annotation equality across all three methods.
@ParthSareen — ready to merge.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The
thinkparameter accepts either aboolor the'low'/'medium'/'high'string levels, and the request model serializes all of them
(
GenerateRequest.thinkinollama/_types.pyisOptional[Union[bool, Literal['low', 'medium', 'high']]]).PR #553 (commit
aa4b476, "add support for high/medium/low think values")widened this annotation across
Client.chat,AsyncClient.generate, andAsyncClient.chat, but missed the syncClient.generateoverloads andimplementation, which stayed at
Optional[bool].As a result, calling the documented string levels through the sync generate path,
e.g.
ollama.generate(model=..., prompt=..., think='high')(the module-levelgenerateis bound to the syncClient.generate), is flagged as a false typeerror by static type checkers, even though it works at runtime and the equivalent
chat()and asyncgenerate()calls type-check cleanly.What this changes
Client.generateoccurrences (the two@overloadstubsand the implementation) from
think: Optional[bool] = Nonetothink: Optional[Union[bool, Literal['low', 'medium', 'high']]] = None, sothey match
Client.chatandAsyncClient.generateexactly.Union,Literal,and
Optionalare already imported; no new imports.test_generate_think_annotation_matches_chat)asserting that
Client.generate,Client.chat, andAsyncClient.generateshare the same
thinkannotation, guarding against this kind of drift in thefuture.
This is a pure type-annotation change: no runtime behavior changes, and the blast
radius is limited to static type checking of
Client.generate/ the module-levelgenerate.Reproduction (before this PR)
A type checker reports, on
main:The same call through
chat()or asyncgenerate()type-checks cleanly today;after this PR the sync
generate()call does too.Testing
uv run pytest ollama tests -q-> all tests pass (the CI scope;examples/is excluded as it needs a live server).
uvx ruff check .anduvx hatch fmt --check-> clean.(
Optional[bool]) and passes after the widening.