feat: add structured output support to OCI Generative AI provider

[fede-kamel]

Summary

• response_model (Pydantic BaseModel) support for structured JSON output
• _build_response_format: converts Pydantic schema → OCI JsonSchemaResponseFormat (generic) or CohereResponseJsonFormat (Cohere)
• _parse_structured_response: validates model output and returns typed Pydantic instance
• response_model threaded through call() → _call_impl / _stream_call_impl → _handle_tool_calls for full coverage
• Handles JSON wrapped in markdown fences via base class _validate_structured_output

Depends on #4962, #4961, #4959. Draft until all merge.
Tracking issue: #4944

Diff breakdown (vs tool calling PR)

Change	Lines
completion.py (structured output methods + threading)	+96/-2
test_oci_structured.py (5 unit tests)	+197
test_oci_integration_structured.py (1 parametrized test)	+38
Total	+333

Test plan

• 5 unit tests: generic structured output, Cohere structured output, fenced JSON, None model, schema creation
• Integration tests against meta.llama-3.3-70b-instruct and google.gemini-2.5-flash
• All 28 prior unit tests (PR 1-3) still pass

[fede-kamel]

OCI GenAI PR series (tracking issue: #4944)

1. feat: add OCI Generative AI provider — basic text completion #4959 — Basic text completion
2. feat: add streaming support to OCI Generative AI provider #4961 — Streaming
3. feat: add tool calling support to OCI Generative AI provider #4962 — Tool calling
4. feat: add structured output support to OCI Generative AI provider #4963 — Structured output ← this PR
5. feat: add multimodal support to OCI Generative AI provider #4964 — Multimodal
6. feat: add OCI Generative AI embeddings provider #4966 — Embeddings
7. feat: add true async support to OCI provider via aiohttp #4982 — True async (aiohttp)

Depends on #4962.

[coderabbitai]

Warning

Review limit reached

@fede-kamel, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 48 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 7fe255c6-6442-46cb-a6e5-e2382aa0ab49

📥 Commits

Reviewing files that changed from the base of the PR and between dc1d0bb and 3fc19ce.

📒 Files selected for processing (9)

• lib/crewai/src/crewai/llms/providers/oci/completion.py
• lib/crewai/tests/llms/oci/conftest.py
• lib/crewai/tests/llms/oci/test_oci_integration_basic.py
• lib/crewai/tests/llms/oci/test_oci_integration_streaming.py
• lib/crewai/tests/llms/oci/test_oci_integration_structured.py
• lib/crewai/tests/llms/oci/test_oci_integration_tools.py
• lib/crewai/tests/llms/oci/test_oci_streaming.py
• lib/crewai/tests/llms/oci/test_oci_structured.py
• lib/crewai/tests/llms/oci/test_oci_tools.py

📝 Walkthrough

Walkthrough

Adds OCI as a native CrewAI LLM provider, including optional dependency registration, model routing, authentication, generic and Cohere request handling, streaming, structured outputs, tool execution, and unit/live integration coverage.

Changes

OCI provider integration

Layer / File(s)	Summary
Provider dependency, routing, and authentication lib/crewai/pyproject.toml, lib/crewai/src/crewai/llm.py, lib/crewai/src/crewai/llms/providers/oci/__init__.py, lib/crewai/src/crewai/utilities/oci.py	Registers the OCI extra, routes OCI model identifiers to OCICompletion, exports the provider, and builds authenticated OCI client configuration.
OCI client setup and request construction lib/crewai/src/crewai/llms/providers/oci/completion.py	Initializes OCI clients, infers provider families, converts CrewAI messages and tools, and builds generic or Cohere request payloads.
Completion, streaming, structured output, and tools lib/crewai/src/crewai/llms/providers/oci/completion.py	Adds synchronous and asynchronous calls, streaming event handling, response metadata extraction, Pydantic response parsing, recursive tool execution, and serialized OCI client access.
Mocked provider behavior validation lib/crewai/tests/llms/oci/conftest.py, lib/crewai/tests/llms/oci/test_oci.py, lib/crewai/tests/llms/oci/test_oci_streaming.py, lib/crewai/tests/llms/oci/test_oci_structured.py, lib/crewai/tests/llms/oci/test_oci_tools.py	Tests routing, initialization, request construction, response normalization, streaming, structured output, tool calling, and SDK interactions with a mocked OCI module.
Live OCI integration coverage lib/crewai/tests/llms/oci/conftest.py, lib/crewai/tests/llms/oci/test_oci_integration_*.py	Adds environment-configured live tests for synchronous and asynchronous completion, streaming, structured output, and tool execution.

Sequence Diagram(s)

sequenceDiagram
  participant Caller
  participant LLM
  participant OCICompletion
  participant OCI
  participant Tool
  Caller->>LLM: provide OCI model
  LLM->>OCICompletion: route request
  OCICompletion->>OCI: send chat request
  OCI-->>OCICompletion: response or stream events
  OCICompletion->>Tool: execute tool call
  Tool-->>OCICompletion: tool result
  OCICompletion-->>Caller: final text, chunks, structured model, or tool calls

Loading

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 64.06% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title accurately summarizes the main change: adding structured output support to the OCI Generative AI provider.
Description check	✅ Passed	The description directly describes the structured output changes, tests, and integration coverage in this PR.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[github-actions]

This PR is stale because it has been open for 45 days with no activity.

[fede-kamel]

Rebased onto current main and un-drafted — mergeable again. Structured output verified live against OCI on-demand (us-chicago-1) with a Pydantic response model on xai.grok-4.3 (json_schema response format). Unit suite: 34 passed.

[coderabbitai]

Actionable comments posted: 2

🧹 Nitpick comments (2)

lib/crewai/tests/llms/oci/test_oci_streaming.py (1)

10-23: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Move import json to module level.

import json inside _make_fake_stream_event should be at the top of the file per PEP 8 conventions.

♻️ Proposed refactor

 """Unit tests for OCI provider streaming (mocked SDK)."""
 
 from __future__ import annotations
 
+import json
 from unittest.mock import MagicMock
 
 import pytest
 

 def _make_fake_stream_event(text: str = "", finish_reason: str | None = None, usage: dict | None = None) -> MagicMock:
     """Build a single SSE event with optional text, finish, and usage."""
     payload: dict = {}
     if text:
         payload["message"] = {"content": [{"text": text}]}
     if finish_reason:
         payload["finishReason"] = finish_reason
     if usage:
         payload["usage"] = usage
 
-    import json
     event = MagicMock()
     event.data = json.dumps(payload)
     return event

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@lib/crewai/tests/llms/oci/test_oci_streaming.py` around lines 10 - 23, Move
the local json import out of _make_fake_stream_event and add it with the
module-level imports at the top of the test file, leaving the helper to use the
module-level json reference.

lib/crewai/tests/llms/oci/test_oci_integration_tools.py (1)

24-39: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Reuse conftest's oci_live_config fixture instead of duplicating _skip_unless_live().

The _skip_unless_live() function duplicates the conftest's _skip_unless_live_config() logic (env var checks, config dict construction). Since the conftest already provides the oci_live_config fixture, the tools test can reuse it directly and only define its own oci_tool_model fixture for OCI_TEST_TOOL_MODELS. This eliminates duplication and would also fix the missing region issue if the conftest includes it.

♻️ Proposed refactor: reuse conftest fixtures

-def _skip_unless_live():
-    compartment = os.getenv("OCI_COMPARTMENT_ID")
-    if not compartment:
-        pytest.skip("OCI_COMPARTMENT_ID not set")
-    region = os.getenv("OCI_REGION")
-    endpoint = os.getenv("OCI_SERVICE_ENDPOINT")
-    if not region and not endpoint:
-        pytest.skip("Set OCI_REGION or OCI_SERVICE_ENDPOINT")
-    config: dict[str, str] = {"compartment_id": compartment}
-    if endpoint:
-        config["service_endpoint"] = endpoint
-    if os.getenv("OCI_AUTH_TYPE"):
-        config["auth_type"] = os.getenv("OCI_AUTH_TYPE", "API_KEY")
-    if os.getenv("OCI_AUTH_PROFILE"):
-        config["auth_profile"] = os.getenv("OCI_AUTH_PROFILE", "DEFAULT")
-    return config
-
-
 TOOL_SPEC = [

-@pytest.fixture()
-def oci_tool_config():
-    return _skip_unless_live()
-
-
 def test_oci_live_tool_call_returns_raw(oci_tool_model: str, oci_tool_config: dict):
-    llm = OCICompletion(model=oci_tool_model, **oci_tool_config)
+    llm = OCICompletion(model=oci_tool_model, **oci_live_config)

 def test_oci_live_tool_call_with_execution(oci_tool_model: str, oci_tool_config: dict):
-    llm = OCICompletion(model=oci_tool_model, **oci_tool_config)
+    llm = OCICompletion(model=oci_tool_model, **oci_live_config)

Then update both test signatures to accept oci_live_config instead of oci_tool_config, and remove the os import if no longer needed.

Also applies to: 69-72

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@lib/crewai/tests/llms/oci/test_oci_integration_tools.py` around lines 24 -
39, Remove the duplicated _skip_unless_live() helper and reuse the
conftest-provided oci_live_config fixture, retaining only the oci_tool_model
fixture for OCI_TEST_TOOL_MODELS. Update both affected tests to accept
oci_live_config instead of oci_tool_config, pass that configuration to the tool
setup, and remove the unused os import.

Source: Coding guidelines

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@lib/crewai/src/crewai/llms/providers/oci/completion.py`:
- Around line 1134-1214: Update iter_stream and astream to preserve the behavior
of _stream_call_impl: either implement handling for tools, callbacks,
available_functions, from_task, and from_agent, including relevant events and
tool recursion, or explicitly reject unsupported arguments instead of silently
discarding them. Ensure the OCI response event iteration in iter_stream executes
under the same _client_lock used by other OCI client calls, including safe lock
handling across the synchronous producer thread.
- Around line 1056-1081: Preserve each streamed tool call’s payload index
instead of using the per-event position from enumerate(stream_tool_calls).
Update _extract_tool_calls_from_stream_event to return the OCI/OpenAI-compatible
index, then use that index as the key for tool_calls_by_index and retain it when
emitting _emit_stream_chunk_event data.

---

Nitpick comments:
In `@lib/crewai/tests/llms/oci/test_oci_integration_tools.py`:
- Around line 24-39: Remove the duplicated _skip_unless_live() helper and reuse
the conftest-provided oci_live_config fixture, retaining only the oci_tool_model
fixture for OCI_TEST_TOOL_MODELS. Update both affected tests to accept
oci_live_config instead of oci_tool_config, pass that configuration to the tool
setup, and remove the unused os import.

In `@lib/crewai/tests/llms/oci/test_oci_streaming.py`:
- Around line 10-23: Move the local json import out of _make_fake_stream_event
and add it with the module-level imports at the top of the test file, leaving
the helper to use the module-level json reference.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: ae3d4c73-2257-48f4-a26e-fb245f99ebf9

📥 Commits

Reviewing files that changed from the base of the PR and between 85c467d and dc1d0bb.

📒 Files selected for processing (15)

• lib/crewai/pyproject.toml
• lib/crewai/src/crewai/llm.py
• lib/crewai/src/crewai/llms/providers/oci/__init__.py
• lib/crewai/src/crewai/llms/providers/oci/completion.py
• lib/crewai/src/crewai/utilities/oci.py
• lib/crewai/tests/llms/oci/__init__.py
• lib/crewai/tests/llms/oci/conftest.py
• lib/crewai/tests/llms/oci/test_oci.py
• lib/crewai/tests/llms/oci/test_oci_integration_basic.py
• lib/crewai/tests/llms/oci/test_oci_integration_streaming.py
• lib/crewai/tests/llms/oci/test_oci_integration_structured.py
• lib/crewai/tests/llms/oci/test_oci_integration_tools.py
• lib/crewai/tests/llms/oci/test_oci_streaming.py
• lib/crewai/tests/llms/oci/test_oci_structured.py
• lib/crewai/tests/llms/oci/test_oci_tools.py

[coderabbitai]

🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🌐 Web query:

Does the OCI Generative AI streaming chat response include an explicit index field for toolCalls deltas, or only positional array entries?

💡 Result:

The OCI Generative AI streaming chat response structure—particularly for OpenAI-compatible endpoints—does not natively include an explicit, persistent index field within the toolCalls delta object itself in the way some other providers might structured it [1]. Instead, OCI streaming chunks often bundle toolCalls data [1]. When integrating with streaming tools on OCI, developers frequently need to implement a client-side layer to manage indices [1]. For instance, integration layers (like those used in AI gateways) must often track and "anchor" the tool_calls[].index value across consecutive streaming chunks to ensure consistency with the OpenAI wire format [1]. Without this client-side transformation, raw OCI streaming responses may lack the explicit index required to correctly map and assemble multi-part tool call deltas [1]. While OCI Generative AI's static (non-streaming) model objects and Python SDK models define ToolCall and FunctionCall structures with explicit id and type fields [2][3], these definitions apply to the final message content, not the ephemeral streaming deltas [1][2]. Therefore, when handling streaming, the responsibility for generating or maintaining the positional index for toolCalls deltas generally falls to the client implementation to match the expected API specifications [1].

Citations:

• 1: fix(oracle): emit tool_calls in streaming responses Portkey-AI/gateway#1537
• 2: https://docs.oracle.com/en-us/iaas/tools/python/latest/api/generative_ai_inference/models/oci.generative_ai_inference.models.FunctionCall.html
• 3: https://docs.oracle.com/en-us/iaas/tools/python/latest/api/generative_ai_inference/models/oci.generative_ai_inference.models.ToolCall.html

---

🏁 Script executed:

python3 - <<'PY'
from pathlib import Path
p = Path('lib/crewai/src/crewai/llms/providers/oci/completion.py')
text = p.read_text()
for needle in ['def _extract_tool_calls_from_stream_event', 'toolCalls', 'tool_calls_by_index', 'index']:
    print(f'\n=== {needle} ===')
    for i, line in enumerate(text.splitlines(), 1):
        if needle in line:
            start = max(1, i-20)
            end = min(len(text.splitlines()), i+40)
            for j in range(start, end+1):
                print(f"{j}: {text.splitlines()[j-1]}")
            break
PY

Repository: crewAIInc/crewAI

Length of output: 11889

---

🏁 Script executed:

python3 - <<'PY'
from pathlib import Path
p = Path('lib/crewai/src/crewai/llms/providers/oci/completion.py')
lines = p.read_text().splitlines()

needles = [
    'def _extract_tool_calls_from_stream_event',
    'toolCalls',
    'tool_calls_by_index',
    'enumerate(stream_tool_calls)',
]
for needle in needles:
    print(f'\n=== {needle} ===')
    found = False
    for i, line in enumerate(lines, 1):
        if needle in line:
            found = True
            start = max(1, i - 20)
            end = min(len(lines), i + 40)
            for j in range(start, end + 1):
                print(f"{j}: {lines[j-1]}")
            break
    if not found:
        print('not found')
PY

Repository: crewAIInc/crewAI

Length of output: 11770

---

🌐 Web query:

OCI Generative AI streaming toolCalls payload index field delta array stable slot OpenAI compatible

💡 Result:

The Oracle Cloud Infrastructure (OCI) Generative AI service provides an OpenAI-compatible API for streaming responses [1][2][3]. When performing streaming tool calls, the API follows the standard OpenAI response format where tool calls are emitted as a sequence of deltas [4][5]. In the streaming response structure, the tool_calls field within the delta object utilizes an index field to identify which specific tool call is being updated [4][5]. This index is essential for handling multiple parallel tool calls, allowing clients to correctly assemble the complete function name and arguments by matching incoming chunks to the same tool call instance [4]. Key technical aspects include: 1. Payload Structure: During streaming, the choices[].delta field contains a tool_calls array [4]. Each element in this array includes an index (an integer) and a function object containing the name and arguments (partial) [4][5]. 2. Stability and Mapping: To ensure compatibility with standard OpenAI client libraries and frameworks (like LangChain or OpenAI SDK), middleware or client-side logic must track these indices to maintain stability across streamed chunks [4]. If a provider omits the id or if multiple calls are initiated, the index serves as the primary mechanism for associating subsequent streaming deltas with the correct, ongoing tool call [6][4]. 3. OpenAI Compatibility: While OCI Generative AI offers OpenAI-compatible endpoints, the underlying model responses may vary (e.g., some models might omit the id field) [7][4]. Consequently, integration layers (like those found in LangChain or gateway services) often implement logic to normalize these payloads, ensuring that tool_calls are emitted in a consistent format that includes the required index for proper assembly [6][4]. In summary, the index field within the tool_calls delta array is the standard, stable slot used to correlate streaming chunks with their respective tool call operations in the OCI Generative AI OpenAI-compatible interface [4][5].

Citations:

• 1: https://docs.oracle.com/en-us/iaas/Content/generative-ai/openai-compatible-api.htm
• 2: https://docs.oracle.com/en-us/iaas/Content/generative-ai/responses-api.htm
• 3: https://docs.oracle.com/en-us/iaas/Content/generative-ai/oci-openai.htm
• 4: fix(oracle): emit tool_calls in streaming responses Portkey-AI/gateway#1537
• 5: https://developers.openai.com/api/docs/guides/function-calling
• 6: https://github.com/oracle/langchain-oracle/blob/main/libs/oci/langchain_oci/chat_models/providers/generic.py
• 7: [Bug] Missing 'id' in toolCalls from Google Gemini models, causing errors with multiple tool calls oracle/oci-typescript-sdk#465

---

Preserve the streamed tool-call index here. enumerate(stream_tool_calls) keys state by per-event list position, but OCI/OpenAI-compatible tool-call deltas are indexed in the payload. If a stream emits only the updated delta or the array order shifts, arguments can be merged into the wrong tool_calls_by_index entry. Thread the payload index through _extract_tool_calls_from_stream_event and key on that instead.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@lib/crewai/src/crewai/llms/providers/oci/completion.py` around lines 1056 -
1081, Preserve each streamed tool call’s payload index instead of using the
per-event position from enumerate(stream_tool_calls). Update
_extract_tool_calls_from_stream_event to return the OCI/OpenAI-compatible index,
then use that index as the key for tool_calls_by_index and retain it when
emitting _emit_stream_chunk_event data.

[coderabbitai]

🎯 Functional Correctness | 🟠 Major | 🏗️ Heavy lift

🧩 Analysis chain

🏁 Script executed:

ast-grep outline lib/crewai/src/crewai/llms/providers/oci/completion.py --view expanded

Repository: crewAIInc/crewAI

Length of output: 2637

---

🏁 Script executed:

sed -n '515,610p;948,1133p;1134,1325p' lib/crewai/src/crewai/llms/providers/oci/completion.py

Repository: crewAIInc/crewAI

Length of output: 19552

---

🏁 Script executed:

rg -n "iter_stream\(" lib/crewai/src tests

Repository: crewAIInc/crewAI

Length of output: 388

---

iter_stream should match _stream_call_impl or explicitly reject unsupported args.

• tools, callbacks, available_functions, from_task, and from_agent are dropped here, so astream() bypasses tool handling and call/stream events without warning.
• response.data.events() also runs outside _client_lock; the OCI client is treated as non-reentrant elsewhere, so this path reopens the concurrency hazard.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@lib/crewai/src/crewai/llms/providers/oci/completion.py` around lines 1134 -
1214, Update iter_stream and astream to preserve the behavior of
_stream_call_impl: either implement handling for tools, callbacks,
available_functions, from_task, and from_agent, including relevant events and
tool recursion, or explicitly reject unsupported arguments instead of silently
discarding them. Ensure the OCI response event iteration in iter_stream executes
under the same _client_lock used by other OCI client calls, including safe lock
handling across the synchronous producer thread.