Assessment: Anthropic Batch fix

[vprashrex]

Issue

Closes #990

Summary

• Route batch create through beta endpoint with the structured-outputs beta so output_config (JSON schema) is no longer stripped
• Raise assessment batch max_tokens to 16384 so long rubric responses complete instead of truncating mid-JSON

Checklist

Before submitting a pull request, please ensure that you mark these task.

• Ran fastapi run --reload app/main.py or docker compose up in the repository root and test.
• If you've fixed a bug or added code that is tested and has test cases.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 23296d9f-474d-4e42-92b6-3c211a0c7086

📥 Commits

Reviewing files that changed from the base of the PR and between 5c999f9 and b5b8926.

📒 Files selected for processing (2)

• backend/app/core/batch/anthropic.py
• backend/app/tests/core/batch/test_anthropic.py

💤 Files with no reviewable changes (1)

• backend/app/core/batch/anthropic.py

---

📝 Walkthrough

Walkthrough

Updates Anthropic batch creation to use the beta batches endpoint with optional structured-outputs support, and raises the assessment batch max-tokens default to 16384 with matching tests.

Changes

Anthropic structured outputs beta and assessment batch token constant

Layer / File(s)	Summary
Anthropic batch beta routing backend/app/core/batch/anthropic.py	Adds STRUCTURED_OUTPUTS_BETA, inspects batch request params for output_config, builds a betas list when needed, and calls self.client.beta.messages.batches.create(...).
Assessment batch max-tokens update backend/app/models/llm/constants.py, backend/app/crud/assessment/batch.py, backend/app/tests/assessment/test_batch.py	Adds DEFAULT_ASSESSMENT_BATCH_MAX_TOKENS = 16384, switches the assessment batch fallback to that constant, and updates the routing test expectation.
Provider and routing tests backend/app/tests/core/batch/test_anthropic.py	Moves Anthropic batch tests to the beta client path, checks empty betas for normal requests, and adds coverage for structured outputs beta selection and error handling.

Estimated code review effort: 2 (Simple) | ~10 minutes

Sequence Diagram(s)

sequenceDiagram
  participant AnthropicBatchProvider
  participant AnthropicClient
  participant AnthropicBetaBatchesAPI
  AnthropicBatchProvider->>AnthropicBatchProvider: inspect requests for output_config
  AnthropicBatchProvider->>AnthropicClient: beta.messages.batches.create(requests, betas)
  AnthropicClient->>AnthropicBetaBatchesAPI: create batch request

Loading

Possibly related PRs

• ProjectTech4DevAI/kaapi-backend#931: Also changes AnthropicBatchProvider.create_batch in the Anthropic batch submission flow.

Suggested labels: ready-for-merge

Suggested reviewers: AkhileshNegi, Ayush8923, Prajna1999

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 66.67% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title is concise and accurately reflects the main change: fixing Anthropic batch handling for assessment requests.
Linked Issues check	✅ Passed	The PR implements both requested fixes: beta batch routing with structured outputs and the assessment batch max_tokens increase to 16384.
Out of Scope Changes check	✅ Passed	The changes stay focused on Anthropic assessment batch handling, with supporting constant and test updates only.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch chore/assessment-anthropic-batch-fix

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[github-actions]

OpenAPI changes   ⚪ No API surface changes

Note

This PR does not modify the API contract.

_{main ↔ 0bc5ffd3 · generated by oasdiff}

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@backend/app/core/batch/anthropic.py`:
- Around line 82-84: The `create_batch` logging in `anthropic.py` is emitting
the full serialized `requests` payload at info level, which can leak
prompts/messages and create oversized logs. Update the `logger.info` call in
`create_batch` to avoid dumping `json.dumps(requests, indent=2)`; either lower
it to debug or replace it with minimal metadata such as request count and
`custom_id` values. Keep the change localized to the batch request logging path
and preserve the rest of the batch creation flow.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 94da541c-68e8-4817-aceb-3d813576b01e

📥 Commits

Reviewing files that changed from the base of the PR and between d7a6e2f and 5c999f9.

📒 Files selected for processing (4)

• backend/app/core/batch/anthropic.py
• backend/app/crud/assessment/batch.py
• backend/app/models/llm/constants.py
• backend/app/tests/assessment/test_batch.py

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[github-actions]

🎉 This PR is included in version 1.3.0-main.2 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀