JET-2518: Log keyed-state key and state name on RocksDB migration failures#22
JET-2518: Log keyed-state key and state name on RocksDB migration failures#22rafael-sotelo wants to merge 3 commits into
Conversation
…lures During savepoint state migration, RocksDBKeyedStateBackend.migrateStateValues iterates raw RocksDB bytes and never decodes the key, so a value deserialization failure logged "key=null" with no state name. Best-effort decode the composite key (key group + key) from the iterator and thread it, plus the state name, into DeserializationContext around the per-entry migrateSerializedValue call (same try/finally pattern as RocksDBValueState.value()). PojoSerializer's failure log now reports [state=<name>, key=<key>], pinpointing the offending entry. Key decoding is best-effort and never masks the real migration failure; migrated bytes are unchanged. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
✨ PR Review
LGTM
Generated by LinearB AI and added by gitStream.
AI-generated content may contain inaccuracies. Please verify before using.
💡 Tip: You can customize your AI Review using Guidelines Learn how
|
🥷 Code experts: kellinwood kellinwood has most 👩💻 activity in the files. See details
Activity based on git-commit:
Knowledge based on git-blame:
Activity based on git-commit:
Knowledge based on git-blame: ✨ Comment |
|
[Automated] Latest version: 2026.6.23-4f28457 |
…d key Adds RocksDBStateBackendMigrationTest#testMigrationFailureSurfacesStateAndKey and a PoisonV1TestTypeSerializer that writes valid bytes but fails to read, forcing migrateStateValues to fail mid-migration. The test captures the DeserializationContext during the failing deserialize and asserts the keyed key and state name are populated, exercising the diagnostics added in this PR. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Up to standards ✅🟢 Issues
|
| Category | Results |
|---|---|
| Complexity | 1 medium |
🟢 Metrics 18 complexity
Metric Results Complexity 18
NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.
…izer Reflow class Javadoc to satisfy the googleJavaFormat (AOSP) 100-column limit enforced by spotless-check. No functional change. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Summary
When a savepoint is restored with an evolved POJO schema, Flink runs state migration:
RocksDBKeyedStateBackend.migrateStateValuesiterates the raw RocksDB bytes, deserializes each value with the old serializer, and re-serializes with the new one. The migration loop never decodes the key (it copiesiterator.key()verbatim) and never callssetCurrentKey, so when a value fails to deserialize,PojoSerializer's error log showskey=nullwith no state name — leaving no way to identify which keyed entry holds the corrupt/incompatible state.This change decodes the composite key during migration and threads it (plus the state name) into
DeserializationContext, so the existingPojoSerializerfailure log now prints[state=<name>, key=<key>].Changes
migrateStateValues, best-effort decode the composite key (key group + key) from each iterator entry using the existingCompositeKeySerializationUtils.migrateSerializedValuecall withDeserializationContext.set(key, stateName)/clear()— the same try/finally pattern used byRocksDBValueState.value().key=nulland never masks the real value-migration failure.Effect
Failure logs change from
Deserialization failure: ...toDeserialization failure [state=<stateName>, key=<realKey>]: ..., pinpointing the offending entry.Verification
flink-statebackend-rocksdbmodule compiles cleanly (./mvnw/ Maven 3.8.6).🤖 Generated with Claude Code
✨ PR Description
Purpose: Add diagnostic logging of keyed-state keys and state names during RocksDB migration failures to improve error attribution in state deserialization.
Main changes:
Generated by LinearB AI and added by gitStream.
AI-generated content may contain inaccuracies. Please verify before using.
💡 Tip: You can customize your AI Description using Guidelines Learn how