fix(batch): add optional logger injection for BatchProcessors (#7553) by Sujit-1509 · Pull Request #8272 · aws-powertools/powertools-lambda-python

Sujit-1509 · 2026-06-14T03:56:51Z

Issue number: closes #7553

Summary

Closes #7553.

Changes

This PR adds optional logger injection to the Batch Processing utility, allowing record-processing exceptions to be logged with their original traceback when desired.

Implementation

Added an optional \logger: Logger | logging.Logger | None\ parameter to the batch processor hierarchy.
Updated \ailure_handler()\ to emit a warning with \exc_info\ when:
- a logger is provided, and
- the exception contains a real traceback.
Preserved existing behavior when no logger is supplied.
Added regression tests covering:
- exception logging with an injected logger,
- backward compatibility when no logger is provided,
- FIFO circuit-breaker scenarios to ensure synthetic exceptions are not logged.

Why logger injection?

This follows existing Powertools dependency-injection patterns and avoids introducing additional boolean configuration flags or undocumented APIs.

User Experience

Before:

Record-processing exceptions were stored internally but not emitted through a customer logger.

After:

Customers can provide a logger to the processor and receive warning logs with full traceback information for failed records.
Existing behavior remains unchanged unless a logger is explicitly provided.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Disclaimer: We value your time and bandwidth. As such, any pull requests created on non-triaged issues might not be successful.

…wertools#7553)

leandrodamascena

Hey @Sujit-1509 thanks for working on this. I just left some reviews before we approve and merge.

leandrodamascena · 2026-06-19T05:53:08Z

+
+        # Log with full traceback when a customer-provided logger is present
+        # and the exception carries a real traceback (e.g. not a synthetic FIFO circuit-breaker)
+        batch_logger = self.logger
+        if batch_logger is not None and exception[2] is not None:
+            # ExceptionInfo allows None on every slot, but logging.warning's exc_info
+            # requires a fully populated tuple. We already excluded synthetic exceptions
+            # (no traceback) above, so the type and value are guaranteed to be set.
+            assert exception[0] is not None
+            assert exception[1] is not None
+            exc_info = cast("tuple[type[BaseException], BaseException, TracebackType]", exception)
+            batch_logger.warning(
+                "Record processing exception; skipping this record",
+                exc_info=exc_info,
+            )


Suggested change

# Log with full traceback when a customer-provided logger is present

# and the exception carries a real traceback (e.g. not a synthetic FIFO circuit-breaker)

batch_logger = self.logger

if batch_logger is not None and exception[2] is not None:

# ExceptionInfo allows None on every slot, but logging.warning's exc_info

# requires a fully populated tuple. We already excluded synthetic exceptions

# (no traceback) above, so the type and value are guaranteed to be set.

assert exception[0] is not None

assert exception[1] is not None

exc_info = cast("tuple[type[BaseException], BaseException, TracebackType]", exception)

batch_logger.warning(

"Record processing exception; skipping this record",

exc_info=exc_info,

)

if self.logger is not None and exception[2] is not None:

self.logger.warning(

"Record processing exception; skipping this record",

exc_info=exception,

)

Can you try this pls? I think we can simplify this block of code.

leandrodamascena · 2026-06-19T05:54:27Z

        event_type: EventType,
        model: BatchTypeModels | None = None,
        raise_on_entire_batch_failure: bool = True,
+        logger: logging.Logger | Logger | None = None,


Suggested change

logger: logging.Logger | Logger | None = None,

logger: logging.Logger | None = None,

Powertools Logger inherits from logging.Logger, so, this is not necessary here.

leandrodamascena · 2026-06-19T05:54:35Z

        raise_on_entire_batch_failure: bool
            Raise an exception when the entire batch has failed processing.
            When set to False, partial failures are reported in the response
+        logger: logging.Logger | Logger | None


Suggested change

logger: logging.Logger | Logger | None

logger: logging.Logger | None

leandrodamascena · 2026-06-19T05:54:53Z

+        self,
+        model: BatchSqsTypeModel | None = None,
+        skip_group_on_error: bool = False,
+        logger: logging.Logger | Logger | None = None,


Suggested change

logger: logging.Logger | Logger | None = None,

logger: logging.Logger | None = None,

Powertools Logger inherits from logging.Logger, so, this is not necessary here.

leandrodamascena · 2026-06-19T05:55:01Z

        skip_group_on_error: bool
            Determines whether to exclusively skip messages from the MessageGroupID that encountered processing failures
            Default is False.
+        logger: logging.Logger | Logger | None


Suggested change

logger: logging.Logger | Logger | None

logger: logging.Logger | None

leandrodamascena · 2026-06-19T05:56:12Z

+    import logging
+
+    from aws_lambda_powertools.utilities.batch import BatchProcessor, EventType, process_partial_response


Please move imports to the top of the file. No need for lazy imports in tests.

leandrodamascena · 2026-06-19T05:56:20Z

+    import logging
+
+    from aws_lambda_powertools.utilities.batch import SqsFifoPartialProcessor, process_partial_response


Please move imports to the top of the file. No need for lazy imports in tests.

leandrodamascena · 2026-06-19T05:56:30Z

+    import logging
+
+    from aws_lambda_powertools.utilities.batch import BatchProcessor, EventType, process_partial_response


Please move imports to the top of the file. No need for lazy imports in tests.

sonarqubecloud · 2026-06-19T07:40:56Z

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

Sujit-1509 · 2026-06-19T07:45:08Z

@leandrodamascena I pushed the review updates for this PR.

Addressed in 97247759:

simplified the warning block in failure_handler while keeping the CI mypy check green for the nullable ExceptionInfo case
removed the redundant Powertools Logger type annotations and kept logging.Logger | None
moved the test imports to the top of the file

I also reran local verification for the updated branch:

ruff format aws_lambda_powertools tests examples --check
ruff check aws_lambda_powertools tests examples
mypy --pretty --platform linux aws_lambda_powertools examples
focused batch regression tests for the logger/FIFO cases

Thanks again for the review.

AI Bot added 2 commits May 26, 2026 00:39

chore: fix minor typos and grammar in docs and comments

a87e0cb

fix(batch): add optional logger injection for BatchProcessors (aws-po…

3090e4d

…wertools#7553)

Sujit-1509 requested a review from a team as a code owner June 14, 2026 03:56

Sujit-1509 requested a review from svozza June 14, 2026 03:56

powertools-for-aws-oss-automation Bot assigned Sujit-1509 Jun 14, 2026

boring-cyborg Bot added documentation Improvements or additions to documentation tests labels Jun 14, 2026

powertools-for-aws-oss-automation Bot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Jun 14, 2026

Merge branch 'develop' into fix/batch-processor-logger-injection

01af30a

powertools-for-aws-oss-automation Bot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Jun 14, 2026

Merge branch 'develop' into fix/batch-processor-logger-injection

54251b8

powertools-for-aws-oss-automation Bot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Jun 14, 2026

Merge branch 'develop' into fix/batch-processor-logger-injection

368cbbc

powertools-for-aws-oss-automation Bot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Jun 18, 2026

fix: satisfy batch logger mypy checks

acd3734

powertools-for-aws-oss-automation Bot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Jun 18, 2026

leandrodamascena requested changes Jun 19, 2026

View reviewed changes

chore: address batch logger review feedback

40ec3f0

powertools-for-aws-oss-automation Bot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Jun 19, 2026

chore: simplify batch logger warning guard

9724775

powertools-for-aws-oss-automation Bot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Jun 19, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix(batch): add optional logger injection for BatchProcessors (#7553)#8272

fix(batch): add optional logger injection for BatchProcessors (#7553)#8272
Sujit-1509 wants to merge 8 commits into
aws-powertools:developfrom
Sujit-1509:fix/batch-processor-logger-injection

Sujit-1509 commented Jun 14, 2026 •

edited

Loading

Uh oh!

leandrodamascena left a comment

Uh oh!

leandrodamascena Jun 19, 2026

Uh oh!

leandrodamascena Jun 19, 2026

Uh oh!

leandrodamascena Jun 19, 2026

Uh oh!

leandrodamascena Jun 19, 2026

Uh oh!

leandrodamascena Jun 19, 2026

Uh oh!

leandrodamascena Jun 19, 2026

Uh oh!

leandrodamascena Jun 19, 2026

Uh oh!

leandrodamascena Jun 19, 2026

Uh oh!

sonarqubecloud Bot commented Jun 19, 2026

Uh oh!

Sujit-1509 commented Jun 19, 2026 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

	logger: logging.Logger \| Logger \| None = None,
	logger: logging.Logger \| None = None,

	logger: logging.Logger \| Logger \| None
	logger: logging.Logger \| None

		import logging

		from aws_lambda_powertools.utilities.batch import BatchProcessor, EventType, process_partial_response

Conversation

Sujit-1509 commented Jun 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Changes

Implementation

Why logger injection?

User Experience

Uh oh!

leandrodamascena left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sonarqubecloud Bot commented Jun 19, 2026

Quality Gate passed

Uh oh!

Sujit-1509 commented Jun 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Sujit-1509 commented Jun 14, 2026 •

edited

Loading

Sujit-1509 commented Jun 19, 2026 •

edited

Loading