Skip to content

fix: implement retry logic for ComponentInformation to handle transient errors#118

Merged
alexlovelltroy merged 2 commits into
mainfrom
bugfix/smd-connection-timeouts
Jul 9, 2026
Merged

fix: implement retry logic for ComponentInformation to handle transient errors#118
alexlovelltroy merged 2 commits into
mainfrom
bugfix/smd-connection-timeouts

Conversation

@alexlovelltroy

Copy link
Copy Markdown
Member

Description

This pull request introduces improved resiliency and reliability to SMD client interactions, particularly during high-load scenarios such as boot storms. The main enhancements include adding retry logic with exponential backoff for component information lookups, updating the SMD client interface, and tuning HTTP client configuration for better performance.

Resiliency Improvements:

  • Added a new ComponentInformationWithRetry method to SMDClient that wraps ComponentInformation with exponential backoff retry logic for transient network errors and timeouts, improving reliability during SMD overloads.
  • Implemented the isRetryableError helper function to determine which errors should trigger a retry, covering timeouts, network errors, and common HTTP error codes like 429 and 503.
  • Updated the MetaDataHandler in metadata_handlers.go to use ComponentInformationWithRetry with up to 3 attempts instead of a single call to ComponentInformation.

Interface and Testing Updates:

  • Extended the SMDClientInterface to include the new ComponentInformationWithRetry method, ensuring all clients (including fakes) conform to the updated contract.
  • Provided a stub implementation of ComponentInformationWithRetry in FakeSMDClient for testing.

HTTP Client Configuration:

  • Increased the default HTTP client timeout from 2 seconds to 10 seconds to better accommodate slow SMD responses.
  • Changed DisableKeepAlives from true to false to allow HTTP keep-alive connections, improving efficiency under load.

Type of Change

  • Bug fix
  • New feature
  • Breaking change
  • Documentation update

For more info, see Contributing Guidelines.

…nt errors

Signed-off-by: Alex Lovell-Troy <alovelltroy@lanl.gov>
Signed-off-by: Alex Lovell-Troy <alovelltroy@lanl.gov>
@alexlovelltroy alexlovelltroy merged commit 5fb3d1f into main Jul 9, 2026
5 checks passed
@synackd synackd deleted the bugfix/smd-connection-timeouts branch July 9, 2026 22:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants