Skip to content

fix(kinesis): Limit Aggregated Records to 1MB#169

Merged
shellcromancer merged 2 commits intomainfrom
jshlbrd/fix/kinesis/agg-size
May 3, 2024
Merged

fix(kinesis): Limit Aggregated Records to 1MB#169
shellcromancer merged 2 commits intomainfrom
jshlbrd/fix/kinesis/agg-size

Conversation

@jshlbrd
Copy link
Contributor

@jshlbrd jshlbrd commented May 3, 2024

Description

  • Adds a 1MB limit to the Kinesis KPL aggregator

Motivation and Context

KPL records still need a limit, otherwise the Kinesis transform may generate a record bigger than the 1 MB service limit.

How Has This Been Tested?

Integration tested on data that is known to cause this issue.

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

  • My code follows the code style of this project.
  • My change requires a change to the documentation.
  • I have updated the documentation accordingly.

@jshlbrd jshlbrd marked this pull request as ready for review May 3, 2024 21:50
@jshlbrd jshlbrd requested a review from a team as a code owner May 3, 2024 21:50
@shellcromancer shellcromancer merged commit 9065f1c into main May 3, 2024
@shellcromancer shellcromancer deleted the jshlbrd/fix/kinesis/agg-size branch May 3, 2024 21:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants