Skip to content

fix(internal/bufio): Default Max Token Size#220

Merged
jshlbrd merged 2 commits intomainfrom
jshlbrd/fix/bufio-scan-token
Jul 29, 2024
Merged

fix(internal/bufio): Default Max Token Size#220
jshlbrd merged 2 commits intomainfrom
jshlbrd/fix/bufio-scan-token

Conversation

@jshlbrd
Copy link
Contributor

@jshlbrd jshlbrd commented Jul 27, 2024

Description

  • Fixes the max token size for bufio scanners

Motivation and Context

Noticed that this was wrong when working on another PR. These are not equivalent: https://go.dev/play/p/rGQz8_p5zR3. This doesn't affect deployments in AWS Lambda.

How Has This Been Tested?

Try to read a file with very large lines using any file reader app -- it should fail using the main branch but succeed with this change.

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

  • My code follows the code style of this project.
  • My change requires a change to the documentation.
  • I have updated the documentation accordingly.

@jshlbrd jshlbrd marked this pull request as ready for review July 29, 2024 16:40
@jshlbrd jshlbrd requested a review from a team as a code owner July 29, 2024 16:40
Copy link
Contributor

@shellcromancer shellcromancer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great catch, also confirming that I don't see anywhere else in the codebase we use bitwise XOR for exponentiation.

@jshlbrd jshlbrd merged commit e3dfa47 into main Jul 29, 2024
@jshlbrd jshlbrd deleted the jshlbrd/fix/bufio-scan-token branch July 29, 2024 17:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants