Skip to content

Comments

Support for Kafka Queues (Share consumer)#3319

Open
ozangunalp wants to merge 2 commits intosmallrye:mainfrom
ozangunalp:kafka_4_shared_consumer
Open

Support for Kafka Queues (Share consumer)#3319
ozangunalp wants to merge 2 commits intosmallrye:mainfrom
ozangunalp:kafka_4_shared_consumer

Conversation

@ozangunalp
Copy link
Collaborator

@ozangunalp ozangunalp commented Feb 20, 2026

Provides a new Kafka Source to support Kafka Queues (Share Consumer client).

Kafka companion: shareConsume* methods

Kafka Connector:

  • share-group flag to create Kafka Share Consumer client.
  • ReactiveKafkaShareConsumer with polling stream without pause/resume. -> Keeps polling the consumer and retries on IllegalStateException thrown by the consumer.
  • ReactiveKafkaAdminClient methods for share Consumer description
  • Fixed commit and failure strategies to handle per-message acknowledgement. -> Tracks received messages and listens to committed acks. Applies a timeout for pending records that are never acknowledged. Sends RENEW acknowledgements for messages still in-processing
  • Graceful shutdown waits for pending messages, trying to commit at each iteration.
  • Topic-based Health check to check partition assignments to the current client.
  • ShareGroupAcknowledgment metadata enables processing code to decide between different per-message ack type. Same nack metadata.
  • Batch consuming mode with per-message acking.
  • Deserialization failure handler flags to choose which ack type to apply on deserialization failure.

@ozangunalp ozangunalp force-pushed the kafka_4_shared_consumer branch from 0fafa9a to 4d6c400 Compare February 20, 2026 13:56
@ozangunalp ozangunalp force-pushed the kafka_4_shared_consumer branch from 4d6c400 to 20dfd8a Compare February 20, 2026 15:11
@ozangunalp
Copy link
Collaborator Author

CI green, but some share group tests may still be flaky. Re-running in case.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant