Skip to content

feat(sink): tuple sink supports format #3954

Merged
ngjaying merged 2 commits intolf-edge:masterfrom
ngjaying:mem
Dec 17, 2025
Merged

feat(sink): tuple sink supports format #3954
ngjaying merged 2 commits intolf-edge:masterfrom
ngjaying:mem

Conversation

@ngjaying
Copy link
Collaborator

@ngjaying ngjaying commented Dec 16, 2025

Use format to decode for TupleSink if it has transformed by DataTemplate

@ngjaying ngjaying changed the title feat(memory sink): add RawTuple decoding with configurable format feat(sink): support format after dataTemplate transform Dec 16, 2025
@codecov
Copy link

codecov bot commented Dec 16, 2025

Codecov Report

❌ Patch coverage is 94.91525% with 3 lines in your changes missing coverage. Please review.
✅ Project coverage is 71.86%. Comparing base (2604093) to head (15eb9eb).
⚠️ Report is 5 commits behind head on master.

Files with missing lines Patch % Lines
internal/topo/node/sink_node.go 94.92% 2 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #3954      +/-   ##
==========================================
+ Coverage   71.77%   71.86%   +0.09%     
==========================================
  Files         455      455              
  Lines       52412    52451      +39     
==========================================
+ Hits        37618    37693      +75     
+ Misses      11872    11842      -30     
+ Partials     2922     2916       -6     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@ngjaying ngjaying changed the title feat(sink): support format after dataTemplate transform feat(sink): tuple sink with datatemplate support format Dec 16, 2025
@ngjaying ngjaying changed the title feat(sink): tuple sink with datatemplate support format feat(sink): tuple sink support format Dec 16, 2025
@ngjaying ngjaying changed the title feat(sink): tuple sink support format feat(sink): tuple sink supports format Dec 16, 2025
- Create converter based on SinkConf.Format and SinkConf.SchemaId
- Replace hardcoded json.Unmarshal with converter.Decode
- Handle map[string]any, []map[string]any, and []any decode results
- For list results, call CollectList instead of Collect
- Add decodeAndCollect helper function for RawTuple handling

Signed-off-by: Jiyong Huang <huangjy@emqx.io>
Signed-off-by: Jiyong Huang <huangjy@emqx.io>
@ngjaying ngjaying marked this pull request as ready for review December 16, 2025 11:11
@ngjaying ngjaying requested a review from Yisaer December 16, 2025 11:11
@ngjaying ngjaying merged commit 66aed42 into lf-edge:master Dec 17, 2025
60 checks passed
@ngjaying ngjaying deleted the mem branch January 7, 2026 08:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants