[feat][trace] input/output configurable preview & agent metadata discovery by fanhua-bytedance · Pull Request #492 · coze-dev/coze-loop

fanhua-bytedance · 2026-04-15T03:30:52Z

What type of PR is this?

Check the PR title

This PR title match the format: [<type>][<scope>] <description>. For example: [fix][backend] flaky fix
The description of this PR title is user-oriented and clear enough for others to understand.
Add documentation if the current PR requires user awareness at the usage level.
This PR is written in English. PRs not in English will not be reviewed.

(Optional) Translate the PR title into Chinese

(Optional) More detailed description for this PR(en: English/zh: Chinese)

en:
zh(optional):

(Optional) Which issue(s) this PR fixes

CLAassistant · 2026-04-15T03:31:02Z

All committers have signed the CLA.

codecov · 2026-04-15T04:14:12Z

Codecov Report

❌ Patch coverage is 83.43750% with 53 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
...dler/coze/loop/apis/observability_trace_service.go	0.00%	22 Missing ⚠️
backend/modules/observability/application/trace.go	80.24%	15 Missing and 1 partial ⚠️
...bservability/domain/trace/service/trace_service.go	91.66%	3 Missing and 3 partials ⚠️
...ility/domain/trace/entity/column_extract_config.go	90.00%	2 Missing and 2 partials ⚠️
...ace/service/trace/span_processor/clip_processor.go	88.46%	1 Missing and 2 partials ⚠️
.../observability/infra/repo/column_extract_config.go	95.34%	1 Missing and 1 partial ⚠️

@@            Coverage Diff             @@
##             main     #492      +/-   ##
==========================================
+ Coverage   77.14%   77.18%   +0.03%     
==========================================
  Files         650      653       +3     
  Lines       72527    72825     +298     
==========================================
+ Hits        55953    56211     +258     
- Misses      13241    13271      +30     
- Partials     3333     3343      +10

Flag	Coverage Δ
unittests	`77.18% <83.43%> (+0.03%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
...plication/convertor/trace/column_extract_config.go	`100.00% <100.00%> (ø)`
backend/pkg/json/ojg.go	`95.95% <100.00%> (+0.35%)`	⬆️
.../observability/infra/repo/column_extract_config.go	`95.34% <95.34%> (ø)`
...ace/service/trace/span_processor/clip_processor.go	`90.32% <88.46%> (-1.57%)`	⬇️
...ility/domain/trace/entity/column_extract_config.go	`90.00% <90.00%> (ø)`
...bservability/domain/trace/service/trace_service.go	`89.04% <91.66%> (+0.10%)`	⬆️
backend/modules/observability/application/trace.go	`77.71% <80.24%> (+1.13%)`	⬆️
...dler/coze/loop/apis/observability_trace_service.go	`0.00% <0.00%> (ø)`

... and 1 file with indirect coverage changes

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7a3ce58...1a279aa. Read the comment docs.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

…to fix CK Missing columns error

…fields to fix AGW body parsing

…onfig, GetAgentMetadata

…definitions

…ql for schema consistency

Reuse BuildBuiltinFilters + CombineFilters to add space_id and call_type filters, consistent with ListSpans logic.

taoyifan89 · 2026-04-16T07:04:57Z

这个不该传上来吧

taoyifan89 · 2026-04-16T14:39:20Z

 	InsertAnnotations(context.Context, *InsertAnnotationParam) error
 	UpsertTrajectoryConfig(context.Context, *UpsertTrajectoryConfigParam) error
 	GetTrajectoryConfig(context.Context, GetTrajectoryConfigParam) (*entity.TrajectoryConfig, error)
+	UpsertColumnExtractConfig(context.Context, *UpsertColumnExtractConfigParam) error


这个单独拆一个repo吧，和trace也没有太多关系

taoyifan89 · 2026-04-16T14:43:05Z

 	GetTrajectories(ctx context.Context, workspaceID int64, traceIDs []string, startTime, endTime int64,
 		platformType loop_span.PlatformType) (map[string]*loop_span.Trajectory, error)
 	MergeHistoryMessagesByRespIDBatch(ctx context.Context, spans []*loop_span.Span, platformType loop_span.PlatformType) error
+	UpsertColumnExtractConfig(ctx context.Context, req *UpsertColumnExtractConfigRequest) error


ColumnExtractConfig 单独拆一个service吧，和trace关系不大。这里逻辑其实非常简单，直接application调repo都行，类似view的写法
GetAgentMetadata查trace的，可以放着

taoyifan89 · 2026-04-16T14:47:57Z

+				IsCustom:  true,
+			},
+			{
+				FieldName: tagKeyGenAIAgentName,


这个不需要适配

taoyifan89 · 2026-04-16T14:48:42Z

+	tagKeyGenAIAgentName = "gen_ai.agent.name"
+)
+
+func extractAgentName(s *loop_span.Span) string {


这个也没必要了

taoyifan89 · 2026-04-16T14:49:04Z

+	}
+	var agentName string
+	_ = filters.Traverse(func(f *loop_span.FilterField) error {
+		if f.FieldName == tagKeyAgentName || f.FieldName == tagKeyGenAIAgentName {


taoyifan89 · 2026-04-16T14:49:30Z

 	spanRedisDao redis_dao.ISpansRedisDao,
 	spanProducer mq.ISpanProducer,
 	trajectoryConfDao mysql.ITrajectoryConfigDao,
+	columnExtractConfDao mysql.IColumnExtractConfigDao,


拆出去吧

taoyifan89 · 2026-04-16T14:52:35Z

 	clipProcessorSuffix             = "..."
 )

+var defaultExtractRules = map[loop_span.SpanListType][]entity.ColumnExtractRule{


这个是什么时候会用到的?

这个是什么时候会用到的?
数据库无记录或配置为空的情况会使用默认配置

taoyifan89 · 2026-04-16T14:53:44Z

-		if span == nil {
+	var rules []entity.ColumnExtractRule
+	if c.traceRepo != nil && c.settings.WorkspaceId > 0 {
+		cfg, err := c.traceRepo.GetColumnExtractConfig(ctx, repo.GetColumnExtractConfigParam{


这里应该缺了这个空间没有配置，取全局的默认配置逻辑吧？

目前是如果无配置则会返回适用于ModelSpan的默认配置，对与自定义上报和其他的类型的还没做默认配置。

这里应该缺了这个空间没有配置，取全局的默认配置逻辑吧？

…ame tag key

…for ClipProcessor

- Move SelectBest/Extract/configScore logic from clip_processor to entity layer - Use workspace_id=0 for global default configs with wildcard '*' matching - Add 4-bit scoring strategy (workspace > agent > spanListType > platformType) - Simplify ClipProcessor.Transform to delegate to entity layer - Add DB error logging consistent with other processors - Seed 3 default extract configs via init SQL with fixed IDs for idempotency - Add comprehensive tests for SelectBest, configScore, Extract

…te default configs - Rewrite SelectBest to use two-phase direct matching instead of bit-score approach - Reduce default column extract configs from 3 to 2: (0,*,llm_span) and (0,prompt,root_span) - Change span_list_type SQL query from wildcard IN to exact match - Update tests with 20 cases for 100% SelectBest coverage

mocayo changed the title ~~[Feat]trace input/output configurable preview & agent metadata discovery~~ [feat][trace] input/output configurable preview & agent metadata discovery Apr 15, 2026

fanhua-bytedance force-pushed the feat/trace_preview branch from fea6639 to a4d80b8 Compare April 15, 2026 03:48

fanhua-bytedance force-pushed the feat/trace_preview branch from a4d80b8 to 129e512 Compare April 16, 2026 07:37

fanhua-bytedance added 5 commits April 16, 2026 15:44

feat: trace input/output configurable preview & agent metadata discovery

97b246f

fix: add span_id and start_time to SelectColumns in GetAgentMetadata …

13c7cdc

…to fix CK Missing columns error

fix: remove api.body annotation from ColumnExtractRule nested struct …

aa34bdf

…fields to fix AGW body parsing

test: add unit tests for UpsertColumnExtractConfig, GetColumnExtractC…

d0c13b8

…onfig, GetAgentMetadata

fix: regenerate kitex_gen after rebase, add missing ListMetadata IDL …

bb67520

…definitions

fanhua-bytedance force-pushed the feat/trace_preview branch from 129e512 to bb67520 Compare April 16, 2026 07:45

fanhua-bytedance added 4 commits April 16, 2026 15:47

fix: add observability_column_extract_config.sql to helm-chart init-s…

93674ec

…ql for schema consistency

ci: trigger CI re-run with latest commits

bd9dfd9

ci: trigger CI re-run with latest commits

499ddf8

fix: add space_id filter to GetAgentMetadata query

ebfc27e

Reuse BuildBuiltinFilters + CombineFilters to add space_id and call_type filters, consistent with ListSpans logic.

taoyifan89 reviewed Apr 16, 2026

View reviewed changes

fanhua-bytedance added 3 commits April 20, 2026 10:47

refactor: extract ColumnExtractConfig to standalone repo, fix agent_n…

a451348

…ame tag key

fix: gofmt trace.go struct field alignment

0a7668f

fix: normalize JSONPath in ClipProcessor to support frontend format

5d9668d

fanhua-bytedance force-pushed the feat/trace_preview branch from 0ac1b95 to 5d9668d Compare April 20, 2026 04:05

fix: handle column[index] format in JSONPath normalization

791b5b5

fanhua-bytedance force-pushed the feat/trace_preview branch from 61d5c40 to 78044f6 Compare April 20, 2026 07:31

fix: escape hyphenated keys in JSONPath to bracket notation

4ae2cc4

fanhua-bytedance force-pushed the feat/trace_preview branch 3 times, most recently from b7bb283 to fea7e85 Compare April 20, 2026 11:56

feat: add config fallback strategy and recursive JSONPath extraction …

d149df0

…for ClipProcessor

fanhua-bytedance force-pushed the feat/trace_preview branch from fea7e85 to d149df0 Compare April 20, 2026 12:23

chore: remove golangci-lint binary and add backend/bin/ to gitignore

02ea69d

fanhua-bytedance force-pushed the feat/trace_preview branch from c2c4da4 to 02ea69d Compare April 22, 2026 03:30

taoyifan89 reviewed Apr 22, 2026

View reviewed changes

fanhua-bytedance force-pushed the feat/trace_preview branch from 5d000b3 to bb1534c Compare April 22, 2026 03:47

Merge branch 'main' into feat/trace_preview

f4f75ea

fanhua-bytedance force-pushed the feat/trace_preview branch from bb1534c to f4f75ea Compare April 22, 2026 04:21

taoyifan89 reviewed Apr 24, 2026

View reviewed changes

Comment thread backend/modules/observability/domain/trace/entity/column_extract_config.go Outdated

fanhua-bytedance added 2 commits April 27, 2026 14:10

merge: resolve conflicts with main, regenerate kitex_gen and wire_gen

1a279aa

Conversation

fanhua-bytedance commented Apr 15, 2026

What type of PR is this?

Check the PR title

(Optional) Translate the PR title into Chinese

(Optional) More detailed description for this PR(en: English/zh: Chinese)

(Optional) Which issue(s) this PR fixes

Uh oh!

CLAassistant commented Apr 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov Bot commented Apr 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

CLAassistant commented Apr 15, 2026 •

edited

Loading

codecov Bot commented Apr 15, 2026 •

edited

Loading