Add bundle version of utf8_range to validate attributes

[owent]

Fixes #3491

Changes

• Add bundle version of utf8_range to validate attribute values
• Update the implementation for the new SPEC(2026-03)

For significant contributions please make sure you have completed the following items:

• CHANGELOG.md updated for non-trivial changes
• Unit tests have been added
• Changes in public API reviewed

[netlify]

✅ Deploy Preview for opentelemetry-cpp-api-docs canceled.

Name	Link
🔨 Latest commit	2ab31e8
🔍 Latest deploy log	https://app.netlify.com/projects/opentelemetry-cpp-api-docs/deploys/68a8d5196a46540008c6326f

[Copilot]

Pull Request Overview

This PR introduces a bundled UTF-8 validation library (utf8_range) and integrates it into attribute validation throughout the SDK, ensuring that span, resource, and instrumentation-scope attributes containing invalid UTF-8 are filtered out with warnings.

• Add attribute_validity module wrapping the new utf8_range library
• Integrate attribute validation/filtering into Span, Resource, and InstrumentationScope
• Add build rules for utf8_range and update CMake/Bazel files accordingly

Reviewed Changes

Copilot reviewed 16 out of 16 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
sdk/src/common/internal/utf8_range/uft8_range.cc	Added bundled UTF-8 validation implementation
sdk/src/common/internal/utf8_range/utf8_range.h	Declared public API for UTF-8 validation (IsValid, ValidPrefix)
sdk/src/common/attribute_validity.cc	Wrapped validation logic around SDK attribute values
sdk/src/common/BUILD	Added utf8_range and attribute_validity libraries
sdk/src/common/CMakeLists.txt	Updated to include new source in common build
sdk/src/trace/span.cc	Applied attribute validation in Span::SetAttribute and AddLink
sdk/src/resource/resource.cc	Applied attribute validation in Resource constructor
sdk/include/opentelemetry/sdk/instrumentationscope/instrumentation_scope.h	Applied attribute validation in instrumentation scope factories and setters
sdk/test/instrumentationscope/CMakeLists.txt	Updated linkage to include opentelemetry_common
sdk/test/instrumentationscope/BUILD	Added dependency on attribute_validity for tests

Comments suppressed due to low confidence (2)

sdk/src/common/internal/utf8_range/uft8_range.cc:1

• The source file is named uft8_range.cc, which looks like a typo. Renaming it to utf8_range.cc would align with the header (utf8_range.h) and improve consistency.

// Copyright 2023 Google LLC

sdk/src/common/BUILD:6

• The new utf8_range library adds core validation logic but there are no accompanying unit tests verifying valid and invalid UTF-8 sequences. Consider adding tests under sdk/test/common/ to cover edge cases.

cc_library(

[codecov]

Codecov Report

❌ Patch coverage is 58.79828% with 96 lines in your changes missing coverage. Please review.
✅ Project coverage is 89.14%. Comparing base (2eba094) to head (49b474a).
⚠️ Report is 8 commits behind head on main.

Files with missing lines	Patch %	Lines
sdk/src/common/attribute_validity.cc	56.10%	36 Missing ⚠️
sdk/src/common/internal/utf8_range/uft8_range.cc	50.75%	33 Missing ⚠️
...lude/opentelemetry/sdk/common/attribute_validity.h	63.64%	12 Missing ⚠️
...y/sdk/instrumentationscope/instrumentation_scope.h	61.12%	7 Missing ⚠️
sdk/src/trace/span.cc	64.71%	6 Missing ⚠️
...ntelemetry/sdk/metrics/view/attributes_processor.h	66.67%	1 Missing ⚠️
sdk/src/resource/resource.cc	92.31%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #3512      +/-   ##
==========================================
- Coverage   90.12%   89.14%   -0.98%     
==========================================
  Files         227      230       +3     
  Lines        7261     7480     +219     
==========================================
+ Hits         6543     6667     +124     
- Misses        718      813      +95     

Files with missing lines	Coverage Δ
...ntelemetry/sdk/metrics/view/attributes_processor.h	85.00% <66.67%> (ø)
sdk/src/resource/resource.cc	95.13% <92.31%> (-1.65%)	⬇️
sdk/src/trace/span.cc	85.57% <64.71%> (-4.08%)	⬇️
...y/sdk/instrumentationscope/instrumentation_scope.h	87.50% <61.12%> (-12.50%)	⬇️
...lude/opentelemetry/sdk/common/attribute_validity.h	63.64% <63.64%> (ø)
sdk/src/common/internal/utf8_range/uft8_range.cc	50.75% <50.75%> (ø)
sdk/src/common/attribute_validity.cc	56.10% <56.10%> (ø)

... and 1 file with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[owent]

The clang-tidy check failure does not appear to be caused by this PR.

[dbarker]

Vendoring this library and always running validation by default adds some maintenance and performance overhead. Ideally we can just get the required package/targets along with protobuf and then string validation can be tested as a preview feature and controlled by the users.

Can we require the utf8_range package to be found/fetched like the other dependencies?

The package is within protobuf source and built with protobuf. For CMake builds it can be installed using the utf8_range_ENABLE_INSTALL CMake option (See protobuf/third_party/utf8_range/CMakeLists.txt#L10 ) and then found with find_package. If protobuf is fetched then we can use FetchContent_Declare on the source directory ${protobuf_SOURCE_DIR}/third_party/utf8_range and build the target directly. Bazel builds relying on the protobuf packaged utf8_range also seem doable.

What do you think?

[owent]

utf8_range was moved into protobuf’s source tree starting from protobuf v22, and it is only included in CMake installation when the protobuf binaries are built.
If an older protobuf version is used, or if only the libraries are built for cross-compilation, utf8_range may not be installed and therefore cannot be found by find_package.

Enforcing the use of FetchContent_Declare may also introduce ABI compatibility issues when users link our project against a prebuilt version of protobuf.

For this reason, I imported it in the same way as the bundled version of abseil-cpp.

Alternatively, do you think we can disable these checks, even though this may result in some records being dropped?

[dbarker]

utf8_range was moved into protobuf’s source tree starting from protobuf v22, and it is only included in CMake installation when the protobuf binaries are built.
If an older protobuf version is used, or if only the libraries are built for cross-compilation, utf8_range may not be installed and therefore cannot be found by find_package.

We currently have v3.21 as the minimum supported/tested protobuf version (it is a 4 year old release) and we can bump it to 3.22 if needed, but only supporting utf8 validation for otlp attributes if utf8_range is installed or fetched seems reasonable.

If protobuf (v3.22+) is fetched now (by default otel-cpp will fetch protobuf v6.31) the utf_range CMake targets (utf8_validity) will be available to link to.

Enforcing the use of FetchContent_Declare may also introduce ABI compatibility issues when users link our project against a prebuilt version of protobuf.

The utf8_range package is independent and used as a static dependency in protobuf and will be used as a private dependency in otel-cpp OTLP exporters. ABI compatibility with protobuf may not be of concern in this case. Either way I think we use the same version bundled with protobuf.

For this reason, I imported it in the same way as the bundled version of abseil-cpp.

Makes sense. Just raising the alternative to optionally enable UTF-8 validation when the SDK is built with OTLP support and installing/importing/fetching utf8_range along with the protobuf dependency. This could be less maintenance for otel-cpp.

Alternatively, do you think we can disable these checks, even though this may result in some records being dropped?

The spec on OTLP attribute type mapping (specification/common/attribute-type-mapping.md) uses Should for UTF8 validation so it appears to be an optional requirement of the SDK. I think we should make it a compile time option that is off by default for now and evaluate performance/stability (similar to other _PREVIEW options).

[dbarker]

Should these attribute values that are not valid UTF-8 sequences be set to byte values here?

Spec language (specification/common/attribute-type-mapping.md#L109):

String Values
String values which are valid UTF-8 sequences SHOULD be converted to AnyValue's string_value field.

String values which are not valid Unicode sequences SHOULD be converted to AnyValue's bytes_value with the bytes representing the string in the original order and format of the source string.

[owent]

Thanks, the spec seems changed after the first version of this PR.I will change it to follow the latest spec.

[dbarker]

Same question as above: ignore or set as bytes?

[dbarker]

If we have to place invalid utf8 sequences into byte attribute values per the spec should the IsValid check always be performed when constructing an attribute map (attribute_utils.h)?

[owent]

Thanks, it looks like the spec changed after the initial version of this PR.
In the earlier version, bytes were allowed in log attributes, but not in traces, metrics, or resources.
I’ll review the spec again over the next few days and update the implementation to match the latest version.

[dbarker]

Looking at the spec it may only apply utf8 validation to OTLP type mapping. If that is the case we may be able to make a more isolated change to OtlpPopulateAttributeUtils::PopulateAnyValue to support this as @lalitb suggests.

[dbarker]

Hey @owent, sorry for the long delay in review. I've added some comments above. Two main questions:

• Can we treat utf8_range as a standard dependency (whose source comes with protobuf and can be installed with protobuf)?
• Do we always need to put invalid utf8 strings into byte values of attributes?

[lalitb]

@owent - Is it possible to scope this fix within OTLP instead of adding SDK-wide validation/framework changes?

The original issue seems much narrower: OTLP/protobuf serialization is where invalid UTF-8 actually becomes a problem. So I’d expect a smaller fix closer to the exporter boundary, for example validating right before writing protobuf string fields / set_string_value(...), and handling invalid data there, rather than introducing a broader SDK-wide mechanism. If we go that route, I’m also not sure we need to bring in utf8_range in this broader way.

[owent]

@owent - Is it possible to scope this fix within OTLP instead of adding SDK-wide validation/framework changes?

The original issue seems much narrower: OTLP/protobuf serialization is where invalid UTF-8 actually becomes a problem. So I’d expect a smaller fix closer to the exporter boundary, for example validating right before writing protobuf string fields / set_string_value(...), and handling invalid data there, rather than introducing a broader SDK-wide mechanism. If we go that route, I’m also not sure we need to bring in utf8_range in this broader way.

Thanks. This is also due to a change in the spec.
The latest baggage API spec (https://opentelemetry.io/docs/specs/otel/baggage/api/) also requires UTF-8 strings. Do you think we can ignore this requirement?