💡 Feature Description and Proposed Solution
AgentOps captures and serializes agent events, LLM calls, and tool results as JSON by default.
At scale this creates compounding overhead that affects both observability costs and the agent pipelines being observed.
The core problem:
~44% of tokens in typical agent payloads are pure JSON syntax overhead before any reasoning begins.
For AgentOps specifically this matters in two ways:
-
OBSERVABILITY COST
Every event AgentOps captures is serialized and transmitted as JSON. At high agent volumes this creates significant egress overhead. 4.1x smaller wire format = 4.1x less egress cost.
-
PIPELINE COST
Teams using AgentOps to monitor inference costs are often unaware that JSON serialization itself is a major cost driver. 44% token reduction = $59K saved per 10M loops.
Proposed solution: pluggable serialization interface allowing ULMEN as an opt-in replacement for JSON.
ULMEN benchmarks on NVIDIA Tesla T4:
Beyond compression, ULMEN adds a Semantic Firewall that validates agent state transitions:
- Rejects orphaned tool calls
- Catches invalid step transitions
- Validates enum states
- Raises structured errors vs silent failures
This is particularly valuable for AgentOps because silent failures that pass through JSON undetected are exactly the kind of events AgentOps should be catching but currently can't see.
Proposed API:
Current
agentops.init(api_key="...")
Proposed
agentops.init(
api_key="...",
serializer="ulmen" # opt-in
)
ULMEN is:
- Drop-in Python/Rust library
- No schema compilation required
- Pure Python fallback if Rust unavailable
- BSL license, free under $10M revenue
- 1,364 tests, 100% statement coverage
Reproducible benchmark notebook:
github.com/makroumi/ulmen
Happy to submit a PR implementing the serializer interface once maintainers confirm preferred integration approach.
🤔 Related Problem
Teams using AgentOps to monitor and reduce inference costs are often surprised to discover that JSON serialization itself is one of their largest hidden cost drivers.
The irony: the observability tool that helps teams track spending is itself contributing to the overhead through JSON serialization of every captured event.
At 10M agent loops monitored through AgentOps:
- JSON overhead: ~$59K in wasted token spend
- AgentOps egress: 4.1x larger than necessary
Both problems solved simultaneously with pluggable ULMEN serialization.
🤝 Contribution
💡 Feature Description and Proposed Solution
AgentOps captures and serializes agent events, LLM calls, and tool results as JSON by default.
At scale this creates compounding overhead that affects both observability costs and the agent pipelines being observed.
The core problem:
~44% of tokens in typical agent payloads are pure JSON syntax overhead before any reasoning begins.
For AgentOps specifically this matters in two ways:
OBSERVABILITY COST
Every event AgentOps captures is serialized and transmitted as JSON. At high agent volumes this creates significant egress overhead. 4.1x smaller wire format = 4.1x less egress cost.
PIPELINE COST
Teams using AgentOps to monitor inference costs are often unaware that JSON serialization itself is a major cost driver. 44% token reduction = $59K saved per 10M loops.
Proposed solution: pluggable serialization interface allowing ULMEN as an opt-in replacement for JSON.
ULMEN benchmarks on NVIDIA Tesla T4:
Beyond compression, ULMEN adds a Semantic Firewall that validates agent state transitions:
This is particularly valuable for AgentOps because silent failures that pass through JSON undetected are exactly the kind of events AgentOps should be catching but currently can't see.
Proposed API:
Current
agentops.init(api_key="...")
Proposed
agentops.init(
api_key="...",
serializer="ulmen" # opt-in
)
ULMEN is:
Reproducible benchmark notebook:
github.com/makroumi/ulmen
Happy to submit a PR implementing the serializer interface once maintainers confirm preferred integration approach.
🤔 Related Problem
Teams using AgentOps to monitor and reduce inference costs are often surprised to discover that JSON serialization itself is one of their largest hidden cost drivers.
The irony: the observability tool that helps teams track spending is itself contributing to the overhead through JSON serialization of every captured event.
At 10M agent loops monitored through AgentOps:
Both problems solved simultaneously with pluggable ULMEN serialization.
🤝 Contribution