Skip to content

Commit 0db0ac7

Browse files
petrotiurinclaude
petrotiurin
and
claude
authored
[OPIK-5326] [SDK] feat: cast job input values to declared param types in runner (#5952)
* [OPIK-5326] [SDK] feat: cast job input values to declared param types in runner Add input type casting to both the TypeScript and Python in-process runner loops so agent functions receive correctly-typed arguments regardless of how the server serialised the values in job.inputs. TypeScript: export castInputValue() from InProcessRunnerLoop, apply it in invokeAgent using each Param's declared type (boolean / number / string). Python: add cast_input_value() to in_process_loop, apply it in _execute_job for all keys that match a registered param (bool / int / float / str); keys such as opik_args that are not in params pass through unchanged. Both implementations follow the same pattern as typeHelpers.ts: primitives are cast natively, complex types (dict/list) are JSON-serialised as strings, and null/None passes through unchanged. Unit tests added for both SDKs using parametrisation to cover each type individually and a set of multi-param combination scenarios. Implements OPIK-5326 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * [OPIK-5326] [SDK] refactor: reuse type_helpers deserialization in runner casting Address baz-reviewer feedback on PR #5952: TypeScript (comment #3009010633): castInputValue now delegates to deserializeValue() from typeHelpers.ts for string→boolean and string→number conversions. Adds a Number.isNaN guard so non-numeric strings (e.g. "abc") throw TypeError instead of silently passing NaN to the agent function. Python (comment #3009010639): extract_params now calls unwrap_optional() from type_helpers.py before extracting the type name, so Optional[int] annotations correctly store type="int" instead of the raw "typing.Optional[int]" string. cast_input_value is rewritten to delegate to backend_value_to_python_value() from type_helpers.py, unifying the conversion logic across AgentConfig and the runner. Python (comment #3009010648): renamed all test functions in test_cast_input_value.py to follow the repo convention test_WHAT__CASE_DESCRIPTION__EXPECTED_RESULT. Added tests for Optional[T] unwrapping in extract_params and for the new backend type name aliases ("boolean", "integer", "string"). Skipping comment #3009010644 (bool "1"→True): strict "true"/"false" only behaviour is intentional and mirrors the TypeScript SDK — the backend serialises booleans as true/false, not "1"/"yes". Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * [OPIK-5326] [SDK] refactor: move type_helpers to shared path type_helpers.py / typeHelpers.ts were under agent_config but are now used by the runner as well. Move them to a neutral location: Python: api_objects/agent_config/type_helpers.py → api_objects/type_helpers.py TypeScript: agent-config/typeHelpers.ts → typeHelpers.ts (opik package root) Update all import sites in both SDKs: agent_config internals (config, blueprint, base, AgentConfig, Blueprint, index), the runner (in_process_loop, registry), the client (Client.ts), and all corresponding test files. No logic changes. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * [OPIK-5303] [SDK] refactor: use module-form import for type_helpers in in_process_loop Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * [OPIK-5326] [SDK] refactor: standardise input casting around backend type names - Remove backend_type param from backend_value_to_python_value (was unused) - Raise TypeError for truncating int casts ("3.9") and bool→int coercion - registry.extract_params now emits backend type names (integer/boolean/string) so Param.type is consistent with what the server expects - cast_input_value delegates directly to backend_type_to_python_type; the dual Python/backend name lookup (type_name_to_python_type) is removed - Add _execute_job integration tests covering multi-typed params in both Python and TypeScript Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * [NA] revert package-lock.json to main Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * test(runner): update extract_params assertions to backend type names * fix(runner): treat '1'/'yes' as truthy bool; report cast errors as failed jobs - Extend bool casting to accept "1" and "yes" as truthy values - Move input casting inside _execute_job's try/except so TypeError from invalid inputs is reported to the backend as a failed job instead of propagating uncaught Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * test(runner): use fake timers in typed-params TS test Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
1 parent af80fec commit 0db0ac7

21 files changed

Lines changed: 689 additions & 62 deletions

File tree

‎sdks/python/src/opik/api_objects/agent_config/base.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@
44

55
from opik.exceptions import AgentConfigNotFound
66
from opik.rest_api import core as rest_api_core
7-
from . import type_helpers
7+
from .. import type_helpers
88
from . import cache as cache_mod
99
from .context import get_active_config_mask
1010

‎sdks/python/src/opik/api_objects/agent_config/blueprint.py‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@
77
from opik.api_objects.prompt.text.prompt import Prompt
88
from opik.api_objects.prompt.chat.chat_prompt import ChatPrompt
99
from opik.rest_api.types.prompt_version_detail import PromptVersionDetail
10-
from . import type_helpers
10+
from .. import type_helpers
1111

1212

1313
def _resolve_prompt_from_commit(
@@ -47,7 +47,7 @@ def _convert_primitives(
4747

4848
if py_type is not None:
4949
values[param.key] = type_helpers.backend_value_to_python_value(
50-
param.value, param.type, py_type
50+
param.value, py_type
5151
)
5252
else:
5353
values[param.key] = param.value

‎sdks/python/src/opik/api_objects/agent_config/config.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@
99
from opik.api_objects import rest_helpers
1010
from opik import id_helpers
1111
from .blueprint import Blueprint
12-
from . import type_helpers
12+
from .. import type_helpers
1313

1414

1515
class AgentConfigManager:

sdks/python/src/opik/api_objects/agent_config/type_helpers.py renamed to sdks/python/src/opik/api_objects/type_helpers.py

Lines changed: 12 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -109,7 +109,6 @@ def python_value_to_metadata_value(
109109

110110
def backend_value_to_python_value(
111111
value: typing.Any,
112-
backend_type: str,
113112
py_type: typing.Any,
114113
) -> typing.Any:
115114
if value is None:
@@ -118,10 +117,20 @@ def backend_value_to_python_value(
118117
if py_type is bool:
119118
if isinstance(value, bool):
120119
return value
121-
return str(value).lower() == "true"
120+
return str(value).lower() in ("true", "1", "yes")
122121

123122
if py_type is int:
124-
return int(float(value)) if not isinstance(value, int) else value
123+
if isinstance(value, bool):
124+
raise TypeError("Cannot cast bool to int")
125+
if isinstance(value, int):
126+
return value
127+
try:
128+
f = float(value) # type: ignore[arg-type]
129+
except (ValueError, TypeError):
130+
raise TypeError(f"Cannot cast {value!r} to int")
131+
if f != int(f):
132+
raise TypeError(f"Cannot cast {value!r} to int: not a whole number")
133+
return int(f)
125134

126135
if py_type is float:
127136
return float(value) if not isinstance(value, float) else value

‎sdks/python/src/opik/runner/in_process_loop.py‎

Lines changed: 43 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -4,11 +4,14 @@
44
import collections
55
import contextvars
66
import inspect
7+
import json
78
import logging
89
import random
910
import threading
1011
import time
11-
from typing import Callable, Optional
12+
from typing import Any, Callable, Optional
13+
14+
from ..api_objects import type_helpers
1215

1316
from ..api_objects.agent_config.context import agent_config_context
1417
from .. import id_helpers
@@ -26,6 +29,37 @@
2629
_CANCELLED_JOBS_MAX_SIZE = 10_000
2730

2831

32+
def cast_input_value(value: object, type_name: str) -> object:
33+
"""Cast *value* to the native Python type indicated by *type_name*.
34+
35+
*type_name* is a backend type name (``"integer"``, ``"boolean"``,
36+
``"float"``, ``"string"``). Unknown names are treated as ``"string"``.
37+
38+
Raises :exc:`TypeError` for values that cannot be safely cast (e.g.
39+
``"3.9"`` for ``"integer"``, or a ``bool`` for ``"integer"``).
40+
``None`` is always returned unchanged. ``dict``/``list`` values are
41+
JSON-serialised when the target type is ``"string"``.
42+
"""
43+
if value is None:
44+
return value
45+
46+
py_type: Any = type_helpers.backend_type_to_python_type(type_name)
47+
if py_type is None:
48+
# Unknown type: pass strings through, JSON-serialize complex types, str() otherwise
49+
if isinstance(value, str):
50+
return value
51+
if isinstance(value, (dict, list)):
52+
return json.dumps(value)
53+
return str(value)
54+
55+
# backend_value_to_python_value uses str(value) for the str case, which gives
56+
# Python repr for dicts/lists instead of JSON — handle that separately first.
57+
if py_type is str and isinstance(value, (dict, list)):
58+
return json.dumps(value)
59+
60+
return type_helpers.backend_value_to_python_value(value, py_type)
61+
62+
2963
def _inject_trace_id(inputs: dict, trace_id: str) -> None:
3064
"""Merge trace_id into inputs["opik_args"]["trace"]["id"].
3165
@@ -205,7 +239,6 @@ async def _execute_job(self, job: LocalRunnerJob) -> None:
205239
mask_id = job.mask_id
206240

207241
trace_id = id_helpers.generate_id()
208-
_inject_trace_id(inputs, trace_id)
209242

210243
self._safe_report_job_result(
211244
job_id=job_id,
@@ -216,6 +249,14 @@ async def _execute_job(self, job: LocalRunnerJob) -> None:
216249
token = set_job_id(job_id)
217250
ctx = contextvars.copy_context()
218251
try:
252+
params_by_name = {p.name: p for p in entry["params"]}
253+
for key in list(inputs.keys()):
254+
if key in params_by_name:
255+
inputs[key] = cast_input_value(
256+
inputs[key], params_by_name[key].type
257+
)
258+
259+
_inject_trace_id(inputs, trace_id)
219260
timeout = job.timeout
220261
if inspect.iscoroutinefunction(func):
221262
with agent_config_context(mask_id):

‎sdks/python/src/opik/runner/registry.py‎

Lines changed: 11 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,8 @@
55
import threading
66
from typing import Any, Callable, Dict, List
77

8+
from opik.api_objects import type_helpers
9+
810
_lock = threading.Lock()
911
REGISTRY: Dict[str, Dict[str, Any]] = {}
1012
_listeners: List[Callable[[str], None]] = []
@@ -13,7 +15,7 @@
1315
@dataclasses.dataclass
1416
class Param:
1517
name: str
16-
type: str = "str"
18+
type: str = "string"
1719

1820

1921
def register(
@@ -52,9 +54,15 @@ def extract_params(fn: Callable) -> List[Param]:
5254
params: List[Param] = []
5355
for param_name, param in sig.parameters.items():
5456
if param.annotation is inspect.Parameter.empty:
55-
type_name = "str"
57+
type_name = "string"
5658
else:
5759
ann = param.annotation
58-
type_name = ann.__name__ if hasattr(ann, "__name__") else str(ann)
60+
inner = type_helpers.unwrap_optional(ann)
61+
if inner is not None:
62+
ann = inner
63+
try:
64+
type_name = type_helpers.python_type_to_backend_type(ann)
65+
except TypeError:
66+
type_name = "string"
5967
params.append(Param(name=param_name, type=type_name))
6068
return params

‎sdks/python/tests/unit/api_objects/agent_config/test_type_helpers.py‎

Lines changed: 26 additions & 37 deletions
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@
55

66
import pytest
77

8-
from opik.api_objects.agent_config import type_helpers
8+
from opik.api_objects import type_helpers
99
from opik.api_objects.prompt.base_prompt import BasePrompt
1010
from opik.api_objects.prompt.text.prompt import Prompt
1111
from opik.api_objects.prompt.chat.chat_prompt import ChatPrompt
@@ -276,17 +276,20 @@ def test_none_value__returns_none(self, py_type):
276276

277277
class TestBackendValueToPythonValue:
278278
@pytest.mark.parametrize(
279-
"value, backend_type, py_type, expected",
279+
"value, py_type, expected",
280280
[
281-
("hello", "string", str, "hello"),
282-
("42", "integer", int, 42),
283-
("42.0", "integer", int, 42),
284-
(42, "integer", int, 42),
285-
("0.6", "float", float, 0.6),
286-
(0.6, "float", float, 0.6),
287-
("true", "boolean", bool, True),
288-
("false", "boolean", bool, False),
289-
(True, "boolean", bool, True),
281+
("hello", str, "hello"),
282+
("42", int, 42),
283+
("42.0", int, 42),
284+
(42, int, 42),
285+
("0.6", float, 0.6),
286+
(0.6, float, 0.6),
287+
("true", bool, True),
288+
("1", bool, True),
289+
("yes", bool, True),
290+
("false", bool, False),
291+
("0", bool, False),
292+
(True, bool, True),
290293
],
291294
ids=[
292295
"str",
@@ -296,17 +299,15 @@ class TestBackendValueToPythonValue:
296299
"float_from_str",
297300
"float_native",
298301
"bool_true_str",
302+
"bool_one_str",
303+
"bool_yes_str",
299304
"bool_false_str",
305+
"bool_zero_str",
300306
"bool_native",
301307
],
302308
)
303-
def test_primitives__deserialized_correctly(
304-
self, value, backend_type, py_type, expected
305-
):
306-
assert (
307-
type_helpers.backend_value_to_python_value(value, backend_type, py_type)
308-
== expected
309-
)
309+
def test_primitives__deserialized_correctly(self, value, py_type, expected):
310+
assert type_helpers.backend_value_to_python_value(value, py_type) == expected
310311

311312
@pytest.mark.parametrize(
312313
"value, py_type, expected",
@@ -319,23 +320,18 @@ def test_primitives__deserialized_correctly(
319320
ids=["list_from_json", "list_native", "dict_from_json", "dict_native"],
320321
)
321322
def test_collections__deserialized_correctly(self, value, py_type, expected):
322-
assert (
323-
type_helpers.backend_value_to_python_value(value, "string", py_type)
324-
== expected
325-
)
323+
assert type_helpers.backend_value_to_python_value(value, py_type) == expected
326324

327325
def test_none__returns_none(self):
328-
assert type_helpers.backend_value_to_python_value(None, "string", str) is None
326+
assert type_helpers.backend_value_to_python_value(None, str) is None
329327

330328
@pytest.mark.parametrize(
331329
"py_type",
332330
[Prompt, ChatPrompt, BasePrompt],
333331
ids=["Prompt", "ChatPrompt", "BasePrompt"],
334332
)
335333
def test_prompt_type__returns_raw_version_id_string(self, py_type):
336-
result = type_helpers.backend_value_to_python_value(
337-
"ver-xyz", "prompt", py_type
338-
)
334+
result = type_helpers.backend_value_to_python_value("ver-xyz", py_type)
339335
assert result == "ver-xyz"
340336

341337
@pytest.mark.parametrize(
@@ -344,21 +340,17 @@ def test_prompt_type__returns_raw_version_id_string(self, py_type):
344340
ids=["Prompt", "ChatPrompt", "BasePrompt"],
345341
)
346342
def test_prompt_type__none_value__returns_none(self, py_type):
347-
assert (
348-
type_helpers.backend_value_to_python_value(None, "prompt", py_type) is None
349-
)
343+
assert type_helpers.backend_value_to_python_value(None, py_type) is None
350344

351345
def test_prompt_version_type__returns_raw_version_id_string(self):
352346
result = type_helpers.backend_value_to_python_value(
353-
"ver-pv-xyz", "prompt_version", PromptVersionDetail
347+
"ver-pv-xyz", PromptVersionDetail
354348
)
355349
assert result == "ver-pv-xyz"
356350

357351
def test_prompt_version_type__none_value__returns_none(self):
358352
assert (
359-
type_helpers.backend_value_to_python_value(
360-
None, "prompt_version", PromptVersionDetail
361-
)
353+
type_helpers.backend_value_to_python_value(None, PromptVersionDetail)
362354
is None
363355
)
364356

@@ -386,11 +378,8 @@ class TestRoundTrip:
386378
],
387379
)
388380
def test_serialize_then_deserialize__recovers_original(self, value, py_type):
389-
backend_type = type_helpers.python_type_to_backend_type(py_type)
390381
backend_value = type_helpers.python_value_to_backend_value(value, py_type)
391-
restored = type_helpers.backend_value_to_python_value(
392-
backend_value, backend_type, py_type
393-
)
382+
restored = type_helpers.backend_value_to_python_value(backend_value, py_type)
394383
assert restored == value
395384
assert isinstance(restored, py_type)
396385

0 commit comments

Comments
 (0)