Proper Realsense and ZED Camera Drivers by alexlin2 · Pull Request #935 · dimensionalOS/dimos

alexlin2 · 2026-01-04T05:51:06Z

PR: Add RealSense + ZED camera drivers (Dimos)

Summary

Add a new RealSense camera module with color/depth streaming, camera info publishing, and optional pointcloud generation.
Refactor the ZED camera module to match Dimos camera conventions (config-driven, TF publishing, depth alignment, pointcloud support).
Clean up camera module plumbing and CameraInfo usage to align with Dimos message types.

Key Changes

New RealSense implementation in dimos/hardware/sensors/camera/realsense/camera.py with proper RS pipeline setup, intrinsics/extrinsics, TF frames, and depth alignment.
ZED module rewrite in dimos/hardware/sensors/camera/zed/camera.py to use ZED SDK runtime params, stream dims from camera info, depth scale, TF publishing, and optional pointcloud generation.
Camera module updates in dimos/hardware/sensors/camera/module.py to simplify image publishing, metadata cadence, and passive skill sampling.
CameraInfo now comes from dimos.msgs.sensor_msgs.CameraInfo for consistency across camera hardware.
New GStreamer TCP camera sender/receiver utilities for remote camera streaming under dimos/hardware/sensors/camera/gstreamer/.

Testing

Not run (not requested).

chatgpt-codex-connector · 2026-01-04T05:51:11Z

Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits.
Credits must be used to enable repository wide code reviews.

…oud2

spomichter · 2026-01-05T18:33:26Z

@codex review

dimos/hardware/sensors/camera/realsense/camera.py

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 03439c5136

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

dimos/hardware/sensors/camera/realsense/camera.py

dimos/hardware/sensors/camera/zed/camera.py

spomichter · 2026-01-05T18:37:12Z

dimos/hardware/sensors/camera/realsense/camera.py

+            on_error=lambda e: print(f"CameraInfo error: {e}"),
+        )
+
+        return "started"


Bad thing to return make this some generic sensor status type

spomichter · 2026-01-05T18:42:26Z

dimos/hardware/sensors/camera/realsense/camera.py

+    camera_info_fps: float = 1.0
+
+
+class RealSenseCamera(Module[RealSenseCameraConfig]):


Need abstract depth camera class. That woudl enforce universal depth properties and things like default base transform access

spomichter · 2026-01-05T18:44:29Z

dimos/hardware/sensors/camera/realsense/camera.py

+            # Publish TF
+            self._publish_tf(ts)
+
+    def _publish_tf(self, ts: float) -> None:


@leshy can you review tf approach maybe some way to abstract stuff and make this leaner - may want transform wrapper for all depth cameras that make some of this logic baked in. For example Zed has global coordinate frame init params that we should probably also have as abstract depth camera InitParams: https://www.stereolabs.com/docs/positional-tracking/coordinate-frames

leshy · 2026-01-06T08:59:37Z

dimos/hardware/sensors/camera/spec.py

+from dimos.protocol.service import Configurable
+
+
+class SensorStatus(Enum):


Resource has start and stop, if you failed throw an exception, no need to define this

Yeah, Stash's comment

Yeah well @leshy has a point I said you would need an Enum in place of returning some string status as a raw string. I forgot that we could access this same thing basically via Resource and check state there.

we have a Resource base class, that defines 'start' and 'stop' calls - if they pass it means you started or stopped, if they throw an exception it means you have an error. no need to invent a new language for this just for depth cameras

dimos/hardware/sensors/camera/zed/camera.py

spomichter · 2026-01-05T18:46:55Z

dimos/hardware/sensors/camera/zed/camera.py

+        if err != sl.ERROR_CODE.SUCCESS:
+            print(f"Failed to open ZED camera: {err}")
+            self._zed = None
+            return "failed"


spomichter · 2026-01-06T14:54:37Z

dimos/hardware/sensors/camera/spec.py

+from dimos.protocol.service import Configurable
+
+
+class SensorStatus(Enum):


Yeah well @leshy has a point I said you would need an Enum in place of returning some string status as a raw string. I forgot that we could access this same thing basically via Resource and check state there.

spomichter · 2026-01-06T21:24:11Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: c8699301f3

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

dimos/hardware/sensors/camera/module.py

dimos/hardware/sensors/camera/realsense/camera.py

spomichter · 2026-01-07T01:49:11Z

dimos/hardware/sensors/camera/realsense/camera.py

+    camera_info_fps: float = 1.0
+
+
+class RealSenseCamera(Module[RealSenseCameraConfig]):


spomichter · 2026-01-07T01:49:28Z

dimos/hardware/sensors/camera/spec.py

+StereoCameraConfigT = TypeVar("StereoCameraConfigT", bound=StereoCameraConfig)
+
+
+class StereoCameraHardware(Protocol[StereoCameraConfigT]):


Not being used. Where is the base class

leshy · 2026-01-07T02:04:37Z

shouldn't be in camera but in depthcamera - this is a fundamentally different sensor
depthcamera should inherit camera spec, and build depth on top

can we truly not reuse code between 2 depth cameras? they are totally independant modules

leshy · 2026-01-06T09:00:30Z

dimos/hardware/sensors/camera/spec.py

+
+OPTICAL_ROTATION = Quaternion(-0.5, 0.5, -0.5, 0.5)
+
+
+def default_base_transform() -> Transform:
+    """Default identity transform for camera mounting."""
+    return Transform(
+        translation=Vector3(0.0, 0.0, 0.0),
+        rotation=Quaternion(0.0, 0.0, 0.0, 1.0),
+    )


this doesn't belong in the camera spec, you can construct your default transform as a default field of the config - check how Camera is implemented

dimos/hardware/sensors/camera/spec.py

leshy · 2026-01-06T09:05:01Z

dimos/hardware/sensors/camera/realsense/camera.py

+        self._latest_color_img: Image | None = None
+        self._latest_depth_img: Image | None = None
+        self._pointcloud_lock = threading.Lock()
+        self._pointcloud_disposable: Disposable | None = None


you shouldn't creating separate disposables, you can just use self._disposables.add for cleanup. pls read documentation

leshy · 2026-01-06T09:06:07Z

dimos/hardware/sensors/camera/realsense/camera.py

+            )
+
+        interval_sec = 1.0 / self.config.camera_info_fps
+        self._camera_info_disposable = backpressure(rx.interval(interval_sec)).subscribe(


there is no need to wrap this in backpressure, you need to read reactivex documentation and understand what you are doing

My bad, I had to wrap the pointcloud publisher with backpressure because it is stochastic and processing will depend on number of points in the scene, removed the backpressure

* Added realsense camera driver and added from_rgbd function to pointcloud2 * added rate limiting and backpressure to pointcloud publishing * CI code cleanup * updated ZED module to the same standard as realsense * added changes from another commit for camera * CI code cleanup * added more changes from another commit * added voxel_downsample to pointcloud2 * fixed stash's comments * CI code cleanup * mypy fixes + comments * CI code cleanup * more mypy fixes * removed property of camera_info * fixed abstract class and config * Made changes to all of Ivan and Stash's comments * CI code cleanup * should pass CI now # PR: Add RealSense + ZED camera drivers (Dimos) ## Summary - Add a new RealSense camera module with color/depth streaming, camera info publishing, and optional pointcloud generation. - Refactor the ZED camera module to match Dimos camera conventions (config-driven, TF publishing, depth alignment, pointcloud support). - Clean up camera module plumbing and CameraInfo usage to align with Dimos message types. ## Key Changes - New RealSense implementation in `dimos/hardware/sensors/camera/realsense/camera.py` with proper RS pipeline setup, intrinsics/extrinsics, TF frames, and depth alignment. - ZED module rewrite in `dimos/hardware/sensors/camera/zed/camera.py` to use ZED SDK runtime params, stream dims from camera info, depth scale, TF publishing, and optional pointcloud generation. - Camera module updates in `dimos/hardware/sensors/camera/module.py` to simplify image publishing, metadata cadence, and passive skill sampling. - CameraInfo now comes from `dimos.msgs.sensor_msgs.CameraInfo` for consistency across camera hardware. - New GStreamer TCP camera sender/receiver utilities for remote camera streaming under `dimos/hardware/sensors/camera/gstreamer/`. ## Testing - Not run (not requested).

@spomichter

Release v0.0.6: Pre-Launch Unitree Go2 Release ## What's Changed * Added is_flying_to_target agent skill and fly_to now return string for agent feeback by @spomichter in #635 * Release v0.0.5 by @spomichter in #697 * Rebase ivan g1 by @paul-nechifor in #709 * Navspec by @leshy in #648 * Remove depth module from base unitree go2 blueprints by @spomichter in #712 * Fix Unitree Go2 (replay and spatial memory) by @paul-nechifor in #714 * Add G1 blueprints, and simulation by @paul-nechifor in #724 * New g1 blueprint runfiles by @spomichter in #706 * Update G1/Go2 skills and remove some Robot interfaces by @paul-nechifor in #717 * Add dimos-robot end-to-end test with agents by @paul-nechifor in #716 * Run DimOS and ROS nav in Docker by @paul-nechifor in #700 * Anim experiment by @leshy in #701 * G1 navigation documentation fixes by @spomichter in #738 * Rename dimos-robot to dimos by @paul-nechifor in #740 * Use a process for MuJoCo by @paul-nechifor in #747 * Remove unneeded code files by @paul-nechifor in #718 * Make pygame G1JoystickModule usable for all modules by @paul-nechifor in #741 * error on conflicts by @paul-nechifor in #763 * Hosted Moondream 3 for VLM queries by @alexlin2 in #751 * transport: Remove DaskTransport dead code by @ym-han in #767 * Add editorconfig by @paul-nechifor in #769 * add `type: ignore` by @paul-nechifor in #768 * exclude .md changes from CICD builds by @spomichter in #770 * Working Ivan g1 detection in blueprints by @spomichter in #737 * small env fixes on a fresh install by @leshy in #778 * autofixes by @paul-nechifor in #744 * Support running local agents by @paul-nechifor in #739 * pin major version of langchain packages by @paul-nechifor in #789 * Deduplicate Unitree connections/entrypoints. by @paul-nechifor in #749 * Add TTS and STT by @paul-nechifor in #753 * fix mypy errors by @paul-nechifor in #791 * Use structlog and store JSON logs on disk by @paul-nechifor in #715 * Rpc fixes merge by @paul-nechifor in #801 * transport improvements by @leshy in #713 * Added concurrency check by @spomichter in #803 * make connections work with string annotations by @paul-nechifor in #807 * Run mypy checks in GitHub Actions by @paul-nechifor in #805 * Fix incorrect `= None` by @paul-nechifor in #802 * increase mujoco timeout by @paul-nechifor in #823 * MacOS Support: tests + devShell + mujoco by @jeff-hykin in #745 * nix flake revert by @leshy in #824 * fix mypy issues by @paul-nechifor in #827 * PRODUCTION Nav skills on drone with tracking by @spomichter in #640 * Fix added memory limit to blueprint global config by @spomichter in #856 * models/ refactor by @leshy in #819 * Point Detections by @leshy in #859 * Add generic ignore to gitignore by @jeff-hykin in #864 * fix set transport by @paul-nechifor in #866 * cli-precedence by @paul-nechifor in #857 * show `get_data` progress by @paul-nechifor in #873 * skip if OPENAI_API_KEY not defined by @paul-nechifor in #872 * build foxglove extension by @paul-nechifor in #871 * New planner by @paul-nechifor in #792 * Use `uv` by @paul-nechifor in #870 * Add direnv to gitignore by @Kaweees in #875 * Cuda mapper by @leshy in #862 * rename agents to agents_deprecated by @paul-nechifor in #877 * new planner new mapper by @paul-nechifor in #879 * odom ts parsing by @leshy in #882 * Sim fix by @paul-nechifor in #881 * navigation tuning by @leshy in #883 * Fix: Module init and agents by @leshy in #876 * Remove old setup.sh by @paul-nechifor in #888 * Release planner by @leshy in #887 * fix replay leak by @paul-nechifor in #890 * first pass on large file deletions by @leshy in #891 * Generalized manipulator driver by @mustafab0 in #831 * Restore MacOS Support (flake.nix) by @jeff-hykin in #863 * check-uv by @paul-nechifor in #902 * Make dimos pip-installable by @paul-nechifor in #731 * Revert "Restore MacOS Support (flake.nix)" by @leshy in #907 * jeff flake without py env stuff by @leshy in #911 * remove deprecated docker files by @paul-nechifor in #912 * command center stop and home by @leshy in #893 * use packages by @paul-nechifor in #915 * Fix agents prompt by @paul-nechifor in #914 * fix manifest by @paul-nechifor in #916 * fix move skill by @paul-nechifor in #913 * Ignore individual errors by @paul-nechifor in #919 * Feat/rerun latency panels by @Nabla7 in #917 * WIP Release detections by @leshy in #889 * Remove old navigation modules by @paul-nechifor in #923 * Feat/rerun latency panels by @Nabla7 in #925 * Repair camera module by @leshy in #929 * Repair Stream by @leshy in #932 * Docs Clean by @leshy in #933 * docs: sensor streams by @leshy in #934 * Docs: bugfixes by @leshy in #940 * Fixed doclinks to use git ls by @spomichter in #943 * Examples: third party language interop by @leshy in #946 * DOCS: temporal alignment docs improvements by @leshy in #944 * filter bots from commits by @leshy in #947 * Fix skills by @paul-nechifor in #950 * Limit Rerun viewer memory to 4GB default by @Nabla7 in #949 * Working dimensional MCP server - tested with Claude Code MCP client by @spomichter in #945 * allow registration of different agents by @paul-nechifor in #951 * Pre commit large files by @leshy in #953 * Proper Realsense and ZED Camera Drivers by @alexlin2 in #935 * Granular deps by @leshy in #894 * class VLMAgent(AgentSpec, Module) for streamed VLM queries over Transport by @spomichter in #960 * mac compatible commit filter by @paul-nechifor in #961 ## New Contributors * @ym-han made their first contribution in #767 * @jeff-hykin made their first contribution in #745 * @Kaweees made their first contribution in #875 * @mustafab0 made their first contribution in #831 * @Nabla7 made their first contribution in #917 **Full Changelog**: v0.0.5...v0.0.6

* Added realsense camera driver and added from_rgbd function to pointcloud2 * added rate limiting and backpressure to pointcloud publishing * CI code cleanup * updated ZED module to the same standard as realsense * added changes from another commit for camera * CI code cleanup * added more changes from another commit * added voxel_downsample to pointcloud2 * fixed stash's comments * CI code cleanup * mypy fixes + comments * CI code cleanup * more mypy fixes * removed property of camera_info * fixed abstract class and config * Made changes to all of Ivan and Stash's comments * CI code cleanup * should pass CI now # PR: Add RealSense + ZED camera drivers (Dimos) ## Summary - Add a new RealSense camera module with color/depth streaming, camera info publishing, and optional pointcloud generation. - Refactor the ZED camera module to match Dimos camera conventions (config-driven, TF publishing, depth alignment, pointcloud support). - Clean up camera module plumbing and CameraInfo usage to align with Dimos message types. ## Key Changes - New RealSense implementation in `dimos/hardware/sensors/camera/realsense/camera.py` with proper RS pipeline setup, intrinsics/extrinsics, TF frames, and depth alignment. - ZED module rewrite in `dimos/hardware/sensors/camera/zed/camera.py` to use ZED SDK runtime params, stream dims from camera info, depth scale, TF publishing, and optional pointcloud generation. - Camera module updates in `dimos/hardware/sensors/camera/module.py` to simplify image publishing, metadata cadence, and passive skill sampling. - CameraInfo now comes from `dimos.msgs.sensor_msgs.CameraInfo` for consistency across camera hardware. - New GStreamer TCP camera sender/receiver utilities for remote camera streaming under `dimos/hardware/sensors/camera/gstreamer/`. ## Testing - Not run (not requested). Former-commit-id: 5896df5

@spomichter

Release v0.0.6: Pre-Launch Unitree Go2 Release ## What's Changed * Added is_flying_to_target agent skill and fly_to now return string for agent feeback by @spomichter in #635 * Release v0.0.5 by @spomichter in #697 * Rebase ivan g1 by @paul-nechifor in #709 * Navspec by @leshy in #648 * Remove depth module from base unitree go2 blueprints by @spomichter in #712 * Fix Unitree Go2 (replay and spatial memory) by @paul-nechifor in #714 * Add G1 blueprints, and simulation by @paul-nechifor in #724 * New g1 blueprint runfiles by @spomichter in #706 * Update G1/Go2 skills and remove some Robot interfaces by @paul-nechifor in #717 * Add dimos-robot end-to-end test with agents by @paul-nechifor in #716 * Run DimOS and ROS nav in Docker by @paul-nechifor in #700 * Anim experiment by @leshy in #701 * G1 navigation documentation fixes by @spomichter in #738 * Rename dimos-robot to dimos by @paul-nechifor in #740 * Use a process for MuJoCo by @paul-nechifor in #747 * Remove unneeded code files by @paul-nechifor in #718 * Make pygame G1JoystickModule usable for all modules by @paul-nechifor in #741 * error on conflicts by @paul-nechifor in #763 * Hosted Moondream 3 for VLM queries by @alexlin2 in #751 * transport: Remove DaskTransport dead code by @ym-han in #767 * Add editorconfig by @paul-nechifor in #769 * add `type: ignore` by @paul-nechifor in #768 * exclude .md changes from CICD builds by @spomichter in #770 * Working Ivan g1 detection in blueprints by @spomichter in #737 * small env fixes on a fresh install by @leshy in #778 * autofixes by @paul-nechifor in #744 * Support running local agents by @paul-nechifor in #739 * pin major version of langchain packages by @paul-nechifor in #789 * Deduplicate Unitree connections/entrypoints. by @paul-nechifor in #749 * Add TTS and STT by @paul-nechifor in #753 * fix mypy errors by @paul-nechifor in #791 * Use structlog and store JSON logs on disk by @paul-nechifor in #715 * Rpc fixes merge by @paul-nechifor in #801 * transport improvements by @leshy in #713 * Added concurrency check by @spomichter in #803 * make connections work with string annotations by @paul-nechifor in #807 * Run mypy checks in GitHub Actions by @paul-nechifor in #805 * Fix incorrect `= None` by @paul-nechifor in #802 * increase mujoco timeout by @paul-nechifor in #823 * MacOS Support: tests + devShell + mujoco by @jeff-hykin in #745 * nix flake revert by @leshy in #824 * fix mypy issues by @paul-nechifor in #827 * PRODUCTION Nav skills on drone with tracking by @spomichter in #640 * Fix added memory limit to blueprint global config by @spomichter in #856 * models/ refactor by @leshy in #819 * Point Detections by @leshy in #859 * Add generic ignore to gitignore by @jeff-hykin in #864 * fix set transport by @paul-nechifor in #866 * cli-precedence by @paul-nechifor in #857 * show `get_data` progress by @paul-nechifor in #873 * skip if OPENAI_API_KEY not defined by @paul-nechifor in #872 * build foxglove extension by @paul-nechifor in #871 * New planner by @paul-nechifor in #792 * Use `uv` by @paul-nechifor in #870 * Add direnv to gitignore by @Kaweees in #875 * Cuda mapper by @leshy in #862 * rename agents to agents_deprecated by @paul-nechifor in #877 * new planner new mapper by @paul-nechifor in #879 * odom ts parsing by @leshy in #882 * Sim fix by @paul-nechifor in #881 * navigation tuning by @leshy in #883 * Fix: Module init and agents by @leshy in #876 * Remove old setup.sh by @paul-nechifor in #888 * Release planner by @leshy in #887 * fix replay leak by @paul-nechifor in #890 * first pass on large file deletions by @leshy in #891 * Generalized manipulator driver by @mustafab0 in #831 * Restore MacOS Support (flake.nix) by @jeff-hykin in #863 * check-uv by @paul-nechifor in #902 * Make dimos pip-installable by @paul-nechifor in #731 * Revert "Restore MacOS Support (flake.nix)" by @leshy in #907 * jeff flake without py env stuff by @leshy in #911 * remove deprecated docker files by @paul-nechifor in #912 * command center stop and home by @leshy in #893 * use packages by @paul-nechifor in #915 * Fix agents prompt by @paul-nechifor in #914 * fix manifest by @paul-nechifor in #916 * fix move skill by @paul-nechifor in #913 * Ignore individual errors by @paul-nechifor in #919 * Feat/rerun latency panels by @Nabla7 in #917 * WIP Release detections by @leshy in #889 * Remove old navigation modules by @paul-nechifor in #923 * Feat/rerun latency panels by @Nabla7 in #925 * Repair camera module by @leshy in #929 * Repair Stream by @leshy in #932 * Docs Clean by @leshy in #933 * docs: sensor streams by @leshy in #934 * Docs: bugfixes by @leshy in #940 * Fixed doclinks to use git ls by @spomichter in #943 * Examples: third party language interop by @leshy in #946 * DOCS: temporal alignment docs improvements by @leshy in #944 * filter bots from commits by @leshy in #947 * Fix skills by @paul-nechifor in #950 * Limit Rerun viewer memory to 4GB default by @Nabla7 in #949 * Working dimensional MCP server - tested with Claude Code MCP client by @spomichter in #945 * allow registration of different agents by @paul-nechifor in #951 * Pre commit large files by @leshy in #953 * Proper Realsense and ZED Camera Drivers by @alexlin2 in #935 * Granular deps by @leshy in #894 * class VLMAgent(AgentSpec, Module) for streamed VLM queries over Transport by @spomichter in #960 * mac compatible commit filter by @paul-nechifor in #961 ## New Contributors * @ym-han made their first contribution in #767 * @jeff-hykin made their first contribution in #745 * @Kaweees made their first contribution in #875 * @mustafab0 made their first contribution in #831 * @Nabla7 made their first contribution in #917 **Full Changelog**: v0.0.5...v0.0.6 Former-commit-id: 26e61a70a9469f2e33e51f1296f082b470009c09 [formerly 7ffc878] Former-commit-id: 725e628

* Added realsense camera driver and added from_rgbd function to pointcloud2 * added rate limiting and backpressure to pointcloud publishing * CI code cleanup * updated ZED module to the same standard as realsense * added changes from another commit for camera * CI code cleanup * added more changes from another commit * added voxel_downsample to pointcloud2 * fixed stash's comments * CI code cleanup * mypy fixes + comments * CI code cleanup * more mypy fixes * removed property of camera_info * fixed abstract class and config * Made changes to all of Ivan and Stash's comments * CI code cleanup * should pass CI now # PR: Add RealSense + ZED camera drivers (Dimos) ## Summary - Add a new RealSense camera module with color/depth streaming, camera info publishing, and optional pointcloud generation. - Refactor the ZED camera module to match Dimos camera conventions (config-driven, TF publishing, depth alignment, pointcloud support). - Clean up camera module plumbing and CameraInfo usage to align with Dimos message types. ## Key Changes - New RealSense implementation in `dimos/hardware/sensors/camera/realsense/camera.py` with proper RS pipeline setup, intrinsics/extrinsics, TF frames, and depth alignment. - ZED module rewrite in `dimos/hardware/sensors/camera/zed/camera.py` to use ZED SDK runtime params, stream dims from camera info, depth scale, TF publishing, and optional pointcloud generation. - Camera module updates in `dimos/hardware/sensors/camera/module.py` to simplify image publishing, metadata cadence, and passive skill sampling. - CameraInfo now comes from `dimos.msgs.sensor_msgs.CameraInfo` for consistency across camera hardware. - New GStreamer TCP camera sender/receiver utilities for remote camera streaming under `dimos/hardware/sensors/camera/gstreamer/`. ## Testing - Not run (not requested). Former-commit-id: 25839fe

@spomichter

Release v0.0.6: Pre-Launch Unitree Go2 Release ## What's Changed * Added is_flying_to_target agent skill and fly_to now return string for agent feeback by @spomichter in #635 * Release v0.0.5 by @spomichter in #697 * Rebase ivan g1 by @paul-nechifor in #709 * Navspec by @leshy in #648 * Remove depth module from base unitree go2 blueprints by @spomichter in #712 * Fix Unitree Go2 (replay and spatial memory) by @paul-nechifor in #714 * Add G1 blueprints, and simulation by @paul-nechifor in #724 * New g1 blueprint runfiles by @spomichter in #706 * Update G1/Go2 skills and remove some Robot interfaces by @paul-nechifor in #717 * Add dimos-robot end-to-end test with agents by @paul-nechifor in #716 * Run DimOS and ROS nav in Docker by @paul-nechifor in #700 * Anim experiment by @leshy in #701 * G1 navigation documentation fixes by @spomichter in #738 * Rename dimos-robot to dimos by @paul-nechifor in #740 * Use a process for MuJoCo by @paul-nechifor in #747 * Remove unneeded code files by @paul-nechifor in #718 * Make pygame G1JoystickModule usable for all modules by @paul-nechifor in #741 * error on conflicts by @paul-nechifor in #763 * Hosted Moondream 3 for VLM queries by @alexlin2 in #751 * transport: Remove DaskTransport dead code by @ym-han in #767 * Add editorconfig by @paul-nechifor in #769 * add `type: ignore` by @paul-nechifor in #768 * exclude .md changes from CICD builds by @spomichter in #770 * Working Ivan g1 detection in blueprints by @spomichter in #737 * small env fixes on a fresh install by @leshy in #778 * autofixes by @paul-nechifor in #744 * Support running local agents by @paul-nechifor in #739 * pin major version of langchain packages by @paul-nechifor in #789 * Deduplicate Unitree connections/entrypoints. by @paul-nechifor in #749 * Add TTS and STT by @paul-nechifor in #753 * fix mypy errors by @paul-nechifor in #791 * Use structlog and store JSON logs on disk by @paul-nechifor in #715 * Rpc fixes merge by @paul-nechifor in #801 * transport improvements by @leshy in #713 * Added concurrency check by @spomichter in #803 * make connections work with string annotations by @paul-nechifor in #807 * Run mypy checks in GitHub Actions by @paul-nechifor in #805 * Fix incorrect `= None` by @paul-nechifor in #802 * increase mujoco timeout by @paul-nechifor in #823 * MacOS Support: tests + devShell + mujoco by @jeff-hykin in #745 * nix flake revert by @leshy in #824 * fix mypy issues by @paul-nechifor in #827 * PRODUCTION Nav skills on drone with tracking by @spomichter in #640 * Fix added memory limit to blueprint global config by @spomichter in #856 * models/ refactor by @leshy in #819 * Point Detections by @leshy in #859 * Add generic ignore to gitignore by @jeff-hykin in #864 * fix set transport by @paul-nechifor in #866 * cli-precedence by @paul-nechifor in #857 * show `get_data` progress by @paul-nechifor in #873 * skip if OPENAI_API_KEY not defined by @paul-nechifor in #872 * build foxglove extension by @paul-nechifor in #871 * New planner by @paul-nechifor in #792 * Use `uv` by @paul-nechifor in #870 * Add direnv to gitignore by @Kaweees in #875 * Cuda mapper by @leshy in #862 * rename agents to agents_deprecated by @paul-nechifor in #877 * new planner new mapper by @paul-nechifor in #879 * odom ts parsing by @leshy in #882 * Sim fix by @paul-nechifor in #881 * navigation tuning by @leshy in #883 * Fix: Module init and agents by @leshy in #876 * Remove old setup.sh by @paul-nechifor in #888 * Release planner by @leshy in #887 * fix replay leak by @paul-nechifor in #890 * first pass on large file deletions by @leshy in #891 * Generalized manipulator driver by @mustafab0 in #831 * Restore MacOS Support (flake.nix) by @jeff-hykin in #863 * check-uv by @paul-nechifor in #902 * Make dimos pip-installable by @paul-nechifor in #731 * Revert "Restore MacOS Support (flake.nix)" by @leshy in #907 * jeff flake without py env stuff by @leshy in #911 * remove deprecated docker files by @paul-nechifor in #912 * command center stop and home by @leshy in #893 * use packages by @paul-nechifor in #915 * Fix agents prompt by @paul-nechifor in #914 * fix manifest by @paul-nechifor in #916 * fix move skill by @paul-nechifor in #913 * Ignore individual errors by @paul-nechifor in #919 * Feat/rerun latency panels by @Nabla7 in #917 * WIP Release detections by @leshy in #889 * Remove old navigation modules by @paul-nechifor in #923 * Feat/rerun latency panels by @Nabla7 in #925 * Repair camera module by @leshy in #929 * Repair Stream by @leshy in #932 * Docs Clean by @leshy in #933 * docs: sensor streams by @leshy in #934 * Docs: bugfixes by @leshy in #940 * Fixed doclinks to use git ls by @spomichter in #943 * Examples: third party language interop by @leshy in #946 * DOCS: temporal alignment docs improvements by @leshy in #944 * filter bots from commits by @leshy in #947 * Fix skills by @paul-nechifor in #950 * Limit Rerun viewer memory to 4GB default by @Nabla7 in #949 * Working dimensional MCP server - tested with Claude Code MCP client by @spomichter in #945 * allow registration of different agents by @paul-nechifor in #951 * Pre commit large files by @leshy in #953 * Proper Realsense and ZED Camera Drivers by @alexlin2 in #935 * Granular deps by @leshy in #894 * class VLMAgent(AgentSpec, Module) for streamed VLM queries over Transport by @spomichter in #960 * mac compatible commit filter by @paul-nechifor in #961 ## New Contributors * @ym-han made their first contribution in #767 * @jeff-hykin made their first contribution in #745 * @Kaweees made their first contribution in #875 * @mustafab0 made their first contribution in #831 * @Nabla7 made their first contribution in #917 **Full Changelog**: v0.0.5...v0.0.6 Former-commit-id: 7ffc878

* Added realsense camera driver and added from_rgbd function to pointcloud2 * added rate limiting and backpressure to pointcloud publishing * CI code cleanup * updated ZED module to the same standard as realsense * added changes from another commit for camera * CI code cleanup * added more changes from another commit * added voxel_downsample to pointcloud2 * fixed stash's comments * CI code cleanup * mypy fixes + comments * CI code cleanup * more mypy fixes * removed property of camera_info * fixed abstract class and config * Made changes to all of Ivan and Stash's comments * CI code cleanup * should pass CI now # PR: Add RealSense + ZED camera drivers (Dimos) ## Summary - Add a new RealSense camera module with color/depth streaming, camera info publishing, and optional pointcloud generation. - Refactor the ZED camera module to match Dimos camera conventions (config-driven, TF publishing, depth alignment, pointcloud support). - Clean up camera module plumbing and CameraInfo usage to align with Dimos message types. ## Key Changes - New RealSense implementation in `dimos/hardware/sensors/camera/realsense/camera.py` with proper RS pipeline setup, intrinsics/extrinsics, TF frames, and depth alignment. - ZED module rewrite in `dimos/hardware/sensors/camera/zed/camera.py` to use ZED SDK runtime params, stream dims from camera info, depth scale, TF publishing, and optional pointcloud generation. - Camera module updates in `dimos/hardware/sensors/camera/module.py` to simplify image publishing, metadata cadence, and passive skill sampling. - CameraInfo now comes from `dimos.msgs.sensor_msgs.CameraInfo` for consistency across camera hardware. - New GStreamer TCP camera sender/receiver utilities for remote camera streaming under `dimos/hardware/sensors/camera/gstreamer/`. ## Testing - Not run (not requested). Former-commit-id: 6ae2e0e [formerly 25839fe] Former-commit-id: 68726a8

@spomichter

Release v0.0.6: Pre-Launch Unitree Go2 Release ## What's Changed * Added is_flying_to_target agent skill and fly_to now return string for agent feeback by @spomichter in #635 * Release v0.0.5 by @spomichter in #697 * Rebase ivan g1 by @paul-nechifor in #709 * Navspec by @leshy in #648 * Remove depth module from base unitree go2 blueprints by @spomichter in #712 * Fix Unitree Go2 (replay and spatial memory) by @paul-nechifor in #714 * Add G1 blueprints, and simulation by @paul-nechifor in #724 * New g1 blueprint runfiles by @spomichter in #706 * Update G1/Go2 skills and remove some Robot interfaces by @paul-nechifor in #717 * Add dimos-robot end-to-end test with agents by @paul-nechifor in #716 * Run DimOS and ROS nav in Docker by @paul-nechifor in #700 * Anim experiment by @leshy in #701 * G1 navigation documentation fixes by @spomichter in #738 * Rename dimos-robot to dimos by @paul-nechifor in #740 * Use a process for MuJoCo by @paul-nechifor in #747 * Remove unneeded code files by @paul-nechifor in #718 * Make pygame G1JoystickModule usable for all modules by @paul-nechifor in #741 * error on conflicts by @paul-nechifor in #763 * Hosted Moondream 3 for VLM queries by @alexlin2 in #751 * transport: Remove DaskTransport dead code by @ym-han in #767 * Add editorconfig by @paul-nechifor in #769 * add `type: ignore` by @paul-nechifor in #768 * exclude .md changes from CICD builds by @spomichter in #770 * Working Ivan g1 detection in blueprints by @spomichter in #737 * small env fixes on a fresh install by @leshy in #778 * autofixes by @paul-nechifor in #744 * Support running local agents by @paul-nechifor in #739 * pin major version of langchain packages by @paul-nechifor in #789 * Deduplicate Unitree connections/entrypoints. by @paul-nechifor in #749 * Add TTS and STT by @paul-nechifor in #753 * fix mypy errors by @paul-nechifor in #791 * Use structlog and store JSON logs on disk by @paul-nechifor in #715 * Rpc fixes merge by @paul-nechifor in #801 * transport improvements by @leshy in #713 * Added concurrency check by @spomichter in #803 * make connections work with string annotations by @paul-nechifor in #807 * Run mypy checks in GitHub Actions by @paul-nechifor in #805 * Fix incorrect `= None` by @paul-nechifor in #802 * increase mujoco timeout by @paul-nechifor in #823 * MacOS Support: tests + devShell + mujoco by @jeff-hykin in #745 * nix flake revert by @leshy in #824 * fix mypy issues by @paul-nechifor in #827 * PRODUCTION Nav skills on drone with tracking by @spomichter in #640 * Fix added memory limit to blueprint global config by @spomichter in #856 * models/ refactor by @leshy in #819 * Point Detections by @leshy in #859 * Add generic ignore to gitignore by @jeff-hykin in #864 * fix set transport by @paul-nechifor in #866 * cli-precedence by @paul-nechifor in #857 * show `get_data` progress by @paul-nechifor in #873 * skip if OPENAI_API_KEY not defined by @paul-nechifor in #872 * build foxglove extension by @paul-nechifor in #871 * New planner by @paul-nechifor in #792 * Use `uv` by @paul-nechifor in #870 * Add direnv to gitignore by @Kaweees in #875 * Cuda mapper by @leshy in #862 * rename agents to agents_deprecated by @paul-nechifor in #877 * new planner new mapper by @paul-nechifor in #879 * odom ts parsing by @leshy in #882 * Sim fix by @paul-nechifor in #881 * navigation tuning by @leshy in #883 * Fix: Module init and agents by @leshy in #876 * Remove old setup.sh by @paul-nechifor in #888 * Release planner by @leshy in #887 * fix replay leak by @paul-nechifor in #890 * first pass on large file deletions by @leshy in #891 * Generalized manipulator driver by @mustafab0 in #831 * Restore MacOS Support (flake.nix) by @jeff-hykin in #863 * check-uv by @paul-nechifor in #902 * Make dimos pip-installable by @paul-nechifor in #731 * Revert "Restore MacOS Support (flake.nix)" by @leshy in #907 * jeff flake without py env stuff by @leshy in #911 * remove deprecated docker files by @paul-nechifor in #912 * command center stop and home by @leshy in #893 * use packages by @paul-nechifor in #915 * Fix agents prompt by @paul-nechifor in #914 * fix manifest by @paul-nechifor in #916 * fix move skill by @paul-nechifor in #913 * Ignore individual errors by @paul-nechifor in #919 * Feat/rerun latency panels by @Nabla7 in #917 * WIP Release detections by @leshy in #889 * Remove old navigation modules by @paul-nechifor in #923 * Feat/rerun latency panels by @Nabla7 in #925 * Repair camera module by @leshy in #929 * Repair Stream by @leshy in #932 * Docs Clean by @leshy in #933 * docs: sensor streams by @leshy in #934 * Docs: bugfixes by @leshy in #940 * Fixed doclinks to use git ls by @spomichter in #943 * Examples: third party language interop by @leshy in #946 * DOCS: temporal alignment docs improvements by @leshy in #944 * filter bots from commits by @leshy in #947 * Fix skills by @paul-nechifor in #950 * Limit Rerun viewer memory to 4GB default by @Nabla7 in #949 * Working dimensional MCP server - tested with Claude Code MCP client by @spomichter in #945 * allow registration of different agents by @paul-nechifor in #951 * Pre commit large files by @leshy in #953 * Proper Realsense and ZED Camera Drivers by @alexlin2 in #935 * Granular deps by @leshy in #894 * class VLMAgent(AgentSpec, Module) for streamed VLM queries over Transport by @spomichter in #960 * mac compatible commit filter by @paul-nechifor in #961 ## New Contributors * @ym-han made their first contribution in #767 * @jeff-hykin made their first contribution in #745 * @Kaweees made their first contribution in #875 * @mustafab0 made their first contribution in #831 * @Nabla7 made their first contribution in #917 **Full Changelog**: v0.0.5...v0.0.6 Former-commit-id: 7ffc878 Former-commit-id: 067332a

@spomichter

Release v0.0.6: Pre-Launch Unitree Go2 Release ## What's Changed * Added is_flying_to_target agent skill and fly_to now return string for agent feeback by @spomichter in #635 * Release v0.0.5 by @spomichter in #697 * Rebase ivan g1 by @paul-nechifor in #709 * Navspec by @leshy in #648 * Remove depth module from base unitree go2 blueprints by @spomichter in #712 * Fix Unitree Go2 (replay and spatial memory) by @paul-nechifor in #714 * Add G1 blueprints, and simulation by @paul-nechifor in #724 * New g1 blueprint runfiles by @spomichter in #706 * Update G1/Go2 skills and remove some Robot interfaces by @paul-nechifor in #717 * Add dimos-robot end-to-end test with agents by @paul-nechifor in #716 * Run DimOS and ROS nav in Docker by @paul-nechifor in #700 * Anim experiment by @leshy in #701 * G1 navigation documentation fixes by @spomichter in #738 * Rename dimos-robot to dimos by @paul-nechifor in #740 * Use a process for MuJoCo by @paul-nechifor in #747 * Remove unneeded code files by @paul-nechifor in #718 * Make pygame G1JoystickModule usable for all modules by @paul-nechifor in #741 * error on conflicts by @paul-nechifor in #763 * Hosted Moondream 3 for VLM queries by @alexlin2 in #751 * transport: Remove DaskTransport dead code by @ym-han in #767 * Add editorconfig by @paul-nechifor in #769 * add `type: ignore` by @paul-nechifor in #768 * exclude .md changes from CICD builds by @spomichter in #770 * Working Ivan g1 detection in blueprints by @spomichter in #737 * small env fixes on a fresh install by @leshy in #778 * autofixes by @paul-nechifor in #744 * Support running local agents by @paul-nechifor in #739 * pin major version of langchain packages by @paul-nechifor in #789 * Deduplicate Unitree connections/entrypoints. by @paul-nechifor in #749 * Add TTS and STT by @paul-nechifor in #753 * fix mypy errors by @paul-nechifor in #791 * Use structlog and store JSON logs on disk by @paul-nechifor in #715 * Rpc fixes merge by @paul-nechifor in #801 * transport improvements by @leshy in #713 * Added concurrency check by @spomichter in #803 * make connections work with string annotations by @paul-nechifor in #807 * Run mypy checks in GitHub Actions by @paul-nechifor in #805 * Fix incorrect `= None` by @paul-nechifor in #802 * increase mujoco timeout by @paul-nechifor in #823 * MacOS Support: tests + devShell + mujoco by @jeff-hykin in #745 * nix flake revert by @leshy in #824 * fix mypy issues by @paul-nechifor in #827 * PRODUCTION Nav skills on drone with tracking by @spomichter in #640 * Fix added memory limit to blueprint global config by @spomichter in #856 * models/ refactor by @leshy in #819 * Point Detections by @leshy in #859 * Add generic ignore to gitignore by @jeff-hykin in #864 * fix set transport by @paul-nechifor in #866 * cli-precedence by @paul-nechifor in #857 * show `get_data` progress by @paul-nechifor in #873 * skip if OPENAI_API_KEY not defined by @paul-nechifor in #872 * build foxglove extension by @paul-nechifor in #871 * New planner by @paul-nechifor in #792 * Use `uv` by @paul-nechifor in #870 * Add direnv to gitignore by @Kaweees in #875 * Cuda mapper by @leshy in #862 * rename agents to agents_deprecated by @paul-nechifor in #877 * new planner new mapper by @paul-nechifor in #879 * odom ts parsing by @leshy in #882 * Sim fix by @paul-nechifor in #881 * navigation tuning by @leshy in #883 * Fix: Module init and agents by @leshy in #876 * Remove old setup.sh by @paul-nechifor in #888 * Release planner by @leshy in #887 * fix replay leak by @paul-nechifor in #890 * first pass on large file deletions by @leshy in #891 * Generalized manipulator driver by @mustafab0 in #831 * Restore MacOS Support (flake.nix) by @jeff-hykin in #863 * check-uv by @paul-nechifor in #902 * Make dimos pip-installable by @paul-nechifor in #731 * Revert "Restore MacOS Support (flake.nix)" by @leshy in #907 * jeff flake without py env stuff by @leshy in #911 * remove deprecated docker files by @paul-nechifor in #912 * command center stop and home by @leshy in #893 * use packages by @paul-nechifor in #915 * Fix agents prompt by @paul-nechifor in #914 * fix manifest by @paul-nechifor in #916 * fix move skill by @paul-nechifor in #913 * Ignore individual errors by @paul-nechifor in #919 * Feat/rerun latency panels by @Nabla7 in #917 * WIP Release detections by @leshy in #889 * Remove old navigation modules by @paul-nechifor in #923 * Feat/rerun latency panels by @Nabla7 in #925 * Repair camera module by @leshy in #929 * Repair Stream by @leshy in #932 * Docs Clean by @leshy in #933 * docs: sensor streams by @leshy in #934 * Docs: bugfixes by @leshy in #940 * Fixed doclinks to use git ls by @spomichter in #943 * Examples: third party language interop by @leshy in #946 * DOCS: temporal alignment docs improvements by @leshy in #944 * filter bots from commits by @leshy in #947 * Fix skills by @paul-nechifor in #950 * Limit Rerun viewer memory to 4GB default by @Nabla7 in #949 * Working dimensional MCP server - tested with Claude Code MCP client by @spomichter in #945 * allow registration of different agents by @paul-nechifor in #951 * Pre commit large files by @leshy in #953 * Proper Realsense and ZED Camera Drivers by @alexlin2 in #935 * Granular deps by @leshy in #894 * class VLMAgent(AgentSpec, Module) for streamed VLM queries over Transport by @spomichter in #960 * mac compatible commit filter by @paul-nechifor in #961 ## New Contributors * @ym-han made their first contribution in #767 * @jeff-hykin made their first contribution in #745 * @Kaweees made their first contribution in #875 * @mustafab0 made their first contribution in #831 * @Nabla7 made their first contribution in #917 **Full Changelog**: v0.0.5...v0.0.6 Former-commit-id: 26e61a70a9469f2e33e51f1296f082b470009c09 [formerly 7ffc878] Former-commit-id: 725e628 Former-commit-id: 2e5f1d4

@spomichter

Release v0.0.6: Pre-Launch Unitree Go2 Release ## What's Changed * Added is_flying_to_target agent skill and fly_to now return string for agent feeback by @spomichter in #635 * Release v0.0.5 by @spomichter in #697 * Rebase ivan g1 by @paul-nechifor in #709 * Navspec by @leshy in #648 * Remove depth module from base unitree go2 blueprints by @spomichter in #712 * Fix Unitree Go2 (replay and spatial memory) by @paul-nechifor in #714 * Add G1 blueprints, and simulation by @paul-nechifor in #724 * New g1 blueprint runfiles by @spomichter in #706 * Update G1/Go2 skills and remove some Robot interfaces by @paul-nechifor in #717 * Add dimos-robot end-to-end test with agents by @paul-nechifor in #716 * Run DimOS and ROS nav in Docker by @paul-nechifor in #700 * Anim experiment by @leshy in #701 * G1 navigation documentation fixes by @spomichter in #738 * Rename dimos-robot to dimos by @paul-nechifor in #740 * Use a process for MuJoCo by @paul-nechifor in #747 * Remove unneeded code files by @paul-nechifor in #718 * Make pygame G1JoystickModule usable for all modules by @paul-nechifor in #741 * error on conflicts by @paul-nechifor in #763 * Hosted Moondream 3 for VLM queries by @alexlin2 in #751 * transport: Remove DaskTransport dead code by @ym-han in #767 * Add editorconfig by @paul-nechifor in #769 * add `type: ignore` by @paul-nechifor in #768 * exclude .md changes from CICD builds by @spomichter in #770 * Working Ivan g1 detection in blueprints by @spomichter in #737 * small env fixes on a fresh install by @leshy in #778 * autofixes by @paul-nechifor in #744 * Support running local agents by @paul-nechifor in #739 * pin major version of langchain packages by @paul-nechifor in #789 * Deduplicate Unitree connections/entrypoints. by @paul-nechifor in #749 * Add TTS and STT by @paul-nechifor in #753 * fix mypy errors by @paul-nechifor in #791 * Use structlog and store JSON logs on disk by @paul-nechifor in #715 * Rpc fixes merge by @paul-nechifor in #801 * transport improvements by @leshy in #713 * Added concurrency check by @spomichter in #803 * make connections work with string annotations by @paul-nechifor in #807 * Run mypy checks in GitHub Actions by @paul-nechifor in #805 * Fix incorrect `= None` by @paul-nechifor in #802 * increase mujoco timeout by @paul-nechifor in #823 * MacOS Support: tests + devShell + mujoco by @jeff-hykin in #745 * nix flake revert by @leshy in #824 * fix mypy issues by @paul-nechifor in #827 * PRODUCTION Nav skills on drone with tracking by @spomichter in #640 * Fix added memory limit to blueprint global config by @spomichter in #856 * models/ refactor by @leshy in #819 * Point Detections by @leshy in #859 * Add generic ignore to gitignore by @jeff-hykin in #864 * fix set transport by @paul-nechifor in #866 * cli-precedence by @paul-nechifor in #857 * show `get_data` progress by @paul-nechifor in #873 * skip if OPENAI_API_KEY not defined by @paul-nechifor in #872 * build foxglove extension by @paul-nechifor in #871 * New planner by @paul-nechifor in #792 * Use `uv` by @paul-nechifor in #870 * Add direnv to gitignore by @Kaweees in #875 * Cuda mapper by @leshy in #862 * rename agents to agents_deprecated by @paul-nechifor in #877 * new planner new mapper by @paul-nechifor in #879 * odom ts parsing by @leshy in #882 * Sim fix by @paul-nechifor in #881 * navigation tuning by @leshy in #883 * Fix: Module init and agents by @leshy in #876 * Remove old setup.sh by @paul-nechifor in #888 * Release planner by @leshy in #887 * fix replay leak by @paul-nechifor in #890 * first pass on large file deletions by @leshy in #891 * Generalized manipulator driver by @mustafab0 in #831 * Restore MacOS Support (flake.nix) by @jeff-hykin in #863 * check-uv by @paul-nechifor in #902 * Make dimos pip-installable by @paul-nechifor in #731 * Revert "Restore MacOS Support (flake.nix)" by @leshy in #907 * jeff flake without py env stuff by @leshy in #911 * remove deprecated docker files by @paul-nechifor in #912 * command center stop and home by @leshy in #893 * use packages by @paul-nechifor in #915 * Fix agents prompt by @paul-nechifor in #914 * fix manifest by @paul-nechifor in #916 * fix move skill by @paul-nechifor in #913 * Ignore individual errors by @paul-nechifor in #919 * Feat/rerun latency panels by @Nabla7 in #917 * WIP Release detections by @leshy in #889 * Remove old navigation modules by @paul-nechifor in #923 * Feat/rerun latency panels by @Nabla7 in #925 * Repair camera module by @leshy in #929 * Repair Stream by @leshy in #932 * Docs Clean by @leshy in #933 * docs: sensor streams by @leshy in #934 * Docs: bugfixes by @leshy in #940 * Fixed doclinks to use git ls by @spomichter in #943 * Examples: third party language interop by @leshy in #946 * DOCS: temporal alignment docs improvements by @leshy in #944 * filter bots from commits by @leshy in #947 * Fix skills by @paul-nechifor in #950 * Limit Rerun viewer memory to 4GB default by @Nabla7 in #949 * Working dimensional MCP server - tested with Claude Code MCP client by @spomichter in #945 * allow registration of different agents by @paul-nechifor in #951 * Pre commit large files by @leshy in #953 * Proper Realsense and ZED Camera Drivers by @alexlin2 in #935 * Granular deps by @leshy in #894 * class VLMAgent(AgentSpec, Module) for streamed VLM queries over Transport by @spomichter in #960 * mac compatible commit filter by @paul-nechifor in #961 ## New Contributors * @ym-han made their first contribution in #767 * @jeff-hykin made their first contribution in #745 * @Kaweees made their first contribution in #875 * @mustafab0 made their first contribution in #831 * @Nabla7 made their first contribution in #917 **Full Changelog**: v0.0.5...v0.0.6 Former-commit-id: 7ffc878 Former-commit-id: 067332a

* Added realsense camera driver and added from_rgbd function to pointcloud2 * added rate limiting and backpressure to pointcloud publishing * CI code cleanup * updated ZED module to the same standard as realsense * added changes from another commit for camera * CI code cleanup * added more changes from another commit * added voxel_downsample to pointcloud2 * fixed stash's comments * CI code cleanup * mypy fixes + comments * CI code cleanup * more mypy fixes * removed property of camera_info * fixed abstract class and config * Made changes to all of Ivan and Stash's comments * CI code cleanup * should pass CI now # PR: Add RealSense + ZED camera drivers (Dimos) ## Summary - Add a new RealSense camera module with color/depth streaming, camera info publishing, and optional pointcloud generation. - Refactor the ZED camera module to match Dimos camera conventions (config-driven, TF publishing, depth alignment, pointcloud support). - Clean up camera module plumbing and CameraInfo usage to align with Dimos message types. ## Key Changes - New RealSense implementation in `dimos/hardware/sensors/camera/realsense/camera.py` with proper RS pipeline setup, intrinsics/extrinsics, TF frames, and depth alignment. - ZED module rewrite in `dimos/hardware/sensors/camera/zed/camera.py` to use ZED SDK runtime params, stream dims from camera info, depth scale, TF publishing, and optional pointcloud generation. - Camera module updates in `dimos/hardware/sensors/camera/module.py` to simplify image publishing, metadata cadence, and passive skill sampling. - CameraInfo now comes from `dimos.msgs.sensor_msgs.CameraInfo` for consistency across camera hardware. - New GStreamer TCP camera sender/receiver utilities for remote camera streaming under `dimos/hardware/sensors/camera/gstreamer/`. ## Testing - Not run (not requested). Former-commit-id: 6ae2e0e [formerly 25839fe] Former-commit-id: 68726a8

@spomichter

Release v0.0.6: Pre-Launch Unitree Go2 Release ## What's Changed * Added is_flying_to_target agent skill and fly_to now return string for agent feeback by @spomichter in #635 * Release v0.0.5 by @spomichter in #697 * Rebase ivan g1 by @paul-nechifor in #709 * Navspec by @leshy in #648 * Remove depth module from base unitree go2 blueprints by @spomichter in #712 * Fix Unitree Go2 (replay and spatial memory) by @paul-nechifor in #714 * Add G1 blueprints, and simulation by @paul-nechifor in #724 * New g1 blueprint runfiles by @spomichter in #706 * Update G1/Go2 skills and remove some Robot interfaces by @paul-nechifor in #717 * Add dimos-robot end-to-end test with agents by @paul-nechifor in #716 * Run DimOS and ROS nav in Docker by @paul-nechifor in #700 * Anim experiment by @leshy in #701 * G1 navigation documentation fixes by @spomichter in #738 * Rename dimos-robot to dimos by @paul-nechifor in #740 * Use a process for MuJoCo by @paul-nechifor in #747 * Remove unneeded code files by @paul-nechifor in #718 * Make pygame G1JoystickModule usable for all modules by @paul-nechifor in #741 * error on conflicts by @paul-nechifor in #763 * Hosted Moondream 3 for VLM queries by @alexlin2 in #751 * transport: Remove DaskTransport dead code by @ym-han in #767 * Add editorconfig by @paul-nechifor in #769 * add `type: ignore` by @paul-nechifor in #768 * exclude .md changes from CICD builds by @spomichter in #770 * Working Ivan g1 detection in blueprints by @spomichter in #737 * small env fixes on a fresh install by @leshy in #778 * autofixes by @paul-nechifor in #744 * Support running local agents by @paul-nechifor in #739 * pin major version of langchain packages by @paul-nechifor in #789 * Deduplicate Unitree connections/entrypoints. by @paul-nechifor in #749 * Add TTS and STT by @paul-nechifor in #753 * fix mypy errors by @paul-nechifor in #791 * Use structlog and store JSON logs on disk by @paul-nechifor in #715 * Rpc fixes merge by @paul-nechifor in #801 * transport improvements by @leshy in #713 * Added concurrency check by @spomichter in #803 * make connections work with string annotations by @paul-nechifor in #807 * Run mypy checks in GitHub Actions by @paul-nechifor in #805 * Fix incorrect `= None` by @paul-nechifor in #802 * increase mujoco timeout by @paul-nechifor in #823 * MacOS Support: tests + devShell + mujoco by @jeff-hykin in #745 * nix flake revert by @leshy in #824 * fix mypy issues by @paul-nechifor in #827 * PRODUCTION Nav skills on drone with tracking by @spomichter in #640 * Fix added memory limit to blueprint global config by @spomichter in #856 * models/ refactor by @leshy in #819 * Point Detections by @leshy in #859 * Add generic ignore to gitignore by @jeff-hykin in #864 * fix set transport by @paul-nechifor in #866 * cli-precedence by @paul-nechifor in #857 * show `get_data` progress by @paul-nechifor in #873 * skip if OPENAI_API_KEY not defined by @paul-nechifor in #872 * build foxglove extension by @paul-nechifor in #871 * New planner by @paul-nechifor in #792 * Use `uv` by @paul-nechifor in #870 * Add direnv to gitignore by @Kaweees in #875 * Cuda mapper by @leshy in #862 * rename agents to agents_deprecated by @paul-nechifor in #877 * new planner new mapper by @paul-nechifor in #879 * odom ts parsing by @leshy in #882 * Sim fix by @paul-nechifor in #881 * navigation tuning by @leshy in #883 * Fix: Module init and agents by @leshy in #876 * Remove old setup.sh by @paul-nechifor in #888 * Release planner by @leshy in #887 * fix replay leak by @paul-nechifor in #890 * first pass on large file deletions by @leshy in #891 * Generalized manipulator driver by @mustafab0 in #831 * Restore MacOS Support (flake.nix) by @jeff-hykin in #863 * check-uv by @paul-nechifor in #902 * Make dimos pip-installable by @paul-nechifor in #731 * Revert "Restore MacOS Support (flake.nix)" by @leshy in #907 * jeff flake without py env stuff by @leshy in #911 * remove deprecated docker files by @paul-nechifor in #912 * command center stop and home by @leshy in #893 * use packages by @paul-nechifor in #915 * Fix agents prompt by @paul-nechifor in #914 * fix manifest by @paul-nechifor in #916 * fix move skill by @paul-nechifor in #913 * Ignore individual errors by @paul-nechifor in #919 * Feat/rerun latency panels by @Nabla7 in #917 * WIP Release detections by @leshy in #889 * Remove old navigation modules by @paul-nechifor in #923 * Feat/rerun latency panels by @Nabla7 in #925 * Repair camera module by @leshy in #929 * Repair Stream by @leshy in #932 * Docs Clean by @leshy in #933 * docs: sensor streams by @leshy in #934 * Docs: bugfixes by @leshy in #940 * Fixed doclinks to use git ls by @spomichter in #943 * Examples: third party language interop by @leshy in #946 * DOCS: temporal alignment docs improvements by @leshy in #944 * filter bots from commits by @leshy in #947 * Fix skills by @paul-nechifor in #950 * Limit Rerun viewer memory to 4GB default by @Nabla7 in #949 * Working dimensional MCP server - tested with Claude Code MCP client by @spomichter in #945 * allow registration of different agents by @paul-nechifor in #951 * Pre commit large files by @leshy in #953 * Proper Realsense and ZED Camera Drivers by @alexlin2 in #935 * Granular deps by @leshy in #894 * class VLMAgent(AgentSpec, Module) for streamed VLM queries over Transport by @spomichter in #960 * mac compatible commit filter by @paul-nechifor in #961 ## New Contributors * @ym-han made their first contribution in #767 * @jeff-hykin made their first contribution in #745 * @Kaweees made their first contribution in #875 * @mustafab0 made their first contribution in #831 * @Nabla7 made their first contribution in #917 **Full Changelog**: v0.0.5...v0.0.6 Former-commit-id: 26e61a70a9469f2e33e51f1296f082b470009c09 [formerly 7ffc878] Former-commit-id: 725e628 Former-commit-id: 2e5f1d4

* Added realsense camera driver and added from_rgbd function to pointcloud2 * added rate limiting and backpressure to pointcloud publishing * CI code cleanup * updated ZED module to the same standard as realsense * added changes from another commit for camera * CI code cleanup * added more changes from another commit * added voxel_downsample to pointcloud2 * fixed stash's comments * CI code cleanup * mypy fixes + comments * CI code cleanup * more mypy fixes * removed property of camera_info * fixed abstract class and config * Made changes to all of Ivan and Stash's comments * CI code cleanup * should pass CI now # PR: Add RealSense + ZED camera drivers (Dimos) ## Summary - Add a new RealSense camera module with color/depth streaming, camera info publishing, and optional pointcloud generation. - Refactor the ZED camera module to match Dimos camera conventions (config-driven, TF publishing, depth alignment, pointcloud support). - Clean up camera module plumbing and CameraInfo usage to align with Dimos message types. ## Key Changes - New RealSense implementation in `dimos/hardware/sensors/camera/realsense/camera.py` with proper RS pipeline setup, intrinsics/extrinsics, TF frames, and depth alignment. - ZED module rewrite in `dimos/hardware/sensors/camera/zed/camera.py` to use ZED SDK runtime params, stream dims from camera info, depth scale, TF publishing, and optional pointcloud generation. - Camera module updates in `dimos/hardware/sensors/camera/module.py` to simplify image publishing, metadata cadence, and passive skill sampling. - CameraInfo now comes from `dimos.msgs.sensor_msgs.CameraInfo` for consistency across camera hardware. - New GStreamer TCP camera sender/receiver utilities for remote camera streaming under `dimos/hardware/sensors/camera/gstreamer/`. ## Testing - Not run (not requested). Former-commit-id: 488be56 [formerly 5896df5] Former-commit-id: 66b0d53

alexlin2 changed the base branch from main to dev January 4, 2026 05:51

alexlin2 force-pushed the alexl_realsense_integration branch from e87af86 to 381e5ab Compare January 4, 2026 06:57

alexlin2 added 2 commits January 4, 2026 12:04

Added realsense camera driver and added from_rgbd function to pointcl…

3b8c818

…oud2

added rate limiting and backpressure to pointcloud publishing

e5dd14d

alexlin2 force-pushed the alexl_realsense_integration branch from 381e5ab to e5dd14d Compare January 4, 2026 20:05

alexlin2 requested a review from a team January 4, 2026 20:05

CI code cleanup

0b862fd

alexlin2 changed the title ~~Alexl realsense integration~~ Proper Realsense and ZED Camera Drivers Jan 5, 2026

dimensionalOS deleted a comment from greptile-apps bot Jan 5, 2026

alexlin2 and others added 3 commits January 5, 2026 10:14

updated ZED module to the same standard as realsense

8bdcb7f

added changes from another commit for camera

d3675d1

CI code cleanup

201f273

alexlin2 requested review from paul-nechifor and spomichter January 5, 2026 18:19

added more changes from another commit

03439c5

spomichter reviewed Jan 5, 2026

View reviewed changes

dimos/hardware/sensors/camera/realsense/camera.py Show resolved Hide resolved

chatgpt-codex-connector bot reviewed Jan 5, 2026

View reviewed changes

dimos/hardware/sensors/camera/realsense/camera.py Show resolved Hide resolved

dimos/hardware/sensors/camera/realsense/camera.py Show resolved Hide resolved

dimos/hardware/sensors/camera/zed/camera.py Show resolved Hide resolved

spomichter requested changes Jan 5, 2026

View reviewed changes

alexlin2 and others added 3 commits January 5, 2026 23:43

added voxel_downsample to pointcloud2

27a7c15

fixed stash's comments

c97078d

CI code cleanup

95419c5

leshy reviewed Jan 6, 2026

View reviewed changes

spomichter reviewed Jan 6, 2026

View reviewed changes

alexlin2 and others added 3 commits January 6, 2026 12:11

mypy fixes + comments

765c404

CI code cleanup

1018030

more mypy fixes

c869930

chatgpt-codex-connector bot reviewed Jan 6, 2026

View reviewed changes

dimos/hardware/sensors/camera/module.py Outdated Show resolved Hide resolved

dimos/hardware/sensors/camera/realsense/camera.py Show resolved Hide resolved

removed property of camera_info

5c575d2

spomichter requested changes Jan 7, 2026

View reviewed changes

leshy reviewed Jan 7, 2026

View reviewed changes

alexlin2 and others added 4 commits January 6, 2026 18:32

fixed abstract class and config

e84ec4d

Made changes to all of Ivan and Stash's comments

347251b

CI code cleanup

6fb191d

should pass CI now

df1e274

spomichter approved these changes Jan 7, 2026

View reviewed changes

spomichter merged commit 5896df5 into dev Jan 7, 2026
13 checks passed

spomichter deleted the alexl_realsense_integration branch January 7, 2026 11:29

spomichter mentioned this pull request Jan 7, 2026

Release v0.0.6: Pre-Launch Unitree Go2 Release #958

Merged

greptile-apps bot mentioned this pull request Jan 13, 2026

Tag v0.0.7 #986

Merged

		camera_info_fps: float = 1.0


		class RealSenseCamera(Module[RealSenseCameraConfig]):

		from dimos.protocol.service import Configurable


		class SensorStatus(Enum):

		StereoCameraConfigT = TypeVar("StereoCameraConfigT", bound=StereoCameraConfig)


		class StereoCameraHardware(Protocol[StereoCameraConfigT]):

Conversation

alexlin2 commented Jan 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR: Add RealSense + ZED camera drivers (Dimos)

Summary

Key Changes

Testing

Uh oh!

chatgpt-codex-connector bot commented Jan 4, 2026

Uh oh!

spomichter commented Jan 5, 2026

Uh oh!

Uh oh!

chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

spomichter commented Jan 6, 2026

Uh oh!

chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

leshy commented Jan 7, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

alexlin2 commented Jan 4, 2026 •

edited

Loading