Docs · Report Bug · Roadmap · Get Help · Watch Demo · Free Swag
GlassFlow is an open-source stream processing engine designed for high-volume data ingestion and transformation from Kafka to ClickHouse. GlassFlow comes with the following core functionalities:
- Stateless transformations: Powered by the expr expression engine, enabling flexible data transformations using helper functions and standard operators (e.g., removing null values or replacing missing timestamps).
- Stateful transformations: A built-in state store allows deduplication logic and temporal joins over configurable time windows.
- Filtering: Drop events you don’t want to ingest into ClickHouse before they reach your tables.
- Ingest only: Direct data transfer from Kafka to ClickHouse without transformations.
- Metrics & OTEL: Built-in pipeline metrics with OpenTelemetry support.
- Dead-Letter-Queue: Keep pipelines running when faulty events occur. Inspect failed events and reprocess them later.
To get started with GlassFlow, you can:
- Try the Live Demo: Experience GlassFlow running on a live cluster at demo.glassflow.dev
- Install on Kubernetes: Follow our Kubernetes Installation Guide for production deployment
- Learn More: Explore our Usage Guide to start creating pipelines
GlassFlow is open source and can be self-hosted on Kubernetes. GlassFlow works with any managed Kubernetes services like AWS EKS, GKE, AKS, and more.
| Method | Use Case | Docs Link |
|---|---|---|
| ☸️ Kubernetes with Helm | Production and development deployment | Kubernetes Helm Guide |
Log in and see a working demo of GlassFlow running on a GPC cluster at demo.glassflow.dev. You will see a Grafana dashboard and the setup that we used.
For detailed documentation, visit docs.glassflow.dev. The documentation includes:
Check out our public roadmap to see what's coming next in GlassFlow. We're actively working on new features and improvements based on community feedback.
Want to suggest a feature? We'd love to hear from you! Please use our GitHub Discussions to share your ideas and help shape the future of GlassFlow.
This project is licensed under the Apache License 2.0.