Workflows

[matthiasfeist]

Right now with AI agents and stuff, workflows with repeatable steps are all the rage.
I was wondering if that's something that might be on the roadmap?

[timgit]

It's not currently on the roadmap, but I've also been interested in this over the years. Basically, you'd build out a graph/flowchart and connect the dots with conditions and different queues, correct?

[timgit]

What do you think about this package? https://github.com/SokratisVidros/pg-workflows

[matthiasfeist]

pg-workflows is pretty much exactly the API that other engines also use.
Another contender is https://dbos.dev

Essentially, these "engines" have sub-queues called steps that where the result of a step is saved in postgres as well so it's available for the next step in the workflow without re-running the previous step.
then there's some sugar on top with delays and waiting-for-events.

I'm currently using Inngest for this and the only reason I do is because they have a very nice UI. I've now seen that pg-boss also comes with a UI which makes me want to refactor my code to use pg-boss 😆

I really think it would be nice to fold a workflow-like function into pg-boss eventually since the developer experience of workflows is actually really nice compared to "only" queues.

[SokratisVidros]

@matthiasfeist You caught that! The pg-workflows API is definitely inspired by Inngest and Trigger.dev. Their DX is great.
However, I’ve found that reducing your stack by one system is a massive win, especially as AI agents gain more autonomy over code and production environments.

Plus, pg-boss is great. We’re running a heavy daily ingestion engine on it right now, and the scalability has been flawless.

[matthiasfeist]

As I wrote, the only reason why I am on Inngest is their UI. I really think that being able to visually see the different runs and steps is a killer feature for any queue or workflow engine.

(OK to be fair, the other reason is that we're running on serverless and I would need to set up a long-living docker container for the background job engine. Not a ton of work, but you know how it is... Further down in the backlog 😅)

[SokratisVidros]

Makes sense. I haven't played with the pg-boss UI much. If not mistaken it was added recently. Currently, we’re just using saved PG queries and materialized views to monitor the queues. Let me install it locally and see if we can piggyback on that and create a richer UI for workflows.