bar.Abort should not cause the bar to render if the render delay hasn't expired

[neilotoole]

First, thanks for a wonderful library 🙏

Problem

I create a progress container with a render delay of, say, 5s. I have multiple bars, some of which complete quickly, and some which take a long time. When a bar completes its work, I call bar.Abort(true). However, if Abort is called before the render delay has expired (e.g. in 2s), the bar will, surprisingly, render immediately.

What I expect to happen

I expect Abort to honor the render delay. If a bar completes its work before the render delay expires, the bar should never be rendered.

See full example in gist: https://gist.github.com/neilotoole/0c799d06ef9d47458328bdfd6054790a

[vbauerster]

Should be fixed. If you can test against latest master branch, please do. I didn't test your example thoroughly.

[neilotoole]

@vbauerster Thanks for the quick response. Alas, this does not fix the issue.

I've distilled my example down to the bare minimum.

// require github.com/vbauerster/mpb/v8 v8.7.1-0.20231205062852-da3162c67234
func main() {
	const delay = time.Second * 5
	start := time.Now()
	delayCh := make(chan struct{})
	time.AfterFunc(delay, func() { close(delayCh) })

	p := mpb.New(mpb.WithAutoRefresh(), mpb.WithRenderDelay(delayCh))
	bar := p.New(0, mpb.BarStyle(), mpb.BarRemoveOnComplete())
	bar.Abort(true)
	bar.Wait() // <-- blocks here until render delay completes
	p.Wait()

	fmt.Println("Elapsed: ", time.Since(start))
}

Running this example, you'll see that it blocks on bar.Wait:

$ go install github.com/neilotoole/mpb-render-delay@latest
$ mpb-render-delay
Elapsed:  5.002145041s

BTW, note that that the behavior is the same with or without mpb.WithAutoRefresh.

[vbauerster]

I don't see any wrong behavior here. Render delay is respected correctly, bar is waiting for a reason. If you don't want to wait for a bar don't call bar.Wait().

[neilotoole]

@vbauerster: Even if I skip the bar.Wait() on the aborted bar, this just pushes the block to p.Wait() 👎.

func main() {
	const delay = time.Second * 5
	start := time.Now()
	delayCh := make(chan struct{})
	time.AfterFunc(delay, func() { close(delayCh) })

	p := mpb.New(mpb.WithAutoRefresh(), mpb.WithRenderDelay(delayCh))
	bar := p.New(0, mpb.BarStyle(), mpb.BarRemoveOnComplete())
	bar.Abort(true)
	// bar.Wait()
	p.Wait() // <-- blocks here until render delay completes

	fmt.Println("Elapsed: ", time.Since(start))
}

Why should p.Wait() continue to block on the render delay, if it has no "alive" bars to wait on?

If we run the example without creating any bars:

func main() {
	const delay = time.Second * 5
	start := time.Now()
	delayCh := make(chan struct{})
	time.AfterFunc(delay, func() { close(delayCh) })

	p := mpb.New(mpb.WithAutoRefresh(), mpb.WithRenderDelay(delayCh))
	p.Wait() // <-- Doesn't block, because no "alive" bars.

	fmt.Println("Elapsed: ", time.Since(start))
}

Then this will finish without waiting for the render delay, as expected 👍.

The behavior should be the same if all of p's bars are aborted/completed; p.Wait() should not block (via render delay) on aborted/completed bars.

That is to say, it should be functionally equivalent to invoke p.Wait() on an "empty" p as invoking p.Wait() on a p with no "alive" bars.

[vbauerster]

bar.Wait and p.Wait is blocking for a reason. In order to remove completed/aborted bar at least one render cycle needs to be run. OnComplete/OnAbort decorators demand at least two render cycle to be run. This is core design of this lib and I'm not going to change it.

[neilotoole]

@vbauerster Yes, I looked at the source code, and I understand your reluctance to address the issue. I think it would be a challenging fix.

I will document my understanding of the issue below, in case any other users encounter it.

Problem summary

mbp.WithRenderDelay and mpb.BarRemoveOnComplete interact to produce undesired behavior.

In the scenario I'm working with, some bars will complete before the render delay, and some will complete after the render delay. In both cases, the bar is to be removed. The expectation is that the short-lived bars are never rendered, because they were born and died before the render delay expired. This is not what happens.

Example

Take this example:

$ go install github.com/neilotoole/mpb-render-delay@example-six-seconds
$ mpb-render-delay

This causes a single-frame "flash" of the aborted (already-dead) bars at the point of the render-delay expiration, and then proceeds correctly.

I've pulled this out into a super-slowmo video (in reality, it's just a single frame):

mpb-render-delay-slowmo.mp4

Obviously this isn't the desired outcome.

Next steps

Per @vbauerster's comment above, the interaction between RenderDelay, BarRemoveOnComplete and early-completing bars wasn't anticipated in the rendering pipeline design, and it seems it would be challenging to fix.

However, the exhibited "flash" behavior isn't acceptable in my use case. I suspect I will address the issue by implementing the render delay in an outer layer of decorators (for Progress and Bar) that wraps the mpb library, and avoid using mpb.RenderDelay entirely. The outer decorators will defer creation of the Progress and Bar until the render delay expires; during this time the Bar-decorator will store the IncrBy calls in a variable. When the render delay expires, any non-aborted Bar will be instantiated, and the stored increment value will be played-back on the newly-instantiated Bar.

It's ugly, but it'll probably work. If I do manage to make it work, I'll follow up with a link to it here, for any other users who encounter the issue.

@vbauerster, thank you for taking the time to look at this issue. If you do ever create a v9 or revisit the rendering pipeline, maybe this scenario can be handled? Thanks again for a great library.

[vbauerster]

Thanks for detailed example, it helped me to apply I believe correct fix, at least "flash" behavior is gone when I tested it.
Hope it is acceptable fix for you.

[neilotoole]

@vbauerster I can confirm that the "flash" seen in the example is fixed on master. Thank you. In the coming days I will incorporate master into my main project (which has substantially more complex behavior) and verify that there's no other hidden gremlins, and then I'll close the ticket.

Keep up the great work, and thanks for being so responsive. It's a wonderful project.

[neilotoole]

@vbauerster In my (limited) testing, I haven't found any issues with the fix. Closing 👍