Skip to content

Write the progress bar attempt number#219

Merged
folbricht merged 1 commit intofolbricht:masterfrom
RyuzakiKK:explicit_stage_pb
Apr 16, 2022
Merged

Write the progress bar attempt number#219
folbricht merged 1 commit intofolbricht:masterfrom
RyuzakiKK:explicit_stage_pb

Conversation

@RyuzakiKK
Copy link
Copy Markdown
Contributor

@RyuzakiKK RyuzakiKK commented Apr 11, 2022

Currently we have three major steps while extracting a bundle:
validation, seed chunking and assembling.

If one or more seeds are invalid, these steps might be repeated until
Desync is able to make a plan that is correct.

To make it easier to parse and understand what is going on, we prepend
the attempt number to the progress bar description.

In this way, for example, we will have an output like the following:
Attempt 1: Validating 0.00%
Attempt 1: Validating 0.45% 0s
Attempt 2: Chunking Seed 1 0.00%
Attempt 2: Chunking Seed 1 0.27% 00m35s
Attempt 2: Chunking Seed 1 100.00% 12s
Attempt 2: Validating 0.00%
Attempt 2: Validating 30.43% 00m01s
Attempt 2: Validating 100.00% 1s
Attempt 2: Assembling 0.00%
[...]

@RyuzakiKK
Copy link
Copy Markdown
Contributor Author

After I proposed #217, I realized that parsing the current progress status was overly complicated. This is due to the fact that when we are in the "Chunking" and "Assembling" stage, there is no indication of how many times we already tried to validate a plan, or how many seeds we recalculated.

With this PR I added an "Attempt" counter that helps keeping track of the whole process (thanks to a colleague suggestion).

Please let me know if this makes sense or if you'd prefer a different wording.

@evelikov
Copy link
Copy Markdown

One small comment - other than that, I this looks great.

I think the example is a little off - validating is always the first step, isn't it?

Attempt 1: Validating 0.00%
Attempt 1: Validating 0.45% 0s
Attempt 1: Chunking Seed 1 0.00% // this and next couple of lines were "2"
Attempt 1: Chunking Seed 1 0.27% 00m35s
Attempt 1: Chunking Seed 1 100.00% 12s
Attempt 2: Validating 0.00%
...

@RyuzakiKK
Copy link
Copy Markdown
Contributor Author

No, that reflects the current implementation where I increase the attempt number after a failed validation.

The reasoning behind it is that if a validation fails, I considered chunking a seed (i.e. fixing it) already the attempt 2, because our first attempt already failed.

Did you prefer having the chunking in the same attempt as the validation, to keep the validations always as the first step?

@evelikov
Copy link
Copy Markdown

As an outsider (aka someone who doesn't know the desync internals), it makes sense to have each attempt encapsulated in the validate/chunk/assemble order. For example:

Attempt 1: Validate // oops need chunking
Attempt 1: Chunk
Attempt 2: Validate
Attempt 2: Assemble // oops writing has failed due to reasons
Attempt 3: Validate // disk has changed, need chunking
Attempt 3: Chunking
Attempt 4: Validate
Attempt 4: Assemble // everything was successful

At a glance the above matches the actual execution order within desync as well. So it maps nicely AFAICT.

Currently we have three major steps while extracting a bundle:
validation, seed chunking and assembling.

If one or more seeds are invalid, these steps might be repeated until
Desync is able to make a plan that is correct.

To make it easier to parse and understand what is going on, we prepend
the attempt number to the progress bar description.

In this way, for example, we will have an output like the following:
Attempt 1: Validating    0.00%
Attempt 1: Validating    0.45% 0s
Attempt 1: Chunking Seed 1    0.00%
Attempt 1: Chunking Seed 1    0.27% 00m35s
Attempt 1: Chunking Seed 1  100.00% 12s
Attempt 2: Validating    0.00%
Attempt 2: Validating   30.43% 00m01s
Attempt 2: Validating  100.00% 1s
Attempt 2: Assembling    0.00%
[...]

Signed-off-by: Ludovico de Nittis <ludovico.denittis@collabora.com>
@RyuzakiKK
Copy link
Copy Markdown
Contributor Author

Yes, makes sense.

Currently if the Assembling phase fails, the whole operation is aborted. But to be consistent and ensure to handle the eventual future features without having to change again the progress bar, it's probably worth to think about an attempt as the union of validating+chunking+assembling.

I pushed a new commit where now a new attempt always starts with the Validation phase.

@folbricht
Copy link
Copy Markdown
Owner

Thank you

@folbricht folbricht merged commit 32d4018 into folbricht:master Apr 16, 2022
@RyuzakiKK RyuzakiKK deleted the explicit_stage_pb branch April 18, 2022 10:39
JeremieA pushed a commit to InSimo/desync that referenced this pull request Apr 10, 2026
Currently we have three major steps while extracting a bundle:
validation, seed chunking and assembling.

If one or more seeds are invalid, these steps might be repeated until
Desync is able to make a plan that is correct.

To make it easier to parse and understand what is going on, we prepend
the attempt number to the progress bar description.

In this way, for example, we will have an output like the following:
Attempt 1: Validating    0.00%
Attempt 1: Validating    0.45% 0s
Attempt 1: Chunking Seed 1    0.00%
Attempt 1: Chunking Seed 1    0.27% 00m35s
Attempt 1: Chunking Seed 1  100.00% 12s
Attempt 2: Validating    0.00%
Attempt 2: Validating   30.43% 00m01s
Attempt 2: Validating  100.00% 1s
Attempt 2: Assembling    0.00%
[...]

Signed-off-by: Ludovico de Nittis <ludovico.denittis@collabora.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants