[BUG] Should't BuildConsensus.py be producing always the same result?

[sralchemab]

Describe the bug
I noticed that re-running the exact same BuildConsensus.py command doesn't always produce the same result, shouldn't it be? I'm running it using 4 threads.
In order to double check that the resulting FastA file didn't just differ in the order of the sequences, I first sorted it and format the nucleotide sequences in one long line.

Steps to reproduce
Here's how I'm running the command:

BuildConsensus.py \
    -s SampleA_align-pass.fastq \
    --bf UMI \
    --cf V_PRIMER C_PRIMER BARCODE C_CALL \
    --act majority majority majority majority \
    -n 1 \
    -q 0 \
    --maxerror 0.1 \
    --maxgap 0.5 \
    --fasta \
    --nproc 4 \
    --log "consensus.log" \
    --outname "SampleA"

Here's how I'm preparing the FastA file for comparison:

./bbmap-39.13/sortbyname.sh \
    fastawrap=2000 \
    in=SampleA_collapse-unique.fasta \
    out=SampleA_collapse-unique.sorted.fasta

Environment and versions:

• OS: Linux
• Relevant immcantation packages and versions: presto v0.6.0
• Docker container: quay.io/biocontainers/presto:0.6.0--py_0

[ssnn-airr]

I was going to say yes, it should reproduce the same result, but I will need to check the code to see how ties are dealt with. Can you share the example data to reproduce the behavior? You can send it to immcantation@googlegroups.com.

[sralchemab]

Hi @ssnn-airr. I'm sorry, but I'm not allowed to share this data. I will try to see if I can find a dataset I could share and see if the same issue happens with it.

[sralchemab]

Hi @ssnn-airr. The file was too big to be sent as an attachment, so here's a Box link to it:

https://alchemab.box.com/s/nmw1gypux3ql52f0mxojdzoj79iufg3x

Let me know if you could download it and unpack it properly.

[ssnn-airr]

Received

[ssnn-airr]

Hi! I think the issue was related to the presence of ties and the use of sorted/un-sorted dictionaries by different versions of python (commit 74740a1). Please, try the updated development version of presto and let me know how it goes.

[sralchemab]

Hi @ssnn-airr. I re-ran it 16 times and, although the MD5 of the outputs were different, after sorting the Fasta files, the contents ended up being the same (same MD5 after sorting). So I consider the problem fixed.

Not that important but worth considering, are the contents of the output file different each time because of how batches of sequences end up being joined due to multiprocessing? Would you consider always producing the exact same result or is this out of scope?

Thanks for the quick fix. Would you have to wait some time to release the patch?

[ssnn-airr]

I can make the release today.
Yes, the different order is due to multiprocessing. I will add a feature request to evaluate if we can update the code to produce the same results in a future release.

[sralchemab]

That's great, thanks!