BART included twice

[danieldeutsch]

Hi,

Thanks for this exciting work. Both the dataset and analysis will be very useful!

It looks like BART is listed as both extractive (#8) and abstractive (#13). I believe the files are also identical. It also appears twice in the pickle files with the scores, both under the system name "bart_out".

Is this a mistake or is there a misunderstanding on my part?

Thanks!

[ZhangShiyue]

looks like they have different summaries (one is abstractive, the other is extractive) if you check the two bart files in human_annotations
https://github.com/neulab/REALSumm/blob/master/human_annotations/SCUs_and_annotations/SCU_rating_results_abs_systems/bart_out_Batch_crowd_3.csv
and https://github.com/neulab/REALSumm/blob/master/human_annotations/SCUs_and_annotations/SCU_rating_results_ext_systems/bart_out.csv

However, in the pickle files, they are indeed identical. Looks like they both are the extractive summaries from https://github.com/neulab/REALSumm/blob/master/human_annotations/SCUs_and_annotations/SCU_rating_results_ext_systems/bart_out.csv.

So I guess they might make a mistake when compiling pickle files?