[Remote Store] Keep metadata files for last N refreshes #5455
Description
Currently, we keep one metadata file per commit and the same file (latest metadata file) gets overwritten per refresh. This worked well with commit level durability as failure to read the latest metadata file would mean the next metadata file which would represent the previous commit.
With refresh level durability, reading second to last file (by upload timestamp) will not provide previous refresh details. In fact, we don't keep the refresh history the same way we are keeping commit history with metadata files. This needs to be changed.
Originally posted by @sachinpkale in #5253 (comment)