[mat-wg] Changes to the rpki archives on ftp.ripe.net

Dear colleagues,

As you may know, the RIPE NCC publishes daily RPKI dumps for all trust anchors
on ftp.ripe.net/rpki. Today, we have made some changes to this longitudinal
dataset.

We have added:
  * The compressed JSON output of a validator (output.json.xz). We added this
    because it contains validated ASPA and BGPsec payloads.
  * A compressed version of the CSV file of ROAs (roas.csv.xz)
  * A xz compressed archive of the validated and unvalidated objects (repo.tar.xz)

We made this change because it significantly reduces the size of the files. At
this point, we do not plan to re-compress historic files. However, to realise
the savings, we must stop producing the larger version at some point and clean
up the "duplicate" files from the transition period.

For now, we will create the old (roas.csv, repo.tar.gz) and new (roas.csv.xz,
repo.tar.xz) side by side during a transition period. We plan to stop creating
the old format after a transition period of at least a month, taking into
account the feedback we receive. We will inform you when this happens.

Ties de Kock
Specialist Software Engineer
RIPE NCC