[mat-wg] RIPE NCC measurement data retention
- Previous message (by thread): [mat-wg] RIPE NCC measurement data retention
- Next message (by thread): [mat-wg] The role of aggregators in RIPE Atlas
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
Joshua Levett
joshua.levett at york.ac.uk
Tue Dec 12 16:30:25 CET 2023
Hi, I'm another researcher that uses quite a bit of the historical data held in these services, and I appreciate the commitment to keeping this data available where possible. In the Labs article <https://labs.ripe.net/author/kistel/ripe-ncc-measurement-data-retention-principles/>, there's a statement that: "For the RIPEstat use-case, we make the data available in a variety of ways which takes up about 800 TB of storage space." This reads to me as if there's a lot of (potentially unnecessary?) data duplication. I think proposal 2 therefore sounds sensible - I would imagine that it's possible to reconstruct some of or all of the formats served, so for older data would producing some of these on-the-fly/converting formats be feasible? Is there a way to get a breakdown of what data forms you're using are most storage-intensive, or which parts of services like RIPEstat are using the most storage? I'm imagining that there probably aren't that many use-cases where getting instant access to historic data is needed, so making accessing older data slower/tiered (and hence cheaper) doesn't seem like a problem, but I'm looking at it very much from a research perspective so I could be way off the mark on that. Kind regards, Josh -------------- next part -------------- An HTML attachment was scrubbed... URL: <https://lists.ripe.net/ripe/mail/archives/mat-wg/attachments/20231212/e54af38d/attachment.html>
- Previous message (by thread): [mat-wg] RIPE NCC measurement data retention
- Next message (by thread): [mat-wg] The role of aggregators in RIPE Atlas
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
[ mat-wg Archives ]