[atlas] Planned changes to "measurement archive"
- Previous message (by thread): [atlas] Credit page 404ed?
- Next message (by thread): [atlas] New on RIPE Labs: Rolling Out the RIPE Atlas Redesign
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
Robert Kisteleki
robert at ripe.net
Thu Nov 12 10:24:27 CET 2020
Dear all, This change has been implemented - the measurement archive is created daily now, with reduced content, as described below. Regards, Robert On 2020-09-09 11:48, Robert Kisteleki wrote: > > Dear RIPE Atlas users, > > You may be aware that we publish "measurement metadata dumps" on "FTP" > (https://ftp.ripe.net/ripe/atlas/measurements/) every week. These are > meant to make the complete measurement metadata downloadable for those > who are interested. > > Even though there is still a point in publishing our metadata in a > downloadable format, the current method is not sustainable: as it is > defined it'll grow forever - which is not realistic. > > Currently *every* dump contains *all* the measurements we *ever* had in > the system. We plan to change that to "everything that's running now + > everything that was running in the last two weeks". This has about 200k > items currently. If we published this every day instead of once a week, > we will still only produce 10% of the output we do today - by virtue of > eliminating 90% of redundancy and repeated work. The syntax and > semantics of the content will not change. > > We will keep the existing dumps as they are mostly because there's no > reason to remove them. > > What this means for users is: > * if you care about "current / recent" measurements, you can just use > the last dump from yesterday > * if you really need all (or a large part of) history, you can fetch one > dump per 2 weeks or so and combine them > * you can still use the API for random, finite searches > > We plan to implement the above change just after RIPE 81 (i.e. early > November). Please be aware of this change if you rely on these dumps. > > Regards, > Robert > >
- Previous message (by thread): [atlas] Credit page 404ed?
- Next message (by thread): [atlas] New on RIPE Labs: Rolling Out the RIPE Atlas Redesign
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]