You are here: Home > Participate > Join a Discussion > RIPE Forum
RIPE Forum v1.4.1

RIPE Atlas

Threaded

[atlas] Planned changes to "measurement archive"

User Image

Robert Kisteleki

2020-09-09 11:48:32 CET

RIPE NCC staff member

Dear RIPE Atlas users,

You may be aware that we publish "measurement metadata dumps" on "FTP"
(https://ftp.ripe.net/ripe/atlas/measurements/) every week. These are
meant to make the complete measurement metadata downloadable for those
who are interested.

Even though there is still a point in publishing our metadata in a
downloadable format, the current method is not sustainable: as it is
defined it'll grow forever - which is not realistic.

Currently *every* dump contains *all* the measurements we *ever* had in
the system. We plan to change that to "everything that's running now +
everything that was running in the last two weeks". This has about 200k
items currently. If we published this every day instead of once a week,
we will still only produce 10% of the output we do today - by virtue of
eliminating 90% of redundancy and repeated work. The syntax and
semantics of the content will not change.

We will keep the existing dumps as they are mostly because there's no
reason to remove them.

What this means for users is:
* if you care about "current / recent" measurements, you can just use
the last dump from yesterday
* if you really need all (or a large part of) history, you can fetch one
dump per 2 weeks or so and combine them
* you can still use the API for random, finite searches

We plan to implement the above change just after RIPE 81 (i.e. early
November). Please be aware of this change if you rely on these dumps.

Regards,
Robert