[atlas] Inconsistencies in anchor API and measurements
- Previous message (by thread): [atlas] RIPE measurement Credits required
- Next message (by thread): [atlas] Inconsistencies in anchor API and measurements
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
Malte Appel
malte at iij-ii.co.jp
Mon Nov 15 08:32:44 CET 2021
Hi everyone,
sorry in advance for the long mail.
tl;dr: Anchor API and UI give inconsistent results. Some anchor mesh
measurements could be fixed, some might target non-anchors. Not sure what to do
about it.
I am currently working a lot with RIPE Atlas and recently wanted to use the
anchors and their mesh measurements in particular. I wanted to answer two simple
queries:
1. Get a list of all active anchors
2. Get a list of all active anchor mesh and probe measurements (traceroute,
for my particular use case)
However, while trying to answer these queries, I stumbled upon quite some
inconsistencies, depending on which interface / API is used. As far as I can
tell there are four ways in which one could technically answer query 1:
1. Look at the website: https://atlas.ripe.net/anchors/list/
-> 840 results
2. Query the /anchors API with attribute is_disabled = false:
https://atlas.ripe.net/api/v2/anchors/?is_disabled=false
-> 844 results
3. Query the /probes API with attributes is_anchor = true and status = 1
(Connected):
https://atlas.ripe.net/api/v2/probes/?is_anchor=True&status=1
-> 729 results
4. Query the /probes API with attributes tags = system-anchor and status = 1
(Connected):
https://atlas.ripe.net/api/v2/probes/?tags=system-anchor&status=1
-> 729 results
Methods 3 and 4 are actually consistent!
The main discrepancy is between the /anchors and /probes API: 116 anchors that
are listed on the webpage and/or /anchors API as ‘active’ are inactive (97
abandoned; 19 disconnected at time of writing) according to their respective
/probes entry.
I understand that disconnects might be temporary, but some anchors seem to be
inactive for years (at least according to their status) and are still listed as
active.
I have attached a text file with some notes that go deeper into the differences,
but might be hard to read.
For query 2, I faced a similar situation:
1. Look at the website: https://atlas.ripe.net/anchors/list/full/
Anchoring Mesh IPv4: 840
Anchoring Mesh IPv6: 739
Anchoring Probes IPv4: 840
Anchoring Probes IPv6: 705
Total: 3124
2. Query the /anchor-measurements API
Anchoring Mesh IPv4: 849
Anchoring Mesh IPv6: 743
Anchoring Probes IPv4: 902
Anchoring Probes IPv6: 753
Total: 3247
3. Query the /measurements API with attributes status = 2 (Ongoing),
type = traceroute and corresponding attributes:
Anchoring Mesh IPv4: af = 4; tags=anchoring,mesh
Anchoring Mesh IPv6: af = 6; tags=anchoring,mesh
Anchoring Probes IPv4: af = 4; tags=anchoring,probes
Anchoring Probes IPv6: af = 6; tags=anchoring,probes
For example:
https://atlas.ripe.net/api/v2/measurements/?status=2&type=traceroute&af=6&tags=anchoring,probes
Anchoring Mesh IPv4: 1026
Anchoring Mesh IPv6: 902
Anchoring Probes IPv4: 668
Anchoring Probes IPv6: 619
Total: 3215
These results are even more mixed:
- Tags can be inconsistent: Some measurements have none, some have the
‘probes’ or ’mesh’ tag, but miss the ’anchoring’ tag.
- Some anchors have multiple measurements (especially probes measurements), of
which most actually are run by the same set of probes, i.e., they are
duplicates.
- Which measurements are contained in which of the three result sets is very
mixed, maybe I should draw a Venn diagram :)
Finally, I looked at the consistency of the IP addresses of the /anchors API
(ip_v4 and ip_v6), the /probes API (address_v4, address_v6), the DNS result for
the FQDN of the anchors, and the target IP of the mesh/probes measurements.
I noticed some problems, since our lab (IIJ) also operates an anchor (probe 6425
[0]) and we updated the IP address some time ago, but are actually not reached
by the mesh measurement, because the measurement still targets the old IP.
I attached a CSV that includes the raw data (of measurements with some form of
problem), but basically there are 93 measurements from connected anchors that
fail, and out of which 68 (from 29 anchors) could work, if the measurement would
target the correct IP. These measurements have matching anchor/probe IPs and DNS
records, so I do not know why the measurement target is stale. There are some
additional measurements that could work, but it is unclear what the intended
‘correct’ IP is.
On that note, there are 48 measurements that ‘work‘, i.e., they get a response
from the target, but it is not clear if the target is the intended receiver:
- 8 target abandoned anchors
- 18 have different probe and anchor IPs and target one of them
- 21 have the same probe and anchor IP but target something else
Again, I am sorry for this long mail. I understand that RIPE Atlas is a huge
project that has grown over time so it might be hard to keep some things
synchronized, and some other things might not be easily decidable (e.g., when to
mark an anchor is inactive).
However, I think especially the IP address of an anchor in the /anchors and
/probes APIs, in the DNS entry, and the target of the mesh/probes measurements
need to be consistent. Currently some mesh measurements might target an entirely
different machine.
I wanted to bring some attention to this, but not sure what else I can do as a
user. I don‘t want to complain too much :)
For now I will just use all data sources as input and apply some sanity checks.
Best,
Malte
P.S.: Some feedback on how we can bring the measurement of our anchor to target
our anchor would be nice though.
[0] https://atlas.ripe.net/probes/6425
-------------- next part --------------
A non-text attachment was scrubbed...
Name: wrong_ip_msm.csv
Type: text/csv
Size: 56448 bytes
Desc: not available
URL: <https://lists.ripe.net/ripe/mail/archives/ripe-atlas/attachments/20211115/2e40a63c/attachment.csv>
-------------- next part --------------
Four ways to get list of active anchors:
1. Look at the website: https://atlas.ripe.net/anchors/list/
-> 840 results
2. Query the /anchors API with attribute is_disabled = false: https://atlas.ripe.net/api/v2/anchors/?is_disabled=false
-> 844 results
3. Query the /probes API with attributes is_anchor = true and status = 1 (Connected): https://atlas.ripe.net/api/v2/probes/?is_anchor=True&status=1
-> 729 results
4. Query the /probes API with attributes tags = system-anchor and status = 1 (Connected): https://atlas.ripe.net/api/v2/probes/?tags=system-anchor&status=1
-> 729 results
3 and 4 are consistent!
Unique to 2:
Probe 6288
Hostname: us-pdp-as15133
Anchor ID: 1063
Abandoned since 2020-09-21T23:34:29Z
Both /probes address_v4 field and /anchors ip_v4 field point to 152.195.252.99
Anchoring Mesh ping and traceroute measurements go to 192.229.156.101, which seems to be an old IP according to the changelog:
2018-12-10 15:18 UTC IPv4 address changed from (u'192.229.156.101', u'152.195.252.35') to 152.195.252.35
Both addresses still reply.
Probe 6399
Hostname: nl-ams-as20857
Anchor ID: 1411
Abandoned since 2020-04-28T09:18:55Z
Definitely dead, ping and traceroute not responding.
Unique to 3/4:
Probe 6890
Hostname: ca-van-as395152
Listed as “decommissioned as of 2020-11-09 09:21 UTC”, but still has is_anchor = true
Probes and webpage, but not API:
Probe 7045
Freshly setup anchor, will probably appear in API at some point.
Missing from webpage, but present in API and probes:
Probe 6959
Hostname: at-hit-as34347
Anchor ID: 2762
Active and responding, no idea why not listed.
Probe 6363
Hostname: fr-ysd-as29169
Anchor ID: 1163
Again, IP change to 217.70.176.109, measurements point to 217.70.181.69, not responding there.
Main problem: 116 anchors listed on the webpage and/or /anchors API, but not active in /probes:
Abandoned: 97
16 reachable: 6176, 6204, 6215, 6259, 6288, 6385, 6397, 6459, 6481, 6529, 6555, 6601, 6720, 6735, 6791, 6976
Disconnected: 19
2 actually reachable: 6907, 6928
6907 uses wrong target IP for measurements: Resolved to {103.122.188.7, 103.122.190.141}, uses 103.122.188.7, but should now use 103.122.190.141.
All reachable anchors are listed in the API and website (except 6288; API only)
98 dead anchors.
Getting active anchoring mesh and probes measurements (traceroute only):
1. Look at the website: https://atlas.ripe.net/anchors/list/full/
Anchoring Mesh IPv4: 840
Anchoring Mesh IPv6: 739
Anchoring Probes IPv4: 840
Anchoring Probes IPv6: 705
Total: 3124
2. Query the /anchor-measurements API
Anchoring Mesh IPv4: 849
Anchoring Mesh IPv6: 743
Anchoring Probes IPv4: 902
Anchoring Probes IPv6: 753
Total: 3247
3. Query the /measurements API with attributes status = 2 (Ongoing), type = traceroute and corresponding attributes:
Anchoring Mesh IPv4: af = 4; tags=anchoring,mesh
https://atlas.ripe.net/api/v2/measurements/?status=2&type=traceroute&af=4&tags=anchoring,mesh
Anchoring Mesh IPv6: af = 6; tags=anchoring,mesh
https://atlas.ripe.net/api/v2/measurements/?status=2&type=traceroute&af=6&tags=anchoring,mesh
Anchoring Probes IPv4: af = 4; tags=anchoring,probes
https://atlas.ripe.net/api/v2/measurements/?status=2&type=traceroute&af=4&tags=anchoring,probes
Anchoring Probes IPv6: af = 6; tags=anchoring,probes
https://atlas.ripe.net/api/v2/measurements/?status=2&type=traceroute&af=6&tags=anchoring,probes
Anchoring Mesh IPv4: 1026
Anchoring Mesh IPv6: 902
Anchoring Probes IPv4: 668
Anchoring Probes IPv6: 619
Total: 3215
Unique to 2:
Measurement IDs: 16442292, 16442295, 16442306, 16442309
Mesh IPv4, IPv6, Probes IPv4, IPv6 for nl-ams-as20857.anchors.atlas.ripe.net
Anchor ID: 1411
Probe ID: 6399
Abandoned since 2020-04-28T09:18:55Z
Unique to 3:
82 Measurements
All from 1 are in 2
1 but not 3: 110 Measurements
2 but not 1: 123 Measurements
2 but not 3: 114 Measurements
3 but not 1: 201 Measurements
3 but not 2: 82 Measurements
From 2:
# Tag Consistency
76 measurements without ‘anchoring’ tag, but with ‘probes’ or ‘mesh’ tag
38 measurements without ‘anchoring’, ‘probes’, or ‘mesh’ tag
# Multiple measurements per anchor
84 anchors / 103 measurements with multiple mesh and/or probes measurements.
3 Mesh IPv4
56 Probes IPv4
44 Probes IPv6
82 anchors / 100 measurements with same participating probes.
3 Mesh IPv4
Anchor ID: 2082 Measurement IDs: 24433722, 24433725
Anchor down
Anchor ID: 2214 Measurement IDs: 25155868, 25155871
Anchor down
Anchor ID: 2572 Measurement IDs: 28329447, 28329450
55 Probes IPv4
42 Probes IPv6
2 anchors / 3 measurements with different participating probes.
1 Probes IPv4:
Anchor ID: 2748 Measurement IDs: 29622276, 29622279
2 Probes IPv6:
Anchor ID: 2748 Measurement IDs: 29622287, 29622290
Anchor ID: 2761 Measurement IDs: 30624813, 30624816
# Measurements for non-anchors
Anchoring mesh and probes measurements for two anchors without /anchors API entry, and generally without any entry.
Anchor ID: 1171
Probe ID: 6356
Hostname: au-syd-as23719
Disconnected since 2021-09-02T00:46:30Z
Mesh:
Anchor Measurement ID: 4095
Measurement ID: 12237261
Probes:
Anchor Measurement ID: 4098
Measurement ID: 12237268
Anchor ID: 1600
Probe ID: 6500
Hostname: ir-thr-as8868
Abandoned since 2019-08-27T06:57:40Z
Mesh:
Anchor Measurement ID: 5824
Measurement ID: 19508319
Probes:
Anchor Measurement ID: 5827
Measurement ID: 19508349
Failing Atlas measurements from /anchor-measurements API
Out of 3247 measurements
166 measurements fail
70 Abandoned
6 (from 2 anchors) would work if their target IP would be updated
3 Disconnected
1 (from 1 anchor) would work if the target IP would be updated
93 Connected
68 (from 29 anchors) would work if their target IP would be updated (probe IP == anchor IP == DNS entry)
2 (from 1 anchor) probe == DNS, works; target == anchor, fails
2 (from 1 anchor) probe != anchor != target; DNS == anchor; both probe and anchor would work.
8 (from 3 anchors) probe works; anchor == target == DNS, fails
1 (from 1 anchor) has wrong target; probe == anchor that works, but no DNS entry? *1)
3 (from 1 anchor) probe == anchor, works; dns == target, fails
84 measurements could be ‘resurrected’
9 are borked (none of the IPs respond)
2 (from 1 anchor) target == anchor == dns, fails; probe has no IPv4 anymore
48 work with problems
8 (from 4 anchors) work, even though anchors are Abandoned -> One-way
6 (from 3 anchors) have no DNS entry anymore
2 (from 1 anchor) have a different target IP, but probe == anchor; both work
2 (from 1 anchors) target == probe; dns == anchor; both work
18 (from 8 anchors) have different probe and anchor IPs
2 (from 1 anchor) target probe (for these the anchor IP does not work)
16 (from 7 anchors) target anchor (the probe IP would also work)
2 (from 1 anchor) resolve DNS to probe, i.e., target does not match DNS
21 (from 11 anchors) have the same probe and anchor IP, but target something else
19 (from 10 anchors) target and probe/anchor work
2 (from 1 anchor) target works, but probe/anchor does not
1 (from 1 anchor) has no DNS entry *1) The same. Different targets for Probes (this one; works) and Mesh (the one above; fails)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_0x877CEAFF9EA25E92.asc
Type: application/pgp-keys
Size: 3138 bytes
Desc: OpenPGP public key
URL: <https://lists.ripe.net/ripe/mail/archives/ripe-atlas/attachments/20211115/2e40a63c/attachment.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_signature
Type: application/pgp-signature
Size: 840 bytes
Desc: OpenPGP digital signature
URL: <https://lists.ripe.net/ripe/mail/archives/ripe-atlas/attachments/20211115/2e40a63c/attachment.sig>
- Previous message (by thread): [atlas] RIPE measurement Credits required
- Next message (by thread): [atlas] Inconsistencies in anchor API and measurements
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]