From robert at ripe.net Tue Oct 1 14:55:28 2013 From: robert at ripe.net (Robert Kisteleki) Date: Tue, 01 Oct 2013 14:55:28 +0200 Subject: [atlas] Probe uptime reporting issues Message-ID: <524AC640.50601@ripe.net> Dear All, On 21 September we deployed a new software version on our core infrastructure components. Unfortunately this contained a bug that we didn't catch in time, which caused "probe uptime reports" not to be propagated properly. This affected the "monthly probe uptime reports" as well as -- in some cases -- the amount of credits received for a probe. We've fixed the bug and re-sent the missing uptime entries. Please let us know (in an email to atlas at ripe.net, mentioning your email address and probe ID) if you received less credits than you should have, and would like to receive compensation. In order to improve our process, we're working on an internal test suite that will help us catching these kind of errors in time. We aplogise for the inconvenience this issue may have casued. Regards, Robert Kisteleki for the RIPE Atlas team From maxtul at netassist.ua Tue Oct 1 17:05:00 2013 From: maxtul at netassist.ua (Max Tulyev) Date: Tue, 01 Oct 2013 18:05:00 +0300 Subject: [atlas] HTTP/HTTPS probe Message-ID: <524AE49C.4080005@netassist.ua> Hi All, What about HTTP/HTTPS requests from Atlas probes? I hear it is technically possible. This test can provide statistics about quality and availability of certain websites, as well as the information about content is delivered unmodified, not blocked, correctly/incorrectly cached, etc. Is it possible to enable that kind of probes? From folkert.van.heusden at gmail.com Tue Oct 1 19:09:52 2013 From: folkert.van.heusden at gmail.com (Folkert van Heusden) Date: Tue, 1 Oct 2013 19:09:52 +0200 Subject: [atlas] HTTP/HTTPS probe In-Reply-To: <524AE49C.4080005@netassist.ua> References: <524AE49C.4080005@netassist.ua> Message-ID: Httping! Op 1 okt. 2013 17:27 schreef "Max Tulyev" het volgende: > Hi All, > > What about HTTP/HTTPS requests from Atlas probes? I hear it is > technically possible. > > This test can provide statistics about quality and availability of > certain websites, as well as the information about content is delivered > unmodified, not blocked, correctly/incorrectly cached, etc. > > Is it possible to enable that kind of probes? > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From robert at ripe.net Tue Oct 1 20:01:28 2013 From: robert at ripe.net (Robert Kisteleki) Date: Tue, 01 Oct 2013 20:01:28 +0200 Subject: [atlas] HTTP/HTTPS probe In-Reply-To: <524AE49C.4080005@netassist.ua> References: <524AE49C.4080005@netassist.ua> Message-ID: <524B0DF8.4000409@ripe.net> On 2013.10.01. 17:05, Max Tulyev wrote: > Hi All, > > What about HTTP/HTTPS requests from Atlas probes? I hear it is > technically possible. > > This test can provide statistics about quality and availability of > certain websites, as well as the information about content is delivered > unmodified, not blocked, correctly/incorrectly cached, etc. > > Is it possible to enable that kind of probes? Hi, This topic has surfaced a couple of times already (in this list: Aug 2012, July 2013), probably on other related lists too. Bottom line: * indeed, technically it is possible to do HTTP requests, and every now and then we're doing tests ourselves or with external researchers * public release needs community discussion, as this feature can have dire consequences to probe hosts (especially in terms of bandwidth usage and privacy consequences) * HTTPS is a different question: the "SSL get certificate" measurement is available to all users See these threads: https://www.ripe.net/ripe/mail/archives/ripe-atlas/2012-August/000552.html https://www.ripe.net/ripe/mail/archives/ripe-atlas/2013-July/000929.html Regards, Robert From bortzmeyer at nic.fr Wed Oct 2 10:13:43 2013 From: bortzmeyer at nic.fr (Stephane Bortzmeyer) Date: Wed, 2 Oct 2013 10:13:43 +0200 Subject: [atlas] HTTP/HTTPS probe In-Reply-To: <524B0DF8.4000409@ripe.net> References: <524AE49C.4080005@netassist.ua> <524B0DF8.4000409@ripe.net> Message-ID: <20131002081343.GA31298@nic.fr> On Tue, Oct 01, 2013 at 08:01:28PM +0200, Robert Kisteleki wrote a message of 33 lines which said: > This topic has surfaced a couple of times already (in this list: Aug > 2012, July 2013), probably on other related lists too. It would surface less often if the doc were fixed . From robert at ripe.net Wed Oct 2 10:42:54 2013 From: robert at ripe.net (Robert Kisteleki) Date: Wed, 02 Oct 2013 10:42:54 +0200 Subject: [atlas] HTTP/HTTPS probe In-Reply-To: <20131002081343.GA31298@nic.fr> References: <524AE49C.4080005@netassist.ua> <524B0DF8.4000409@ripe.net> <20131002081343.GA31298@nic.fr> Message-ID: <524BDC8E.8080907@ripe.net> On 2013.10.02. 10:13, Stephane Bortzmeyer wrote: > On Tue, Oct 01, 2013 at 08:01:28PM +0200, > Robert Kisteleki wrote > a message of 33 lines which said: > >> This topic has surfaced a couple of times already (in this list: Aug >> 2012, July 2013), probably on other related lists too. > > It would surface less often if the doc were fixed > . We updated the doc right after the last time you asked :-) "Important note: HTTP(6) measurements are not available to all UDM users; they are restricted while the potential impact is evaluated." Regards, Robert From kix at kix.es Wed Oct 2 11:21:37 2013 From: kix at kix.es (Rodolfo =?utf-8?b?R2FyY8OtYSA=?= =?utf-8?b?UGXDsWFz?= (kix)) Date: Wed, 02 Oct 2013 09:21:37 +0000 Subject: [atlas] HTTP/HTTPS probe In-Reply-To: <524BDC8E.8080907@ripe.net> References: <524AE49C.4080005@netassist.ua> <524B0DF8.4000409@ripe.net> <20131002081343.GA31298@nic.fr> <524BDC8E.8080907@ripe.net> Message-ID: <20131002092137.Horde.vjUFJNi2YDSy8Gl2w01atw2@mail.kix.es> Robert Kisteleki escribi?: > On 2013.10.02. 10:13, Stephane Bortzmeyer wrote: >> On Tue, Oct 01, 2013 at 08:01:28PM +0200, >> Robert Kisteleki wrote >> a message of 33 lines which said: >> >>> This topic has surfaced a couple of times already (in this list: Aug >>> 2012, July 2013), probably on other related lists too. >> >> It would surface less often if the doc were fixed >> . > > We updated the doc right after the last time you asked :-) > > "Important note: HTTP(6) measurements are not available to all UDM users; > they are restricted while the potential impact is evaluated." Hi, my proposal: - Two different checks in the probe configuration: - http:// small file download - http:// big file download Then, the user can select if their probe will download big or small files (bw impact). - Fixed targets. The probe can connect only selected hosts and files (privacy impact). IMO, the targets should be non commercial sites, with good bw. Regards, kix > Regards, > Robert Rodolfo Garc?a Pe?as (kix) http://www.kix.es/ From maxtul at netassist.ua Wed Oct 2 14:51:17 2013 From: maxtul at netassist.ua (Max Tulyev) Date: Wed, 02 Oct 2013 15:51:17 +0300 Subject: [atlas] HTTP/HTTPS probe In-Reply-To: <524BDC8E.8080907@ripe.net> References: <524AE49C.4080005@netassist.ua> <524B0DF8.4000409@ripe.net> <20131002081343.GA31298@nic.fr> <524BDC8E.8080907@ripe.net> Message-ID: <524C16C5.5020406@netassist.ua> So if I have the http measurement could be globally interested, what is the algorythm to realize it? On 02.10.13 11:42, Robert Kisteleki wrote: > On 2013.10.02. 10:13, Stephane Bortzmeyer wrote: >> On Tue, Oct 01, 2013 at 08:01:28PM +0200, >> Robert Kisteleki wrote >> a message of 33 lines which said: >> >>> This topic has surfaced a couple of times already (in this list: Aug >>> 2012, July 2013), probably on other related lists too. >> >> It would surface less often if the doc were fixed >> . > > We updated the doc right after the last time you asked :-) > > "Important note: HTTP(6) measurements are not available to all UDM users; > they are restricted while the potential impact is evaluated." > > Regards, > Robert > > From rlb at ipv.sx Wed Oct 2 20:13:11 2013 From: rlb at ipv.sx (Richard Barnes) Date: Wed, 2 Oct 2013 14:13:11 -0400 Subject: [atlas] HTTP/HTTPS probe In-Reply-To: <20131002092137.Horde.vjUFJNi2YDSy8Gl2w01atw2@mail.kix.es> References: <524AE49C.4080005@netassist.ua> <524B0DF8.4000409@ripe.net> <20131002081343.GA31298@nic.fr> <524BDC8E.8080907@ripe.net> <20131002092137.Horde.vjUFJNi2YDSy8Gl2w01atw2@mail.kix.es> Message-ID: I was one of the people that urged caution with regard to HTTP probes. There's some ambiguity that needs to be resolved first, and then some security risks to consider. The basic question is "what is being measured"? There are several possible answers: 1. Transport-layer connectivity 2. HTTP header information 3. HTTP body information Each of these has some risks that would beed to be controlled. (1) is obviously the safest; that's part of what the SSL cert measurement does. But (1) is not really an HTTP measurement, it's a TCP measurement, and it would be better to cast it that way. (2) could probably be implemented pretty safely by sending a HEAD request. However, there's still a risk that private user information would leak in such requests. For example, if a web site is doing IP-address based access control, and the probe is behind the same NAT as a user's laptop, then even a HEAD request might return user information (e.g., session cookies). (3) is a huge security risk, because of the wide variety of things that are done with HTTP requests. For simplicity, let's assume the probe would send a GET request, and not anything more sophisticated (POST, PUT, DELETE, etc.). You could use a GET request to download a file, but you can also a GET request to do things to supply responses to HTTP forms. Want to make sure your favorite band wins the EuroVision Song Contest? Just task the Atlas network have 1000 probes vote for them every 5 minutes. There's also the question of what you do with the downloaded content. Returning it to the measurement owner would raise huge security issues, not to mention bandwidth issues. But if you don't return it, then the system will need to constrain the questions the experimenter can ask, e.g., "How many bytes were received?" "What was the SHA-1 digest of the file?". --Richard On Wed, Oct 2, 2013 at 5:21 AM, Rodolfo Garc?a Pe?as (kix) wrote: > > Robert Kisteleki escribi?: > > > On 2013.10.02. 10:13, Stephane Bortzmeyer wrote: >> >>> On Tue, Oct 01, 2013 at 08:01:28PM +0200, >>> Robert Kisteleki wrote >>> a message of 33 lines which said: >>> >>> This topic has surfaced a couple of times already (in this list: Aug >>>> 2012, July 2013), probably on other related lists too. >>>> >>> >>> It would surface less often if the doc were fixed >>> >>> **>. >>> >> >> We updated the doc right after the last time you asked :-) >> >> "Important note: HTTP(6) measurements are not available to all UDM users; >> they are restricted while the potential impact is evaluated." >> > > Hi, > > my proposal: > > - Two different checks in the probe configuration: > - http:// small file download > - http:// big file download > Then, the user can select if their probe will download big or small > files (bw impact). > - Fixed targets. The probe can connect only selected hosts and files > (privacy impact). IMO, the targets should be non commercial sites, with > good bw. > > Regards, > kix > > Regards, >> Robert >> > > > Rodolfo Garc?a Pe?as (kix) > http://www.kix.es/ > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From klaus.mailinglists at pernau.at Wed Oct 2 20:40:25 2013 From: klaus.mailinglists at pernau.at (Klaus Darilion) Date: Wed, 02 Oct 2013 20:40:25 +0200 Subject: [atlas] Problems deleting measurements Message-ID: <524C6899.3000105@pernau.at> Hi! I have problems with lots of measurements, I can not delete them. e.g: A) https://atlas.ripe.net/api/v1/measurement/1027747/ The status is "Failed" and in the web I can not Archive or Stop it. Also deletion via REST fails: curl -H 'Accept: application/json' -X DELETE https://atlas.ripe.net/api/v1/measurement/1027747/?key=01....6d {"error": {"code": 200, "message": "That measurement cannot be stopped."}} btw: is above REST call correct? Same problem for measurment 1012558 which is in status Stopped. B) Is "archive" in the web interface identical to "DELETE" via REST? E.g. the web interface offers me "Archive" for measurement 1029338 (and I am quite sure it would work), but the REST call fails with same result: curl -H 'Accept: application/json' -X DELETE https://atlas.ripe.net/api/v1/measurement/1029338/?key=01....d {"error": {"code": 200, "message": "That measurement cannot be stopped."}} Further, the web interface offers "Stop" and "Archive", but the REST call only offers "DELETE". So, how do I "Stop" via REST? For me it is confusing to have different terms in the web interface and REST API. C) 1031725: The measurement is currently (UTC 18:30) in status Ongoing. Start Time is UTC 19:45, Stop Time is 19:55. When I stop the measurement, the measurements stop time changes to the current time, but status is still Ongoing, and I can not Archive (not offered), but it still offers always Stop. But when I delete via REST, the measurement gets stopped (status Stopped in web interface). So does REST-DELETE is identical to "Stop" in web interface? If yes, how do I archive via REST? Further, DELETE a stopped message should not produce error "That measurement cannot be stopped.", but rather "That measurement is already stopped." D) For the web interface it would be cool to stop/archive/delete measurements more comfortable. Eg. select multiple measurements and then choose "stop and archive". Thanks Klaus Thanks Klaus From dquinn at ripe.net Thu Oct 3 00:45:56 2013 From: dquinn at ripe.net (Daniel Quinn) Date: Thu, 03 Oct 2013 00:45:56 +0200 Subject: [atlas] Problems deleting measurements In-Reply-To: <524C6899.3000105@pernau.at> References: <524C6899.3000105@pernau.at> Message-ID: <524CA224.70008@ripe.net> You've asked for a lot here, and for most of it, the best I can offer you is: "I understand (and even share) your concerns, but what you're asking for takes a lot longer to implement than you may like. However, we are working on clarifying the interface in this area." Now, regarding some of your confusion, I'll put it all in point-form to keep it simple: * There is no way to delete a measurement. Not via the website, and not via the REST API. * As REST offers 4 basic verbs for interface: GET, PUT, POST, and DELETE, we opted to use DELETE as the "STOP" method, as we have no intention of ever actually deleting measurements. * The concept of archiving measurements is being re-worked and may disappear only to be re-used later as a means of hiding the measurements from your list. We're still working on what the Right Way to handle this should be. I hope that clears things up for you. From klaus.mailinglists at pernau.at Thu Oct 3 10:19:26 2013 From: klaus.mailinglists at pernau.at (Klaus Darilion) Date: Thu, 03 Oct 2013 10:19:26 +0200 Subject: [atlas] Problems deleting measurements In-Reply-To: <524CA224.70008@ripe.net> References: <524C6899.3000105@pernau.at> <524CA224.70008@ripe.net> Message-ID: <524D288E.1020509@pernau.at> Hi Daniel! Thanks for the info. IMO https://atlas.ripe.net/doc/rest -> last paragraph: "Deleting a Measurement" should be renamed and extended with the text below. And as I said, it would be nice to have an easy way to remove the measurements from the web interface (e.g. marking measurements with the "hide" flag, and in the web interface one can choose to see the normal view or also "show also hidden measurements"). This should be possible with REST too (setting/unsetting the "hide measurement" flag). Thanks Klaus On 03.10.2013 00:45, Daniel Quinn wrote: > You've asked for a lot here, and for most of it, the best I can offer > you is: "I understand (and even share) your concerns, but what you're > asking for takes a lot longer to implement than you may like. However, > we are working on clarifying the interface in this area." > > Now, regarding some of your confusion, I'll put it all in point-form to > keep it simple: > > * There is no way to delete a measurement. Not via the website, and > not via the REST API. > * As REST offers 4 basic verbs for interface: GET, PUT, POST, and > DELETE, we opted to use DELETE as the "STOP" method, as we have no > intention of ever actually deleting measurements. > * The concept of archiving measurements is being re-worked and may > disappear only to be re-used later as a means of hiding the > measurements from your list. We're still working on what the Right Way > to handle this should be. > > I hope that clears things up for you. > From maxtul at netassist.ua Thu Oct 3 11:21:12 2013 From: maxtul at netassist.ua (Max Tulyev) Date: Thu, 03 Oct 2013 12:21:12 +0300 Subject: [atlas] Measurement of impact of the Russian censorship Message-ID: <524D3708.3060101@netassist.ua> Hello Atlas, recently I found some of Websites was blocked in Ukraine with PutinWall outside Russia. These resources was not hosted in Russia as well. Further investigation shows that the trace to these resources are either through Russia, or (which is more interesting) through Russian-based carriers which are do filtering on their networks boundary which are outside Russia as well, for example, in Frankfurt. This is some impaced nets in Ukraine, but probably countries whole connected through Russia, like Kazakhstan, are more suffering. Is it possible to use Atlas probes to figure out impact of PutinWall to countries should not be impacted by Russian censorship? And may be the efficient of filtering inside Russia? That's the sample of censored URL: http://stervozzinka.dreamwidth.org/15580.html The task is to probe it from Atlas probes and put on map the mark is it accessible or not for the first result. The second stage is to figure out carriers do the filtering of trans-Russia connectivity. I believe it will be interesting for a lot of people. I see this test is possible, but possible only manually by the RIPE NCC staff. From BECHA at ripe.net Thu Oct 3 13:36:36 2013 From: BECHA at ripe.net (Vesna Manojlovic) Date: Thu, 03 Oct 2013 13:36:36 +0200 Subject: [atlas] Announcing RIPE Atlas Service Terms Documentation for Anchor Hosts In-Reply-To: <524D4F8D.1010003@ripe.net> References: <524D4F8D.1010003@ripe.net> Message-ID: <524D56C4.5050006@ripe.net> Dear colleagues, In the process of transitioning RIPE Atlas anchors towards being a production service, we have developed two documents that describe the service terms: - Draft RIPE Atlas Service Terms and Conditions - Memorandum of Understanding - Hosting a RIPE Atlas Anchor Probe Please see the details in this RIPE Labs article: https://labs.ripe.net/Members/becha/ripe-atlas-anchors-terms-and-conditions If you are currently hosting an anchor, or if you are interested in hosting a RIPE Atlas anchor in the future, we would like to hear any feedback you have on these draft documents. Our goal is to finalise the documents by 11 October so that we can announce the production service at the upcoming RIPE 67 Meeting in Athens. Regards, Vesna -- Vesna Manojlovic BECHA at ripe.net Senior Community Builder @Ms_Measurements for Measurements Tools RIPE NCC http://ripe.net From klaus.mailinglists at pernau.at Fri Oct 4 11:50:11 2013 From: klaus.mailinglists at pernau.at (Klaus Darilion) Date: Fri, 04 Oct 2013 11:50:11 +0200 Subject: [atlas] Question about rt definition of DNS measurements Message-ID: <524E8F53.1040000@pernau.at> Hi! When doing DNS measurements I see that the average "rt" time is considerable higher when setting the DO bit. When I check my name servers manually with dig (with and without DO) I do not see any considerable differences. Thus, I wonder how "rt" is measured in the Atlas probes and if the probe does some DNS/DNSSEC relevant processing during the "rt" interval, or is it really just the time between sending and receiving of the packets on IP layer? Thanks Klaus From bortzmeyer at nic.fr Sun Oct 6 17:44:09 2013 From: bortzmeyer at nic.fr (Stephane Bortzmeyer) Date: Sun, 6 Oct 2013 08:44:09 -0700 Subject: [atlas] Question about rt definition of DNS measurements In-Reply-To: <524E8F53.1040000@pernau.at> References: <524E8F53.1040000@pernau.at> Message-ID: <20131006154409.GA21884@laperouse.bortzmeyer.org> On Fri, Oct 04, 2013 at 11:50:11AM +0200, Klaus Darilion wrote a message of 14 lines which said: > When doing DNS measurements I see that the average "rt" time is > considerable higher when setting the DO bit. I cannot reproduce it. Are you sure it isn't a caching effect? I tried on a hot cache (using the same set of probes for three measurements, one with the DO bit to warm the cache, then one without the DO bit and one with, both using a hot cache). Compare public measurements #1032220 and #1032221, the one with the DO bit is even a bit faster: Without DO : Measurement #1032220 for www.afnic.fr uses 488 probes 467 probes, 432 got a reply, 15 replies unparseable, 0 probes were apparently using a validator Average RT is 66.67 ms Test done at 2013-10-06T15:36:25Z With DO : Measurement #1032221 for www.afnic.fr uses 489 probes 478 probes, 424 got a reply, 18 replies unparseable, 144 probes were apparently using a validator Average RT is 65.26 ms Test done at 2013-10-06T15:40:57Z From robert at ripe.net Mon Oct 7 12:29:40 2013 From: robert at ripe.net (Robert Kisteleki) Date: Mon, 07 Oct 2013 12:29:40 +0200 Subject: [atlas] Problems with one-off measurements In-Reply-To: <523AF3A0.90603@ripe.net> References: <523AF3A0.90603@ripe.net> Message-ID: <52528D14.9050703@ripe.net> On 2013.09.19. 14:52, Robert Kisteleki wrote: > Dear RIPE Atlas users, > > It has come to our attention that one-off measurements at the moment don't > produce the expected number of results. So far our investigation shows that > this is most likely a probe firmware issue. We'll let you know once we find > the real cause and have a fix for this. In the meantime please accept our > apologies for the reduced usefulness of this function. > > Regards, > Robert Kisteleki > for the RIPE Atlas team Dear All, Last week we released a new firmware version (4560) to address this issue. The situation seems to have improved, and we'll keep it monitoring in order to see if the issue is completely solved. Regards, Robert Kisteleki for the RIPE Atlas team From the.lists at mgm51.com Thu Oct 10 20:54:26 2013 From: the.lists at mgm51.com (Mike.) Date: Thu, 10 Oct 2013 14:54:26 -0400 Subject: [atlas] Huge credit debit Message-ID: <201310101454260738.011DB521@smtp.24cl.home> One of my measurements (that usually uses about 1000 credits per day) all of a sudden posted a credit usage of 2,293,788.00 credits for a single measurement. 2013-10-10 18:13 UTC UDM 764596 samples of msm #1013277 -2,293,788.00 -1,279,520.00 I don't know where or why the 764596 samples came from, usually only 90 samples are performed. Was there a proble in the measurement or credit system recently? Thanks. From bortzmeyer at nic.fr Thu Oct 10 23:10:50 2013 From: bortzmeyer at nic.fr (Stephane Bortzmeyer) Date: Thu, 10 Oct 2013 23:10:50 +0200 Subject: [atlas] Huge credit debit In-Reply-To: <201310101454260738.011DB521@smtp.24cl.home> References: <201310101454260738.011DB521@smtp.24cl.home> Message-ID: <20131010211050.GA4396@nic.fr> On Thu, Oct 10, 2013 at 02:54:26PM -0400, Mike. wrote a message of 20 lines which said: > I don't know where or why the 764596 samples came from, usually only > 90 samples are performed. > > > Was there a proble in the measurement or credit system recently? Someone reported exactly the same problem on Twitter: https://twitter.com/bastien_durel/status/388407290262736897 From robert at ripe.net Fri Oct 11 10:05:49 2013 From: robert at ripe.net (Robert Kisteleki) Date: Fri, 11 Oct 2013 10:05:49 +0200 Subject: [atlas] Huge credit debit In-Reply-To: <20131010211050.GA4396@nic.fr> References: <201310101454260738.011DB521@smtp.24cl.home> <20131010211050.GA4396@nic.fr> Message-ID: <5257B15D.3070901@ripe.net> On 2013.10.10. 23:10, Stephane Bortzmeyer wrote: > On Thu, Oct 10, 2013 at 02:54:26PM -0400, > Mike. wrote > a message of 20 lines which said: > >> I don't know where or why the 764596 samples came from, usually only >> 90 samples are performed. >> >> >> Was there a proble in the measurement or credit system recently? > > Someone reported exactly the same problem on Twitter: > > https://twitter.com/bastien_durel/status/388407290262736897 I can confirm that the billing cycle yesterday afternoon went out of whack: we see that in some cases it overcharged (and in some other cases it undercharged) for measurement results. We've suspended billing for the time being and are investigating the cause for the issue. Of course we'll compensate the affected users for the overcharge. We'll get back with more details later. Thank you for your patience, Robert From robert at ripe.net Fri Oct 11 14:48:01 2013 From: robert at ripe.net (Robert Kisteleki) Date: Fri, 11 Oct 2013 14:48:01 +0200 Subject: [atlas] Huge credit debit In-Reply-To: <5257B15D.3070901@ripe.net> References: <201310101454260738.011DB521@smtp.24cl.home> <20131010211050.GA4396@nic.fr> <5257B15D.3070901@ripe.net> Message-ID: <5257F381.2040202@ripe.net> On 2013.10.11. 10:05, Robert Kisteleki wrote: > On 2013.10.10. 23:10, Stephane Bortzmeyer wrote: >> On Thu, Oct 10, 2013 at 02:54:26PM -0400, >> Mike. wrote >> a message of 20 lines which said: >> >>> I don't know where or why the 764596 samples came from, usually only >>> 90 samples are performed. >>> >>> >>> Was there a proble in the measurement or credit system recently? >> >> Someone reported exactly the same problem on Twitter: >> >> https://twitter.com/bastien_durel/status/388407290262736897 > > I can confirm that the billing cycle yesterday afternoon went out of whack: > we see that in some cases it overcharged (and in some other cases it > undercharged) for measurement results. We've suspended billing for the time > being and are investigating the cause for the issue. > > Of course we'll compensate the affected users for the overcharge. > > We'll get back with more details later. Update: so far we haven't identified the root cause of the problem. However, in order to prevent the issue form reoccuring, we deployed an emergency update of the billing code that includes sanity checks about the charges and refuses to bill unreasonable amounts. (This piece of code was scheduled for deployment after the RIPE76 meeting.) We also identified overcharged measurements and provided compensation for these. From robert at ripe.net Fri Oct 11 14:59:22 2013 From: robert at ripe.net (Robert Kisteleki) Date: Fri, 11 Oct 2013 14:59:22 +0200 Subject: [atlas] Huge credit debit In-Reply-To: <5257F381.2040202@ripe.net> References: <201310101454260738.011DB521@smtp.24cl.home> <20131010211050.GA4396@nic.fr> <5257B15D.3070901@ripe.net> <5257F381.2040202@ripe.net> Message-ID: <5257F62A.9090408@ripe.net> (... and now also sending the missing part of this mail ...) > Update: so far we haven't identified the root cause of the problem. However, > in order to prevent the issue form reoccuring, we deployed an emergency > update of the billing code that includes sanity checks about the charges and > refuses to bill unreasonable amounts. (This piece of code was scheduled for > deployment after the RIPE76 meeting.) > > We also identified overcharged measurements and provided compensation for > these. We believe this covers virtually all of the affected measurements. We plan to give you another update tomorrow, once we're through 2-3 more billing cycles. Regards, Robert Kisteleki for the RIPE Atlas team From the.lists at mgm51.com Fri Oct 11 17:05:52 2013 From: the.lists at mgm51.com (Mike.) Date: Fri, 11 Oct 2013 11:05:52 -0400 Subject: [atlas] Huge credit debit In-Reply-To: <5257F62A.9090408@ripe.net> References: <201310101454260738.011DB521@smtp.24cl.home> <20131010211050.GA4396@nic.fr> <5257B15D.3070901@ripe.net> <5257F381.2040202@ripe.net> <5257F62A.9090408@ripe.net> Message-ID: <201310111105520109.003EB922@smtp.24cl.home> On 10/11/2013 at 2:59 PM Robert Kisteleki wrote: |(... and now also sending the missing part of this mail ...) | |> Update: so far we haven't identified the root cause of the problem. |However, |> in order to prevent the issue form reoccuring, we deployed an emergency |> update of the billing code that includes sanity checks about the charges |and |> refuses to bill unreasonable amounts. (This piece of code was scheduled |for |> deployment after the RIPE76 meeting.) |> |> We also identified overcharged measurements and provided compensation for |> these. |We believe this covers virtually all of the affected measurements. | |We plan to give you another update tomorrow, once we're through 2-3 more |billing cycles. | |Regards, |Robert Kisteleki |for the RIPE Atlas team ============= Thanks for the follow-up. From robert at ripe.net Sat Oct 12 17:28:28 2013 From: robert at ripe.net (Robert Kisteleki) Date: Sat, 12 Oct 2013 17:28:28 +0200 Subject: [atlas] Huge credit debit In-Reply-To: <5257F62A.9090408@ripe.net> References: <201310101454260738.011DB521@smtp.24cl.home> <20131010211050.GA4396@nic.fr> <5257B15D.3070901@ripe.net> <5257F381.2040202@ripe.net> <5257F62A.9090408@ripe.net> Message-ID: <52596A9C.4050208@ripe.net> Dear All, Today's update, as promised: we believe that the situation has been successfully resolved. If you see any reamining issues, please write to us at atlas at ripe.net and we'll look into the details. Regards, Robert From bortzmeyer at nic.fr Fri Oct 25 12:15:28 2013 From: bortzmeyer at nic.fr (Stephane Bortzmeyer) Date: Fri, 25 Oct 2013 12:15:28 +0200 Subject: [atlas] Strong doubts about the option "DNS recursion" Message-ID: <20131025101528.GA25649@nic.fr> I fear that the option "recursion_desired" does not work as expected. Here was my experiment. I registered a new domain name which never appeared before, d7b16b2a97627e8ccbcccf93e773ec3f.cosmogol.fr. Because of its unreadability, there is zero chance it has ever been in a DNS cache. I then ask Atlas probes to dig it, with "recursion_desired" set to false: {'definitions': [{'query_class': 'IN', 'use_probe_resolver': True, 'af': 4, 'query_argument': 'd7b16b2a97627e8ccbcccf93e773ec3f.cosmogol.fr', 'query_type': 'TXT', 'type': 'dns', 'is_oneoff': True, 'recursion_desired': False, 'description': 'DNS resolution of d7b16b2a97627e8ccbcccf93e773ec3f.cosmogol.fr'}], 'probes': [{'requested': 10, 'type': 'area', 'value': 'WW'}]} But all the probes got the content of this TXT record! Measurement #1034372 if you want to check (unfortunately, this parameter is not displayed in ). It seems wrong, they should all have retrieved an empty answer (here, with dig, the behaviour I expected): % dig +norecurse @MYSERVER TXT d7b16b2a97627e8ccbcccf93e773ec3f.cosmogol.fr ; <<>> DiG 9.8.4-rpz2+rl005.12-P1 <<>> +norecurse @relay1 TXT d7b16b2a97627e8ccbcccf93e773ec3f.cosmogol.fr ; (2 servers found) ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 33141 ;; flags: qr ra ad; QUERY: 1, ANSWER: 0, AUTHORITY: 8, ADDITIONAL: 21 ;; OPT PSEUDOSECTION: ; EDNS: version: 0, flags: do; udp: 4096 ;; QUESTION SECTION: ;d7b16b2a97627e8ccbcccf93e773ec3f.cosmogol.fr. IN TXT ;; AUTHORITY SECTION: fr. 165392 IN NS d.ext.nic.fr. fr. 165392 IN NS d.nic.fr. ... From bortzmeyer at nic.fr Fri Oct 25 12:41:32 2013 From: bortzmeyer at nic.fr (Stephane Bortzmeyer) Date: Fri, 25 Oct 2013 12:41:32 +0200 Subject: [atlas] Strong doubts about the option "DNS recursion" In-Reply-To: <20131025101528.GA25649@nic.fr> References: <20131025101528.GA25649@nic.fr> Message-ID: <20131025104132.GA28636@nic.fr> On Fri, Oct 25, 2013 at 12:15:28PM +0200, Stephane Bortzmeyer wrote a message of 35 lines which said: > {'definitions': [{'query_class': 'IN', 'use_probe_resolver': True, 'af': 4, 'query_argument': 'd7b16b2a97627e8ccbcccf93e773ec3f.cosmogol.fr', 'query_type': 'TXT', 'type': 'dns', 'is_oneoff': True, 'recursion_desired': False, 'description': 'DNS resolution of d7b16b2a97627e8ccbcccf93e773ec3f.cosmogol.fr'}], 'probes': [{'requested': 10, 'type': 'area', 'value': 'WW'}]} I created a second domain by the same method, changed only the recursion_desired option and got exactly the same result (measurement #1034374): {'definitions': [{'query_class': 'IN', 'use_probe_resolver': True, 'af': 4, 'query_argument': 'ea4f6f5b9176b3d55061e5bd85410c1b.cosmogol.fr', 'query_type': 'TXT', 'type': 'dns', 'is_oneoff': True, 'recursion_desired': True, 'description': 'DNS resolution of ea4f6f5b9176b3d55061e5bd85410c1b.cosmogol.fr'}], 'probes': [{'requested': 10, 'type': 'area', 'value': 'WW'}]} So, it seems as if recursion_desired is ignored and the reality is that the RD bit is always set. By the way, this investigation was done because some people, to monitor the content of the DNS caches, use open resolvers ( and ) and I tought it would be better to use Atlas probes. But the tests may have to be run without RD, otherwise you risk "poisoning" the caches, if you use the measurement to test a hijacking, for instance, as in the first example mentioned. From philip.homburg at ripe.net Fri Oct 25 13:14:41 2013 From: philip.homburg at ripe.net (Philip Homburg) Date: Fri, 25 Oct 2013 13:14:41 +0200 Subject: [atlas] Strong doubts about the option "DNS recursion" In-Reply-To: <20131025104132.GA28636@nic.fr> References: <20131025101528.GA25649@nic.fr> <20131025104132.GA28636@nic.fr> Message-ID: <526A52A1.2030405@ripe.net> Hi Stephane, On 2013/10/25 12:41 , Stephane Bortzmeyer wrote: > So, it seems as if recursion_desired is ignored and the reality is that > the RD bit is always set. > > By the way, this investigation was done because some people, to > monitor the content of the DNS caches, use open resolvers > ( > and ) and I tought it would be > better to use Atlas probes. But the tests may have to be run without > RD, otherwise you risk "poisoning" the caches, if you use the > measurement to test a hijacking, for instance, as in the first example > mentioned. The RD bit is set because Atlas is supposed to do active measurements and not probe the host's systems. One obvious bug is that this behavior is not documented. Whether setting this flag is a good idea is of course subject to debate. Another thing is that just silently setting the flag is not optimal, maybe the measurement should have been rejected instead. Philip From bortzmeyer at nic.fr Fri Oct 25 13:59:12 2013 From: bortzmeyer at nic.fr (Stephane Bortzmeyer) Date: Fri, 25 Oct 2013 13:59:12 +0200 Subject: [atlas] Strong doubts about the option "DNS recursion" In-Reply-To: <526A52A1.2030405@ripe.net> References: <20131025101528.GA25649@nic.fr> <20131025104132.GA28636@nic.fr> <526A52A1.2030405@ripe.net> Message-ID: <20131025115912.GA2543@nic.fr> On Fri, Oct 25, 2013 at 01:14:41PM +0200, Philip Homburg wrote a message of 25 lines which said: > One obvious bug is that this behavior is not documented. It is much worse: it _is_ documented and the documentation is wrong. I spent two hours on that this morning and, as you can imagine, I'm not happy. > Another thing is that just silently setting the flag is not optimal, It is not "not optimal", it is _wrong_. > maybe the measurement should have been rejected instead. And the documentation fixed! From philip.homburg at ripe.net Fri Oct 25 14:04:57 2013 From: philip.homburg at ripe.net (Philip Homburg) Date: Fri, 25 Oct 2013 14:04:57 +0200 Subject: [atlas] Strong doubts about the option "DNS recursion" In-Reply-To: <20131025115912.GA2543@nic.fr> References: <20131025101528.GA25649@nic.fr> <20131025104132.GA28636@nic.fr> <526A52A1.2030405@ripe.net> <20131025115912.GA2543@nic.fr> Message-ID: <526A5E69.3030500@ripe.net> On 2013/10/25 13:59 , Stephane Bortzmeyer wrote: > On Fri, Oct 25, 2013 at 01:14:41PM +0200, > Philip Homburg wrote > a message of 25 lines which said: > >> One obvious bug is that this behavior is not documented. > > It is much worse: it _is_ documented and the documentation is wrong. I > spent two hours on that this morning and, as you can imagine, I'm not > happy. To avoid further confusion, which part of the documentation did you read? The measurement API? (https://atlas.ripe.net/doc/measurement-creation-api/) Philip From bortzmeyer at nic.fr Fri Oct 25 14:07:32 2013 From: bortzmeyer at nic.fr (Stephane Bortzmeyer) Date: Fri, 25 Oct 2013 14:07:32 +0200 Subject: [atlas] Strong doubts about the option "DNS recursion" In-Reply-To: <526A5E69.3030500@ripe.net> References: <20131025101528.GA25649@nic.fr> <20131025104132.GA28636@nic.fr> <526A52A1.2030405@ripe.net> <20131025115912.GA2543@nic.fr> <526A5E69.3030500@ripe.net> Message-ID: <20131025120732.GA4088@nic.fr> On Fri, Oct 25, 2013 at 02:04:57PM +0200, Philip Homburg wrote a message of 16 lines which said: > which part of the documentation did you > read? The measurement API? > (https://atlas.ripe.net/doc/measurement-creation-api/) Yes: Name Description Default Required? Price Modifier recursion_desired false From philip.homburg at ripe.net Fri Oct 25 15:12:09 2013 From: philip.homburg at ripe.net (Philip Homburg) Date: Fri, 25 Oct 2013 15:12:09 +0200 Subject: [atlas] Strong doubts about the option "DNS recursion" In-Reply-To: <20131025120732.GA4088@nic.fr> References: <20131025101528.GA25649@nic.fr> <20131025104132.GA28636@nic.fr> <526A52A1.2030405@ripe.net> <20131025115912.GA2543@nic.fr> <526A5E69.3030500@ripe.net> <20131025120732.GA4088@nic.fr> Message-ID: <526A6E29.10600@ripe.net> On 2013/10/25 14:07 , Stephane Bortzmeyer wrote: > On Fri, Oct 25, 2013 at 02:04:57PM +0200, > Philip Homburg wrote > a message of 16 lines which said: > >> which part of the documentation did you >> read? The measurement API? >> (https://atlas.ripe.net/doc/measurement-creation-api/) > > Yes: > > Name Description Default Required? Price Modifier > recursion_desired false The documentation is now updated to reflect the actual behavior. Philip From bortzmeyer at nic.fr Fri Oct 25 15:21:02 2013 From: bortzmeyer at nic.fr (Stephane Bortzmeyer) Date: Fri, 25 Oct 2013 15:21:02 +0200 Subject: [atlas] Strong doubts about the option "DNS recursion" In-Reply-To: <526A6E29.10600@ripe.net> References: <20131025101528.GA25649@nic.fr> <20131025104132.GA28636@nic.fr> <526A52A1.2030405@ripe.net> <20131025115912.GA2543@nic.fr> <526A5E69.3030500@ripe.net> <20131025120732.GA4088@nic.fr> <526A6E29.10600@ripe.net> Message-ID: <20131025132102.GA11840@nic.fr> On Fri, Oct 25, 2013 at 03:12:09PM +0200, Philip Homburg wrote a message of 17 lines which said: > The documentation is now updated to reflect the actual behavior. OK, it makes sense, thanks. From jerome.benoit at grenouille.com Sat Oct 26 06:20:23 2013 From: jerome.benoit at grenouille.com (=?ISO-8859-1?B?Suly9G1l?= Benoit) Date: Sat, 26 Oct 2013 06:20:23 +0200 Subject: [atlas] Strong doubts about the option "DNS recursion" In-Reply-To: <20131025115912.GA2543@nic.fr> References: <20131025101528.GA25649@nic.fr> <20131025104132.GA28636@nic.fr> <526A52A1.2030405@ripe.net> <20131025115912.GA2543@nic.fr> Message-ID: <20131026062023.67dac910@nemesis> On Fri, 25 Oct 2013 13:59:12 +0200 in <20131025115912.GA2543 at nic.fr>, Stephane Bortzmeyer wrote: > > > Another thing is that just silently setting the flag is not optimal, > > It is not "not optimal", it is _wrong_. > > > maybe the measurement should have been rejected instead. > > And the documentation fixed! Give us the source code also, the software will be also fixed in less than one hour. Does RIPE plan to release source code or not ? How can a user (its working on whatever you what) trust a measurement system that just do not give the basic level of transparency that a measurement system *need* to be trusted ? The availability of the source code to every single user ! the code can be modified or not, I just do not care about license issue, what I care about is the measurement system transparency, the rest is poetry for brainless persons). -- J?r?me Benoit aka fraggle OpenPGP Key ID : 9FE9161D Key fingerprint : 9CA4 0249 AF57 A35B 34B3 AC15 FAA0 CB50 9FE9 161D -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: not available URL: From vnaumov at ripe.net Sat Oct 26 10:54:51 2013 From: vnaumov at ripe.net (Viktor Naumov) Date: Sat, 26 Oct 2013 10:54:51 +0200 Subject: [atlas] Strong doubts about the option "DNS recursion" In-Reply-To: <20131026062023.67dac910@nemesis> References: <20131025101528.GA25649@nic.fr> <20131025104132.GA28636@nic.fr> <526A52A1.2030405@ripe.net> <20131025115912.GA2543@nic.fr> <20131026062023.67dac910@nemesis> Message-ID: <526B835B.8070901@ripe.net> Hi J?r?me, It is there: https://atlas.ripe.net/get-involved/source-code/ /vty On 10/26/13 6:20 AM, J?r?me Benoit wrote: > Give us the source code also, the software will be also fixed in less > than one hour. > Does RIPE plan to release source code or not ? > > How can a user (its working on whatever you what) trust a measurement > system that just do not give the basic level of transparency that a > measurement system *need* to be trusted ? The availability of the source > code to every single user ! the code can be modified or not, I just do > not care about license issue, what I care about is the measurement > system transparency, the rest is poetry for brainless persons). > > From jerome.benoit at grenouille.com Sat Oct 26 21:04:41 2013 From: jerome.benoit at grenouille.com (=?ISO-8859-1?B?Suly9G1l?= Benoit) Date: Sat, 26 Oct 2013 21:04:41 +0200 Subject: [atlas] Strong doubts about the option "DNS recursion" In-Reply-To: <526B835B.8070901@ripe.net> References: <20131025101528.GA25649@nic.fr> <20131025104132.GA28636@nic.fr> <526A52A1.2030405@ripe.net> <20131025115912.GA2543@nic.fr> <20131026062023.67dac910@nemesis> <526B835B.8070901@ripe.net> Message-ID: <20131026210441.2f0afcae@nemesis> On Sat, 26 Oct 2013 10:54:51 +0200 in <526B835B.8070901 at ripe.net>, Viktor Naumov wrote: > Hi J?r?me, > > It is there: > https://atlas.ripe.net/get-involved/source-code/ Great ! Should have search before posting :) ++ -- J?r?me Benoit aka fraggle OpenPGP Key ID : 9FE9161D Key fingerprint : 9CA4 0249 AF57 A35B 34B3 AC15 FAA0 CB50 9FE9 161D -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: not available URL: From bortzmeyer at nic.fr Mon Oct 28 09:06:18 2013 From: bortzmeyer at nic.fr (Stephane Bortzmeyer) Date: Mon, 28 Oct 2013 09:06:18 +0100 Subject: [atlas] Strong doubts about the option "DNS recursion" In-Reply-To: <20131026210441.2f0afcae@nemesis> References: <20131025101528.GA25649@nic.fr> <20131025104132.GA28636@nic.fr> <526A52A1.2030405@ripe.net> <20131025115912.GA2543@nic.fr> <20131026062023.67dac910@nemesis> <526B835B.8070901@ripe.net> <20131026210441.2f0afcae@nemesis> Message-ID: <20131028080618.GA26363@nic.fr> On Sat, Oct 26, 2013 at 09:04:41PM +0200, J?r?me Benoit wrote a message of 34 lines which said: > > It is there: > > https://atlas.ripe.net/get-involved/source-code/ > > Great ! Should have search before posting :) It is very recent, many people asked for it in the previous years. Also, I believe it is "only" the Atlas probe's source code, not the C&C source code. From robert at ripe.net Wed Oct 30 14:14:36 2013 From: robert at ripe.net (Robert Kisteleki) Date: Wed, 30 Oct 2013 15:14:36 +0200 Subject: [atlas] Replacing "ping RRDs" with "zoomable ping graphs" In-Reply-To: <5242B718.4090007@ripe.net> References: <5242B718.4090007@ripe.net> Message-ID: <5271063C.3060205@ripe.net> Dear All, We'd like to let you know that as described in the mail below, we stopped the generation of the "built-in ping RRDs" (though a bit later than projected). The RRDs that visualise UDM ping measurements will be decommissioned next; the seismograph is provided as a replacement visualisation tool. Regards, Robert Kisteleki On 2013.09.25. 13:12, Robert Kisteleki wrote: > > Dear RIPE Atlas users, > > As you may have read in the recent RIPE Labs article > (https://labs.ripe.net/Members/becha/seismograph-and-other-new-ripe-atlas-features), > we've replaced the old style, RRD based visualisation for built-in ping > measurements with client side, zoomable graphs. Since then we've also > changed the graphs that represent the probe's bandwidth usage. > > The new graphs have the benefit that they retain all details, even for > longer time intervals. (These details were lost in the RRD based graphs over > time.) Other reasons why we changed include that generating the RRD graphs > did not scale well with the increasing number of RIPE Atlas probes, and > making them required us to maintain a large set of backend servers, each > being a single point of failure. > > With the new approach you still have visualisations for this information, > with more details and interactivity (though it does require running > JavaScript on the client side). > > We'll change the remaining RRDs to this new model as well in the coming > period, and stop generating the RRD based images altogether. We expect to > stop with the "built-in pings" and the probe traffic RRDs first, likely in > the second week of October. > > If you would like to embed these new style visualisations into your pages > (maybe because you used the RRD graphs before), then you'll have to embed a > RIPEstat widget; see > https://stat.ripe.net/widgets/demo/atlas_probe_widgets.html for details. > > Regards, > Robert Kisteleki > for the RIPE Atlas team > > >