From mir at ripe.net Mon Jul 11 16:02:21 2011 From: mir at ripe.net (Mirjam Kuehne) Date: Mon, 11 Jul 2011 16:02:21 +0200 Subject: [dns-wg] Analysis of Increased Query Load on Root Name Servers In-Reply-To: <4E0B27C0.1030709@ripe.net> References: <4E0B27C0.1030709@ripe.net> Message-ID: <4E1B026D.2010606@ripe.net> [apologies for duplicates] Dear colleagues, We did some more analysis of the recent increase in query load on K-root and other root name servers. Please read on RIPE Labs: http://labs.ripe.net/Members/wnagele/analysis-of-increased-query-load-on-root-name-servers Kind Regards, Mirjam Kuehne RIPE NCC From dougb at dougbarton.us Tue Jul 12 02:41:03 2011 From: dougb at dougbarton.us (Doug Barton) Date: Mon, 11 Jul 2011 17:41:03 -0700 Subject: [dns-wg] Analysis of Increased Query Load on Root Name Servers In-Reply-To: <4E1B026D.2010606@ripe.net> References: <4E0B27C0.1030709@ripe.net> <4E1B026D.2010606@ripe.net> Message-ID: <4E1B981F.20209@dougbarton.us> On 07/11/2011 07:02, Mirjam Kuehne wrote: > [apologies for duplicates] > > Dear colleagues, > > We did some more analysis of the recent increase in query load on K-root > and other root name servers. Please read on RIPE Labs: > > http://labs.ripe.net/Members/wnagele/analysis-of-increased-query-load-on-root-name-servers This analysis is interesting from the traffic standpoint, but doesn't seem to answer one of the questions that I had, which is what caused the sudden increase? Historically this kind of thing has happened in the case of a misconfiguration for the name service for a popular domain, but (unless I missed it, and if so apologies) the question of, "Was misconfigured?" isn't answered in this paper. I'm not (necessarily) asking you to "name and shame" the domain in question, but I still believe that it's an interesting question on several levels. Meanwhile congrats to the K staff for handling this issue so adroitly. Doug -- Nothin' ever doesn't change, but nothin' changes much. -- OK Go Breadth of IT experience, and depth of knowledge in the DNS. Yours for the right price. :) http://SupersetSolutions.com/ From emile.aben at ripe.net Tue Jul 12 09:20:51 2011 From: emile.aben at ripe.net (Emile Aben) Date: Tue, 12 Jul 2011 09:20:51 +0200 Subject: [dns-wg] Analysis of Increased Query Load on Root Name Servers In-Reply-To: <4E1B981F.20209@dougbarton.us> References: <4E0B27C0.1030709@ripe.net> <4E1B026D.2010606@ripe.net> <4E1B981F.20209@dougbarton.us> Message-ID: <4E1BF5D3.6020302@ripe.net> On 12/07/2011 02:41, Doug Barton wrote: > On 07/11/2011 07:02, Mirjam Kuehne wrote: >> [apologies for duplicates] >> >> Dear colleagues, >> >> We did some more analysis of the recent increase in query load on K-root >> and other root name servers. Please read on RIPE Labs: >> >> http://labs.ripe.net/Members/wnagele/analysis-of-increased-query-load-on-root-name-servers > > This analysis is interesting from the traffic standpoint, but doesn't > seem to answer one of the questions that I had, which is what caused the > sudden increase? Historically this kind of thing has happened in the > case of a misconfiguration for the name service for a popular domain, > but (unless I missed it, and if so apologies) the question of, "Was > misconfigured?" isn't answered in this paper. Hi Doug, We don't have all the answers, but it appears not to be related to a misconfigured zone, the zone looked (and still looks) like this: .com. 7200 IN SOA ns1.. root.ns1..com. 20091027 28800 600 604800 86400 .com. 300 IN A .com. 300 IN A .com. 7200 IN NS ns1.. .com. 7200 IN NS ns2.. .com. 7200 IN NS ns3.. .com. 7200 IN NS ns4.. www..com. 300 IN A www..com. 300 IN A .com. 7200 IN SOA ns1.. root.ns1..com. 20091027 28800 600 604800 86400 As mentioned in the article, we have several indications that this was caused by a botnet. It is unlikely this was a reflector attack with spoofed source addresses, as there are some 60,000 unique source IPs per hour in the queries for this specific domain. For targeted spoofing I'd would expect this number to be very low, for random spoofing I'd expect this number would be far higher. If you have any clue or indication on things we could further investigate, let us know, here or on RIPE Labs. best regards, Emile Aben RIPE NCC From dougb at dougbarton.us Wed Jul 13 02:43:34 2011 From: dougb at dougbarton.us (Doug Barton) Date: Tue, 12 Jul 2011 17:43:34 -0700 Subject: [dns-wg] Analysis of Increased Query Load on Root Name Servers In-Reply-To: <4E1BF5D3.6020302@ripe.net> References: <4E0B27C0.1030709@ripe.net> <4E1B026D.2010606@ripe.net> <4E1B981F.20209@dougbarton.us> <4E1BF5D3.6020302@ripe.net> Message-ID: <4E1CEA36.6010105@dougbarton.us> On 07/12/2011 00:20, Emile Aben wrote: > We don't have all the answers, but it appears not to be related to a > misconfigured zone Thank you for satisfying my idle curiosity. :) I did not mean to imply that your report was in any way deficient at describing what you think the problem was actually caused by. My curiosity about this particular issue was raised for 2 reasons, one being (as I said previously) history of previous incidents. The other is given that if this were a DDOS attempt it's a rather weak one (on several levels) I can't help finding that unlikely. (Which again, is not a criticism of your analysis, merely a disturbing lack of pieces falling neatly into previously-known patterns.) I did note this from your scrubbed zone file: .com. 7200 IN NS ns1.. .com. 7200 IN NS ns2.. .com. 7200 IN NS ns3.. .com. 7200 IN NS ns4.. Are we to conclude from that that is different from .com? If so, and is misconfigured somehow, that would start to look more like misconfiguration patterns that we've seen in the past; particularly if is not in COM, and therefore the COM zone has no glue for those hostnames. I also note that 2 hours seems to be a ridiculously short TTL for NS records, which would seem to put a little more weight on the "possible misconfiguration" side of the balance. One could imagine a moderately popular game site receiving the CN equivalent of being slashdotted, and previously-painless minor misconfigurations suddenly causing much larger problems. hth, Doug -- Nothin' ever doesn't change, but nothin' changes much. -- OK Go Breadth of IT experience, and depth of knowledge in the DNS. Yours for the right price. :) http://SupersetSolutions.com/ From minmin at jprs.co.jp Wed Jul 13 10:30:23 2011 From: minmin at jprs.co.jp (Masato Minda) Date: Wed, 13 Jul 2011 17:30:23 +0900 Subject: [dns-wg] Response size of JP's DNSKEY was changed Message-ID: <4E1D579F.5080803@jprs.co.jp> Folks In RIPE 62, I had a presentation about response size of DNS with DNSSEC. Somebody was interested about reply size of JP's DNSKEY. (slide 9) In this slide, the response size of JP's DNSKEY was 1203 octets. Last week(July 7), we have changed it. $ dig +dnssec jp dnskey | grep SIZE ;; MSG SIZE rcvd: 893 Here is the size of packet. ----------------------- KSK of DNSKEY 276 ZSK of DNSKEY 148 RRSIG by KSK 290 RRSIG by ZSK 162 ----------------------- ---------------------- DNS Header 12 Question section 8 JP:4 class:2 type:2 EDNS0 11 ---------------------- Before July 7, response of DNSKEY had 1 KSK, 3 ZSK, 1 RRSIG by KSK, and 1 RRSIG by ZSK. 12 + 8 + 11 + 276*1 + 148*3 + 290*1 + 162*1 = 1203 After July 7, response of DNSKEY has 1 KSK, 2 ZSK and 1 RRSIG by KSK. 12 + 8 + 11 + 276*1 + 148*2 + 290*1 + 162*0 = 893 It is current result. * KSK rollover In KSK rollover, we will use the double signature key rollover. 12 + 8 + 11 + 276*2 + 148*2 + 290*2 + 162*0 = 1459 Of course, IP and UDP header are needed in real packet, IPv4 IPv6 IP 20 40 UDP 8 8 -------------------- total 28 48 The size of packet in KSK rollover, IPv4 is 1487, IPv6 is 1507. 1507 is bigger than traditional MTU. :-( If the ZSK is only one when KSK rollover, its response size is 1311. 12 + 8 + 11 + 276*2 + 148*1 + 290*2 + 162*0 = 1311 In this condition, IPv4 is 1339, IPv6 is 1359. It's ok. :-) It is a bit trouble. But, we will do our best. Unfortunately it is impossible to less than 1280 in current condition. I think that ECC (Elliptic Curve Cryptography) can clear under 1280. Regards, -- minmin / Masato Minda Research and Development Dept. Japan Registry Services Co., Ltd. (JPRS) From emile.aben at ripe.net Wed Jul 13 11:32:49 2011 From: emile.aben at ripe.net (Emile Aben) Date: Wed, 13 Jul 2011 11:32:49 +0200 Subject: [dns-wg] Analysis of Increased Query Load on Root Name Servers In-Reply-To: <4E1CEA36.6010105@dougbarton.us> References: <4E0B27C0.1030709@ripe.net> <4E1B026D.2010606@ripe.net> <4E1B981F.20209@dougbarton.us> <4E1BF5D3.6020302@ripe.net> <4E1CEA36.6010105@dougbarton.us> Message-ID: <4E1D6641.1060503@ripe.net> On 13/07/2011 02:43, Doug Barton wrote: > On 07/12/2011 00:20, Emile Aben wrote: >> We don't have all the answers, but it appears not to be related to a >> misconfigured zone > > Thank you for satisfying my idle curiosity. :) I did not mean to imply > that your report was in any way deficient at describing what you think > the problem was actually caused by. My curiosity about this particular > issue was raised for 2 reasons, one being (as I said previously) history > of previous incidents. The other is given that if this were a DDOS > attempt it's a rather weak one (on several levels) I can't help finding > that unlikely. (Which again, is not a criticism of your analysis, merely > a disturbing lack of pieces falling neatly into previously-known patterns.) I agree that this is a bit strange, my guess would be that this was a test of capabilities of some kind. Not too strange to pick the root-servers as a target, since it is relatively well instrumented. > I did note this from your scrubbed zone file: > > .com. 7200 IN NS ns1.. > .com. 7200 IN NS ns2.. > .com. 7200 IN NS ns3.. > .com. 7200 IN NS ns4.. > > Are we to conclude from that that is different from > .com? If so, and is misconfigured somehow, that would Yes, it was a different domain, not in COM. We asked folks that operate COM and they didn't see the same query-storm for this domain though. If these were all 'normal' resolvers dealing with a misconfigured zone, I'd expect them to follow the delegation chain. Also when spot-checking some 20 source IPs for these queries we didn't find these did any other queries to K-root then for things in .com. But again, we don't have the definite answer and are not excluding any possible explanation, so thanks for inquiring deeper into this. > start to look more like misconfiguration patterns that we've seen in the > past; particularly if is not in COM, and therefore the COM > zone has no glue for those hostnames. > > I also note that 2 hours seems to be a ridiculously short TTL for NS > records, which would seem to put a little more weight on the "possible > misconfiguration" side of the balance. One could imagine a moderately > popular game site receiving the CN equivalent of being slashdotted, and > previously-painless minor misconfigurations suddenly causing much larger > problems. I just looked at the query load for www..com on 20110628, and before 16:28 UTC (0:28 Chinese Standard time) we have 2 queries for this domain, then it all starts: #queries timestamp 1 1309252434 1 1309274472 8603 1309278521 9630 1309278522 11277 1309278523 14123 1309278524 12271 1309278525 12457 1309278526 12118 1309278527 12369 1309278528 12234 1309278529 12402 1309278530 12202 1309278531 12469 1309278532 12138 1309278533 12149 1309278534 ... (continues to be in 10-12kps range for a while) So either the misconfiguration started at around 16:28 UTC, or this wasn't a misconfiguration. The third possibility, already misconfigurated+CN-slashdotted, I think is not impossible but unlikely, both because of it being past midnight at the ASes that were a major source of queries, and the very sudden increase in load. regards, Emile Aben RIPE NCC From bortzmeyer at nic.fr Wed Jul 13 11:29:16 2011 From: bortzmeyer at nic.fr (Stephane Bortzmeyer) Date: Wed, 13 Jul 2011 11:29:16 +0200 Subject: [dns-wg] Re: Response size of JP's DNSKEY was changed In-Reply-To: <4E1D579F.5080803@jprs.co.jp> References: <4E1D579F.5080803@jprs.co.jp> Message-ID: <20110713092916.GA11870@nic.fr> On Wed, Jul 13, 2011 at 05:30:23PM +0900, Masato Minda wrote a message of 71 lines which said: > In this condition, IPv4 is 1339, IPv6 is 1359. It's ok. :-) > It is a bit trouble. But, we will do our best. > > Unfortunately it is impossible to less than 1280 in current condition. What's the purpose of this exercice? Many TLD have larger DNSKEY sets (for instance, .FR) and "it works". Is it really a good idea to change the DNSKEY set, just to avoid problems with the minority of broken sites? What is your goal in doing so? From dougb at dougbarton.us Wed Jul 13 19:38:43 2011 From: dougb at dougbarton.us (Doug Barton) Date: Wed, 13 Jul 2011 10:38:43 -0700 Subject: [dns-wg] Analysis of Increased Query Load on Root Name Servers In-Reply-To: <4E1D6641.1060503@ripe.net> References: <4E0B27C0.1030709@ripe.net> <4E1B026D.2010606@ripe.net> <4E1B981F.20209@dougbarton.us> <4E1BF5D3.6020302@ripe.net> <4E1CEA36.6010105@dougbarton.us> <4E1D6641.1060503@ripe.net> Message-ID: <4E1DD823.3000002@dougbarton.us> On 07/13/2011 02:32, Emile Aben wrote: > Yes, it was a different domain, not in COM. > We asked folks that operate COM and they didn't see the same query-storm > for this domain though. If these were all 'normal' resolvers dealing > with a misconfigured zone, I'd expect them to follow the delegation > chain. Also when spot-checking some 20 source IPs for these queries we > didn't find these did any other queries to K-root then for things in > .com. Interesting stuff! Thanks for once again indulging my curiosity. I always find it interesting when more data makes a problem muddier instead of clearer. :) Doug -- Nothin' ever doesn't change, but nothin' changes much. -- OK Go Breadth of IT experience, and depth of knowledge in the DNS. Yours for the right price. :) http://SupersetSolutions.com/ From denis at ripe.net Thu Jul 14 15:47:56 2011 From: denis at ripe.net (Denis Walker) Date: Thu, 14 Jul 2011 15:47:56 +0200 Subject: [dns-wg] DOMAIN attributes review Message-ID: <4E1EF38C.2010701@ripe.net> [Apologies for duplicate emails.] Dear Colleagues, At the RIPE 62 Meeting we were asked to consider the attributes in DOMAIN objects. With the removal of any hierarchy in reverse delegations and the removal of forward domain data there is no longer any use case for the "mnt-lower:" attribute. Other attributes that are considered to be used only for forward domains are: - "refer:" - "sub-dom:" - "dom-net:" The statistics listed below show how many of the above attributes have been added to reverse delegations and ENUMs. The RIPE NCC would like to ask the community if anyone sees any justification for the use of these attributes in reverse delegations or ENUMs. If there is no justification, all four of these attributes can be removed from all DOMAIN objects when the last two TLD operators move their data out of the RIPE Database. This is anticipated to occur by the end of 2011. To keep any discussion focussed please send any comments or feedback to the DNS Working Group Mailing list. Regards Denis Walker Business Analyst RIPE NCC Database Group Statistics ---------- Number of each attribute used in reverse delegations or ENUMs and the number of unique maintainers who manage these objects. "refer:" = 6 unique maintainers = 1 "sub-dom:" = 271 unique maintainers = 23 "dom-net:" = 2219 unique maintainers = 140 "mnt-lower:" = 46485 unique maintainers = 4208 From pk at DENIC.DE Tue Jul 19 08:09:58 2011 From: pk at DENIC.DE (Peter Koch) Date: Tue, 19 Jul 2011 08:09:58 +0200 Subject: [dns-wg] DOMAIN attributes review In-Reply-To: <4E1EF38C.2010701@ripe.net> References: <4E1EF38C.2010701@ripe.net> Message-ID: <20110719060958.GG30334@x27.adm.denic.de> On Thu, Jul 14, 2011 at 03:47:56PM +0200, Denis Walker wrote: > With the removal of any hierarchy in reverse delegations and the removal > of forward domain data there is no longer any use case for the > "mnt-lower:" attribute. > > Other attributes that are considered to be used only for forward domains > are: > - "refer:" > - "sub-dom:" > - "dom-net:" thanks for getting the ball rolling, Denis. > Number of each attribute used in reverse delegations or ENUMs and the > number of unique maintainers who manage these objects. > > "refer:" = 6 unique maintainers = 1 all these objects share the same target that i currently cannot connect to (port 43 or otherwise). Also I do not believe this makes much sense for the reverse space. It could be intersting for ENUM, though, but then for similar reasons as in the forward case (ACL maintenance and the likes) it is probably practically useless. (maybe repost the first message to the ENUM list ...) > "sub-dom:" = 271 unique maintainers = 23 > "dom-net:" = 2219 unique maintainers = 140 dom-net seems to list the exact address space that corresponds to the IN-ADDR.ARPA subdomain in question, so that appears rather redundant. sub-dom is almost always used in a way inconsistent with the syntax description (which requires non-FQDNs) and most uses seem to be enumerations of sibling domains rather than subdomains. For ENUM, the choice of available subdomains is even more limited, so i doubt it's useful there, either. > "mnt-lower:" = 46485 unique maintainers = 4208 With the lack of ability to register subdomains in the reverse tree this might be obsolete. The relative popularity might suggest kind of an awareness campaign, though. For ENUM, I'm not sure whether this is still necesary as a protective measure. In any case, centrally changing the Tier1 objects will take some extra effort. -Peter (no hats)