[RETRANSMISSION] DNS problems with nameservers for .be and193.in-addr.arpa

[Brad Knowles <]

[ The previous version of this message (with attachments included) 
was too large to be accepted by certain recipient addresses, so I am 
retransmitting the message with the attachments instead available at 
the URL <ftp://ftp.shub-internet.org/pub/shub/brad/dns/>.  I also 
mistakenly left off a few recipients on the original version, which 
are included here.  -Brad ]


Folks,

	A couple of problems with some pretty important nameservers has 
been brought to my attention in the last couple of weeks.  I would 
not normally bring this issue directly to such a wide group of 
people, but I believe that this matter is sufficiently serious to 
warrant such an approach.


	In particular, we have discovered that holem.belnet.be 
[193.190.198.10] (acting as one of the two nameservers for 
ns.belnet.be) is acting as both an authoritative nameserver for .be 
(among many others, I'm sure), as well as acting as a general caching 
nameserver.

	This has hit us a few times when we've changed machines on our 
own networks, made the changes in our authoritative nameservers and 
then reloaded the zones, and then finally restarted our caching 
nameservers -- only to have the data in the caching nameservers 
poisoned by incorrect information from holem.belnet.be 
[193.190.198.2].

	Earlier we had the same problem with vivaldi.belnet.be 
[193.190.198.2] (the other machine acting as ns.belnet.be).  However, 
the problem with vivaldi.belnet.be appears to have since been 
corrected.  However, I note that DNS.CS.KULEUVEN.AC.be, NS.EU.NET, 
SPARKY.ARL.MIL, and SUNIC.SUNET.SE appear to have the same problem as 
holem.belnet.be [193.190.198.10].


	This is an extremely dangerous situation, one that has long since 
been recognized by experienced nameserver administrators.  This is 
why section 2.5 of RFC 2870 "Root Nameserver Operational 
Requirements" says:

   2.5 Servers MUST provide authoritative responses only from the zones
       they serve.  The servers MUST disable recursive lookup,
       forwarding, or any other function that may allow them to provide
       cached answers.  They also MUST NOT provide secondary service for
       any zones other than the root and root-servers.net zones.  These
       restrictions help prevent undue load on the root servers and
       reduce the chance of their caching incorrect data.

	While these machines are not root nameservers (to the best of my 
knowledge), the same rules and regulations should be applied to their 
maintenance and operation, for the same reasons.  Therefore, I 
recommend that these machines be fixed ASAP, and if they cannot be 
fixed within a reasonable period of time, they should be removed from 
the list of authoritative nameservers for .be.

	Furthermore, I recommend that if these machines cannot be fixed 
within a reasonable period of time, they be removed from all 
nameserver duties for all upper-level domains, including root 
nameservice, gTLD and ccTLD duties, service for any of the 
in-addr.arpa zones, etc....


	I have also noticed that while vivaldi.belnet.be [193.190.198.2] 
and AUTH02.NS.UU.NET do not appear to be caching data directly, they 
do appear to somehow be secondaries for the .com zone, as the 
referrals they give for www.aol.com go directly to the AOL 
nameservers, and not through the root or .com gTLD nameservers.  Try 
as I might, I cannot find the IP addresses 193.190.198.2  and 
198.6.1.82 on the list of published nameservers for the .com gTLD. 
This concerns me, but without additional information, I can't call 
this an "error" per se.


	You can confirm my test results by executing the following Bourne 
shell script:

		#!/bin/sh
		BENS="DNS.CS.KULEUVEN.AC.be. SECDNS.EUNET.be. SPARKY.ARL.MIL. 
SUNIC.SUNET.SE. AUTH02.NS.UU.NET. NS.EU.NET. 193.190.198.10 
193.190.198.2 NS.DNS.be."

		for NS in $BENS
		do
        		dig @$NS www.aol.com. a +aa
		done

	Note that because of the lossy nature of sending UDP packets via 
the Internet, you may need to run this script multiple times in order 
to get complete output.  I know I did.

	The results I get can be found at 
<ftp://ftp.shub-internet.org/pub/shub/brad/dns/be.txt>.  I would also 
refer you to the output of running "doc -d be" at 
<ftp://ftp.shub-internet.org/pub/shub/brad/dns/log-be.txt>.


	I have also discovered that NS.EU.NET is not properly handling 
DNS queries for 193.in-addr.arpa.  In particular, I was trying to do 
a reverse lookup on 193.74.108.3, and was getting SERVFAIL responses, 
something that I should not be getting -- especially not from 
nameservers that are supposed to be serving 193.in-addr.arpa.

	I wrote the following script to check out the nameservers for 
this top-level reverse zone:

		#!/bin/sh
		REVNS="NS.RIPE.NET. NS.EU.NET. AUTH03.NS.UU.NET. NS2.NIC.FR. 
SUNIC.SUNET.SE. MUNNARI.OZ.AU. NS.APNIC.NET."

		for NS in $REVNS
		do
        		dig @$NS -x 193.74.108.3
		done

	The results I get are at 
<ftp://ftp.shub-internet.org/pub/shub/brad/dns/reverse.txt>.  I would 
recommend that you also look at the output of running "doc -d 
3.108.74.193.in-addr.arpa" and "doc -d 74.193.in-addr.arpa" at 
<ftp://ftp.shub-internet.org/pub/shub/brad/dns/log-reverse.txt>).


	You will note that NS.EU.NET is not the only nameserver here that 
is giving me SERVFAIL.  I also get it from SUNIC.SUNET.SE and 
MUNNARI.OZ.AU.  Again, if these machines cannot be fixed in a 
reasonable period of time, I suggest that they should be removed from 
the list of authoritative nameservers for 193.in-addr.arpa.

	Also note some of the interesting and different answers given 
back by the various machines for the IP address of ns.ripe.net in 
particular -- some list the IPv6 address and some don't.  Is this an 
error?  I honestly don't know....



	With this information, I hope that you will understand the 
issues, and share my very deep concern that this is a systemic 
problem that affects everyone on the Internet, and goes far beyond 
the operational problems that any one ISP may be currently 
experiencing.

	I don't want to come across as a self-styled DNS expert, although 
I do believe that I know more on this topic than the average system 
administrator.  I hope that you won't hold against me the fact that I 
am the current maintainer of the DNS debugging tool "doc", and that I 
have used the output of this tool to help document my position.


	If I wanted to be really nasty, I'd ask that you take away the 
high-level authoritative nameserver privileges for the above named 
machines, and that you instead give them to us, thus increasing the 
prestige of my employer.

	However, all I really want to do is get these problems fixed, and 
stop having so many complaints being delivered to me which ultimately 
result from the issues I have raised here -- and therefore, problems 
that I cannot do anything about, at least not by myself.

	That said, if it would help for Skynet to provide a nameserver to 
fill in for these various different roles, I'm sure that I and my 
employers would be more than happy to move Heaven and Earth to make 
sure that we got a suitable machine up and running as quickly as 
humanly possible.



ADDENDUM:

	Please note that we have previously brought up these issues 
regarding ns.belnet.be with Marc Roger at Belnet.  So far as I can 
tell, he didn't seem to care.  We also brought these issues up with 
DNS.BE, and again we haven't seen any results.  Also note that the 
information at <http://www.iana.org/root-whois/be.htm> appears to be 
out-of-date, as I keep getting bounces back from the supposedly 
official address "marc.vanwezemael@localhost" as the Administrative POC 
for .be.

	I know that the management of EUnet is aware of their reverse DNS 
problems, because we have been having running problems with them for 
at least the last couple of weeks, and our servers don't accept mail 
from any of theirs, because they don't have reverse DNS.  Of course, 
the reason they don't have reverse DNS is that the same servers are 
used for serving various important zones within Europe as well as 
serving those local to EUnet, and these machines are the ones that 
are screwed-up.

	Unfortunately, while they have been aware of the problems for a 
while and we've been trying to get them to fix their local problems 
in that time, it wasn't until more recently that I realized just how 
severe the issue with their nameservers really is.  Again, I have yet 
to see anything useful at all come from them on this issue.


	Frankly, I just don't know what else to do.  If I have not used 
the proper escalation procedure, then I would appreciate it if 
someone would tell me what the proper escalation procedure is.  I 
would also like to see where this is documented on the appropriate 
web sites, because I looked everywhere I could think of, and still 
haven't found it.

--
  These are my opinions -- not to be taken as official Skynet policy
======================================================================
Brad Knowles, blk@localhost                || Belgacom Skynet SA/NV
Systems Architect, Mail/News/FTP/Proxy Admin || Rue Colonel Bourg, 124
Phone/Fax: +32-2-706.13.11/12.49             || B-1140 Brussels
http://www.skynet.be                         || Belgium