[ncc-services-wg] Updated Hostcount++ proposal
Andrei Robachevsky andrei at ripe.net
Thu Oct 6 14:21:27 CEST 2005
Dear Colleagues, Please find below an updated proposal for Hostcount++. Hostcount ++ is a set of enhancements to the hostcount service that the RIPE NCC has been running for 15 years. The Hostcount++ concept was presented at RIPE 50: http://www.ripe.net/ripe/meetings/ripe-50/presentations/ripe50-serv-hostcount.pdf The general feedback was positive, and there were some follow-up discussions. For examples of these discussions see: http://www.ripe.net/ripe/maillists/archives/ncc-services-wg/2005/msg00046.html http://www.ripe.net/ripe/maillists/archives/ncc-services-wg/2005/msg00051.html The Hostcount ++ proposal has been refined based on this feedback. Your comments and suggestions are appreciated. The RIPE NCC will make a short update on this project at the RIPE NCC Services Working Group at RIPE 51. Best regards, Andrei Robachevsky RIPE NCC Hostcount++ for the RIPE NCC Service Region =========================================== Motivation ---------- Every month the RIPE NCC coordinates the collection of data from the DNS zones of ccTLDs in the RIPE NCC Service Region and publishes summary statistics derived from this data. The first hostcount was in October 1990 and involved 19 ccTLDs. Since then, the hostcount has expanded to include around 100 ccTLDs. The hostcount is provided as an informational service to the Internet community. The current implementation of the Hostcount has several problems: - Connectivity problems with certain hosts - Lack of data accuracy due to access policies of some ccTLD domains - The increasing number of domains There are also issues related to privacy and acceptable use of the results. Objectives and Deliverables --------------------------- 1) To increase the precision and reliability of data, especially for trend analysis, more sources will be added. Our final aim is to collect data from: - Forward DNS - Reverse DNS - BGP view on availability of prefixes 2) Re-designing the software to be more scalable is very important, since the amount of input to be analysed is already of considerable size and is increasing every day. 3) To protect the privacy of domain holders and data, the following changes will be made: - Use cases will be defined in more detail and "Acceptable Use Policy" documents will be written accordingly. - Raw data will not be made available. 4) The following information will be made available, via textual dumps and graphs and tables on the web site: - Average visibility of RIPE NCC allocations: Visibility will be calculated by the information provided from the Routing Information Service (RIS) project. - Number of hosts and Average number of hosts: Host information will be retrieved via zone transfers. If the zone transfers fail due to blocking, partial zone information will be generated from reverse scan. Blocked zone transfers will be constructed by enumeration of the zones. - Breakdown of DNS record types: This information will depend on zone transfers of each allocation. 5) Results will be sorted into: - visible/invisible: This is based on whether a given IP range has been announced in the preceding month for longer than a particular threshold. - ccTLDs: This is based on the ccTLDs in the RIPE NCC service region. - RIPE NCC allocations: These are the allocations made by the RIPE NCC for its members. - IANA allocations: These are the /8 ranges for IPv4, and /23 ranges IPv6, allocated to the RIPE NCC. 6) In addition, timestamps of the results will be more consistent. 7) Hostcounts for certain ccTLDs will be performed locally by contributors. We look forward to increasing the number of contributors to distribute the load of the overall process. 8) Resulting software will be open-source, enabling a do-it-yourself kit for local statistics gathering.