Skip to main content

Methodology for Passive Analysis of a Commodity Internet Link

Nevil Brownlee, University of Auckland, Auckland, New Zealand
KC Claffy, & Margaret Murray, CAIDA at San Diego Supercomputer Center, University of California, San Diego
Evi Nemeth, University of Colorado, Boulder, Colorado

Passive monitoring of Internet links can provide important data on a variety of Internet performance parameters. We use two publicly available monitoring tools, exploring their synergy and relevance for collecting and analyzing Internet flow data. Our passive network measurements involve snooping packet headers on a link to gather information for traffic analysis rather than actively injecting packets to gauge network behavior. We use a CoralReef monitor to gather link packet data and the NeTraMet real-time traffic flow measurement (RTFM) meter to filter the packet stream and analyze flows of interest. Both tools focus on information in the packet headers, not the payload contents.

In our campus environment, all commercial Internet traffic to and from the University traverses one commodity ATM OC3 link, rate-limited to 20Mbps. While this Internet link is not the only path off-campus, it provides a good source of Internet data for research and operations monitoring. Campus personnel responsible for network operations can use the CoralReef / NeTraMet measurement methodology for both real-time incident detection and long range capacity planning. Internet researchers find such data valuable for empirical validation of models and assumptions, and for generating realistic input to network simulators. We use this methodology for several case studies concerning the campus commodity Internet connection. As expected, total traffic on this link shows the clear effect of student workload trends as our measurements span the end of a semester and beginning of summer break. More surprising are results showing:

  1. high loss rates in DNS flows to root name servers, suggesting that the robustness of DNS is masking significant congestion based packet loss, and
  2. TCP flows, though generally longer than UDP flows, are still quite short: over 75% contain fewer than 10 packets and fewer than 2 kBytes. NeTraMet's RTFM approach to collecting traffic flow data allows flexibility in defining flows, using a high-level language to configure a traffic meter. As much data reduction as possible is performed on the meter. The data is then post-processed with Perl scripts to produce graphs. NeTraMet is currently used by ISP operations personnel and university network administrators mainly to collect traffic data for billing and network engineering purposes, but our investigation demonstrates its additional utility as a research tool.

As in any measurement project, one must decide beforehand what to measure. One can begin with a hypothesis, then develop a ruleset to collect data that tests that hypothesis. As understanding improves, one can modify the ruleset, and so on. In this kind of study, it can be very helpful to use a Coral monitor to capture a header trace file and test many different NeTraMet rulesets against the same data. Such flexibility makes the CoralReef-coupled version of NeTraMet of tremendous value for research.