Sensitivity Analysis of Event Driven Simulation Results
Murray Pearson and Tony McGregor
The University of Waikato, Hamilton, New Zealand
One use of passive measurement is to provide traces that can be used as
the input to event driven simulation. Such simulations have been used to
study a wide range of network problems including strategies to achieve good
performance over high bandwidth delay satellite[1], terrestrial[2] and
asymmetric satellite/terrestrial[3] international links, without requiring
users to upgrade or tune their TCP stacks.
The simulator used in these studies was based on the ATM-TN
simulator[4]. ATM-TN's TCP model includes the actual TCP code from 4.4 BSD
Lite, modified to suit the simulation environment. Connections are
simulated on a packet by packet basis and including slow start, congestion
control, fast retransmit, and fast recovery algorithms.[5]
The simulation can be considered as a number of connected components.
These are the HTTP requesters, the local proxy, the international link,
including the routers that feed it (which is the key component under
study), the US network and the HTTP servers.
An HTTP traffic model is responsible for creating TCP connections,
sending HTTP GET requests, receiving the request at the destination and
returning and results and for recording the time required to complete the
HTTP requests. The HTTP model makes use of information about hosts and a
URL trace collected from a real network using passive measurement.
The delays in the US part of the network are simulated by the HTTP
traffic model which releases the packets that make up the HTTP response at
a regulated rate so that the complete response arrives at the US proxy at
the same mean rate as applied when the page was fetched over the real
network.
Simulations, by their nature, are approximations. Some details of the
real system are intentionally omitted to make the simulation more
manageable. For example it is possible to drive a simulation from a trace
of high level protocol events (e.g. HTTP requests) or from a packet level
trace. In the case of TCP packet control level behaviours (such as slow
start) are important. They are not always important with UDP based traffic
such as a game or mpeg replay. It is part of the art of simulation to omit
details that are difficult to provide but make little difference to the
final result while maintaining the essential elements of the situation
being modelled. In trace-driven simulation we want to omit details that
are difficult to collect. This might be either because they are not readily
available in passive traces, or because they require a lot of resources to
collect, store and simulate. Unfortunately it is not always obvious which
aspects of a simulation can be omitted without unduly affecting the result.
In this paper we examine the sensitivity of the simulation results
(described above) to the following parameters:
- Connection establishment and termination
Sometimes it is only possible to capture parts of a connection
in a trace. This is particularly true for long lived
connections. This parameter compares the results with and without
the overhead of connection management.
- The effect of overlaying workloads
In many communications contexts, burstiness exists at all layers
of traffic abstraction. As a consequence of this, it is not
always valid to add together multiple copies of the same
workload to generate a higher load. We describe the effect of
layering identical workloads, with and without a time offset,
compared with a single higher bandwidth workload.
- Distribution of arrivals from feeder networks
A great deal of complexity is contained in the structure and
performance of the network that collects data to be carried over
the link under study (in these studies these are the local and
the US network). Under this heading we show the result of
substituting a more detailed model with several alternative
simple arrival distributions.
- MSS
While it is well known that MSS has a significant affect on the
performance of an individual TCP connection it is less clear how
it affects the overall aggregate performance of a link that is
carrying many TCP connections.
- TCP mechanisms.
The effect of slow start, fast recovery and fast retransmit on the
simulation results are presented.
- Transmission Errors.
Most connections have very low error rates but there are some
important exceptions, including satellite and wireless networks.
The goal of this work is to guide the selection of parameters to be
included in a measurement program that will feed event driven simulation.
In most cases our results indicate that simulations of the behaviour of
links with composite traffic, made from many simultaneous TCP connections,
are not sensitive to variation of the parameters described above. The main
exception is the "TCP mechanism class". There are many possible event
driven simulations and the needs many of these simulations vary. However,
many simulations have elements in common with ours and, to that extent, we
believe our results will be generally useful.
- Pearson M. and McGregor A. "A simulation study of network
architectures to support HTTP Traffic on Symetric high-bandwidth*delay
circuits" Proceedings of the Asia Pacific Web Conference,
pp19-25, 2000.
- Pearson M.W., McGregor A., and Cleary J.G. "Reducing US/NZ Web
Page Latencies" Networks99 pp 54 - 63, Janurary 1999.
- A. McGregor, M. Pearson and J. Cleary\\ "The effect of
multiplexing HTTP connections over asymmetric high bandwidth-delay
product circuits" SPIE Photonics East 1998: Routing in the Internet:
New Concepts. pp 398-401, November 1998
- M. Arlitt, Y. Chen, R. Gurski, and C. Williamson, "Traffic modeling
in the
ATM-TN TeleSim project: Design, implementation, and performance
evaluation", in Proceedings of the 1995 Summer Computer Simulation
Conference, (Ottawa, Ontario), July 1995.
- W. Stevens, "TCP slow start, congestion avoidance, fast retransmit,
and
fast recovery algorithms", Tech. Rep. RFC2001, IETF, Jan. 1997.
|
