*IX administrative and technical frameworks discussion paper.
- Date: Sun, 23 Jan 94 13:40:46 +0100
To the coming EEPG meeting in Amsterdam I have tried to address some
of the administrative and technical aspects of traffic exchange
installations. Find a very draft paper below. It shall be clearly
pointed out that this paper is far from complete. I have tried to use
experiences from EBONE and PIPEX (Peter Dawe) to create some ideas on
what can be said about this such installations. It is my hope,
although very late distributed, that we can initiate an improvement of
the text at the EEPG meeting.
See You all in Amsterdam,
Bernhard.
=============================================================================
DRAFT DRAFT DRAFT DRAFT DRAFT DRAFT DRAFT DRAFT DRAFT DRAFT
=============================================================================
Some aspects of the administrative and technical models for
installation of Internet traffic exchange points.
Bernhard Stockman
January 23, 1994
1. Introduction
With the event of multiple Internet Service Providers (ISP) where an
ISP is defined as giving full global Internet connectivity, there is
an obvious and urgent need to formalize administrative and technical
models for implementing inter-connectivity between ISP's.
Such traffic exchanges have been installed in various parts of the
Internet. For example the NSFnet DMZ's providing interconnectivity
between the NSFnet backbone and the US regional networks, the FIX's
and the CIX. In Europe there are de facto installed traffic
exchanges in several locations such as Amsterdam, Geneva, London,
Paris, Stockholm and Vienna.
It is believed a more distributed model will be needed with traffic
exchanges on local, regional and global level to facilitate for a
continued high quality of service and a manageable routing
environment between ISP's. There is today no technical or
administrative models defining the requirements for traffic exchange
points. This paper tries to address some administrative and
technical aspects both within a single traffic exchange installation
and in a distributed system of traffic exchanges.
This paper is clearly a discussion paper and should not be seen as
anything more than that. It is the hope that it will create
discussions within relevant groups which eventually will result in
commonly accepted specifications of administrative and technical
models.
Throughout the paper the acronym IX will be used to describe any
kind of Internet traffic eXchanges.
2. An administrative model for IX installations
The most straight forward situation is that a group of ISP's forms a
joint venture for the provision of interconnectivity among
themselves. Later more organizations may join as full members of the
group or as purchaser of the interconnectivity service from that
group.
An administrative model should cover issues like
- management of an IX installation
- general cost principles and models
- changes within an IX installation
- resolution of IX member conflicts
- legal aspects
2.1 Management of an IX installation
Is there a need to have a dedicated legal entity defined taking the
responsibility and, if so, how is this implemented? If not, where
will decisions regarding an IX installation as such be taken?
In general the management of an IX has to guarantee that there is
- no discrimination between users.
- no discrimination of traffic.
- no discrimination between carriers
The IX managers must facilitate the transfer of management to
another body when required
The area of responsibility for the IX installation as such will
include all equipment commonly used up to the connectors interface
to this equipment. That is the physical media as such and commonly
used hard- and software within the IX installation such as route
servers, statistical servers, etc. There is also a need to
facilitate for connectors by having agreement on simple operational
tasks being undertaken by the IX management on behalf of IX
connectors. Such operational service could be undertaken by the
operational staff from one of the owners of the IX installation or
outsourced as necessary.
2.2 General cost principles and models
The joint venture for the provision of IX service is a not for
profit organization (?). The opposite may create a situation with
inflation of low quality IX installation and by that be
counterproductive to the IX general ambitions of high quality of
service and manageable routing between ISP's.
2.2.1 Pricing principles
IX facilities must be provided at the lowest cost but sufficient for
the required interconnect bandwidth ( Ethernet, FDDI, Switched
Ethernet, backplane,...) as much of the attributable costs to a
connection are already paid for by the connector.
A problem here is to define what should be intended with lowest cost
and how this is implemented without endangering the provided
interconnectivity service. To be able to define an acceptable cost
it will be necessary with an adequate and rather detailed basic
requirement specification for an IX installation.
2.2.2 Cost recovery models
There exist today a variety of ideas on how to charge for Internet
services. Which model(s) should be used for IX connectivity?
- free of charge
- voluntary contribution,
- a levy, flat fixed rate,
- a levy, flat rate depending on e.g. connected bandwidth?
Should only serial connections to an IX be allowed? If not, how is
the problems with LAN connected members solved with respect to
connected bandwidth equivalence? If a flat fixed fee is used this
problem will not occur as everyone will be free to install whatever
bandwidth seen as necessary for his needs. On the other hand this
might give rise to situation where the commonly used IX equipment
will not be able to handle the total amount of traffic being
injected. That is, how is a situation when an IX connector injects
90 percent of all traffic by having much more connected bandwidth
than anyone else and by that overloading the entire IX system where
all its members pays the same fixed fee?
2.3 Changes within an IX installation
At some point in time it will be necessary with changes of installed
commonly used equipment such as the LAN, route servers, other
servers, etc. It may be useful to have an understanding on how such
changes should be managed within existing cost models.
An obvious situation is where the IX installation is not sufficient
for injected traffic and there is a need either to limit the amount
of injected traffic or to upgrade the system. There might also be
other situation where additional equipment has to be installed for
new type of requirements not known today.
In general there shall be a model on cost sharing in an already
installed system when new costs are to be covered. In the simplest
case the additional costs will be divided equally among the owners
of the IX installation and higher charges will be given out to
connected ISP's to cover the increases in costs for the owners. A
keyed system of cost recovery is also a possibility where the keys
may be dependent on e.g. connected bandwidth.
2.4 Resolution of IX member conflicts
To be able to function also under conflict conditions there is need
to define a framework for problem resolution. In simple cases IX
members should be able handle this between themselves. If not, there
may be the need to escalate to a general IX arbitration authority.
(See also under section XX below). How such an authority has to be
defined and installed to be acceptable to a majority of the today
ISP's is an open question.
2.5 Legal aspects
To avoid legal problems with for example anti-trust laws it will be
necessary to formally declare every IX installation as being fully
open to all ISP's as described in section XX above. There might as
well be other legal concerns that has to be covered.
3. Technical and operational requirements of an IX installation
There is a need to guarantee an extremely stable service. How is
this achieved? For stability reasons there are certainly also
technical requirements on connecting service providers.
3.1 Physical requirements
3.1.1 Power supply
The IX installation delivers 1-phase voltage according to approved
energy distribution system within the country where the IX is
installed.
The primary power feeder for the entire IX installation (including
all external components such as terminals and other systems in
direct communication like transmission equipments) should be unique
to the IX system. Therefore, power for all components should be
derived from the same power distribution panel. All equipments must
be grounded to the same single point.
If, for any reason, this cannot be implemented, provide special
signal interference handling whenever a communication path goes
between two devices powered from different feeders. Special signal
interface handling may include opto-isolation, or any means which
maintains logic reference isolation between the communicating
components.
The IX installation shall be powered from uninterruptable power
source (UPS) able to support the IX installation for no less than a
6 hours outage of commercial power.
It shall be possible to bypass the UPS and remove it for maintenance
without interruption the IX service.
The IX installation current drained through a fuse shall be less
than 50% of the rated fuse value.
3.1.2. Air Conditioning and Cooling Recommendations.
The room housing a IX installation must be equipped with sufficient
air conditioning capability effective enough to meet the
environmental specification for both the hottest and coldest day of
the year. In case of malfunctioning, a backup air conditioning
system shall be provided. In case of centralized water cooling
systems the building must have sufficient emergency backup or an
alternate source of cooling water shall be used. To secure cooling
capacity also during commercial power outage all cooling equipment
shall be powered via UPS with no less than 6 hours capacity.
3.1.3. Fire protection.
As the rules are different from country to country, no general rules
could be set. A non-destructive extinguishing system is preferred.
Examples of non-destructive extinguishing system are carbon dioxide
and bromotriflouromethane (halon 1301).
3.1.4. Physical security.
The area surrounding the IX installation shall provide such physical
protection that it is unlikely that it may be sabotaged or
manipulated by unauthorized personnel.
3.1.5 Mounting of equipment
As much as possible of the needed IX equipment shall be placed in
the same rack. The goal is to make the IX installation as compact
as possible to minimize the risk of interruptions caused by accident
while working on other equipment housed in the same location.
The IX equipment shall be installed in 19-inch cabinets which have
both front and back doors that can be closed. There shall be enough
air-flow through the cabinet to take away all the heat produced by
the housed equipment with the doors closed.
3.1.6 No single point of failure
To be completed.
3.1.7 Full operational coverage
An IX installation shall be operational 24*365, that is it shall
provide access at 24*365 and provide operational staff at 24*365.
In case of malfunctioning, repair and/or replacement of
malfunctioning equipment belonging to the IX installation or, by its
placement, having impacts on the IX service performance, shall be
completed within 4 hours.
3.1.8 Multiple telecom entry points
To minimize problems with links delivered by carriers to the IX it
shall be requested that physically different paths are being used to
provide requested link connectivity.
3.2 Traffic capacity requirements
As the IX installation will serve many ISP's with contracted
services to its customers the IX itself may not for a bottleneck of
interconnectivity between ISP's. There is hence the need to define
minimum capacity of the IX installation as such. An obvious
parameter in the calculation is the sum of incoming bandwidth. For
pure broadcast LAN's the total sum of incoming connection should not
exceed 50 percent (?) of the IX media total capacity. For other
types of interconnectivity equipment other requirements may have to
be defined (e.g. ether-switches, fddi-switches).
3.3 Routing requirements
IX common service must support basic routing requirement to be able
to interoperate with IX member equipment to provide a stable and
manageable routing environment. This means that an IX route server
has to support
- modern policy based routing protocols (today BGP-4 and IDRP),
- full routing (it must be allowed to default within the IX
installation),
- promptly updates from routing registry databases,
Commonly used routing equipment must be scalable in the sense that
it shall be able to handle the amount of routing in the today
Internet both in terms of cpu, memory and disk capacity. This is to
a large extent depending on the specifications for such equipment
which is outside the scope of this paper. For more information on
this see relevant papers on the route server concept.
3.4 Statistical requirements
There is an overall need to have an understanding of the major
traffic flows within the Internet. To facilitate for such knowledge
it is requested than an IX installation makes commonly agreed
statistical data available in a commonly agreed format. [See also
RFC 1404]
The IX shall thus make available statistical data of general
interest for the Internet without violating legal concerns of
privacy among connected networks. Depending on the owners of an IX
installation this could be done in a couple of ways.
- the IX owners decides to install a commonly used statistical
server and the management of the IX is chartered to make
agreed statistical data available.
- the owners of the IX installation makes such data available
derived from there own systems. In this case it must be possible
to merge such data to achieve a total view of the IX traffic.
Such statistic may also prove useful for the owners and connectors
to understand the load behavior within the IX installation as such.
3.5 Other requirements
There may be situations when there is a need for an optimized basic
network application service that could be facilitated by the
installation of certain hard- and software at an IX. Obvious
examples are TTS server, NTP servers, MBONE servers, DNS servers,
general network information servers, etc. (See also Aggarval for
basic network services).
As long as such services could not be seen as being in competition
with IX member service providers but delivering a service of mutual
benefit for all IX members this should be supported.
4. The distributed system of IX's and its administrative and technical
requirements
4.1 Who looks after the IX managers
Is there a need for a dedicated body to specify the IX managers
responsibilities and ensure they are fulfilled and to appoint
replacement managers when necessary.
Such authority may also provide the arbiter function to aid in the
resolvation of conflicts that could not be resolved within a certain
IX installation.
Is there a risk that in some point in time one IX group or set
thereof may try to engulf the remaining IX installation and by that
have total control of all IX installations. If so, is this an issue
that could be addressed here? by an arbiter authority?
4.2 Changes in the IX system
There will be points in time when changes to the IX system has to be
undertaken which will change the the overall system both technically
and maybe financially. How should such changes be done to minimize
disturbances. The overall responsibility should be to evaluate such
changes with the intention of maintain or improve overall quality of
service and manageability of the entire IX system.
An example:
- Creation of an IX
An interconnection point becomes another IX installation when
more than 60%? of service providers connected to other traffic
exchange are connected
- Deletions of an IX
A IX installation ceases when less then 60%? of service providers
connected to other traffic exchange are connected
5. Membership and responsibilities for IX connectors
Anyone who contracts to fulfill the responsibilities of a connecting
organization shall be allowed to connect to an IX installation. The
connector area of responsibility is up to an including the port
where common IX equipment is connected. So far it has been stated
that all network service providers shall be able to connect to an IX
installation. What is lacking is a knowledge of what should be
intended with a network service provider in this context.
5.1 Requirement on member connectivity and traffic
- Minimum connection 1 year, resign only on 1/2 year boundaries.
- Must connect to ALL IX's? (locally/regionally/globally?)
- Connection equipment must interoperate and be safe etc
- One network causing problems on another must rectify or is disconnected
- Minimum connection bandwidth 56kbs/256kbs/2Mbs?
- Installed equipment must not consume abnormal amounts of power
or produce abnormal amounts of heat. Equipment must comply
with FCC norms for electromagnetic radiation.
- Bandwidth between IX and connector must be delivered by recognized
RPOA. In other cases this has to be approved by the IX management.
- Traffic must comply with standard RFCs
5.2 Requirement on member routing capabilities
- Mandatory registration of routes in routing registry
- Agrees to implement decisions of the Routing Authority
- Must implement full routing (default routing is not accepted)
- Except in special circumstances e,g, line failure, ISP's will keep
national traffic with a country and regional traffic within the
region. (But if a connector connects with two routers and pay
double fee ???)
- Must implement BGP-4
5.3 Requirement on member network management and security
- 24*365 member NOC contact (or disconnection on fault?)
- Active membership of CERT
- Must contribute funds to relevant routing registry
APPENDIX:
As a comparison, the working rules for the Washington DC GIX
installation:
- only network providers (no end users).
- any network provider can join.
- AUP-free on the traffic exchange.
- any provider can peer (or not peer) with any other provider
(mutual consent).
- providers pay for 'their' connect point.
- should any costs be assessed for the FIX-East connectivity,
they will be shared on a pro-rata basis (there are no
costs being charged at this time.)
- no internal traffic on the traffic exchange.
=============================================================================
DRAFT DRAFT DRAFT DRAFT DRAFT DRAFT DRAFT DRAFT DRAFT DRAFT
=============================================================================
