<<< Chronological >>> Author Index    Subject Index <<< Threads >>>

*IX administrative and technical frameworks discussion paper.

  • From:
  • Date: Sun, 23 Jan 94 13:40:46 +0100

To the coming EEPG meeting in Amsterdam I have tried to address some
of the administrative and technical aspects of traffic exchange
installations. Find a very draft paper below. It shall be clearly
pointed out that this paper is far from complete. I have tried to use
experiences from EBONE and PIPEX (Peter Dawe) to create some ideas on
what can be said about this such installations. It is my hope,
although very late distributed, that we can initiate an improvement of
the text at the EEPG meeting.

See You all in Amsterdam,

Bernhard.

=============================================================================
DRAFT   DRAFT   DRAFT   DRAFT   DRAFT   DRAFT   DRAFT   DRAFT   DRAFT   DRAFT
=============================================================================

    Some aspects of the administrative and technical models for
    installation of Internet traffic exchange points.

    Bernhard Stockman
    January 23, 1994

 1. Introduction

    With the event of multiple Internet Service Providers (ISP) where an
    ISP is defined as giving full global Internet connectivity, there is
    an obvious and urgent need to formalize administrative and technical
    models for implementing inter-connectivity between ISP's.

    Such traffic exchanges have been installed in various parts of the
    Internet. For example the NSFnet DMZ's providing interconnectivity
    between the NSFnet backbone and the US regional networks, the FIX's
    and the CIX. In Europe there are de facto installed traffic
    exchanges in several locations such as Amsterdam, Geneva, London,
    Paris, Stockholm and Vienna.

    It is believed a more distributed model will be needed with traffic
    exchanges on local, regional and global level to facilitate for a
    continued high quality of service and a manageable routing
    environment between ISP's.  There is today no technical or
    administrative models defining the requirements for traffic exchange
    points. This paper tries to address some administrative and
    technical aspects both within a single traffic exchange installation
    and in a distributed system of traffic exchanges.

    This paper is clearly a discussion paper and should not be seen as
    anything more than that. It is the hope that it will create
    discussions within relevant groups which eventually will result in
    commonly accepted specifications of administrative and technical
    models.

    Throughout the paper the acronym IX will be used to describe any
    kind of Internet traffic eXchanges.


 2. An administrative model for IX installations

    The most straight forward situation is that a group of ISP's forms a
    joint venture for the provision of interconnectivity among
    themselves. Later more organizations may join as full members of the
    group or as purchaser of the interconnectivity service from that
    group.

    An administrative model should cover issues like

    - management of an IX installation
    - general cost principles and models
    - changes within an IX installation
    - resolution of IX member conflicts
    - legal aspects
    
 2.1 Management of an IX installation

    Is there a need to have a dedicated legal entity defined taking the
    responsibility and, if so, how is this implemented? If not, where
    will decisions regarding an IX installation as such be taken?

    In general the management of an IX has to guarantee that there is
    
    - no discrimination between users.
    - no discrimination of traffic.
    - no discrimination between carriers

    The IX managers must facilitate the transfer of management to
    another body when required

    The area of responsibility for the IX installation as such will
    include all equipment commonly used up to the connectors interface
    to this equipment. That is the physical media as such and commonly
    used hard- and software within the IX installation such as route
    servers, statistical servers, etc. There is also a need to
    facilitate for connectors by having agreement on simple operational
    tasks being undertaken by the IX management on behalf of IX
    connectors. Such operational service could be undertaken by the
    operational staff from one of the owners of the IX installation or
    outsourced as necessary.

 2.2 General cost principles and models

    The joint venture for the provision of IX service is a not for
    profit organization (?). The opposite may create a situation with
    inflation of low quality IX installation and by that be
    counterproductive to the IX general ambitions of high quality of
    service and manageable routing between ISP's.

 2.2.1 Pricing principles

    IX facilities must be provided at the lowest cost but sufficient for
    the required interconnect bandwidth ( Ethernet, FDDI, Switched
    Ethernet, backplane,...) as much of the attributable costs to a
    connection are already paid for by the connector.

    A problem here is to define what should be intended with lowest cost
    and how this is implemented without endangering the provided
    interconnectivity service. To be able to define an acceptable cost
    it will be necessary with an adequate and rather detailed basic
    requirement specification for an IX installation.

 2.2.2 Cost recovery models

    There exist today a variety of ideas on how to charge for Internet
    services. Which model(s) should be used for IX connectivity?

    - free of charge 
    - voluntary contribution,
    - a levy, flat fixed rate,
    - a levy, flat rate depending on e.g. connected bandwidth?

    Should only serial connections to an IX be allowed? If not, how is
    the problems with LAN connected members solved with respect to
    connected bandwidth equivalence? If a flat fixed fee is used this
    problem will not occur as everyone will be free to install whatever
    bandwidth seen as necessary for his needs.  On the other hand this
    might give rise to situation where the commonly used IX equipment
    will not be able to handle the total amount of traffic being
    injected. That is, how is a situation when an IX connector injects
    90 percent of all traffic by having much more connected bandwidth
    than anyone else and by that overloading the entire IX system where
    all its members pays the same fixed fee?

 2.3 Changes within an IX installation

    At some point in time it will be necessary with changes of installed
    commonly used equipment such as the LAN, route servers, other
    servers, etc. It may be useful to have an understanding on how such
    changes should be managed within existing cost models.

    An obvious situation is where the IX installation is not sufficient
    for injected traffic and there is a need either to limit the amount
    of injected traffic or to upgrade the system. There might also be
    other situation where additional equipment has to be installed for
    new type of requirements not known today.

    In general there shall be a model on cost sharing in an already
    installed system when new costs are to be covered.  In the simplest
    case the additional costs will be divided equally among the owners
    of the IX installation and higher charges will be given out to
    connected ISP's to cover the increases in costs for the owners. A
    keyed system of cost recovery is also a possibility where the keys
    may be dependent on e.g. connected bandwidth.

 2.4 Resolution of IX member conflicts

    To be able to function also under conflict conditions there is need
    to define a framework for problem resolution.  In simple cases IX
    members should be able handle this between themselves. If not, there
    may be the need to escalate to a general IX arbitration authority.
    (See also under section XX below). How such an authority has to be
    defined and installed to be acceptable to a majority of the today
    ISP's is an open question.

 2.5 Legal aspects

    To avoid legal problems with for example anti-trust laws it will be
    necessary to formally declare every IX installation as being fully
    open to all ISP's as described in section XX above.  There might as
    well be other legal concerns that has to be covered.


 3. Technical and operational requirements of an IX installation

    There is a need to guarantee an extremely stable service. How is
    this achieved? For stability reasons there are certainly also
    technical requirements on connecting service providers.

 3.1 Physical requirements

 3.1.1 Power supply

    The IX installation delivers 1-phase voltage according to approved
    energy distribution system within the country where the IX is
    installed.

    The primary power feeder for the entire IX installation (including
    all external components such as terminals and other systems in
    direct communication like transmission equipments) should be unique
    to the IX system. Therefore, power for all components should be
    derived from the same power distribution panel.  All equipments must
    be grounded to the same single point.

    If, for any reason, this cannot be implemented, provide special
    signal interference handling whenever a communication path goes
    between two devices powered from different feeders. Special signal
    interface handling may include opto-isolation, or any means which
    maintains logic reference isolation between the communicating
    components.

    The IX installation shall be powered from uninterruptable power
    source (UPS) able to support the IX installation for no less than a
    6 hours outage of commercial power.

    It shall be possible to bypass the UPS and remove it for maintenance
    without interruption the IX service.

    The IX installation current drained through a fuse shall be less
    than 50% of the rated fuse value.

 3.1.2. Air Conditioning and Cooling Recommendations.

    The room housing a IX installation must be equipped with sufficient
    air conditioning capability effective enough to meet the
    environmental specification for both the hottest and coldest day of
    the year.  In case of malfunctioning, a backup air conditioning
    system shall be provided. In case of centralized water cooling
    systems the building must have sufficient emergency backup or an
    alternate source of cooling water shall be used. To secure cooling
    capacity also during commercial power outage all cooling equipment
    shall be powered via UPS with no less than 6 hours capacity.

 3.1.3. Fire protection.

    As the rules are different from country to country, no general rules
    could be set. A non-destructive extinguishing system is preferred.
    Examples of non-destructive extinguishing system are carbon dioxide
    and bromotriflouromethane (halon 1301).

 3.1.4. Physical security.

    The area surrounding the IX installation shall provide such physical
    protection that it is unlikely that it may be sabotaged or
    manipulated by unauthorized personnel.

 3.1.5 Mounting of equipment

    As much as possible of the needed IX equipment shall be placed in
    the same rack.  The goal is to make the IX installation as compact
    as possible to minimize the risk of interruptions caused by accident
    while working on other equipment housed in the same location.

    The IX equipment shall be installed in 19-inch cabinets which have
    both front and back doors that can be closed. There shall be enough
    air-flow through the cabinet to take away all the heat produced by
    the housed equipment with the doors closed.

 3.1.6  No single point of failure

    To be completed.

 3.1.7  Full operational coverage

    An IX installation shall be operational 24*365, that is it shall
    provide access at 24*365 and provide operational staff at 24*365.
    In case of malfunctioning, repair and/or replacement of
    malfunctioning equipment belonging to the IX installation or, by its
    placement, having impacts on the IX service performance, shall be
    completed within 4 hours.

 3.1.8 Multiple telecom entry points

    To minimize problems with links delivered by carriers to the IX it
    shall be requested that physically different paths are being used to
    provide requested link connectivity.

 3.2 Traffic capacity requirements

    As the IX installation will serve many ISP's with contracted
    services to its customers the IX itself may not for a bottleneck of
    interconnectivity between ISP's. There is hence the need to define
    minimum capacity of the IX installation as such. An obvious
    parameter in the calculation is the sum of incoming bandwidth. For
    pure broadcast LAN's the total sum of incoming connection should not
    exceed 50 percent (?) of the IX media total capacity. For other
    types of interconnectivity equipment other requirements may have to
    be defined (e.g. ether-switches, fddi-switches).

 3.3 Routing requirements

    IX common service must support basic routing requirement to be able
    to interoperate with IX member equipment to provide a stable and
    manageable routing environment. This means that an IX route server
    has to support

    - modern policy based routing protocols (today BGP-4 and IDRP), 
    - full routing (it must be allowed to default within the IX
      installation), 
    - promptly updates from routing registry databases,

    Commonly used routing equipment must be scalable in the sense that
    it shall be able to handle the amount of routing in the today
    Internet both in terms of cpu, memory and disk capacity.  This is to
    a large extent depending on the specifications for such equipment
    which is outside the scope of this paper. For more information on
    this see relevant papers on the route server concept.

 3.4 Statistical requirements 

    There is an overall need to have an understanding of the major
    traffic flows within the Internet. To facilitate for such knowledge
    it is requested than an IX installation makes commonly agreed
    statistical data available in a commonly agreed format.  [See also
    RFC 1404]

    The IX shall thus make available statistical data of general
    interest for the Internet without violating legal concerns of
    privacy among connected networks. Depending on the owners of an IX
    installation this could be done in a couple of ways.

      - the IX owners decides to install a commonly used statistical 
        server and the management of the IX is chartered to make
        agreed statistical data available.

     - the owners of the IX installation makes such data available
       derived from there own systems. In this case it must be possible
       to merge such data to achieve a total view of the IX traffic.

    Such statistic may also prove useful for the owners and connectors
    to understand the load behavior within the IX installation as such.

 3.5 Other requirements

    There may be situations when there is a need for an optimized basic
    network application service that could be facilitated by the
    installation of certain hard- and software at an IX.  Obvious
    examples are TTS server, NTP servers, MBONE servers, DNS servers,
    general network information servers, etc. (See also Aggarval for
    basic network services).

    As long as such services could not be seen as being in competition
    with IX member service providers but delivering a service of mutual
    benefit for all IX members this should be supported.


 4. The distributed system of IX's and its administrative and technical 
    requirements

 4.1 Who looks after the IX managers

    Is there a need for a dedicated body to specify the IX managers
    responsibilities and ensure they are fulfilled and to appoint
    replacement managers when necessary.

    Such authority may also provide the arbiter function to aid in the
    resolvation of conflicts that could not be resolved within a certain
    IX installation.

    Is there a risk that in some point in time one IX group or set
    thereof may try to engulf the remaining IX installation and by that
    have total control of all IX installations. If so, is this an issue
    that could be addressed here? by an arbiter authority?

 4.2 Changes in the IX system

    There will be points in time when changes to the IX system has to be
    undertaken which will change the the overall system both technically
    and maybe financially.  How should such changes be done to minimize
    disturbances. The overall responsibility should be to evaluate such
    changes with the intention of maintain or improve overall quality of
    service and manageability of the entire IX system.

    An example:

    - Creation of an IX

      An interconnection point becomes another IX installation when
      more than 60%? of service providers connected to other traffic
      exchange are connected
		
    - Deletions of an IX

      A IX installation ceases when less then 60%? of service providers
      connected to other traffic exchange are connected


 5. Membership and responsibilities for IX connectors

    Anyone who contracts to fulfill the responsibilities of a connecting
    organization shall be allowed to connect to an IX installation.  The
    connector area of responsibility is up to an including the port
    where common IX equipment is connected.  So far it has been stated
    that all network service providers shall be able to connect to an IX
    installation. What is lacking is a knowledge of what should be
    intended with a network service provider in this context.

 5.1 Requirement on member connectivity and traffic

    - Minimum connection 1 year, resign only on 1/2 year boundaries.
    - Must connect to ALL IX's? (locally/regionally/globally?)
    - Connection equipment must interoperate and be safe etc
    - One network causing problems on another must rectify or is disconnected
    - Minimum connection bandwidth 56kbs/256kbs/2Mbs?
    - Installed equipment must not consume abnormal amounts of power
      or produce abnormal amounts of heat. Equipment must comply
      with FCC norms for electromagnetic radiation.
    - Bandwidth between IX and connector must be delivered by recognized
      RPOA. In other cases this has to be approved by the IX management.
    - Traffic must comply with standard RFCs 

 5.2 Requirement on member routing capabilities

    - Mandatory registration of routes in routing registry
    - Agrees to implement decisions of the Routing Authority
    - Must implement full routing (default routing is not accepted)
    - Except in special circumstances e,g, line failure, ISP's will keep
      national traffic with a country and regional traffic within the 
      region. (But if a connector connects with two routers and pay
      double fee ???)
    - Must implement BGP-4

 5.3 Requirement on member network management and security

    - 24*365 member NOC contact (or disconnection on fault?)
    - Active membership of CERT
    - Must contribute funds to relevant routing registry

APPENDIX:

    As a comparison, the working rules for the Washington DC GIX
    installation:

    - only network providers (no end users).
    - any network provider can join.
    - AUP-free on the traffic exchange.
    - any provider can peer (or not peer) with any other provider
      (mutual consent).
    - providers pay for 'their' connect point.
    - should any costs be assessed for the FIX-East connectivity,
      they will be shared on a pro-rata basis (there are no
      costs being charged at this time.)
    - no internal traffic on the traffic exchange.

=============================================================================
DRAFT   DRAFT   DRAFT   DRAFT   DRAFT   DRAFT   DRAFT   DRAFT   DRAFT   DRAFT
=============================================================================



  • Post To The List:
<<< Chronological >>> Author    Subject <<< Threads >>>