Draft minutes RIPE50 TTM session #2
From: Lorenzo Colitti <>
Date: Thu, 12 May 2005 22:42:31 +0200
Apologies for getting Rickard and Niall's names wrong!
Here is a second draft with those corrections made.
RIPE 50 TTM WORKING GROUP MINUTES
Thursday, May 5, 14:00-15:30
Lorenzo Colitti (RIPE NCC) volunteered to scribe.
The minutes of the previous meetings were approved.
AS = Ad Spelt
AT = Alexander Tudor
BN = Brian Nisbet
CP = Christian Panigl
GN = Geert Nijpels
HU = Henk Uijterwaal
LC = Lorenzo Colitti
LD = Luca Deri
NR = Niall O'Reilly
RD = Rickard Dahlstrand
RR = Rainer Rudiger
TS = Thomas Schmid
Henk Uijterwaal: TTM status update
- Internal restructuring done
- DNSMON v1 out, now a self-supporting service. Request list for
- Technical improvements: taildaemon bug fixed, timekeeping
- CDMA clocks: looking for a volunteer to help debug problems
- OWAMP: currently planning implementation
- Other work in progress
- Bandwidth, VOIP measurements, v4/v6 comparison
- Look at Neil O'Reilly's ISP quality measurements
Thomas Wana: Improving Time Measurement in TTM
- Evaluated precision of current TTM setup
- Looked at NTP alternatives
- ATOM is a little better than standard PPS_SYNC and will be used
- ntpns only works on FreeBSD 5
- Improved sending timestamping
- Implemented kernel-level timestamping for TTM sender
- Lower offset, lower variation
- Production ready
- Looked at receiver timestamping
- This is the major source of noise in current TTM
- Not easy to fix, depends on FreeBSD architecture
- Proposal: equip testboxes with DAG cards
- Accuracy is extremely good
- Requires calculating delays centrally
- Results on the testboxes themselves would continue using the
- Would be optional
- A DAG card costs ~800 euros
LD: You should use something like nCap to obtain direct access to
Discussion on Third Party Verified SLA's
AT: It's not clear to the membership what the value proposition of
TTM is. The original idea was that ISP would be interested in
comparing their performance to others, but it was not until very
recently that someone proposed to use TTM in operations.
There is a place for an organisation such as RIPE NCC that could
provide an unbiased metric. ISPs or IX operators could then point to
the data to show that their service is good.
Points for discussion:
- Does it make sense for RIPE NCC to play the role of a trusted
- If so, what do we want it to measure? Loss, delay, etc. are
probably not sufficient? What does it take to turn the service
into something that's useful for members and their customers?
RD: We must distinguish between several markets: service providers,
government, content providers, end users
TS: Let me add "the press"
AT: The press always turns any measurement into a competition or a
race. This is meant to help ISPs. The measurements must be data
that the members can point to, not something to pit one ISP
RR: As soon as you measure something, you are comparing
RD: Compare with DNSMON: it can be used to see which of the three
competitors for .se is best. They don't like the fact that DNSMON
is publicly available. Similarly if TTM data is available that
could be used to make ISPs look bad.
AT: We need to allow the ISP to provide info on SLAs
HU: 1. This is not about monitoring the quality of service of one ISP
(meaningless since there are different packages, SLAs,
overprovisioning, prices...). What is interesting for a customer
is "does ISP X deliver what they promise?"
2. The .se operators might not be happy, but they *are* committed
to provide good service, and if there is an objective tool that
shows that they are not then it is interesting both for users and
for them themselves.
RD: This is still an obstacle. The test-boxes are placed in the network
of the ISPs that they are going to monitor and the ISPs won't want
to host them if they fear the results might make them look bad.
AT: Again this is not about ISP x vs ISP y but measuring ISP x against
their claims. This is not contentious and is good for the service
BN: It's a fine distinction. Users may say "well, I'll move to the
ISP who *does* fulfil their promises"
AT: This is an important concern. But sooner or later users will
become more savvy. Do ISPs want to be prepared or not?
BS: Well, for us the point is that we are not a commercial service...
AT: How many people here are ISPs?
Show of hands: 5
AT: So you ISPs: if you went to marketing and asked to do this, would
AS: In the Netherlands we already have a service that ranks ISPs
TS: You can cheat and apply qos to the addresses that measure quality
AS: We can't: they just buy our products and use them to go to
AT: This is getting close to Neil's talk. Who would be happy with
providing such information to their customers?
RR: We do publish information. Actually, we did the measurements
to prove to the customer that our service was ok and that the
problem was on their side.
AT: So the customer doesn't trust your numbers and wants a third party.
RD: ISP networks are complex. What are we testing? DSL? Distribution?
AT: We don't know. What should we do? You tell us. Those ISPs who say
this is not an idea, why would this not be a good idea?
Reaction from the participants: none
AT: So RIPE NCC is a good party to measure this? It would be
GN: we chose RIPE NCC precisely because it's is a trusted third party
RR: The residential customer does not know who RIPE NCC is.
AT: So you would need to build up the brand
HU: You just say "this is a third party"
RR: If we document exactly how the measurement is done, it doesn't
matter who is doing it.
AT: So you could base your SLAs on standard numbers that TTM measures.
AT: Second point: we agree that RIPE NCC is a good third party to do
this work, but what is this work? What is it that is interesting
for ISPs and their customers to measure?
RD: In Sweden we have 3 types of services that are used to test
bandwidth, etc. We could complement that not with a network of
measurement points (too expensive to deploy many boxes in Sweden)
but by providing a test suite or a tool which could be used by
AT: So we could put a test-box at a public exchange point and people
can sign up for measurements by providing a pingable address.
Would this be acceptable?
LC: How do you know it's not the user's fault if his connection is
AT: You don't...
RD: I think we need something intermediate between the current ttm
network and this solution. Maybe a 100 euro box that could be
deployed where the customers want it? There is no single point in
the network that you can use as reference for measurement
HU: You could provide a box at the ISP's edge and measure from there
to the Internet and from there to the customers. Internet users
installing hardware at home is a logistical nightmare.
AT: We need to walk before we run. First we need to show that there
is community interest in a proposal. So is a public exchange
point a reasonable place from which to use the TTM metrics as
they stand today?
LC: An exchange point LAN doesn't make sense, most of the time they
don't even use globally routed addresses.
GN: I talked to people at Euro-IX and there is interest from other
IXs to use TBs for multi-location L2 exchange point monitoring.
That's a new market.
CP: Operating a reference point at an exchange point is logical, but
the exchange points in europe are very neutral with their
customers (the ISPs). Providing data for ISP's would seem to be
discrimination between ISPs. Also, if we were a reference point
for measuring we would need to answer user requests too.
RD: In Sweden we have a bandwidth measurement project and there is a
reference point for that at Netnod.
AT: The customer doesn't care where the box is. He knows that the box
is operated by a third party and that his ISP has given him a URL
to a page where he can check the measurements. But is an exchange
point a good location?
BN: I don't think the ISP would think the IX was taking sides. But the
exchange is not always the best egress point in the network from
which to measure.
AT: Ok, so to wrap things up:
- We agree that this is an idea that makes sense
- We should continue discussion amongst people who are interested
- Who would like to provide a proposal?
RD, RR, NR volunteer
AT: Ok, so we expect something before next meeting.