Thursday-Amal-2PM

Thursday, 30 October 2008 14:00

Plenary

The Plenary session commenced on the 30th October 2008 at 2 p.m.:

CHAIRMAN: Hello. Welcome to this after lunch Plenary Session. We are doing a couple of changes here the in the meantime, let's go through the programme.

I am chairing this session. However, I'll try to speak as an little as an possible. We have two talks in this session and one speaker.

I am sure you all know him. Lorenzo Colitti, RIPE NCC, now Google. One of the talks, initial talk is based on work by one of his colleagues, but unfortunately he fell ill and could not make it here. So Lorenzo has been kind enough to present in place of his colleague. After that, we have a talk on Lorenzo's work itself. So here you go.

LORENZO COLITTI: Thanks. As an /SKWRO*U said, unfortunately Steinar is not here. It is indeed unfortunately because I don't know as much as about his, about his methodology and data as an he does, maybe he is lurking on the Jabber channels and he'll be able to chip in in case he knows something more.

So, Steinar did some work in order to measure the adoption of IPv6 that we see among users of Google. The focus here is because we don't have good measurements of how much connectivity is out there, we have a few numbers, they are not conducted on large scales. They are all different and there is a general, including inside Google, that turning or IPv6 will break users and we wanted to find out exactly how true that is. Both because of highlighted C and because of middle boxes somehow breaking connections.

So we want to figure out how common IPv6 is. How many IPv6 users would be out there and how many users have v6, but they would break if we turned on a AAAA for Dublin /TKUB.google.com. So the idea of this presentation of this research and the idea of these numbers is to answer the question, what would happen if we put a AAAA for a big website if we published it? How much people how many people boo to it. How many people would get to it suboptimaley and how many people would completely break? That's one of the reasons for example we don't have any statistics on the use of Teredo because the implementations that� the nonTeredo implementations do not, by default, use Teredo if they can get an IPv4 address first. So, from the perspective of content provider, we are obviously not going to turn off IPv4, so Teredo will not be used to connect to a dual stack hose that we'll have.

So, this is the question again: What is the impact, if we were to turn on a AAAA for a /TKUB, dub.google tomorrow, what would be the impact? How do we do it? This is similar to other work that's out there. Various Java scripts implementations. Basically what we do is, we enrol a small traction of Google users, not really Google users, a small fraction of requests more properly. This is less than 1 percent into a sort of socalled IPv6 experiment, so the browser, when it gets to a Google result page, it will fetch, it will be asked to fetch a URL from either dual stack site or an IPv4 only website. And by looking at the  by comparing the number of hits that we see on each, we can see how many users would have  would be able to connect to the dual stack site and how many wouldn't, because if we see a user able to connect to the v4 only website, but not to the dual stack website, it means that his IPv6 connection is somehow breaking his ability. It doesn't work obviously. It doesn't work over v6 but that's not really a problem as long as the browser falls back to v4. More importantly, publishing the AAAA record means that the user is not able to reach website at all. Which is obviously a bad problem. Because we try to maintain a website that works for everyone.

So, a couple more details. The users, it involves users from all data centres but the background request is only sent to a couple of the locations in our network, one is in the US one is in Europe. The request to sign so that you can't just make up one of these requests and attempt to influence our data, even if you happen to see it. So we call the IPv4 and IPv6 addresses, the latency and what kind of user agent it was.

So, without further ado, these are the results as an of a couple of weeks ago I think. So, 0.2 percent of Google users have I think it's more appropriate to speak of IP addresses, anyway 0.2 percent of users have IPv6 and will use IPv6 if you publish a AAAA. So this is not too many, but it's /O higher than other numbers out there that we have seen.

Worryingly, 0.09 percent of users, that's one in 10,000 have broken IPv6 connectivity. So, if you turn on v6 for your website today, those users will not be able to reach you any more.

So, this is a much less accurate figure because the numbers are much smaller. So, this is take it with a grain of salt, but it is non zero. So this is one of the reasons why, and I'll come back to that in my talk later, this is one of the reasons why we can't just say, okay, you can get to Google over IPv6. It's not a question of technical capabilities on or sites.

Steinar also did some interesting research involving I think the birthday paradox. I forget the details of the implementation, but he looked at how many requests happened to come from the same IP and he estimates that there are probably a million, at least a million IPv6 clients out there.

So, we tracked it, we have tracked it over a couple of months. It's growing slowly, but not too slowly. Unfortunately we don't have a long time horizon here, but we'll be maintaining this to see, to track the state of IPv6 connectivity.

Again, these are over a very large sample, so they are good enough for us.

Latency: So, these are  the other question that we asked was what is the impact on user latency to enabling a AAAA on your website because there is not only brokenness, there is also a general worry that IPv6 routing is suboptimal, that there are long tunnels, there are very, very long paths and that's true to some extent and I have more data on that in the following presentation. But, this is seen from the, from the web server perspective.

Now, a couple of caveats here. Number one, as an it says below, this graph is not indicative of Google service latency. Even though it's a v4 host, it doesn't go to all the data centres you would normally go to. It doesn't necessarily go to the right place for you. And it doesn't include any sort of back end processing. It's very random. It includes your browser rendering time. It's not very accurate. But it is, it does give a general picture.

So, basically, the good news is, well maybe it's not you'd expect, is that if you don't support IPv6, then putting a AAAA in a DNS doesn't hurt you. Well, that's good to know. I think it's what we expected. But, it's good to prove to your management that whatever you do, even if you put a AAAA in your website, then you will at most be influencing those 0.2 percent of users that happen to have IPv6. So, if you put a AAAA on your website, nothing will happen to 99.8 percent of you users. So this graph does tell us something.

So, we can't graph IPv4 versus IPv6 latency, because our samples are  because we can't directly compare all the v4 numbers to all the v6 numbers because the v6 numbers are bias and they are only in some networks and they need to remove this bias, so what we do is find pairs of hits from the same IPv4 network by /24 that have both v4 and v6 and we compare those numbers to each other. So basically you have a set of pairs that you can use to estimate the difference in connectivity.

So, what it looks like is this: It's not too bad, but on the other hand, we do see a definite 150ms in the mode in the graph and that is mostly attributable to increased network latency. So, it might not seem like a lot, but of course 150ms is the IT T from Frankfurt to Paulo Alto, so we  this is something that does worry us in general, because we all like our Google searches to come back fast. So, if you look at TCP setup, the minimum time that you'll need to request a web page is a couple of round trip times, because you want the TCP setup and at least packets from the other side. And so you need to multiply this delay by two. It's, in effect, hadding 300 milliseconds to every web page you load. So this is not something that we can immediately do either.

So, other interesting things that Steinar did. Connectivity by weekday. Apparently people use v6 more at home than in the office, well this is what the slide suggests. I don't know what the difference is between Saturday and Sunday, but it might be due to people not always having Sunday as an a holiday here. I am not sure. And of course Saturday is not particularly meaningful unless you know where it is. So, this is I think  I think this is I think Saturday means specific time.

But there is definitely a larger proportion of v4, a slightly larger proportion of v4 on Saturday and Sunday. This is repeatable, so it's probably indicative of people using IPv6 more at home.

By country: So, we look  remember what we have of pairs of v4 and v6 addresses. What we look is we look very v4 address. Gee allocate the user with the best data we have available and then assume that that applies to the v6 addresses as well and do it by country.

Of course, we remove countries that don't have a lot of Internet traffic. Countries like Nigeria, because the sample set is too small that the results vary wildly. So...

Yes, numbers. Russia, I can't explain it. I don't know what the deal is here, but we see lots of hits from Russia.
A lot of stuff in France, the Ukraine, etc., etc.. We don't see evidence of there being an increased uptake of IPv6 in Japan or China, based on this data. They are pretty much sort of close to the average. China is slightly above the average and Japan is slightly below the average. Based on what he have seen.

Here we have a antispoofing graph. The US is not doing too badly despite people saying US doesn't have IPv6. But this is partly at least something we'll come to later.

So, another thing we do is based on the IPv6 address. We try to find out how they get IPv6 access. We can't distinguish native access from tunnel access because they don't look different. We can see if people come in through 6to4 example. We don't look at Teredo again, because the Teredo implementation on Vista prefers IPv4 and we don't have an IPv6 only host. And the reason for that again is that we are trying to determine the impact of adding a AAAA to an existing v4 website.

So, some countries stand out again. US and Canada, 95 percent, 6to4. That's quite high. There is very little native IPv6 in these countries that we see.

France: 95 native. This is possibly the result of the free .fr deployment of well universal deployment of IPv6 to the user. So, we can see that these numbers can be swayed by large deployments. So, it's indicative how much traffic there is out there.

China, mostly native. We see a lot of AS ATAP from China. By the way, if you have any questions I'll do my best to field them, but Steinar might be lurking on the Jabber and also he, I am sure he'll be glad to reply to your emails, and the email addresses are on the slide at the end.

By OS, we see lots and lots of Mac OS. So far and away the highest. We see some Linux, not a lot of Vista and then decreasing from there.

Most Mac users are on using 6to4 and we think that this might be due to the effect of the iPod extreme which automatically configures a 6to4 tunnels if it happens to have a public IPv4 address. So,  and this might also be the result of our large latency increases because in the US at least, 6to4, at least from our point of view, 6to4 is particularly bad. My closest relay at home from comcast is I think in Amsterdam announced by tele too, I think my colleague was going to Chicago and he considers himself lucky. So a 6to4 does have a large latency hit.

So, we see a lot of 52 percent of all IPv6 hits are from Macs using 6to4 and we presume this might be the effect of the airport extreme.

97 percent of all Teredo users are on Windows, we probably, /PHEUR he'd owe and the other implementations don't seem to be very widely adopted.

So, in brief: IP adoption is low, IPv6 adoption is low. It's growing and we are tracking it. There are large variations between individual countries, and this can be heavily influenced by single deployments, for example, the free .fr deployment. This plays into things that I'll say later about how we intend to try to kick start IPv6 deployment.

Steinar says it's not that broken. So, if  I mean, if you don't care about losing one user in ten thousand, you can turn IPv6 on today. If you don't care about breaking users, or breaking the 0.2 percent of users that have IPv6 turned on, then putting a AAAA in your website will have no effect obviously. So that definitely limits the damage. On the other hand, the argument is if there are no users, why would you even bother? So...

The default policy matters a lot. Well this is something we know. Nobody changes defaults.

And what we see is 6to4 is extremely prevalent.

So if you operate a network and you want to make at least a first step towards IPv6, then please set up a 6to4 relay so that your users will get good paths. That will make a huge difference in the latency they will see if they happen to go to an IPv6 website. Which there will be some of in the future and even now. So, if you want to deploy  if you don't want to employ IPv6. Please even just install a 6to4 relay and your users will go much faster.

So, what do we plan to do? We plan to keep this running. We are gathering more data as an time goes by, we'll track adoption. We'll try to figure out why these users are broken and we'll try to fix it. And we'll try to run different experiments to gain more accurate numbers. We might also look into trying to find out how much Teredo is out there, although again this doesn't really impact us in any way.

It would be good to do these experiments on the clients side, but that is something that it is not straightforward to do.

So, with that, if there are any questions, I'll trying to answer them and if not, we can punt them to signer, and his email should be somewhere  at the beginning, I believe. Oh, no.

JIM REID: Lorenzo, I am a little bit confused because if I remember correctly, when you gave a presentation in Berlin, you were saying you  Google didn't want to break their users by breaking AAAA records for things specially dubdubdub.google.com. You seem to be saying the exact opposite of that. That adding a AAAA record is not going to be harmful so just go ahead and do it.

SPEAKER: I don't think I said that. I said that, that it will break 1 in 10,000 of your users if you do that. We don't do that. We will not do that. It will not affect 99.8 percent

AUDIENCE SPEAKER: But you have also got the situation where a host computer might think it's got IPv6 when it hasn't. Of your users because they don't have IPv6. Now that maybe a very small percentage.

SPEAKER: That's the 1 in 10,000.

AUDIENCE SPEAKER: That's me then.

SPEAKER: Sorry?

AUDIENCE SPEAKER: That's me then. I am the one in ten thousand. I am special, I am special. My social worker says it all the time.

SPEAKER: The best brokenness I have seen just for kicks, is my broadband router in Ireland, as an it happens. If you asked it  it was a NAT box and DNS resolver, if you asked it a AAAA question, it would dutifully relay your question to the authoritative server, get back an answer. Take the first four bites of the IPv6 address and put them in an A answer. And my resolver got hopelessly confused. It just timed out. So this is what we are worried about and this is hard to fix. So, the deployments that are likely to be errorfree are those that are very homogenous and those that use technology that doesn't play with v4 too much. For example, ether net, we expect educational networks to be flat topologies, layer 2 with relatively few problems on IPv6. So...

AUDIENCE SPEAKER: Leslie Nagle. Just one quick question for clarification. The numbers you are showing are measured from v4 capable clients towards v4 advertising sites, right? I mean because of the channel you are routing the experting through, so there is nothing to measure the putative possibility of v6 only routes?

SPEAKER: No. This is on the web pages of you know on the results pages for google.com. Although it is  at the moment, v6 only use is not something that we are 

AUDIENCE SPEAKER: I appreciate that. I just wanted to make sure I was clear on the scope.

SPEAKER: That's correct.

CHAIRMAN: Any other questions then? If not, we will continue with Lorenzo.

SPEAKER: Okay. So, this is what the talk on what we have done so far for  oh yes, I have to use it on the web because I need the dancing Google logo. Anyway, this is what, a talk on what we have done so far and what we think is a way forward for a content providers to start experimenting and deploying IPv6 in order to hopefully provide one side of the chicken and egg problem.

So, why are we doing this again? So, when the day comes that there will be the first IPv6 only user on the Internet, Google needs to be there for that user.

That's in the future. Today, if we can serve our users better over IPv6 than IPv4 then we will. There are cases of network that have better v6 connectivity than v4. Some these are educational networks that have separate IPv6 if you think than they for v4 and the v4 is commercial and the v6 is funded by research projects.

We have reports of users that tell us this. And of course, the problem with excessive ports used by Ajax applications, this is quite well known, presented by [] sham I remember could you a, essentially you'd run out of ports on your public IP NAT device. Also developer time being wasted in developing NAT reversal for applications, for peer to peer applications like Google doc. And also we believe that IPv6 is good for the Internet and we want to help it get deployed.

What have we done so far? We have a few websites. We have fixed the IPv6 cache link which previously pointed to an IPv4 address. You can use Google on an IPv6 only address with NATPT. What you can't do is go any of the search results over IPv4 because they are all  because the search results are all IPv4. We have a network we have been trying to be active in promotes IPv6. We have organised a conference internally.

So, the route of the root the /TPHARB /KWEUF lib /RUPL for IPv6 is to do nothing. Because you have no way of unilaterally increasing your position if no one else deploys it. So, you just don't do anything. There is a chicken and egg problem so the user ISP says there is no content. The content provider say there are no users. But, on the other hand, we are heading towards the cliff the so how do we break the cycle?

So, we can try to create a chicken and then maybe the chicken will lay an egg for us. So, if convent provider provide content over IPv6 that providers one side of the equation. It might provide an incentive for clients to move to IPv6. This is better if the content is more desirable that that that's available over IPv4. Why you would do that it open to discussion, but one idea is that if you have  if the economics of your v4 and v6 network are different. Then perhaps you can deliver content more cheaply over v6 or if there is  if there are applications that happen don't work over IPv4, simply because they require too much address space, then you can deliver it over v6 and not over v4.

Unfortunately the problem that I'll be spending most time with here is there is another vicious cycle is that low adoption causes low traffic. Low traffic leads to bad connectivity and bad connectivity hampers adoption the so again we have a vicious cycle that hurts us here.

The basic problem for consent provider such as an ourselves is how do we offer IPv6 content without harming the user experience. That one is ten thousand and the latency hit, we can't do that. Again one in ten thousand users is too many. We have a lot of users. And if you have a problem, the first thing you'll lie to do is go and Google and see how do you fix it. If Google is broken for you, you can't do it that. 150ms RTT penalty doesn't help. It's like going from Europe to the west coast and you know, that really slows your searches down and that's not something we'd like to do.

So, but let's see what actually causes these problems first.

So, what are the problems exactly? So, apart from people not having any v6 at all, okay, that will presumably get fixed at some point. Hopefully. But, if you do have v6, the problem is that many v6 connections are slower than IPv4 and some of them just don't work all together. It's particularly bad if they timeout because if you fail cleanly and fallback to V had that's not so much of an have you. These are protocol problems the v6 is not somehow inherently less reliable that IPv4. It's just less deployed. So all these things that we see here are deployment problems the

Or at least, if it is slightly less reliable, then it's not at the scale of these problems we are seeing. Maybe it's a little which is less material in some implementations but it is, that's in no way comparable to these effects that we see here. So we can neglect those for the time being.

What are the causes, at least the ones that we can see? Long paths. Nonoptimal muting, broken middle boxes, MTU issues, etc..

This is something that we were  this is a path we saw actually trying to get to China from the west coast. This was due to various leaks, various routing policies. But the takehome message here is we do not want to do this to our users, we cannot afford this because our users will have terrible performance and we don't want that.

In the particular case here it was one those long AS paths that Martin commented on earlier that Bernhard Schmidt commented on in RIPE 56. When you have long AS paths you have routing convergence slowly. Your latency is high and it's hard to debug and fix and you have got simply people in the AS paths before you and any one of these might be making a mistake or not filtering or something like that. So these are a couple of examples taken completely at random. They say nothing about the IPv4 err performance of these networks. They are just taken at random. So this one here, okay, so this, these are examples in particular of the problem that Bernhard pointed out last RIPE meeting and it's basically they are routes that get advertised to downstreams and a downstream is then readvertise them to their transits as an customer routes. And so what you end up getting is something like this where you have, so 3257  okay. Sorry, this should say 3 /# 57 peers with 6939. So basically you have, you should have peering between 6939 and 3257, but it's not there because that's not a route that should be announced over a peering but somehow the route survives.

Again 6939 peers most of those ASs down there but they are still seeing those routes.

See Bernhard's RIPE 56 presentation. See Martin's presentation earlier today. Look at the ghost true hunter, there is a lot of these around.

So, again, usually this is because of interdomain routing over the tunnels, the indiscriminate transit and prefixes without real up streams. And the solution to this from our point of view is, well, we don't want these routes. So, we will not take transit. We will  we prefer no connectivity than bad connectivity to these places. Since we are not a transit provider, we can live with partial routing. We can live with the partial routing table. If you want to reach Google. You can reach it over IPv4 and that's not going to change. We provide service to everyone obviously. But in IPv6, we don't need to provide universal service but we do peer with almost everyone, so we see something like 95 or 99 percent of the v6 routing table. So  and if we don't, if we don't see you, you can come and talk to us and we can discuss that. We have an open peering policy.

So, if these ASs with prefixes with circuitous routing, if they peer with us and take proper transits, we'll see them again.

This is nothing about IPv4 performance. This is only at IPv6.

This is another example. You can't see T it's wrapped around. But anyway we peer this ASX in Amsterdam and in Equinix Ashbourne and it so happens that ASX decides to send all traffic to Google through Amsterdam. So the US customers of X, which there are some of. Cross the Atlantic twice if they want to come to Google. And we mailed X and somehow nothing happened. So these problems are hard to fix. We could definitely try harder to fix this problem, but I am sure there are other problems that we are not aware of and the route cause here is that there is basically little incentive for X to fix this problem.

So why is that? If you try to opt  normally in v4 networks where there is a lot of traffic flowing you try to minimise your costs, so you use as few links as an possible. If you have no traffic it doesn't matter what route your packets take because you are not using any real resources to carry them essentially. There is no incentive to fix nonoptimal routing. But to us latency matters, so if you increase your ITT your searches will become much slower and we don't like that because we like to give good performance. And then the point here is that if there are IPv6 networks with end users on them, they have much more incentive to fix these problems and to get good performance than the transit providers that are carrying their routes in general. Because the transit providers don't see any traffic.

So, what are we doing? So, again, we'll try to avoid bad routes by not taking transit. We don't have a v6 transit provider. We do peer with almost everyone. And if we can't see your routes or you can't see our routes, then we can talk, we are happy to peer with you or close to you if we are there already. We have an aggressive rollout and it's mostly user driven, so where we can go to get interconnection, then we will roll out v6 on those routers.

So check peering for /AL db, they are increasing and email peering at google.com.

So the idea is that we peer directly with a user networks as an clothes as an we can. So in that case, we guarantee good service, good quality of services, it guarantees good latency because we try to do the right thing in routing and hopefully the user network is also trying to do the right thing and more importantly since both networks care about this on both sides, the IPv6 issues get reported and fixed. So, this is what we are trying to do going forward.

What else are we doing? We have  we have finally  this is an idea that I think will be useful going onwards. It's the only idea that really we have thought of that might actually help adoption on the content side. So, what this basically means is that if we agree to, and if we think that there is good connectivity to a particular user network, then we can enable IPv6 for that network. You can get dubdubdub, you can get email, gmail, you can get calendar, you can't get YouTube yet. You can tell us which ones you want. We'll see what we can do. The way it works is by looking at the IPv4 address of your resolver. It looks at your v4 address because we don't have any v6 glue and because getting v6 glue is something that's likely to take a long time and it also has latency considerations. It looks at the v4 address of your resolver. If the v4 address of your DNS resolver is in the white list, then it will get AAAA answers for us and therefore your users will be able to get to dubdubdub.google.com on IPv6 and all these other properties. It's working here, this network is whitelisted so if you do host dubdubdub.google.com you should get a Google address and it should still be working. Did you notice? Are you using v6, if you have v6 turned on. Anything that you use on the Google website should be using v6. See if that's happening. Fire up a packet sniffer, do a host look up, it's working here.

AUDIENCE SPEAKER: Dubdubdub does, but google.com doesn't.

SPEAKER: Technical limitation. Actually it's possible to fix that I think based on how  but, yeah, true. Yeah, I won't go into further detail. Till recently, google.com naked domain didn't resolve to anything useful. It now does, but that's a different story.

Anyway in case that wasn't clear, this is how it works. Your user is somewhere here. Your user is in the cloud. So, if you are a normal network, you'll ask for a AAAA and our GNS servers will say there is no AAAA, this is the way we do this by default is because we don't want to break that user in 10,000. We don't want to take the latency penalty to the v6 users. We don't want to harm our /AOESers. If you are in a trust the Ned work, then we'll see the v4 address of your resolver and you say yes, this is a AAAA for dubdubdub.google.com.

So, what are the requirements for in? We need good connectivity and you need good connectivity to us. That's a given. This is the most important part. So, we probably need a couple of peerings in two different places or we need one direct peering and good transit. But we do want direct peering because as an we have seen, transit providers don't always do a good job, depending on who they are. Don't always do a good job of being proactive on these issues and to us latency and reliability do matter. Because this is a production service and it's important to us.

And also, we want  we would like  we want to see a production quality IPv6 on the user network side and we would love to see commitment to fix your users if they break, and let us know if we are broken somehow. We have tested it and it should be working but of course it's relatively new, so please let us know. But more importantly, fix the users if they break. Because driving the one in ten thousand number down is the only way we can get IPv6 adoption on the content side.

Of course there is nothing special about Google in doing this. This is something that anyone can do. I believe that Wikipedia is also doing it already. It took a long time for us to implement T this is definitely something that other people can do obviously.

So, step 2 which is a little more controversial is yes you email us and you save a production quality IPv6 network, we have good connectivity here, please enable this resolver to receive IPv6 answers and we'll say yes, but we can't do a thousand of those and there are 1,200 ASs in the routing table. We can't scale up that way manually. We need a clear signal that somehow tells us, I want IPv6 from Google and I am commited to fixing breakage if it happens. So, the idea, and I am here and you can shoot me down is to use BGP communities for that. And this is the only thing that really I think will be useful in scaling up this experiment from essentially a zero volume test to something that really has a few percent of users and that's critical thing that we have not been able to do so far.

So, the idea is you tag your, the prefixes that your DNS resolvers are in with a community value, this simply because we define anything that starts with 15149. This is something definitely that other people could use. It could be standardised at the ITF, other content providers could use it, it wouldn't be 15169 of course, but we are definitely  it we are definitely willing to collaborate with others and we'd like to see other people use this as well.

So, basically on our side, something will look at the v6 routing table, see whether connectivity is good and automatically put your DNS resolver in the white list. For now, this is probably mean that you will have to be a direct v6 peer with us, but as an v6 connectivity improves, we can work on relaxing that requirement and again we have an open peering policy. So if you have IPv6 user policy, talk to us and we'll work with you.

This is the idea. We are not doing it yet. We will be doing the manual  we will be doing manual whitelisting, so, please let us know what you think.

So a few more thoughts before I conclude. These are some of these are peeves of mine, some of these are things that I heard around here.

IPv6 licensing. So some vendors charge separately for IPv6 support. We think it's a bad idea. Or I think it's a bad idea. Suppose you have to pay 10,000 dollars to enable a router to run IPv6. That's a reasonable price. It's in the region of the list prices that I have seen. So, this is bad for two reasons: Number one, you can't set up a v6 lab because the red tape blocks you. To set up labs on three routers a lap with three routers, you have to get a PO card, you have to get spending approval, budgetary approval, you have to submit the PO and you have to wait and this doesn't make sense. It really blocks initial experimentation and I have said elsewhere that experimentation by enthusiasts is one of the real powers you can tap to get this transition there. There is enthusiasm out there and all you need to do is just don't stand in their way and they'll do it for you. And of course if you have, if it costs you 10 thousand dollars per router, if you have 100 routers, that will be a million dollars please, if you want to hand me a million dollars, I'll take it. So what we are suggesting here obviously is not to, not to you know, dictate pricing strategies, but we are pointing out that if you try separately for IPv6 it will hinder at option and why don't you raise the cost of the base image instead and include IPv6 in the base image instead. The Internet will thank you for that.

The same goes for ISPs and exchanges. There is at least one exchange that we can't deploy at because they have asked us to pay separately for an IPv6 aaddress and that is getting into serious red tape on our side. So, if you want to help IPv6, please include it in your normal prices, don't charge for it separately, because that's the first thing that will get stripped out when some person looks at the bill trying to cut something.

And this is simply because I I can. I'd like to point out, we have seen lots of proposals to maintain backup compatibility with OSs that don't support v6. I am sure this will be controversial, we carry A plus P, C G N, DS light.

Some of the arguments are we have to maintain backwards compatibility with devices that don't support IPv6 and when we asked what those devices are we get said people still have Windows 95 and 98 boxes, so Windows 98 was end of life in July 2006, that's more than two years ago. We looked at our server logs and it said that Windows 95 and ME are about 1 percent of all hits. The RIPE NCC graciously looked at their server logs and it turned out that it was much less than 1 percent. Presumably the audience is much more text savvy and localised in regions where we have more recent software.

So, if it's 1 percent now, what will the adoption of Windows 95 and 98 be in three years when we hit the v4 address cliff? So the question there is: If these NAT appliances are going to cost you hundreds of thousands of dollars, are you sure you want to spend all this money on 1 percent of your users? That's just, far be it for me to dictate business models but this is a question that you might want to ask yourself.

And applications: This is one piece of the puzzle that we haven't touched on. Mostly I haven't talked about it because almost everything Google does runs in the browser and everything that runs in the browser already supports IPv6. But there are applications that don't. It's not as an bad as an you might think but torrent supports IPv6 and that's a major traffic generator. Everything that runs in the browser supports v6 and NATPT can take care of many of the rest of the applications. They don't have to be substantially rewritten.

But the point I'd like to make is IPv6 applications will only emerge when users can use them. This is especially true in the open source world. Why would a developer develop IPv6 support and add IPv6 support to his application when nobody can use it and not even he can use it? So, let's not block roll out of IPv6 to users by saying that there are no applications. Because expecting applications to be there when there is no network is probably not a good idea.

So, with that, that's it. Is there any questions? I am happy to take them.
(Applause)

CHAIRMAN: Thank you Lorenzo.

Daniel: Am I allowed to make a speech other than a question? It's more polite to ask.

When I saw this BGP hack, this is basically making a contract using a BGP community saying I say I have  ISP, say I have a reasonably performing IPv6 network and I am commited to improve it and fix users that are broken. I was very sceptical and I have lots of grey hair and other people with grey hair around hearing the same story were laughing and very sceptical, because it's just overloading and overloading some specific protocol that supposed to be routing with signalling of a contract and the way to turn it off and turn it on. I have to say I have thought about this a little and after I heard this talk and specially after I see that Google is actually taking really positive steps, my scepticism is a lot less. I think this is maybe one dirty hack that we might need.

SPEAKER: Not the IPv6 in itself is particularly beautiful, right. This is the only thing we have. V6 is the only thing we have and I think this community  this is at least the only thing I could think of, but if there are other ideas, please let us know, because we will all need some solution:

AUDIENCE SPEAKER: Lorenzo, thank you very much. This is Neill O'Reilly from UCD. Thank you especially for the slide that encourages vendors to adopt a v6 friendly tariff. This is very important for universities like the one that pays me and probably for lots of other people too. Well done for saying so. And congratulations on saying so so diplomatically.

SPEAKER: You know, it's important for us as well. You can imagine we have a fair number of them and... you know... also, you know, we are screaming at our vendors  well, we are talking to our vendors to fix this and if you would like to do the same, then if v6 is important to you, then that might be a way of helping.

Randy Bush: Great stuff. And I'll even survive the BGP signalling. But what I'm worried about that 1 percent of ME and 95 users is they are in an enterprise that's my customer and so that 1 percent is really affecting a bigger than 1 percent chunk and it's, the people in the enterprise running that chunk probably wear suits.

SPEAKER: Yeah,. I fully appreciate that. I expected it to be  I don't expect it to be a solution but maybe for some ISPs it will be a way up.

Moshen: Congratulations, Lorenzo, for this good job. Have you done any statistics about how many hits you get on IPv6.google.com and where they come from? I am actually, for instance, from those  from free.fr and I have IPv6.google.com on my default web page and I am really keen on being, for some experimentation, so are you suggesting maybe I sign a petition or bring people from free.fr users to sign a petition so that we get this stuff with you?

SPEAKER: I think we are talking to free.fr now. It's one of the largest v6 deployments on the planet. I think we are talking to them. We peer with them already. And as soon as the kinks in the implementation, they should be fixed this week. But as soon as they say kinks are worked out, if you have v6 networking, talk to us, we want to do this. So, yes, free.fr is already on it. To your point you are one of the people that has IPv6.google.com in that you are search box. So do I, but how many of us are there?

AUDIENCE SPEAKER: That was my first question. Have you done any statistics about the hits and their origin?

SPEAKER: No, because we believe that it's not representative of the Internet as a whole. So, it's representative of people like you and me who know that this exists and who bother to change their  and of course, I am sure people would say we don't release numbers, so, that would be a problem as well.

AUDIENCE SPEAKER: Hi, Gert Doering. Congratulations are being  really, bringing this forward and while I share some of the concerns about using BGP as a kitchen sink protocol, it seems to be the only thing that scales in being used for about everything.

The actual question why I stood up was: Is this already operational? Can I use it?

SPEAKER: BGP? No.

AUDIENCE SPEAKER: BGP community announcing to you and get 

SPEAKER: No.

AUDIENCE SPEAKER: What's the time frame you expect this to work? I want to have it, I want to see it.

SPEAKER: I am the one who has to implement it. I don't know. We usually don't make statements about future policies. So it's already a lot that I am saying that we will probably do it. It was also something I'd like to  I wanted to bounce off the operational community first, because I am sure that there are many whose initial reaction would be what are you doing? This is horrible. Please don't do it and if those were the majority, then I would have you know, I wouldn't have tried to push it.

AUDIENCE SPEAKER: Anyway. Consider us, the testers and we are committed to fix problems that happen for our users.

SPEAKER: You can send me mail today and we can do that as soon as 

AUDIENCE SPEAKER: Consider it done before you are done from there.

SPEAKER: Okay.

AUDIENCE SPEAKER: Mike Hughes, Linx. You are talking about v6 on exchanges. This is more of an observation and point of information for the room rather than a question. I don't know why I didn't raise it any earlier this week. But we actually had our first v6 only application to join the exchange. They do not have v4. They do not want to peer with v4, they only want to peer with v6.

SPEAKER: Who are they?

AUDIENCE SPEAKER: Nokia. That's it. Everybody said I question the validity of this application. Well, no, it's a perfectly valid application. V6, it's visible in the global routing table. I just wanted to let people know as far as what people are doing with v6 it's just the observation that springs to mind.

SPEAKER: It may be that some ISPs that are late to the v4 game will find greener pastures in v6, depending on what what content is out there, it might be a cheaper way of handling traffic than v4.

AUDIENCE SPEAKER: Coming back to your charging point, we will charge v6 any normal price that any member would charge for a v4 pool.

AUDIENCE SPEAKER: Do you have any statistics about the adoption of IPv6 in research networks, how many of the hits you get from there?

SPEAKER: In which networks?

AUDIENCE SPEAKER: Research networks.

SPEAKER: No, we didn't look at that particularly.

AUDIENCE SPEAKER: Because I would say that the research community is the first one that adopts these new technologies so it's nice to have a picture where we are right now. This is the first thing that I would like to know.

The next thing is that instead of asking networks to get connected with you in this trust programme, I think it would be more  it would be better to try to do communicate these kind of activities with the community as a general, I mean, to get connected with the TransEuropean research network like in Europe and get from that particular network all the routing information you need in order to provide IPv6 to the rest of NRNs all around Europe. Of course the same for the states or the Asia Pacific research networks. Thank you.

SPEAKER: To your first point. We don't know specific research networks. We wouldn't really know how to classify them except based on IS number. There is also the issue that some of these networks we have no route to any more because we don't  because they don't, some of them don't have proper upstreams so we can't reach them. So that would be difficult at the moment.

So your second point, yes, that's something we could do, but on the other hand, the operational responsibility  remember, that this is  remember that the user side of the bargain is that they have to commit fixing v6 if it breaks. Now if you talk to Jean or I talk to Internet2 or [] ab lean, first of all they don't operate the DNS server that I will get the requests from, that Google will get the requests from, so we can't  they would be effectively be speaking from somebody else's, for someone's DNS server.

And second, they are not involved necessarily, they might be but they might not be in fixing the user networks that will break because breakage usually occurs much closer to the user. Breaker in the core is much rarer. We can definitely reach out to these networks and tell and ask them to tell their members that this opportunity exists but we can't simply wholesale by talking to them only.

AUDIENCE SPEAKER: If I may comment. Does the N C better events actually apply the policies and they are leading these particular  I mean, network, to one specific direction. So, the enter inner events are actually controlling the network. Instead of talking with each of the NRNs separately, you are talking with let's say the representative board and if the deJanuary boards decides that they are going to join this programme, then this means that all the NRNs have agreed on that and they have agreed on the rules and the preconditions that have defined. So, this is my suggestion, okay. Thank you. Thank you for your work.

AUDIENCE SPEAKER: I am coming from Central Europe, speaking for some Central Europe users. First, one more notice for man who spoke before me. It's not only Google task. Your GR net may talk to Dante to peer it to Google. This is another problem. This is not only Google side.

Another thing is that in Central Europe we still have a problem with the transit. For example, in the Czech Republic there are only two networks having dual stack. Everything else is styled or you know, big players but still tunnels. Google is in Czech Republic I know and Google is not in Czech peering centre. If you want to promote Google IPv6 services in the Czech Republic, please come to local peering centre and peer there. Same problem in Budapest and probably in [] paragraph sue you'll, this is I think is not only chicken and egg problem. This is chicken Google egg problem.

SPEAKER: Okay. So if I can reply to that.

AUDIENCE SPEAKER: Sorry, this is just a notice. I don't need an answer. The main message I can raise is, there is another thing which promotes IPv6. Some content providers may need some reliable and not overloaded channel for some crisis situations, for example voting process or, I don't know, some catastrophic, all of us know that CNN is overloaded by plain users and promote these guys to use IPv6 for having such a service for premium people it would be great. It's better than using VPNs or stuff like that.

AUDIENCE SPEAKER: I am Ronan Perry from RIPE NCC monitoring the Jabber channel and Eric Kline of Google says, regarding research networks, the 6to4 still stands out strongly over the native. He goes on to say one would presume that research networks would fall into the native bucket.

SPEAKER: What he said. As an regards the previous question, so the realities of deployment dictate that physical deployment of gear in two POPs is determined by business needs. For us as an the same as an anyone else. So, it's a very different proposition to turn on v6 in a place that we are already present in than to build out a new POP just for the purpose of getting v6 there. I know  I mean, I am not speaking for the peering team but I do think that we have local interconnections in Prague.

AUDIENCE SPEAKER: This is getting a bit unrelated but I want to point out that a couple of eastern Europe ISPs have connectivity and maybe just breached ether net connectivity to the DECIX in Frankfurt and Google is there with IPv6. I have too many hats on my head, don't take this as a DECIX plug, but something to consider how to connect to other IPv6 capable networks.

CHAIRMAN: Thank you very much Lorenzo for your talk.
(Applause).

This is last of the presentations in the regular Plenary. We still have the Closing Plenary and I am inviting Rob, the RIPE Chair, to say something.

Rob: We have 20 minutes left till the official start of the coffee break but we have too much items for the next session, so I don't want to rush them through now. I want to call your attention to one of these items, the United States Government through the Department of Commerce is going through, I don't know what they call it exactly, but they are calling for opinions on a set of possible plans to sign the root of the domain name system. I think that is an extremely important activity and subject. The DNS Working Group has had it on its agenda and comes up with a draft position that they want to get agreement on to put it into the process as our contribution.

I do want to remind you that we, as a community, issued a statement a year ago urging parties involved in root signing to start signing the root. So I think it's important that we follow up this. This is one of the items we will deal with after the break. So, I think we can start the coffee break now and we resume at four o'clock.
Thank you.

(Coffee break)