From brettcarr at ripe.net Mon Apr 2 09:17:37 2007 From: brettcarr at ripe.net (Brett Carr) Date: Mon, 2 Apr 2007 09:17:37 +0200 Subject: [ncc-services-wg] Maintenance on ns-sec.ripe.net Message-ID: [Apologies for duplicates] Dear Colleagues, On 4 April, 2007, authoritative Reverse DNS Services on ns- sec.ripe.net will be unavailable between 19:00 and 20:00 UTC. During this time we need to carry out maintenance work. We apologise for any inconvenience this causes. If you have any questions or concerns about this, please e-mail: dns- help at ripe.net -- Brett Carr Manager -- DNS Services Group RIPE Network Coordination Centre Amsterdam From markguz at ripe.net Tue Apr 3 16:00:22 2007 From: markguz at ripe.net (Mark Guz) Date: Tue, 03 Apr 2007 16:00:22 +0200 Subject: [ncc-services-wg] RIPE NCC Network Maintenance 1700 UTC (1900 CEST) Thurday 5 April 2007 Message-ID: <46125DF6.2000208@ripe.net> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 [apologies for duplicates] Dear Colleagues, On Thursday 05/04/2007, between 17:00 and 19:00 (UTC), the RIPE NCC will carry out planned maintenance work on our core network infrastructure. RIPE NCC services will be temporarily unavailable during this period. We apologise for any inconvenience that this may cause. This work is due to the deployment of a new section of dark fibre between two of our PoPs If you have any questions about this, please send an e-mail to: ops at ripe.net Regards Mark Guz Senior Engineer Operations Department RIPE NCC http://www.ripe.net -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (Darwin) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFGEhqgl42z2L2N9gERAofHAKClFauaor8xhKENLD4dp5PSfN9MFQCfZhCh D1RBdhQbWlox3dtzLzLPvLE= =g9jU -----END PGP SIGNATURE----- From markd at ripe.net Thu Apr 5 11:01:33 2007 From: markd at ripe.net (Mark Dranse) Date: Thu, 05 Apr 2007 11:01:33 +0200 Subject: [ncc-services-wg] Test Traffic Task Force Message-ID: <4614BAED.4090600@ripe.net> [Apologies for duplicate e-mails] Dear Colleagues, At RIPE 52, a task force was formed to discuss the future developments of the RIPE NCC Test Traffic Measurements (TTM) platform. In response to feedback received on the task force list, we have a new draft proposal for network improvements that we plan to distribute for discussion next week. We encourage anyone with an interest in this subject to participate in the task force by subscribing to the mailing list. You can subscribe to the list at: http://www.ripe.net/mailman/listinfo/tt-tf More information about TTM is available at: http://www.ripe.net/ttm The presentation that led to the creation of the task force is available at: http://www.ripe.net/ripe/meetings/ripe-52/presentations/ripe52-tt-future.pdf Kind regards, Mark Dranse Information Services Manager RIPE NCC From markguz at ripe.net Thu Apr 5 16:12:31 2007 From: markguz at ripe.net (Mark Guz) Date: Thu, 05 Apr 2007 16:12:31 +0200 Subject: [ncc-services-wg] RIPE NCC Network Maintenance 1700 UTC (1900 CEST) Thurday 5 April 2007 Postponed In-Reply-To: <46125F03.4080103@ripe.net> References: <46125F03.4080103@ripe.net> Message-ID: <461503CF.3040204@ripe.net> Dear Colleagues, Due to the fact that the RIPE NCC will be closed tomorrow and Monday we have decided to postpone this maintenance until next Wednesday (11/04/07 ). Apologies for the inconvenience this may cause. If you have any questions please send them to ops at ripe.net Kind Regards Mark Guz Senior Engineer OPS/IT RIPE NCC Amsterdam From markguz at ripe.net Thu Apr 12 14:25:14 2007 From: markguz at ripe.net (Mark Guz) Date: Thu, 12 Apr 2007 14:25:14 +0200 Subject: [ncc-services-wg] RIPE NCC Network Maintenance Tuesday April 17 at 17:00 UTC Message-ID: <461E252A.4090909@ripe.net> [apologies for duplicates] Dear Colleagues, On Tuesday 17/04/2007, between 17:00 and 19:00 (UTC), the RIPE NCC will carry out planned maintenance work on our core network infrastructure. RIPE NCC services will be temporarily unavailable during this period. We apologise for any inconvenience that this may cause. This work is due to the deployment of a new section of dark fibre between 2 of our PoPs, which was aborted on April 12. If you have any questions about this, please send an e-mail to: ops at ripe.net Regards Mark Guz Senior Engineer Operations Department RIPE NCC http://www.ripe.net From BSanghani at flagtelecom.com Thu Apr 12 17:50:37 2007 From: BSanghani at flagtelecom.com (Sanghani, Bijal) Date: Thu, 12 Apr 2007 16:50:37 +0100 Subject: [ncc-services-wg] RIPE 54 Agenda Items for NCC Services WG Message-ID: <147955311569D511AD0D00508B6675300577DC4B@lon-emailcl.flagtelecom.com> Hi all, We are currently compiling the draft agenda for the RIPE NCC Services Working Group for RIPE 54 in Tallinn. If anyone would like to suggest a presentation or raise a topic for discussion please let either Kurtis or me know and we'll arrange a spot on the agenda. Regards, Bijal ********************************************************************** This e-mail message is confidential and is intended only for the use of the individual or entity named above and contains information which is or may be confidential, non-public or legally privileged. Any dissemination or distribution of this message other than to its intended recipient is strictly prohibited. You should not copy it or use it for any purpose nor disclose the contents to any other person. If you have received this message in error, please notify us by email to postmaster at flagtelecom.com immediately and delete the original message and all copies from all locations in your computer systems. This e-mail has been swept by Mailsweeper TM for viruses. However, FLAG Telecom cannot accept liability for any damage which you may sustain as a result of software viruses. ********************************************************************** This message has been scanned for viruses by MailControl - www.mailcontrol.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From markguz at ripe.net Mon Apr 16 17:32:40 2007 From: markguz at ripe.net (Mark Guz) Date: Mon, 16 Apr 2007 17:32:40 +0200 Subject: [ncc-services-wg] RIPE NCC Maintenance Wednesday 18/04/2007 http://e-learning.ripe.net 1700 UTC Message-ID: <46239718.2060804@ripe.net> Dear Colleagues, On Wednesday 18/04/07 between 17:00 and 19:00 UTC we will be renumbering the website http://e-learning.ripe.net. This will result in a small outage during the posted maintenance window while the new IP address filters through DNS. We apologise for any inconvenience this might cause. As usual, if you have any questions please forward them to ops at ripe.net Regards Mark Guz Senior Engineer OPS/IT Department RIPE NCC http://www.ripe.net From denis at ripe.net Tue Apr 17 17:21:19 2007 From: denis at ripe.net (Denis Walker) Date: Tue, 17 Apr 2007 17:21:19 +0200 Subject: [ncc-services-wg] Updated RIPE Database Manuals Message-ID: <4624E5EF.9090404@ripe.net> [apologies for duplicates] Dear Colleagues, We are pleased to announce the publication of three updated RIPE Database Documents. ripe-401 RIPE Database Query Reference Manual ripe-402 RIPE Database User Manual: Getting Started DRAFT RIPE Database Update Reference Manual Previously, the Query and Update Reference Manuals were part of ripe-252, The RIPE Database Reference Manual. The content has been thoroughly revised and updated, and we have also improved the wording to make it clearer. The Update Reference Manual also contains a lot of new content, reflecting many of the changes made to the database software over recent years. Because it is so long since this manual has been revised, the list of changes is considerable. It has not been possible to reflect all these changes in the first revision. This is why it is only released as a draft document at this stage. A second revision will take place after the RIPE 54 Meeting. Together, the Query Reference Manual and Update Reference Manual (currently in draft format) will completely obsolete ripe-252. You can find ripe-401 and ripe-402 at: http://www.ripe.net/ripe/docs/database.html The draft of the RIPE Database Update Reference Manual is available at: http://www.ripe.net/ripe/draft-documents/draft_db_manual.pdf Regards Denis Walker Database Group RIPE NCC From denis at ripe.net Wed Apr 18 13:34:54 2007 From: denis at ripe.net (Denis Walker) Date: Wed, 18 Apr 2007 13:34:54 +0200 Subject: [ncc-services-wg] Clean up of unreferenced person/role objects Message-ID: <4626025E.4020909@ripe.net> Introduction ------------ According to our database consistency statistics program (dbconstat [1]) we currently have 460,573 unreferenced person/role objects [2]. Some of these may be maintained, but are still unreferenced. Any personal data not referenced by Internet resources do not fit within the purpose of the RIPE Database. They should not be stored in the RIPE Database beyond a reasonable 'work in progress' period. The RIPE NCC has had a mandate to delete these unreferenced person/role objects since RIPE 40: http://www.ripe.net/ripe/wg/db/minutes/ripe-40.html (2001) At RIPE 41, the Database Working Group agreed that "maintained objects will now be removed" and "gave a mandate to the RIPE NCC to continue with the cleanup process." http://www.ripe.net/ripe/wg/db/minutes/ripe-41.html (2002) The cleanup process of 2003 (http://www.ripe.net/db/news/unref-cleanup-200304.html) involved using a script to run periodic cleanups. This script was put in place. However, it failed about 18 months ago. Because of other priorities, we have not had the time to examine this issue again until now. The graph showing the increase in these unreferenced person objects [3] indicates that the cleanup script appeared not to be performing correctly. At the start of 2006, there were about 300,000 unreferenced person/role objects. Since then there has been a steady increase with a large increase of almost 50,000 in February 2007. The graph also shows a slightly higher rate of increases in these objects since February 2007. Because of the high number now of unreferenced person objects, we want to raise the issue with the community again and propose the below procedure to clean them up. Because redundant personal data is a serious data protection issue, we want to take a new approach to this in the future. Once the initial cleanup is in progress, we will create a new proposal for a new, regular cleanup procedure. This will be sent to the Data Protection Task Force [4] and then to the rest of the community. One time bulk cleanup procedure ------------------------------- Month 1 * Select 80,000 unreferenced person/role objects. Month 2 * Check selected person/role objects. * Those still unreferenced: o Delete using normal update process. o 2000 objects per update message. o Run updates overnight (Saturday/Sunday). o One update every 15 minutes. o This should avoid any unnecessary load on the servers. * Select next 80,000 unreferenced person/role objects. Month 3 * Repeat process until complete. Notes ----- This procedure will take about 6 months to clear the current backlog, not including the extra time that may be necessary due to the increases over that period. Because of the high numbers involved, we prefer NOT to send out individual e-mail notifications, either before or after deletion. This is to prevent a high load on our mail servers, especially in the event of a high number of bounced e-mails. This means that there will be no individual announcements to listed e-mail addresses before the deletion and none of the usual update notifications. This also means that even if the objects are maintained, the maintainer will not be notified directly about the deletion of their unreferenced person/role object. We will, however, announce the cleanup to the Working Group mailing lists and as a news item on our web site home page. The worst problem that can occur is that someone will enter a reference to their person/role object just as we delete it. However, as we are only deleting unreferenced person/role objects, the time needed to re-create them is minimal. We suspect that a very large proportion of the unreferenced person/role objects that we will be deleting are abandoned objects that are no longer used. References ---------- [1] dbconstat http://www.ripe.net/projects/dbconstat/index.html [2] current unreferenced person/role objects http://www.ripe.net/projects/dbconstat/html/cons-current.html [3] graph of unreferenced person object increase http://www.ripe.net/projects/dbconstat/cons-unrefpero.html [4] Data Protection Task Force http://www.ripe.net/ripe/tf/dp/index.html From denis at ripe.net Wed Apr 18 14:49:41 2007 From: denis at ripe.net (Denis Walker) Date: Wed, 18 Apr 2007 14:49:41 +0200 Subject: [ncc-services-wg] Re: [db-wg] Clean up of unreferenced person/role objects In-Reply-To: <462605EF.4010405@noc.ntua.gr> References: <4626025E.4020909@ripe.net> <462605EF.4010405@noc.ntua.gr> Message-ID: <462613E5.60600@ripe.net> Andreas Polyrakis wrote: > > > Denis Walker wrote: >> Introduction >> ------------ >> >> According to our database consistency statistics program (dbconstat >> [1]) we >> currently have 460,573 unreferenced person/role objects [2]. Some of >> these >> may be maintained, but are still unreferenced. Any personal data not >> referenced by Internet resources do not fit within the purpose of the >> RIPE >> Database. They should not be stored in the RIPE Database beyond a >> reasonable >> 'work in progress' period. >> >> The RIPE NCC has had a mandate to delete these unreferenced person/role >> objects since RIPE 40: >> >> http://www.ripe.net/ripe/wg/db/minutes/ripe-40.html (2001) >> >> At RIPE 41, the Database Working Group agreed that "maintained >> objects will >> now be removed" and "gave a mandate to the RIPE NCC to continue with the >> cleanup process." >> >> http://www.ripe.net/ripe/wg/db/minutes/ripe-41.html (2002) >> >> The cleanup process of 2003 >> (http://www.ripe.net/db/news/unref-cleanup-200304.html) involved using a >> script to run periodic cleanups. This script was put in place. >> However, it >> failed about 18 months ago. Because of other priorities, we have not had >> the time to examine this issue again until now. >> >> The graph showing the increase in these unreferenced person objects [3] >> indicates that the cleanup script appeared not to be performing >> correctly. >> At the start of 2006, there were about 300,000 unreferenced person/role >> objects. Since then there has been a steady increase with a large >> increase >> of almost 50,000 in February 2007. The graph also shows a slightly >> higher >> rate of increases in these objects since February 2007. >> >> Because of the high number now of unreferenced person objects, we >> want to >> raise the issue with the community again and propose the below procedure >> to clean them up. >> >> Because redundant personal data is a serious data protection issue, >> we want >> to take a new approach to this in the future. Once the initial >> cleanup is in >> progress, we will create a new proposal for a new, regular cleanup >> procedure. This will be sent to the Data Protection Task Force [4] >> and then >> to the rest of the community. >> >> >> One time bulk cleanup procedure >> ------------------------------- >> >> Month 1 >> >> * Select 80,000 unreferenced person/role objects. >> >> Month 2 >> >> * Check selected person/role objects. >> * Those still unreferenced: >> o Delete using normal update process. >> o 2000 objects per update message. >> o Run updates overnight (Saturday/Sunday). >> o One update every 15 minutes. >> o This should avoid any unnecessary load on the servers. > > Hello, > > I suppose you have already foreseen this, but just in case: > > What about RIPE DB mirroring? Is there a chance that this mass > deletion will cause anomalies in the NRTM mirroring? We are using the normal update process. So these updates will be passed out on the mirror stream in the usual way. By spreading the updates over several hours each time it should not cause too much load on the mirror servers. Regards denis > > Regards, > Andreas > >> * Select next 80,000 unreferenced person/role objects. >> >> Month 3 >> >> * Repeat process until complete. >> >> >> Notes >> ----- >> >> This procedure will take about 6 months to clear the current backlog, >> not >> including the extra time that may be necessary due to the increases over >> that period. >> >> Because of the high numbers involved, we prefer NOT to send out >> individual >> e-mail notifications, either before or after deletion. This is to >> prevent a >> high load on our mail servers, especially in the event of a high >> number of >> bounced e-mails. >> >> This means that there will be no individual announcements to listed >> e-mail >> addresses before the deletion and none of the usual update >> notifications. >> This also means that even if the objects are maintained, the >> maintainer will >> not be notified directly about the deletion of their unreferenced >> person/role object. >> >> We will, however, announce the cleanup to the Working Group mailing >> lists >> and as a news item on our web site home page. >> >> The worst problem that can occur is that someone will enter a >> reference to >> their person/role object just as we delete it. However, as we are only >> deleting unreferenced person/role objects, the time needed to >> re-create them >> is minimal. We suspect that a very large proportion of the unreferenced >> person/role objects that we will be deleting are abandoned objects >> that are >> no longer used. >> >> >> References >> ---------- >> >> [1] dbconstat >> http://www.ripe.net/projects/dbconstat/index.html >> >> [2] current unreferenced person/role objects >> http://www.ripe.net/projects/dbconstat/html/cons-current.html >> >> [3] graph of unreferenced person object increase >> http://www.ripe.net/projects/dbconstat/cons-unrefpero.html >> >> [4] Data Protection Task Force >> http://www.ripe.net/ripe/tf/dp/index.html >> > From apolyr at noc.ntua.gr Wed Apr 18 13:50:07 2007 From: apolyr at noc.ntua.gr (Andreas Polyrakis) Date: Wed, 18 Apr 2007 14:50:07 +0300 Subject: [ncc-services-wg] Re: [db-wg] Clean up of unreferenced person/role objects In-Reply-To: <4626025E.4020909@ripe.net> References: <4626025E.4020909@ripe.net> Message-ID: <462605EF.4010405@noc.ntua.gr> Denis Walker wrote: > Introduction > ------------ > > According to our database consistency statistics program (dbconstat [1]) we > currently have 460,573 unreferenced person/role objects [2]. Some of these > may be maintained, but are still unreferenced. Any personal data not > referenced by Internet resources do not fit within the purpose of the RIPE > Database. They should not be stored in the RIPE Database beyond a reasonable > 'work in progress' period. > > The RIPE NCC has had a mandate to delete these unreferenced person/role > objects since RIPE 40: > > http://www.ripe.net/ripe/wg/db/minutes/ripe-40.html (2001) > > At RIPE 41, the Database Working Group agreed that "maintained objects will > now be removed" and "gave a mandate to the RIPE NCC to continue with the > cleanup process." > > http://www.ripe.net/ripe/wg/db/minutes/ripe-41.html (2002) > > The cleanup process of 2003 > (http://www.ripe.net/db/news/unref-cleanup-200304.html) involved using a > script to run periodic cleanups. This script was put in place. However, it > failed about 18 months ago. Because of other priorities, we have not had > the time to examine this issue again until now. > > The graph showing the increase in these unreferenced person objects [3] > indicates that the cleanup script appeared not to be performing correctly. > At the start of 2006, there were about 300,000 unreferenced person/role > objects. Since then there has been a steady increase with a large increase > of almost 50,000 in February 2007. The graph also shows a slightly higher > rate of increases in these objects since February 2007. > > Because of the high number now of unreferenced person objects, we want to > raise the issue with the community again and propose the below procedure > to clean them up. > > Because redundant personal data is a serious data protection issue, we want > to take a new approach to this in the future. Once the initial cleanup is in > progress, we will create a new proposal for a new, regular cleanup > procedure. This will be sent to the Data Protection Task Force [4] and then > to the rest of the community. > > > One time bulk cleanup procedure > ------------------------------- > > Month 1 > > * Select 80,000 unreferenced person/role objects. > > Month 2 > > * Check selected person/role objects. > * Those still unreferenced: > o Delete using normal update process. > o 2000 objects per update message. > o Run updates overnight (Saturday/Sunday). > o One update every 15 minutes. > o This should avoid any unnecessary load on the servers. Hello, I suppose you have already foreseen this, but just in case: What about RIPE DB mirroring? Is there a chance that this mass deletion will cause anomalies in the NRTM mirroring? Regards, Andreas > * Select next 80,000 unreferenced person/role objects. > > Month 3 > > * Repeat process until complete. > > > Notes > ----- > > This procedure will take about 6 months to clear the current backlog, not > including the extra time that may be necessary due to the increases over > that period. > > Because of the high numbers involved, we prefer NOT to send out individual > e-mail notifications, either before or after deletion. This is to prevent a > high load on our mail servers, especially in the event of a high number of > bounced e-mails. > > This means that there will be no individual announcements to listed e-mail > addresses before the deletion and none of the usual update notifications. > This also means that even if the objects are maintained, the maintainer will > not be notified directly about the deletion of their unreferenced > person/role object. > > We will, however, announce the cleanup to the Working Group mailing lists > and as a news item on our web site home page. > > The worst problem that can occur is that someone will enter a reference to > their person/role object just as we delete it. However, as we are only > deleting unreferenced person/role objects, the time needed to re-create them > is minimal. We suspect that a very large proportion of the unreferenced > person/role objects that we will be deleting are abandoned objects that are > no longer used. > > > References > ---------- > > [1] dbconstat > http://www.ripe.net/projects/dbconstat/index.html > > [2] current unreferenced person/role objects > http://www.ripe.net/projects/dbconstat/html/cons-current.html > > [3] graph of unreferenced person object increase > http://www.ripe.net/projects/dbconstat/cons-unrefpero.html > > [4] Data Protection Task Force > http://www.ripe.net/ripe/tf/dp/index.html > -- ========================================= Andreas Polyrakis GRNET/NTUA NOC apolyr at noc.ntua.gr Office: +302107722409 Cell: +306972832445 ========================================= From denis at ripe.net Thu Apr 19 19:04:08 2007 From: denis at ripe.net (Denis Walker) Date: Thu, 19 Apr 2007 19:04:08 +0200 Subject: [ncc-services-wg] Re: [db-wg] Clean up of unreferenced person/role objects In-Reply-To: <20070419152243.GB29847@borg.c-l-i.net> References: <4626025E.4020909@ripe.net> <20070419152243.GB29847@borg.c-l-i.net> Message-ID: <4627A108.7010500@ripe.net> Shane Kerr wrote: > All, > > On Wed, Apr 18, 2007 at 01:34:54PM +0200, Denis Walker wrote: > >> The worst problem that can occur is that someone will enter a >> reference to their person/role object just as we delete it. However, >> as we are only deleting unreferenced person/role objects, the time >> needed to re-create them is minimal. We suspect that a very large >> proportion of the unreferenced person/role objects that we will be >> deleting are abandoned objects that are no longer used. >> > > A bit of thinking can come up with some scenerios that might be > worse(*). > > I definitely think any minor problems are more than outweighed by the > removal of unused personal data! > > -- > Shane > > (*) For example, a worse problem would be referencing the wrong > person/role object. This can happen like so: > > - Person object X created > - Months pass... > - Person object X deleted by this process > - Person object Y created with same "nic-hdl:" as object X > - User who created person object X decides to use it, but actually > refers to object Y (since it has the same "nic-hdl:") > > The user who created person object X assumes it is still fine, because > the usual notification was not received, and in any case it was > probably protected by "mnt-by:". > > Of course, this is an unlikely corner case. :) > This case can already occur: - Person A creates object X without "mnt-by:" - Months pass... - Person B thinks he creates object Y with same "nic-hdl:" as object X - Person B actually modifies object X and changes it into object Y - Person A who created person object X decides to use it, but actually refers to object Y (since it has the same "nic-hdl:") Person A who created person object X assumes it is still fine, because he did not maintain his data and did not include a "notify:" attribute. This is also an unlikely corner case, but the first part does occasionally happen. denis From leo.vegoda at icann.org Thu Apr 19 19:11:25 2007 From: leo.vegoda at icann.org (Leo Vegoda) Date: Thu, 19 Apr 2007 19:11:25 +0200 Subject: [ncc-services-wg] Re: [db-wg] Clean up of unreferenced person/role objects In-Reply-To: <20070419152243.GB29847@borg.c-l-i.net> References: <4626025E.4020909@ripe.net> <20070419152243.GB29847@borg.c-l-i.net> Message-ID: <0194BC2B-64BF-4045-8123-49D427B458B1@icann.org> On Apr 19, 2007, at 5:22 PM, Shane Kerr wrote: [...] > (*) For example, a worse problem would be referencing the wrong > person/role object. This can happen like so: > > - Person object X created > - Months pass... > - Person object X deleted by this process > - Person object Y created with same "nic-hdl:" as object X > - User who created person object X decides to use it, but actually > refers to object Y (since it has the same "nic-hdl:") > > The user who created person object X assumes it is still fine, because > the usual notification was not received, and in any case it was > probably protected by "mnt-by:". > > Of course, this is an unlikely corner case. :) This case could also be avoided by making it impossible to re-use a nic-hdl. I believe that is already the case for organisation objects. Putting it in place for person and role objects would probably be a good thing. But I doubt it's an urgent requirement and I wouldn't want to delay this cleanup. Regards, -- Leo Vegoda IANA Numbers Liaison From shane at time-travellers.org Thu Apr 19 17:22:43 2007 From: shane at time-travellers.org (Shane Kerr) Date: Thu, 19 Apr 2007 17:22:43 +0200 Subject: [ncc-services-wg] Re: [db-wg] Clean up of unreferenced person/role objects In-Reply-To: <4626025E.4020909@ripe.net> References: <4626025E.4020909@ripe.net> Message-ID: <20070419152243.GB29847@borg.c-l-i.net> All, On Wed, Apr 18, 2007 at 01:34:54PM +0200, Denis Walker wrote: > > One time bulk cleanup procedure > ------------------------------- > > Month 1 > > * Select 80,000 unreferenced person/role objects. > > Month 2 > > * Check selected person/role objects. > * Those still unreferenced: > o Delete using normal update process. > o 2000 objects per update message. > o Run updates overnight (Saturday/Sunday). > o One update every 15 minutes. > o This should avoid any unnecessary load on the servers. > * Select next 80,000 unreferenced person/role objects. > > Month 3 > > * Repeat process until complete. This makes sense to me, for a one-time cleanup. > The worst problem that can occur is that someone will enter a > reference to their person/role object just as we delete it. However, > as we are only deleting unreferenced person/role objects, the time > needed to re-create them is minimal. We suspect that a very large > proportion of the unreferenced person/role objects that we will be > deleting are abandoned objects that are no longer used. A bit of thinking can come up with some scenerios that might be worse(*). I definitely think any minor problems are more than outweighed by the removal of unused personal data! -- Shane (*) For example, a worse problem would be referencing the wrong person/role object. This can happen like so: - Person object X created - Months pass... - Person object X deleted by this process - Person object Y created with same "nic-hdl:" as object X - User who created person object X decides to use it, but actually refers to object Y (since it has the same "nic-hdl:") The user who created person object X assumes it is still fine, because the usual notification was not received, and in any case it was probably protected by "mnt-by:". Of course, this is an unlikely corner case. :) From jon at fido.net Fri Apr 20 10:36:14 2007 From: jon at fido.net (Jon Morby) Date: Fri, 20 Apr 2007 09:36:14 +0100 Subject: [ncc-services-wg] Re: [db-wg] Clean up of unreferenced person/role objects In-Reply-To: <20070419152243.GB29847@borg.c-l-i.net> References: <4626025E.4020909@ripe.net> <20070419152243.GB29847@borg.c-l-i.net> Message-ID: On 19 Apr 2007, at 16:22, Shane Kerr wrote: > >> The worst problem that can occur is that someone will enter a >> reference to their person/role object just as we delete it. However, >> as we are only deleting unreferenced person/role objects, the time >> needed to re-create them is minimal. We suspect that a very large >> proportion of the unreferenced person/role objects that we will be >> deleting are abandoned objects that are no longer used. > > A bit of thinking can come up with some scenerios that might be > worse(*). > > I definitely think any minor problems are more than outweighed by the > removal of unused personal data! I personally think it would be a mistake not to at least attempt to notify the contact by email. And sending 80,000 emails isn't exactly a massive amount to send in the grand scheme of things ... and whilst processing 80,000 bounces wouldn't be fun there are ways to mitigate the whole bounce process. If there is concern about load on the current RIPE mail servers, then simply setup another mail server / MX specifically for this project where these mails can be sent from. Personally I think the bigger problem isn't unreferenced objects, but the bigger issue is with "orphaned" objects (*). The "new" role objects overcome a lot of this, but every day I still see objects with contact details of people who left the company 5-10 years earlier and are still referenced .. and unravelling that mess becomes a whole lot harder. -- Jon Morby FidoNet Registration Services Ltd tel: 0845 004 3050 / fax: 0845 004 3051 web: http://www.fido.net/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From gert at space.net Fri Apr 20 11:57:15 2007 From: gert at space.net (Gert Doering) Date: Fri, 20 Apr 2007 11:57:15 +0200 Subject: [ncc-services-wg] Re: [db-wg] Clean up of unreferenced person/role objects In-Reply-To: <20070420094923.GA9585@borg.c-l-i.net> References: <4626025E.4020909@ripe.net> <20070419152243.GB29847@borg.c-l-i.net> <20070420094923.GA9585@borg.c-l-i.net> Message-ID: <20070420095715.GY63243@Space.Net> Hi, On Fri, Apr 20, 2007 at 11:49:23AM +0200, Shane Kerr wrote: > This points out the failure of policy... what happens if objects are > not maintained properly? Right now, there is nothing that can be done. > What I would like to see done is the resources made unavailable for > use until the maintainer confirms that the objects about them are > correct. Playing devil's advocate: how can the RIPE NCC make an IP address block "unavailable"? Or an AS number? Gert Doering -- NetMaster -- Total number of prefixes smaller than registry allocations: 113403 SpaceNet AG Vorstand: Sebastian v. Bomhard Joseph-Dollinger-Bogen 14 Aufsichtsratsvors.: A. Grundner-Culemann D-80807 Muenchen HRB: 136055 (AG Muenchen) Tel: +49 (89) 32356-444 USt-IdNr.: DE813185279 From shane at time-travellers.org Fri Apr 20 11:49:23 2007 From: shane at time-travellers.org (Shane Kerr) Date: Fri, 20 Apr 2007 11:49:23 +0200 Subject: [ncc-services-wg] Re: [db-wg] Clean up of unreferenced person/role objects In-Reply-To: References: <4626025E.4020909@ripe.net> <20070419152243.GB29847@borg.c-l-i.net> Message-ID: <20070420094923.GA9585@borg.c-l-i.net> On Fri, Apr 20, 2007 at 09:36:14AM +0100, Jon Morby wrote: > > I personally think it would be a mistake not to at least attempt to > notify the contact by email. For the record, I really don't think it matters much. These person or role objects are ones that are: - unused for months, and - trivially easy to recreate. > Personally I think the bigger problem isn't unreferenced objects, > but the bigger issue is with "orphaned" objects (*). The "new" role > objects overcome a lot of this, but every day I still see objects > with contact details of people who left the company 5-10 years > earlier and are still referenced .. and unravelling that mess > becomes a whole lot harder. This is a separate issue, and is IMHO a failure of both database design and policy. The failure of database design is that data needs to be checked for correctness periodically. Objects in the RIPE Database are not. Such a mechanism can be simple. For example, each maintainer could get an e-mail each year pointing to a web page listing objects maintained asking "click to continue to use these values". This points out the failure of policy... what happens if objects are not maintained properly? Right now, there is nothing that can be done. What I would like to see done is the resources made unavailable for use until the maintainer confirms that the objects about them are correct. I think that LIRs will never accept such a policy. But I've been wrong before. -- Shane From slz at baycix.de Sat Apr 21 01:35:12 2007 From: slz at baycix.de (Sascha Lenz) Date: Sat, 21 Apr 2007 01:35:12 +0200 Subject: [ncc-services-wg] Clean up of unreferenced person/role objects In-Reply-To: <4626025E.4020909@ripe.net> References: <4626025E.4020909@ripe.net> Message-ID: <46294E30.8050101@baycix.de> Denis Walker schrieb: > Introduction > ------------ > > According to our database consistency statistics program (dbconstat [1]) we > currently have 460,573 unreferenced person/role objects [2]. Some of these > may be maintained, but are still unreferenced. Any personal data not > referenced by Internet resources do not fit within the purpose of the RIPE > Database. They should not be stored in the RIPE Database beyond a reasonable > 'work in progress' period. [...] so, my 0.02EUR: unreferenced person/role objects should not stay in the RIPE whois db forever and should been cleaned up on a regular basis, i agree since 2001 :-) I don't really know how that vast amount of useless(?) data affects the database operation nowadays (lookup/update times, mirror synchronization , databse sizes ect.), if at all with the current database machines, but the issue is more general. It just makes no real sense to keep them in there. > One time bulk cleanup procedure > ------------------------------- [...] But like some other people i'm not really happy about the "no e-mail notification" thing. Not even for the one-time procedure. I really really would like to have at least the regular update/delete-notificiations. It's just a bad thing to delete something without notifying the "owner". I wouldn't like that to happen to my objects - probably i keep one old Internic person object i copied to the RIPE db some decades ago alive but unreferenced since it's without the xyz-RIPE tag and i don't like to really use it anymore but .... at least i'd like to know if it gets deleted and the WHY in the delete: reason filed. Bottom line: If it's agreed on that it's not possible for the RIPE NCC to send out the notifications for whatever reason, i'd actually rather like to keep the objects there - or take smaller steps so it IS possible to send out notifications (hey, it took so long to re-activate that 'project', so it can't be that time-critical ;-). -- ======================================================================== = Sascha Lenz SLZ-RIPE slz at baycix.de = = Network Operations = = BayCIX GmbH, Landshut * PGP public Key on demand * = ======================================================================== From andy at nosignal.org Sun Apr 22 20:58:37 2007 From: andy at nosignal.org (Andy Davidson) Date: Sun, 22 Apr 2007 14:58:37 -0400 Subject: [ncc-services-wg] Clean up of unreferenced person/role objects In-Reply-To: <4626025E.4020909@ripe.net> References: <4626025E.4020909@ripe.net> Message-ID: On 18 Apr 2007, at 07:34, Denis Walker wrote: > According to our database consistency statistics program (dbconstat > [1]) we > currently have 460,573 unreferenced person/role objects [2]. Some > of these > may be maintained, but are still unreferenced. Any personal data not > referenced by Internet resources do not fit within the purpose of > the RIPE > Database. They should not be stored in the RIPE Database beyond a > reasonable > 'work in progress' period. > Do you have any data on the age of these unreferenced people/ objects ? I don't get warm-and-fuzzies from the thought of deleting objects.... On 19 Apr 2007, at 13:11, Leo Vegoda wrote: > This case could also be avoided by making it impossible to re-use a > nic-hdl. > I used to work for LIR-X and now I'm working for new LIR-Y. Whilst internal paperwork is done my person object is orphaned, but I a. want to keep my handle b. want to keep my maintainer stuff intact Please lets try to understand why objects are orphaned before we take a bullet to them. cheers -a From leo.vegoda at icann.org Sun Apr 22 21:33:45 2007 From: leo.vegoda at icann.org (Leo Vegoda) Date: Sun, 22 Apr 2007 15:33:45 -0400 Subject: [db-wg] Re: [ncc-services-wg] Clean up of unreferenced person/role objects In-Reply-To: References: <4626025E.4020909@ripe.net> Message-ID: <599D50F5-D0B5-4FD4-AD25-F11F64A2C3FE@icann.org> On Apr 22, 2007, at 2:58 PM, Andy Davidson wrote: > On 19 Apr 2007, at 13:11, Leo Vegoda wrote: > >> This case could also be avoided by making it impossible to re-use >> a nic-hdl. > > I used to work for LIR-X and now I'm working for new LIR-Y. Whilst > internal paperwork is done my person object is orphaned, but I > > a. want to keep my handle > b. want to keep my maintainer stuff intact I don't think I wrote very clearly. I didn't mean "make it impossible to use the same nic-hdl for two different organisations' objects". I meant that if a person object is deleted then its nic-hdl could not be reassigned to a different person object at a later date. As long as you clean up as you go keeping your handle and maintainer shouldn't be an issue - if this feature was ever added. Regards, -- Leo Vegoda IANA Numbers Liaison From andy at nosignal.org Mon Apr 23 00:25:00 2007 From: andy at nosignal.org (Andy Davidson) Date: Sun, 22 Apr 2007 18:25:00 -0400 Subject: [db-wg] Re: [ncc-services-wg] Clean up of unreferenced person/role objects In-Reply-To: <599D50F5-D0B5-4FD4-AD25-F11F64A2C3FE@icann.org> References: <4626025E.4020909@ripe.net> <599D50F5-D0B5-4FD4-AD25-F11F64A2C3FE@icann.org> Message-ID: <2D27B254-7A79-44D7-968A-AD46332D5FF6@nosignal.org> On 22 Apr 2007, at 15:33, Leo Vegoda wrote: >> I used to work for LIR-X and now I'm working for new LIR-Y. >> Whilst internal paperwork is done my person object is orphaned, but I > I don't think I wrote very clearly. Nor did I. :-) My person object is orphaned during my move between the companies. But I don't want it deleting ! cheers -a From leo.vegoda at icann.org Mon Apr 23 00:32:25 2007 From: leo.vegoda at icann.org (Leo Vegoda) Date: Sun, 22 Apr 2007 18:32:25 -0400 Subject: [db-wg] Re: [ncc-services-wg] Clean up of unreferenced person/role objects In-Reply-To: <2D27B254-7A79-44D7-968A-AD46332D5FF6@nosignal.org> References: <4626025E.4020909@ripe.net> <599D50F5-D0B5-4FD4-AD25-F11F64A2C3FE@icann.org> <2D27B254-7A79-44D7-968A-AD46332D5FF6@nosignal.org> Message-ID: <8CD08077-BF0E-4FC9-BA90-6B9824AA44CB@icann.org> On Apr 22, 2007, at 6:25 PM, Andy Davidson wrote: >>> >>> I used to work for LIR-X and now I'm working for new LIR-Y. >>> Whilst internal paperwork is done my person object is orphaned, >>> but I >> I don't think I wrote very clearly. > > Nor did I. :-) My person object is orphaned during my move between > the companies. But I don't want it deleting ! I suppose the thing to do is to stick it in a role or organisation object so it is no longer unreferenced. It seems a simple thing to do to distinguish an object that is wanted from one that is not. Regards, -- Leo Vegoda IANA Numbers Liaison From shane at time-travellers.org Sun Apr 22 11:45:28 2007 From: shane at time-travellers.org (Shane Kerr) Date: Sun, 22 Apr 2007 11:45:28 +0200 Subject: [ncc-services-wg] Re: [db-wg] Clean up of unreferenced person/role objects In-Reply-To: <20070420095715.GY63243@Space.Net> References: <4626025E.4020909@ripe.net> <20070419152243.GB29847@borg.c-l-i.net> <20070420094923.GA9585@borg.c-l-i.net> <20070420095715.GY63243@Space.Net> Message-ID: <20070422094528.GA17471@borg.c-l-i.net> Gert, On Fri, Apr 20, 2007 at 11:57:15AM +0200, Gert Doering wrote: > > On Fri, Apr 20, 2007 at 11:49:23AM +0200, Shane Kerr wrote: > > This points out the failure of policy... what happens if objects > > are not maintained properly? Right now, there is nothing that can > > be done. What I would like to see done is the resources made > > unavailable for use until the maintainer confirms that the objects > > about them are correct. > > Playing devil's advocate: how can the RIPE NCC make an IP address > block "unavailable"? Or an AS number? How would *you* do it if someone asked you to set up a revokation procedure? I imagine it would look something like: - Flag the resources as possibly abandonded internally at the NCC. - Try to contact the maintainers (or LIR if possible). - After a time, flag the resources as possibly abandoned externally. - Try harder to contact the maintainers (contact peers to try to get contact information, for instance). - Move the resources to an "abandoned" status, removing them from public databases. - After a time, do a debogonizing effort on the resources. - Mark the resources available for use again. Mind you, this is just a possibility. There are costs and benefits at each step (for example, publically flagging a resource as unmaintained in the Whois gives hijackers an easy way to locate likely targets... but looking at the routing table can do this too). The RIPE NCC's policies and procedures have done a fairly good job of handling the task of issuing new resources, and making sure that active LIRs keep their information accurate. But when resources go to non-LIRs, for both PI blocks and AS numbers, the system basically fails completely. Maybe this will all solve itself in 2 or 3 years, when we run out of new IPv4 space. I imagine then there will be a lot of people hijacking this space, so this problem may disappear. -- Shane From florian at frotzler.priv.at Mon Apr 23 00:33:01 2007 From: florian at frotzler.priv.at (Florian Frotzler) Date: Mon, 23 Apr 2007 00:33:01 +0200 Subject: AW: [db-wg] Re: [ncc-services-wg] Clean up of unreferenced person/role objects In-Reply-To: <2D27B254-7A79-44D7-968A-AD46332D5FF6@nosignal.org> References: <4626025E.4020909@ripe.net> <599D50F5-D0B5-4FD4-AD25-F11F64A2C3FE@icann.org> <2D27B254-7A79-44D7-968A-AD46332D5FF6@nosignal.org> Message-ID: <000001c7852e$310305e0$930911a0$@priv.at> Andy, I think it was clearly stated that only objects that are orphaned _for a long time_ are deleted, so it won't hit you. Cheers, Florian > >> I used to work for LIR-X and now I'm working for new LIR-Y. > >> Whilst internal paperwork is done my person object is orphaned, but > I > > I don't think I wrote very clearly. > > Nor did I. :-) My person object is orphaned during my move between > the companies. But I don't want it deleting ! > > cheers > -a From denis at ripe.net Mon Apr 23 15:20:25 2007 From: denis at ripe.net (Denis Walker) Date: Mon, 23 Apr 2007 15:20:25 +0200 Subject: [ncc-services-wg] Cleanup of unreferenced person objects Message-ID: <462CB299.9060200@ripe.net> Hi There has been some useful feedback already on this proposal. Perhaps I need to clarify some points and suggest some changes to the proposal. We have always wanted to keep the database as clean as possible. So whilst half a million extra person objects has no effect on the operation, we prefer them not to be there if they are serving no useful purpose. We have also been looking more closely at Data Protection issues recently. We have a number of points to discus with the Data Protection Task Force. But the bottom line is, we have to define the purpose(s) of the RIPE Database. We are only allowed to keep personal data in the database if it fits the defined purpose. Although we are only at the stage of drafting these definitions, it is difficult to see how unreferenced person objects could fit any definition that we finally agree on. As for e-mails, I have discussed the numbers with our IT operations guys. They are confident we can handle the worst case scenario with our mail servers. So I suggest we process the deletions with the normal notification procedure where e-mails are sent to any "notify:" addresses in the person objects and to the maintainers if maintained. We will use a modified notification text for this purpose. It will refer to a web page that explains why these person objects are being deleted. Regarding the issue of temporary unreferenced person objects. For example when someone is moving to a new company and de-references their person object from the old companies data and intends to reference it with the new companies data. The proposal allows for a (minimum) one month waiting period before deletion. It could be up to two months, depending on when it is de-referenced. If there is a feeling that this needs to be a longer period it can be extended. I hope this addresses the issues so far raised. Regards Denis Walker Database Group RIPE NCC From denis at ripe.net Thu Apr 26 15:16:56 2007 From: denis at ripe.net (Denis Walker) Date: Thu, 26 Apr 2007 15:16:56 +0200 Subject: [ncc-services-wg] RIPE Database update problem Message-ID: <4630A648.4060609@ripe.net> [Apologies for duplicate e-mails] Dear Colleagues, >From 21:30 (UTC) on 24 April 2007 until 10:20 (UTC) on 26 April 2007, e-mail updates sent by users to the RIPE Database update service were not processed by the RIPE Database. This problem has now been resolved. ALL the updates submitted have now been processed. No updates were lost. We apologise for this problem with the RIPE Database update service. This was caused by some network adjustments affecting our firewall and mail servers. It had no effect on updates submitted using syncupdates or webupdates, or changes made using the LIR Portal interface. Any changes made by the RIPE NCC Registration Services Department on behalf of RIPE NCC members were also unaffected. We would like to thank all our users for their patience while we fixed this problem. Regards Denis Walker Database Group RIPE NCC