From mis at wari.net Thu Jan 3 19:29:57 2008 From: mis at wari.net (Manfredo Miserocchi) Date: Thu, 03 Jan 2008 18:29:57 +0000 Subject: [dp-tf] Re: [ncc-services-wg] Unreferenced person/role objects In-Reply-To: <477CE053.5030100@ripe.net> References: <477CE053.5030100@ripe.net> Message-ID: -----Original Message----- From: Denis Walker To: ncc-services-wg , Database WG Date: Thu, 03 Jan 2008 14:17:07 +0100 Subject: [ncc-services-wg] Unreferenced person/role objects Denis, > We contacted them and they responded very quickly. They gave the RIPE > NCC permission to delete all these unreferenced objects, and they have > now adjusted their procedures. it looks to me quite strange to obtain the permission so quickly and without any resistence. All that objects were unuseful ?? > Over the New Year period, when we expected the load on the RIPE > Database > to be quite low, we deleted 161062 person/role objects. You can see the > drop in the graph here: well done. This will help a lot > In the long term, the RIPE NCC will implement an automatic clean-up > process, but this may take some time to develop. this will help more.... Happy 2008 to all of you Manfredo Si precisa che le informazioni contenute in questo messaggio sono riservate e ad uso esclusivo del destinatario. Qualora il presente messaggio Le fosse pervenuto per errore, La invitiamo ad eliminarlo senza copiarlo ed a non inoltrarlo a terzi, dandocene gentilmente comunicazione. Grazie. You are hereby informed that this message contains confidential informations intended for the addressee's use only. If yu're not the addressee and have received this message by mistake, please delete it and immediately notify us. You may not copy or disseminate this message to anyone. Thank you. From denis at ripe.net Fri Jan 4 11:43:49 2008 From: denis at ripe.net (Denis Walker) Date: Fri, 04 Jan 2008 11:43:49 +0100 Subject: [dp-tf] Re: [ncc-services-wg] Unreferenced person/role objects In-Reply-To: References: <477CE053.5030100@ripe.net> Message-ID: <477E0DE5.6020109@ripe.net> Manfredo Miserocchi wrote: > -----Original Message----- > From: Denis Walker > To: ncc-services-wg , Database WG > Date: Thu, 03 Jan 2008 14:17:07 +0100 > Subject: [ncc-services-wg] Unreferenced person/role objects > > Denis, > > >> We contacted them and they responded very quickly. They gave the RIPE >> NCC permission to delete all these unreferenced objects, and they have >> now adjusted their procedures. >> > > it looks to me quite strange to obtain the permission so quickly and > without any resistence. All that objects were unuseful ?? > I sent them a list of all the nic-hdls and they presumably looked at a few to see where they came from. They had two big cleanup operations last year where they deleted lots of inetnum objects from past customers. But they did not delete any of the referenced person objects. You can see the two big increases on the graph in Feb and July. A few from the list were referenced when I came to delete them, but less than 100. cheers denis > > >> Over the New Year period, when we expected the load on the RIPE >> Database >> to be quite low, we deleted 161062 person/role objects. You can see the >> drop in the graph here: >> > > well done. This will help a lot > > >> In the long term, the RIPE NCC will implement an automatic clean-up >> process, but this may take some time to develop. >> > > this will help more.... > > > Happy 2008 to all of you > Manfredo > > > > Si precisa che le informazioni contenute in questo messaggio sono riservate e ad uso esclusivo del destinatario. Qualora il presente messaggio Le fosse pervenuto per errore, La invitiamo ad eliminarlo senza copiarlo ed a non inoltrarlo a terzi, dandocene gentilmente comunicazione. Grazie. > > You are hereby informed that this message contains confidential informations intended for the addressee's use only. If yu're not the addressee and have received this message by mistake, please delete it and immediately notify us. You may not copy or disseminate this message to anyone. Thank you. > > > From denis at ripe.net Thu Jan 24 16:29:14 2008 From: denis at ripe.net (Denis Walker) Date: Thu, 24 Jan 2008 16:29:14 +0100 Subject: [dp-tf] Limit on access to personal data Message-ID: <4798AECA.70800@ripe.net> Dear Colleagues The RIPE NCC has been working with the RIPE Data Protection Task Force on many areas, including access to personal data held in the RIPE Database. We have now developed a new Near Real Time Mirroring (NTRM) stream to filter out identifiable personal information. NRTM is a mechanism whereby users can receive filtered updates to the RIPE Database almost immediately. It maintains the references to nic-hdls. So anyone using this stream can find the references and then query the RIPE Database for the personal data they really need. This, of course, will be subject to our normal access limits. Initially, those requesting NRTM streams will be provided with the new stream and must sign a new contract. The data protection laws make it very difficult to justify giving anyone full access to all personal data held in the RIPE Database. The RIPE NCC now has a dilemma. We often get requests from organisations claiming to be spam or abuse fighters. They always ask for full access based on an unpredictable need to query large numbers of PERSON objects. A typical recent example is this: "You see, the number of queries doesn't depend on me; it depends on how many different IP addresses attack that server ... December was quiet and I probably made 4,000 or 5,000 queries, while in the first week of January alone I was hit by about 18,500 bots on distinct addresses and had to make just as many queries." To query a large number of inetnums without using the "-r" query flag could return tens of thousands of PERSON objects. This is much higher than our default access limit, which is the total number of personal data sets a user can receive from queries to the RIPE Database in a set period of time. (For security reasons, we do not disclose what the default value is or the time period.) We can raise the limit on the number of PERSON object queries allowed by a specific IP address, subject to the user signing a contract with the RIPE NCC. But we have some questions that need to be considered here: 1. How do we verify the validity of a claim that an organisation is a genuine spam or abuse fighter? We have no knowledge of these organisations. Often their website does not provide conclusive evidence. It could even be a spamming organisation that claims to fight spam and ask for an increased access limit. 2. If we are able to verify the claim, what is an acceptable increased limit for organisations involved in this type of work? From a data protection point of view, lower is better. We cannot accept an open-ended, unpredictable need. 3. Perhaps a better approach would be to provide training on how to use the RIPE Database. Rather than raising limits, advise people to query the IP addresses with the "-r" flag and see how many networks these individual addresses fall into. Find which of these networks have either an "abuse-mailbox:" attribute or reference to an IRT object. Then only query for personal data with those that are still difficult to trace. The RIPE NCC would appreciate any input and guidance on these questions. It is a topic that will be added to the agenda of the DP TF meeting in February. Regards Denis Walker Business Analyst RIPE NCC