[ripe-list] [db-wg] Reminder: RIPE Database Requirements Task Force – Draft Report Published

Dear Denis,

Thank you for your feedback. You will find the task force’s reply to your suggestions and comments below. If you need clarifications feel free to contact us.

-- Data management principles
Regarding data accuracy, we decided to stick with the current wording as we think it encompasses all database users and not only network operators. Regarding data minimisation, the question of “data inheritance” is interesting but probably out of scope for this document.

-- Purposes
We chose to use existing RIPE Database purposes to build our requirements list as they were still relevant. Even though there is a use case for geolocation, we didn’t think that it constituted a purpose. The RIPE Database only offers partial geolocation data and the “geoloc:” attribute data is user-maintained and mainly unreliable.

The task force suggested to rephrase the research purpose to “enabling research and analysis into IP networking in the RIPE region”. This way, it will also include IP research from the general public which was one of your suggestions. We will also give this purpose its own section and add more background information.

— Requirements
5.1.2 Requirement: IPv4 PA assignments
Under the current policy which we are recommending to remove, for reasons previously explained (https://ripe82.ripe.net/archives/video/542/ <https://ripe82.ripe.net/archives/video/542/>), there are a lot of discrepancies in how resource holders document assignments and it’s already possible for them to delete assignments. Also, removing this policy requirement will be in line with the data minimisation principle.

5.1.3 Other consideration: Using the RIPE Database as an IPAM solution
As stated in the document, IPAM goes against the data minimisation principle and is not directly matching the purposes of the RIPE Database. This is why we didn’t consider it as a requirement.

5.1.4 Requirement: Historical data and personal data filtering
The historical data was added since 2001, but we implemented a feature to return historical data in 2013. We will edit the draft to make the distinction. We also appreciate your suggestion regarding historical data to: “Make as much historical data publicly available as is legally permitted.” but decided to stick with the current recommendation. This recommendation came as the result of discussions we had at previous BoFs and seems to be responding to the needs of the community.

5.2 Purpose: Provisioning of Reverse Domain Name System (rDNS)
We didn’t identify any specific requirements that will be worth mentioning in this section. Regarding ENUM data, we will mention it as a side note in the document as it holds a special status in the RIPE Database.

5.3.1 Requirement: Routing information and 5.3.2 Requirement: Maintaining accurate routing origin information
These requirements were developed in line with today’s best practices and might indeed reflect some of the status quo.

5.3.3 Other Consideration: Routing Policy Specification Language (RPSL)
We do think that it’s worth clarifying that RPSL is not a requirement of the RIPE Database and that the relevant working groups should look into deprecating this old standard.

5.4 Purpose: Facilitating Internet operations and coordination
Adding a stakeholder list is complicated as you’ll always end up excluding some people even if it’s a non-exhaustive list. This is why the task force decided to stick with a broader definition of RIPE Database users.

Kind regards,
Bijal Sanghani


> On 11 Aug 2021, at 13:50, denis walker via db-wg <db-wg at ripe.net> wrote:
> 
> Colleagues
> 
> The DBTF asked for comments about their latest draft document by 13
> August. So far I have seen only one comment. So I am taking off my
> co-chairs hat and putting my devil's advocate hat on again and I will
> make some comments of my own and raise a number of questions. I make
> no apologies for the length of this email and I accept some people
> won't read it for that reason. But as the DBTF is approaching the end
> of it's work, there are many things that need to be said about this
> document. Whether the DBTF can or will or should take these into
> account is for others to decide. The document can be found here
> https://www.ripe.net/participate/ripe/tf/rdb-requirements-tf/the-ripe-database-requirements-task-force-draft-report-july-2021.pdf
> 
> '4.1 Data accuracy'
> Here you say data should be accurate for "all parties involved in
> network operations.". But In section 3 you accept that there are
> different user groups who use the database now. So data should be
> accurate for "all parties.". It may sound irrelevant, but you also
> accept in section 3 that there is friction between different user
> groups. If this document appears to favour one user group, network
> operators, it may be used in the future to argue against changes that
> benefit user groups other than network operators.
> 
> '4.3 Data minimisation'
> You only refer to personal data and your definition only relates to
> limiting data to match the purposes. While they are valid examples,
> they are not the only issues regarding data minimisation. You haven't
> covered the issue of inheritance for example. Having one "tech-c:"
> attribute which is inherited by 10,000 INETNUM objects is far better
> than having 10,000 identical copies of that one piece of data. The
> concept of inheritance is certainly within scope of the business
> requirements as it can lead to improved data accuracy.
> 
> '5. Purposes, Requirements and Recommendations'
> You start by saying:
> 
> "To produce its requirements, the task force looked at four purposes
> of the RIPE Database.
> 
> ● Providing authoritative and accurate registration of Internet number resources
> ● Provisioning of the Reverse Domain Name System (rDNS)
> ● Publishing routing policies by network operators (RIPE IRR)
> ● Facilitating Internet operations and coordination
> 
> Even though this document is based around the four purposes mentioned
> above, the task force is aware that there is a fifth purpose that
> should be taken into consideration:
> ● Enabling scientific research into network operations and topology"
> 
> This suggests that the DBTF considers that these 5 items are the only
> purposes of the RIPE Database. Although you say the fifth purpose
> "should be taken into consideration", you don't consider it in this
> document. This confuses me about the nature of this document. If it is
> to be the business requirements of the RIPE Database how can you not
> discuss one of the defined purposes?
> 
> When I wrote the first set of Terms & Conditions for the RIPE Database
> 10+ years ago (which were approved at the time by the community) the
> purposes were listed as:
> 
> ● Ensuring the uniqueness of Internet number resource usage through
> registration of information related to the resources and Registrants
> ● Publishing routing policies by network operators (IRR)
> ● Facilitating coordination between network operators (network problem
> resolution, outage notification etc.)
> ● Provisioning of Reverse Domain Name System (DNS) and ENUM delegations
> ● Providing information about the Registrant and Maintainer of
> Internet number resources when the resources are suspected of being
> used for unlawful activities, to parties who are authorised under the
> law to receive such information.
> ● Scientific research into network operations and topology
> ● Providing information to parties involved in disputes over Internet
> number resource registrations to parties who are authorised under the
> law to receive such information.
> 
> Do you not regard these items as valid purposes of the RIPE Database now:
> 
> ● Providing information about the Registrant and Maintainer of
> Internet number resources when the resources are suspected of being
> used for unlawful activities, to parties who are authorised under the
> law to receive such information.
> ● Providing information to parties involved in disputes over Internet
> number resource registrations to parties who are authorised under the
> law to receive such information.
> 
> All of the 5 purposes you have listed are the historic purposes of the
> RIPE Database. Has the DBTF not recognised any new, valid purposes of
> the RIPE Database? (Besides IPAM that you have rejected.) Accepting
> that some historic purposes have been dropped (like forward domain
> registrations) do you consider that the purpose of the RIPE Database
> in 2021 has not moved on or developed from what it was, say, in 2001?
> It seems to me that there is a fundamental piece of information
> missing from this document. Who are the main user groups who enter
> data into and consume data from the RIPE Database, for what purposes
> and what data do they need? Do they have an equal stake or are there
> priorities? Do conflicts exist between stakeholders and data
> requirements? I don't see how you can document the business
> requirements and functionality without first defining the main
> stakeholders, actors, end users, data consumers and their data
> requirements. (Technically the Business Enterprise Requirements would
> be followed by the Stakeholder Requirements then the Solution
> Requirements. But as this document is shifting between requirements
> and functional review it seems to have merged all these into one, but
> without including some key elements.)
> 
> In section 3 you say:
> 
> "Some database users, such as ISPs or IXPs, have been part of the RIPE
> community for years, while others are relatively new, such as Law
> Enforcement Agencies (LEAs) or regulators. These user groups have
> different needs and expectations regarding the database which is
> creating friction within the community."
> 
> So you have recognised that there are new user groups with different
> needs but this is not reflected in the historic purposes of the
> database. The purposes clearly have changed but these changes have not
> been recognised or formally accepted. Hence the friction you refer to.
> Unless we address the purposes these new user groups want from the
> database this friction is not going to be resolved. If these issues
> are not addressed then I would consider this document does not fully
> represent the business/stakeholder requirements.
> 
> Geolocation is another purpose the database has long been used for and
> this has taken on a higher profile recently by a significant user
> group. This does not fit under the purpose 'Facilitating Internet
> operations and coordination' as some other features do like abuse
> contact definitions. Being a completely separate and external service
> that requires some new data to be stored in the RIPE Database for this
> service to function it is a different type of purpose to the other
> defined purposes. This should be listed in this document as a new
> purpose.
> 
> Should it be generally accepted now that the RIPE Database is a public
> registry of all users of blocks of IP addresses? This would be similar
> to the public set of domain name registries. Anyone who wants to know
> who operates a domain name can look it up in the appropriate domain
> registry.  It has become quite common that anyone who wants to know
> who is using a block of IP addresses will look up those addresses in
> the RIPE Database. Has it evolved from just being a tool for network
> operators to a more general public service? Is it time to recognise
> this as a purpose of the RIPE Database?
> 
> '5.1.2 Requirement: IPv4 PA assignments'
> You refer to "A core reason for registration of IPv4 PA assignments"
> in the address policy. It should be pointed out that this is also a
> 'historic' reason. Perhaps in 2021 it is not the only reason. If it is
> accepted that the RIPE Database has evolved into a public service
> registry and this is defined as a purpose of the database then that
> provides another reason for documenting all assignments in the
> database.
> 
> For the DBTF to recommend not deleting assignments after making them
> optional is like a government recommending wearing masks after
> dropping the legal requirement. Most people don't wear masks now. Most
> optional assignments will probably be deleted. Why would operators
> spend time and money maintaining optional assignment data? Unless
> there is a clear benefit to the operator or some greater need covered
> by a requirement to provide assignments, I suspect most will simply
> ditch this public data. This would destroy the RIPE Database as a
> public service registry. If some operators maintained the optional
> data and some deleted it then we end up with an incomplete and
> inconsistent data set, which is what happened with forward domain
> data.
> 
> '5.1.3 Other consideration: Using the RIPE Database as an IPAM solution'
> According to your survey results almost 54% of the respondents said
> they use the RIPE Database for their IPAM. That is the second largest
> percentage of all the listed uses. Only slightly behind general number
> resource management at 57%. Given the level of usage, why do you not
> accept IPAM as a valid, new purpose of the RIPE Database? Many years
> ago forward domain data was stored in the database. Many of the
> smaller domain registries used the database as their primary registry
> database. Most of the bigger registries only published a limited data
> set and used a referral mechanism to direct queries to their whois.
> One of the main reasons for deleting this data was the inconsistent
> and incomplete, public facing, data set. What is your reasoning for
> recommending not accepting IPAM as a new purpose of the database and
> maybe a resource holder benefit?
> 
> '5.1.4 Requirement: Historical data and personal data filtering'
> This is the third draft doc you have published containing this statement:
> "Since 2013, the RIPE Database has stored historical data, as
> requested by the RIPE community"
> and this is the third time I have commented that this is simply not
> correct. The database contains every version of every object that has
> ever existed since April 2001. Historical queries were implemented in
> 2013, but the data was already there going all the way back to 2001. I
> trust the DBTF will finally correct this in the next draft.
> 
> Your recommendation makes no sense whatsoever. Availability of
> historical data should depend on the type of data, not on the use case
> for accessing it. To suggest that researchers could be given more data
> access on a case by case basis is madness. The principle of
> availability of historical data is basically that operational data is
> available, personal data and any data subject to privacy is not
> available. There is no reason to limit any of the available data by
> user group or use case. It should all be publicly available. No user
> group or use case is going to make the excluded personal and privacy
> data available to anyone for very strict legal reasons. (In particular
> the right to be forgotten granted by GDPR. Of course that can be
> overruled by a court order but that is an exceptional use case.)
> 
> The recommendation should be to make as much historical data publicly
> available as is legally permitted. I believe some of the data
> currently redacted could still be made available. I won't go into
> details here as it gets too technical for this doc.
> 
> '5.2 Purpose: Provisioning of Reverse Domain Name System (rDNS)'
> ENUM may be a small part of this but it is still significant and
> should not be overlooked in terms of defining the purpose of the RIPE
> Database.
> 
> "The task force didn’t find the need for any requirements or
> recommendations attached to this purpose and therefore recommend
> maintaining the current status quo."
> Comments like this really confuse me. It again makes me question what
> is this document? If it is a business requirements document then you
> cannot say there is no need to document any requirement for one of the
> purposes of the database.
> 
> '5.3.1 Requirement: Routing information'
> The DBTF makes a long list of recommendations, but the whole list is
> simply a definition of the status quo. If you want to document this
> functionality that is fine, but that is then inconsistent with the way
> you have handled other functionality. There is nothing new or
> different in these recommendations so you could have said the same as
> you did for rDNS "we recommend the status quo".
> 
> '5.3.2 Requirement: Maintaining accurate routing origin information'
> Again the list of recommendations are simply the status quo. I find
> this very confusing. Initially you appear to be recommending
> something. But you are only documenting the current practise.
> 
> '5.3.3 Other Consideration: Routing Policy Specification Language (RPSL)'
> I think this section is out of scope for this document. This is choice
> of technology and technical implementation. First of all, what the
> DBTF says here is not strictly accurate. The RIPE Database data model
> is 'based' on RPSL and RPSLng. It never has been a strict
> implementation. Although the RPSL standard may not have been
> maintained for decades, the implementation in the RIPE Database has
> been modified, added to and cut down many times according to the needs
> of the RIPE community. The way data is input and output does not need
> to exactly match the way data is stored internally in the database.
> All the data, not only the routing information, is currently still
> stored internally in an RPSL like format. This is because the
> community has never had the appetite to redesign the data model, not
> even with a small iterative approach (despite numerous attempts by
> myself to push for this over the last 10 years).
> 
> If this document is the business requirements then it may be in scope
> to recommend a review of the format(s) of data input to and output
> from the database. The internal representation of that data is
> definitely out of scope. Even discussing what detailed routing
> information is useful is more for the level of technical specification
> than a high level business requirements document.
> 
> '5.4 Purpose: Facilitating Internet operations and coordination'
> "The RIPE Database should facilitate communication and cooperation
> among stakeholders for the following reasons:"
> This needs to say, "for reasons including the following:". The list is
> not exhaustive now and there may well be other reasons in the future.
> 
> 'Accuracy'
> (hair splitting comment) RFC7020 does not define 'registration
> accuracy'. It simply uses the words as in "provide accurate
> registration information". So what 'accurate' means in terms of
> registration information does not appear to have been defined
> anywhere. I think correct would be a more appropriate word than
> accurate.
> 
> 'Resource Holder:'
> It should say "allocated or assigned....from the RIPE NCC directly" to
> be correct.
> 
> General closing comment from me..
> Having read this document several times I am still confused about what
> it is. It is not (some form of) a Requirements document, nor is it a
> review of RIPE Database functionality. In either of those cases it
> only presents a select subset of what would be needed. What it appears
> to be, to me, is mostly a review of RIPE Database functionality that
> has been the subject of (recent) discussion and has outstanding open
> issues, with some requirements added in some cases. But even that is
> not complete.
> 
> I really do appreciate the work the task force members have done. I am
> just not sure how to interpret the outcome.
> 
> In the first bof in Iceland, Daniel's opening remark was "it is time
> to stop tinkering around the edges...". Looking around the room at the
> time I saw a lot of heads nodding in approval at that comment, even
> from some of the 'old grandees' of the community. Unfortunately most
> of this document, in my opinion, is still tinkering around the edges.
> 
> I don't know if the DBTF was meant to document where we are now with
> the database or produce a guiding light to move onwards and upwards.
> Given that there are many recommendations it suggests it was to be the
> way forward. For me it falls well short of taking me to the place I
> imagined after listening to Daniel's inspiring speech. Perhaps an
> opportunity missed....or maybe I misunderstood the destination and/or
> route...or maybe that is the next step?
> 
> cheers
> denis
> 
> On Wed, 11 Aug 2021 at 12:26, Bijal Sanghani via db-wg <db-wg at ripe.net> wrote:
>> 
>> Dear all,
>> 
>> A gentle reminder for feedback on the report the RIPE Database Requirements Task Force published last month.
>> 
>> Please let us know if you have any questions or concerns on the draft or if you’re happy with our recommendations. This will give us input into finalising the report which we plan to publish before RIPE82.
>> 
>> We look forward to your feedback, thank you for your time - https://www.ripe.net/participate/ripe/tf/rdb-requirements-tf/the-ripe-database-requirements-task-force-draft-report-july-2021.pdf
>> 
>> Kind regards,
>> Bijal Sanghani
>> 
>> 
>> 
>> On 1 Jul 2021, at 13:31, Bijal Sanghani <bijal at euro-ix.net> wrote:
>> 
>> Dear colleagues,
>> 
>> The RIPE Database Requirements Task Force has just published a draft report for your review: https://www.ripe.net/participate/ripe/tf/rdb-requirements-tf/the-ripe-database-requirements-task-force-draft-report-july-2021.pdf
>> 
>> In this third draft, we’ve added new recommendations and edited our document based on feedback received via emails, survey and during our third BoF in May.
>> 
>> Please review this draft and share any comments on the ripe-list before Friday, 13 August 2021.
>> 
>> The task force will reconvene in August to discuss the feedback received and work on its final report that will be published ahead of RIPE 83.
>> 
>> We look forward to your feedback.
>> 
>> Best Regards,
>> 
>> Bijal Sanghani
>> 
>> 
>> 
>> 
>> 
>> 
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.ripe.net/ripe/mail/archives/ripe-list/attachments/20210929/e0edb4b1/attachment.html>