[db-wg] proposal for a review of the RIPE Database data model

Seconded!

Two more suggestions:

1. Data format
It would also be good to finally move away from the RPSL format too,
to JSON or YAML (or any other human-readable standard format). RPSL
may seem simple, but it actually allows for horrible abominations with
the multiline attributes scattered with comments. Also, some data can
be given as multiple attributes, some as one attribute list, or a mix
of these.

Just by looking at the data, people would think it's an easy text
parser, but it's actually unnecessarily complex to work with,
especially in the advent of proper message formats widely available.


2. Versioning
At the moment, updates to the RIPE DB are considered a 'latest source
of truth'. However, each object does have a unique version number
associated with it. To avoid accidental overwrites, updates should
only be allowed against the latest version of any object.

This is easily achieved with e.g. having an 'updates: XXX'
meta-attribute that specifies either the last modification date+time,
or the unique object version (in which case it has to be made
available).


Agoston


On Mon, May 23, 2016 at 9:20 PM,  <ripedenis at yahoo.co.uk> wrote:
> HI All
>
> Piotr asked me if I would have time to write a short problem statement about
> the data model for consideration as an NWI. For once I will try to make it
> brief(er).
>
> Current model developed in late 90s as a further development of the previous
> RIPE Database design from much earlier. Nothing fundamental has changed in
> the last 15 years. It makes virtually no use of inheritance which has led to
> massive amounts of duplicated data, with all the problems of managing and
> interpreting that data. It was designed in a day when it was more normal to
> manage data by sending and receiving emails. Although I welcome the
> improvements to Webupdates, this feature is still something bolted on to the
> side and not fundamental to the design. There is no concept of 'user
> accounts' beyond SSO (another feature bolted on) so it is not possible to
> manage your data online and set permissions or do any configuration.
>
> The database contains two basic types of data:
> 1/ Operational data as determined by the purpose of the database
> 2/ Organisational and security data for the management of the operational
> data
> Far too much of the security and organisational management data is public.
> How I secure my data and who, within my organisation, has permission to do
> what is non of your business. That information is confidential to my
> organisation. With user accounts that can all be kept confidential.
>
> The lack of business rules and constraints in the early days of the database
> allowed massive misuse of the original design. Especially in the area of
> PERSON, ROLE and MNTNER objects. A lot of the relationships between these
> objects and others is totally screwed up now making management, security and
> interpretation of the data far more complex than it needs to be.
>
> Over the last 12 months I have pointed out so many examples on the mailing
> list of how things are over complicated, or cannot be changed, or features
> that cannot be implemented because of the constraints of this data model.
> The most recent example being a very good suggestion by someone to customise
> the notification message and define a simple rule set for sending them out
> as part of a bulk update. This cannot be done by the database software based
> on this data model. It has to be done externally by the NCC staff by
> analysing the data involved in the bulk update, building the customised
> notification messages and sending them out with an email client.
>
> What I am suggesting is a serious review of the data model and the database
> design to identify areas that could be improved in an organised, backwards
> compatible, step by step process.
>
> It is also important to stress what I am not suggesting as this topic seems
> to carry as much negative propaganda as the UK's foolish, brexit campaign. I
> am not suggesting a team of engineers hide in a dark room for a year then
> launch a completely re-designed data model on the community and force you
> all to change all your tools used to engage with the database overnight.
> That is not how software is developed these days.
>
> Small steps, backwards compatible, testing, reviews, run in parallel, slowly
> moving forwards. That is how software is developed these days. On this basis
> I put this forward for consideration as an NWI.
>
> cheers
> denis