[db-wg] More-specific abuse-c

Hi Tim

Looks like I have some competition now for writing long emails late at
night :)

You have some interesting ideas, but I would also like to point out some 
concerns.

On 13/11/2016 01:55, Tim Bruijnzeels wrote:
> Hi,
>
> Even though we don't have a formal problem definition yet, I felt it
> would be useful to be more specific about possible solutions that we
> see.
>
> The short version of the below is that I believe we can have an
> implementation that: - allows the use inheritance for "admin-c:",
> "tech-c:" *and* "abuse-c:" - allows for a simple opt-in, but ensures
> that nothing will break or change if you do not - simplifies

developing new ideas with an opt-in and being backwards compatible is 
always a good way to move forward.

> management of these contacts (in particular simple LIRs / end-users
> with a few PI resources will be able to manage all on their
> ORGANISATION object only) - but allows for more complicated
> structures (inheritance can apply to just parts of the tree, and can
> differ between branches) - simplifies finding the contacts
>
> The long version, well, is a bit longer:
>
>
> = Optional abuse-c on resource* objects
>
> [*: for readability I will use the term "resource objects" to
> indicate INET(6)NUM and AUTNUM objects]
>
> Allowing an optional "abuse-c:" attribute on resource objects,
> referencing a ROLE object with an "abuse-mailbox:" attribute would
> already solve a lot of issues. I believe this would be better than
> having "abuse-mailbox:" attributes directly on these objects, because
> this way the model will be more consistent, and it will be possible
> to reference the same ROLE object from various resources - avoiding
> data duplication.

The reference to a ROLE object was to allow other options for reporting 
abuse, besides email. Also to allow for the possibility of separating 
automated reporting from one off, manual complaints. I still have issues 
about adding "abuse-c:" directly into resource objects but accept that 
is what many users want.

>
> In principle it's an option to only do this part.
>
> Once done this would also allow us to migrate existing
> "abuse-mailbox:" attributes to an "abuse-c:" ROLE object instead. We
> can do this as a one time operation and generate ROLE objects using
> the existing "abuse-mailbox:" - using the same maintainers as the
> resource object has. We can of course avoid creating duplicate ROLE
> objects - and we can avoid creating such objects altogether if the
> value of the "abuse-mailbox:" is the same as the "abuse-c:" that
> already occurs in the hierarchy.

I am not sure which "abuse-mailbox:" attribute you are thinking about 
here. This attribute has never existed in any resource object type. It 
was in PERSON, ROLE, MNTNER, ORGANISATION and IRT objects. It should now 
only be in the ROLE object. The others should all be deprecated. You 
cannot use any of these other object's data in a reliable way to auto 
generate any new data. That was one of the reasons why "abuse-c:" was 
introduced as there were so many places to put this data no one knew 
which was the reliable one.

I also don't think there should be any need to auto generate any data. 
AFAIK all the member's allocations were covered by an "abuse-c:" 
(subject to a few that were created during the re-write of the new LIR 
process). Many/most of the PI objects should also be covered now. It is 
a long time since the RIPE NCC produced any stats on "abuse-c:" 
coverage. So as an aside it would be a good idea to update the community 
on the present coverage before any changes are made.

>
> But, I believe that we can improve things much more.
>
> = Finding relevant abuse-c information
>
> As David pointed out there is quite a bit of hassle involved in
> finding out what the appropriate "abuse-c:" is for a resource object.
> This problem already exists today, but allowing "abuse-c:" on
> resource objects add step #3 in the algorithm described by David.
>
> I believe it would be better if we would just generate the proper
> "abuse-c:" for resource objects on the server side instead, and we
> save users (and tools) of the database the hassle of resolving this.
>
> = Inheritance vs specified values
>
> If a resource object is created or updated and the "abuse-c:"
> attribute is left out, then this would indicate "inherit". In that
> case the server can resolve and generate the appropriate value. We
> can then "tag" the object so that we know that inheritance is desired
> for this attribute.
>
> If a user then queries and re-submits the object with minor changes
> (a common practice), we can detect whether any change was made to the
> "abuse-c:" attribute. If so, we can assume that the user intended to
> stop using inheritance for this attribute. But, if the user made
> other changes and kept the same "abuse-c:", then we can assume that
> inheritance is still desired.

This is a bit confusing, but it may just be the wording. I presume you 
mean if a user makes a minor change to a 'resource' object. If they add 
an "abuse-c:" attribute directly to the resource object they wish to 
stop using inheritance from this point in the hierarchy. If they didn't 
add an "abuse-c:" attribute in the minor change to the resource object 
then it is still inherited from the ORGANISATION object.

With many object types involved in this process it is better to always 
spell out explicitly which object you mean at each step. It makes the 
spec longer, but there is no doubt about exactly what you mean.

>
> Now if something changes higher up in the hierarchy, e.g. the
> "abuse-c:" attribute of the ORGANISATION object is modified, then the
> server can follow all the resource objects below that use inheritance
> and generate the appropriate new values.
>
> Note that this is not duplication of data, it's just explicit
> duplication of references. And since it's handled automatically there
> is no risk of human error.

It is also not clear exactly what you mean here. I am guessing you want 
to expand out the inherited reference to an abuse ROLE object that is 
specified in the ORGANISATION object by allowing the server to add the 
"abuse-c:" attribute to each resource object referencing the 
ORGANISATION object. And in some way tag these resource object to say 
they are using the inherited reference rather than a directly added user 
reference. Then if the "abuse-c:" value is changed in the ORGANISATION 
object this change is replicated throughout the hierarchy of resource 
objects that are tagged as using inheritance.

If this is what you man then there are issues and consequences of doing 
this. Firstly this IS duplication of data as you are adding an attribute 
to all the child objects. But this is an acceptable way these days of 
handling inheritance. But you have to balance out the cost of 
replicating this change vs looking up the inherited value from the 
parent object. In doing this calculation keep in mind that some of the 
large telcos have 200k or even by now 300k resource objects referencing 
an ORGANISATION object. So by changing the value of the "abuse-c:" in 
their ORGANISATION object they will trigger an update to maybe 300k 
objects in the database. That will put quite a load on the update 
server. Especially if they realise they made a mistake and change it 
again a few minutes later. Given that this update can take minutes and 
two such updates could be running in parallel and there is no versioning 
of objects, you could hit a race condition with these updates being done 
in the wrong order on some objects.

You also need to think about the history of the resource objects. What 
would be the intended or desired action here? Do you want a change to an 
ORGANISATION object to effect the "last-modified:" attribute in resource 
objects? Will this be a real update to the resource objects?

>
>
> = Going further -> using inheritance on "admin-c:" and "tech-c:"
>
> Currently both "admin-c:" and "tech-c" are mandatory on resource
> objects, and they are optional on ORGANISATION objects. Contrary to
> "abuse-c:" there is currently no inheritance for these attributes.
>
> I propose that inheritance is added for these attributes and values
> are always generated server side, exactly as I described above for
> "abuse-c:". This means that in order to simplify the management of
> "admin-c:" and "tech-c:" you would only have to re-submit your
> resource object and leave these attributes out. The server would then
> know to use inheritance and track it per attribute, e.g. you can use
> inherit on "admin-c:", but not the other "tech-c:" if you prefer.

I am pleased to see you are considering using inheritance on these other 
attributes. That was always the plan when we first introduce "abuse-c:". 
I presume you will also include the fourth contact attribute "zone-c:" 
in the DOMAIN object as well for inheritance.

>
> Because these attributes are optional on ORGANISATION objects, the
> server can generate an error in case inheritance was asked, but there
> is no value in the hierarchy.
>
> = Wrapping it up..
>
> I believe that this approach for "abuse-c:" *and* "admin-c:" and
> "tech-c:" will make the management of all this contact information
> much easier, and consistent.

and "zone-c:"

>
> It will not require any changes in approach from users. If you are
> currently managing your resource objects and maintaining "admin-c:"
> and "tech-c:" there, you can continue to do so and nothing will break
> or change. If you do not wish to use a different "abuse-c:" you can
> just continue to leave it out and the server will generate it for
> you.

OK I am thinking now of other scenarios. These may be covered in your 
plans but maybe not specified here.

Whatif I choose to opt-in to inheritance, add the values in the 
ORGANISATION object and remove them from resource objects. The inherited 
values are auto generated by the server in all the resource objects. So 
if it is the same value I started with, my resource objects just look 
the same as they did before I opted in. As a user, how do I know I am 
using inheritance? If I now query and re-submit an INETNUM object with a 
change to the "notify:" attribute, I am submitting a resource object 
'with' an "abuse-c:", "admin-c:" and "tech-c". How will the server know 
I still want to use inheritance? It looks like I have just turned off 
inheritance from this point in the hierarchy by directly specifying 
these values that were shown in the query. But to me as a user it looks 
the same. If I now update the values in the ORGANISATION object, the 
auto generated, replicated data will not be changed in these resource 
objects I have just updated. My data is now out of step and as a user I 
don't have a clue this has gone wrong.

(Incidentally this is why I proposed NWI 1)

>
> On the other hand it's very simple to opt-in for users who do want to
> use this. Just set up "admin-c:", "tech-c:" and "abuse-c:" on your
> ORGANISATION object, and resubmit your resource objects without these
> attributes to start using inheritance.

Again large members may have hundreds of thousands of resource objects. 
To resubmit these objects without the attributes you want to inherit is 
a major undertaking.

>Then when for example you want
> to remove one "admin-c:" you only need to change your ORGANISATION
> object and everything else will follow.
>
> For more complicated topologies you can override values for specific
> resource objects later, or simply not opt-in to inheritance to begin
> with - since this is a per attribute and per object choice. You can
> also override values at one point in the resource tree, and use
> inheritance of those values again below.

Are you saying here that if a user manually sets an "admin-c:" at some 
point in their network, this manually added value will then be inherited 
by all the more specific resource objects? The server will then 
replicate this value in all the more specific objects.

>
>
> I hope the above makes sense and I would love to hear what you think.
> Does this solve the issues? Does this introduce problems that I
> overlooked?

Another issue you need to think about is documenting all the fine points 
about this process and see if it can be described in simple terms. The 
trainers also need to explain this to members on the RIPE Database courses.

Some food for thought :)
cheers
denis

>
>
> Kind regards,
>
> Tim Bruijnzeels
>
>
>
>
>
>
>
>> On 11 Nov 2016, at 01:42, fransossen at yahoo.com wrote:
>>
>>
>>
>> Hi,
>>
>> Going down the path of allowing some form of abuse-email directly
>> on the resources would indeed work for almost everyone and keep the
>> centralised abuse-c system for the ones who do not need a more
>> complicated setup.
>>
>> The IPv4/6 PI holders would not be able to do the "granularity"
>> with their single assignments where several abuse-mail might be
>> needed in network segments from within a single inet(6)num. But
>> that would be one more of the restrictions to add to the lists of
>> differences between PI and PA since assignments, a large PI
>> assignment already cannot be broken into smaller assignment unless
>> converted to Allocated PA.
>>
>>
>> It would still confuse any person inheriting the job of reviewing
>> the Data once all the entries are already made. One would actually
>> have to review the abuse contacts on:
>>
>>
>> 1) Your own main org object and its abuse-c:.
>>
>> 2) Any potential org objects that may contain an abuse-c:
>> referenced on your more specific inet(6)nums
>>
>> 3) Any abuse-mail on any inet(6)nums (parent and more specifics)
>> and aut-nums.
>>
>> 4) Figure out the hierarchy and order of which abuse mail-box is
>> displayed based on the intricate DB rules that were decided in
>> order to know if the displayed email address is the intended one or
>> not for those DB entries.
>>
>>
>> This reminds me of the RIPE DB update notifications system:
>>
>> https://www.ripe.net/manage-ips-and-asns/db/support/documentation/ripe-database-documentation/notifications/9-2-notification-messages
>>
>>
>>
"notify:"
>> "mnt-nfy:" "upd-to:" "irt-nfy:" "ref-nfy:"
>>
>> Each one of them makes sense individually, but really confuses
>> people in general, notifiers require quite some DB knowledge once
>> you want to know why you received a notification and even more when
>> you want to know why you didn't receive one!
>>
>> Complicated to setup, complicated to debug, but once understood, it
>> does exactly what it is intended for and provides flexibility. All
>> and all it might be the only way to achieve the intended results
>> though.
>>
>> Cheers, David Hilario
>>
>>
>>
>>
>> On Tuesday, November 8, 2016 7:02 PM, Tim Bruijnzeels
>> <tim at ripe.net> wrote: Dear WG,
>>
>>> Problem statement: ==================
>>>
>>> It is currently not possible to specify alternative abuse
>>> contacts for different resources held by the same organisation in
>>> the RIPE Database. This can be problematic.
>>>
>>> For example for abuse contact management for big organisations
>>> which want multiple abuse contacts for different parts of the
>>> organisation. The org is using a lot of PI resources, some of
>>> them legacy. Currently they all use the same ORGANISATION object
>>> which is inextricably linked to the same abuse-c mailbox.
>>> [example taken from Sebastian's first email]
>>>
>>> Another example applies to different allocations held by a single
>>> LIR. The reference to the LIR's ORGANISATION object is maintained
>>> as part of the registry managed by RIPE NCC, and therefore all
>>> allocations will have the same abuse-c mailbox. There may however
>>> be good operational reasons for having different contacts. [same
>>> issue, slightly different example also mentioned by David]
>>>
>>> It should be noted however that it is considered useful that in
>>> cases where there is no need to have a different abuse-c mailbox,
>>> the abuse-c can be defined on an ORGANISATION object that is
>>> referenced from an INET(6)NUM object, and have it apply to more
>>> specific INET(6)NUM objects through inheritance. This avoids data
>>> duplication, and makes it easier to manage this information.
>>>
>>> But on the other hand it should also be noted that for a lot of
>>> less experienced users of the RIPE database it has proven to be
>>> difficult to find the applicable abuse contact for a resource,
>>> following this hierarchy. For this reason the abuse contact is
>>> now mentioned as a comment in whois output, and is highlighted
>>> explicitly on the web query results.
>>>
>>> As a somewhat related issue I would also mention the following:
>>>
>>> Management of administrative and technical contacts in the RIPE
>>> Database is done differently, and the challenges of maintaining
>>> that data is somewhat different. While here it is possible, or
>>> even mandatory, to have explicit references to admin-c or tech-c
>>> contacts for each resource object that may differ from the
>>> objects, it is *not* possible here to inherit these contacts from
>>> a covering INET(6)NUM or ORGANISATION object. This results in an
>>> increase in maintenance burden for this data and therefore makes
>>> it more difficult to maintain accurate data.
>>
>
>