[db-wg] "status:" attribute values in the RIPE database

Dear Colleagues,

After starting work on the implementation, we discovered that "status:" values are *already* automatically converted to uppercase on update. This was implemented in Whois release 1.73.1 and deployed on 27th May 2014, but existing non-uppercase values were not cleaned up at the time.

I propose simplifying the implementation plan as follows:

(1) Update Whois business rule validation (already implemented)

When a non-uppercase "status:" value is converted to uppercase, a global informational message is added to the response:

	***Info:    Value %s converted to %s

(2) Cleanup existing non-uppercase "status:" values in the RIPE database

On next *Monday 30th August*, we will run a cleanup script to convert all existing non-uppercase "status:" values to uppercase.

We found approximately 21,492 out of 4,217,303 inetnum objects; 2,577 out of 655,856 inet6num objects; 0 out of 37,056 aut-num objects, with a non-uppercase "status:" value.

As only the case of the "status:" value is changed, the objects remain syntactically the same. We will *not* send notification emails to maintainers for the change, apart from notifying the db-wg. The "last-modified:" value will not change, but a new object version will be created (so the change will be visible in the version history).

All changes will be visible in NRTM as the objects are updated. The change can be considered a NOOP.

All changes will be visible in that night's database dump and split files.

We expect the cleanup script to take approximately an hour to complete all updates. 

Cleanup script updates will be applied cooperatively between normal updates, we do not expect any interruption to Whois updates.

Regards
Ed Shryane
RIPE NCC



> On 28 Jul 2021, at 17:15, Edward Shryane via db-wg <db-wg at ripe.net> wrote:
> 
> Dear Colleagues,
> 
> Thanks to the co-chairs for declaring a consensus, the DB team will start working on the implementation.
> 
> The implementation plan consists of two parts:
> 
> (1) Update Whois business rule validation
> 
> We will add a business rule to require the "status:" attribute value in inetnum, inet6num and aut-num objects to be completely in UPPERCASE.
> 
> This business rule will apply when creating or updating these object types.
> 
> If the "status:" value is not completely in uppercase, the software will automatically convert the value to uppercase and add a warning in the update response, immediately following the "status:" attribute: "***Warning:   The status value was converted to uppercase". The update will not fail if this happens.
> 
> We plan to deploy this change in the next Whois release (provisionally, version 1.102).
> 
> (2) Cleanup existing non-uppercase "status:" values in the RIPE database
> 
> Following the Whois release, we will run a cleanup script to convert all existing non-uppercase "status:" values to uppercase.
> 
> We found approximately 21,492 out of 4,217,303 inetnum objects; 2,577 out of 655,856 inet6num objects; 0 out of 37,056 aut-num objects, with a non-uppercase "status:" value.
> 
> As only the case of the "status:" value is changed, the objects remain syntactically the same. We will not send notification emails to maintainers for the change, apart from notifying the db-wg. The "last-modified:" value will not change, but a new object version will be created (so the change will be visible in the version history).
> 
> All changes will be visible in NRTM as the objects are updated. The change can be considered a NOOP.
> 
> All changes will be visible in that night's database dump and split files.
> 
> We expect the cleanup script to take approximately an hour to complete all updates. 
> 
> Cleanup script updates will be applied cooperatively between normal updates, we do not expect any interruption to Whois updates.
> 
> Please let me know if you have any questions.
> 
> Regards
> Ed Shryane
> RIPE NCC
> 
> 
>> On 27 Jul 2021, at 15:59, denis walker <ripedenis at gmail.com> wrote:
>> 
>> Colleagues
>> 
>> The chairs recognise a consensus to force "status:" values to
>> uppercase on updates and for the RIPE NCC to perform a one time update
>> across the database to convert existing "status:" values to uppercase
>> where necessary.
>> 
>> We ask the RIPE NCC to go ahead with this process and inform the
>> community of their implementation plans.
>> 
>> regards
>> William & denis
>> co-chairs DB-WG
>> 
>> On Mon, 7 Jun 2021 at 11:09, Edward Shryane via db-wg <db-wg at ripe.net> wrote:
>>> 
>>> Dear colleagues,
>>> 
>>> In Remco van Mook's presentation to the Address Policy WG at RIPE 82: "What Colour Is My IP(v4) space? PI Addresses and LIRs", he pointed out that resource "status:" attribute values may not always be all UPPERCASE.
>>> 
>>> This is because the RIPE database is case-insensitive and case preserving in general, meaning a "status:" attribute value is the same regardless of the case.
>>> 
>>> However, this may make parsing this value more difficult as the case should be ignored.
>>> 
>>> Should the DB team implement a rule to always force the "status:" attribute value to uppercase, for consistency?
>>> 
>>> Regards
>>> Ed Shryane
>>> RIPE NCC
>>> 
>>> 
> 
>