[db-wg] Redevelopment of the RIPE Database software is complete

Congratulations to all of the RIPE DB team members on the new system. Hard to believe the previous one lasted that long :)

Joao Damas

On 25 Feb 2013, at 13:23, Kaveh Ranjbar <kranjbar at ripe.net> wrote:

> Dear Colleagues,
> 
> The RIPE NCC is proud to announce that our redevelopment of the RIPE Database software is complete.
> 
> At RIPE 63, the RIPE NCC announced that we were starting a project to replace the 11-year old legacy RIPE Database software with a new system. At RIPE 64, we were able to report completion of the new query software and the start of the update project. The new query software has been handling all queries for almost a year, and the new update software is now in place and handling all updates.
> 
> This new software is able to scale the RIPE Database to meet the expanding requirements of the members, the RIPE community and the RIPE NCC. It will also improve our response times on bug fixes, changes and major feature requests. With near complete unit test coverage of the code, higher level integration tests and extensive end to end 'use case' testing we have greatly improved the maintainability and reliability of the entire RIPE Database infrastructure.
> 
> The new systems were rolled out incrementally, so the process was almost invisible to users of the service. The new software is mostly backwards compatible, although we have introduced some long-anticipated features and requirements. On our website, we are publishing software release notes that are updated for each new release. These can be found here:
> 
> http://www.ripe.net/data-tools/db/release-notes
> 
> With the new software now in place, we have a lot of opportunity for future expansion of the service and to implement features desired by the members and the wider community. During the redevelopment process we also had the chance to improve the behaviour of our system. Some of the areas improved are:
> 
> - Some community-approved policies which were not fully enforced within the RIPE Database are now enforced by the new software for all RIPE address space.
> - A lot of issues in the authorisation are resolved. Resource holders now have full control over the resources they maintain and situations where they might lose control of their resources by mistake are no longer possible.
> - All resources allocated or assigned by the RIPE NCC have their entries in the RIPE Database protected against accidental deletion by users.
> - The new system accepts IPv6 network addresses in different formats, but always normalises the input to a single format which is used for storing, searching and displaying the records. If you query, create, modify or delete an object, no matter what format you specify the address in, it will first be normalised, then the operation will be actioned. In general, the new system is much smarter in the interpretation of IPv4 and IPv6 addresses on input and can handle different notations.
> - New error messages are more comprehensive and the system provides more specific error messages and logging and in many different situations.
> - The new system is about four times faster on queries and three times faster on processing updates.
> 
> We have published an article on RIPE Labs with details about the features in the new system:
> https://labs.ripe.net/ripe-database/changes-to-the-processing-of-ripe-database-updates
> 
> Internally, the new system has a lot of benefits as well:
> 
> - The new software has been designed from scratch with distribution in mind. Additional query and update nodes can easily be added for redundancy and performance.
> - Hot deployments of new software are now possible with no interruption to the service.
> - Some additional security measures have been introduced to improve the overall security of the system.
> - Logging and debugging facilities have been improved for easier support and maintenance.
> - The whole system is integrated and self-contained. It doesn't rely on external scripts for any task (such as the import of external data for mirroring or automatic data clean-up of unreferenced objects) and so is much easier to maintain.
> - Internal management and monitoring facilities have been improved and integrated into the system, so support and maintenance can provide users with direct feedback from the production system.
> - The memory footprint of the new system is much smaller.
> - The code base is much cleaner, making it easier to understand and maintain. Java is now the single language, replacing a mix of C, Perl and Java code.
> 
> So, what's next?
> 
> - Currently we are working on a new set of RIPE Database documentation that will be clean, concise and linked directly to each release, including a further review of error messages and all information provided when interacting with RIPE Database.
> - Investigate with the community the integration of RIPE Database authentication with the single sign-on (SSO) service for RIPE NCC services.
> - Implementing the IETF WEIRDS RDAP Protocol, fully integrated into the RIPE Database API, as a cooperative software development project with APNIC.
> - Using business rules to 'fix' parts of resource objects that users should not change, then allowing all objects to be updated by normal methods rather than special editors in the LIR Portal.
> - Providing the new software to the public and making the project open source and accessible for third parties.
> - Liaise with the community on new features and data clean-up as announced at the last RIPE Meeting.
> 
> As always, you can contact us at <ripe-dbm at ripe.net> for any issues, feature requests or suggestions.
> 
> Kind regards,
> 
> Kaveh Ranjbar
> Database Group Manager
> RIPE NCC
>