New database software update

[Marten Terpstra <]

all,

It's time to let you know what has happened wth RIPE database software
development.  After some more experimenting to check feasibility we have
progressed quite far into a complete rewrite of the database software. 
We have completed the system design and started to implement various
modules. 

The new design has the following features

	- fully (and we *mean* fully!) configurable from a config file

	- separate files (and indices) to store different databases

	- dynamic updates and deletes as opposed to batch only

	- possibility for update protocol other than e-mail

	- guarded fields designed in

	- dynamic update of secondary databases possible


Features which have been requested but are not designed in:

	- preserve the order of tags within object 

	- preserve the position of remark or comment lines within objects

These two features turned out to be hard to specify in the light of
updates and even harder to implement.  They might go in the next release
if someone can come up with a consistent and intuitive spec. 


The new software will be implemented in the perl language.  So those of
you who do not have it installed and want to use the RIPE database
software should plan to install perl.  Of course you won't need perl if
you just want to use a whois client and send in updates by mail. 
Implementing in perl instead of shell, awk and C has three benefits:
coding in perl ist fast, subroutines can be used (missing from std awk)
and the result is portable.  We have a copy of the alpha software
running on a PC under UNIX.  Together with a notebook this gives you a
"portable" RIPE database. 

The first module completed is the whois server (net.c and netdbm.c
replacement).  This server runs as its own daemon process rather than
under inetd.  The measured performance on a SUN ELC before optimising is
5 queries/s on average and the server can sustain peaks of 50 queries
arriving in parallel.  The new server is compatible with the current one
and has some added features: the source databases (RIPE, MERIT, NIC
etc.) can be selected individually rather than either RIPE or all, the
same connection can be used for multiple queries (nice for tools like
prtraceroute), a grep type query as well as nice logging. 

We have an alpha version of the whois server and the database indexing
tool running.  The next part to be tackled is the update procedure
including the guarded attributes.  The last part will be the current
mail interface for the update procedure. 

Impact: Sometime in the next few weeks (after another announcement) we
will start using the new server on whois.ripe.net.  This should be
transparent.  Once this is running we will implement the backlogged new
tags such as the DMZ specifics in the network and the operatinal contacts. 
Sometime later we will start using the new update procedures.  This will
mean that updates mailed in will appear in the database more quickly. 
It will also mean that we will be more stringent in checking updates,
because there will be less human intervention from NCC staff.  So you
should expect more bounced updates, but they will also happen much more
quickly.  Diagnostics will be better because the dynamic update
procedure will have access to the current database.  This will mean that
heuristic checks of updates can be better. 

If you have any questions, please do not hesitate to ask them either
puclicly for the benefit of all or privately. 

The NCC people working on this are me (design, coding and testing) and
Daniel (design and testing). 

Cheers,

-Marten