Announcement: Release of RIPE datase software 2.00

Dear all,

This message is to announce the new 2.00 release version of the RIPE
database. Many new features and bug fixes have been added since the
previous beta release. The software is available from:

ftp://ftp.ripe.net/ripe/dbase/software/ripe-dbase-2.00.tar.gz

The following lists the most interesting new features since the previous
non-beta release (you can do 'whois -h whois.ripe.net HELP' to get the
RIPE database HOWTO that includes more detailed information on the new
features):

- AUTO NIC handle assignment support. The (slow) finger tool is not
  needed any more (see the HOWTO for more details on how to use this
  new feature).

- hierarchical authorization for creation of 'inetnum' and 'domain'
  objects. Use a 'mnt-lower: YOUR-MAINTAINER' attribute for these
  objects to use the authorization of YOUR-MAINTAINER for the creation of
  objects direct below in the hierarchy your object with the 'mnt-lower:' 
  line.

- role object support. Get a template by doing 'whois -h whois.ripe.net
  -t role'. All attributes are already known, except for the 'trouble:'
  attribute that can contain any free text. Note: references to the role
  object may not be used in the 'admin-c:' attribute, however the use of
  the object in 'tech-c:' & 'zone-c:' is fine.

- prefix notation for 'inetnum:' updates & classless range support for
  whois lookups

- extended 'country:' attribute syntax implemented, more then one country
  on a line and more 'country:' attributes per object are allowed now.

- the special RIPE whois client is now fully supported and included in
  the distribution.

- much stronger syntax checking (=as strong as specified by the original
  database specification) for NIC handles, phone numbers, E-mail
  addresses, domain names (Some people might not like this ;-)).

- network updates, a method for directly updating objects by use of the
  program 'networkupdate'. The use of this method is restricted by an
  accesslist and currently only intended for the maintenance people of
  the database.
  
- BSD DB dbm package is fully supported. This dbm package is a fast,
  platform independant and reliable dbm package.

- No limitations on the number of hits (no matter what indexing package
  you use): 
  a query like 'David' will give you an answer and person names like
  'John George' will be found
  
- new and more intelligent indexing program 'indexdb' that can do a raw
  index or a clean index (-c option) and automatically detects if it is
  dealing with a classless index. netdbm & cleandb are now obsoleted.
  Indexing speed has been improved by a factor of at least two on average
  although the number of indexed values is more then twice the previous
  number. RAM memory consumption is also limited to about 6 Mb depending
  on your architecture.

- reverse lookup of objects (whois -i option), this enables you to find
  for example the routes with a given origin by doing:

  whois -h whois.ripe.net -i origin AS3333

- Nearly real time mirroring of the RIPE database. This feature is fully
  supported but has a few shortcomings that will involve a bit more
  administration work until the 'stored/processed' attribute and better
  crash recovery routines are implemented.

- experimental inet6num support for ipv6 network objects, including
  support for -L, -m, -M queries! Note: this is not an approved object
  whatsoever. It is intended for testing ipv6 indexing but can serve as a
  good starting point for further discussion. Get a template by doing:
  'whois -h whois.ripe.net -t inet6num'

and many more small changes and fixes. Please read the release notes at
the bottom of this message for more information and discussion about
these topics.

Please send bug reports/fixes directly to <ripe-dbm at ripe.net> and I will
fix the problem but be fast since I will move to new job at ISI very
soon.

And finally:

Thanks for all your help, patience with bugs, bug reports and patches!

Kind regards,

David K.
---

***Warning*** for upgraders:

A reindex (indexdb -c -s SOURCE) is necessary since the internal index
formats changed!


RELEASE NOTES (RIPE database version 2.00):


- AUTO NIC handle assignment support. The (slow) finger tool is not
  needed any more (see the HOWTO for more details on how to use this
  new feature). Gabor Kiss provided be with some very usefull patches for
  part of the work.


- hierarchical authorization for creation of 'inetnum' and 'domain'
  objects. Use a 'mnt-lower: YOUR-MAINTAINER' attribute for these
  objects to use the authorization of YOUR-MAINTAINER for the creation of
  objects direct below in the hierarchy your object with the 'mnt-lower:' 
  line.


- role object support. Get a template by doing 'whois -h whois.ripe.net
  -t role'. All attributes are already known, except for the 'trouble:'
  attribute that can contain any free text. Note: references to the role
  object should and may not be used in the 'admin-c:' attribute, however
  the use of the object in 'tech-c:' & 'zone-c:' is fine.


- Nearly real time mirroring support:
  
  What is it:
  
  The database software now keeps serial numbers of all updates. If you
  have been added to an accesslist, it it possible to retrieve all the
  changes to the database. A program 'syncdb' has been added to the
  distribution that can retrieve these changes and update your mirror
  database automatically. This program is intended to be put in a
  cronjob.
  
  The whois erver has an access list so you first need to ask the site
  to be mirrored to add your host to the access list.

  Please let me now at <ripe-dbm at ripe.net> if you are interested in
  trying out this feature, and I will add the IP number of your site to
  the access list.
  
  Short discussion:
  
  Note that this is the first version, it currently uses seperate files
  internally for every update. We plan to change this since this is not
  really what we want with 2000 (atomic) updates on some days. 
  
  Furthermore, we use currently an ever increasing integer serial number.
  A proposal (stored/processed attribute) on how to this right has
  already been circulated to the RIPE database working group and discussed
  on the RIPE meeting. Implementation of this attribute will also allow
  for a better crash recovery behavior.


- No limitations on the number of hits (no matter what indexing package
  you use): a query like 'David' will give you an answer and person
  names like 'John George' will be found

  
- New and more intelligent indexing program 'indexdb' that can do a raw
  index or a clean index (-c option) and automatically detects if it is
  dealing with a classless index (from the config file). netdbm & cleandb
  are now obsoleted. The indexing speed has also been improved.

  
- BSD berkeley DB support:

  The database software can now use the Berkeley DB dbm package.
  According to many people this package is superior to most other dbm
  packages that perl can use (unlimited size of keys/values, platform
  independent db files). We have put the current sources for this
  package at ftp://ftp.ripe.net/tools/db.1.85.tar.gz

  It should be possible with the BSD DB package to increase the
  OVERFLOWSIZE and as result get rid of (most) of the overflow files.
  However, it is not clear if this is desirable since it doesn't seem to
  make the software more memory/space/speed efficient.


- a new help and HOWTO use the RIPE database file is available
  
  use 'whois -h whois.ripe.net HELP' to get the most recent version.
  
  The document gives a short introduction on the database, tells you how
  to create/update/delete objects, contains a section with tips and common
  problems and gives pointers to other documentation about the database.
  
  
- BSDI/perl memory leak problems

  Due to a memory leak with perl on BSDI machines we needed to make the
  code more memory conscious. enread.pl has been rewritten completely and
  uses now less memory and as an (intended) side effect became faster
  when reading big objects. Also the special input routine for reading in
  an object from an E-mail has undergone the same optimizations.


- Most patches (for greater speed and less memory consumption) that I
  received from Curtis Villamizar <curtis at ans.net> have been included.
  Thanks Curtis!
  
      
- The direct network updating method is now fully supported.

  Currently this method is only intended for use by the registry itself,
  however it is possible to add support in the maintainer objects if
  desired by the RIPE community.
  
  You can now do you updates very rapidly by giving the command
  'networkupdate' and input your object via STDIN. Of course, this is
  also protected with an access list.


- The special magic to overrule the maintainer protection needs a
  password now. Syntax will be published after the main registries
  support this...


- All support for the historical 'guarded' objects have been removed. The
  guardian attribute can be obsoleted if the database working group
  wishes to. Present guardian attributes will act as a 'notify:'
  attribute.

- the special RIPE whois version is now fully supported and included in
  the distribution.


- cryptpw

  is a new (small) program that encrypts a password with a choosen salt
  for use in the maintainer objects.
  
  
- dbupdate -T
  
  Makes it very easy to install your own test database. The only
  difference with a normal update is that it doesn't need manual
  authorization for creation of maintainer objects and that it will add a
  warning to your acknowlegment message that this is not the normal
  behavior...


- status: attribute as described in ripe-127 fully supported


- whoisd changes:

  -D runs in daemon mode otherwise in 'inetd' mode ('inetd' mode is
     untested)
  
  The number of disk accesses have been further minimized and should
  result in a better performance.

    
- line continuation by use of a "\" character at the end of an attribute
  line in your update message. The database software will then
  automatically concatenate the next line with the previous one.


- prefix notation for 'inetnum:' updates & whois lookups


- extended 'country:' attribute syntax implemented. You can now specify
  multiple countries in a 'country:' attribute line.


- much stronger syntax checking for NIC handles, phone numbers, E-mail
  addresses, domain names (Some people might not like this ;-)). The
  syntax was too relaxed and follows our own specifications more closely.


- perl5 patches. perl5 is still not supported. However, the biggest
  problems should be gone now (the software runs with perl5 at home with
  Linux & perl5.001m). The Makefiles have been changed to support
  perl5 and some of the database tools as 'whois' can be configured to
  run with perl5 by default (and work with it!). See the Makefiles on how
  to configure this. You are welcome to experiment with perl5 and we are
  interested in your comments!

  
- Tony Bates <Tony.Bates at mci.net> recent patch for a problem with the
  maintainer notifications. Thank you Tony. 


- databases and serial number files that don't exist are now
  autogenerated.


- Config file has been simplified by removing some options that can be
  determined by the database software itself. Of course, also a few new
  options appeared. Everything is documented in the config file itself.


- The Makefiles have been rewritten to allow for easier installation and
  more logical behavior when doing make [[un]install].

  
- many small fixes and clean up of the code to make above changes
  possible. 
  
  - bug that caused problems when deleting objects with(out) syntactic sugar

  - bug that caused a crash when one tries to delete an object
    that cannot be updated (object is mirrored from other database)


- List of OS's that we know the RIPE database works with:
  
  - SUNOS
  - BSDI
  - Linux
  
  (Please inform me with you own results so that I can update this list!)
  
  Automatic on the fly patches implemented in the new Makefiles should
  make it possible to run the database on Solaris. This is untested for
  now and you will need a special perl4 that is available from
  ftp://ftp.ripe.net/tools/perl4.036-for-solaris.tar.gz. Peter Haag has
  made the perl version and patches available to me. Thanks!
 

What didn't make it in this release:

- checking for relations between objects:

  - disallow deletion of referenced object
  - disallow creation of objects that reference non exiting objects


- the list of current proposals (soon to be reposted to the
  <db-wg at ripe.net> maillist.