[db-wg] Two Proposals

PROPOSED

    1) RIPE NCC shall be tasked to begin updating the following file on a near
       real-time basis, as number resource assignments are transferred either
       into or out of the authority of RIPE:

       ftp://ftp.ripe.net/pub/stats/ripencc/delegated-ripencc-extended-latest

    2) RIPE NCC shall be tasked to begin making the above file available for
       incremental remote update via the rsync protrocol.

RATIONALE


Background

At the present time the most reliable and up-to-the-minute information
indicating which RIR any given number resource is being administered by
is contained within the so-called "daily stats files" that are generated
and published by the five RIRs:

  ftp://ftp.arin.net/pub/stats/arin/delegated-arin-extended-latest
  ftp://ftp.ripe.net/pub/stats/ripencc/delegated-ripencc-extended-latest
  ftp://ftp.apnic.net/public/apnic/stats/apnic/delegated-apnic-extended-latest
  ftp://ftp.lacnic.net/pub/stats/lacnic/delegated-lacnic-extended-latest
  ftp://ftp.afrinic.net/stats/afrinic/delegated-afrinic-extended-latest

Additionally, a single file representing the union of all of the above five
files, along with additional data relating to (some but not all) IANA
reserved number resources is also available:

  https://nro-extended-stats-from-rpki-team-prod.s3.eu-west-1.amazonaws.com/nro-extended-stats

Note however that the data in this (NRO stats) file lags behind the
contents of the five RIR daily stats files, and the content of this
file is therfore often less fresh and up-to-date than the information
contained in the five RIR stats files.


Problem Statement (Proposal 1)

Because the five RIR stats files are only updated once a day, and at
different times for each RIR, and because number resources are, with
increasing frequency, being transfered between regions, it is nowadays
frequently the case that there will be, in effect, instances of "version
skew" between the five RIR stats files.  Instance of such inter-RIR
stats files version skew can render it impossible to know, based on the
stats files, which RIR currently administers a given number resource.

Instances of internet-RIR stats file version skew can arise, and in practice
they actually do arise as follows:

   *) RIR 'A' transfers a number resource `R' to RIR `B'.

   *) RIR `B' records its administration of `R' in its stats file which
      is then published within 1 hour.

   *) RIR `A' records the fact that it no longer administers `R' by removing
      the associated line from its stats file, and RIR `A's stats file is
      then published 23 hours later.

Under the above scenario, the union of all five of the daily RIR stats files
will indicate that *both* RIR `A" and also RIR `B' are the responsible RIRs
for the resource `R' for a period of 22 hours.

An actual example:

    As of approximately 07:13 (-0700 / PDT) on 2021-09-17 the following blocks
    were listed in *both* the stats file for ARIN and also the stats file for
    RIPE:

    140.228.32.0/19
    140.228.64.0/19

    (Evidence indicates that these 2 block have just been transferred from
    administration by the ARIN region to administration by the RIPE region.)

The problem of version skew between the various daily stats files that
are currently published by the various RIRs could be largely or entirely
eliminated if all five RIRs henceforth began to update these files on a
near real-time basis.  RIPE can and should begin doing this as an example
to the other RIRs, which may then follow suit.


Problem Statement (Proposal 2)

The RIR daily stats files, including the one that is generated and published
by RIPE NCC, consist of plain text files which are themselves composed of
a sequence of plain text lines.  As a general matter, very few of these
lines change, even over time scales as long as a month.  Changes to these
files, if any, are entirely minimal when viewed over even shorter time scales,
e.g. a day or less.

The rsync protocol and rsync servers and clients are especially adept at 
keeping remote copies of exactly such plain text files up-to-date and
synchronized with a master copy stored elsewhere on the Internet.  They
do so while minimizing bandwidth usage by transfering only those portions
of a given file that have changed since the last time a local mirrored
copy of the file was updated.  Publishing RIPE's stats file via rsync
would thus be beneficial to all concerned by minimizing bandwidth usage
on both the sending and receiving end.  Ideally, RIPE NCC could and should
take the lead in making its stats file available via rsync.  Once it has
done so, other RIRs can be encouraged to follow suit and do likewise with
their own daily stats files.


Regards,
rfg


P.S.  On a personal note I find it nearly unfathomable that these "daily"
stats files are being updated and published only once a day.  This seems to
me almost reminicent of technological solutions considered appropriate
in the prior century.