problems loading ripe.db ...
ripe at foobar.dk ripe at foobar.dk
Tue May 15 14:08:30 CEST 2007
Hello there I had some problems in the beginning of 2007 and I got some direction from Agoston Horvath (thanks btw). It kinda solved some of the problems, but I had to run the DB as a static DB to avoid crashes... Anyway, due to some hardware problems we decided to rebuild the whois server (version 3.3) on a new box. Agoston Horvath wrote that there would be a new whois server avail. and 3.3 was old... but I can only find ver. 3.3. Question: Is ver. 3.3 the latest avil. version? If not, where do I pickup the latest and best (not via you CVS link ...) ? version 3.3: Compiling took ages due to a stripped RHEL (prerequisites not in place) but it looks like I've succeeded at last. Loading SAMPLE went OK - but I have problems with RIPE DB (would it be an idea for future releases to store RIPE.CURRENTSERIAL within the ripe.db.gz file ...just for the ease of it ?) First - I'm running a MySQL 4.1.20 but that is the only *big* difference in the list of prerequisites. Second - a minor hack applied to <installdir>/bin/SQL/create.first-stage since I apparently had som problems in the "auto" creation of the RIPADMIN. I added the following vars. to the script which helped (somehow these vars. was not passed on from my rip.config file: RIPADMIN_DBHOST=localhost RIPADMIN_DBPORT=3306 RIPADMIN_DBUSER=<my_db_user> RIPADMIN_DBPASS=<my_passwd> RIPADMIN_DBNAME=RIPADMIN Running make_db with option -0 runs fine :-) Running make_db (in the ex. below with DEBUG enabled) with option -1 fails: nohup /usr/local/whoisserver/bin/make_db -c /usr/local/whoisserver/conf/rip.config -s RIPE -1 2>&1 [cut..cut] [0] 0.03s OK [aut-num:AS8387][U:ADD] [0] 0.04s OK [aut-num:AS1205][U:ADD] [0] 0.02s OK [aut-num:AS15631][U:ADD] [0] 0.02s OK [aut-num:AS8925][U:ADD] [0] 0.06s OK [aut-num:AS12565][U:ADD] [0] 0.04s OK [aut-num:AS8787][U:ADD] [0] 0.03s OK [aut-num:AS15646][U:ADD] [0] 0.02s OK [aut-num:AS15649][U:ADD] [0] 0.02s OK [aut-num:AS15650][U:ADD] [0] 0.02s OK [aut-num:AS15651][U:ADD] [0] 0.03s OK [aut-num:AS8602][U:ADD] [0] 0.04s OK [aut-num:AS13211][U:ADD] [0] 0.04s OK [aut-num:AS12508][U:ADD] [0] 0.03s OK [aut-num:AS13004][U:ADD] [0] 0.06s OK [aut-num:AS15654][U:ADD] [0] 0.04s OK [aut-num:AS15655][U:ADD] [0] 0.03s OK [aut-num:AS15656][U:ADD] [0] 0.03s OK [aut-num:AS15508][U:ADD] *** ERROR: Loader failure [139]. Exiting\n + '[' 1 -ne 0 ']' + error_exit 'Error loading database=RIPE for source=RIPE, /usr/local/whoisserver/bin/make_db exiting' + error Error loading database=RIPE for source=RIPE, /usr/local/whoisserver/bin/make_db exiting + echo '************** ERROR ***************' ************** ERROR *************** ++ eval /bin/date ''\''+%H:%M:%S'\''' +++ /bin/date +%H:%M:%S + echo '*** ' 12:31:36 Error loading database=RIPE for source=RIPE, /usr/local/whoisserver/bin/make_db exiting *** 12:31:36 Error loading database=RIPE for source=RIPE, /usr/local/whoisserver/bin/make_db exiting + echo '************************************' ************************************ + echo + exit 1 [end..end] I've looked for the error code [139] in the src, but with no *hey-here-is-the-solution* result. Question: What does the error code imply ? The exit/error messages on tty says: [root at powder DB]# 0 rows affected [ INSERT mnt_routes SELECT 0, 19974, mntner.object_id, 2 FROM mntner WHERE mntner.mntner='planetwebhost-mnt' ] *** died: +1576 modules/ud/ud_core.c *** Backtrace: Using host libthread_db library "/lib/tls/libthread_db.so.1". [Thread debugging using libthread_db enabled] [New Thread -1208465184 (LWP 30785)] 0x005777a2 in _dl_sysinfo_int80 () from /lib/ld-linux.so.2 ./load_all.sh: line 15: 30782 Broken pipe ${UNZIP1} ${object_file} 30783 Exit 141 | eval ${FILTER} 30785 Segmentation fault | $LOADER -L $1 -p ${PROPERTIES} -s ${SOURCE} + check_error 'Loader failure' 139 + '[' 139 -ne 0 ']' + echo '*** ERROR: Loader failure [139]. Exiting\n' + exit 1 I thought there was an error/inconsistency in the input file (ripe.db.gz) but ripe.db.gz from yesterday (May 14th) causes the same problem. So I guess the problem lies somewhere in my binaries (the loader) or in mysql ? Running make_db with the -2 option crashes as well - but it succeeds loading much more data. Apparently it fails trying to update: [root at powder DB]# tail -30 RIPE-load-type-2-run.log [0] 0.03s OK [as-block:AS3416 - AS3459][U:UPD] [0] 0.03s OK [as-block:AS3412 - AS3415][U:UPD] [0] 0.03s OK [as-block:AS4383 - AS4404][U:UPD] [0] 0.03s OK [as-block:AS4405 - AS4430][U:UPD] [0] 0.03s OK [as-block:AS34816 - AS35839][U:UPD] [0] 0.03s OK [as-block:AS35840 - AS36863][U:UPD] [0] 0.03s OK [as-block:AS30720 - AS30979][U:UPD] [0] 0.03s OK [as-block:AS30981 - AS30981][U:UPD] [0] 0.03s OK [as-block:AS31000 - AS31743][U:UPD] [0] 0.04s OK [as-block:AS30980 - AS30999][U:UPD] [0] 0.03s OK [as-block:AS36864 - AS37887][U:UPD] [0] 0.03s OK [as-block:AS37888 - AS38911][U:UPD] [0] 0.03s OK [as-block:AS38912 - AS39935][U:UPD] [0] 0.03s OK [as-block:AS39936 - AS40959][U:UPD] [0] 0.03s OK [as-block:AS40960 - AS43007][U:UPD] *** ERROR: Loader failure [139]. Exiting\n + '[' 1 -ne 0 ']' + error_exit 'Error loading database=RIPE for source=RIPE, /usr/local/whoisserver/bin/make_db exiting' + error Error loading database=RIPE for source=RIPE, /usr/local/whoisserver/bin/make_db exiting + echo '************** ERROR ***************' ************** ERROR *************** ++ eval /bin/date ''\''+%H:%M:%S'\''' +++ /bin/date +%H:%M:%S + echo '*** ' 13:54:00 Error loading database=RIPE for source=RIPE, /usr/local/whoisserver/bin/make_db exiting *** 13:54:00 Error loading database=RIPE for source=RIPE, /usr/local/whoisserver/bin/make_db exiting + echo '************************************' ************************************ + echo + exit 1 The tty output for the option "-2" run says: [cut..cut] + echo 'finished create.second-stage...' + '[' 2 -eq 2 ']' ++ date + echo Tue May 15 13:53:43 CEST 2007 + echo 'Loading tables - 2 of 2 pass' + loadpass 2 + for object_file in '${LIST}' ++ date + echo Tue May 15 13:53:43 CEST 2007 Loading /usr/local/whoisserver/var/tmp/load/RIPE/ripe.db.gz + test -f /usr/local/whoisserver/var/tmp/load/RIPE/ripe.db.gz + check_error '/usr/local/whoisserver/var/tmp/load/RIPE/ripe.db.gz not found' 0 + '[' 0 -ne 0 ']' + gzip -cd /usr/local/whoisserver/var/tmp/load/RIPE/ripe.db.gz + eval cat ++ cat + /usr/local/whoisserver/bin/loader -L 2 -p /usr/local/whoisserver/conf/rip.config -s RIPE D: Making SQL connection to RIPE at localhost ...OK starting processing stream Object cannot be retrieved:[as-block: AS0 - AS0 descr: IANA held ASN block remarks: These AS numbers are reserved by IANA org: ORG-IANA1-RIPE admin-c: IANA1-RIPE tech-c: IANA1-RIPE mnt-by: RIPE-DBM-MNT mnt-lower: RIPE-DBM-MNT changed: ripe-dbm at ripe.net 20070101 source: RIPE ] *** died: +1144 modules/ud/ud_process_stream.c *** Backtrace: Using host libthread_db library "/lib/tls/libthread_db.so.1". [Thread debugging using libthread_db enabled] [New Thread -1208882976 (LWP 31113)] 0x005777a2 in _dl_sysinfo_int80 () from /lib/ld-linux.so.2 ./load_all.sh: line 15: 31111 Broken pipe ${UNZIP1} ${object_file} 31112 Exit 141 | eval ${FILTER} 31113 Segmentation fault | $LOADER -L $1 -p ${PROPERTIES} -s ${SOURCE} + check_error 'Loader failure' 139 + '[' 139 -ne 0 ']' + echo '*** ERROR: Loader failure [139]. Exiting\n' + exit 1 [end..end] So...the loader errorcode is still "139" but it fails during UDP, not ADD Any hints ? I'm a little lost ;-( Should I give it a try with the "spilt" version of the current ripe.db.gz ? If yes, is the right way to point at the split files like this (in ./bin/make_db): # List of object files (wildcards are allowed) #LIST=${LIST:-$OBJDIR/sample.db.gz} #LIST=${LIST:-$OBJDIR/ripe.db.gz} LIST=${LIST:-$OBJDIR/*.db.gz} I hope there is enough (and not to much...) info. to help pin-pointing where to look. Best regards, Peter
[ db-help Archives ]