From wilhelm at ripe.net Tue Sep 6 15:30:04 2005 From: wilhelm at ripe.net (Rene Wilhelm) Date: Tue, 6 Sep 2005 15:30:04 +0200 (CEST) Subject: [ris-int] riswhois request: reporting parent routes (fwd) Message-ID: Hi, Though it's nice to have an active user providing suggestions, I have the feeling his latest request would fall in the category "feature creep", making the program larger, more complex and the output harder to understand. However, before sending him a polite no-no, I'd like to ask how others in RIS feel about this. Should RISwhois embark on the trail of cross examining other routes (parents, daughters, siblings, ...) or not? -- Rene ---------- Forwarded message ---------- Message-ID: <431C426B.1060807 at priv.onet.pl> From: Andrzej Adam Filip Sender: ris-users-admin at ripe.net To: ris-users at ripe.net Date: Mon, 05 Sep 2005 15:04:43 +0200 Subject: riswhois: reporting parent routes Could riswhois generate statistics of "parent routes" for given route? [ parent, not ancestor] My goal is achieve better coverage of IP->AS mappings based on riswhois provided data. Unanimous AS reports (after ignoring default route) give very good starting point I would like to extend. For 65.54.160.0/19 I would like to know how many of 58 rispeers reporting the route reports also 65.52.0.0/14 route as belonging to AS8070. e.g. route: 65.54.160.0/19 origin: AS12076 [...] num-rispeers: 58 numrp-parent: 58 65.54.160.0/19 AS8070 P.S. As security precaution riwhois may limit number of parent routes reported to e.g. <10 ---------------------------------------------------------- $whois -h riswhois.ripe.net -- 65.54.162.200 [leading comments and default route entry skipped] route: 65.52.0.0/14 origin: AS8070 descr: MICROSOFT-CORP---MSN-AS-BLOCK - Microsoft Corp lastupd-frst: 2005-07-21 08:28Z 202.12.28.190 at rrc00 lastupd-last: 2005-09-05 07:49Z 193.111.172.55 at rrc03 seen-at: rrc00,rrc01,rrc03,rrc04,rrc05,rrc06,rrc07,rrc10,rrc11,rrc12,rrc13,rrc14 num-rispeers: 58 source: RISWHOIS route: 65.54.160.0/19 origin: AS12076 descr: HOTMAIL-AS - Hotmail Corporation lastupd-frst: 2005-06-29 08:05Z 195.66.224.151 at rrc01 lastupd-last: 2005-09-05 07:49Z 193.111.172.55 at rrc03 seen-at: rrc00,rrc01,rrc03,rrc04,rrc05,rrc06,rrc07,rrc10,rrc11,rrc12,rrc13,rrc14 num-rispeers: 58 source: RISWHOIS ---------------------------------------------------------- -- Andrzej [en:Andrew] Adam Filip anfi at priv.onet.pl anfi at xl.wp.pl All that is necessary for the triumph of evil is that good men do nothing -- Edmund Burke (1729-1797) From arife at ripe.net Tue Sep 6 15:32:13 2005 From: arife at ripe.net (Arife Vural) Date: Tue, 6 Sep 2005 15:32:13 +0200 Subject: [ris-int] Re: [Fwd: backup bottlenecks and possible improvements] In-Reply-To: <4305B39B.6080607@ripe.net> References: <4302022D.5040607@ripe.net> <4305B39B.6080607@ripe.net> Message-ID: <20050906133213.GC28526@ripe.net> > It looks like we are backing up the correct data for RIS. It is only the > "raw data". > > I had a quick look at the raw data, and I think we can save a *lot* of > space. A comparison tells me that if we convert from gzip to bzip2, we > would save something like 25% of the space. This is only 100 Gbyte or > so, but nice to have. I think it's a good idea. We can convert from gzip to bzip2. Few applications that use rawdata need to be changed to use bzip2. If you agree to use bzip2, I can proceed, and do the change. I guess we need to announce our users. Arife > > In fact, perhaps more generally we should look at our various compressed > files and convert from gzip to bzip2. We already did this for the Whois > database logs a few years ago, and saw a large savings. > > Andrei Robachevsky wrote: > > >Shane, > > > >FYI. This may help in the RIS backup design. > > > >Andrei > > > >-------- Original Message -------- > >Subject: backup bottlenecks and possible improvements > >Date: Tue, 9 Aug 2005 16:23:59 +0200 (CEST) > >From: gerard at ripe.net (Gerard Leurs) > >To: andrei at ripe.net, gerard at ripe.net, ruud at ripe.net > > > >Andrei + Ruud. > > > >I promised to write something about our current backup. > > > >Pls have a read, and when there are questions/suggestions > >I'm happy to answer them. > > > > Gerard. > > > > > >++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ > >Backup system. > > > >Short description. > > > >We backup (or rather synchronise) unique filesystems from servers > >on a frequent basis to a huge EMC diskvolume. > >At least twice a day, but for some servers-filesystems up to every > >half hour. > > > >We keep the backups for a period, so we can restore from online-backup. > >The retentionperiod depends a little bit on the server, but in general > >we have snapshots for the last 14 days + 1 week + 1 month. This provides > >for 'online' restoring files of the last 7 weeks. > > > >This strategy proves to be sufficient for almost all cases we have > >wrt restoring files, or installing and synchronisation of a replacement > >server. > > > >When we purchased our current hardware we already anticipated possible > >future growth. And possible addition of servers and services. > > > > > >Current status. > > > >Although we reduced the amount of snapshots to keep drastically we > >do not gain lots of diskspace. Most data on any filesystem does not > >change over time. And only the changed data requires extra diskspace > >on our backupserver. > > > >There is still enough EMC-diskspace for the serverbackup. > >We can even add some new servers, without hassle. > >But for the filer backups it is getting awkward. > > > > size used available > >emc1 (serverbackup) 1.6 TB 1.4 TB 247 GB > >emc2 (filers + RIS + DNSMon) 1.6 TB 1.5 TB 184 GB > > > > > >We did not anticipate that the diskusage for the former NP-services > >would have such an enormous impact on the EMC. Approx. 1 GB is solely > >taken by the full backup for RIS, TTM and dnsmon. > > > >Some rough estimates of diskusage are: > >For EMC2: > >- filer2 189 GB > > 80 GB homedirs > > 46 GB groupdirs > > 32 GB roledirs > > 21 GB ticketDB > >- filer3 183 GB > > 95 GB datadirs > > 39 GB web dirs > > 12 GB FTP dirs > >- weesp > > 236, 111, 2 GB for RIS > >- dolphin > > 94, 102 GB for dnsmon > > > >For EMC1: > >- ceiba > > 34, 34, 34, 33, 11, 132 GB, all for TTM data > >- too many other servers to report > > > > > >Future plans (2006). > > > >- Replacing our current filers with more powerfull filer, with more > > diskspac, increases demands for diskspace. > >- Adding services on the TTM-cloud, and collecting this data on a > > central server, also requires extra diskspace. > >- Adding backup of laptops also requires extra diskspace. > >- "Natural growth" of diskusage requires extra diskspace. > > > > > >How can we be pro-active for possible diskspace problems? > > > >- Reduce amount of snapshots to the absolute minimum. > > This will however not free half of the current EMC-diskspace. > > But possibly only 10%. > > And the number of snapshots is not that high anymore. > > I think not the option, which solves our diskgreed for years. > > > >- Change backup strategy. > > + Expire all files older than 1 year. And do not sync them onto the > > EMC. > > pro: saves diskspace > > con: hard and time-consuming to restore a filesystem/server > > from backups > > hard to implement > > will ot save time for the sync-operation > > admin. overhead increased by factors > > I would not consider this a good option. > > > > + Staff removes files older than x months/years. > > pro: saves diskspace > > con: time consuming for everyone > > hard to convince everyone > > will not work for dirs of websites, ftpsite, > > ticketDB and lots of other dirs. > > I would not consider this a good option. > > > > + Do not add new servers, filesystems and laptops to the backup > > Obviously one of the worst options possible. > > > >- Buy an expansion diskarray, similar to the current diskarray. > > This gives us 3 TB of extra diskspace. > > pro: no administrative workload > > no reduced quality of service > > get dedicated EMC-filesystem for filers > > be able to backup the new filers (larger diskspace) > > anticipated for additional data of services on TBs > > possibility to add more filesystems, and servers to backup > > con: invest 17.150 euro > > spend time on configuration (and on data-migration) > > > >++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ > > > > -- Arife Vural SED, RIPE NCC From gerard at ripe.net Tue Sep 6 17:15:45 2005 From: gerard at ripe.net (Gerard Leurs) Date: Tue, 6 Sep 2005 17:15:45 +0200 Subject: [ris-int] Re: [Fwd: backup bottlenecks and possible improvements] Message-ID: <200509061515.j86FFjbq016078@birch.ripe.net> Dear all.. > > space. A comparison tells me that if we convert from gzip to bzip2, we > > would save something like 25% of the space. This is only 100 Gbyte or > > so, but nice to have. Nice idea, but saving only 100GB is not much on a 1.6TB filesystem. And it is not a real solution to the diskspace-problem I was talking about. One extra concern: if you do so we will have 300GB of gzipped files on the backupserver, and 200GB of bzipped2 files. And as a result we do not have any space anymore on the backupserver. And that is exactly what I/ops is trying to solve. Pls do not do this on your own and consult ops before actually planning this! Gerard. From wilhelm at ripe.net Tue Sep 6 17:24:13 2005 From: wilhelm at ripe.net (Rene Wilhelm) Date: Tue, 06 Sep 2005 17:24:13 +0200 Subject: [ris-int] Re: [Fwd: backup bottlenecks and possible improvements] In-Reply-To: <20050906133213.GC28526@ripe.net> References: <4302022D.5040607@ripe.net> <4305B39B.6080607@ripe.net> <20050906133213.GC28526@ripe.net> Message-ID: <20050906152413.CEED66006B@localhost.ripe.net> Hi Arife, >> I had a quick look at the raw data, and I think we can save a *lot* of >> space. A comparison tells me that if we convert from gzip to bzip2, we >> would save something like 25% of the space. This is only 100 Gbyte or >> so, but nice to have. > > I think it's a good idea. We can convert from gzip to bzip2. Few > applications that use rawdata need to be changed to use bzip2. > If you agree to use bzip2, I can proceed, and do the change. I have no principal objections against a switch to bzip2 but it has to be done in a coordinated way: RISwhois depends on raw data !!! Two changes are needed: 1. libbgdump MUST support bzip2 (dynamically, user shouldn't have to worry about it, just pass any filename to bgpdump_open and that routine should do the magic of reading the gzip or bzip file in the correct way). 2. riswhoisd MUST recognise .bz2 as a valid extension for RIS raw data files when searching archive dirs for dumps Currently, I don't know when I could work on #2, but as it depends on #1 anyhow, I'll first wait for a new libbgpdump to be ready before worrying about riswhois code itself. Cheers, -- Rene > gerard wrote: > > >We did not anticipate that the diskusage for the former NP-services > >would have such an enormous impact on the EMC. Approx. 1 GB is solely > >taken by the full backup for RIS, TTM and dnsmon. :-)) no surprise, NP projects were always data hungry :P From henk at ripe.net Tue Sep 6 18:05:19 2005 From: henk at ripe.net (Henk Uijterwaal) Date: Tue, 06 Sep 2005 18:05:19 +0200 Subject: [ris-int] Re: [Fwd: backup bottlenecks and possible improvements] In-Reply-To: <20050906152413.CEED66006B@localhost.ripe.net> References: <4302022D.5040607@ripe.net> <4305B39B.6080607@ripe.net> <20050906133213.GC28526@ripe.net> <20050906152413.CEED66006B@localhost.ripe.net> Message-ID: <6.2.1.2.2.20050906180439.02cca4a0@localhost> At 17:24 06/09/2005, Rene Wilhelm wrote: >Hi Arife, > > >> I had a quick look at the raw data, and I think we can save a *lot* of > >> space. A comparison tells me that if we convert from gzip to bzip2, we > >> would save something like 25% of the space. This is only 100 Gbyte or > >> so, but nice to have. > > > > I think it's a good idea. We can convert from gzip to bzip2. Few > > applications that use rawdata need to be changed to use bzip2. > > > If you agree to use bzip2, I can proceed, and do the change. > >I have no principal objections against a switch to bzip2 but it has >to be done in a coordinated way: RISwhois depends on raw data !!! I agree, we should make the software aware of the change _and_ coordinate this carefully with our users, so that their tools don't break either. Henk >Two changes are needed: > >1. libbgdump MUST support bzip2 (dynamically, user shouldn't have > to worry about it, just pass any filename to bgpdump_open and > that routine should do the magic of reading the gzip or bzip > file in the correct way). > >2. riswhoisd MUST recognise .bz2 as a valid extension for > RIS raw data files when searching archive dirs for dumps > >Currently, I don't know when I could work on #2, but as it depends >on #1 anyhow, I'll first wait for a new libbgpdump to be ready >before worrying about riswhois code itself. > >Cheers, > >-- Rene > > > gerard wrote: > > > > >We did not anticipate that the diskusage for the former NP-services > > >would have such an enormous impact on the EMC. Approx. 1 GB is solely > > >taken by the full backup for RIS, TTM and dnsmon. > >:-)) > >no surprise, NP projects were always data hungry :P ------------------------------------------------------------------------------ Henk Uijterwaal Email: henk.uijterwaal(at)ripe.net RIPE Network Coordination Centre http://www.amsterdamned.org/~henk P.O.Box 10096 Singel 258 Phone: +31.20.5354414 1001 EB Amsterdam 1016 AB Amsterdam Fax: +31.20.5354445 The Netherlands The Netherlands Mobile: +31.6.55861746 ------------------------------------------------------------------------------ Look here junior, don't you be so happy. And for Heaven's sake, don't you be so sad. (Tom Verlaine) From arife at ripe.net Tue Sep 6 18:19:11 2005 From: arife at ripe.net (Arife Vural) Date: Tue, 6 Sep 2005 18:19:11 +0200 Subject: [ris-int] Re: [Fwd: backup bottlenecks and possible improvements] In-Reply-To: <20050906152413.CEED66006B@localhost.ripe.net> References: <4302022D.5040607@ripe.net> <4305B39B.6080607@ripe.net> <20050906133213.GC28526@ripe.net> <20050906152413.CEED66006B@localhost.ripe.net> Message-ID: <20050906161911.GD28526@ripe.net> rene wrote: > >> I had a quick look at the raw data, and I think we can save a *lot* of > >> space. A comparison tells me that if we convert from gzip to bzip2, we > >> would save something like 25% of the space. This is only 100 Gbyte or > >> so, but nice to have. > > > > I think it's a good idea. We can convert from gzip to bzip2. Few > > applications that use rawdata need to be changed to use bzip2. > > > If you agree to use bzip2, I can proceed, and do the change. > > I have no principal objections against a switch to bzip2 but it has > to be done in a coordinated way: RISwhois depends on raw data !!! > > Two changes are needed: > > 1. libbgdump MUST support bzip2 (dynamically, user shouldn't have > to worry about it, just pass any filename to bgpdump_open and > that routine should do the magic of reading the gzip or bzip > file in the correct way). > > 2. riswhoisd MUST recognise .bz2 as a valid extension for > RIS raw data files when searching archive dirs for dumps > this plan sounds good. I will let you know when libbgpdump is ready. btw, I think before doing the change we should send the announcement to ris-users list, then people also can make the changes on their side. arife > Currently, I don't know when I could work on #2, but as it depends > on #1 anyhow, I'll first wait for a new libbgpdump to be ready > before worrying about riswhois code itself. > > Cheers, > > -- Rene > > > gerard wrote: > > > > >We did not anticipate that the diskusage for the former NP-services > > >would have such an enormous impact on the EMC. Approx. 1 GB is solely > > >taken by the full backup for RIS, TTM and dnsmon. > > :-)) > > no surprise, NP projects were always data hungry :P -- Arife Vural SED, RIPE NCC From arife at ripe.net Tue Sep 6 18:19:49 2005 From: arife at ripe.net (Arife Vural) Date: Tue, 6 Sep 2005 18:19:49 +0200 Subject: [ris-int] Re: [Fwd: backup bottlenecks and possible improvements] In-Reply-To: <200509061515.j86FFjbq016078@birch.ripe.net> References: <200509061515.j86FFjbq016078@birch.ripe.net> Message-ID: <20050906161949.GE28526@ripe.net> > > > space. A comparison tells me that if we convert from gzip to bzip2, we > > > would save something like 25% of the space. This is only 100 Gbyte or > > > so, but nice to have. > > Nice idea, but saving only 100GB is not much on a 1.6TB filesystem. > And it is not a real solution to the diskspace-problem I was talking > about. > > One extra concern: if you do so we will have 300GB of gzipped files > on the backupserver, and 200GB of bzipped2 files. And as a result we do > not have any space anymore on the backupserver. And that is exactly > what I/ops is trying to solve. Pls do not do this on your own and > consult ops before actually planning this! sure, I will coordinate with you guys. I do not want to mess your backup schedule. arife > > Gerard. -- Arife Vural SED, RIPE NCC From daniel.karrenberg at ripe.net Wed Sep 7 08:39:39 2005 From: daniel.karrenberg at ripe.net (Daniel Karrenberg) Date: Wed, 7 Sep 2005 08:39:39 +0200 Subject: [ris-int] riswhois request: reporting parent routes (fwd) In-Reply-To: References: Message-ID: <20050907063939.GS821@reifer-karrenberg-net.local> On 06.09 15:30, Rene Wilhelm wrote: > Hi, > > Though it's nice to have an active user providing suggestions, > I have the feeling his latest request would fall in the > category "feature creep", making the program larger, more > complex and the output harder to understand. > > However, before sending him a polite no-no, I'd like to > ask how others in RIS feel about this. Should RISwhois > embark on the trail of cross examining other routes > (parents, daughters, siblings, ...) or not? I do not understand the question fully but I would point him to the RIS itself. If this is a general question we could do a specialised query to the RIS for it. But not complicate riswhois. Small is beautiful. Daniel From daniel.karrenberg at ripe.net Wed Sep 7 09:05:29 2005 From: daniel.karrenberg at ripe.net (Daniel Karrenberg) Date: Wed, 7 Sep 2005 09:05:29 +0200 Subject: [ris-int] Re: [Fwd: backup bottlenecks and possible improvements] In-Reply-To: <200509061515.j86FFjbq016078@birch.ripe.net> References: <200509061515.j86FFjbq016078@birch.ripe.net> Message-ID: <20050907070529.GT821@reifer-karrenberg-net.local> [Sorry to be repetitive at the frequency of 1/year] This is experimental data which is immutable once aquired; read: it does not change anyore, ever. It is also not read very often and less so the older it gets. It also needs no high-speed access, we have copies of it in databases for that. The costs of keeping it around are roughly: equipment + ops time. It appears that particular equipment (netapps filer), which is easy to operate, becomes too expensive. It might be a good idea to invest a little ops time in a cheap storage boxes for stuff that does not need filer-quality storage. ATA disks are available at about half a Euro per Gigabute in units of 300GB. Thus it is possible to put up about a Terabyte of storage in any simple Wintel box. Slightly more without redundancy, slightly less with software RAID. Any (old) Wintel box will be fine. The equipment cost will be negligible: EUR600/terabyte if you use old wintel boxes, otherwise add the cost of the simplest wintel box. When building a couple of them and operating them all the same, the ops cost will not be too high. One does not even need RAID. Just build two of them and have a cron job rsyncing between them for full hot redundancy. Name them cheapfiler-1 and cheapfiler-1-copy, make the copy read-only to users. Make as many as we need. Spread them around for physical redundancy. Not rocket science. What's the problem? Daniel From henk at ripe.net Wed Sep 7 08:27:23 2005 From: henk at ripe.net (Henk Uijterwaal) Date: Wed, 07 Sep 2005 08:27:23 +0200 Subject: [ris-int] riswhois request: reporting parent routes (fwd) In-Reply-To: References: Message-ID: <6.2.1.2.2.20050907082051.02ce6b10@localhost> Hi, >Though it's nice to have an active user providing suggestions, >I have the feeling his latest request would fall in the >category "feature creep", making the program larger, more >complex and the output harder to understand. I tend to agree. RISwhois is a tool with a known and limited functionality, and we should keep it that way. I'm also a little reluctant to add every feature one can think of just because one person asks for it. However, what he wants is in the data and I can see some use for it. So, what I'd suggest is that this is put in the PDP ;-) Put the suggestion in the RIS update for R51, ask if there is interest from the community. If so, we can build "something" that does this (and we have to figure out internally who actually will do the job and when), if not, drop it. Henk ------------------------------------------------------------------------------ Henk Uijterwaal Email: henk.uijterwaal(at)ripe.net RIPE Network Coordination Centre http://www.amsterdamned.org/~henk P.O.Box 10096 Singel 258 Phone: +31.20.5354414 1001 EB Amsterdam 1016 AB Amsterdam Fax: +31.20.5354445 The Netherlands The Netherlands Mobile: +31.6.55861746 ------------------------------------------------------------------------------ Look here junior, don't you be so happy. And for Heaven's sake, don't you be so sad. (Tom Verlaine) From daniel.karrenberg at ripe.net Wed Sep 7 09:26:01 2005 From: daniel.karrenberg at ripe.net (Daniel Karrenberg) Date: Wed, 7 Sep 2005 09:26:01 +0200 Subject: [ris-int] Re: [Fwd: backup bottlenecks and possible improvements] In-Reply-To: <20050907070529.GT821@reifer-karrenberg-net.local> References: <200509061515.j86FFjbq016078@birch.ripe.net> <20050907070529.GT821@reifer-karrenberg-net.local> Message-ID: <20050907072601.GU821@reifer-karrenberg-net.local> On 07.09 09:05, Daniel Karrenberg wrote: > > Not rocket science. What's the problem? Forgot to say: I speak from experience. This is how I do file storage at home: /dev/hda1 7,4G 5,3G 1,8G 76% / /dev/hdc1 150G 111G 39G 75% /silo1 /dev/hdd4 145G 106G 40G 73% /silo2 /dev/hda3 67G 24G 43G 37% /silo3 /dev/hdb3 67G 53G 15G 79% /silo4 /dev/hdb1 7,4G 5,2G 2,0G 73% /root2 Whenever the family has collected too much, I just replace the oldest disk with the newest for more space. Important data is rsynced between spindles. More important data is rsynced to a different box. All this runs on an old PII 350MHz with 256MB which also is print server, VoIP phone switch, router, NAT and firewall. The key is to use a good case with extra fans that keeps eveerything cool including the disks: Sep 7 09:23:08 houser hddtemp[4332]: /dev/hdc: SAMSUNG SV1604N: 33 C Sep 7 09:23:08 houser hddtemp[4332]: /dev/hdd: SAMSUNG SP1604N: 35 C I have never had one break before I swapped it because it became got too small. Daniel From daniel.karrenberg at ripe.net Wed Sep 7 09:36:43 2005 From: daniel.karrenberg at ripe.net (Daniel Karrenberg) Date: Wed, 7 Sep 2005 09:36:43 +0200 Subject: [ris-int] riswhois request: reporting parent routes (fwd) In-Reply-To: <6.2.1.2.2.20050907082051.02ce6b10@localhost> References: <6.2.1.2.2.20050907082051.02ce6b10@localhost> Message-ID: <20050907073643.GV821@reifer-karrenberg-net.local> On 07.09 08:27, Henk Uijterwaal wrote: > > However, what he wants is in the data and I can see some use for it. So, > what I'd suggest is that this is put in the PDP ;-) .... Fine, as long as it is speced as a tool that is separate from riswhois and that produces this data on demand from the risdb. Daniel From shane at ripe.net Wed Sep 7 10:28:00 2005 From: shane at ripe.net (Shane Kerr) Date: Wed, 07 Sep 2005 10:28:00 +0200 Subject: [ris-int] Re: [Fwd: backup bottlenecks and possible improvements] In-Reply-To: <20050907070529.GT821@reifer-karrenberg-net.local> References: <200509061515.j86FFjbq016078@birch.ripe.net> <20050907070529.GT821@reifer-karrenberg-net.local> Message-ID: <431EA490.7020703@ripe.net> Daniel Karrenberg wrote: >[Sorry to be repetitive at the frequency of 1/year] > >This is experimental data which is immutable once aquired; read: it does >not change anyore, ever. It is also not read very often and less so the >older it gets. It also needs no high-speed access, we have copies of it >in databases for that. > >The costs of keeping it around are roughly: equipment + ops time. > >It appears that particular equipment (netapps filer), which is easy to >operate, becomes too expensive. It might be a good idea to invest a >little ops time in a cheap storage boxes for stuff that does not need >filer-quality storage. > >ATA disks are available at about half a Euro per Gigabute in units of >300GB. Thus it is possible to put up about a Terabyte of storage in any >simple Wintel box. Slightly more without redundancy, slightly less with >software RAID. Any (old) Wintel box will be fine. The equipment cost >will be negligible: EUR600/terabyte if you use old wintel boxes, >otherwise add the cost of the simplest wintel box. When building a >couple of them and operating them all the same, the ops cost will not be >too high. One does not even need RAID. Just build two of them and have >a cron job rsyncing between them for full hot redundancy. Name them >cheapfiler-1 and cheapfiler-1-copy, make the copy read-only to users. >Make as many as we need. Spread them around for physical redundancy. > >Not rocket science. What's the problem? > > I more-or-less agree. For 12000 Euros you can put 11 TB in a rack-mounted box (prices from alternate.nl, a month or so ago): Procase Procase C4EE Case, 24x3.5" drives, 950 Watt PSU 1 EUR 2.499,00 EUR 2.499,00 Hitachi Deskstar 7K500 SATA hard disk, 500 GB, 8,5 ms, 16 MB, 7200 RPM 22 EUR 359,00 EUR 7.898,00 Promise SATA II 150 SX8 SATA controller, PCI-X 64-bit 133 MHz, 8xSATA, 150 MB/s 2 EUR 189,00 EUR 378,00 Tyan Tyan Thunder K8S Pro (S2882G3NR) 2x Opteron, PCI-X, 8xDDR-SDRAM, ATI Rage XL, 2x1GHz LAN 1 EUR 459,00 EUR 459,00 AMD Opteron 244 1.8 GHz CPU 2 EUR 199,00 EUR 398,00 Kingston Kingston DIMM 2 GB 333 MHz DDR, 2 GB (PC2700) 1 EUR 549,00 EUR 549,00 EUR 12.181,00 The bottleneck here will probably be the dual Gigabit Ethernet controllers, rather than the disks, if we are using it for backup. This is the *worst-case* cost, mind you. If we were to build a system to backup RIS raw data, it could be as simple as putting a USB drive on my desk next to someone's workstation (I vote for Arife, since her workstation is in the corner and not likely to get bumped by someone walking by) - total cost, 359 Euros: http://www.alternate.nl/html/shop/productDetails.html?artno=A9UE03& Which is a bit extreme, but it shows what we could do if we decided to optimise for cost/performance instead of risk- and work-aversion. -- Shane p.s. With 11 TB I could achieve my dream of putting all of RIS data on-line. :) -------------- next part -------------- An HTML attachment was scrubbed... URL: From gerard at ripe.net Wed Sep 7 10:56:06 2005 From: gerard at ripe.net (Gerard Leurs) Date: Wed, 7 Sep 2005 10:56:06 +0200 Subject: [ris-int] Re: [Fwd: backup bottlenecks and possible improvements] Message-ID: <200509070856.j878u6pa017930@birch.ripe.net> Daniel. > It appears that particular equipment (netapps filer), which is easy to > operate, becomes too expensive. > it might be a good idea to invest a > little ops time in a cheap storage boxes for stuff that does not need > filer-quality storage. I'll keep it short: For over one year we have cheap IDE disks in an EMC-CX300. And our F810c (=NetApp at singel258) is no longer in use as backup-medium. Gerard. From wilhelm at ripe.net Wed Sep 7 11:16:00 2005 From: wilhelm at ripe.net (Rene Wilhelm) Date: Wed, 07 Sep 2005 11:16:00 +0200 Subject: [ris-int] riswhois request: reporting parent routes (fwd) In-Reply-To: Message from Daniel Karrenberg of "Wed, 07 Sep 2005 08:39:39 +0200." <20050907063939.GS821@reifer-karrenberg-net.local> Message-ID: <200509070916.j879G0Vw023755@birch.ripe.net> > I do not understand the question fully but > I would point him to the RIS itself. If this is a general question > we could do a specialised query to the RIS for it. > But not complicate riswhois. Small is beautiful. Right, that's also what I had in mind; RISwhois provides a first quick overview, an overal looking glass on various RRC bgp tables. More complex queries, analysing updates, combining data from different prefixes, are out of scope, should be answered from RIS DB. -- Rene From daniel.karrenberg at ripe.net Wed Sep 7 11:21:53 2005 From: daniel.karrenberg at ripe.net (Daniel) Date: Wed, 7 Sep 2005 11:21:53 +0200 Subject: [ris-int] Re: [Fwd: backup bottlenecks and possible improvements] In-Reply-To: <200509070856.j878u6pa017930@birch.ripe.net> References: <200509070856.j878u6pa017930@birch.ripe.net> Message-ID: <20050907092153.GA821@reifer-karrenberg-net.local> On 07.09 10:56, Gerard Leurs wrote: > Daniel. > > > It appears that particular equipment (netapps filer), which is easy to > > operate, becomes too expensive. > > it might be a good idea to invest a > > little ops time in a cheap storage boxes for stuff that does not need > > filer-quality storage. > > I'll keep it short: > For over one year we have cheap IDE disks in an EMC-CX300. And our > F810c (=NetApp at singel258) is no longer in use as backup-medium. So why not put the RIS data there? From shane at ripe.net Wed Sep 7 11:48:09 2005 From: shane at ripe.net (Shane Kerr) Date: Wed, 07 Sep 2005 11:48:09 +0200 Subject: [ris-int] Re: [Fwd: backup bottlenecks and possible improvements] In-Reply-To: <20050907092153.GA821@reifer-karrenberg-net.local> References: <200509070856.j878u6pa017930@birch.ripe.net> <20050907092153.GA821@reifer-karrenberg-net.local> Message-ID: <431EB759.9010803@ripe.net> Daniel wrote: >On 07.09 10:56, Gerard Leurs wrote: > > >>Daniel. >> >> > It appears that particular equipment (netapps filer), which is easy to >> > operate, becomes too expensive. >> > it might be a good idea to invest a >> > little ops time in a cheap storage boxes for stuff that does not need >> > filer-quality storage. >> >>I'll keep it short: >>For over one year we have cheap IDE disks in an EMC-CX300. And our >>F810c (=NetApp at singel258) is no longer in use as backup-medium. >> >> > >So why not put the RIS data there? > > Maybe we have a communication problem. I thought that Ops was annoyed at the cost of supporting RIS, and was trying to help. It seems like Gerard doesn't really think RIS is a problem, because it is only 13% or so of our backup space. So even saving 25% of that doesn't really matter. In which case we should take this off the ris-int list, and Ops should just arrange to buy as much additional storage as they need in whatever way they want. We'll probably still switch to bzip2, as this is helpful for anyone downloading the raw data. We will co-ordinate with Ops, of course. And of course Ops will never complain about people wasting space, unless they are willing to hear the suggestions of cheap & easy ways to provide more space. ;) -- Shane From henk at ripe.net Thu Sep 8 07:49:42 2005 From: henk at ripe.net (Henk Uijterwaal) Date: Thu, 08 Sep 2005 07:49:42 +0200 Subject: [ris-int] Re: [Fwd: backup bottlenecks and possible improvements] In-Reply-To: <200509070856.j878u6pa017930@birch.ripe.net> References: <200509070856.j878u6pa017930@birch.ripe.net> Message-ID: <6.2.1.2.2.20050908074809.02c5a8e8@localhost> Gerard, >I'll keep it short: >For over one year we have cheap IDE disks in an EMC-CX300. And our >F810c (=NetApp at singel258) is no longer in use as backup-medium. If we use cheap disks, then why is every request for a few Gb of disk-space answered with "we do not have space on the filer"? Last week it was space for the kroot traces (1Gb/week), this week it is the RIS... Henk ------------------------------------------------------------------------------ Henk Uijterwaal Email: henk.uijterwaal(at)ripe.net RIPE Network Coordination Centre http://www.amsterdamned.org/~henk P.O.Box 10096 Singel 258 Phone: +31.20.5354414 1001 EB Amsterdam 1016 AB Amsterdam Fax: +31.20.5354445 The Netherlands The Netherlands Mobile: +31.6.55861746 ------------------------------------------------------------------------------ Look here junior, don't you be so happy. And for Heaven's sake, don't you be so sad. (Tom Verlaine) From henk at ripe.net Thu Sep 8 07:51:52 2005 From: henk at ripe.net (Henk Uijterwaal) Date: Thu, 08 Sep 2005 07:51:52 +0200 Subject: [ris-int] riswhois request: reporting parent routes (fwd) In-Reply-To: <20050907073643.GV821@reifer-karrenberg-net.local> References: <6.2.1.2.2.20050907082051.02ce6b10@localhost> <20050907073643.GV821@reifer-karrenberg-net.local> Message-ID: <6.2.1.2.2.20050908075116.02d6dea0@localhost> At 09:36 07/09/2005, Daniel Karrenberg wrote: >On 07.09 08:27, Henk Uijterwaal wrote: > > > > However, what he wants is in the data and I can see some use for it. So, > > what I'd suggest is that this is put in the PDP ;-) .... > >Fine, as long as it is speced as a tool that is separate from riswhois and >that produces this data on demand from the risdb. Yes, sure, I said "build something". I don't mind doing this, but I don't want to do this for 1 user. Henk >Daniel ------------------------------------------------------------------------------ Henk Uijterwaal Email: henk.uijterwaal(at)ripe.net RIPE Network Coordination Centre http://www.amsterdamned.org/~henk P.O.Box 10096 Singel 258 Phone: +31.20.5354414 1001 EB Amsterdam 1016 AB Amsterdam Fax: +31.20.5354445 The Netherlands The Netherlands Mobile: +31.6.55861746 ------------------------------------------------------------------------------ Look here junior, don't you be so happy. And for Heaven's sake, don't you be so sad. (Tom Verlaine) From bourq at nocs.com Tue Sep 27 15:16:48 2005 From: bourq at nocs.com (Elijah Bourquin) Date: Tue, 27 Sep 2005 08:16:48 -0500 Subject: [ris-int] impetus Pfharmaceutical Message-ID: <003801c5c365$b70b2000$5846a8c0@outdrove> Sav % On your Med harmacyByM e up to 70 ications today with P ail XanaViagCialVali xraisum $1 $3 $1 $3 42.33.21.75 and many s in our s other medication hop - http://www.jne-medds.chatpusher.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From arife at ripe.net Wed Sep 28 11:51:41 2005 From: arife at ripe.net (Arife Vural) Date: Wed, 28 Sep 2005 11:51:41 +0200 Subject: [ris-int] RIS status report at RIPE51 Message-ID: <20050928095140.GD30995@ripe.net> Hi guys, There will be 5-10 mins status update at Routing-WG, RIPE51. Presentation will roughly contain the following main points: - RIS: What is it? Contact info - Stats (if we can find some interesting stats) - RIS Performance study - Central server trouble keeping up with RRC - Analysis check for memory, CPU, and disk I/O - Result, disk is bottleneck - Solution: add disks -> even better, split DBs across servers - Getting away from central server design - Add more CPU and Disk. - What are user needs? - Fast applications - Less time lag - More data in DB - Feedback from users. Please let me know if something is missing. -- Arife Vural SED, RIPE NCC