TLD cvsup mirrors

[Ken Smith-3]

I'm looking for several new cvsup mirrors for the top level domain
(cvsup*.freebsd.org).  My preference is to "promote" sites that have
been working for a while as sites within a country code that feel they
can take on more traffic.  At this point most of the TLD cvsup servers
are in the US (and not to put too fine a point on it a depressing number
are in buffalo.edu because I "temporarily" took over for another server
that went AWOL and I'm just now getting around to looking for a real
replacement...) so sites in other countries are definitely encouraged to
make an offer.  I'd like to diversify it a bit if possible.

Just to give you a feel for what you'd be getting into...  Some of these
sites currently handle as many as 1000 to 2000 connects/day.  From my
experiences if you want to support that many connects/day in a
reasonable fashion it takes a machine that can handle 20 to 30
concurrent connections.  To do that it helps to have learned a few
tricks like making sure to turn off atime on the filesystem the
repository is in, and bumping up kern.maxvnodes to encourage more file
caching.

Thanks.

--
                                                Ken Smith
- From there to here, from here to      |       kensmith@...
  there, funny things are everywhere.   |
                      - Theodore Geisel |

[Adam McDougall]

Ken Smith wrote:

> I'm looking for several new cvsup mirrors for the top level domain
> (cvsup*.freebsd.org).  My preference is to "promote" sites that have
> been working for a while as sites within a country code that feel they
> can take on more traffic.  At this point most of the TLD cvsup servers
> are in the US (and not to put too fine a point on it a depressing number
> are in buffalo.edu because I "temporarily" took over for another server
> that went AWOL and I'm just now getting around to looking for a real
> replacement...) so sites in other countries are definitely encouraged to
> make an offer.  I'd like to diversify it a bit if possible.
>
> Just to give you a feel for what you'd be getting into...  Some of these
> sites currently handle as many as 1000 to 2000 connects/day.  From my
> experiences if you want to support that many connects/day in a
> reasonable fashion it takes a machine that can handle 20 to 30
> concurrent connections.  To do that it helps to have learned a few
> tricks like making sure to turn off atime on the filesystem the
> repository is in, and bumping up kern.maxvnodes to encourage more file
> caching.
>
> Thanks.
>
>  

I'd be happy to take on more cvsupd clients if you want, lately I've
been getting around 550 working clients per day and it feels like
nothing (cvsup15.freebsd.org).  I have 20G of ram in the system and
lately a lot of the cvsup data seems to get cached. Plenty of CPU to go
around too.

What would you recommend for kern.maxvnodes?  Thanks
_______________________________________________
freebsd-hubs@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hubs
To unsubscribe, send any mail to "freebsd-hubs-unsubscribe@..."

[Ken Smith-3]

On Sun, 2009-06-14 at 16:09 -0400, Adam McDougall wrote:
> I'd be happy to take on more cvsupd clients if you want, lately I've
> been getting around 550 working clients per day and it feels like
> nothing (cvsup15.freebsd.org).  I have 20G of ram in the system and
> lately a lot of the cvsup data seems to get cached. Plenty of CPU to go
> around too.

Thanks.  If I don't get enough offers I might take you up on that but at
least for now the goal is to get back to having each of the
cvsup*.freebsd.org entries being unique machines.  The higher numbered
machines do seem to wind up with less traffic, one of mine is cvsup18
and it only sees around 200/day.

> What would you recommend for kern.maxvnodes?  Thanks

That depends on how much physical memory the machine has and what else
it's doing.  For my machines, which are just mirror systems, I have:

  Dual Opteron 2.3GHz, 4GB RAM:
        kern.maxvnodes = 250000
        maximum cvsupd clients = 40

  Quad Xeon 2.4GHZ, 8GB RAM:
        kern.maxvnodes = 350000
        maximum cvsupd clients = 40

  Dual Xeon 1.7GHz, 2GB RAM:
        kern.maxvnodes = 150000
        maximum cvsupd clients = 20

The second one is ftp-archive.freebsd.org as well as being a cvsupd
server.

With 20GB RAM if the machine isn't doing much else you could probably
crank maxvnodes up to over the number of files in the repository itself
(looks to be a little over 570000 files in it at the moment).

The other performance tweak that helps a *lot* is to make sure you mount
the partition the repository is in using noatime.

The first machine listed above handles around 3000 connects per day
comfortably with that configuration.  It's the one I let accumulate a
few too many of the TLD names though (three at the moment), those are
some of what I'm trying to find a new home for.  :-)

I only have the second machine doing significant-ish cvsupd stuff when I
need it to take over for site(s) that died while I look to find a new
site to take it over.  If a cvsupd server has been down for a while it
can wind up getting *slammed* when it comes back to life.  I took over
for one that had been down for quite a while on Friday and it wound up
handling over 7,000 requests the first day.  So... I usually try to take
over on one of my machines first to get through that and then get an
idea of what the normal daily traffic is so I can tell the new site what
they're in for...

--
                                                Ken Smith
- From there to here, from here to      |       kensmith@...
  there, funny things are everywhere.   |
                      - Theodore Geisel |

[Garrett Wollman-6]

<<On Sun, 14 Jun 2009 18:19:53 -0400, Ken Smith <kensmith@...> said:

> Thanks.  If I don't get enough offers I might take you up on that but at
> least for now the goal is to get back to having each of the
> cvsup*.freebsd.org entries being unique machines.  The higher numbered
> machines do seem to wind up with less traffic, one of mine is cvsup18
> and it only sees around 200/day.

cvsup3 gets about 600 clients/day (averaged over the past week) with a
connection limit of 16.

-GAWollman

_______________________________________________
freebsd-hubs@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hubs
To unsubscribe, send any mail to "freebsd-hubs-unsubscribe@..."

[Rutger Bevaart-2]

I'd be happy to add our CVSup machine to the list, being located in  
the Netherlands. To me it seems a lot of cvsup mirrors in the  
Netherlands are aliases of the one located at SURFnet.

Anybody on the list who can check/test and add ours?

cvsup.illian.net located in AS39292.

Met vriendelijke groet / Kind Regards,

Rutger Bevaart

illian.networks B.V.

Costerweg 5
6702AA Wageningen

T: 0317.712.400
F: 0317.712.401
W: http://www.illian-networks.nl
E: rutger.bevaart@...

On Jun 15, 2009, at 2:01 AM, Garrett Wollman wrote:

> <<On Sun, 14 Jun 2009 18:19:53 -0400, Ken Smith <kensmith@...
> > said:
>
>> Thanks.  If I don't get enough offers I might take you up on that  
>> but at
>> least for now the goal is to get back to having each of the
>> cvsup*.freebsd.org entries being unique machines.  The higher  
>> numbered
>> machines do seem to wind up with less traffic, one of mine is cvsup18
>> and it only sees around 200/day.
>
> cvsup3 gets about 600 clients/day (averaged over the past week) with a
> connection limit of 16.
>
> -GAWollman
>
> _______________________________________________
> freebsd-hubs@... mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-hubs
> To unsubscribe, send any mail to "freebsd-hubs-
> unsubscribe@..."
>

_______________________________________________
freebsd-hubs@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hubs
To unsubscribe, send any mail to "freebsd-hubs-unsubscribe@..."

[ben wilber-3]

On Sun, Jun 14, 2009 at 06:19:53PM -0400, Ken Smith wrote:
> Thanks.  If I don't get enough offers I might take you up on that but at
> least for now the goal is to get back to having each of the
> cvsup*.freebsd.org entries being unique machines.  The higher numbered
> machines do seem to wind up with less traffic, one of mine is cvsup18
> and it only sees around 200/day.

If you need any more US-based servers, I volunteer cvsup5.us.  It gets
~500 connections per day and runs basically idle.

bw.
_______________________________________________
freebsd-hubs@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hubs
To unsubscribe, send any mail to "freebsd-hubs-unsubscribe@..."

[Mark Huizer-5]

On Mon, Jun 15, 2009 at 08:57:18AM +0200, Rutger Bevaart wrote:
> I'd be happy to add our CVSup machine to the list, being located in  
> the Netherlands. To me it seems a lot of cvsup mirrors in the  
> Netherlands are aliases of the one located at SURFnet.
>
> Anybody on the list who can check/test and add ours?
>
> cvsup.illian.net located in AS39292.
>

Actually, they are aliases of a machine that is not at surfnet but at
ClaraNet Netherlands. :-)
But if you want to be added as cvsup2.nl.freebsd.org, be my guest, just
contact me at hostmaster at nl.freebsd.org

Greetings

Mark
_______________________________________________
freebsd-hubs@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hubs
To unsubscribe, send any mail to "freebsd-hubs-unsubscribe@..."