archzoom

[Andy Tai]

Hi, regarding the recent thread about Archzoom, I was thinking of its performance as a separate issue from some of the performance issues of tla itself.  In my brief understanding of Archzoom, it operates kind of like cgi scripts of the early web where mod-perl achieved better performance by being resident in a server context.

So the high CPU load of Archzoom may suggest that it would be beneficial for tla to be able to operate in a resident manner to carry out operations from a client such as a perl script. 

Of course this may relate back to a lot of Tom's writings about "libify" tla, having a scripting layer, etc.  The work of making the tla internals into a library is largely already in place...   so if I have time I may try to look into making tla to run like a daemon... for web style applications.   Of course anyone who wants to do this would be greatly encouraged...

Using a small scripting language to drive the library... maybe something nice to try as well..

--
Andy Tai, atai@...
_______________________________________________
Gnu-arch-users mailing list
Gnu-arch-users@...
http://lists.gnu.org/mailman/listinfo/gnu-arch-users

GNU arch home page:
http://savannah.gnu.org/projects/gnu-arch/

[Mikhael Goikhman]

[I was offline for a long time, so the late answer.]

It does not seem the "bad performance of archzoom" issue is presented
correctly. I can't say whether the Savannah administrator ever enabled
archzoom. As for gna.org, the issue looks pretty simple. The
administrator decided he has no many servers and wants to dedicate his
single server to subversion browsers and not to arch.

Here is some info that may help. The default archzoom configuration is
conservative in that it does not try to write things to a user's disk.
This is good for a demo installation, but requires tuning for production.
The ArchZoom FAQ explains how to trivially define revision library that
may cause tla to be drastically faster. It also explains how to keep the
size of this revision library constant using "axp revlib prune --params".

Another related problem is that many developers have unresponsively huge
branches of thousands of revisions without a single cacherev, that makes
certain tla operations totally stuck when working on these branches. A
policy of automatic creation a cacherev every 50 revisions partially
solves this problem.

By default, archzoom forbids search engines to crawl its pages, but some
web server misconfiguration or misbehaved robots may cause some unwanted
bombing too. To solve this problem archzoom has a number of configuration
options, for example it may limit the number of its instances. Problems
like these described in the last 3 paragraphs are the real issue and it
has nothing to do with archzoom.

As a side note, certain tla operations (see threads about "tla abrowse",
for example) are needlessly unoptimal, and may be improved noticeably.
Many tla commands miss a limit, as others mentioned.

I think the perl layer has a quite little overhead that is minor compared
to the real problems related to tla and a web server. I don't really see
"tla librification" essential, as long as tla behaves like a good library
(and it is pretty close to it); fork+system is cheap on unix.

As for the idea of making tla to behave more in a daemon manner, I am a
bit skeptical about this, but I should see more info to form any opnion.
ArchZoom implements some of this stuff, it has own mechanism of caching
many of the views for several hours, so that tla is not called. I think
it may also work under mod_perl if one wants this, however the standard
CGI mode together with limiting the number of archzoom instances may work
as well.

Regards,
Mikhael.


_______________________________________________
Gnu-arch-users mailing list
Gnu-arch-users@...
http://lists.gnu.org/mailman/listinfo/gnu-arch-users

GNU arch home page:
http://savannah.gnu.org/projects/gnu-arch/

[Sylvain Beucler-2]

Hi,

I remember Michael installed ArchZoom at Savannah and, seeing the
comment at Gna, we decided not to load the (at that time) much-loaded
Savannah some more.

Since then, we performed a bit of restructuration which improved the
load. Moreover, there aren't that many Arch archives right now
(compared to CVS), so this would worth a try.


Would you be interested in setting up ArchZoom at Savannah? Or for a
start, do you have some advice on what I should look at? :)


By the way, I also have a friend who disabled Arch at his website
(patapouf.org - vcs.patapouf.org) because ArchZoom would apparently
consume too much /tmp space for its own good. Does that ring a bell?

Thanks,

--
Sylvain


On Wed, Oct 18, 2006 at 08:40:36PM +0000, Mikhael Goikhman wrote:

> [I was offline for a long time, so the late answer.]
>
> It does not seem the "bad performance of archzoom" issue is presented
> correctly. I can't say whether the Savannah administrator ever enabled
> archzoom. As for gna.org, the issue looks pretty simple. The
> administrator decided he has no many servers and wants to dedicate his
> single server to subversion browsers and not to arch.
>
> Here is some info that may help. The default archzoom configuration is
> conservative in that it does not try to write things to a user's disk.
> This is good for a demo installation, but requires tuning for production.
> The ArchZoom FAQ explains how to trivially define revision library that
> may cause tla to be drastically faster. It also explains how to keep the
> size of this revision library constant using "axp revlib prune --params".
>
> Another related problem is that many developers have unresponsively huge
> branches of thousands of revisions without a single cacherev, that makes
> certain tla operations totally stuck when working on these branches. A
> policy of automatic creation a cacherev every 50 revisions partially
> solves this problem.
>
> By default, archzoom forbids search engines to crawl its pages, but some
> web server misconfiguration or misbehaved robots may cause some unwanted
> bombing too. To solve this problem archzoom has a number of configuration
> options, for example it may limit the number of its instances. Problems
> like these described in the last 3 paragraphs are the real issue and it
> has nothing to do with archzoom.
>
> As a side note, certain tla operations (see threads about "tla abrowse",
> for example) are needlessly unoptimal, and may be improved noticeably.
> Many tla commands miss a limit, as others mentioned.
>
> I think the perl layer has a quite little overhead that is minor compared
> to the real problems related to tla and a web server. I don't really see
> "tla librification" essential, as long as tla behaves like a good library
> (and it is pretty close to it); fork+system is cheap on unix.
>
> As for the idea of making tla to behave more in a daemon manner, I am a
> bit skeptical about this, but I should see more info to form any opnion.
> ArchZoom implements some of this stuff, it has own mechanism of caching
> many of the views for several hours, so that tla is not called. I think
> it may also work under mod_perl if one wants this, however the standard
> CGI mode together with limiting the number of archzoom instances may work
> as well.
>
> Regards,
> Mikhael.

_______________________________________________
Gnu-arch-users mailing list
Gnu-arch-users@...
http://lists.gnu.org/mailman/listinfo/gnu-arch-users

GNU arch home page:
http://savannah.gnu.org/projects/gnu-arch/

[Sylvain Beucler-2]

Hi,

I tried setting up ArchZoom and it was pretty easy indeed (especially
with the Debian Etch package) :)

http://arch.savannah.gnu.org/archzoom/

I activated the revlib, we'll see how it goes. Currently we have
enough free space at Savannah to test it out.

--
Sylvain

On Wed, Nov 29, 2006 at 11:41:27PM +0100, Sylvain Beucler wrote:

> Hi,
>
> I remember Michael installed ArchZoom at Savannah and, seeing the
> comment at Gna, we decided not to load the (at that time) much-loaded
> Savannah some more.
>
> Since then, we performed a bit of restructuration which improved the
> load. Moreover, there aren't that many Arch archives right now
> (compared to CVS), so this would worth a try.
>
>
> Would you be interested in setting up ArchZoom at Savannah? Or for a
> start, do you have some advice on what I should look at? :)
>
>
> By the way, I also have a friend who disabled Arch at his website
> (patapouf.org - vcs.patapouf.org) because ArchZoom would apparently
> consume too much /tmp space for its own good. Does that ring a bell?
>
> Thanks,
>
> --
> Sylvain
>
>
> On Wed, Oct 18, 2006 at 08:40:36PM +0000, Mikhael Goikhman wrote:
> > [I was offline for a long time, so the late answer.]
> >
> > It does not seem the "bad performance of archzoom" issue is presented
> > correctly. I can't say whether the Savannah administrator ever enabled
> > archzoom. As for gna.org, the issue looks pretty simple. The
> > administrator decided he has no many servers and wants to dedicate his
> > single server to subversion browsers and not to arch.
> >
> > Here is some info that may help. The default archzoom configuration is
> > conservative in that it does not try to write things to a user's disk.
> > This is good for a demo installation, but requires tuning for production.
> > The ArchZoom FAQ explains how to trivially define revision library that
> > may cause tla to be drastically faster. It also explains how to keep the
> > size of this revision library constant using "axp revlib prune --params".
> >
> > Another related problem is that many developers have unresponsively huge
> > branches of thousands of revisions without a single cacherev, that makes
> > certain tla operations totally stuck when working on these branches. A
> > policy of automatic creation a cacherev every 50 revisions partially
> > solves this problem.
> >
> > By default, archzoom forbids search engines to crawl its pages, but some
> > web server misconfiguration or misbehaved robots may cause some unwanted
> > bombing too. To solve this problem archzoom has a number of configuration
> > options, for example it may limit the number of its instances. Problems
> > like these described in the last 3 paragraphs are the real issue and it
> > has nothing to do with archzoom.
> >
> > As a side note, certain tla operations (see threads about "tla abrowse",
> > for example) are needlessly unoptimal, and may be improved noticeably.
> > Many tla commands miss a limit, as others mentioned.
> >
> > I think the perl layer has a quite little overhead that is minor compared
> > to the real problems related to tla and a web server. I don't really see
> > "tla librification" essential, as long as tla behaves like a good library
> > (and it is pretty close to it); fork+system is cheap on unix.
> >
> > As for the idea of making tla to behave more in a daemon manner, I am a
> > bit skeptical about this, but I should see more info to form any opnion.
> > ArchZoom implements some of this stuff, it has own mechanism of caching
> > many of the views for several hours, so that tla is not called. I think
> > it may also work under mod_perl if one wants this, however the standard
> > CGI mode together with limiting the number of archzoom instances may work
> > as well.
> >
> > Regards,
> > Mikhael.

_______________________________________________
Gnu-arch-users mailing list
Gnu-arch-users@...
http://lists.gnu.org/mailman/listinfo/gnu-arch-users

GNU arch home page:
http://savannah.gnu.org/projects/gnu-arch/

[Sylvain Beucler-2]

> By the way, I also have a friend who disabled Arch at his website
> (patapouf.org - vcs.patapouf.org) because ArchZoom would apparently
> consume too much /tmp space for its own good. Does that ring a bell?

Ok, apparently is was old v0.4.5-1 (Debian Sarge).

Probably that was fixed since then :)

--
Sylvain


_______________________________________________
Gnu-arch-users mailing list
Gnu-arch-users@...
http://lists.gnu.org/mailman/listinfo/gnu-arch-users

GNU arch home page:
http://savannah.gnu.org/projects/gnu-arch/

[Miles Bader-2]

Sylvain Beucler <beuc@...> writes:
> http://arch.savannah.gnu.org/archzoom/
>
> I activated the revlib, we'll see how it goes. Currently we have
> enough free space at Savannah to test it out.

Hi, is there a reason only a few archives are viewable ?

Thanks,

-Miles

--
`To alcohol!  The cause of, and solution to,
 all of life's problems' --Homer J. Simpson


_______________________________________________
Gnu-arch-users mailing list
Gnu-arch-users@...
http://lists.gnu.org/mailman/listinfo/gnu-arch-users

GNU arch home page:
http://savannah.gnu.org/projects/gnu-arch/

[Sylvain Beucler-2]

On Mon, Dec 04, 2006 at 05:17:06AM +0900, Miles Bader wrote:
> Sylvain Beucler <beuc@...> writes:
> > http://arch.savannah.gnu.org/archzoom/
> >
> > I activated the revlib, we'll see how it goes. Currently we have
> > enough free space at Savannah to test it out.
>
> Hi, is there a reason only a few archives are viewable ?

Yes, archives need to be referenced one by one, and since we don't
follow a strict archive layout (one or several archive per project,
top-level or subdir...), I can't guess them all and I only activated
some of mine :)

Basically we need a bit of code where a project member can list the
path to all his/her GNU Arch archives, and have the system update
/etc/archzoom/.arch-params/=locations/

Would you be willing to write this code?

I recently wrote similar code to list the CVS modules and attach mail
notifications to them:
http://svn.sv.gnu.org/viewcvs/trunk/frontend/php/cvs/admin/?root=administration
(that code and SVN support are also in test btw)

--
Sylvain


_______________________________________________
Gnu-arch-users mailing list
Gnu-arch-users@...
http://lists.gnu.org/mailman/listinfo/gnu-arch-users

GNU arch home page:
http://savannah.gnu.org/projects/gnu-arch/