git copies of cvs modules available

[spz]

Dear all,

htdocs, othersrc, pkgsrc, src and xsrc are now available as
(bare) git repository copies at
http://ftp.netbsd.org/pub/NetBSD/misc/repositories/git.
These repositories are currently updated every 30 minutes.

They are not authoritative and are not meant to replace cvs,
but if you want to track your development that is not yet fit
for committing into cvs head or sending in as a PR, and want to
use git for that, they may serve as a convenient starting point
or reference.

regards,
        spz

[Rhialto]

On Tue 13 Oct 2009 at 09:38:27 +0200, S.P.Zeidler wrote:
> Dear all,
>
> htdocs, othersrc, pkgsrc, src and xsrc are now available as
> (bare) git repository copies at
> http://ftp.netbsd.org/pub/NetBSD/misc/repositories/git.

That is great! Thanks!

> These repositories are currently updated every 30 minutes.
>
> They are not authoritative and are not meant to replace cvs,

indeed, since user need to be aware of a caveat: All $NetBSD$ tags
appear to be wrong. They have the value as of the checkin of each
version of each file, not the checkout, so they are always 1 version
behind. (I checked this on src/Makefile which shows this in cvs

# $NetBSD: Makefile,v 1.274 2009/09/04 17:21:33 pooka Exp $

and this in the git version:

#       $NetBSD: Makefile,v 1.273 2009/08/30 01:49:42 dyoung Exp $

but the edits regarding "do-sys-rump-dev-lib" are actually in it.)

(There has been a very heated discussion among GIT people why keyword
expansion is supposed to be evil. They are not going to be convinced to
implement it. I am very annoyed by that.)

-Olaf.
--
___ Olaf 'Rhialto' Seibert    -- You author it, and I'll reader it.
\X/ rhialto/at/xs4all.nl      -- Cetero censeo "authored" delendum esse.

[Eric Gillespie-2]

Rhialto <rhialto@...> writes:

> That is great! Thanks!

Indeed, this is awesome!  I've experimented with importing pkgsrc
into bzr, but even with all the recent improvements, it's still
no good.  Quite a shame, I think.  I've just been working with
silly non-history-preserving branches.  I use git at work and
really hate it, but not so much as to prevent me from switching
over for pkgsrc work :).

> > They are not authoritative and are not meant to replace cvs,

It's a good start.  Maybe people will experiment with it and
learn how we could be working.

> indeed, since user need to be aware of a caveat: All $NetBSD$ tags
> appear to be wrong. They have the value as of the checkin of each
> version of each file, not the checkout, so they are always 1 version
> behind. (I checked this on src/Makefile which shows this in cvs
>
> # $NetBSD: Makefile,v 1.274 2009/09/04 17:21:33 pooka Exp $
>
> and this in the git version:
>
> #       $NetBSD: Makefile,v 1.273 2009/08/30 01:49:42 dyoung Exp $

That's how it works in CVS.  The keywords are expanded by your
client in the working copy.  Then the expanded form is committed
when you commit.  Thus, the repository is always one behind.

> (There has been a very heated discussion among GIT people why keyword
> expansion is supposed to be evil. They are not going to be convinced to
> implement it. I am very annoyed by that.)

They are evil.  Evil evil evil evil.  EVIL!  No modern system
supports them, nor are they likely, and we're all the better
for it.

--
Eric Gillespie <*> epg@...

[Adam Hoka]

On 10/14/09 04:44, Eric Gillespie wrote:

> Rhialto<rhialto@...>  writes:
>
>> That is great! Thanks!
>
> Indeed, this is awesome!  I've experimented with importing pkgsrc
> into bzr, but even with all the recent improvements, it's still
> no good.  Quite a shame, I think.  I've just been working with
> silly non-history-preserving branches.  I use git at work and
> really hate it, but not so much as to prevent me from switching
> over for pkgsrc work :).
>
>>> They are not authoritative and are not meant to replace cvs,
>
> It's a good start.  Maybe people will experiment with it and
> learn how we could be working.
>
>> indeed, since user need to be aware of a caveat: All $NetBSD$ tags
>> appear to be wrong. They have the value as of the checkin of each
>> version of each file, not the checkout, so they are always 1 version
>> behind. (I checked this on src/Makefile which shows this in cvs
>>
>> # $NetBSD: Makefile,v 1.274 2009/09/04 17:21:33 pooka Exp $
>>
>> and this in the git version:
>>
>> #       $NetBSD: Makefile,v 1.273 2009/08/30 01:49:42 dyoung Exp $
>
> That's how it works in CVS.  The keywords are expanded by your
> client in the working copy.  Then the expanded form is committed
> when you commit.  Thus, the repository is always one behind.
>
>> (There has been a very heated discussion among GIT people why keyword
>> expansion is supposed to be evil. They are not going to be convinced to
>> implement it. I am very annoyed by that.)
>
> They are evil.  Evil evil evil evil.  EVIL!  No modern system
> supports them, nor are they likely, and we're all the better
> for it.

hg supports it

[Holger Weiss-2]

* Adam Hoka <adam.hoka@...> [2009-10-14 13:47]:

> On 10/14/09 04:44, Eric Gillespie wrote:
> >Rhialto<rhialto@...>  writes:
> >>(There has been a very heated discussion among GIT people why keyword
> >>expansion is supposed to be evil. They are not going to be convinced to
> >>implement it. I am very annoyed by that.)
> >
> >They are evil.  Evil evil evil evil.  EVIL!  No modern system
> >supports them, nor are they likely, and we're all the better
> >for it.
>
> hg supports it

These days, Git actually supports the expansion of $Id$ on checkout and
allows for implementing other stuff yourself, too; see gitattributes(5).

Holger

[Rhialto]

On Wed 14 Oct 2009 at 15:06:29 +0200, Holger Weiß wrote:

> * Adam Hoka <adam.hoka@...> [2009-10-14 13:47]:
> > On 10/14/09 04:44, Eric Gillespie wrote:
> > >Rhialto<rhialto@...>  writes:
> > >>(There has been a very heated discussion among GIT people why keyword
> > >>expansion is supposed to be evil. They are not going to be convinced to
> > >>implement it. I am very annoyed by that.)
> > >
> > >They are evil.  Evil evil evil evil.  EVIL!  No modern system
> > >supports them, nor are they likely, and we're all the better
> > >for it.
> >
> > hg supports it
>
> These days, Git actually supports the expansion of $Id$ on checkout and
> allows for implementing other stuff yourself, too; see gitattributes(5).

Yes, but the $Id$ that it supports is a hash value which is absolutely
meaningless to a human reader.

Why I want (cvs-like) $Id$ is that I can see at a glance which version a
file is, compare its version with that of another file, that sort of
things, without help from the VCS.  Files seem to have a habit of
"escaping from" the VCS so saying "but you can just look up the hash
value in the VCS" are not very helpful.

(There is some extra functionality with $Format$ if you "export" a tree,
but that requires making another full copy and is therefore even more
expensive.)

The only real reason I have found behind the statement "keyword
expansion is evil" is "it is inefficient". The reasoning behind that is
that if you switch from one branch to another, it may need to change
keywords in files which it doesn't otherwise change. There may be after
all files which are identical in two branches but with a different
history for each. (Not that that sounds too likely, anyway). The
argument then is that it would have to touch every file to get these
cases right.

But the way around that seems to me that it is easy enough to find the
path through the revision graph from the revision on the first branch to
that on the second one.  If a VCS is patch-based you have to do that
anyway but even if not it shouldn't be too expensive. You only have to
re-expand in those files that have edits anywhere on the path.

> Holger
-Olaf.
--
___ Olaf 'Rhialto' Seibert    -- You author it, and I'll reader it.
\X/ rhialto/at/xs4all.nl      -- Cetero censeo "authored" delendum esse.

[Brian de Alwis]

On 13-Oct-2009, at 10:44 PM, Eric Gillespie wrote:
> They are evil.  Evil evil evil evil.  EVIL!  No modern system
> supports them, nor are they likely, and we're all the better
> for it.


Actually, bzr has support for keyword expansion too with the bzr-
keywords plugin:

        https://launchpad.net/bzr-keywords

Brian.

--
"Amusement to an observing mind is study." - Benjamin Disraeli

[Lloyd Parkes]

S.P.Zeidler wrote:
> Dear all,
>
> htdocs, othersrc, pkgsrc, src and xsrc are now available as
> (bare) git repository copies at
> http://ftp.netbsd.org/pub/NetBSD/misc/repositories/git.
> These repositories are currently updated every 30 minutes.

Thanks for that. Git looks handy for those of us that aren't committers.

It keeps crashing my NetBSD/sparc64 5.0.1 box though. I run git from
devel/scmgit-base and sooner or later my system hangs or reboots while git is
running. It's really weird. I get some messages about running out of vnodes, but
increasing the maximum number of vnodes in the kernel by an order of magnitude
doesn't seem to help. I don't get crash dumps either. Savecore complains about a
"(null)" pointer.

Anyone else seeing this? If not I'll dig a bit deeper and try and come up with
enough information for a useful bug report.

Cheers,
Lloyd

[Arnaud Lacombe-6]

On Tue, Oct 13, 2009 at 3:38 AM, S.P.Zeidler <spz@...> wrote:
> Dear all,
>
> htdocs, othersrc, pkgsrc, src and xsrc are now available as
> (bare) git repository copies at
> http://ftp.netbsd.org/pub/NetBSD/misc/repositories/git.
> These repositories are currently updated every 30 minutes.
>
*Wonderfull* ! Thanks a lot !

I will no longer have to keep a copy of that 7G cvsroot, finished also
hours lost waiting for an update to complete :/ humm, XXIth century
old tools... :)

Btw, how did the convertion occur ? I guess with git-cvsimport ? I
remind that several ,v files were corrupted/unparsable and keithp'
parsecvs was unable to work with them.

Can we consider this tree stable (ie. do real work on top of it), or
may it be recreated to say fix the Id issue or stuff like that ?

Thanks,
 - Arnaud

On Mon, Oct 19, 2009 at 08:32:40AM +1300, Lloyd Parkes wrote:
> I don't get crash dumps either. Savecore complains about a "(null)" pointer.

I don't remember the outcome - the work around was specifying the /netbsd,
so e.g.,

  savecore -z -N netbsd /var/crash

-current is happy. I remember a discussion on how whatever savecore looks
for in /netbsd, it ought to find in what was written to the dump device,
but don't remember what the conclusion was.

Cheers,

Patrick

[Adam Hamsik-2]

On Oct,Monday 19 2009, at 12:14 AM, Arnaud Lacombe wrote:

> On Tue, Oct 13, 2009 at 3:38 AM, S.P.Zeidler <spz@...> wrote:
>> Dear all,
>>
>> htdocs, othersrc, pkgsrc, src and xsrc are now available as
>> (bare) git repository copies at
>> http://ftp.netbsd.org/pub/NetBSD/misc/repositories/git.
>> These repositories are currently updated every 30 minutes.
>>
> *Wonderfull* ! Thanks a lot !
>
> I will no longer have to keep a copy of that 7G cvsroot, finished also
> hours lost waiting for an update to complete :/ humm, XXIth century
> old tools... :)
>
> Btw, how did the convertion occur ? I guess with git-cvsimport ? I
> remind that several ,v files were corrupted/unparsable and keithp'
> parsecvs was unable to work with them.

Conversion was done by fromcvs tool. IT can do one time conversion and  
incremental updates, too.

>
> Can we consider this tree stable (ie. do real work on top of it), or
> may it be recreated to say fix the Id issue or stuff like that ?

For now this tree should be stable, but it was done for testing to  
found out if there can be some usage for git trees inside of NetBSD  
users group.

Regards

Adam.

[Justin C. Sherrill]

On Sun, October 18, 2009 7:06 pm, Adam Hamsik wrote:

> For now this tree should be stable, but it was done for testing to
> found out if there can be some usage for git trees inside of NetBSD
> users group.

For users outside of NetBSD, it's also useful.  There's already a rough
DragonFly conversion of pkgsrc to git, just to avoid using CVS when we're
using git for the rest of the system:

http://gitweb.dragonflybsd.org/pkgsrc.git

[Greg A. Woods-3]

At Mon, 19 Oct 2009 01:06:49 +0200, Adam Hamsik <haaaad@...> wrote:
Subject: Re: git copies of cvs modules available

>
>
> On Oct,Monday 19 2009, at 12:14 AM, Arnaud Lacombe wrote:
>
> > On Tue, Oct 13, 2009 at 3:38 AM, S.P.Zeidler <spz@...> wrote:
> >> Dear all,
> >>
> >> htdocs, othersrc, pkgsrc, src and xsrc are now available as
> >> (bare) git repository copies at
> >> http://ftp.netbsd.org/pub/NetBSD/misc/repositories/git.
> >> These repositories are currently updated every 30 minutes.
> >>
> > *Wonderfull* ! Thanks a lot !

Indeed!  Amazing!  This will be so incredibly useful!

I was hoping somehow someone would set this up!


> Conversion was done by fromcvs tool. IT can do one time conversion and
> incremental updates, too.

Good to know.

> For now this tree should be stable, but it was done for testing to
> found out if there can be some usage for git trees inside of NetBSD
> users group.

Git access to the "official" NetBSD source repository will be invaluable
to everyone who needs to tract NetBSD changes while also keeping track
of their own changes.  For those of us who submit patches back to NetBSD
it will be infinitely more usable than CVS alone.

--
                                                Greg A. Woods
                                                Planix, Inc.

<woods@...>       [phone]        http://www.planix.com/

[Rhialto]

On Mon 19 Oct 2009 at 08:32:40 +1300, Lloyd Parkes wrote:
> It keeps crashing my NetBSD/sparc64 5.0.1 box though. I run git from  
> devel/scmgit-base and sooner or later my system hangs or reboots while
> git is running. It's really weird. I get some messages about running out
> of vnodes, but increasing the maximum number of vnodes in the kernel by
> an order of magnitude doesn't seem to help. I don't get crash dumps
> either. Savecore complains about a "(null)" pointer.

Git seems *very* resource-intensive on repositories this size.

For testing, I run it on a machine with 512 MB memory, and with default
resource limits it often runs out of file descriptors and/or memory. It
also wants to compress the repository "for optimal efficiency" every
second blink of an eye (or so it seems). Since this takes rather a lot
of time and is very disk- and memory intensive (and hence fails more
often than not), that doesn't appear to be efficient at all to me.

-Olaf.
--
___ Olaf 'Rhialto' Seibert    -- You author it, and I'll reader it.
\X/ rhialto/at/xs4all.nl      -- Cetero censeo "authored" delendum esse.

[matthew sporleder]

Is there a git:// repo setup?  It's usually much faster than http://

On 10/19/09, Rhialto <rhialto@...> wrote:

> On Mon 19 Oct 2009 at 08:32:40 +1300, Lloyd Parkes wrote:
>> It keeps crashing my NetBSD/sparc64 5.0.1 box though. I run git from
>> devel/scmgit-base and sooner or later my system hangs or reboots while
>> git is running. It's really weird. I get some messages about running out
>> of vnodes, but increasing the maximum number of vnodes in the kernel by
>> an order of magnitude doesn't seem to help. I don't get crash dumps
>> either. Savecore complains about a "(null)" pointer.
>
> Git seems *very* resource-intensive on repositories this size.
>
> For testing, I run it on a machine with 512 MB memory, and with default
> resource limits it often runs out of file descriptors and/or memory. It
> also wants to compress the repository "for optimal efficiency" every
> second blink of an eye (or so it seems). Since this takes rather a lot
> of time and is very disk- and memory intensive (and hence fails more
> often than not), that doesn't appear to be efficient at all to me.
>
> -Olaf.
> --
> ___ Olaf 'Rhialto' Seibert    -- You author it, and I'll reader it.
> \X/ rhialto/at/xs4all.nl      -- Cetero censeo "authored" delendum esse.
>

--
Sent from my mobile device

[S.P.Zeidler]

Thus wrote matthew sporleder (msporleder@...):

> Is there a git:// repo setup?  It's usually much faster than http://

No, I don't want to run Yet Another Network Daemon there, and git isn't
supposed to be the only VCS for testing (and playing with).

regards,
        spz
--
spz@... (S.P.Zeidler)

[matthew sporleder]

On Tue, Oct 20, 2009 at 7:02 PM, S.P.Zeidler <spz@...> wrote:
> Thus wrote matthew sporleder (msporleder@...):
>
>> Is there a git:// repo setup?  It's usually much faster than http://
>
> No, I don't want to run Yet Another Network Daemon there, and git isn't
> supposed to be the only VCS for testing (and playing with).
>

If you get a chance to try it out temporarily, I would be interested
in timing clone/pull (http vs git) vs checkout/update (git vs cvs),
etc.

[Silas Silva]

On Mon, Oct 19, 2009 at 10:59:06PM +0200, Rhialto wrote:
> Git seems *very* resource-intensive on repositories this size.

After cloning the NetBSD git repository (which took some hours with a
not too fast connection), I pulled changes made on it while I was
cloning. After cloning it, it automatically ran "git gc" that is taking
hours to complete.

I asked about that in #git@freenode and people told me that, if this
command is run on the server, it doesn't need to be run frequently for
clients that clone it.

I'm a ordinary git user, so I don't know what exactly must be done.

Anybody runs this command on the server?

--
Silas Silva

[Marko Schütz-2]

While it is a little off topic, it reminded me of the following.

Wouldn't it be nice if we'd convert lots (most) PRs to automated tests
which would then automatically be run against each and every
commit...?

I have used Aegis (http://aegis.sourceforge.net/) which explicitly
supports this approach and found it very helpful. I understand that in
e.g. cvs or hg something similar can be achieved using triggers or
event hooks. Whatever the implementation, in my opinion it would be
great to have a growing collection of tests and to ensure that the
current version in the repository has passed all mandatory tests.

Best regards,

Marko

At Mon, 19 Oct 2009 14:09:25 -0400,
Greg A Woods wrote:

>
> At Mon, 19 Oct 2009 01:06:49 +0200, Adam Hamsik <haaaad@...> wrote:
> Subject: Re: git copies of cvs modules available
> >
> >
> > On Oct,Monday 19 2009, at 12:14 AM, Arnaud Lacombe wrote:
> >
> > > On Tue, Oct 13, 2009 at 3:38 AM, S.P.Zeidler <spz@...> wrote:
> > >> Dear all,
> > >>
> > >> htdocs, othersrc, pkgsrc, src and xsrc are now available as
> > >> (bare) git repository copies at
> > >> http://ftp.netbsd.org/pub/NetBSD/misc/repositories/git.
> > >> These repositories are currently updated every 30 minutes.
> > >>
> > > *Wonderfull* ! Thanks a lot !
>
> Indeed!  Amazing!  This will be so incredibly useful!
>
> I was hoping somehow someone would set this up!
>
>
> > Conversion was done by fromcvs tool. IT can do one time conversion and
> > incremental updates, too.
>
> Good to know.
>
> > For now this tree should be stable, but it was done for testing to
> > found out if there can be some usage for git trees inside of NetBSD
> > users group.
>
> Git access to the "official" NetBSD source repository will be invaluable
> to everyone who needs to tract NetBSD changes while also keeping track
> of their own changes.  For those of us who submit patches back to NetBSD
> it will be infinitely more usable than CVS alone.
>
> --
> Greg A. Woods
> Planix, Inc.
>
> <woods@...>       [phone]        http://www.planix.com/

[Joerg Sonnenberger]

On Mon, Oct 26, 2009 at 04:42:39PM -0400, Marko Schütz wrote:
> Wouldn't it be nice if we'd convert lots (most) PRs to automated tests
> which would then automatically be run against each and every
> commit...?

Who is going to do the work? There are a number of regression tests in
src/tests, they can even be run automatically. There is work on a fully
automated test setup as well. It still needs to be prepared though.

> I have used Aegis (http://aegis.sourceforge.net/) which explicitly
> supports this approach and found it very helpful.

The problem is that the approach doesn't scale. You can't run all
test cases for every single commit. It is simply not feasible. It is
also questionable how much problems are actually detected that way.

So you have good test cases, we like to include them. They just don't
grow on trees.

Joerg