[rvm-core] an official git mirror of the SVN repository?

[Will Benton-2]

I'm wondering if it would be useful for the community to have a  
public, official (or semi-official) git mirror of the subversion  
repository.  (I mentioned this in an email to Ian Rogers and he  
suggested that I send a message to jikesrvm-core.)  The advantage to a  
public git repository is that people could easily clone and branch the  
main code and share their changes with one another, while tracking  
updates to the main repository and having access to the full commit  
history and every branch.

Mirroring via git-svn directly from sourceforge is sort of a pain,  
since it can take an extremely long time, and the server seems prone  
to disconnection and glitches when one asks for the entire project  
history and every branch.  However, I've had good experiences using a  
combination of rsync and git-svn to maintain a private git repository  
on github.com that mirrors and tracks the entire Jikes RVM svn tree  
but also includes my personal research branches.  GitHub is a  
particularly nice place to host git repositories:  they offer free  
hosting for open-source projects and provide a great interface for  
browsing repositories, making personal forks of repositories, and so on.

I'd like to volunteer to set up a public mirror of the Jikes RVM  
repository on github and keep it up-to-date by syncing twice daily or  
so.  My thinking on the matter is that this mirror would basically be  
read-only and track the SVN; folks hacking on Jikes could then easily  
fork the repository to make local changes and send patches back to the  
core team for inclusion in the main tree or in the research archive.  
(While git-svn supports bidirectional changes between git and svn  
repositories, this is not compatible with my rsync-based workflow due  
to how git-svn creates commit IDs.)

I'm also happy to share my scripts and workflow if a member of the  
core team is interested in taking on creation and maintenance (via  
cron, of course) of a public git mirror.

Thanks for your time, and thanks to all of you for your hard work on  
this excellent research infrastructure.


best,
wb

---
Will Benton <willb@...>
http://pages.cs.wisc.edu/~willb/


-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Jikesrvm-core mailing list
Jikesrvm-core@...
https://lists.sourceforge.net/lists/listinfo/jikesrvm-core

[Steve Blackburn]

Hi Will,

I think you've hit on a great point.

Let's pop up a level.  What's the problem your suggestion is trying to  
solve?

I suspect that there are two primary issues:


1.  Managing local code development.

Arguably the main purpose of Jikes RVM is not as an end-product JVM,  
but rather as a research platform (hence the name).   So while for  
most projects the number of people accessing the code is orders of  
magnitude less than the number of people using the system, for us the  
two are approximately equal. This is unusual and may mean we should do  
things differently to other projects.  This also means that our choice  
of scm is particularly important, since it is the interface between  
our "end users" and our "product".

As researchers we need to be able to coherently develop our work  
locally (before ready to be public) and under an scm (so we can sanely  
manage our work).  A centralized scm (such as svn) does not directly  
provide such an option.


2. Sharing our work

As researchers we should be sharing our artifacts.  This is key to  
research credibility and good science.

We (end users) depend on a stable, well-performing, and coherent  
head.   As in all projects that are concerned with stability and  
performance, we therefore must be very selective about what we add to  
the code base.   Each addition to the code base must be consistent  
with our objectives of stability and performance, but must also have a  
compelling story for how it will be maintained.  Arguably we are over-
extended in terms of maintainability and should prune some of our  
lesser used code.  The bottom line is that most of the cool work that  
goes on in Jikes RVM won't make its way into the head.  This seems to  
conflict with point 3 above, and has generated a fair amount of angst  
in the past.  Our existing solution is to provide a tracker (http://jikesrvm.org/Research+Archive 
).  Some researcher also publish patches against releases along with  
their papers.



Furthermore, we are apparently constrained to using one of the scms on  
offer from sourceforge (I'm not sure how true this statement is, but  
in any case we would not take lightly a shift to a different scm).  As  
an aside, if there was a strong consensus that we needed some kind of  
mirror, we have resources at ANU for that (in fact we currently run an  
svn mirror here which we use for all of our regression testing---it is  
updated every few minutes and protects the regression system against  
outages at sourceforge or in the network).


If I've understood the problem right, the question then is, what is  
the best solution?

Arguably a system like svk (http://svk.bestpractical.com/view/ 
HomePage) addresses the first problem given our current use of svn.

The research tracker exists to address the second problem (whether it  
does a good job is a different matter).

So, Will,  does my analysis above make sense?  If so, can say a little  
more about your specific solution and how it addresses these concerns  
better than alternatives?  If not, can you fill in some of the gaps?

Thanks again for the input.  I think the issue you're touching on is  
an important one.

Cheers,

--Steve


On 14/10/2008, at 2:43 PM, Will Benton wrote:

> I'm wondering if it would be useful for the community to have a
> public, official (or semi-official) git mirror of the subversion
> repository.  (I mentioned this in an email to Ian Rogers and he
> suggested that I send a message to jikesrvm-core.)  The advantage to a
> public git repository is that people could easily clone and branch the
> main code and share their changes with one another, while tracking
> updates to the main repository and having access to the full commit
> history and every branch.
>
> Mirroring via git-svn directly from sourceforge is sort of a pain,
> since it can take an extremely long time, and the server seems prone
> to disconnection and glitches when one asks for the entire project
> history and every branch.  However, I've had good experiences using a
> combination of rsync and git-svn to maintain a private git repository
> on github.com that mirrors and tracks the entire Jikes RVM svn tree
> but also includes my personal research branches.  GitHub is a
> particularly nice place to host git repositories:  they offer free
> hosting for open-source projects and provide a great interface for
> browsing repositories, making personal forks of repositories, and so  
> on.
>
> I'd like to volunteer to set up a public mirror of the Jikes RVM
> repository on github and keep it up-to-date by syncing twice daily or
> so.  My thinking on the matter is that this mirror would basically be
> read-only and track the SVN; folks hacking on Jikes could then easily
> fork the repository to make local changes and send patches back to the
> core team for inclusion in the main tree or in the research archive.
> (While git-svn supports bidirectional changes between git and svn
> repositories, this is not compatible with my rsync-based workflow due
> to how git-svn creates commit IDs.)
>
> I'm also happy to share my scripts and workflow if a member of the
> core team is interested in taking on creation and maintenance (via
> cron, of course) of a public git mirror.
>
> Thanks for your time, and thanks to all of you for your hard work on
> this excellent research infrastructure.
>
>
> best,
> wb
>
> ---
> Will Benton <willb@...>
> http://pages.cs.wisc.edu/~willb/
>
>
> -------------------------------------------------------------------------
> This SF.Net email is sponsored by the Moblin Your Move Developer's  
> challenge
> Build the coolest Linux based applications with Moblin SDK & win  
> great prizes
> Grand prize is a trip for two to an Open Source event anywhere in  
> the world
> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> _______________________________________________
> Jikesrvm-core mailing list
> Jikesrvm-core@...
> https://lists.sourceforge.net/lists/listinfo/jikesrvm-core

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Jikesrvm-core mailing list
Jikesrvm-core@...
https://lists.sourceforge.net/lists/listinfo/jikesrvm-core

[Peter Donald-2]

On Thu, Oct 16, 2008 at 4:10 PM, Steve Blackburn
<steve.blackburn@...> wrote:
> Furthermore, we are apparently constrained to using one of the scms on
> offer from sourceforge (I'm not sure how true this statement is, but
> in any case we would not take lightly a shift to a different scm).

Not really that true ;) We don't really use much of the other
infrastructure at sourceforge (IIRC the research tracker and svn are
the only remaining components). It would be relatively simple to move
that infrastructure elsewhere if anyone had the inclination.

However I don't think Will is suggesting that primary development be
moved across to another scm (at this stage). The authoritative copy of
the rvm is in sf and a mirror is present on github (that will host for
free) for git users.

As most of the "audience" for the project is likely to find git more
consistent with their workflows. i.e. Anyone can have their own
version controlled branch of rvm and publish it with no difficulty. My
guess is that after trialing github in this fashion you may find that
most of the core team move to using it aswell ;)

> Arguably a system like svk (http://svk.bestpractical.com/view/
> HomePage) addresses the first problem given our current use of svn.

svk is great for what it is but it can be annoying to setup and if you
work across multiple computers you need to figure out a way to publish
non-mainline branches and it can be annoying.

> The research tracker exists to address the second problem (whether it
> does a good job is a different matter).

I would say it does not do a good job ;) i.e. It is easy to imagine a
situation where person A wants to have a branch that is a merge of
person Bs work and person Cs work plus some of their own
modifications. To do this via the research trackers requires lots of
manual labor for everyone who wants to check it out

Anyhoo I don't really have a stake in it atm but mirroring a tree on
github seems like a really good idea to me and if the mirroring
process could be setup on one of the anu boxes even better.

--
Cheers,

Peter Donald
Dennis Miller  - "A recent police study found that you're much more
likely to get shot by a fat cop if you run."

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Jikesrvm-core mailing list
Jikesrvm-core@...
https://lists.sourceforge.net/lists/listinfo/jikesrvm-core

[Will Benton-2]

Steve and Peter,

Thanks for reading and responding.  I'll reply to both of your  
messages here.

On Oct 16, 2008, at 12:10 AM, Steve Blackburn wrote:

> So, Will,  does my analysis above make sense?  If so, can say a little
> more about your specific solution and how it addresses these concerns
> better than alternatives?  If not, can you fill in some of the gaps?

Steve, I think you're exactly right.  I don't have a particular agenda  
with regard to how those aims are accomplished (after all, I can  
continue using the workflow that works for me no matter what other  
people do), but I think git is strictly no worse than any other  
solution, and works better than the alternatives on both of these  
fronts:

1.  Managing local code development.

   *  git (and many other distributed development tools) gives you a  
working directory that is also a low-cost mirror of the entire  
repository.  While this feature is shared by most distributed  
development tools, I note that the git repository format seems to be  
very space-efficient (for my local mirror, the git representation of  
the repository is about 1/3 the size of my subversion mirror -- other  
projects have reported greater space savings).

   *  In my experience, git makes it easier to branch than other  
tools.  While this is not a hard data point, it has surprised me how  
much the ease of branching changes the way I approach development --  
it is much easier to attempt substantial changes independently of  
other work and then integrate them with orthogonal work when convenient.

2.  Sharing our work.

   *  This is where the "your working directory is a first-class  
repository" really shines.  If I have a branch that you want to use,  
you can simply track my branch in your repository, merging changes  
from my branch and one of your branches as you see fit.  (Contrast  
this with the svk workflow, in which, if I recall correctly, two svk  
users have to synchronize by committing to the centralized repository.)

   *  git's approach to merging is also, in my opinion, better than  
sharing patches.  When you merge in git, you essentially replay a  
series of commits against a common ancestor (so no history is lost).  
Incorporating a patch in subversion means that all of the (local)  
history for those changes is lost and that authorship of every change  
is ascribed to the person who committed the patch.

   *  git is also completely compatible with the Research Archive; as  
with most tools, it is trivial to generate patches from a git  
repository.

I will also add, before responding to Peter's comments, that the  
github environment is a particularly nice place to work with git,  
especially because it is easy to see any branches of a particular  
repository that are also hosted on github.

On Oct 16, 2008, at 5:27 PM, Peter Donald wrote:

> However I don't think Will is suggesting that primary development be
> moved across to another scm (at this stage). The authoritative copy of
> the rvm is in sf and a mirror is present on github (that will host for
> free) for git users.

This is what I'm suggesting.  I think that the nice thing about using  
git (and, specifically, github) is that it supports sort of the best  
of all worlds.  I can publish my changes as a custom branch and  
trivially generate a patch against any upstream version for inclusion  
in the research archive.  Furthermore, given git's excellent support  
for branching and merging, it is much easier to keep my local changes  
in sync with the upstream source and to merge multiple external  
changes into a new (personal) branch.

> svk is great for what it is but it can be annoying to setup and if you
> work across multiple computers you need to figure out a way to publish
> non-mainline branches and it can be annoying.


svk is a really nice tool, but it addresses one of the limitations of  
svn (lack of support for disconnected development) while inheriting  
most of the other limitations (most notably, information/history loss  
while merging and an essentially centralized model).  Since I am  
suggesting something that will work for people who are hacking on the  
RVM but most likely not committing upstream, I don't necessarily see  
slightly tighter svn integration as a win.  (I only evaluated svk  
briefly a while ago; as I understand it, svk's support for branching  
and merging is improved from standard svn, but not quite as nice as  
git.)



Thanks again for the great discussion.


thanks,
wb

---
Will Benton <willb@...>
http://pages.cs.wisc.edu/~willb/


-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Jikesrvm-core mailing list
Jikesrvm-core@...
https://lists.sourceforge.net/lists/listinfo/jikesrvm-core