Re: hg convert from git doesn't follow renames

[Greg Ward-17]

[switching to users list, since this is more of a support question
than a development question]

On Mon, Nov 2, 2009 at 9:54 PM, Paul Molodowitch <elrond79@...> wrote:
> Was looking to switch my repo over from git to hg, but I ran into a
> bit of a snag - it seems hg convert from git doesn't preserve renaming
> information.

One possibility is to use fastimport.  That is, dump the git repo with
'git fast-export', then use the hg-fastimport extension
(http://mercurial.selenic.com/wiki/FastImportExtension) to load it
into Mercurial.  *If* it works, I think it will do a more accurate
conversion than hg convert does.

Disclaimer: although I am the maintainer of hg-fastimport, I honestly
don't know how well this will work for you.  I haven't tested
hg-fastimport with exports from git.  And I apologize for the clunky
setup procedure.  Let us know how it goes!

Greg
_______________________________________________
Mercurial mailing list
Mercurial@...
http://selenic.com/mailman/listinfo/mercurial

[chadrik]

>> Was looking to switch my repo over from git to hg, but I ran into a
>> bit of a snag - it seems hg convert from git doesn't preserve  
>> renaming
>> information.
>
> One possibility is to use fastimport.  That is, dump the git repo with
> 'git fast-export', then use the hg-fastimport extension
> (http://mercurial.selenic.com/wiki/FastImportExtension) to load it
> into Mercurial.  *If* it works, I think it will do a more accurate
> conversion than hg convert does.

i've tried hg-fastimport/export, the hg-git extension, and hg convert,  
and none of them implement renames.  is this something that is  
inherently difficult to do or just something that is easily  
forgotten?  it's frightening to me to think that every person who has  
converted to hg from git has permanently butchered their history.

-chad


_______________________________________________
Mercurial mailing list
Mercurial@...
http://selenic.com/mailman/listinfo/mercurial

[Augie Fackler-2]

On Tue, Nov 3, 2009 at 9:40 AM, Chad Dombrova <chadrik@...> wrote:

>
>>> Was looking to switch my repo over from git to hg, but I ran into a
>>> bit of a snag - it seems hg convert from git doesn't preserve
>>> renaming
>>> information.
>>
>> One possibility is to use fastimport.  That is, dump the git repo with
>> 'git fast-export', then use the hg-fastimport extension
>> (http://mercurial.selenic.com/wiki/FastImportExtension) to load it
>> into Mercurial.  *If* it works, I think it will do a more accurate
>> conversion than hg convert does.
>
> i've tried hg-fastimport/export, the hg-git extension, and hg convert,
> and none of them implement renames.  is this something that is
> inherently difficult to do or just something that is easily
> forgotten?  it's frightening to me to think that every person who has
> converted to hg from git has permanently butchered their history.

hg-git *should* be using the same rename-checking algorithm as git. If
it's not, we should fix that.

> -chad
>
>
> _______________________________________________
> Mercurial mailing list
> Mercurial@...
> http://selenic.com/mailman/listinfo/mercurial
>

_______________________________________________
Mercurial mailing list
Mercurial@...
http://selenic.com/mailman/listinfo/mercurial

[chadrik]

hg-git *should* be using the same rename-checking algorithm as git. If
it's not, we should fix that.

we've had the problem with this fork: http://bitbucket.org/abderrahim/hg-git/

is there another we should be using?

_______________________________________________
Mercurial mailing list
Mercurial@...
http://selenic.com/mailman/listinfo/mercurial

[Mads Kiilerich-3]

On 11/03/2009 05:03 PM, Chad Dombrova wrote:
> we've had the problem with this fork:
> http://bitbucket.org/abderrahim/hg-git/
>
> is there another we should be using?

http://mercurial.selenic.com/wiki/HgGit points to
http://hg-git.github.com/ which points to
http://bitbucket.org/durin42/hg-git/ which also includes the changes
from abderrahims repo.

/Mads
_______________________________________________
Mercurial mailing list
Mercurial@...
http://selenic.com/mailman/listinfo/mercurial

[Isaac Jurado]

Replying Chad Dombrova:
> >
> >hg-git *should* be using the same rename-checking algorithm as git. If
> >it's not, we should fix that.
>
> we've had the problem with this fork:
> http://bitbucket.org/abderrahim/hg-git/
>
> is there another we should be using?

There is also this one:

    http://github.com/schacon/hg-git
    http://hg-git.github.com

_______________________________________________
Mercurial mailing list
Mercurial@...
http://selenic.com/mailman/listinfo/mercurial

[chadrik]

i've found the "official" github repo to be pretty out of date.  i'll  
try http://bitbucket.org/durin42/hg-git/, since it has abderrahims'  
changes too.

-chad


On Nov 3, 2009, at 8:10 AM, Mads Kiilerich wrote:

> On 11/03/2009 05:03 PM, Chad Dombrova wrote:
>> we've had the problem with this fork:
>> http://bitbucket.org/abderrahim/hg-git/
>>
>> is there another we should be using?
>
> http://mercurial.selenic.com/wiki/HgGit points to
> http://hg-git.github.com/ which points to
> http://bitbucket.org/durin42/hg-git/ which also includes the changes  
> from abderrahims repo.
>
> /Mads

_______________________________________________
Mercurial mailing list
Mercurial@...
http://selenic.com/mailman/listinfo/mercurial

[Paul Molodowitch]

>> Was looking to switch my repo over from git to hg, but I ran into a
>> bit of a snag - it seems hg convert from git doesn't preserve renaming
>> information.
>
> One possibility is to use fastimport.  That is, dump the git repo with
> 'git fast-export', then use the hg-fastimport extension
> (http://mercurial.selenic.com/wiki/FastImportExtension) to load it
> into Mercurial.  *If* it works, I think it will do a more accurate
> conversion than hg convert does.

Hmm.. this seems promising! Unfortunately, when I tested it out with a
simple git repo, which I'd dumped using:

cd gitRepo
git fast-export --all -M -C > ..\gitDump
cd ..
hg init hgFastImp
cd hgFastImp
hg fastimport ..\gitDump

...I got the following traceback:

Traceback (most recent call last):
  File "C:\Dev\Python25\Scripts\hg", line 5, in <module>
    pkg_resources.run_script('mercurial==1.3.1', 'hg')
  File "C:\Dev\Python25\lib\site-packages\pkg_resources.py", line 448,
in run_script
    self.require(requires)[0].run_script(script_name, ns)
  File "C:\Dev\Python25\lib\site-packages\pkg_resources.py", line
1166, in run_script
    execfile(script_filename, namespace, namespace)
  File "c:\dev\python25\lib\site-packages\mercurial-1.3.1-py2.5-win32.egg\EGG-INFO\scripts\hg",
line 27, in <module>
    mercurial.dispatch.run()
  File "C:\Dev\Python25\lib\site-packages\mercurial-1.3.1-py2.5-win32.egg\mercurial\dispatch.py",
line 16, in run
    sys.exit(dispatch(sys.argv[1:]))
  File "C:\Dev\Python25\lib\site-packages\mercurial-1.3.1-py2.5-win32.egg\mercurial\dispatch.py",
line 27, in dispatch
    return _runcatch(u, args)
  File "C:\Dev\Python25\lib\site-packages\mercurial-1.3.1-py2.5-win32.egg\mercurial\dispatch.py",
line 43, in _runcatch
    return _dispatch(ui, args)
  File "C:\Dev\Python25\lib\site-packages\mercurial-1.3.1-py2.5-win32.egg\mercurial\dispatch.py",
line 449, in _dispatch
    return runcommand(lui, repo, cmd, fullargs, ui, options, d)
  File "C:\Dev\Python25\lib\site-packages\mercurial-1.3.1-py2.5-win32.egg\mercurial\dispatch.py",
line 317, in runcommand
    ret = _runcommand(ui, options, cmd, d)
  File "C:\Dev\Python25\lib\site-packages\mercurial-1.3.1-py2.5-win32.egg\mercurial\dispatch.py",
line 501, in _runcommand
    return checkargs()
  File "C:\Dev\Python25\lib\site-packages\mercurial-1.3.1-py2.5-win32.egg\mercurial\dispatch.py",
line 454, in checkargs
    return cmdfunc()
  File "C:\Dev\Python25\lib\site-packages\mercurial-1.3.1-py2.5-win32.egg\mercurial\dispatch.py",
line 448, in <lambda>
    d = lambda: util.checksignature(func)(ui, *args, **cmdoptions)
  File "C:\Dev\Python25\lib\site-packages\mercurial-1.3.1-py2.5-win32.egg\mercurial\util.py",
line 402, in check
    return func(*args, **kwargs)
  File "C:\Dev\Python25\Lib\site-packages\hg-fastimport\hgfastimport\__init__.py",
line 44, in fastimport
    c.convert(sortmode)
  File "C:\Dev\Python25\lib\site-packages\mercurial-1.3.1-py2.5-win32.egg\hgext\convert\convcmd.py",
line 311, in convert
    heads = self.source.getheads()
  File "C:\Dev\Python25\lib\site-packages\hg-fastimport\hgfastimport\hgimport.py",
line 50, in getheads
    self._parse()
  File "C:\Dev\Python25\lib\site-packages\hg-fastimport\hgfastimport\hgimport.py",
line 112, in _parse
    processor.parseMany(self.sources, parser.ImportParser, self.processor)
  File "C:\Dev\Python25\Lib\site-packages\pyfastimport\fastimport\processor.py",
line 219, in parseMany
    processor.process(parser.parse())
  File "C:\Dev\Python25\Lib\site-packages\pyfastimport\fastimport\processor.py",
line 76, in process
    handler(self, cmd)
  File "C:\Dev\Python25\lib\site-packages\hg-fastimport\hgfastimport\hgimport.py",
line 275, in commit_handler
    commit_handler.process()
  File "C:\Dev\Python25\Lib\site-packages\pyfastimport\fastimport\processor.py",
line 154, in process
    handler(self, fc)
  File "C:\Dev\Python25\lib\site-packages\hg-fastimport\hgfastimport\hgimport.py",
line 408, in rename_handler
    self.copies[filecmd.oldname] = filecmd.newname
AttributeError: 'FileRenameCommand' object has no attribute 'newname'

I'll poke around in hg-fastimport's code a little later - hopefully
there's an easy fix!

- Paul

_______________________________________________
Mercurial mailing list
Mercurial@...
http://selenic.com/mailman/listinfo/mercurial

[Augie Fackler-2]

On Tue, Nov 3, 2009 at 10:20 AM, Chad Dombrova <chadrik@...> wrote:
> i've found the "official" github repo to be pretty out of date.  i'll
> try http://bitbucket.org/durin42/hg-git/, since it has abderrahims'
> changes too.

The official github repo and my repo on bitbucket are kept in sync.
They shouldn't be any different. Looking at our tests directory [0],
it appears that we don't have a sane rename test. Can someone
contribute one (and preferably, although less needed, a patch to fix
rename detection? the hooks should all be there...)?

Thanks,
Augie

[0] http://bitbucket.org/durin42/hg-git/src/tip/tests/

>
> -chad
>
>
> On Nov 3, 2009, at 8:10 AM, Mads Kiilerich wrote:
>
>> On 11/03/2009 05:03 PM, Chad Dombrova wrote:
>>> we've had the problem with this fork:
>>> http://bitbucket.org/abderrahim/hg-git/
>>>
>>> is there another we should be using?
>>
>> http://mercurial.selenic.com/wiki/HgGit points to
>> http://hg-git.github.com/ which points to
>> http://bitbucket.org/durin42/hg-git/ which also includes the changes
>> from abderrahims repo.
>>
>> /Mads
>
> _______________________________________________
> Mercurial mailing list
> Mercurial@...
> http://selenic.com/mailman/listinfo/mercurial
>

_______________________________________________
Mercurial mailing list
Mercurial@...
http://selenic.com/mailman/listinfo/mercurial

[Greg Ward-17]

On Tue, Nov 3, 2009 at 11:23 AM, Paul Molodowitch <elrond79@...> wrote:
> Hmm.. this seems promising! Unfortunately, when I tested it out with a
> simple git repo, which I'd dumped using:
[...]
> ...I got the following traceback:
>
> Traceback (most recent call last):
>  File "C:\Dev\Python25\Scripts\hg", line 5, in <module>
>    pkg_resources.run_script('mercurial==1.3.1', 'hg')
[...]
> line 154, in process
>    handler(self, fc)
>  File "C:\Dev\Python25\lib\site-packages\hg-fastimport\hgfastimport\hgimport.py",
> line 408, in rename_handler
>    self.copies[filecmd.oldname] = filecmd.newname
> AttributeError: 'FileRenameCommand' object has no attribute 'newname'

Ooooops.  Guess which part of the code is missing tests?

> I'll poke around in hg-fastimport's code a little later - hopefully
> there's an easy fix!

Very likely.  fastimport syntax can represent renames, and hg can
handle renames.  And I'm pretty sure that hg-fastimport at least
*tries* to hook the two up.  So it should be an easy fix.  If you can
throw in a test fix, too, that would be super.

Greg

_______________________________________________
Mercurial mailing list
Mercurial@...
http://selenic.com/mailman/listinfo/mercurial

[Greg Ward-17]

On Tue, Nov 3, 2009 at 11:23 AM, Paul Molodowitch <elrond79@...> wrote:
> Hmm.. this seems promising! Unfortunately, when I tested it out with a
> simple git repo, which I'd dumped using:
>
> cd gitRepo
> git fast-export --all -M -C > ..\gitDump
> cd ..
> hg init hgFastImp
> cd hgFastImp
> hg fastimport ..\gitDump

Hey, I just realized that this means you got hg-fastimport working on
Windows.  Fantastic!  If you want to be even more super nice, would
you mind writing up in the wiki page how you did it?  Just add a
separate section after my long clunky Unix setup procedure.

Thanks!

Greg
_______________________________________________
Mercurial mailing list
Mercurial@...
http://selenic.com/mailman/listinfo/mercurial

[Abderrahim Kitouni-2]

Hi,

2009/11/3, Augie Fackler <lists@...>:
> On Tue, Nov 3, 2009 at 10:20 AM, Chad Dombrova <chadrik@...> wrote:
>> i've found the "official" github repo to be pretty out of date.  i'll
>> try http://bitbucket.org/durin42/hg-git/, since it has abderrahims'
>> changes too.
Just to make this clear, Augie is now the maintainer of hg-git, his
repo and the repo on github are the reference. My repo just contains a
couple bugfixes that I haven't got around to have them merged.

>
> The official github repo and my repo on bitbucket are kept in sync.
> They shouldn't be any different. Looking at our tests directory [0],
> it appears that we don't have a sane rename test. Can someone
> contribute one (and preferably, although less needed, a patch to fix
> rename detection? the hooks should all be there...)?
There are no hooks at all, rename detection was removed (even before I
started working on hg-git) because it conflicted with lossless
conversion both ways.

Just to say that rename detection is non-trivial, and I see two ways
to implement it :
* at runtime (like this is done in git) : this will require somehow
monkey-patching hg.
* at convert time : and add the guessed renamed files to an extra field.
(I prefer the latter is easier since it has less chances to depend on
the version of hg used).

If no one is interesed in this, I'll try to do it when I have some time.

Peace,
Abderrahim
_______________________________________________
Mercurial mailing list
Mercurial@...
http://selenic.com/mailman/listinfo/mercurial