The Bugzilla Migration Framework

[Max Kanat-Alexander]

        I just checked in migrate.pl, a framework for migrating from other
bug-tracking systems to Bugzilla.

        The exciting news is that this makes it very easy to write new
migrators to migrate from any bug-tracking system to Bugzilla.

        Also, this makes migration a first-class part of Bugzilla, not part of
contrib/. This means that we will maintain the migration framework, and
the migrators themselves, and if they break, that's a regression.

        The first system implemented is GNATS, because (a) a client hired me to
write a modern Bugzilla->GNATS migrator (which was actually the whole
reason I wrote this framework) and (b) GNATS is dead (as far as I know)
and so we won't have to change anything about the migrator in the future.

        The GNATS migrator is Bugzilla::Migrate::GNATS, if you want to see a
sample of a migrator.

        The migration framework has the following features:

        1) It is non-destructive. You can migrate into an existing Bugzilla
installation!

        2) It uses Bugzilla::Object stuff to create products, components, bugs,
users, and attachments, so it will continue to work into the future, and
automatically validates all data as it is inserted.

        3) All of the inserting of data is handled by Bugzilla::Migrate, so
your migration code doesn't have to do that.

        4) It's relatively easy to implement new migrators for systems.
Basically all you have to do is provide arrays of hashes describing the
products, users, and bugs in your bug-tracker, and Bugzilla::Migrate
does the rest of the work. It even contains a facility to translate
values from the old bug-tracker to Bugzilla, and allows the end user to
specify how that translation should work.

        5) It is capable of doing a "dry-run" of your migration, so that you
can see if everything is going to go well before actually committing
anything to the database.

        6) Bugzilla::Migtate prints output, handles configuration of your
migrator--basically, it really does *everything* except for reading the
data from your bug-tracker.


        So now, I'd really love to see some new migrators! The bug-trackers
that I'd most like to see new migrators for are:

        JIRA
        Trac
        Mantis

        And any other bug-tracker that you want to write a migrator for will be
welcome!

        The review requirements on migrators are slightly relaxed--I will
believe you when you say that it works, as long as it's coded well and
follows the Bugzilla Developers' Guide.

        Also, if anybody who wants to provide me (or the Bugzilla project in
general) a dump of a large working installation of a bug-tracker, that
will help people who want to write a migrator.

        Happy migrating! :-)

        -Max
--
http://www.everythingsolved.com/
Competent, Friendly Bugzilla and Perl Services. Everything Else, too.
-
To view or change your list settings, click here:
<http://bugzilla.org/cgi-bin/mj_wwwusr?user=lists@...>

[Max Kanat-Alexander]

On 10/23/2009 10:42 PM, Max Kanat-Alexander wrote:
> (a) a client hired me to
> write a modern Bugzilla->GNATS migrator (which was actually the whole
> reason I wrote this framework)

        Hahaha, that of course should read:

        "a modern GNATS->Bugzilla migrator"

> The GNATS migrator is Bugzilla::Migrate::GNATS, if you want to see a
> sample of a migrator.

        Oh, and that should be "Bugzilla::Migrate::Gnats".

        -Max
--
http://www.everythingsolved.com/
Competent, Friendly Bugzilla and Perl Services. Everything Else, too.
-
To view or change your list settings, click here:
<http://bugzilla.org/cgi-bin/mj_wwwusr?user=lists@...>

[David Miller]

Max Kanat-Alexander wrote on 10/24/09 1:42 AM:
> I just checked in migrate.pl, a framework for migrating from other
> bug-tracking systems to Bugzilla.

This is *huge*.  :)

You should write up a version of this post targeted at integrators and
consultants and send it to the announce list, might get some good
traction on it.

Should we make any kind of rule about not breaking import from an older
version of something in order to make an import from a newer version
work?  I'm pretty sure the gnats2bz script in contrib got updated for a
newer version of gnats at one point and it got wholesale replaced with
the code for the new version instead of checking to see which schema it
needed to work with.

On the other hand, if it always works with the latest version I suppose
we could tell people they need to ugprade whatever they're migrating
from to the latest first.  But then again, sometimes that costs money
you wouldn't want to spend if you're moving away from that product anyway...

--
Dave Miller                                   http://www.justdave.net/
System Administrator, Mozilla Corporation      http://www.mozilla.com/
Project Leader, Bugzilla Bug Tracking System  http://www.bugzilla.org/
-
To view or change your list settings, click here:
<http://bugzilla.org/cgi-bin/mj_wwwusr?user=lists@...>

[Gervase Markham]

On 24/10/09 06:42, Max Kanat-Alexander wrote:
> I just checked in migrate.pl, a framework for migrating from other
> bug-tracking systems to Bugzilla.

Cool :-)

> Also, this makes migration a first-class part of Bugzilla, not part of
> contrib/. This means that we will maintain the migration framework, and
> the migrators themselves, and if they break, that's a regression.

Do we then have an install of GNATS on Landfill that we are planning to
use to test the migrator with?

> 1) It is non-destructive. You can migrate into an existing Bugzilla
> installation!

An obvious consequence of this is that someone could write a Bugzilla
migrator, and you could use it to merge Bugzilla installations.

> 5) It is capable of doing a "dry-run" of your migration, so that you
> can see if everything is going to go well before actually committing
> anything to the database.

That's particularly brilliant.

Gerv
_______________________________________________
dev-apps-bugzilla mailing list
dev-apps-bugzilla@...
https://lists.mozilla.org/listinfo/dev-apps-bugzilla
-
To view or change your list settings, click here:
<http://bugzilla.org/cgi-bin/mj_wwwusr?user=lists@...>

[Olivier Berger]

Hi.

Le vendredi 23 octobre 2009 à 22:42 -0700, Max Kanat-Alexander a écrit :
> I just checked in migrate.pl, a framework for migrating from other
> bug-tracking systems to Bugzilla.
>
> The exciting news is that this makes it very easy to write new
> migrators to migrate from any bug-tracking system to Bugzilla.
>

Interesting news.

I have one question : if I understand it correctly, it works by
connecting to both bugtrackers' databases to do the "on the fly"
conversion, right ?

Have you thought about using some dump + restore approach involving some
intermediate interchange format ?

Thanks in advance.

Best regards,
--
Olivier BERGER <olivier.berger@...>
http://www-public.it-sudparis.eu/~berger_o/ - OpenPGP-Id: 1024D/6B829EEC
Ingénieur Recherche - Dept INF
Institut TELECOM, SudParis (http://www.it-sudparis.eu/), Evry (France)

-
To view or change your list settings, click here:
<http://bugzilla.org/cgi-bin/mj_wwwusr?user=lists@...>

[Max Kanat-Alexander]

On 10/24/2009 12:42 AM, David Miller wrote:
> You should write up a version of this post targeted at integrators and
> consultants and send it to the announce list, might get some good
> traction on it.

        I did this today, basically, as part of the latest Bugzilla Update:

        http://bugzillaupdate.wordpress.com/2009/11/05/release-3-4-3/

        By the way, I highly encourage anybody who reads this list to also
subscribe to the Bugzilla Update--it's a good concise summary of what's
been going on with the Bugzilla Project, and I hope to update it more
often than we release--that is, on a somewhat regular schedule.

> Should we make any kind of rule about not breaking import from an older
> version of something in order to make an import from a newer version
> work?  I'm pretty sure the gnats2bz script in contrib got updated for a
> newer version of gnats at one point and it got wholesale replaced with
> the code for the new version instead of checking to see which schema it
> needed to work with.
>
> On the other hand, if it always works with the latest version I suppose
> we could tell people they need to ugprade whatever they're migrating
> from to the latest first.  But then again, sometimes that costs money
> you wouldn't want to spend if you're moving away from that product anyway...

        I think the thing to do for now is to play it by ear. If there's a
situation that's going to break import from an old version, but the
version is like 10 years old, then I think it's less important than if
we're breaking import from the latest stable version in order to get
compatibility with a development version. (Those would be the two extremes.)

        -Max
--
http://www.everythingsolved.com/
Competent, Friendly Bugzilla and Perl Services. Everything Else, too.
-
To view or change your list settings, click here:
<http://bugzilla.org/cgi-bin/mj_wwwusr?user=lists@...>

[Max Kanat-Alexander]

On 11/03/2009 09:28 AM, Olivier Berger wrote:
> I have one question : if I understand it correctly, it works by
> connecting to both bugtrackers' databases to do the "on the fly"
> conversion, right ?

        It doesn't *have* to do anything. It's just Perl modules that have
methods that return data.

> Have you thought about using some dump + restore approach involving some
> intermediate interchange format ?

        If you wanted to write a migrator that did that, you would be welcome
to do so, provided that it were well-architected and maintainable by us.

        -Max
--
http://www.everythingsolved.com/
Competent, Friendly Bugzilla and Perl Services. Everything Else, too.
-
To view or change your list settings, click here:
<http://bugzilla.org/cgi-bin/mj_wwwusr?user=lists@...>

[Max Kanat-Alexander]

On 10/26/2009 03:22 AM, Gervase Markham wrote:
> Do we then have an install of GNATS on Landfill that we are planning to
> use to test the migrator with?

        We don't, currently. I have the client's database that I used to
develop the migrator with, but I don't think I can make that public.

> An obvious consequence of this is that someone could write a Bugzilla
> migrator, and you could use it to merge Bugzilla installations.

        Totally. Though it would require quite a bit more work on
Bugzilla::Migrate itself (and probably on Bugzilla itself) to make that
feasible.

>>     5) It is capable of doing a "dry-run" of your migration, so that you
>> can see if everything is going to go well before actually committing
>> anything to the database.
>
> That's particularly brilliant.

        Thanks!! :-)

        -Max
--
http://www.everythingsolved.com/
Competent, Friendly Bugzilla and Perl Services. Everything Else, too.
-
To view or change your list settings, click here:
<http://bugzilla.org/cgi-bin/mj_wwwusr?user=lists@...>