Savannah backup strategy

[Sylvain Beucler-2]

Hi,

I'd like to start a discussion about the Savannah backup.

Currently Savannah2004 is a single filesystem that is rsync'd in a
single pass. That synchro then goes to magnetic tapes, and some of
them are stored forever.  (at least that's what I understand)

Since we're moving to Xen, the data are going to be split in multiple
LVs. How do you see the new backup setup? Some DomUs are external and
be directly accessed through the network, but some others are
internal.

(Currently I implemented a backup from savannah-backup.gnu.org but
there's no backup history.  As previously mentioned it's based on LVM
snapshots, and it done per-domU/VServer rather than in single-pass).

--
Sylvain

[Sylvain Beucler-2]

On Tue, Nov 10, 2009 at 11:19:29AM -0500, Daniel Clark via RT wrote:

> > [beuc - Sat Oct 31 18:06:20 2009]:
> >
> > Hi,
> >
> > I'd like to start a discussion about the Savannah backup.
> >
> > Currently Savannah2004 is a single filesystem that is rsync'd in a
> > single pass. That synchro then goes to magnetic tapes, and some of
> > them are stored forever.  (at least that's what I understand)
> >
> > Since we're moving to Xen, the data are going to be split in multiple
> > LVs. How do you see the new backup setup? Some DomUs are external and
> > be directly accessed through the network, but some others are
> > internal.
> >
> > (Currently I implemented a backup from savannah-backup.gnu.org but
> > there's no backup history.  As previously mentioned it's based on LVM
> > snapshots, and it done per-domU/VServer rather than in single-pass).
>
> Your plan to get stuff to savannah-backup via lvm snapshots on the
> savannah dom0 makes sense to us.
>
> We would suggest that we then backup only savannah-backup, not savannah
> directly, to our tape system - that way the transition from savannah to
> cloud9 should be nearly unnoticeable to us, and we won't be hitting
> production savannah disk for backups more then we need to.
>
> Does that sound good to you?

That sounds good.

Currently the backup is done at 12:00GMT and needs around 1h to work -
we should try to make this backup and your backup at different times.

Currently the data is in /mnt/backup - let me know if that's a
problem.

I think it would be good to backup the whole system, because the DNS
server is hosted there too.

--
Sylvain

[Sylvain Beucler-2]

On Thu, Nov 12, 2009 at 12:41:38AM +0100, Sylvain Beucler wrote:

> On Tue, Nov 10, 2009 at 11:19:29AM -0500, Daniel Clark via RT wrote:
> > > [beuc - Sat Oct 31 18:06:20 2009]:
> > >
> > > Hi,
> > >
> > > I'd like to start a discussion about the Savannah backup.
> > >
> > > Currently Savannah2004 is a single filesystem that is rsync'd in a
> > > single pass. That synchro then goes to magnetic tapes, and some of
> > > them are stored forever.  (at least that's what I understand)
> > >
> > > Since we're moving to Xen, the data are going to be split in multiple
> > > LVs. How do you see the new backup setup? Some DomUs are external and
> > > be directly accessed through the network, but some others are
> > > internal.
> > >
> > > (Currently I implemented a backup from savannah-backup.gnu.org but
> > > there's no backup history.  As previously mentioned it's based on LVM
> > > snapshots, and it done per-domU/VServer rather than in single-pass).
> >
> > Your plan to get stuff to savannah-backup via lvm snapshots on the
> > savannah dom0 makes sense to us.
> >
> > We would suggest that we then backup only savannah-backup, not savannah
> > directly, to our tape system - that way the transition from savannah to
> > cloud9 should be nearly unnoticeable to us, and we won't be hitting
> > production savannah disk for backups more then we need to.
> >
> > Does that sound good to you?
>
> That sounds good.
>
> Currently the backup is done at 12:00GMT and needs around 1h to work -
> we should try to make this backup and your backup at different times.
>
> Currently the data is in /mnt/backup - let me know if that's a
> problem.
>
> I think it would be good to backup the whole system, because the DNS
> server is hosted there too.

Is the new backup currently operational?

If not, our last external backup of all downloads/audio-video/bazaar
is 1 month old, so we should raise the priority of this issue.

--
Sylvain

[Ward Vandewege via RT]

FYI the first savannah-backup.gnu.org backup is still plodding along (at about 500kb/sec).

BTW we were promised a donation of rack space and bandwidth by MIT CSAIL last week, so this should start
sucking substantially less sometime next year.

[Sylvain Beucler-2]

On Mon, Dec 14, 2009 at 02:08:27PM -0500, Daniel Clark via RT wrote:

> The first savannah-backup backup finished:
>
> Number of files: 2099942
> Number of files transferred: 1843832
> Total file size: 123259679448 bytes
> Total transferred file size: 122393771613 bytes
> Literal data: 122469455639 bytes
> Matched data: 0 bytes
> File list size: 47007178
> File list generation time: 628.366 seconds
> File list transfer time: 0.000 seconds
> Total bytes sent: 36165433
> Total bytes received: 122610448822
>
> sent 36165433 bytes  received 122610448822 bytes  358676.13 bytes/sec
> total size is 123259679448  speedup is 1.00
> rsync warning: some files vanished before they could be transferred (code 24) at main.c(1524)
> [generator=3.0.4]
> rsync of savannah-backup.gnu.org finished at Sat Dec 12 10:53:15 EST 2009
>
> I've now enabled savannah-backup in cron, and disabled savannah.

Thanks!


I have 2 comments:

- Currently the first (local) backup is ran at noon (12:00) _GMT_ ,
  and takes less than 2h to complete.

  For your backup to be clean, is it possible to start it at 3PM GMT
  (so that both backups don't run at once) ?

- I wonder what files were excluded (25K files / 8GB), since files are
  normally already excluded with the first backup.  Would it be
  possible to give me a temporary access to your backup so I can
  inspect it?

--
Sylvain