Re: svn dump format

[Syed Uzair Aqeel]

Hello,

I'm looking for a description of the format used by Subversion's dumpfiles.
I haven't found very much documentation on it, but from what I've seen in
dumpfiles from my own repository, it looks something like this:

1) dumpfile format version
2) repository UUID (if available)
3) Revision headers in RFC-822 header format (ie, log message, author, date,
revision #)

After that, each node in the repository is dumped with roughly the following
format:
a) node path, type, action in RFC-822 header format
b) integer lengths for content and properties, and a checksum for the
content in RFC-822 header format
c) key-value pairs for each property, as below, where the numbers denote the
length of the field value
<blockquote>
K 14
svn:executable
V 1
*
</blockquote>
d) the 'PROPS-END' delimiter
e) node content, if any


Am I missing anything? Is there a format design document I can consult? I'm
a little worried about how particularly the parser is about whitespaces
etc., particularly between node dumps. For the root directory (ie,
'Node-path: '), I've also been getting 'file already exists' messages when
loading into the repository. I've checked to make sure the node-path isn't
repeated elsewhere in the dumpfile, and it isn't.

Any pointers much appreciated. Thanks,

Uzair

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...

[Ben Collins-Sussman-3]

On 8/10/06, Syed Uzair Aqeel <uzair@...> wrote:

> Am I missing anything? Is there a format design document I can consult?

http://svn.collab.net/repos/svn/trunk/notes/fs_dumprestore.txt

Be warned of two things:

1. it looks like a text format, but it's really a binary format.

2. in svn_repos.h, there are already public APIs you can call to
produce or consume a dumpfile.  You probably don't want to reinvent
this wheel.

(What are you doing, anyway?)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...

[Syed Uzair Aqeel]

I've written a Clearcase-to-Subversion dumper.

Since the versioning philosophies don't match exactly (individual-files vs.
repository-wide-changesets), I decided the best way to do it would be to
dump Clearcase labels (which represent changesets as well), and to convert
them as dumpfiles rather than simply importing them into Subversion, for
maximum flexibility. The downside is that this requires more disk space,
since I'm not doing delta-based dumps (ie, nothing like the '--incremental'
flag here).

When I tried it yesterday, it seemed to work pretty well. I produced 8.54GB
worth of dumpfiles (255 revisions) in 128 minutes, and loading them into my
SVN repository went through OK (no numbers there yet). I'm going to be doing
more tests today.

Anyway, cheers for the link, Ben, I'll check to make sure I'm doing things
right.

Uzair

> -----Original Message-----
> From: sussman@... [mailto:sussman@...] On Behalf Of Ben
> Collins-Sussman
> Sent: Saturday, August 12, 2006 8:33 PM
> To: Syed Uzair Aqeel
> Cc: dev@...
> Subject: Re: svn dump format
>
> On 8/10/06, Syed Uzair Aqeel <uzair@...> wrote:
>
> > Am I missing anything? Is there a format design document I can consult?
>
> http://svn.collab.net/repos/svn/trunk/notes/fs_dumprestore.txt
>
> Be warned of two things:
>
> 1. it looks like a text format, but it's really a binary format.
>
> 2. in svn_repos.h, there are already public APIs you can call to
> produce or consume a dumpfile.  You probably don't want to reinvent
> this wheel.
>
> (What are you doing, anyway?)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...

[Julian Foad]

Ben Collins-Sussman wrote:
> http://svn.collab.net/repos/svn/trunk/notes/fs_dumprestore.txt
>
> Be warned of two things:
>
> 1. it looks like a text format, but it's really a binary format.

It would seem a good idea to add that statement to the document; would you?

- Julian

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...

[Ben Collins-Sussman-3]

On 8/23/06, Julian Foad <julianfoad@...> wrote:
> Ben Collins-Sussman wrote:
> > http://svn.collab.net/repos/svn/trunk/notes/fs_dumprestore.txt
> >
> > Be warned of two things:
> >
> > 1. it looks like a text format, but it's really a binary format.
>
> It would seem a good idea to add that statement to the document; would you?
>

Definitely, go for it!

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...

[Daniel Rall-2]

On Wed, 23 Aug 2006, Ben Collins-Sussman wrote:

> On 8/23/06, Julian Foad <julianfoad@...> wrote:
> >Ben Collins-Sussman wrote:
> >> http://svn.collab.net/repos/svn/trunk/notes/fs_dumprestore.txt
> >>
> >> Be warned of two things:
> >>
> >> 1. it looks like a text format, but it's really a binary format.
> >
> >It would seem a good idea to add that statement to the document; would you?
> >
>
> Definitely, go for it!

I threw a word in in r21353.