Limitation of Mercurial with xx files or filesizes

[Gaunet Sylvain]

Hello,

I've found some informations on the mailing list and Internet about
Mercurial performances but I still got some questions.
Mercurial is running on a red hat enterprise 5 (4*3.6Ghz with 4 GB of
ram)

Here are my questions :
- From how many files should the performance severaly decrease ?
- What's the limit of file size which should decrease the performance ?

Actually, we don't fix a limit for the size of a project. We just use an
other tool if we have one or more files over 200MB.
To give you an example, we could have some projects : more than 100 000
files, 2.5 GB (containing sources and binary files).

Thanks


_______________________________________________
Mercurial mailing list
Mercurial@...
http://selenic.com/mailman/listinfo/mercurial

[Benoit Boissinot]

On Wed, Nov 04, 2009 at 04:11:45PM +0100, Gaunet Sylvain wrote:
> Hello,
>
> I've found some informations on the mailing list and Internet about
> Mercurial performances but I still got some questions.
> Mercurial is running on a red hat enterprise 5 (4*3.6Ghz with 4 GB of
> ram)
>
> Here are my questions :
> - From how many files should the performance severaly decrease ?

There isn't any fixed limit, it depends on the performance of the
clients computer (where you issue commands like status/commit/etc.).

There are projects with 100k files (netbeans is 96k).

In any case it's a good idea to split if the files are independant,
since it's easier to join repo (with forest or subrepo) than to split
them.
> - What's the limit of file size which should decrease the performance ?

Same here, usually hg needs in the order of O(biggest file) memory, so I
guess it doesn't matter for the server. But if you have clones running
on computer with less memory they could have some troubles.

>
> Actually, we don't fix a limit for the size of a project. We just use an
> other tool if we have one or more files over 200MB.

The bfiles extension is meant for that I think. And I think it's a good
idea to keep big binary files separate.

In addition, you need to think about the bandwidth needed for cloning,
some users might not be happy cloning a 2+GB tree...

regards,

Benoit

--
:wq
_______________________________________________
Mercurial mailing list
Mercurial@...
http://selenic.com/mailman/listinfo/mercurial

[Dirkjan Ochtman]

On Wed, Nov 4, 2009 at 16:51, Benoit Boissinot
<benoit.boissinot@...> wrote:
>> - What's the limit of file size which should decrease the performance ?
>
> Same here, usually hg needs in the order of O(biggest file) memory, so I
> guess it doesn't matter for the server. But if you have clones running
> on computer with less memory they could have some troubles.

To be a little more specific, I think hg keeps at least 2x file
contents in memory for diffing, so with 4GB machines you can probably
handle 200M files just fine.

Cheers,

Dirkjan
_______________________________________________
Mercurial mailing list
Mercurial@...
http://selenic.com/mailman/listinfo/mercurial

[Greg Ward-17]

On Wed, Nov 4, 2009 at 10:11 AM, Gaunet Sylvain
<sylvain.gaunet@...> wrote:
> Here are my questions :
> - From how many files should the performance severaly decrease ?
> - What's the limit of file size which should decrease the performance ?

You might want to see http://mercurial.selenic.com/wiki/BfilesExtension

In particular, my design.txt and usage.txt explain some of the
problems Mercurial has with large binary files (and why I wrote
bfiles).

Note that bfiles is under active development and currently makes
absolutely no guarantees of backwards compatibility in any regard.

Also: http://mercurial.selenic.com/wiki/HandlingLargeFiles .

Greg
_______________________________________________
Mercurial mailing list
Mercurial@...
http://selenic.com/mailman/listinfo/mercurial