performance enhancement by working copy svn server

View: New views
20 Messages — Rating Filter:   Alert me  
< Prev | 1 - 2 - 3 - 4 | Next >

performance enhancement by working copy svn server

by Edward Ned Harvey-5 :: Rate this Message:

| View Threaded | Show Only this Message

Basically, just wondering if anyone has ever thought along these lines ...

I was thinking about how to speed up operations, such as svn status, etc, which are currently required to walk the tree.  With large numbers of files, this can be expensive.  The root cause is simple - peoples files can be changed by any process without the knowledge of svn; therefore svn must inspect each file to see if it's changed.  To improve this would require some process to have the ability to constantly monitor all file access within the working copy.

So the idea is - Suppose there were a special "working copy" svn server, which could have exclusive access to some folder on the system, containing the working copy, and then export those files to localhost, where they could be mounted into a user's home directory (or wherever).  The user would work in his/her home directory as always, but the "working copy svn server" would be able to monitor all file access.  No change of behavior would be necessary; the working copy could contain versioned and unversioned files exactly as it always has, but the "working copy svn server" could keep track of all the local modifications on the fly, and eliminate the need to ever walk the tree.

Granted, root would have to assist somehow, to let the filesystem mount.  And I'm sure it's not a trivial code to write.

This e-mail message may contain proprietary, confidential or legally privileged information for the sole use of the person or entity to whom this message was originally addressed. Any review, e-transmission dissemination or other use of or taking of any action in reliance upon this information by persons or entities other than the intended recipient is prohibited. If you have received this e-mail in error kindly delete  this e-mail from your records. If it appears that this mail has been forwarded to you without proper authority, please notify us immediately at netadmin@... and delete this mail.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...


Re: performance enhancement by working copy svn server

by Greg Hudson :: Rate this Message:

| View Threaded | Show Only this Message

As I understand it, Perforce does this, although I believe it requires
the file server to be hosted by the repository.  It is, in fact, quite
fast as long as the mount is going over a fast link.  It's a bit
unlikely that Subversion will ever go in this direction since it
requires a lot of system-dependent code and/or documentation.

Another possibile way to avoid scans is to require an "svn edit" (actual
syntax may vary) to make Subversion notice an edit to a file.  This idea
is most attractive to people who have Subversion integrated with their
editor or IDE; since not everyone is in that camp, it would probably
have to be a per-checkout option.  Peering at trunk/notes/wc-ng-design,
it looks like Subversion might conceivably go in this direction if a WC
rewrite effort succeeds.

On Sat, 2008-04-05 at 19:28 -0400, Harvey, Edward wrote:

> Basically, just wondering if anyone has ever thought along these lines ...
>
> I was thinking about how to speed up operations, such as svn status, etc, which are currently required to walk the tree.  With large numbers of files, this can be expensive.  The root cause is simple - peoples files can be changed by any process without the knowledge of svn; therefore svn must inspect each file to see if it's changed.  To improve this would require some process to have the ability to constantly monitor all file access within the working copy.
>
> So the idea is - Suppose there were a special "working copy" svn server, which could have exclusive access to some folder on the system, containing the working copy, and then export those files to localhost, where they could be mounted into a user's home directory (or wherever).  The user would work in his/her home directory as always, but the "working copy svn server" would be able to monitor all file access.  No change of behavior would be necessary; the working copy could contain versioned and unversioned files exactly as it always has, but the "working copy svn server" could keep track of all the local modifications on the fly, and eliminate the need to ever walk the tree.
>
> Granted, root would have to assist somehow, to let the filesystem mount.  And I'm sure it's not a trivial code to write.
>
> This e-mail message may contain proprietary, confidential or legally privileged information for the sole use of the person or entity to whom this message was originally addressed. Any review, e-transmission dissemination or other use of or taking of any action in reliance upon this information by persons or entities other than the intended recipient is prohibited. If you have received this e-mail in error kindly delete  this e-mail from your records. If it appears that this mail has been forwarded to you without proper authority, please notify us immediately at netadmin@... and delete this mail.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@...
> For additional commands, e-mail: dev-help@...
>


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...


Re: performance enhancement by working copy svn server

by David O'Shea-3 :: Rate this Message:

| View Threaded | Show Only this Message

This is similar to how Clearcase works also. Something like inotify
would be helpful too I guess, but that would be linux only.

David.

On 06/04/2008 05:30, Greg Hudson wrote:

> As I understand it, Perforce does this, although I believe it requires
> the file server to be hosted by the repository.  It is, in fact, quite
> fast as long as the mount is going over a fast link.  It's a bit
> unlikely that Subversion will ever go in this direction since it
> requires a lot of system-dependent code and/or documentation.
>
> Another possibile way to avoid scans is to require an "svn edit" (actual
> syntax may vary) to make Subversion notice an edit to a file.  This idea
> is most attractive to people who have Subversion integrated with their
> editor or IDE; since not everyone is in that camp, it would probably
> have to be a per-checkout option.  Peering at trunk/notes/wc-ng-design,
> it looks like Subversion might conceivably go in this direction if a WC
> rewrite effort succeeds.
>
> On Sat, 2008-04-05 at 19:28 -0400, Harvey, Edward wrote:
>> Basically, just wondering if anyone has ever thought along these lines ...
>>
>> I was thinking about how to speed up operations, such as svn status, etc, which are currently required to walk the tree.  With large numbers of files, this can be expensive.  The root cause is simple - peoples files can be changed by any process without the knowledge of svn; therefore svn must inspect each file to see if it's changed.  To improve this would require some process to have the ability to constantly monitor all file access within the working copy.
>>
>> So the idea is - Suppose there were a special "working copy" svn server, which could have exclusive access to some folder on the system, containing the working copy, and then export those files to localhost, where they could be mounted into a user's home directory (or wherever).  The user would work in his/her home directory as always, but the "working copy svn server" would be able to monitor all file access.  No change of behavior would be necessary; the working copy could contain versioned and unversioned files exactly as it always has, but the "working copy svn server" could keep track of all the local modifications on the fly, and eliminate the need to ever walk the tree.
>>
>> Granted, root would have to assist somehow, to let the filesystem mount.  And I'm sure it's not a trivial code to write.
>>
>> This e-mail message may contain proprietary, confidential or legally privileged information for the sole use of the person or entity to whom this message was originally addressed. Any review, e-transmission dissemination or other use of or taking of any action in reliance upon this information by persons or entities other than the intended recipient is prohibited. If you have received this e-mail in error kindly delete  this e-mail from your records. If it appears that this mail has been forwarded to you without proper authority, please notify us immediately at netadmin@... and delete this mail.
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@...
>> For additional commands, e-mail: dev-help@...
>>
>
>
> ----------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@...
> For additional commands, e-mail: dev-help@...
>

--

The information contained in this e-mail and in any attachments is confidential and is designated solely for the attention of the intended recipient(s). If you are not an intended recipient, you must not use, disclose, copy, distribute or retain this e-mail or any part thereof. If you have received this e-mail in error, please notify the sender by return e-mail and delete all copies of this e-mail from your computer system(s).
Please direct any additional queries to: communications@....
Thank You.
Silicon and Software Systems Limited. Registered in Ireland no. 378073.
Registered Office: South County Business Park, Leopardstown, Dublin 18

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...


Re: performance enhancement by working copy svn server

by jrep :: Rate this Message:

| View Threaded | Show Only this Message


On Apr 5, 2008, at 4:28 PM, Harvey, Edward wrote:

> So the idea is - Suppose there were a special "working copy" svn  
> server, which could have exclusive access to some folder on the  
> system, containing the working copy, and then export those files to  
> localhost, where they could be mounted into a user's home directory  
> (or wherever).  The user would work in his/her home directory as  
> always, but the "working copy svn server" would be able to monitor  
> all file access.  No change of behavior would be necessary; the  
> working copy could contain versioned and unversioned files exactly  
> as it always has, but the "working copy svn server" could keep track  
> of all the local modifications on the fly, and eliminate the need to  
> ever walk the tree.


FUSE is a fairly portable (Linux, Mac OS X) basis for such a thing:
   http://fuse.sourceforge.net/

Chris Frost has worked on serving Subversion working copies out of a  
FUSE volume, in his "scord" project:
   http://scord.sourceforge.net/

His goal is different from yours: to reduce the disc storage costs of  
the pristine copies. But he's inserted himself into the file system in  
the right spot to do these other things as well.

-==-
Jack Repenning
Chief Technology Officer
CollabNet, Inc.
8000 Marina Boulevard, Suite 600
Brisbane, California 94005
office: +1 650.228.2562
mobile: +1 408.835.8090
raindance: +1 877.326.2337, x844.7461
aim: jackrepenning
skype: jackrepenning






---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...

jr

Re: performance enhancement by working copy svn server

by Chris Frost :: Rate this Message:

| View Threaded | Show Only this Message

On Mon, Apr 07, 2008 at 01:09:23PM -0700, Jack Repenning wrote:

>  FUSE is a fairly portable (Linux, Mac OS X) basis for such a thing:
>    http://fuse.sourceforge.net/
>
>  Chris Frost has worked on serving Subversion working copies out of a FUSE
>  volume, in his "scord" project:
>    http://scord.sourceforge.net/
>
>  His goal is different from yours: to reduce the disc storage costs of the
>  pristine copies. But he's inserted himself into the file system in the right
>  spot to do these other things as well.

Fwiw, I see two relevant parts to the speed of the working copy operations
status, diff, etc:
1) read the necessary data from disk
2) traverse the in-memory file system
Part 1 happens to be what I notice most as limiting my svn use. However, it
sounds like only part 2 is a limitation in your situation. (And you could,
of course, aim to improve both parts.)

Routing working copy access through a FUSE file system would definitely be a
way to go. Fwiw, I think the main three limitations to watch out for would be:
- The FUSE API only works on Linux, MacOSX, and BSD
- Serving large data chunks or running many filesystem system calls via
  FUSE incurs a significant increase in the number of kernel-user crossings,
  data copy operations, and kernel file system cache memory used
- Supporting multiple users with a FUSE server instance might be hard

And easier, but still something to think about, is server/os crash recovery.
I suspect you might make it safe on file systems like ext3 in its default
journaling mode, ordered journaling, and not safe on other file systems,
as the working copy library does itself. Fwiw, I have considered augmenting
the underlying working copy representation that scord itself stores, but
each time decided that the benefit didn't justify the synchronization work.
But these issues might work differently for this scenario.

A David mentioned, file notification mechanisms, like at least Linux and OSX
have, would be another route. This would mostly-to-completely avoid the speed,
multi-user, and safety FUSE limitations.

--
Chris Frost  |  <http://www.frostnet.net/chris/>
-------------+----------------------------------
PGP: <http://www.frostnet.net/chris/pgpkey.txt>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...


Re: performance enhancement by working copy svn server

by Ben Collins-Sussman-3 :: Rate this Message:

| View Threaded | Show Only this Message

As Greg Hudson mentioned, a number of us are really quite enthusiastic
about a libsvn_wc rewrite which (1) centralizes metadata in one place
and (2) allows users the option of emulating perforce -- 'svn edit' to
tell svn what you're changing, so that 'svn status/update/commit' no
longer need to traverse the tree.

We're simply laying low, waiting for svn 1.5 to finish.  When 1.5 is
out, I encourage you to participate in this design.  I think it's a
better bet than digging into FUSE-like things.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...


RE: performance enhancement by working copy svn server

by Oren Eini (Murphy & Associates) :: Rate this Message:

| View Threaded | Show Only this Message

From my perspective, I really like SVN's ability to figure what changed on its own, without having to be told.
And having the metadata localized allows to do wierd things like manually merge WC temporarily, or easily take a smaller piece of the WC and move it elsewhere.

________________________________________
From: sussman@... [sussman@...] On Behalf Of Ben Collins-Sussman [sussman@...]
Sent: Wednesday, April 09, 2008 9:16 AM
To: Chris Frost; Edward.Harvney@...
Cc: dev@...
Subject: Re: performance enhancement by working copy svn server

As Greg Hudson mentioned, a number of us are really quite enthusiastic
about a libsvn_wc rewrite which (1) centralizes metadata in one place
and (2) allows users the option of emulating perforce -- 'svn edit' to
tell svn what you're changing, so that 'svn status/update/commit' no
longer need to traverse the tree.

We're simply laying low, waiting for svn 1.5 to finish.  When 1.5 is
out, I encourage you to participate in this design.  I think it's a
better bet than digging into FUSE-like things.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...


Re: performance enhancement by working copy svn server

by Eric Gillespie :: Rate this Message:

| View Threaded | Show Only this Message

"Ben Collins-Sussman" <sussman@...> writes:

> As Greg Hudson mentioned, a number of us are really quite enthusiastic
> about a libsvn_wc rewrite which (1) centralizes metadata in one place
> and (2) allows users the option of emulating perforce -- 'svn edit' to

Emulating Perforce isn't something anyone sane wants to do.  In
this specific example, 'svn edit' will be a pure-local operation,
and will never be held up by some lock or high load or whatever
on the server.  I know you know this, but I couldn't help
reacting to this "emulating perforce" ;->.

--
Eric Gillespie <*> epg@...

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...


Re: performance enhancement by working copy svn server

by Ben Collins-Sussman-3 :: Rate this Message:

| View Threaded | Show Only this Message

On Wed, Apr 9, 2008 at 12:23 PM, Oren Eini (Murphy & Associates)
<v-orene@...> wrote:
> From my perspective, I really like SVN's ability to figure what changed on its own, without having to be told.

So do we all.  That's why we've always talked about making the 'svn
edit' model into an *optional* mode of working.  Try using subversion
with a 700MB source tree;  running 'svn status' takes minutes to crawl
the tree (along with commits and updates).

>  And having the metadata localized allows to do wierd things like manually merge WC temporarily, or easily take a smaller piece of the WC and move it elsewhere.

That's what we've been telling ourselves for 8 years;  it turns out
that nobody actually ever breaks working copies into small portable
pieces.  And the price we  pay for scattered metadata is huge.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...


RE: performance enhancement by working copy svn server

by Oren Eini (Murphy & Associates) :: Rate this Message:

| View Threaded | Show Only this Message

Um, do I count as someone who actually breaks working copies?
That is not a highly used feature, I'll admit, and not one that I'll particulary miss.

Thanks for clarifying on its optionality.
________________________________________
From: sussman@... [sussman@...] On Behalf Of Ben Collins-Sussman [sussman@...]
Sent: Wednesday, April 09, 2008 3:38 PM
To: Oren Eini (Murphy & Associates)
Cc: Chris Frost; Edward.Harvney@...; dev@...
Subject: Re: performance enhancement by working copy svn server

On Wed, Apr 9, 2008 at 12:23 PM, Oren Eini (Murphy & Associates)
<v-orene@...> wrote:
> From my perspective, I really like SVN's ability to figure what changed on its own, without having to be told.

So do we all.  That's why we've always talked about making the 'svn
edit' model into an *optional* mode of working.  Try using subversion
with a 700MB source tree;  running 'svn status' takes minutes to crawl
the tree (along with commits and updates).

>  And having the metadata localized allows to do wierd things like manually merge WC temporarily, or easily take a smaller piece of the WC and move it elsewhere.

That's what we've been telling ourselves for 8 years;  it turns out
that nobody actually ever breaks working copies into small portable
pieces.  And the price we  pay for scattered metadata is huge.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...


Re: performance enhancement by working copy svn server

by Ph. Marek :: Rate this Message:

| View Threaded | Show Only this Message

On Donnerstag, 10. April 2008, Ben Collins-Sussman wrote:

> Try using subversion
> with a 700MB source tree;  running 'svn status' takes minutes to crawl
> the tree (along with commits and updates).
>
> >  And having the metadata localized allows to do wierd things like
> > manually merge WC temporarily, or easily take a smaller piece of the WC
> > and move it elsewhere.
>
> That's what we've been telling ourselves for 8 years;  it turns out
> that nobody actually ever breaks working copies into small portable
> pieces.  And the price we  pay for scattered metadata is huge.
I just want to chime in with some hard data for the interested: FSVS uses a
single file for most of the needed data (all that's needed for "status"), and
it's bit faster.

See here for an example:
http://fsvs.tigris.org/source/browse/fsvs/trunk/fsvs/doc/PERFORMANCE?rev=972&view=markup

This is when a single file (sorted by inode-number) is read, the entries
lstat()ed in inode order, and no keyword substitution is possible (so that a
size change is immediately seen as changed data).


Regards,

Phil

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...


Re: performance enhancement by working copy svn server

by Hyrum K. Wright-3 :: Rate this Message:

| View Threaded | Show Only this Message

Oren Eini (Murphy & Associates) wrote:
> Um, do I count as someone who actually breaks working copies?
> That is not a highly used feature, I'll admit, and not one that I'll particulary miss.

We've also kicked around the idea for 'svn sever', which would let you
detach a part of a working copy.  So the operation would still be
supported, it would just take one more step.

(Oh, and just a reminder that we bottom post on this list.  Not a big
deal, but it does help to the flow of the conversation.)

-Hyrum

> Thanks for clarifying on its optionality.
> ________________________________________
> From: sussman@... [sussman@...] On Behalf Of Ben Collins-Sussman [sussman@...]
> Sent: Wednesday, April 09, 2008 3:38 PM
> To: Oren Eini (Murphy & Associates)
> Cc: Chris Frost; Edward.Harvney@...; dev@...
> Subject: Re: performance enhancement by working copy svn server
>
> On Wed, Apr 9, 2008 at 12:23 PM, Oren Eini (Murphy & Associates)
> <v-orene@...> wrote:
>> From my perspective, I really like SVN's ability to figure what changed on its own, without having to be told.
>
> So do we all.  That's why we've always talked about making the 'svn
> edit' model into an *optional* mode of working.  Try using subversion
> with a 700MB source tree;  running 'svn status' takes minutes to crawl
> the tree (along with commits and updates).
>
>>  And having the metadata localized allows to do wierd things like manually merge WC temporarily, or easily take a smaller piece of the WC and move it elsewhere.
>
> That's what we've been telling ourselves for 8 years;  it turns out
> that nobody actually ever breaks working copies into small portable
> pieces.  And the price we  pay for scattered metadata is huge.


signature.asc (265 bytes) Download Attachment

RE: performance enhancement by working copy svn server

by Edward Ned Harvey-5 :: Rate this Message:

| View Threaded | Show Only this Message

> libsvn_wc rewrite which
> (1) centralizes metadata in one place and
> (2) allows users the option of 'svn edit' to tell svn what you're
> changing, so that 'svn status/update/commit' no longer need
> to traverse the tree.

With the wc rewrite, are you talking about somehow preventing extraneous processes, such as "vi somefile" from accessing files in a WC "behind the back" of the svn client?  Or just telling users, "don't do that" and assuming during the stat/up/commit that they've behaved properly?

The ideal, which may be very difficult to achieve, is the ability to monitor FS access, to automatically "notice" a file changed on the fly.  This extra FS overhead is analogous to journaling, where you spread out the fsck overhead during normal operation, instead of letting it all pile up into one big operation.

This e-mail message may contain proprietary, confidential or legally privileged information for the sole use of the person or entity to whom this message was originally addressed. Any review, e-transmission dissemination or other use of or taking of any action in reliance upon this information by persons or entities other than the intended recipient is prohibited. If you have received this e-mail in error kindly delete  this e-mail from your records. If it appears that this mail has been forwarded to you without proper authority, please notify us immediately at netadmin@... and delete this mail.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...


Re: performance enhancement by working copy svn server

by Ben Collins-Sussman-3 :: Rate this Message:

| View Threaded | Show Only this Message

On Thu, Apr 10, 2008 at 10:01 AM, Harvey, Edward
<Edward.Harvey@...> wrote:
> > libsvn_wc rewrite which
>  > (1) centralizes metadata in one place and
>  > (2) allows users the option of 'svn edit' to tell svn what you're
>
> > changing, so that 'svn status/update/commit' no longer need
>  > to traverse the tree.
>
>  With the wc rewrite, are you talking about somehow preventing extraneous processes, such as "vi somefile" from accessing files in a WC "behind the back" of the svn client?

Just like perforce, the whole working copy would have read-only
permissions set by default;  'svn edit' would make a file read-write.

Yes, this system is imperfect, but it works pretty well in practice in
conjunction with user education.  We already have this behavior when
the 'svn:needs-lock' property is attached to the file... we'd just be
generalizing the behavior to cover the whole working copy.    (Some
naughty tools ignore read-only permissions, but typcially not text
editors.)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...


Re: performance enhancement by working copy svn server

by David Glasser-4 :: Rate this Message:

| View Threaded | Show Only this Message

On Thu, Apr 10, 2008 at 8:22 AM, Ben Collins-Sussman
<sussman@...> wrote:

> On Thu, Apr 10, 2008 at 10:01 AM, Harvey, Edward
>  <Edward.Harvey@...> wrote:
>  > > libsvn_wc rewrite which
>  >  > (1) centralizes metadata in one place and
>  >  > (2) allows users the option of 'svn edit' to tell svn what you're
>  >
>  > > changing, so that 'svn status/update/commit' no longer need
>  >  > to traverse the tree.
>  >
>  >  With the wc rewrite, are you talking about somehow preventing extraneous processes, such as "vi somefile" from accessing files in a WC "behind the back" of the svn client?
>
>  Just like perforce, the whole working copy would have read-only
>  permissions set by default;  'svn edit' would make a file read-write.
>
>  Yes, this system is imperfect, but it works pretty well in practice in
>  conjunction with user education.  We already have this behavior when
>  the 'svn:needs-lock' property is attached to the file... we'd just be
>  generalizing the behavior to cover the whole working copy.    (Some
>  naughty tools ignore read-only permissions, but typcially not text
>  editors.)

By the way, a nice thing about this model that epg pointed out
recently is that this mode meshes well with optional text-base-free
operation.  You can get away without storing a separate text-base for
the read-only files; 'svn edit' would add a copy of the text-base to
the metadata area before making the file writable.

--dave

--
David Glasser | glasser@... | http://www.davidglasser.net/

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...


Re: performance enhancement by working copy svn server

by Mattias Engdegård :: Rate this Message:

| View Threaded | Show Only this Message

"Ben Collins-Sussman" <sussman@...> writes:

>So do we all.  That's why we've always talked about making the 'svn
>edit' model into an *optional* mode of working.  Try using subversion
>with a 700MB source tree;  running 'svn status' takes minutes to crawl
>the tree (along with commits and updates).

No it doesn't, at least not with the WC on a decent local file system
and there is RAM to cache enough of it - it is usually a matter of
seconds or less once heated up. (I frequently develop on tmpfs because
of this, but ext3 works well too.)

But of course I agree, the current WC organisation scatters
information in lots and lots of files and directories and with very
expensive locking; there is a lot to gain by saner arrangements.

Of course, for those stuck with a slow file system (NTFS, UFS, NFS
etc), even a serious WC reorganisation may not help enough. This
remains to be seen - an 'svn edit' option would feel like admitting
defeat.



---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...


Re: performance enhancement by working copy svn server

by David Summers :: Rate this Message:

| View Threaded | Show Only this Message


On Thu, 10 Apr 2008, Mattias Engdegård wrote:

> "Ben Collins-Sussman" <sussman@...> writes:
>
>> So do we all.  That's why we've always talked about making the 'svn
>> edit' model into an *optional* mode of working.  Try using subversion
>> with a 700MB source tree;  running 'svn status' takes minutes to crawl
>> the tree (along with commits and updates).
>
> No it doesn't, at least not with the WC on a decent local file system
> and there is RAM to cache enough of it - it is usually a matter of
> seconds or less once heated up. (I frequently develop on tmpfs because
> of this, but ext3 works well too.)
>
> But of course I agree, the current WC organisation scatters
> information in lots and lots of files and directories and with very
> expensive locking; there is a lot to gain by saner arrangements.
>
> Of course, for those stuck with a slow file system (NTFS, UFS, NFS
> etc), even a serious WC reorganisation may not help enough. This
> remains to be seen - an 'svn edit' option would feel like admitting
> defeat.
>
Yes, I agree it would "feel like admitting defeat".  I have no problem
with it if it is an option.

Just as a data point:

I work with a 6 Gig working copy (14 Gig repository) and can do status and
commit and update operations in seconds on Win XP SP2 with 2 Gig of RAM and
NTFS.

--
David Wayne Summers        "Linux: Because reboots are for hardware upgrades!"
david@... PGP Key: http://summersoft.fay.ar.us/~david/pgp.txt
PGP Key fingerprint =  0B44 B118 85CC F4EC 7021  1ED4 1516 5B78 E320 2001

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...

RE: performance enhancement by working copy svn server

by Oren Eini (Murphy & Associates) :: Rate this Message:

| View Threaded | Show Only this Message

With or without svn edit?
________________________________________
From: David Summers [david@...]
Sent: Thursday, April 10, 2008 10:36 AM
To: Mattias Engdegård
Cc: dev@...
Subject: Re: performance enhancement by working copy svn server


I work with a 6 Gig working copy (14 Gig repository) and can do status and
commit and update operations in seconds on Win XP SP2 with 2 Gig of RAM and
NTFS.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...


RE: performance enhancement by working copy svn server

by David Summers :: Rate this Message:

| View Threaded | Show Only this Message


Ummm, we don't have "svn edit" currently, so it would have to be without
it.  :-)

(Or am I missing someting?)

    - David

On Thu, 10 Apr 2008, Oren Eini (Murphy & Associates) wrote:

> With or without svn edit?
> ________________________________________
> From: David Summers [david@...]
> Sent: Thursday, April 10, 2008 10:36 AM
> To: Mattias Engdegård
> Cc: dev@...
> Subject: Re: performance enhancement by working copy svn server
>
>
> I work with a 6 Gig working copy (14 Gig repository) and can do status and
> commit and update operations in seconds on Win XP SP2 with 2 Gig of RAM and
> NTFS.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@...
> For additional commands, e-mail: dev-help@...
>
>
--
David Wayne Summers        "Linux: Because reboots are for hardware upgrades!"
david@... PGP Key: http://summersoft.fay.ar.us/~david/pgp.txt
PGP Key fingerprint =  0B44 B118 85CC F4EC 7021  1ED4 1516 5B78 E320 2001

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...

Re: performance enhancement by working copy svn server

by Peter Wemm-2 :: Rate this Message:

| View Threaded | Show Only this Message

On Wed, Apr 9, 2008 at 3:41 PM, Oren Eini (Murphy & Associates)
<v-orene@...> wrote:
> Um, do I count as someone who actually breaks working copies?
>  That is not a highly used feature, I'll admit, and not one that I'll particulary miss.
>
>  Thanks for clarifying on its optionality.

It's also something we use in freebsd.org fairly extensively in cvs.
svn's ability to cope with that sort abuse was one of the things in
its favour when we were looking at svn vs git vs hg.  (We're in the
process of doing a trial conversion right now before we commit. We've
still got some huge problems with svn to solve.)

The main things we do are partial subtree checkouts (eg: check out
src/sys or or src/usr.bin/ls or all of src) and moving subtrees around
(eg: mv src/sys src/sys.wip).  We also have people who merge
independent checkouts together.  eg: checkout -d foo src/sys; edit
foo; mv src/sys /src/sys.clean; mv foo src/sys.

Nobody from our camp would mind having to issue commands to inform svn
of the evil that we've just done behind its back, just so long as it
is possible somehow to do it.  Be it commands in the CLI, or scripts
using API calls to massage the metadata, or whatever.

--
Peter Wemm - peter@...; peter@...; peter@...
"All of this is for nothing if we don't go to the stars" - JMS/B5
"If Java had true garbage collection, most programs would delete
themselves upon execution." -- Robert Sewell

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...

< Prev | 1 - 2 - 3 - 4 | Next >