cygwin copy problems usb 2.0

[Igor Peshansky]

On Thu, 3 Aug 2006, Corinna Vinschen wrote:

> On Aug  3 01:54, Eric Blake wrote:
> > > >I'm really seeing the non-optimized cygwin cp behaviour causing bad
> > > >reputation, which could be easily patched and maybe even accepted
> > > >upstream. Who knows. Eric what do think? Would it be worthful to think
> > > >about?
> >
> > I don't really want to maintain a Windows API patch, and doubt that
>
> Good.
>
> > it would be accepted upstream.  Now if there were something more
> > POSIX-y that we could do to speed things up, such as posix_fadvise,
>
> posix_fadvise can't be implemented nicely, AFAICS.  The POSIX semantics
> require an already opened file and the advice is given for an offset and
> a length.  The Windows semantics only allow to give the advice for the
> whole file, and only switching between FILE_SEQUENTIAL_ONLY or "normal",
> using ZwSetInformationFile.  By re-opening the file using ZwOpenFile it
> would also be possible to toggle the FILE_RANDOM_ACCESS flag.  Still,
> it's always for the whole file, not for an area giving offset and length.

Theoretically, it's possible to implement posix_fadvise only for offset=0
and length=<length-of-file>, and have it fail with EINVAL otherwise...
While technically not POSIX-compliant, it would still allow for better
implementation of things like copy...
        Igor
--
                                http://cs.nyu.edu/~pechtcha/
      |\      _,,,---,,_    pechtcha@... | igor@...
ZZZzz /,`.-'`'    -.  ;-;;,_ Igor Peshansky, Ph.D. (name changed!)
     |,4-  ) )-,_. ,\ (  `'-' old name: Igor Pechtchanski
    '---''(_/--'  `-'\_) fL a.k.a JaguaR-R-R-r-r-r-.-.-.  Meow!

"Las! je suis sot... -Mais non, tu ne l'es pas, puisque tu t'en rends compte."
"But no -- you are no fool; you call yourself a fool, there's proof enough in
that!" -- Rostand, "Cyrano de Bergerac"

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

[Corinna Vinschen-2]

On Aug  3 13:22, Igor Peshansky wrote:

> On Thu, 3 Aug 2006, Corinna Vinschen wrote:
> > On Aug  3 01:54, Eric Blake wrote:
> > > it would be accepted upstream.  Now if there were something more
> > > POSIX-y that we could do to speed things up, such as posix_fadvise,
> >
> > posix_fadvise can't be implemented nicely, AFAICS.  The POSIX semantics
> > require an already opened file and the advice is given for an offset and
> > a length.  The Windows semantics only allow to give the advice for the
> > whole file, and only switching between FILE_SEQUENTIAL_ONLY or "normal",
> > using ZwSetInformationFile.  By re-opening the file using ZwOpenFile it
> > would also be possible to toggle the FILE_RANDOM_ACCESS flag.  Still,
> > it's always for the whole file, not for an area giving offset and length.
>
> Theoretically, it's possible to implement posix_fadvise only for offset=0
> and length=<length-of-file>, and have it fail with EINVAL otherwise...
> While technically not POSIX-compliant, it would still allow for better
> implementation of things like copy...

Right.  Now for the next problem.  Could anybody be bothered to actually
test the performance effect of setting FILE_SEQUENTIAL_ONLY when reading
and writing files sequentially as cp does?  It would be quite interesting
to get some real numbers on FAT and NTFS, before implementing posix_fadvice
just to find out that it has no practical effect.  If it's less than 10%
it's not worth the effort, imo.


Corinna

--
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Project Co-Leader          cygwin AT cygwin DOT com
Red Hat

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

[Eric Blake-2]

>
> Theoretically, it's possible to implement posix_fadvise only for offset=0
> and length=<length-of-file>, and have it fail with EINVAL otherwise...
> While technically not POSIX-compliant, it would still allow for better
> implementation of things like copy...

Even better, have it return 0 for success but do nothing when offset!=0
or len!= 0.  It's called posix_fadvise for a reason - the implementation is
free to ignore the advice when it can't reasonably use it.  EINVAL should
only be used when passing a bad argument (ie. outside the range of
defined POSIX_FADV_* constants).

There is also posix_fallocate, which may help the case of cp.

--
Eric Blake

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

[Corinna Vinschen-2]

On Aug  3 19:08, Eric Blake wrote:

> >
> > Theoretically, it's possible to implement posix_fadvise only for offset=0
> > and length=<length-of-file>, and have it fail with EINVAL otherwise...
> > While technically not POSIX-compliant, it would still allow for better
> > implementation of things like copy...
>
> Even better, have it return 0 for success but do nothing when offset!=0
> or len!= 0.  It's called posix_fadvise for a reason - the implementation is
> free to ignore the advice when it can't reasonably use it.  EINVAL should
> only be used when passing a bad argument (ie. outside the range of
> defined POSIX_FADV_* constants).
>
> There is also posix_fallocate, which may help the case of cp.

Does cp already use ftruncate?


Corinna

--
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Project Co-Leader          cygwin AT cygwin DOT com
Red Hat

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

[Eric Blake]

Corinna Vinschen <corinna-cygwin <at> cygwin.com> writes:

>
> Does cp already use ftruncate?

Yes, when copying from one regular file to another, cp (and mv and install,
which share the same code for file copies) uses ftruncate.  But it currently
only uses it for creating holes at the tail end of sparse files, relying on
lseek for holes in the middle.  It currently does not use posix_f
{allocate/advise), although I think such a patch would be useful independently
of cygwin.

--
Eric Blake





--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/