Possible Bug or limitation in Cygwin 1.7 and Rsync and file number limit

[lists-94]

I'm not sure if this is a bug or a limitation that can be worked around with a setting somewhere, but I have found a problem with cygwin 1.7 while using rsync.  I have been using rsync and cygwin v1.5 for quite some time.  I recently started testing cygwin v1.7 and I ran into a problem with an apparent limit on the number of files cygwin 1.7 along with rsync can process in a directory.  First let me say that I don't store this many files in a directory manually but rather we have a program that does.

I have not been able to identify the exact number of files, but it seems to be somewhere over 6,000 in a single directory.  To help sort this out, I whipped up the following perl code that creates 9,000 dummy files and puts them in a c:\temp directory.  Code is as follows if anyone would like to try and reproduce this:

my $path="c:/temp/";
my $number_of_files=9000;
my $base_file_name="file";
my $count=1;
my $file_name = $path . $base_file_name . $count;
while ($count <= $number_of_files) {
  open FILE, ">$file_name" or die $!;
  $count++;
  $file_name = $path . $base_file_name . $count;
  print FILE "asdfasdf";
  close(FILE);
}

For those familiar with rsync, here are the options I am using in rsync and by the way, I am currently reproducing this by copying from and to the same workstation, however this same issue has been reproduced on more than one machine.  This doesn't appear to be an rsync problem but just the same, I have tried several different options for rsync and none seem to make a difference.

--recursive
--human-readable
--delete
--times
--ignore-errors
--stats
--progress
-v -v

If I then run rsync 3.0.5 using cygwin 1.7, the processor will spike to 100% utilization and it will process several thousand files before it just locks up with the processor still spiked at 100%.  No errors are shown, it just hangs.  If I then delete the directory I am using rsync to copy to and repeat the same procedure with the same version of rsync only this time using cygwin 1.5.25-15, the processor will again spike to 100% however the copy will finish as expected and everything returns to normal.  Again, I have been able to repeat this on more than one computer.

One more interesting thing.  If I first using rsync and cygwin v1.5 to copy the files and THEN come back and use rsync and cygwin 1.7, it make it all the way through.  

See the attached cygcheck.out as per posting instructions.  Note I have also tried this on machines with much stronger hardware with the exact same results.  ALso note that the machine never runs out of memory so that doesn't appear to be the problem either.  

Is this a bug in cygwin 1.7 or is there some setting that will allow me to work around this limitation?

Thank you.


- Kyle



Cygwin Configuration Diagnostics
Current System Time: Thu Jun 18 13:39:33 2009

Windows XP Professional Ver 5.1 Build 2600 Service Pack 3

Path: C:\WINDOWS\system32
        C:\WINDOWS
        C:\WINDOWS\System32\Wbem

Output from C:\cygwin_1.7\bin\id.exe (nontsec)
UID: 10500(administrator) GID: 10513(Domain Users)
0(root)                   544(Administrators)       545(Users)
10513(Domain Users)

Output from C:\cygwin_1.7\bin\id.exe (ntsec)
UID: 10500(administrator) GID: 10513(Domain Users)
0(root)                   544(Administrators)       545(Users)
10513(Domain Users)

SysDir: C:\WINDOWS\system32
WinDir: C:\WINDOWS

Path = 'C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem'

ALLUSERSPROFILE = 'C:\Documents and Settings\All Users'
APPDATA = 'C:\Documents and Settings\Administrator.TRC\Application Data'
CLIENTNAME = 'Console'
CommonProgramFiles = 'C:\Program Files\Common Files'
COMPUTERNAME = 'TEMP'
ComSpec = 'C:\WINDOWS\system32\cmd.exe'
FP_NO_HOST_CHECK = 'NO'
HOMEDRIVE = 'C:'
HOMEPATH = '\Documents and Settings\Administrator.TRC'
LOGONSERVER = '\\TRCBT1'
NUMBER_OF_PROCESSORS = '1'
OS = 'Windows_NT'
PATHEXT = '.COM;.EXE;.BAT;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH'
PROCESSOR_ARCHITECTURE = 'x86'
PROCESSOR_IDENTIFIER = 'x86 Family 6 Model 5 Stepping 2, GenuineIntel'
PROCESSOR_LEVEL = '6'
PROCESSOR_REVISION = '0502'
ProgramFiles = 'C:\Program Files'
PROMPT = '$P$G'
SESSIONNAME = 'Console'
SystemDrive = 'C:'
SystemRoot = 'C:\WINDOWS'
TEMP = 'C:\DOCUME~1\ADMINI~1.TRC\LOCALS~1\Temp'
TMP = 'C:\DOCUME~1\ADMINI~1.TRC\LOCALS~1\Temp'
USERDNSDOMAIN = 'TRCINTL.COM'
USERDOMAIN = 'TRC'
USERNAME = 'administrator'
USERPROFILE = 'C:\Documents and Settings\Administrator.TRC'
windir = 'C:\WINDOWS'

HKEY_CURRENT_USER\Software\Cygnus Solutions\Cygwin
HKEY_CURRENT_USER\Software\Cygnus Solutions\Cygwin\mounts v2
HKEY_CURRENT_USER\Software\Cygnus Solutions\Cygwin\Program Options
HKEY_CURRENT_USER\Software\Cygwin
HKEY_CURRENT_USER\Software\Cygwin\Program Options
HKEY_CURRENT_USER\Software\Cygwin\setup
HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Explorer\MenuOrder\Start Menu\Programs\Cygwin
  (default) = (unsupported type)
HKEY_LOCAL_MACHINE\SOFTWARE\Cygnus Solutions\Cygwin
HKEY_LOCAL_MACHINE\SOFTWARE\Cygnus Solutions\Cygwin\mounts v2
  (default) = '/cygdrive'
  cygdrive flags = 0x00000022
HKEY_LOCAL_MACHINE\SOFTWARE\Cygnus Solutions\Cygwin\mounts v2\/
  (default) = 'C:\cygwin'
  flags = 0x0000000a
HKEY_LOCAL_MACHINE\SOFTWARE\Cygnus Solutions\Cygwin\mounts v2\/usr/bin
  (default) = 'C:\cygwin/bin'
  flags = 0x0000000a
HKEY_LOCAL_MACHINE\SOFTWARE\Cygnus Solutions\Cygwin\mounts v2\/usr/lib
  (default) = 'C:\cygwin/lib'
  flags = 0x0000000a
HKEY_LOCAL_MACHINE\SOFTWARE\Cygnus Solutions\Cygwin\Program Options
HKEY_LOCAL_MACHINE\SOFTWARE\Cygwin
HKEY_LOCAL_MACHINE\SOFTWARE\Cygwin\Program Options
HKEY_LOCAL_MACHINE\SOFTWARE\Cygwin\setup
  (default) = 'C:\cygwin_1.7'

obcaseinsensitive set to 1

a:  fd  FAT          1Mb   6% CP    UN          
c:  hd  NTFS      8048Mb  68% CP CS UN PA FC    
d:  cd             N/A    N/A                    
z:  net NTFS    547127Mb  58% CP CS UN PA FC     Root

C:\cygwin_1.7  /          system  binmode
.              /cygdrive  user    binmode,cygdrive,noacl,posix=0

Not Found: awk
Not Found: bash
Not Found: cat
Not Found: cp
Not Found: cpp (good!)
Not Found: crontab
Found: C:\WINDOWS\system32\find.exe
Not Found: gcc
Not Found: gdb
Not Found: grep
Not Found: kill
Not Found: ld
Not Found: ls
Not Found: make
Not Found: mv
Not Found: patch
Not Found: perl
Not Found: rm
Not Found: sed
Not Found: ssh
Not Found: sh
Not Found: tar
Not Found: test
Not Found: vi
Not Found: vim

   61k 2009/03/02 .\cygbz2-1.dll - os=4.0 img=1.0 sys=4.0
                  "cygbz2-1.dll" v0.0 ts=2009/3/1 21:11
   40k 2009/03/01 .\cygform-8.dll - os=4.0 img=1.0 sys=4.0
                  "cygform-8.dll" v0.0 ts=2009/3/1 0:32
   41k 2009/03/27 .\cygform-9.dll - os=4.0 img=1.0 sys=4.0
                  "cygform-9.dll" v0.0 ts=2009/3/27 0:02
   42k 2009/03/12 .\cyggcc_s-1.dll - os=4.0 img=1.0 sys=4.0
                  "cyggcc_s-1.dll" v0.0 ts=2009/3/6 5:54
  315k 2009/06/06 .\cyggmp-3.dll - os=4.0 img=1.0 sys=4.0
                  "cyggmp-3.dll" v0.0 ts=2009/6/6 6:23
   24k 2008/11/29 .\cyghistory6.dll - os=4.0 img=1.0 sys=4.0
                  "cyghistory6.dll" v0.0 ts=2008/11/29 8:30
  270k 2009/05/07 .\cygicons-0.dll - os=4.0 img=1.0 sys=4.0
                  "cygicons-0.dll" v0.0 ts=2009/5/6 19:58
  981k 2009/05/30 .\cygiconv-2.dll - os=4.0 img=1.0 sys=4.0
                  "cygiconv-2.dll" v0.0 ts=2009/5/30 14:05
   31k 2005/11/20 .\cygintl-3.dll - os=4.0 img=1.0 sys=4.0
                  "cygintl-3.dll" v0.0 ts=2005/11/19 20:04
   31k 2009/04/03 .\cygintl-8.dll - os=4.0 img=1.0 sys=4.0
                  "cygintl-8.dll" v0.0 ts=2009/4/2 20:15
    5k 2009/06/08 .\cyglsa.dll - os=4.0 img=1.0 sys=4.0
                  "cyglsa.dll" v0.0 ts=2009/6/8 15:14
    9k 2009/06/08 .\cyglsa64.dll - os=4.0 img=0.0 sys=5.2
   21k 2009/03/01 .\cygmenu-8.dll - os=4.0 img=1.0 sys=4.0
                  "cygmenu-8.dll" v0.0 ts=2009/3/1 0:31
   21k 2009/03/27 .\cygmenu-9.dll - os=4.0 img=1.0 sys=4.0
                  "cygmenu-9.dll" v0.0 ts=2009/3/27 0:01
  211k 2009/06/06 .\cygmp-3.dll - os=4.0 img=1.0 sys=4.0
                  "cygmp-3.dll" v0.0 ts=2009/6/6 6:23
   66k 2009/03/01 .\cygncurses++-8.dll - os=4.0 img=1.0 sys=4.0
                  "cygncurses++-8.dll" v0.0 ts=2009/3/1 0:39
  334k 2009/03/27 .\cygncurses++-9.dll - os=4.0 img=1.0 sys=4.0
                  "cygncurses++-9.dll" v0.0 ts=2009/3/27 0:12
  237k 2009/03/01 .\cygncurses-8.dll - os=4.0 img=1.0 sys=4.0
                  "cygncurses-8.dll" v0.0 ts=2009/3/1 0:28
  164k 2009/03/27 .\cygncurses-9.dll - os=4.0 img=1.0 sys=4.0
                  "cygncurses-9.dll" v0.0 ts=2009/3/26 23:58
   11k 2009/03/01 .\cygpanel-8.dll - os=4.0 img=1.0 sys=4.0
                  "cygpanel-8.dll" v0.0 ts=2009/3/1 0:30
   11k 2009/03/27 .\cygpanel-9.dll - os=4.0 img=1.0 sys=4.0
                  "cygpanel-9.dll" v0.0 ts=2009/3/27 0:00
  190k 2009/04/17 .\cygpcre-0.dll - os=4.0 img=1.0 sys=4.0
                  "cygpcre-0.dll" v0.0 ts=2009/4/17 11:07
    7k 2009/04/17 .\cygpcreposix-0.dll - os=4.0 img=1.0 sys=4.0
                  "cygpcreposix-0.dll" v0.0 ts=2009/4/17 11:07
   22k 2002/06/09 .\cygpopt-0.dll - os=4.0 img=1.0 sys=4.0
                  "cygpopt-0.dll" v0.0 ts=2002/6/9 0:45
  155k 2008/11/29 .\cygreadline6.dll - os=4.0 img=1.0 sys=4.0
                  "cygreadline6.dll" v0.0 ts=2008/11/29 8:30
   66k 2009/03/27 .\cygtic-9.dll - os=4.0 img=1.0 sys=4.0
                  "cygtic-9.dll" v0.0 ts=2009/3/26 23:54
   64k 2009/03/02 .\cygz.dll - os=4.0 img=1.0 sys=4.0
                  "cygz.dll" v0.0 ts=2009/3/1 19:33
 2376k 2009/06/08 .\cygwin1.dll - os=4.0 img=1.0 sys=4.0
                  "cygwin1.dll" v0.0 ts=2009/6/8 15:13
    Cygwin DLL version info:
        DLL version: 1.7.0
        DLL epoch: 19
        DLL old termios: 5
        DLL malloc env: 28
        Cygwin conv: 181
        API major: 0
        API minor: 210
        Shared data: 5
        DLL identifier: cygwin1
        Mount registry: 3
        Cygwin registry name: Cygwin
        Program options name: Program Options
        Cygdrive default prefix:
        Build date: Mon Jun 8 22:13:39 CEST 2009
        Shared id: cygwin1S5

   61k 2009/03/02 C:\cygwin_1.7\bin\cygbz2-1.dll - os=4.0 img=1.0 sys=4.0
                  "cygbz2-1.dll" v0.0 ts=2009/3/1 21:11
   40k 2009/03/01 C:\cygwin_1.7\bin\cygform-8.dll - os=4.0 img=1.0 sys=4.0
                  "cygform-8.dll" v0.0 ts=2009/3/1 0:32
   41k 2009/03/27 C:\cygwin_1.7\bin\cygform-9.dll - os=4.0 img=1.0 sys=4.0
                  "cygform-9.dll" v0.0 ts=2009/3/27 0:02
   42k 2009/03/12 C:\cygwin_1.7\bin\cyggcc_s-1.dll - os=4.0 img=1.0 sys=4.0
                  "cyggcc_s-1.dll" v0.0 ts=2009/3/6 5:54
  315k 2009/06/06 C:\cygwin_1.7\bin\cyggmp-3.dll - os=4.0 img=1.0 sys=4.0
                  "cyggmp-3.dll" v0.0 ts=2009/6/6 6:23
   24k 2008/11/29 C:\cygwin_1.7\bin\cyghistory6.dll - os=4.0 img=1.0 sys=4.0
                  "cyghistory6.dll" v0.0 ts=2008/11/29 8:30
  270k 2009/05/07 C:\cygwin_1.7\bin\cygicons-0.dll - os=4.0 img=1.0 sys=4.0
                  "cygicons-0.dll" v0.0 ts=2009/5/6 19:58
  981k 2009/05/30 C:\cygwin_1.7\bin\cygiconv-2.dll - os=4.0 img=1.0 sys=4.0
                  "cygiconv-2.dll" v0.0 ts=2009/5/30 14:05
   31k 2005/11/20 C:\cygwin_1.7\bin\cygintl-3.dll - os=4.0 img=1.0 sys=4.0
                  "cygintl-3.dll" v0.0 ts=2005/11/19 20:04
   31k 2009/04/03 C:\cygwin_1.7\bin\cygintl-8.dll - os=4.0 img=1.0 sys=4.0
                  "cygintl-8.dll" v0.0 ts=2009/4/2 20:15
    5k 2009/06/08 C:\cygwin_1.7\bin\cyglsa.dll - os=4.0 img=1.0 sys=4.0
                  "cyglsa.dll" v0.0 ts=2009/6/8 15:14
    9k 2009/06/08 C:\cygwin_1.7\bin\cyglsa64.dll - os=4.0 img=0.0 sys=5.2
   21k 2009/03/01 C:\cygwin_1.7\bin\cygmenu-8.dll - os=4.0 img=1.0 sys=4.0
                  "cygmenu-8.dll" v0.0 ts=2009/3/1 0:31
   21k 2009/03/27 C:\cygwin_1.7\bin\cygmenu-9.dll - os=4.0 img=1.0 sys=4.0
                  "cygmenu-9.dll" v0.0 ts=2009/3/27 0:01
  211k 2009/06/06 C:\cygwin_1.7\bin\cygmp-3.dll - os=4.0 img=1.0 sys=4.0
                  "cygmp-3.dll" v0.0 ts=2009/6/6 6:23
   66k 2009/03/01 C:\cygwin_1.7\bin\cygncurses++-8.dll - os=4.0 img=1.0 sys=4.0
                  "cygncurses++-8.dll" v0.0 ts=2009/3/1 0:39
  334k 2009/03/27 C:\cygwin_1.7\bin\cygncurses++-9.dll - os=4.0 img=1.0 sys=4.0
                  "cygncurses++-9.dll" v0.0 ts=2009/3/27 0:12
  237k 2009/03/01 C:\cygwin_1.7\bin\cygncurses-8.dll - os=4.0 img=1.0 sys=4.0
                  "cygncurses-8.dll" v0.0 ts=2009/3/1 0:28
  164k 2009/03/27 C:\cygwin_1.7\bin\cygncurses-9.dll - os=4.0 img=1.0 sys=4.0
                  "cygncurses-9.dll" v0.0 ts=2009/3/26 23:58
   11k 2009/03/01 C:\cygwin_1.7\bin\cygpanel-8.dll - os=4.0 img=1.0 sys=4.0
                  "cygpanel-8.dll" v0.0 ts=2009/3/1 0:30
   11k 2009/03/27 C:\cygwin_1.7\bin\cygpanel-9.dll - os=4.0 img=1.0 sys=4.0
                  "cygpanel-9.dll" v0.0 ts=2009/3/27 0:00
  190k 2009/04/17 C:\cygwin_1.7\bin\cygpcre-0.dll - os=4.0 img=1.0 sys=4.0
                  "cygpcre-0.dll" v0.0 ts=2009/4/17 11:07
    7k 2009/04/17 C:\cygwin_1.7\bin\cygpcreposix-0.dll - os=4.0 img=1.0 sys=4.0
                  "cygpcreposix-0.dll" v0.0 ts=2009/4/17 11:07
   22k 2002/06/09 C:\cygwin_1.7\bin\cygpopt-0.dll - os=4.0 img=1.0 sys=4.0
                  "cygpopt-0.dll" v0.0 ts=2002/6/9 0:45
  155k 2008/11/29 C:\cygwin_1.7\bin\cygreadline6.dll - os=4.0 img=1.0 sys=4.0
                  "cygreadline6.dll" v0.0 ts=2008/11/29 8:30
   66k 2009/03/27 C:\cygwin_1.7\bin\cygtic-9.dll - os=4.0 img=1.0 sys=4.0
                  "cygtic-9.dll" v0.0 ts=2009/3/26 23:54
   64k 2009/03/02 C:\cygwin_1.7\bin\cygz.dll - os=4.0 img=1.0 sys=4.0
                  "cygz.dll" v0.0 ts=2009/3/1 19:33
 2376k 2009/06/08 C:\cygwin_1.7\bin\cygwin1.dll - os=4.0 img=1.0 sys=4.0
                  "cygwin1.dll" v0.0 ts=2009/6/8 15:13
    Cygwin DLL version info:
        DLL version: 1.7.0
        DLL epoch: 19
        DLL old termios: 5
        DLL malloc env: 28
        Cygwin conv: 181
        API major: 0
        API minor: 210
        Shared data: 5
        DLL identifier: cygwin1
        Mount registry: 3
        Cygwin registry name: Cygwin
        Program options name: Program Options
        Cygdrive default prefix:
        Build date: Mon Jun 8 22:13:39 CEST 2009
        Shared id: cygwin1S5

Warning: There are multiple cygwin1.dlls on your path

Can't find the cygrunsrv utility, skipping services check.


Cygwin Package Information
Last downloaded files to: C:\Documents and Settings\Administrator.TRC\Desktop
Last downloaded files from: ftp://mirror.mcs.anl.gov/pub/cygwin/

Package              Version
_update-info-dir     00823-1
alternatives         1.3.30c-10
ash                  20040127-4
base-cygwin          2.0-1
base-files           3.8-3
base-passwd          3.1-1
bash                 3.2.48-21
bzip2                1.0.5-10
coreutils            7.0-2
cygutils             1.4.0-1
cygwin               1.7.0-49
cygwin-doc           1.5-1
editrights           1.01-2
findutils            4.5.4-1
gawk                 3.1.6-2
grep                 2.5.3-1
groff                1.19.2-2
gzip                 1.3.12-2
ipc-utils            1.0-1
less                 429-1
libbz2_1             1.0.5-10
libgcc1              4.3.2-2
libgmp3              4.3.1-3
libiconv2            1.13-10
libintl3             0.14.5-1
libintl8             0.17-11
libncurses8          5.5-10
libncurses9          5.7-14
libpcre0             7.9-1
libpopt0             1.6.4-4
libreadline6         5.2.13-11
login                1.10-10
man                  1.6e-1
rebase               3.0-2
rsync                3.0.5-1
run                  1.1.10-1
sed                  4.1.5-2
tar                  1.22-1
termcap              20050421-1
terminfo             5.7_20090228-10
terminfo0            5.5_20061104-10
texinfo              4.13-3
tzcode               2008h-1
which                2.20-2
zlib0                1.2.3-10
Use -h to see help about each section


--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

[Larry Hall (Cygwin)]

lists wrote:
<snip>

> See the attached cygcheck.out as per posting instructions. Note I have
> also tried this on machines with much stronger hardware with the exact same
> results. ALso note that the machine never runs out of memory so that doesn't
> appear to be the problem either.

<snip>

>
> Not Found: awk
> Not Found: bash
> Not Found: cat
> Not Found: cp
> Not Found: cpp (good!)
> Not Found: crontab
> Found: C:\WINDOWS\system32\find.exe
> Not Found: gcc
> Not Found: gdb
> Not Found: grep
> Not Found: kill
> Not Found: ld
> Not Found: ls
> Not Found: make
> Not Found: mv
> Not Found: patch
> Not Found: perl
> Not Found: rm
> Not Found: sed
> Not Found: ssh
> Not Found: sh
> Not Found: tar
> Not Found: test
> Not Found: vi
> Not Found: vim

Can you upgrade to the latest Cygwin 1.7 package and try again.  From your
cygcheck output, it looks like things are not correct but this may be just
a problem with an old cygcheck that doesn't know how to find the implicit
mounts.  If the issue is still the same, resending the output of the new
cygcheck would be helpful.

--
Larry Hall                              http://www.rfk.com
RFK Partners, Inc.                      [phone] - RFK Office
216 Dalton Rd.                          [phone] - FAX
Holliston, MA 01746

_____________________________________________________________________

A: Yes.
 > Q: Are you sure?
 >> A: Because it reverses the logical flow of conversation.
 >>> Q: Why is top posting annoying in email?

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

[lists-94]

<snip>

> Can you upgrade to the latest Cygwin 1.7 package and try again.  From your
> cygcheck output, it looks like things are not correct but this may be just
> a problem with an old cygcheck that doesn't know how to find the implicit
> mounts.  If the issue is still the same, resending the output of the new
> cygcheck would be helpful.
>
> --
> Larry Hall                              http://www.rfk.com
> RFK Partners, Inc.                      [phone] - RFK Office
> 216 Dalton Rd.                          [phone] - FAX
> Holliston, MA 01746
>

I just downloaded the newest version of cygwin 1.7 as of 6-19-09 at
approximately 3:30 CST and had the exact same results.  Please find the new
cygcheck.out file attached.  You mentioned above that their might be a
problem with the mounts.  Not sure if it helps to know this, but I can
successfully use rsync to copy many gigabytes of files.  It just hangs when
there are thousands of files in a single directory (9,000 seems to always do
it.).  Also note that this cygcheck was done on a different computer than
the first just to make sure this isn't computer specific.  Again, I have
tried it on several.

Thank you in advance.

- Kyle

> _____________________________________________________________________
>
> A: Yes.
> > Q: Are you sure?
> >> A: Because it reverses the logical flow of conversation.
> >>> Q: Why is top posting annoying in email?
>
> --
> Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
> Problem reports:       http://cygwin.com/problems.html
> Documentation:         http://cygwin.com/docs.html
> FAQ:                   http://cygwin.com/faq/
>

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

[Larry Hall (Cygwin)]

Lists wrote:

> Larry Hall wrote:
>> <snip>
>> Can you upgrade to the latest Cygwin 1.7 package and try again.  From
>> your
>> cygcheck output, it looks like things are not correct but this may be
>> just
>> a problem with an old cygcheck that doesn't know how to find the implicit
>> mounts.  If the issue is still the same, resending the output of the new
>> cygcheck would be helpful.
>
> I just downloaded the newest version of cygwin 1.7 as of 6-19-09 at
> approximately 3:30 CST and had the exact same results.  Please find the
> new cygcheck.out file attached.  You mentioned above that their might be
> a problem with the mounts.  Not sure if it helps to know this, but I can
> successfully use rsync to copy many gigabytes of files.  It just hangs
> when there are thousands of files in a single directory (9,000 seems to
> always do it.).  Also note that this cygcheck was done on a different
> computer than the first just to make sure this isn't computer specific.  
> Again, I have tried it on several.
>
> Thank you in advance.

Your cygcheck output looks more like what I would expect except that it
is truncated.  Does this happen when you run it or did it just get stopped
early?

--
Larry Hall                              http://www.rfk.com
RFK Partners, Inc.                      [phone] - RFK Office
216 Dalton Rd.                          [phone] - FAX
Holliston, MA 01746

_____________________________________________________________________

A: Yes.
 > Q: Are you sure?
 >> A: Because it reverses the logical flow of conversation.
 >>> Q: Why is top posting annoying in email?

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

[lists-94]

> Lists wrote:

>> Larry Hall wrote:
>>> <snip>
>>> Can you upgrade to the latest Cygwin 1.7 package and try again.  From
>>> your
>>> cygcheck output, it looks like things are not correct but this may be
>>> just
>>> a problem with an old cygcheck that doesn't know how to find the
>>> implicit
>>> mounts.  If the issue is still the same, resending the output of the new
>>> cygcheck would be helpful.
>>
>> I just downloaded the newest version of cygwin 1.7 as of 6-19-09 at
>> approximately 3:30 CST and had the exact same results.  Please find the
>> new cygcheck.out file attached.  You mentioned above that their might be
>> a problem with the mounts.  Not sure if it helps to know this, but I can
>> successfully use rsync to copy many gigabytes of files.  It just hangs
>> when there are thousands of files in a single directory (9,000 seems to
>> always do it.).  Also note that this cygcheck was done on a different
>> computer than the first just to make sure this isn't computer specific.
>> Again, I have tried it on several.
>>
>> Thank you in advance.
>
> Your cygcheck output looks more like what I would expect except that it
> is truncated.  Does this happen when you run it or did it just get stopped
> early?
>
> --
> Larry Hall                              http://www.rfk.com
> RFK Partners, Inc.                      [phone] - RFK Office
> 216 Dalton Rd.                          [phone] - FAX
> Holliston, MA 01746
>

It didn't get stoped early, that is just what it looks like on this machine.
I ran it again to verify and got what appears to be the same thing, at least
it seems to stop in the same place.  This particular machine runs Windows
Vista so perhaps that has something to do with it?  Either way, the rsync
results were the same.  Attached you should find another copy created on yet
another machine.  Rsync results remain the same.  This machine runs XP.

Thank you again.

- Kyle

> _____________________________________________________________________
>
> A: Yes.
> > Q: Are you sure?
> >> A: Because it reverses the logical flow of conversation.
> >>> Q: Why is top posting annoying in email?
>
> --
> Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
> Problem reports:       http://cygwin.com/problems.html
> Documentation:         http://cygwin.com/docs.html
> FAQ:                   http://cygwin.com/faq/
>

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/

[Larry Hall (Cygwin)]

Lists wrote:

>> Lists wrote:
>>> Larry Hall wrote:
>>>> <snip>
>>>> Can you upgrade to the latest Cygwin 1.7 package and try again.  
>>>> From your
>>>> cygcheck output, it looks like things are not correct but this may
>>>> be just
>>>> a problem with an old cygcheck that doesn't know how to find the
>>>> implicit
>>>> mounts.  If the issue is still the same, resending the output of the
>>>> new
>>>> cygcheck would be helpful.
>>>
>>> I just downloaded the newest version of cygwin 1.7 as of 6-19-09 at
>>> approximately 3:30 CST and had the exact same results.  Please find
>>> the new cygcheck.out file attached.  You mentioned above that their
>>> might be a problem with the mounts.  Not sure if it helps to know
>>> this, but I can successfully use rsync to copy many gigabytes of
>>> files.  It just hangs when there are thousands of files in a single
>>> directory (9,000 seems to always do it.).  Also note that this
>>> cygcheck was done on a different computer than the first just to make
>>> sure this isn't computer specific. Again, I have tried it on several.
>>>
>>> Thank you in advance.
>>
>> Your cygcheck output looks more like what I would expect except that it
>> is truncated.  Does this happen when you run it or did it just get
>> stopped
>> early?
>>
>> --
>> Larry Hall                              http://www.rfk.com
>> RFK Partners, Inc.                      [phone] - RFK Office
>> 216 Dalton Rd.                          [phone] - FAX
>> Holliston, MA 01746
>>
>
> It didn't get stoped early, that is just what it looks like on this
> machine. I ran it again to verify and got what appears to be the same
> thing, at least it seems to stop in the same place.  This particular
> machine runs Windows Vista so perhaps that has something to do with it?  
> Either way, the rsync results were the same.  Attached you should find
> another copy created on yet another machine.  Rsync results remain the
> same.  This machine runs XP.

OK, this makes me very suspicious that something is getting in your way.
'cygcheck -srv' should contain allot more information than that.  And while
I might suspect Vista (though was always able to get valid output on the
Vista installation I used to have), I would never expect a problem getting
the complete output on XP unless something was interfering.  Perhaps you're
experiencing the effects of <http://cygwin.com/acronyms/#BLODA>?
Ironically, if this is the problem, the cygcheck output is designed to
help spot this kind of problem...


--
Larry Hall                              http://www.rfk.com
RFK Partners, Inc.                      [phone] - RFK Office
216 Dalton Rd.                          [phone] - FAX
Holliston, MA 01746

_____________________________________________________________________

A: Yes.
 > Q: Are you sure?
 >> A: Because it reverses the logical flow of conversation.
 >>> Q: Why is top posting annoying in email?

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

[Corinna Vinschen-2]

On Jun 19 15:55, Lists wrote:

> <snip>
>> Can you upgrade to the latest Cygwin 1.7 package and try again.  From your
>> cygcheck output, it looks like things are not correct but this may be just
>> a problem with an old cygcheck that doesn't know how to find the implicit
>> mounts.  If the issue is still the same, resending the output of the new
>> cygcheck would be helpful.
>
> I just downloaded the newest version of cygwin 1.7 as of 6-19-09 at  
> approximately 3:30 CST and had the exact same results.  Please find the
> new cygcheck.out file attached.  You mentioned above that their might be
> a problem with the mounts.  Not sure if it helps to know this, but I can  
> successfully use rsync to copy many gigabytes of files.  It just hangs
> when there are thousands of files in a single directory (9,000 seems to
> always do it.).  Also note that this cygcheck was done on a different
> computer than the first just to make sure this isn't computer specific.  
> Again, I have tried it on several.

Works fine for me under the latest Cygwin 1.7.0-50 using your perl
script and your rsync options.  I tend to agree to Larry's assumption
that some 3PP is interferring.


Corinna

--
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Project Co-Leader          cygwin AT cygwin DOT com
Red Hat

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

[lists-94]

> On Jun 19 15:55, Lists wrote:
>> <snip>
>>> Can you upgrade to the latest Cygwin 1.7 package and try again.  From
>>> your
>>> cygcheck output, it looks like things are not correct but this may be
>>> just
>>> a problem with an old cygcheck that doesn't know how to find the
>>> implicit
>>> mounts.  If the issue is still the same, resending the output of the new
>>> cygcheck would be helpful.
>>
>> I just downloaded the newest version of cygwin 1.7 as of 6-19-09 at
>> approximately 3:30 CST and had the exact same results.  Please find the
>> new cygcheck.out file attached.  You mentioned above that their might be
>> a problem with the mounts.  Not sure if it helps to know this, but I can
>> successfully use rsync to copy many gigabytes of files.  It just hangs
>> when there are thousands of files in a single directory (9,000 seems to
>> always do it.).  Also note that this cygcheck was done on a different
>> computer than the first just to make sure this isn't computer specific.
>> Again, I have tried it on several.
>
> Works fine for me under the latest Cygwin 1.7.0-50 using your perl
> script and your rsync options.  I tend to agree to Larry's assumption
> that some 3PP is interferring.
>
>
> Corinna
>

Sorry to just now be replying but I have been out of the country.  I tried
the same procedure on yet another computer.  I actually got it to run one
time without a problem.  I then deleted the copied files and repeated the
procedure and got the same problem again.

I am totally convienced there is a problem here.  One of the machines I have
been working on is a "clean" machine, meaning that it has no other software
on it other than the OS and this software.  I keep a machine around like
this for just such purposes.  After using it for a while, I reload it.  By
the way, some of the machines I have tried are not even on the same network.

So, I have tried this on at least 10 machines now and all exhibit the same
behavior.  I could attach another output from  but they all look pretty much
like the previous ones I attached.

Can anyone else try this and confirm the problem?  If it works, I suspect if
you try it again a time or two, you too will see the problem.  Before trying
it again, you need to make sure and delete the files from the destination
directory.

Thank you.

- Kyle


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

[Corinna Vinschen-2]

On Jun 26 23:42, Lists wrote:

>> On Jun 19 15:55, Lists wrote:
>>> <snip>
>> Works fine for me under the latest Cygwin 1.7.0-50 using your perl
>> script and your rsync options.  I tend to agree to Larry's assumption
>> that some 3PP is interferring.
>
> Sorry to just now be replying but I have been out of the country.  I
> tried the same procedure on yet another computer.  I actually got it to
> run one time without a problem.  I then deleted the copied files and
> repeated the procedure and got the same problem again.
>
> I am totally convienced there is a problem here.

Still, I can't reproduce it.  I just ran this with the latest Cygwin
from CVS and it works fine for me, multiple times, every time removing
the copied files from the former run.

I ran the command the same way you did, I guess, just using different
paths for the source and destination directories:

  rsync --recursive \
        --human-readable \
        --delete \
        --times \
        --ignore-errors \
        --stats \
        --progress \
        -v -v \
        ~/tmp/rsync ~/tmp/rsync-tgt

> Can anyone else try this and confirm the problem?  If it works, I suspect
> if you try it again a time or two, you too will see the problem.  Before
> trying it again, you need to make sure and delete the files from the
> destination directory.

I tried it 6 times in a row, every time erasing the destination files.
Works fine for me.


Corinna

--
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Project Co-Leader          cygwin AT cygwin DOT com
Red Hat

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

[lists-94]

> On Jun 26 23:42, Lists wrote:
>>> On Jun 19 15:55, Lists wrote:
>>>> <snip>
>>> Works fine for me under the latest Cygwin 1.7.0-50 using your perl
>>> script and your rsync options.  I tend to agree to Larry's assumption
>>> that some 3PP is interferring.
>>
>> Sorry to just now be replying but I have been out of the country.  I
>> tried the same procedure on yet another computer.  I actually got it to
>> run one time without a problem.  I then deleted the copied files and
>> repeated the procedure and got the same problem again.
>>
>> I am totally convienced there is a problem here.
>
> Still, I can't reproduce it.  I just ran this with the latest Cygwin
> from CVS and it works fine for me, multiple times, every time removing
> the copied files from the former run.
>
> I ran the command the same way you did, I guess, just using different
> paths for the source and destination directories:
>
>  rsync --recursive \
>        --human-readable \
> --delete \
> --times \
> --ignore-errors \
> --stats \
> --progress \
> -v -v \
> ~/tmp/rsync ~/tmp/rsync-tgt
>
>> Can anyone else try this and confirm the problem?  If it works, I suspect
>> if you try it again a time or two, you too will see the problem.  Before
>> trying it again, you need to make sure and delete the files from the
>> destination directory.
>
> I tried it 6 times in a row, every time erasing the destination files.
> Works fine for me.
>
>
> Corinna
>

Perhaps you have hit upon the problem.  I am just downloading and running
setup-1.7.exe and it definetly has this problem.  I tried to download and
build it from CVS as you did but I have never done that before and for some
reason it isn't working.  I'm not even sure if I am downloading code for
v1.5 or v1.7 but when I follow the instructions under the FAX, it won't
build.  So when will the latest changes be incorporated in the version
downloaded in the setup-1.7.exe because it defintely has this problem.  I
have no doubt about it.  Or, is there somewhere else I can find a pre-built
version of the latest CVS cygwin1.dll?

- Kyle


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

[Corinna Vinschen-2]

On Jul  1 07:36, Lists wrote:
>> I tried it 6 times in a row, every time erasing the destination files.
>> Works fine for me.
>>
>>
>> Corinna
>>
> Perhaps you have hit upon the problem.  I am just downloading and running
> setup-1.7.exe and it definetly has this problem.  I tried to download and

Setup-1.7.exe can't have this problem.  What you mean is the Cygwin DLL.

> build it from CVS as you did but I have never done that before and for
> some reason it isn't working.  I'm not even sure if I am downloading code

What are you trying to download and build from CVS?  Setup or Cygwin?

> for v1.5 or v1.7 but when I follow the instructions under the FAX, it

No idea what you mean.  When you use Setup-1.7 you'll download the 1.7
packages.  When you download CVS HEAD, you'll download the 1.7 sources.

> won't build.  So when will the latest changes be incorporated in the
> version downloaded in the setup-1.7.exe because it defintely has this
> problem.  I have no doubt about it.

That's fine, but unless I can reproduce it, I can't fix it.  You could
try to debug it yourself.  There's GDB, strace, and sources for everything.

> problem.  I have no doubt about it.  Or, is there somewhere else I can
> find a pre-built version of the latest CVS cygwin1.dll?

http://cygwin.com/snapshots/


Corinna

--
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Project Co-Leader          cygwin AT cygwin DOT com
Red Hat

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

[lists-94]

> On Jul  1 07:36, Lists wrote:
>>> I tried it 6 times in a row, every time erasing the destination files.
>>> Works fine for me.
>>>
>>>
>>> Corinna
>>>
>> Perhaps you have hit upon the problem.  I am just downloading and running
>> setup-1.7.exe and it definetly has this problem.  I tried to download and
>
> Setup-1.7.exe can't have this problem.  What you mean is the Cygwin DLL.
>
>> build it from CVS as you did but I have never done that before and for
>> some reason it isn't working.  I'm not even sure if I am downloading code
>
> What are you trying to download and build from CVS?  Setup or Cygwin?
>
>> for v1.5 or v1.7 but when I follow the instructions under the FAX, it
>
> No idea what you mean.  When you use Setup-1.7 you'll download the 1.7
> packages.  When you download CVS HEAD, you'll download the 1.7 sources.
>
>> won't build.  So when will the latest changes be incorporated in the
>> version downloaded in the setup-1.7.exe because it defintely has this
>> problem.  I have no doubt about it.
>
> That's fine, but unless I can reproduce it, I can't fix it.  You could
> try to debug it yourself.  There's GDB, strace, and sources for
> everything.
>
>> problem.  I have no doubt about it.  Or, is there somewhere else I can
>> find a pre-built version of the latest CVS cygwin1.dll?
>
> http://cygwin.com/snapshots/
>
>
> Corinna
>

In your earlier reply, you said that you used the latest cygwin from CVS and
that is not what I am using.  I am using the cygwin that installs after
running setup-1.7.exe on the web site.  I don't know if they are the same.
I do know that the one I am using has this problem as I have reproduced it
far too many times.

By the way, when I try to build it from CVS, I get the following:

cc1: /oss/src/newlib/libc/argz/dummy.c: No such file or directory
cc1: /oss/src/newlib/libc/argz/dummy.c: No such file or directory
cc1: /oss/src/newlib/libc/argz/dummy.c: No such file or directory
cc1: /oss/src/newlib/libc/argz/dummy.c: No such file or directory
cc1: /oss/src/newlib/libc/argz/dummy.c: No such file or directory
cc1: /oss/src/newlib/libc/argz/dummy.c: No such file or directory
make[5]: *** [lib_a-dummy.o] Error 1
make[5]: Leaving directory `/oss/build/i686-pc-cygwin/newlib/libc/argz'
make[4]: *** [all-recursive] Error 1
make[4]: Leaving directory `/oss/build/i686-pc-cygwin/newlib/libc'
make[3]: *** [all-recursive] Error 1
make[3]: Leaving directory `/oss/build/i686-pc-cygwin/newlib'
make[2]: *** [all] Error 2
make[2]: Leaving directory `/oss/build/i686-pc-cygwin/newlib'
make[1]: *** [all-target-newlib] Error 2
make[1]: Leaving directory `/oss/build'
make: *** [all] Error 2

- Kyle


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

[lists-94]

>> On Jul  1 07:36, Lists wrote:
>>>> I tried it 6 times in a row, every time erasing the destination files.
>>>> Works fine for me.
>>>>
>>>>
>>>> Corinna
>>>>
>>> Perhaps you have hit upon the problem.  I am just downloading and
>>> running
>>> setup-1.7.exe and it definetly has this problem.  I tried to download
>>> and
>>
>> Setup-1.7.exe can't have this problem.  What you mean is the Cygwin DLL.
>>
>>> build it from CVS as you did but I have never done that before and for
>>> some reason it isn't working.  I'm not even sure if I am downloading
>>> code
>>
>> What are you trying to download and build from CVS?  Setup or Cygwin?
>>
>>> for v1.5 or v1.7 but when I follow the instructions under the FAX, it
>>
>> No idea what you mean.  When you use Setup-1.7 you'll download the 1.7
>> packages.  When you download CVS HEAD, you'll download the 1.7 sources.
>>
>>> won't build.  So when will the latest changes be incorporated in the
>>> version downloaded in the setup-1.7.exe because it defintely has this
>>> problem.  I have no doubt about it.
>>
>> That's fine, but unless I can reproduce it, I can't fix it.  You could
>> try to debug it yourself.  There's GDB, strace, and sources for
>> everything.
>>
>>> problem.  I have no doubt about it.  Or, is there somewhere else I can
>>> find a pre-built version of the latest CVS cygwin1.dll?
>>
>> http://cygwin.com/snapshots/
>>
>>
>> Corinna
>>
> In your earlier reply, you said that you used the latest cygwin from CVS
> and that is not what I am using.  I am using the cygwin that installs
> after running setup-1.7.exe on the web site.  I don't know if they are the
> same. I do know that the one I am using has this problem as I have
> reproduced it far too many times.
>
> By the way, when I try to build it from CVS, I get the following:
>
> cc1: /oss/src/newlib/libc/argz/dummy.c: No such file or directory
> cc1: /oss/src/newlib/libc/argz/dummy.c: No such file or directory
> cc1: /oss/src/newlib/libc/argz/dummy.c: No such file or directory
> cc1: /oss/src/newlib/libc/argz/dummy.c: No such file or directory
> cc1: /oss/src/newlib/libc/argz/dummy.c: No such file or directory
> cc1: /oss/src/newlib/libc/argz/dummy.c: No such file or directory
> make[5]: *** [lib_a-dummy.o] Error 1
> make[5]: Leaving directory `/oss/build/i686-pc-cygwin/newlib/libc/argz'
> make[4]: *** [all-recursive] Error 1
> make[4]: Leaving directory `/oss/build/i686-pc-cygwin/newlib/libc'
> make[3]: *** [all-recursive] Error 1
> make[3]: Leaving directory `/oss/build/i686-pc-cygwin/newlib'
> make[2]: *** [all] Error 2
> make[2]: Leaving directory `/oss/build/i686-pc-cygwin/newlib'
> make[1]: *** [all-target-newlib] Error 2
> make[1]: Leaving directory `/oss/build'
> make: *** [all] Error 2
>
> - Kyle

I just discovered some additional information that may help get to the
bottom of this.  Not sure what made me think of this, but I decided to try
an older version of rsync.  If I run rsync 3.0.4-1 or 3.0.5-1, I experience
the problem.  However, when I run 2.6.9-2, the only other version of rsync I
can find built specifically for cygwin 1.7, the problem goes away and all
works fine?

By the way Corinna, in case I didn't say it before, thank you very much for
trying to help out and for the time you spend on this project!

- Kyle



--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple