Issue with rsync 3.0.0 and iconv

[Sergi Baila]

Hello there,

I have a windows box (spanish locale, charset cp1252) which is backup to a linux server via rsync. Until now I've had problems with file names containing non us-ascii characters. Since the new stable version of rsync with support for iconv I've upgraded rsync on my linux (Debian) to 3.0.0 and also on my windows (cygwin, compiled from source).

It works quite right, getting the proper treatment of file names. But after a while running I reach a point on the filesystem with this error:

2008/03/23 19:27:37 [3760] received request to transfer non-regular file: 86300 [sender]
2008/03/23 19:27:37 [3760] rsync error: protocol incompatibility (code 2) at rsync.c(298) [sender=3.0.0]

This is the command line from the linux box:

rsync -azP --ignore-errors --password-file=/etc/backup/pcmama.pwd --delete-after --timeout=300 --iconv=.  --delete-excluded --exclude-from=/etc/backup/exclude.pcmama bkp@...::docs /bkps/bkppcmama/docs/

I've found this post but I've been unable to find any bug related (perhaps I haven't searched enough).
http://www.nabble.com/rsync-error:-protocol-incompatibility-(code-2)-at-rsync.c,-using---iconv%3D.-and-(code-2)-and-rsync-3.0.0pre8-td15341068.html

Any idea? Is this a bug?

Thank you.

--
www.sargue.net
--
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Wayne Davison-2]

On Sun, Mar 23, 2008 at 07:54:09PM +0100, Sergi Baila wrote:
> 2008/03/23 19:27:37 [3760] received request to transfer non-regular file: 86300 [sender]

This means either that file-list number 86300 is not matching up between
the sender and the receiver, or that the data stream is somehow out of
whack, so the sender misinterpreted what the generator was asking it to
do.  I have not seen this failure, so I'll need some more information to
diagnose what is going wrong.  An easy way to start to narrow it down is
to run the rsync command with 3 -v options.  For each transfer, you'll
see a set of verbose messages, such as:

recv_generator(byteorder.h,53)
generating and sending sums for 53
send_files(53, ../rsync/byteorder.h)

This shows us that the two sides are in sync on both the # and the name.
If you get a failure, send me the tail end of the -vvv output and it
should reveal how things have gone wrong.

..wayne..
--
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Sergi Baila]

Thanks for your answer Wayne,

I've run the command with -vvv and got a 13 megs output. ;-)

The file bzipped is just under 1M, I can send to you directly by email if you want.

There seems to be certain unsync between the moment this line appears:

received request to transfer non-regular file: 86290 [sender]

(line number 147370 of output)

And the final part (149396 total lines), which is:

rsync: connection unexpectedly closed (2245617 bytes received so far) [generator]
rsync error: error in rsync protocol data stream (code 12) at io.c(600) [generator=3.0.0]
_exit_cleanup(code=12, file=io.c, line=600): about to call exit(12)

On Mon, Mar 24, 2008 at 4:14 AM, Wayne Davison <wayned@...> wrote:

On Sun, Mar 23, 2008 at 07:54:09PM +0100, Sergi Baila wrote:
> 2008/03/23 19:27:37 [3760] received request to transfer non-regular file: 86300 [sender]

This means either that file-list number 86300 is not matching up between
the sender and the receiver, or that the data stream is somehow out of
whack, so the sender misinterpreted what the generator was asking it to
do.  I have not seen this failure, so I'll need some more information to
diagnose what is going wrong.  An easy way to start to narrow it down is
to run the rsync command with 3 -v options.  For each transfer, you'll
see a set of verbose messages, such as:

recv_generator(byteorder.h,53)
generating and sending sums for 53
send_files(53, ../rsync/byteorder.h)

This shows us that the two sides are in sync on both the # and the name.
If you get a failure, send me the tail end of the -vvv output and it
should reveal how things have gone wrong.

..wayne..

--
www.sargue.net
--
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Wayne Davison-2]

On Sun, Mar 23, 2008 at 07:54:09PM +0100, Sergi Baila wrote:
> 2008/03/23 19:27:37 [3760] received request to transfer non-regular file: 86300 [sender]

OK: thanks to Sergi's off-list debug info, I've fixed the problem.  It
was caused by a name that failed to convert on the sending side, and
that failure caused the file-list on the sender and the receiver to get
out of sync.  The new code properly discards the file from the sender's
file list (marking the transfer as incomplete), and should otherwise
complete normally.

The latest git-repository version and nightly tar file have this fix
committed.

..wayne..
--
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Sergi Baila]

Great, that was fast!

Thanks a lot for your work Wayne. I'll test it asap.

On Tue, Mar 25, 2008 at 6:58 PM, Wayne Davison <wayned@...> wrote:

On Sun, Mar 23, 2008 at 07:54:09PM +0100, Sergi Baila wrote:

> 2008/03/23 19:27:37 [3760] received request to transfer non-regular file: 86300 [sender]

OK: thanks to Sergi's off-list debug info, I've fixed the problem.  It
was caused by a name that failed to convert on the sending side, and
that failure caused the file-list on the sender and the receiver to get
out of sync.  The new code properly discards the file from the sender's
file list (marking the transfer as incomplete), and should otherwise
complete normally.

The latest git-repository version and nightly tar file have this fix
committed.

..wayne..

--
www.sargue.net
--
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Kurt Martinsen 543]

Wayne Davison-2 wrote:

The latest git-repository version and nightly tar file have this fix
committed.

Hi,

I have the same problem:
received request to transfer non-regular file: 88183 [sender]
rsync error: protocol incompatibility (code 2) at rsync.c(298) [sender=3.0.0]

Is it just the one file that gets discarded or is it the whole process that stops at this point?

Any plans on releasing a new binary version including this fix anytime soon?

Regards,

Kurt

[Matt McCutchen-7]

On Tue, 2008-04-01 at 01:59 -0700, Kurt Martinsen 543 wrote:
> I have the same problem:
> received request to transfer non-regular file: 88183 [sender]
> rsync error: protocol incompatibility (code 2) at rsync.c(298)
> [sender=3.0.0]
>
> Is it just the one file that gets discarded or is it the whole process that
> stops at this point?

For protocol-corruption errors like this one, the whole process stops.

> Any plans on releasing a new binary version including this fix anytime soon?

Wayne plans to release rsync 3.0.1 in the next few days.  Until then,
you can use a nightly snapshot or exclude the offending file.  (This is
a major, easily overlooked advantage of rsync's filters: if one file
gives rsync trouble, one can simply exclude it so that the rest of the
run can proceed.)

Matt

--
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html