Re: OpenJDK Community Code Review server rollout

[john.coomes]

Tim Bell (Tim.Bell@...) wrote to discuss@...:

> ...
> With that introduction, I would like to announce the initial rollout
> of an OpenJDK Community Code Review server.  The home page is
> http://cr.openjdk.java.net.
>
> Users having push access to the Mercurial repositories also have scp,
> sftp, and rsync access to the code review server.  Please refer to the
> link above for more information and an example showing how to push a
> webrev to the server.
>
> This is an initial step, and I am sure there will be many more.
> Please give the site a try and send feedback to
> web-discuss(at)openjdk.java.net.

First, many thanks, this is something we obviously need.  Second, scp
and sftp work fine, but rsync does not:

$ rsync -avv 4396719-fatstack jcoomes@...:
opening connection using: ssh cr.openjdk.java.net -l jcoomes rsync --server -vvlogDtpre.is .

insecure -e option not allowed.
This account is restricted by rssh.
Allowed commands: scp sftp rsync

If you believe this is in error, please contact your system
administrator.

rsync: connection unexpectedly closed (0 bytes received so far) [sender]
rsync error: error in rsync protocol data stream (code 12) at io.c(600) [sender=3.0.5]


I can live with scp, but rsync would be nice for partial transfers.
Has anyone else used rsync successfully?

-John

[tim.bell]

John Coomes wrote:

> scp and sftp work fine, but rsync does not:
>
> $ rsync -avv 4396719-fatstack jcoomes@...:
> opening connection using: ssh cr.openjdk.java.net -l jcoomes rsync --server -vvlogDtpre.is .
>
> insecure -e option not allowed.

<... snip ...>

> I can live with scp, but rsync would be nice for partial transfers.
> Has anyone else used rsync successfully?

Yes, I can rsync to cr.openjdk.java.net with no problems:

> % rsync -avv 6797463 tbell@...:
> opening connection using ssh cr.openjdk.java.net -l tbell rsync --server -vvlogDtpr .
> building file list ...
> done
> delta-transmission enabled
> 6797463/
> 6797463/webrev/
> 6797463/webrev/6797463.patch
> 6797463/webrev/TotalChangedLines
      [....]
> total: matches=2258  hash_hits=2258  false_alarms=0 data=0
>
> sent 12400 bytes  received 14634 bytes  18022.67 bytes/sec
> total size is 1568560  speedup is 58.02

This is from my office system using /usr/bin/rsync on OpenSolaris 2008.11.


I wonder where your command line got the extra flags "e.is"

Do you have an RSYNC_RSH environment variable set?


Tim

[john.coomes]

Tim Bell (Tim.Bell@...) wrote:

> John Coomes wrote:
>
> > scp and sftp work fine, but rsync does not:
> >
> > $ rsync -avv 4396719-fatstack jcoomes@...:
> > opening connection using: ssh cr.openjdk.java.net -l jcoomes rsync --server -vvlogDtpre.is .
> >
> > insecure -e option not allowed.
>
> <... snip ...>
>
> > I can live with scp, but rsync would be nice for partial transfers.
> > Has anyone else used rsync successfully?
>
> Yes, I can rsync to cr.openjdk.java.net with no problems:
>
> > % rsync -avv 6797463 tbell@...:
> > opening connection using ssh cr.openjdk.java.net -l tbell rsync --server -vvlogDtpr .
> > building file list ...
>       [....]
> > total: matches=2258  hash_hits=2258  false_alarms=0 data=0
> >
> > sent 12400 bytes  received 14634 bytes  18022.67 bytes/sec
> > total size is 1568560  speedup is 58.02
>
> This is from my office system using /usr/bin/rsync on OpenSolaris 2008.11.
>
>
> I wonder where your command line got the extra flags "e.is"
>
> Do you have an RSYNC_RSH environment variable set?

No, nothing with RSYNC in my environment:

$ env | egrep RSYNC
$

I also tried as a different user and after removing most of the
environment variables.

FWIW, it fails the same way for me on solaris 10 (my build of rsync
3.0.5 from source) and on fedora 9 with the standard rpm
(rsync-3.0.4-0.fc9.i386).  On fedora 9:

$ rsync --version
rsync  version 3.0.4  protocol version 30
Copyright (C) 1996-2008 by Andrew Tridgell, Wayne Davison, and others.
Web site: http://rsync.samba.org/
Capabilities:
    64-bit files, 64-bit inums, 32-bit timestamps, 64-bit long ints,
    socketpairs, hardlinks, symlinks, IPv6, batchfiles, inplace,
    append, ACLs, xattrs, iconv, symtimes

Grasping at straws ... could it be an rsync version problem?  What
version do you have, and what is on cr.openjdk.java.net?

-John

[tim.bell]

John Coomes wrote:

> FWIW, it fails the same way for me on solaris 10 (my build of rsync
> 3.0.5 from source) and on fedora 9 with the standard rpm
> (rsync-3.0.4-0.fc9.i386).  On fedora 9:
>
> $ rsync --version
> rsync  version 3.0.4  protocol version 30
> Copyright (C) 1996-2008 by Andrew Tridgell, Wayne Davison, and others.
> Web site: http://rsync.samba.org/
> Capabilities:
>     64-bit files, 64-bit inums, 32-bit timestamps, 64-bit long ints,
>     socketpairs, hardlinks, symlinks, IPv6, batchfiles, inplace,
>     append, ACLs, xattrs, iconv, symtimes
>
> Grasping at straws ... could it be an rsync version problem?  What
> version do you have, and what is on cr.openjdk.java.net?

"version problem" - John nailed it...


Both my office machine and cr.o.j.n are running OpenSolaris 2008.11, which has:

% /usr/bin/rsync --version
rsync  version 2.6.9  protocol version 29
Copyright (C) 1996-2006 by Andrew Tridgell, Wayne Davison, and others.
<http://rsync.samba.org/>
Capabilities: 64-bit files, socketpairs, hard links, symlinks, batchfiles,
               inplace, no IPv6, 64-bit system inums, 64-bit internal inums

rsync comes with ABSOLUTELY NO WARRANTY.  This is free software, and you
are welcome to redistribute it under certain conditions.  See the GNU
General Public Licence for details.



If your rsync is using a protocol version newer than 29, you need to add
"--protocol=29" to your command line.

I got on John's system and ran his build of rsync - it worked when I called for
protocol version 29:

> % /export/pub/SunOS-5-sparc/bin/rsync --protocol=29 -av 6797463 \
>     tbell@...:6797463.01
> Enter passphrase for key '/home/tbell/.ssh/id_dsa':
> building file list ... done
> created directory 6797463.01
> 6797463/
> 6797463/webrev/
> 6797463/webrev/6797463.patch
> 6797463/webrev/TotalChangedLines
  [...]
> sent 1572224 bytes  received 1086 bytes  349624.44 bytes/sec
> total size is 1568560  speedup is 1.00


I will add this information to the example on http://cr.openjdk.java.net/


Tim

[john.coomes]

I (John.Coomes@...) wrote:

> Tim Bell (Tim.Bell@...) wrote:
> > John Coomes wrote:
> >
> > > scp and sftp work fine, but rsync does not:
> > >
> > > $ rsync -avv 4396719-fatstack jcoomes@...:
> > > opening connection using: ssh cr.openjdk.java.net -l jcoomes rsync --server -vvlogDtpre.is .
> > >
> > > insecure -e option not allowed.
> >
> ...
> Grasping at straws ... could it be an rsync version problem?  What
> version do you have, and what is on cr.openjdk.java.net?

FYI, I tried rsync 2.6.9 on a different box in the lab and that
worked.  So it looks like an rsync version problem.  A solution that
allows 3.0.x clients would be nice, but I can work around it for now.

-John

[Mark Reinhold]

> Date: Tue, 10 Feb 2009 14:10:31 -0800
> From: tim.bell@...

> "version problem" - John nailed it...

Thanks for the detective work!

> ...
>
> I will add this information to the example on http://cr.openjdk.java.net/

Would it be better instead to put a newer build of rsync on cr.ojn?

Not in /usr/bin but in /usr/local/bin, though this assumes that there's
a way to tell the restricted shell to allow that program to be run.

- Mark

[tim.bell]

Mark Reinhold wrote:

> Would it be better instead to put a newer build of rsync on cr.ojn?

Yes

> Not in /usr/bin but in /usr/local/bin, though this assumes that there's
> a way to tell the restricted shell to allow that program to be run.

I will investigate that.  rssh is very strict about what it will allow users to
run, but we should be able to update to a newer version.

Tim

[john.coomes]

Tim Bell (Tim.Bell@...) wrote:

> John Coomes wrote:
> > ...
> > Grasping at straws ... could it be an rsync version problem?  What
> > version do you have, and what is on cr.openjdk.java.net?
>
> "version problem" - John nailed it...
>
>
> Both my office machine and cr.o.j.n are running OpenSolaris 2008.11, which has:
>
> % /usr/bin/rsync --version
> rsync  version 2.6.9  protocol version 29
> Copyright (C) 1996-2006 by Andrew Tridgell, Wayne Davison, and others.
>
> If your rsync is using a protocol version newer than 29, you need to add
> "--protocol=29" to your command line.
> ...
> I will add this information to the example on http://cr.openjdk.java.net/

Thanks!

-John

[Mark Wielaard]

On Wed, 2009-02-04 at 22:20 -0800, Tim Bell wrote:
> This is an initial step, and I am sure there will be many more.
> Please give the site a try and send feedback to
> web-discuss(at)openjdk.java.net.

Nice thanks.

Would it be possible to tell the server to give the patches type
text/plain instead of application/octet-stream like
webrev.invokedynamic.info does?

That would make integration with some programs easier for direct viewing
of the files.

Cheers,

Mark

[tim.bell]

Hi Mark:

> Would it be possible to tell the server to give the patches type
> text/plain instead of application/octet-stream like
> webrev.invokedynamic.info does?

Great idea, thanks.  I am working on making this better.

Tim