Patch for SSL in TSI and XNJS
|
View:
New views
2 Messages
—
Rating Filter:
Alert me
|
 |
Patch for SSL in TSI and XNJS
by Clement COUSSIRAT
::
Rate this Message:
Dear UNICORE developers,
I am currently working at CEA on adding some HA capabilities to UNICORE, I thought adding SSL support on all XNJS<->TSI communications could be a good thing to increase security and deactivate TSI's IP checks (which would allow the connection of several XNJS on one TSI).I've made two patches, one for the TSI and the other for the XNJS. TSI's part of the patch applies to the SVN trunk version. It activates SSL if the keystore file is specified in tsi.properties. Keystore and truststore must be in pem format, when an XNJS connects, its certificate is checked with the CA cert. If this cert is correctly signed and if it's present in the truststore, the TSI allows the connection. In SSL mode, the XNJS IP checks are deactivated. An other thing added in this patch is the XNJS port negotiation. If the XNJS port read in the tsi conf file is incorrect or not specified, the TSI will try to read it from the XNJS shepherd connection. Technically, the patch uses IO::Socket::SSL perl module. This module is actively maintained and is present on the most of package managers. To simplify the code, the clear listening socket is created with IO::Socket which has the same methods as IO::Socket::SSL. To avoid changes in the rest of the code, the socket's filehandles are extracted from IO::Socket objects and used as classic sockets. The XNJS patch is based on xnjs-core-1.2.0 available in UNICORE-6.2.0, the connections to the TSI presents in "TSIConnectionFactory" have been split into 3 files. Now, "TSIConnectionFactory" uses the abstract class "TSISocket" do call accept() and signalShepherd methods. The TSISocket object is instantiated using a class name read in the xnjs_legacy.xml conf file. This parameter can be set to 2 class names. The first is "TSISocketPlain" which is the default value. This class keeps the communications methods unchanged. The second class is "TSISocketSSL" which add SSL support. It uses the same trustsfiles and keyfiles as wsrflite. The transmission of the NJS port number has also been added in the shepherd. To activate SSL, the following parameters must be set in tsi.properties: tsi.keystore=/my/keystore.pem tsi.keypass=keystore-password tsi.truststore=/my/truststore.pem and in xnjs_legacy.xml: <eng:Property name="TSISocket.class" value="de.fzj.unicore.xnjs.legacy.TSISocketSSL"/> I would be grateful to you if you could send me a short review about these patches and about upcoming updates. Regards, Clément Index: tsi/sun_gridengine/tsi =================================================================== --- tsi/sun_gridengine/tsi (revision 5222) +++ tsi/sun_gridengine/tsi (working copy) @@ -56,7 +56,27 @@ $main::njs_machine = shift || "set name for NJS machine"; $main::njs_port = shift || "set port for NJS machine"; $main::my_port = shift || "set port for this (the TSID)"; +$main::keystore = shift; +$main::truststore = shift; +# Keypass is the password used to encrypt the private key +$main::keypass = $ENV{KEYPASS}; +if ($main::keystore ne q{}) { + $main::ssl = 1; + die "Invalid Keystore filename: $main::keystore." + unless -f $main::keystore; + if ($main::truststore ne q{}) { + if (! -f $main::truststore) { + warn "Invalid Truststore filename: $main::truststore, using Keystore file."; + $main::truststore = $main::keystore; + } + } + else { + warn "Trustore filename not specified, using Keystore file."; + $main::truststore = $main::keystore; + } +} + # Where to find the SGE commands used by the TSI my $sge_dir = "/opt/gridengine/bin/lx26-x86"; Index: tsi/linux_slurm/tsi =================================================================== --- tsi/linux_slurm/tsi (revision 5222) +++ tsi/linux_slurm/tsi (working copy) @@ -57,7 +57,27 @@ $main::njs_machine = shift || "set name for NJS machine"; $main::njs_port = shift || "set port for NJS machine"; $main::my_port = shift || "set port for this (the TSID)"; +$main::keystore = shift; +$main::truststore = shift; +# Keypass is the password used to encrypt the private key +$main::keypass = $ENV{KEYPASS}; +if ($main::keystore ne q{}) { + $main::ssl = 1; + die "Invalid Keystore filename: $main::keystore." + unless -f $main::keystore; + if ($main::truststore ne q{}) { + if (! -f $main::truststore) { + warn "Invalid Truststore filename: $main::truststore, using Keystore file."; + $main::truststore = $main::keystore; + } + } + else { + warn "Trustore filename not specified, using Keystore file."; + $main::truststore = $main::keystore; + } +} + # Where to find the SLURM commands used by the TSI my $slurm_dir="/usr/bin"; Index: tsi/linux_condor/tsi =================================================================== --- tsi/linux_condor/tsi (revision 5222) +++ tsi/linux_condor/tsi (working copy) @@ -18,7 +18,27 @@ $main::njs_machine = shift || "set name for NJS machine"; $main::njs_port = shift || "set port for NJS machine"; $main::my_port = shift || "set port for this (the TSID)"; +$main::keystore = shift; +$main::truststore = shift; +# Keypass is the password used to encrypt the private key +$main::keypass = $ENV{KEYPASS}; +if ($main::keystore ne q{}) { + $main::ssl = 1; + die "Invalid Keystore filename: $main::keystore." + unless -f $main::keystore; + if ($main::truststore ne q{}) { + if (! -f $main::truststore) { + warn "Invalid Truststore filename: $main::truststore, using Keystore file."; + $main::truststore = $main::keystore; + } + } + else { + warn "Trustore filename not specified, using Keystore file."; + $main::truststore = $main::keystore; + } +} + # Condor install directory my $condor_bin_dir = "/usr/local/condor/bin"; Index: tsi/linux_lsf/tsi =================================================================== --- tsi/linux_lsf/tsi (revision 5222) +++ tsi/linux_lsf/tsi (working copy) @@ -61,7 +61,27 @@ $main::njs_machine = shift || "set name for NJS machine"; $main::njs_port = shift || "set port for NJS machine"; $main::my_port = shift || "set port for this (the TSID)"; +$main::keystore = shift; +$main::truststore = shift; +# Keypass is the password used to encrypt the private key +$main::keypass = $ENV{KEYPASS}; +if ($main::keystore ne q{}) { + $main::ssl = 1; + die "Invalid Keystore filename: $main::keystore." + unless -f $main::keystore; + if ($main::truststore ne q{}) { + if (! -f $main::truststore) { + warn "Invalid Truststore filename: $main::truststore, using Keystore file."; + $main::truststore = $main::keystore; + } + } + else { + warn "Trustore filename not specified, using Keystore file."; + $main::truststore = $main::keystore; + } +} + # Where to find the LSF commands used by the TSI my $lsf_dir = "/usr/local/lsf/5.1/sgi6.5.8/bin"; Index: tsi/SHARED/Initialisation.pm =================================================================== --- tsi/SHARED/Initialisation.pm (revision 5222) +++ tsi/SHARED/Initialisation.pm (working copy) @@ -10,6 +10,9 @@ use Socket; use Reporting qw(initial_report report_and_die); +use IO::Socket::SSL; +use IO::Socket; + use strict; sub GENTLY { @@ -41,10 +44,15 @@ $njs_machine = $1; } - my $njs_port = $main::njs_port; - if ($njs_port =~ m/(.*)/s ) { + my $njs_port; + # NJS port is now optionnal, if it isn't specified or + # invalid, it will be read on the socket from the NJS + if (defined $main::njs_port) { + $njs_port = $main::njs_port; + if ($njs_port =~ m/(.*)/s ) { $njs_port = $1; - } + } + } my $my_port = $main::my_port; if ($my_port =~ m/(.*)/s ) { @@ -52,14 +60,14 @@ } my $proto = getprotobyname('tcp'); - - if($njs_port =~ /\D/) {$njs_port = getservbyname($njs_port, 'tcp')}; - die "No NJS port\n" unless $njs_port; - - my $iaddr = inet_aton($njs_machine); - my $ip_address = inet_ntoa($iaddr); - my $paddr = sockaddr_in($njs_port, $iaddr); - + my $iaddr; + my $ip_address; + my $paddr; + if (!defined $main::ssl) { + $iaddr = inet_aton($njs_machine); + $ip_address = inet_ntoa($iaddr); + } + my $old_fh = select(STDOUT); $| = 1; select($old_fh); @@ -68,82 +76,188 @@ $| = 1; select($old_fh); - # Open a server socket to listen for new TSI requests from the NJS - socket(FROM_NJS, PF_INET, SOCK_STREAM, $proto); - setsockopt(FROM_NJS, SOL_SOCKET, SO_REUSEADDR, 1); - my $from_njs_addr = sockaddr_in($my_port, INADDR_ANY); - bind(FROM_NJS, $from_njs_addr) or die "Could not bind to port $my_port: $!\n"; - listen(FROM_NJS, SOMAXCONN) or die "Could not listen on port $my_port: $!\n"; - + # Open a server socket to listen for new TSI requests from the NJS + my $FROM_NJS; + if (defined $main::ssl) { + # If using SSL connections + if (!($FROM_NJS = IO::Socket::SSL->new( Listen => SOMAXCONN, + LocalPort => $my_port, + Proto => 'tcp', + ReuseAddr => 1, + SSL_use_cert => 1, + SSL_verify_mode => 0x01, + SSL_key_file => $main::keystore, + SSL_cert_file => $main::keystore, + SSL_ca_file => $main::truststore, + SSL_passwd_cb => sub {$main::keypass}, + )) ) { + warn "unable to create socket: ", &IO::Socket::SSL::errstr, "\n"; + exit(1); + } + } + else { + # If using CLASSIC connections + # Use of IO::Socket instead of classic socket API + # in order to unify the use of SSL and plain Socket + # the main changes in the code are that FROM_NJS is now + # used as a reference to a filehandle and not directly as + # a filehandle. + if (!($FROM_NJS = IO::Socket::INET->new( Listen => SOMAXCONN, + LocalPort => $my_port, + Proto => 'tcp', + ReuseAddr => 1, + )) ) { + warn "unable to create socket: ", &IO::Socket::INET::errstr, "\n"; + exit(1); + } + } + my ($other_end, $other_iaddr, $other_port, $pid); - while(accept(NJS_CLIENT, FROM_NJS)) { - - # Got connection, from NJS? - - if($other_end = getpeername(NJS_CLIENT)) { - - ($other_port, $other_iaddr) = unpack_sockaddr_in($other_end); + my $NJS_CLIENT; + # accept() system call will create plain connection instead of + # SSL connection. Use instead $FROM_NJS->accept(), it works + # with both IO::Socket and IO::Socket::SSL sockets + while ( $NJS_CLIENT = $FROM_NJS->accept() ) { + if (!cert_is_trusted($NJS_CLIENT, $main::truststore)) { + # if the peer cert isn't in the truststore, + # close the Socket and go back to accept + initial_report("NJS is using an unknown cert"); + $NJS_CLIENT->close(); + next; + } + # Got connection, from NJS? + if($other_end = getpeername($NJS_CLIENT)) { + ($other_port, $other_iaddr) = unpack_sockaddr_in($other_end); my $other_ip_address = inet_ntoa($other_iaddr); + if(defined $main::ssl || ($other_ip_address eq $ip_address)) { - if($other_ip_address eq $ip_address) { - # V4 Read message from NJS to see if we need to go - my $message = <NJS_CLIENT>; + my $message = <$NJS_CLIENT>; $message =~ /get down shep\n/ && do { initial_report("NJS requested a stop"); GENTLY(); }; # V4 END - # Write anything to tell NJS that this is OK - $old_fh = select(NJS_CLIENT); + # if $njs_port is not a number, look in /etc/services + if($njs_port =~ /\D/) {$njs_port = getservbyname($njs_port, 'tcp')}; + + if (!$njs_port) { + # if $njs_port keeps invalid, try to read it from the NJS + initial_report("Waiting NJS port"); + $njs_port = <$NJS_CLIENT>; + if ($njs_port =~ m/^(.+)/m ) { + $njs_port = $1; + } + # if NJS sends a name, try to get port with /etc/services + if($njs_port =~ /\D/) {$njs_port = getservbyname($njs_port, 'tcp')}; + die "No NJS port\n" unless $njs_port; + } + + # Write anything to tell NJS that this is OK + $old_fh = select($NJS_CLIENT); $| = 1; select($old_fh); - print NJS_CLIENT "OK\n"; - close(NJS_CLIENT); + print $NJS_CLIENT "OK\n"; + close($NJS_CLIENT); # -------------------------------------------------------------------- # Open a pair of sockets to the NJS, one to carry commands (text) # and the other to carry data (files read and written). - initial_report("Contacting the NJS on <$njs_machine> at port number <$njs_port>"); + initial_report("Contacting the NJS on <$other_ip_address> at port number <$njs_port>"); + if (defined $main::ssl) { + $main::CMD_SSL = IO::Socket::SSL->new (PeerAddr => $other_ip_address, + PeerPort => $njs_port, + Proto => 'tcp', + SSL_use_cert => 1, + SSL_verify_mode => 0x01, + SSL_key_file => $main::keystore, + SSL_cert_file => $main::keystore, + SSL_ca_file => $main::truststore, + SSL_passwd_cb => sub {$main::keypass}, + ) + || report_and_die("unable to create SSL command socket: ". &IO::Socket::SSL::errstr . "\n"); - # Open the commands channel, wait here for the NJS to start listening - socket(main::CMD_SOCK, PF_INET, SOCK_STREAM, $proto) or report_and_die("command socket: $!"); - while(! connect(main::CMD_SOCK, $paddr) ) { - initial_report("Waiting for NJS at $njs_machine:$njs_port to come up (last connect attempt said: $!)"); - sleep 3; + $main::DATA_SSL = IO::Socket::SSL->new (PeerAddr => $other_ip_address, + PeerPort => $njs_port, + Proto => 'tcp', + SSL_use_cert => 1, + SSL_verify_mode => 0x01, + SSL_key_file => $main::keystore, + SSL_cert_file => $main::keystore, + SSL_ca_file => $main::truststore, + SSL_passwd_cb => sub {$main::keypass}, + ) + || report_and_die("unable to create SSL data socket: ". &IO::Socket::SSL::errstr . "\n"); + + + # Ensure that output is flushed + $old_fh = select($main::CMD_SSL); + $| = 1; + select($old_fh); + + $old_fh = select($main::DATA_SSL); + $| = 1; + select($old_fh); + + } + else { + # Open the commands channel, wait here for the NJS to start listening + $paddr = sockaddr_in($njs_port, $iaddr); + socket(main::CMD_SOCK, PF_INET, SOCK_STREAM, $proto) or report_and_die("command socket: $!"); + while(! connect(main::CMD_SOCK, $paddr) ) { + initial_report("Waiting for NJS at $other_ip_address:$njs_port to come up (last connect attempt said: $!)"); + sleep 3; socket(main::CMD_SOCK, PF_INET, SOCK_STREAM, $proto) or report_and_die("command socket: $!"); } + # Open the data channel + socket(main::DATA_SOCK, PF_INET, SOCK_STREAM, $proto) or report_and_die("data socket: $!"); + connect(main::DATA_SOCK, $paddr) or report_and_die("data connect: $!"); - # Open the data channel - socket(main::DATA_SOCK, PF_INET, SOCK_STREAM, $proto) or report_and_die("data socket: $!"); - connect(main::DATA_SOCK, $paddr) or report_and_die("data connect: $!"); + + # Ensure that output is flushed + $old_fh = select(main::CMD_SOCK); + $| = 1; + select($old_fh); + + $old_fh = select(main::DATA_SOCK); + $| = 1; + select($old_fh); + + } + + - # Ensure that output is flushed - $old_fh = select(main::CMD_SOCK); - $| = 1; - select($old_fh); - - $old_fh = select(main::DATA_SOCK); - $| = 1; - select($old_fh); - initial_report("Connection to NJS established."); # -------------------------------------------------------------------- # Now fork the child which becomes the requested TSI if( $pid = fork ) { - # Parent, do not want the NJS sockets - close(main::CMD_SOCK); - close(main::DATA_SOCK); - next; # go back to accept + # Parent, do not want the NJS sockets + if (defined $main::ssl) { + # close Socket but keep the SSL session opened for the child + $main::CMD_SSL->close(SSL_no_shutdown => 1); + $main::DATA_SSL->close(SSL_no_shutdown => 1); + } + else { + close(main::CMD_SOCK); + close(main::DATA_SOCK); + } + next; # go back to accept } elsif(defined $pid) { # $pid is actually 0 in child # Child, do not want the listener to NJS connections close(FROM_NJS); + if (defined $main::ssl) { + # this lines translate reference to sockets into + # typeglobs as they are used in the rest of the code + *main::CMD_SOCK = $main::CMD_SSL; + *main::DATA_SOCK = $main::DATA_SSL; + } + # Clear the signal handlers, CHLD screws script execution $SIG{CHLD} = 'DEFAULT'; $SIG{TERM} = 'DEFAULT'; @@ -175,12 +289,12 @@ } else { # IP addresses differ, ignore - close(NJS_CLIENT); + close($NJS_CLIENT); initial_report("The other end of a start TSI request does not come from the NJS, ignoring: $other_ip_address"); } } else { - close(NJS_CLIENT); + close($NJS_CLIENT); initial_report("Could not identify other end of a start TSI request, ignoring."); } } @@ -189,6 +303,26 @@ die "Failed while listening for NJS connections: $!\n"; } + +sub cert_is_trusted { + my ($sock, $trustfile) = @_; + # get an IO::Socket::SSL and a PEM file + # return 1 if the peer certificate is present in the PEM file. + + my $cert = $sock->peer_certificate(); + my $peer = Net::SSLeay::PEM_get_string_X509($cert); + + open(F, $trustfile); + my $trusted; + # check if the peer certificate is in the truststore + while (<F>) { + $trusted = q{} if (/BEGIN CERTIFICATE/); # reset $trusted string if a new certificate begin + $trusted .= $_ if (/BEGIN CERTIFICATE/ .. /END CERTIFICATE/); # copy the new certificate in $trusted + close F && return 1 if (/END CERTIFICATE/ && $peer eq $trusted); # when the trusted cert is read, return 1 if they match + } + close F; + return 0; +} # # Copyright (c) Fujitsu Ltd 2000 - 2004 # Index: tsi/aix_ll/tsi =================================================================== --- tsi/aix_ll/tsi (revision 5222) +++ tsi/aix_ll/tsi (working copy) @@ -59,7 +59,27 @@ $main::njs_machine = shift || "set name for NJS machine"; $main::njs_port = shift || "set port for NJS machine"; $main::my_port = shift || "set port for this (the TSID)"; +$main::keystore = shift; +$main::truststore = shift; +# Keypass is the password used to encrypt the private key +$main::keypass = $ENV{KEYPASS}; +if ($main::keystore ne q{}) { + $main::ssl = 1; + die "Invalid Keystore filename: $main::keystore." + unless -f $main::keystore; + if ($main::truststore ne q{}) { + if (! -f $main::truststore) { + warn "Invalid Truststore filename: $main::truststore, using Keystore file."; + $main::truststore = $main::keystore; + } + } + else { + warn "Trustore filename not specified, using Keystore file."; + $main::truststore = $main::keystore; + } +} + # Where to find the LL commands used by the TSI my $ll_dir = "/usr/lpp/LoadL/full/bin/"; Index: tsi/ccs/tsi =================================================================== --- tsi/ccs/tsi (revision 5222) +++ tsi/ccs/tsi (working copy) @@ -58,7 +58,27 @@ $main::njs_machine = shift || "set name for NJS machine"; $main::njs_port = shift || "set port for NJS machine"; $main::my_port = shift || "set port for this (the TSID)"; +$main::keystore = shift; +$main::truststore = shift; +# Keypass is the password used to encrypt the private key +$main::keypass = $ENV{KEYPASS}; +if ($main::keystore ne q{}) { + $main::ssl = 1; + die "Invalid Keystore filename: $main::keystore." + unless -f $main::keystore; + if ($main::truststore ne q{}) { + if (! -f $main::truststore) { + warn "Invalid Truststore filename: $main::truststore, using Keystore file."; + $main::truststore = $main::keystore; + } + } + else { + warn "Trustore filename not specified, using Keystore file."; + $main::truststore = $main::keystore; + } +} + # Where to find the CCS commands used by the TSI my $ccs_bin_dir = "/usr/local/ccs/bin/LINUX"; Index: tsi/NOBATCH/tsi.HPUX =================================================================== --- tsi/NOBATCH/tsi.HPUX (revision 5222) +++ tsi/NOBATCH/tsi.HPUX (working copy) @@ -51,7 +51,28 @@ $main::njs_machine = shift || "set name for NJS machine"; $main::njs_port = shift || "set port for NJS machine"; $main::my_port = shift || "set port for this (the TSID)"; +$main::keystore = shift; +$main::truststore = shift; +# Keypass is the password used to encrypt the private key +$main::keypass = $ENV{KEYPASS}; +if ($main::keystore ne q{}) { + $main::ssl = 1; + die "Invalid Keystore filename: $main::keystore." + unless -f $main::keystore; + if ($main::truststore ne q{}) { + if (! -f $main::truststore) { + warn "Invalid Truststore filename: $main::truststore, using Keystore file."; + $main::truststore = $main::keystore; + } + } + else { + warn "Trustore filename not specified, using Keystore file."; + $main::truststore = $main::keystore; + } +} + + # Get all the processes on the system, with sufficient information # to be able to derive the Unicore ones $main::qstat_cmd = "ksh -c 'UNIX95=1 ps -e -ostate,args'"; # show _all_ jobs submitted by TSI Index: tsi/NOBATCH/tsi.MACOSX =================================================================== --- tsi/NOBATCH/tsi.MACOSX (revision 5222) +++ tsi/NOBATCH/tsi.MACOSX (working copy) @@ -51,7 +51,27 @@ $main::njs_machine = shift || "set name for NJS machine"; $main::njs_port = shift || "set port for NJS machine"; $main::my_port = shift || "set port for this (the TSID)"; +$main::keystore = shift; +$main::truststore = shift; +# Keypass is the password used to encrypt the private key +$main::keypass = $ENV{KEYPASS}; +if ($main::keystore ne q{}) { + $main::ssl = 1; + die "Invalid Keystore filename: $main::keystore." + unless -f $main::keystore; + if ($main::truststore ne q{}) { + if (! -f $main::truststore) { + warn "Invalid Truststore filename: $main::truststore, using Keystore file."; + $main::truststore = $main::keystore; + } + } + else { + warn "Trustore filename not specified, using Keystore file."; + $main::truststore = $main::keystore; + } +} + # Get all the processes on the system, with sufficient information # to be able to derive the Unicore ones $main::qstat_cmd = "/bin/ps -e -ostate,command"; # show _all_ jobs submitted by TSI Index: tsi/NOBATCH/tsi.IRIX =================================================================== --- tsi/NOBATCH/tsi.IRIX (revision 5222) +++ tsi/NOBATCH/tsi.IRIX (working copy) @@ -51,7 +51,27 @@ $main::njs_machine = shift || "set name for NJS machine"; $main::njs_port = shift || "set port for NJS machine"; $main::my_port = shift || "set port for this (the TSID)"; +$main::keystore = shift; +$main::truststore = shift; +# Keypass is the password used to encrypt the private key +$main::keypass = $ENV{KEYPASS}; +if ($main::keystore ne q{}) { + $main::ssl = 1; + die "Invalid Keystore filename: $main::keystore." + unless -f $main::keystore; + if ($main::truststore ne q{}) { + if (! -f $main::truststore) { + warn "Invalid Truststore filename: $main::truststore, using Keystore file."; + $main::truststore = $main::keystore; + } + } + else { + warn "Trustore filename not specified, using Keystore file."; + $main::truststore = $main::keystore; + } +} + # Get all the processes on the system, with sufficient information # to be able to derive the Unicore ones $main::qstat_cmd = "ps -e -ostate,args"; # show _all_ jobs submitted by TSI Index: tsi/NOBATCH/tsi.SOLARIS =================================================================== --- tsi/NOBATCH/tsi.SOLARIS (revision 5222) +++ tsi/NOBATCH/tsi.SOLARIS (working copy) @@ -51,7 +51,27 @@ $main::njs_machine = shift || "set name for NJS machine"; $main::njs_port = shift || "set port for NJS machine"; $main::my_port = shift || "set port for this (the TSID)"; +$main::keystore = shift; +$main::truststore = shift; +# Keypass is the password used to encrypt the private key +$main::keypass = $ENV{KEYPASS}; +if ($main::keystore ne q{}) { + $main::ssl = 1; + die "Invalid Keystore filename: $main::keystore." + unless -f $main::keystore; + if ($main::truststore ne q{}) { + if (! -f $main::truststore) { + warn "Invalid Truststore filename: $main::truststore, using Keystore file."; + $main::truststore = $main::keystore; + } + } + else { + warn "Trustore filename not specified, using Keystore file."; + $main::truststore = $main::keystore; + } +} + # Get all the processes on the system, with sufficient information # to be able to derive the Unicore ones $main::qstat_cmd = "ps -e -os,args"; # show _all_ jobs submitted by TSI Index: tsi/NOBATCH/tsi.LINUX =================================================================== --- tsi/NOBATCH/tsi.LINUX (revision 5222) +++ tsi/NOBATCH/tsi.LINUX (working copy) @@ -51,7 +51,27 @@ $main::njs_machine = shift || "set name for NJS machine"; $main::njs_port = shift || "set port for NJS machine"; $main::my_port = shift || "set port for this (the TSID)"; +$main::keystore = shift; +$main::truststore = shift; +# Keypass is the password used to encrypt the private key +$main::keypass = $ENV{KEYPASS}; +if ($main::keystore ne q{}) { + $main::ssl = 1; + die "Invalid Keystore filename: $main::keystore." + unless -f $main::keystore; + if ($main::truststore ne q{}) { + if (! -f $main::truststore) { + warn "Invalid Truststore filename: $main::truststore, using Keystore file."; + $main::truststore = $main::keystore; + } + } + else { + warn "Trustore filename not specified, using Keystore file."; + $main::truststore = $main::keystore; + } +} + # Get all the processes on the system, with sufficient information # to be able to derive the Unicore ones $main::qstat_cmd = "ps -e -os,args"; # show _all_ jobs submitted by TSI Index: tsi/linux_torque/tsi =================================================================== --- tsi/linux_torque/tsi (revision 5222) +++ tsi/linux_torque/tsi (working copy) @@ -55,7 +55,27 @@ $main::njs_machine = shift || "zam470.zam.kfa-juelich.de"; $main::njs_port = shift || "8884"; $main::my_port = shift || "8885"; +$main::keystore = shift; +$main::truststore = shift; +# Keypass is the password used to encrypt the private key +$main::keypass = $ENV{KEYPASS}; +if ($main::keystore ne q{}) { + $main::ssl = 1; + die "Invalid Keystore filename: $main::keystore." + unless -f $main::keystore; + if ($main::truststore ne q{}) { + if (! -f $main::truststore) { + warn "Invalid Truststore filename: $main::truststore, using Keystore file."; + $main::truststore = $main::keystore; + } + } + else { + warn "Trustore filename not specified, using Keystore file."; + $main::truststore = $main::keystore; + } +} + # Where to find the PBS commands used by the TSI my $pbs_bin_dir = "/usr/bin"; Index: conf/tsi.properties =================================================================== --- conf/tsi.properties (revision 5222) +++ conf/tsi.properties (working copy) @@ -21,3 +21,12 @@ # Logging directory to be used, by default log to base TSI dir # tsi.logdir=${INSTALL_PATH}/tsi/logs + +# +# SSL parameters +# Keystore must contain the private TSI key and certificate +# Trustore must contain the certificate of the CA + +#tsi.keystore=/root/keystore.pem +#tsi.keypass=the!njs +#tsi.truststore=/root/keystore.pem Index: bin/start_tsi =================================================================== --- bin/start_tsi (revision 5222) +++ bin/start_tsi (working copy) @@ -140,6 +140,9 @@ NJS_HOST="`perl -ne 'if(/^\s*tsi.njs_machine\s*=\s*(\S+)/) {print $1}' $TSI_PROPS`" NJS_PORT="`perl -ne 'if(/^\s*tsi.njs_port\s*=\s*(\S+)/) {print $1}' $TSI_PROPS`" MY_PORT="`perl -ne 'if(/^\s*tsi.my_port\s*=\s*(\S+)/) {print $1}' $TSI_PROPS`" +KEYSTORE="`perl -ne 'if(/^\s*tsi.keystore\s*=\s*(\S+)/) {print $1}' $TSI_PROPS`" +export KEYPASS="`perl -ne 'if(/^\s*tsi.keypass\s*=\s*(\S+)/) {print $1}' $TSI_PROPS`" +TRUSTSTORE="`perl -ne 'if(/^\s*tsi.truststore\s*=\s*(\S+)/) {print $1}' $TSI_PROPS`" # check log dir LOG_DIR="`perl -ne 'if(/^\s*tsi.logdir\s*=\s*(\S+)/) {print $1}' $TSI_PROPS`" @@ -158,6 +161,8 @@ echo "Found XNJS port $NJS_PORT" echo "Found shepherd port $MY_PORT" echo "Found log directory $LOG_DIR" +echo "Found Keystore File $KEYSTORE" +echo "Found Keystore File $TRUSTSTORE" echo "" date=`date +_%Y_%m_%d` @@ -176,8 +181,8 @@ echo "perl -d $TSI/tsi $NJS_HOST $NJS_PORT $MY_PORT" perl -d $TSI/tsi $NJS_HOST $NJS_PORT $MY_PORT else - echo "nohup perl $TSI/tsi $NJS_HOST $NJS_PORT $MY_PORT > $tsilog 2>&1 &" - nohup perl $TSI/tsi $NJS_HOST $NJS_PORT $MY_PORT > $tsilog 2>&1 & + echo "nohup perl $TSI/tsi $NJS_HOST $NJS_PORT $MY_PORT $KEYSTORE > $tsilog 2>&1 &" + nohup perl "$TSI/tsi" "$NJS_HOST" "$NJS_PORT" "$MY_PORT" "$KEYSTORE" "$TRUSTSTORE"> $tsilog 2>&1 & echo $! >> $TSI_CONF/LAST_TSI_PIDS $TSI_BIN/find_pids $TSI_CONF Index: src/main/java/de/fzj/unicore/xnjs/legacy/TSIConnectionFactory.java =================================================================== --- src/main/java/de/fzj/unicore/xnjs/legacy/TSIConnectionFactory.java (revision 5220) +++ src/main/java/de/fzj/unicore/xnjs/legacy/TSIConnectionFactory.java (working copy) @@ -35,7 +35,6 @@ import java.io.IOException; import java.io.InterruptedIOException; import java.net.InetAddress; -import java.net.ServerSocket; import java.net.Socket; import java.util.ArrayList; import java.util.List; @@ -62,10 +61,12 @@ private final List<TSIConnection> pool=new ArrayList<TSIConnection>(); private InetAddress source_addr=null; - private ServerSocket server=null; + private TSISocket server=null; private String machine=""; private String tsiDescription=""; - + private Socket commands_socket; + private Socket data_socket; + private int port=-1; private int replyport=-1; @@ -73,7 +74,8 @@ public static final String TSI_PORT="CLASSICTSI.port"; public static final String TSI_MYPORT="CLASSICTSI.replyport"; public static final String TSI_BSSUSER="CLASSICTSI.priveduser"; - + public static final String TSI_SOCKET_CLASS="TSISocket.class"; + private int count=0; private String bssuser; @@ -136,7 +138,9 @@ // Ask shepherd for a new worker try { - signalShepherd("newtsiprocess\n"); + // sends this XNJS port to allow the use of multiple XNJS ports + // on the same TSI + server.signalShepherd("newtsiprocess\n" + replyport + "\n"); } catch(Exception ex) { log.error("TSI is not available",ex); @@ -145,26 +149,35 @@ try { // Wait for connection requests (commands first, then data) - Socket commands_socket = server.accept(); - Socket data_socket = server.accept(); - - - // Make sure that pair comes from same machine - if(!commands_socket.getInetAddress().equals(data_socket.getInetAddress())) { + commands_socket = server.accept(); + } + catch(Exception ex) { + try { commands_socket.close(); - data_socket.close(); - log.error("TSI unavailable"); - throw new Exception("TSI unavailable"); + } catch (IOException e) { + //e.printStackTrace(); } - - // and want them both to be from the correct place - if(!commands_socket.getInetAddress().equals(source_addr)) { - commands_socket.close(); + log.error("failed to initialize command socket"); + throw new TSIUnavailableException("failed to initialize command socket"); + } + + try { + // Wait for connection requests (commands first, then data) + data_socket = server.accept(); + } + catch(Exception ex) { + try { data_socket.close(); - throw new RuntimeException("Invalid new TSI connection (wrong machine). Contact site administration"); // ?? OK + } catch (IOException e) { + //e.printStackTrace(); } + log.error("failed to initialize data socket"); + throw new TSIUnavailableException("failed to initialize data socket"); + } + + try { // Build the Channel on them newConn = new TSIConnection(commands_socket, data_socket, this); @@ -181,19 +194,6 @@ return newConn; } - private void signalShepherd(String message) throws Exception { - // Signal TSID that we want a new TSI process - if(log.isDebugEnabled()){ - log.debug("Signalling TSI at "+source_addr+":"+port - +"\n"+message); - } - Socket s = new Socket(source_addr, port); - s.getOutputStream().write(message.getBytes()); - s.getOutputStream().flush(); - // Read from the TSI daemon, just an ack that all is OK - try {s.getInputStream().read(); s.close();} catch(IOException ex) {} - } - //return a connection to the pool void done(TSIConnection connection){ synchronized (pool){ @@ -235,7 +235,9 @@ " ** Listening on port "+replyport+"\n"+ " ** User id for querying list of jobs on BSS: '"+bssuser+"'"); - server=new ServerSocket(replyport); + String className = getConfiguration().getProperty(TSI_SOCKET_CLASS, "de.fzj.unicore.xnjs.legacy.TSISocketPlain"); + server = (TSISocket)Class.forName(className).newInstance(); + server.init(replyport, port, source_addr); //getConfiguration().registerMBeans(this.getClass()); tsiDescription="classic TSI at "+machine+":"+port+", XNJS listens on port "+replyport; } Index: src/main/java/de/fzj/unicore/xnjs/legacy/TSISocket.java =================================================================== --- src/main/java/de/fzj/unicore/xnjs/legacy/TSISocket.java (revision 0) +++ src/main/java/de/fzj/unicore/xnjs/legacy/TSISocket.java (revision 0) @@ -0,0 +1,48 @@ +package de.fzj.unicore.xnjs.legacy; + +import java.io.IOException; +import java.net.InetAddress; +import java.net.ServerSocket; +import java.net.Socket; + +import org.apache.log4j.Logger; + +import de.fzj.unicore.xnjs.util.LogUtil; + +public abstract class TSISocket { + + protected static final Logger log=LogUtil.getLogger(LogUtil.TSI,TSIConnectionFactory.class); + protected ServerSocket socket = null; + protected InetAddress tsi_addr = null; + protected int listening_port = -1; + protected int tsi_port = -1; + + /** + * Initialize the TSISocket object + * @param njs_port The NJS listening port + * @param dest_port The TSI port + * @param tsi_address The TSI INET address + * @throws Exception + */ + public abstract void init(int njs_port, int dest_port, InetAddress tsi_address) throws Exception; + + /** + * return a Socket connected with the remote host, + * @return a valid connection object or null in case of errors + */ + public abstract Socket accept() throws IOException; + + + /** + * Signal TSID that we want a new TSI process + */ + public abstract void signalShepherd(String message) throws Exception; + + /** + * Close the listening socket + */ + public void close() throws IOException { + socket.close(); + } + +} Index: src/main/java/de/fzj/unicore/xnjs/legacy/TSISocketSSL.java =================================================================== --- src/main/java/de/fzj/unicore/xnjs/legacy/TSISocketSSL.java (revision 0) +++ src/main/java/de/fzj/unicore/xnjs/legacy/TSISocketSSL.java (revision 0) @@ -0,0 +1,133 @@ +package de.fzj.unicore.xnjs.legacy; + +import java.io.FileInputStream; +import java.io.IOException; +import java.net.InetAddress; +import java.net.ServerSocket; +import java.net.Socket; +import java.security.KeyStore; + +import javax.net.ssl.KeyManagerFactory; +import javax.net.ssl.SSLContext; +import javax.net.ssl.SSLServerSocket; +import javax.net.ssl.SSLServerSocketFactory; +import javax.net.ssl.SSLSocket; +import javax.net.ssl.SSLSocketFactory; +import javax.net.ssl.TrustManagerFactory; + +import de.fzj.unicore.wsrflite.Kernel; + +/** + * create a ServerSocket supporting SSL connections + * connections are secured by controlling hosts keys + */ +public class TSISocketSSL extends TSISocket{ + + + private String trustFile; + private String trustPassword; + private String trustType; + private String keyFile; + private String keyPassword; + private String keyType; + + public void init(int njs_port, int dest_port, InetAddress tsi_address) throws Exception { + + listening_port = njs_port; + tsi_port = dest_port; + tsi_addr = tsi_address; + + trustFile = Kernel.getKernel().getProperty("unicore.wsrflite.ssl.truststore"); + trustPassword = Kernel.getKernel().getProperty("unicore.wsrflite.ssl.truststorepass"); + trustType = Kernel.getKernel().getProperty("unicore.wsrflite.ssl.truststoretype"); + keyFile = Kernel.getKernel().getProperty("unicore.wsrflite.ssl.keystore"); + keyPassword = Kernel.getKernel().getProperty("unicore.wsrflite.ssl.keypass"); + keyType = Kernel.getKernel().getProperty("unicore.wsrflite.ssl.keytype"); + + SSLContext sc = getSSLContext(); + SSLServerSocketFactory ssf = sc.getServerSocketFactory(); + socket = (ServerSocket) ssf.createServerSocket(listening_port); + + String[] suites = ((SSLServerSocket)socket).getSupportedCipherSuites(); + ((SSLServerSocket)socket).setEnabledCipherSuites(suites); + ((SSLServerSocket)socket).setNeedClientAuth(true); + ((SSLServerSocket)socket).setEnableSessionCreation(true); + } + + + + public Socket accept() throws IOException { + Socket sock = null; + log.info("New listening connection"); + + log.info("SSL enabled"); + sock = ((SSLServerSocket)socket).accept(); + // the utilization of SSL authentication doesn't guaranty + // that the connection comes from the right TSI + // so we use the same authentication mechanism + // as TSISocketPlain + + // control net addresses + // Make sure that pair comes from same machine + if(!sock.getInetAddress().equals(tsi_addr)) { + sock.close(); + log.warn("TSI unavailable, wrong peer"); + throw new IOException("TSI unavailable"); + } + // control certificates + ((SSLSocket)sock).startHandshake(); + + return sock; + } + + + + public void signalShepherd(String message) throws Exception { + Socket s = null; + + // Signal TSID that we want a new TSI process + if(log.isDebugEnabled()){ + log.debug("Signalling TSI at "+tsi_addr+":"+tsi_port + +"\n"+message); + } + + SSLContext sc = getSSLContext(); + SSLSocketFactory ssf = sc.getSocketFactory(); + s = (Socket) ssf.createSocket(tsi_addr, tsi_port); + + s.getOutputStream().write(message.getBytes()); + s.getOutputStream().flush(); + // Read from the TSI daemon, just an ack that all is OK + try {s.getInputStream().read(); s.close();} catch(IOException ex) {} + + } + + private SSLContext getSSLContext() { + SSLContext sc = null; + try { + // keystore init + KeyStore ks = KeyStore.getInstance(keyType); + ks.load(new FileInputStream(keyFile), keyPassword.toCharArray()); + KeyManagerFactory kmf = KeyManagerFactory.getInstance("SunX509"); + kmf.init(ks, keyPassword.toCharArray()); + + // trustore init + KeyStore ts = KeyStore.getInstance(trustType); + ts.load(new FileInputStream(trustFile), trustPassword.toCharArray()); + TrustManagerFactory tmf = TrustManagerFactory.getInstance("SunX509"); + tmf.init(ts); + + sc = SSLContext.getInstance("TLS"); + sc.init(kmf.getKeyManagers(), tmf.getTrustManagers(), null); + } + catch(Exception e) { + log.error("Error initialising SSLContext"); + throw new Error("Error initialising SSLContext"); + + } + + return sc; + + } + +} Index: src/main/java/de/fzj/unicore/xnjs/legacy/TSISocketPlain.java =================================================================== --- src/main/java/de/fzj/unicore/xnjs/legacy/TSISocketPlain.java (revision 0) +++ src/main/java/de/fzj/unicore/xnjs/legacy/TSISocketPlain.java (revision 0) @@ -0,0 +1,52 @@ +package de.fzj.unicore.xnjs.legacy; + +import java.io.IOException; +import java.net.InetAddress; +import java.net.ServerSocket; +import java.net.Socket; + +/** + * create a ServerSocket supporting normal connections + * connections are secured by controlling IP addresses + */ +public class TSISocketPlain extends TSISocket { + + public void init(int njs_port, int dest_port, InetAddress tsi_address) throws Exception { + listening_port = njs_port; + tsi_port = dest_port; + tsi_addr = tsi_address; + + socket = new ServerSocket(listening_port); + } + + public Socket accept() throws IOException { + Socket sock = ((ServerSocket)socket).accept(); + + // control net addresses + // Make sure that pair comes from same machine + if(!sock.getInetAddress().equals(tsi_addr)) { + sock.close(); + log.warn("TSI unavailable, wrong peer"); + throw new IOException("TSI unavailable"); + } + return sock; + } + + public void signalShepherd(String message) throws Exception { + Socket s = null; + + // Signal TSID that we want a new TSI process + if(log.isDebugEnabled()){ + log.debug("Signalling TSI at "+tsi_addr+":"+tsi_port + +"\n"+message); + } + + s = new Socket(tsi_addr, tsi_port); + s.getOutputStream().write(message.getBytes()); + s.getOutputStream().flush(); + // Read from the TSI daemon, just an ack that all is OK + try {s.getInputStream().read(); s.close();} catch(IOException ex) {} + } + + +} Index: pom.xml =================================================================== --- pom.xml (revision 5220) +++ pom.xml (working copy) @@ -16,6 +16,15 @@ <connection>scm:svn:https://unicore.svn.sourceforge.net/svnroot/unicore/xnjs/tags/xnjs-module-core/xnjs-module-core-1.2.0</connection> <developerConnection>scm:svn:https://unicore.svn.sourceforge.net/svnroot/unicore/xnjs/tags/xnjs-module-core/xnjs-module-core-1.2.0</developerConnection> </scm> + <dependencyManagement> + <dependencies> + <dependency> + <groupId>de.fzj.unicore.wsrflite</groupId> + <artifactId>wsrflite</artifactId> + <version>1.8.12</version> + </dependency> + </dependencies> + </dependencyManagement> <distributionManagement> <!-- use the following if you're not using a snapshot version. --> @@ -106,6 +115,10 @@ <artifactId>persistence</artifactId> <version>0.3</version> </dependency> + <dependency> + <groupId>de.fzj.unicore.wsrflite</groupId> + <artifactId>wsrflite</artifactId> + </dependency> </dependencies> ------------------------------------------------------------------------------ Come build with us! The BlackBerry® Developer Conference in SF, CA is the only developer event you need to attend this year. Jumpstart your developing skills, take BlackBerry mobile applications to market and stay ahead of the curve. Join us from November 9-12, 2009. Register now! http://p.sf.net/sfu/devconf _______________________________________________ Unicore-devel mailing list Unicore-devel@... https://lists.sourceforge.net/lists/listinfo/unicore-devel |
|
|
Re: Patch for SSL in TSI and XNJS
by Bernd Schuller
::
Rate this Message:
Hi Clement,
thank you for this contribution! It looks quite OK to me. I'd like to integrate this into the SVN trunk as soon as possible. The XNJS part I'll probably change a bit to avoid having a direct dependency to WSRFlite. Best regards, Bernd. On Mo, 2009-10-05 at 16:42 +0200, Clement COUSSIRAT wrote: > Dear UNICORE developers, > > I am currently working at CEA on adding some HA capabilities to > UNICORE, I thought adding SSL support on all XNJS<->TSI communications > could be a good thing to increase security and deactivate TSI's IP > checks (which would allow the connection of several XNJS on one > TSI).I've made two patches, one for the TSI and the other for the > XNJS. > > TSI's part of the patch applies to the SVN trunk version. It activates > SSL if the keystore file is specified in tsi.properties. Keystore and > truststore must be in pem format, when an XNJS connects, its > certificate is checked with the CA cert. If this cert is correctly > signed and if it's present in the truststore, the TSI allows the > connection. In SSL mode, the XNJS IP checks are deactivated. > > An other thing added in this patch is the XNJS port negotiation. If > the XNJS port read in the tsi conf file is incorrect or not specified, > the TSI will try to read it from the XNJS shepherd connection. > > Technically, the patch uses IO::Socket::SSL perl module. This module is > actively maintained and is present on the most of package managers. To > simplify the code, the clear listening socket is created with > IO::Socket which has the same methods as IO::Socket::SSL. To avoid > changes in the rest of the code, the socket's filehandles are > extracted from IO::Socket objects and used as classic sockets. > > > The XNJS patch is based on xnjs-core-1.2.0 available in UNICORE-6.2.0, > the connections to the TSI presents in "TSIConnectionFactory" have > been split into 3 files. Now, "TSIConnectionFactory" uses the abstract > class "TSISocket" do call accept() and signalShepherd methods. The > TSISocket object is instantiated using a class name read in the > xnjs_legacy.xml conf file. > > This parameter can be set to 2 class names. The first is > "TSISocketPlain" which is the default value. This class keeps the > communications methods unchanged. The second class is "TSISocketSSL" > which add SSL support. It uses the same trustsfiles and keyfiles as > wsrflite. > > The transmission of the NJS port number has also been added in the > shepherd. > > To activate SSL, the following parameters must be set in tsi.properties: > tsi.keystore=/my/keystore.pem > tsi.keypass=keystore-password > tsi.truststore=/my/truststore.pem > > and in xnjs_legacy.xml: > <eng:Property name="TSISocket.class" > value="de.fzj.unicore.xnjs.legacy.TSISocketSSL"/> > > I would be grateful to you if you could send me a short review about > these patches and about upcoming updates. > > Regards, > Clément > Dr. Bernd Schuller Distributed Systems and Grid Computing Juelich Supercomputing Centre, http://www.fz-juelich.de/jsc Phone: +49 246161-8736 (fax -8556) Personal blog: www.jroller.com/page/gridhaus ------------------------------------------------------------------------------------------------ ------------------------------------------------------------------------------------------------ Forschungszentrum Juelich GmbH 52425 Juelich Sitz der Gesellschaft: Juelich Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 Vorsitzende des Aufsichtsrats: MinDir'in Baerbel Brumme-Bothe Geschaeftsfuehrung: Prof. Dr. Achim Bachem (Vorsitzender), Dr. Ulrich Krafft (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt, Prof. Dr. Sebastian M. Schmidt ------------------------------------------------------------------------------------------------ ------------------------------------------------------------------------------------------------ ------------------------------------------------------------------------------ Come build with us! The BlackBerry(R) Developer Conference in SF, CA is the only developer event you need to attend this year. Jumpstart your developing skills, take BlackBerry mobile applications to market and stay ahead of the curve. Join us from November 9 - 12, 2009. Register now! http://p.sf.net/sfu/devconference _______________________________________________ Unicore-devel mailing list Unicore-devel@... https://lists.sourceforge.net/lists/listinfo/unicore-devel |
| Free embeddable forum powered by Nabble | Forum Help |
