Re: [openssl.org #2094] AutoReply: BUG: 0.9.8x md5-x86_64.s:41: Error: 0xd76aa478 out range of signed 32bit displacement

[Robin Seggelmann via RT]

On Friday 06 of November 2009, The default queue via RT wrote:

And the fix from md5-x86_64 implementation author -  Marc Bevand with explanation

"Well my assembler code doesn't really care whether the
displacement are interpreted as 32-bit signed or unsigned values
because in the end the result is truncated to 32-bit. So in order to
fix it replace each hex value with a 32-bit signed value (eg.
0xd76aa478 with -680876936)."

diff -ur openssl-0.9.8l.org/crypto/md5/asm/md5-x86_64.pl openssl-0.9.8l/crypto/md5/asm/md5-x86_64.pl
--- openssl-0.9.8l.org/crypto/md5/asm/md5-x86_64.pl 2007-11-11 14:34:06.000000000 +0100
+++ openssl-0.9.8l/crypto/md5/asm/md5-x86_64.pl 2009-11-06 23:23:15.528442820 +0100
@@ -19,6 +19,7 @@
 sub round1_step
 {
     my ($pos, $dst, $x, $y, $z, $k_next, $T_i, $s) = @_;
+    $T_i = unpack("l",pack("l", hex($T_i))); # convert to 32-bit signed decimal
     $code .= " mov 0*4(%rsi), %r10d /* (NEXT STEP) X[0] */\n" if ($pos == -1);
     $code .= " mov %edx, %r11d /* (NEXT STEP) z' = %edx */\n" if ($pos == -1);
     $code .= <<EOF;
@@ -42,6 +43,7 @@
 sub round2_step
 {
     my ($pos, $dst, $x, $y, $z, $k_next, $T_i, $s) = @_;
+    $T_i = unpack("l",pack("l", hex($T_i))); # convert to 32-bit signed decimal
     $code .= " mov 1*4(%rsi), %r10d /* (NEXT STEP) X[1] */\n" if ($pos == -1);
     $code .= " mov %ecx, %r11d /* (NEXT STEP) y' = %ecx */\n" if ($pos == -1);
     $code .= <<EOF;
@@ -65,6 +67,7 @@
 sub round3_step
 {
     my ($pos, $dst, $x, $y, $z, $k_next, $T_i, $s) = @_;
+    $T_i = unpack("l",pack("l", hex($T_i))); # convert to 32-bit signed decimal
     $code .= " mov 5*4(%rsi), %r10d /* (NEXT STEP) X[5] */\n" if ($pos == -1);
     $code .= " mov %ecx, %r11d /* (NEXT STEP) y' = %ecx */\n" if ($pos == -1);
     $code .= <<EOF;
@@ -87,6 +90,7 @@
 sub round4_step
 {
     my ($pos, $dst, $x, $y, $z, $k_next, $T_i, $s) = @_;
+    $T_i = unpack("l",pack("l", hex($T_i))); # convert to 32-bit signed decimal
     $code .= " mov 0*4(%rsi), %r10d /* (NEXT STEP) X[0] */\n" if ($pos == -1);
     $code .= " mov \$0xffffffff, %r11d\n" if ($pos == -1);
     $code .= " xor %edx, %r11d /* (NEXT STEP) not z' = not %edx*/\n"


--
Arkadiusz Miśkiewicz        PLD/Linux Team
arekm / maven.pl            http://ftp.pld-linux.org/


______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
Development Mailing List                       openssl-dev@...
Automated List Manager                           majordomo@...