segmentation fault with libcrypto.so (but not libcrypto.a)

[N N-3]

Apologies if this is the wrong list.  If so, please direct me to the appropriate one.

Consider the following C code:

include <openssl/hmac.h>

#include <stdio.h>

int main(int argc, char** argv) {

  unsigned char foo[10] = "boo";

  printf("%s\n", SHA1(foo, 3, 0));

}

in file test-hmac.c.

gcc -static test-hmac.c -lcrypto; ./a.out

This works correctly, spewing garbage to the terminal.

gcc test-hmac.c -lcrypto; ./a.out

This segmentation faults.

Why?  What is wrong here?  So far, my best guess is that it has to do with how SHA1 allocates the return value when passed the null pointer (the third argument, 0).  The SHA1 function creates a static pointer large enough to hold the result which it then returns.  Does the fact that this operation occurs in a shared library change the allocation to the static pointer so that accessing after it returns is outside the allowed memory for the calling program?  Any help is appreciated.

Thanks.

P.S. libcrypto version 0.9.8k, Debian version squeeze/sid.

[Peter Samuelson]

[N N]
> #include <openssl/hmac.h>
> #include <stdio.h>
>
> int main(int argc, char** argv) {
>   unsigned char foo[10] = "boo";
>   printf("%s\n", SHA1(foo, 3, 0));
> }

Don't use printf("%s") on binary data that is not null-terminated.
What you probably want is to convert each byte to 2 hex digits and
print it that way.  See below.
--
Peter Samuelson | org-tld!p12n!peter | http://p12n.org/


#include <openssl/sha.h>
#include <stdlib.h>
#include <stdio.h>

int main(int argc, char** argv) {
    int i;
    char hex[] = "0123456789abcdef";
    unsigned char foo[10] = "boo";
    unsigned char *out = SHA1(foo, 3, 0);

    for (i = 0; i < 20; i++)
        printf("%c%c",
               hex[out[i] >> 4],
               hex[out[i] & 0x0f]);
    putc('\n', stdout);
    exit(0);
}


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Goswin von Brederlow-2]

N N <nothere44@...> writes:

> Apologies if this is the wrong list.  If so, please direct me to the
> appropriate one.
>
> Consider the following C code:
>
> include <openssl/hmac.h>
> #include <stdio.h>
>
> int main(int argc, char** argv) {
>   unsigned char foo[10] = "boo";
>   printf("%s\n", SHA1(foo, 3, 0));
> }
>
> in file test-hmac.c.
>
> gcc -static test-hmac.c -lcrypto; ./a.out
>
> This works correctly, spewing garbage to the terminal.

Since accidentally the SHA1 sum is followed by some allocated memory
containing a 0 byte at some point.

> gcc test-hmac.c -lcrypto; ./a.out
>
> This segmentation faults.

Here you don't have a 0 byte after the SHA1 before the allocated
memory runs out and you get a segfault.

> Why?  What is wrong here?  So far, my best guess is that it has to
> do with how SHA1 allocates the return value when passed the null
> pointer (the third argument, 0).  The SHA1 function creates a static
> pointer large enough to hold the result which it then returns.  Does
> the fact that this operation occurs in a shared library change the
> allocation to the static pointer so that accessing after it returns
> is outside the allowed memory for the calling program?  Any help is
> appreciated.

What is wrong is your code.

While SHA1 allocates the value it does not allocate a 0 terminated C
string but rather a fixed size array filled with binary data. You can
never print that with %s or even %32s. Apart from your segfault just
think what happens if the SHA1 starts with a zero byte or contains a
zero byte.

You can also not compare 2 SHA1 with strncmp. Don't even try.

> Thanks.
>
> P.S. libcrypto version 0.9.8k, Debian version squeeze/sid.

MfG
        Goswin


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[N N-3]

Since people didn't like the previous code, try the following version with safer memory handling.  It still segfaults when using the shared library and doesn't when using the static one.  Replacing the line above with the commented out line works in both cases (since we're no longer relying on SHA1 to allocate the memory for the return result for us).

#include <openssl/hmac.h>

#include <stdio.h>

int main(int argc, char** argv) {

  unsigned char foo[10] = "boo";

  unsigned char* res = malloc(20);

  unsigned char* res2 = res;

  res = SHA1(foo, 3, 0);

  //res = SHA1(foo, 3, res);                                                    

  int i;

  for(i = 0; i < 20; i++)

    printf("%x ", res[i]);

  printf("\n");

  free(res2);

}

[Michael Banck]

Hi,

On Tue, Nov 03, 2009 at 12:29:46PM -0500, N N wrote:
> Since people didn't like the previous code, try the following version with
> safer memory handling.  

Debian-devel is not a bug-reporting or programming-discussion list.
Please post elsewhere.


thanks,

Michael


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Gabor Gombas]

On Tue, Nov 03, 2009 at 12:29:46PM -0500, N N wrote:

> #include <openssl/hmac.h>
> #include <stdio.h>
>
> int main(int argc, char** argv) {
>   unsigned char foo[10] = "boo";
>   unsigned char* res = malloc(20);
>   unsigned char* res2 = res;
>   res = SHA1(foo, 3, 0);
>   //res = SHA1(foo, 3, res);
>
>   int i;
>   for(i = 0; i < 20; i++)
>     printf("%x ", res[i]);
>   printf("\n");
>   free(res2);
> }

Try to compile it. Read and understand the gcc warnings, fix the bugs,
and the segfault is gone.

Gabor

--
     ---------------------------------------------------------
     MTA SZTAKI Computer and Automation Research Institute
                Hungarian Academy of Sciences
     ---------------------------------------------------------


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...