[PATCH] libc/stdlib/malloc/realloc.c: Fix failure when doing realloc(mem, -1).

[James Coleman-3]

Now check that new_size is > ((unsigned long)-(MALLOC_HEADER_SIZE*2)),
which is the same test that is found in malloc.

This fixes a test failure in test/malloc/tst-mcheck.

Signed-off-by: James Coleman <jcoleman@...>
---

Please ignore the last patch that I sent, this one is correct and
actually compiles.

Many apologies.

James.
 
 libc/stdlib/malloc/realloc.c |    6 ++++--
 1 files changed, 4 insertions(+), 2 deletions(-)

diff --git a/libc/stdlib/malloc/realloc.c b/libc/stdlib/malloc/realloc.c
index a827199..aad2b3f 100644
--- a/libc/stdlib/malloc/realloc.c
+++ b/libc/stdlib/malloc/realloc.c
@@ -27,8 +27,10 @@ realloc (void *mem, size_t new_size)
   size_t size;
   char *base_mem;
 
-  /* Check for special cases.  */
-  if (! new_size)
+  /* Check for special cases, such as realloc(mem, 0) or if they are
+     doing something dumb like realloc(mem, -1) */
+  if (unlikely(! new_size) ||
+      unlikely(((unsigned long)new_size > (unsigned long)(MALLOC_HEADER_SIZE*-2))))
     {
       free (mem);
       return malloc (new_size);
--
1.6.1.1

_______________________________________________
uClibc mailing list
uClibc@...
http://lists.busybox.net/mailman/listinfo/uclibc

[Mike Frysinger]

On Thursday 30 July 2009 15:58:31 James Coleman wrote:

> Now check that new_size is > ((unsigned long)-(MALLOC_HEADER_SIZE*2)),
> which is the same test that is found in malloc.
>
> This fixes a test failure in test/malloc/tst-mcheck.
>
> -  /* Check for special cases.  */
> -  if (! new_size)
> +  /* Check for special cases, such as realloc(mem, 0) or if they are
> +     doing something dumb like realloc(mem, -1) */
> +  if (unlikely(! new_size) ||
> +      unlikely(((unsigned long)new_size > (unsigned
>  long)(MALLOC_HEADER_SIZE*-2)))) {
>        free (mem);
>        return malloc (new_size);

if we do overflow the size field, i dont think we should bother calling down
to malloc().  it's going to come to the same conclusion and return NULL.  so i
updated the realloc() code to return NULL itself if this case.
-mike


_______________________________________________
uClibc mailing list
uClibc@...
http://lists.busybox.net/mailman/listinfo/uclibc