file handle exhaustion with openvpn and pam_ldap

[Andreas Metzler]

Hello,

this is <http://bugs.debian.org/543941>:

When using openvpn and pam_ldap against an LDAP server with TLS
support on every authentication, a file handle to /dev/urandom is
created but never released. (libldap-2.4-2 is using gnutls, openvpn
isn't.)

------------------------
Lars Ellenberg has debuged the issue, I am forwarding his comments:
attached is a simple program to reproduce,
and workaround the issue.

libgcrypt standard behaviour is, at least on linux,
to open /dev/urandom once,
save that file descriptor in some static variable,
and re-use it wherever appropriate.
and never ever close that file descriptor,
but on exit or fork.

problem is:
pam_start() via various indirections may dlopen()s libgcrypt,
pam_stop() will dlclose() it again.

which means the libgcrypt will be unloaded,
and its static urandom fd with it.

but there is no destructor to close the FD.

on the next iteration,
a new instance of libgcrypt will be loaded,
with freshly initialized data segment,
resulting in an additional open of urandom.

that is the leak.

Workaround:
grab an additional reference on libgcrypt.
these workarounds seem to have precedence, see the
void nasty_pthread_hack (void) __attribute__ ((constructor));
void nasty_ssl_hack (void) __attribute__ ((constructor));
in libpam_ldap: pam_ldap.c

This should only be done as a short term workaround, though.

Real fix would be for libgcrypt to properly clean up on unload,
i.e. to provide proper destructor functions.

try.c is attached.
example session:

~/src/try$ gcc -o try try.c -lpam -ldl -pthread
~/src/try$ strace -e open ./try 2>&1 | grep urandom
open("/dev/urandom", O_RDONLY)          = 4
open("/dev/urandom", O_RDONLY)          = 6
open("/dev/urandom", O_RDONLY)          = 7

~/src/try$ gcc -DFIXIT -o try try.c -lpam -ldl -pthread
~/src/try$ strace -e open ./try 2>&1 | grep urandom
open("/dev/urandom", O_RDONLY)          = 4


----------------------------------------------
#include <stdio.h>
#include <unistd.h>
#include <stdlib.h>
#include <string.h>
#include <dlfcn.h>
#include <security/pam_appl.h>

#define USERNAME "dummy1"
#define PASSWORD "dummy2"

/*
 * PAM conversation function
 */

/* copied and shortened from openvpn source */
static int
my_conv (int n, const struct pam_message **msg_array,
         struct pam_response **response_array, void *appdata_ptr)
{
  struct pam_response *aresp;
  int i;
  int ret = PAM_SUCCESS;

  *response_array = NULL;

  if (n <= 0 || n > PAM_MAX_NUM_MSG)
    return (PAM_CONV_ERR);
  if ((aresp = calloc (n, sizeof *aresp)) == NULL)
    return (PAM_BUF_ERR);

  /* loop through each PAM-module query */
  for (i = 0; i < n; ++i)
    {
      const struct pam_message *msg = msg_array[i];
      aresp[i].resp_retcode = 0;
      aresp[i].resp = NULL;

      /* use PAM_PROMPT_ECHO_x hints */
      switch (msg->msg_style)
        {
        case PAM_PROMPT_ECHO_OFF:
          aresp[i].resp = strdup (PASSWORD);
          if (aresp[i].resp == NULL)
            ret = PAM_CONV_ERR;
          break;

        case PAM_PROMPT_ECHO_ON:
          aresp[i].resp = strdup (USERNAME);
          if (aresp[i].resp == NULL)
            ret = PAM_CONV_ERR;
          break;

        case PAM_ERROR_MSG:
        case PAM_TEXT_INFO:
          break;

        default:
          ret = PAM_CONV_ERR;
          break;
        }
    }

  if (ret == PAM_SUCCESS)
    *response_array = aresp;
  return ret;
}

int main(int argc, char **argv)
{
#ifdef FIXIT
        void *dlh = dlopen("libgcrypt.so", RTLD_LAZY);
#endif
        struct pam_conv conv = { .conv = my_conv, };
        pam_handle_t *pamh;
        int i;
        for (i = 0; i < 3; i++) {
                pam_start("openvpn", USERNAME, &conv, &pamh);
                pam_authenticate(pamh, 0);
                pam_end(pamh, PAM_SUCCESS);
        }
        return 0;
}
----------------------------------------------

cu andreas
--
`What a good friend you are to him, Dr. Maturin. His other friends are
so grateful to you.'
`I sew his ears on from time to time, sure'


_______________________________________________
Gcrypt-devel mailing list
Gcrypt-devel@...
http://lists.gnupg.org/mailman/listinfo/gcrypt-devel

[Werner Koch]

On Sun, 25 Oct 2009 09:35, ametzler@... said:

> When using openvpn and pam_ldap against an LDAP server with TLS
> support on every authentication, a file handle to /dev/urandom is
> created but never released. (libldap-2.4-2 is using gnutls, openvpn
> isn't.)

The problem is that you can't load/unload/load libgcrypt using dlopen
tricks.  This is simply not defined unless dlopen/dlclose implements a
complete process initialization/termination.  True, there is a function
to terminate the secure memory which needs to be called before the
process terminates but this is not a complete shutdown of libgcrypt, the
OS needs to cleanup some of the resources.

The documentation os FIPS required state machine says:

  [The state transition from] Operational to Shutdown is an artifical
  state without any direct action in Libgcrypt.  When reaching the
  Shutdown state the library is deinitialized and can't return to any
  other state again.

Thus to change this you would need to implement the required OS parts in
your dlopen/dlclose.


Salam-Shalom,

   Werner

--
Die Gedanken sind frei.  Auschnahme regelt ein Bundeschgesetz.


_______________________________________________
Gcrypt-devel mailing list
Gcrypt-devel@...
http://lists.gnupg.org/mailman/listinfo/gcrypt-devel

[Andreas Metzler]

On 2009-10-26 Werner Koch <wk@...> wrote:
> On Sun, 25 Oct 2009 09:35, ametzler@... said:

> > When using openvpn and pam_ldap against an LDAP server with TLS
> > support on every authentication, a file handle to /dev/urandom is
> > created but never released. (libldap-2.4-2 is using gnutls, openvpn
> > isn't.)

> The problem is that you can't load/unload/load libgcrypt using dlopen
> tricks.  This is simply not defined unless dlopen/dlclose implements a
> complete process initialization/termination.  True, there is a function
> to terminate the secure memory which needs to be called before the
> process terminates but this is not a complete shutdown of libgcrypt, the
> OS needs to cleanup some of the resources.

> The documentation os FIPS required state machine says:

>   [The state transition from] Operational to Shutdown is an artifical
>   state without any direct action in Libgcrypt.  When reaching the
>   Shutdown state the library is deinitialized and can't return to any
>   other state again.

> Thus to change this you would need to implement the required OS parts in
> your dlopen/dlclose.

Hello,

just to clarify. -  You are saying that:

* This issue cannot be fixed in gcrypt itself (and therefore will not
  be fixed).
* The way dlopen works on $OS would need to be changed (I guess on
  Linux this would be glibc.)

thanks, cu andreas

_______________________________________________
Gcrypt-devel mailing list
Gcrypt-devel@...
http://lists.gnupg.org/mailman/listinfo/gcrypt-devel

[Werner Koch]

On Mon, 26 Oct 2009 13:17, ametzler@... said:

> * This issue cannot be fixed in gcrypt itself (and therefore will not
>   be fixed).

Well, this is fix not that easy.  The open file descriptor is just one
sign thatthe process has not really be terminated.  Sure, it is possible
to do that but it is quite some work for a rare use case.

> * The way dlopen works on $OS would need to be changed (I guess on
>   Linux this would be glibc.)

Frankly, I doubt that this will be possible on Unix.  A process is a
fundamental resource and tweaking it to behave similar to an independant
process but not really is a bit weird.


Shalom-Salam,

   Werner

--
Die Gedanken sind frei.  Auschnahme regelt ein Bundeschgesetz.


_______________________________________________
Gcrypt-devel mailing list
Gcrypt-devel@...
http://lists.gnupg.org/mailman/listinfo/gcrypt-devel

[Andreas Metzler]

On 2009-10-26 Werner Koch <wk@...> wrote:
> On Mon, 26 Oct 2009 13:17, ametzler@... said:

> > * This issue cannot be fixed in gcrypt itself (and therefore will not
> >   be fixed).

> Well, this is fix not that easy.  The open file descriptor is just one
> sign thatthe process has not really be terminated.  Sure, it is possible
> to do that but it is quite some work for a rare use case.

> > * The way dlopen works on $OS would need to be changed (I guess on
> >   Linux this would be glibc.)

> Frankly, I doubt that this will be possible on Unix.  A process is a
> fundamental resource and tweaking it to behave similar to an independant
> process but not really is a bit weird.

Hello,

thanks for the clarification. So it is basically the other way round
than I understood it. The issue *might* be fixed in libgcrypt, but is
hard.

"but it is quite some work for a rare use case"

It possibly breaks every pam or nsswitch modules that uses GnuTLS. In
Debian this includes some of the popular ones (e.g. samba, ldap
postgresql). I do not claim that pam/nss is a brilliant design
especially due to dlopen problems like this one but it is not that
unpopular.

cu andreas

_______________________________________________
Gcrypt-devel mailing list
Gcrypt-devel@...
http://lists.gnupg.org/mailman/listinfo/gcrypt-devel