Unexpected "Exiting normally" 2.1.8?

[Craig Campbell]

Some parts of this message have been removed. Learn more about Nabble's security policy.

I'm running an unreleased '"development?" version of freeradius (2.1.8?).

 

So far it is working well, but it is terminating for reasons I cannot determine.

 

The log contains the following,

 

Mon Oct 26 15:48:57 2009 : Info: rlm_sql (sql): Driver rlm_sql_mysql (module rlm_sql_mysql) loaded and linked
Mon Oct 26 15:48:57 2009 : Info: rlm_sql (sql): Attempting to connect to radiusd@...
Mon Oct 26 15:48:57 2009 : Info: rlm_sql_mysql: Starting connect to MySQL server for #0
Mon Oct 26 15:48:57 2009 : Info: rlm_sql_mysql: Starting connect to MySQL server for #1
Mon Oct 26 15:48:57 2009 : Info: rlm_sql_mysql: Starting connect to MySQL server for #2
Mon Oct 26 15:48:57 2009 : Info: rlm_sql_mysql: Starting connect to MySQL server for #3
Mon Oct 26 15:48:57 2009 : Info: rlm_sql_mysql: Starting connect to MySQL server for #4
Mon Oct 26 15:48:57 2009 : Info: Loaded virtual server inner-tunnel
Mon Oct 26 15:48:57 2009 : Info: Loaded virtual server copy-acct-to-home-server
Mon Oct 26 15:48:57 2009 : Info: Loaded virtual server copy-acct-to-radius-c
Mon Oct 26 15:48:57 2009 : Info: Loaded virtual server <default>
Mon Oct 26 15:48:57 2009 : Info: Ready to process requests.
Mon Oct 26 17:57:33 2009 : Error: PROXY: Marking home server 192.168.1.226 port 1813 as zombie (it looks like it is dead).
Mon Oct 26 17:58:13 2009 : Info: PROXY: Marking home server 192.168.1.226 port 1813 as dead.
Mon Oct 26 20:05:36 2009 : Info: Exiting normally.

 

The zombie messages are suspicious, since neither host is experiencing any significant load. (The zombie server is also 2.1.8.  There is a 2.1.7 server as well NOT being zombied..)

The exit message is much later, but no hint as to WHY it is exiting normally.

 

Any hints would be greatly appreciated.

 

Thanks,

-craig

---

Craig Campbell
craig.campbell@...
CampbellCraft Consulting Inc
2 Kenny Court
Whitby, Ontario
Canada
L1R 2L8
905 922-2789

 

__________ Information from ESET Smart Security, version of virus signature database 4546 (20091027) __________

The message was checked by ESET Smart Security.

http://www.eset.com

-
List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html

[Alexander Clouter]

Craig Campbell <craig@...> wrote:
>
> I'm running an unreleased '"development?" version of freeradius (2.1.8?).
>
"me too", I get exactly what you are getting.  If you are always
fiddling with FreeRADIUS I recommend you always run it in gdb as then
you can get things fixed easily.

I usually build FreeRADIUS (under Debian stable) with:
----
git clone http://git.freeradius.org/freeradius-server.git
cd freeradius-server
git checkout release_2_1_7
git checkout -b soas

git cherry-pick c7a9d2aa1b3fa91591ce95f19aa5ba42c102f4f7
git cherry-pick fbdc02ad699b9bc4d410daaa54f76df7141d2f64
git cherry-pick fa0e98d1056e22fa413078dbd8c3fe0d85532826
git cherry-pick 92ab5fef40320d1dbc3fe59db82cb20a3ec69249
git cherry-pick 4ca219b1f1ab68fc8434072e51a8e4b95cf37c16
git cherry-pick 52880d0020b7b900ae8383b142b08e4e11cde639
git cherry-pick 137e3759b2ffc0c4f99064dadbd7461d3e86ae2a
git cherry-pick 9491d6eb7b963532855ccc8a63a523a2a1e3af2b
git cherry-pick 4baebf8202d7db372a9ad2ce5026ec6c986f0de7
git cherry-pick 382b6c2223ba1a233ca9f4d248beb888a0123f3e
git cherry-pick 751e9a39b2221a2623001a4611021a8e01cf4375
git cherry-pick 1013e94b66064f24170e394e63ba4f093c141d74
git cherry-pick 1628ef2387d9f7a89b3c2ff8945f49777eb135f1
git cherry-pick 83c2cd412b1208e67381372baa73c779cd2848f6
git cherry-pick f6e2dba8a7e4dd31d36d5b8ee434d21600e3f99f
git cherry-pick 64700e41098a874581d683c8606c94f9ad23079d
git cherry-pick e69be18535bd8b9a2cfb50a9df7cb44e3129ab4c
git cherry-pick 9261f3e0026323b2c397af13d02fbc5780908143

DEB_BUILD_OPTIONS='debug nostrip noopt' CFLAGS='-DIE_LIBTOOL_DIE' debuild -us -b
----


It's when I add (I am pretty sure it's the in the first 8 or so
patches) the following I get the same problem with FreeRADIUS:
----
git cherry-pick 12ead56dffca9b3ecddc8a7860a1ef5b5361b374
git cherry-pick d711a368ebf0e057e54596d22584ca2ce37e209c
git cherry-pick 057c7ac764a4639f715edcbde7dc22491b79be62
git cherry-pick a4202aeb848174ed430fd29573e3dd2db78ae2a1
git cherry-pick a92700b3fb88239ccb0db9f5ece68dd430937df3
git cherry-pick b1e815d0b4bec01f9721d4b92786960b2218f149
git cherry-pick 30adbf8230730a7503f5e3654df90c5c2a38a8ed
git cherry-pick f2d96581f98990d24991c99a681d018a3df85e92
git cherry-pick 5aa01c58d91063b5bbbf5aef941137d7cf638bbe
git cherry-pick 9b70af0c517daad7d374f4cc948488429d3a9cc0
git cherry-pick 98b22609015439b16cc62cf45e4472a14377da2a
git cherry-pick 092f0ea30cdfc2d669afe47061fafb9407269641
git cherry-pick b853a84e6c4ccd5d9e2c4ad9da2c421a234e887f
git cherry-pick d9dd62aae7baa5346f19236cead4414c03546d45
git cherry-pick 1700127c8a7150f57056495a2980fd132dafdb92
git cherry-pick 9dbc8974fdd2300a70293eda9c62bce20a3c9165
----

I guess at this point I am going to be told to be a good boy and run off
and use git bisect? :)

Looking through the patches normally I cannot see what could have caused
the graceful exit...which is exactly what I am getting:
----
garibaldi:/usr/src# gdb freeradius
GNU gdb 6.8-debian
Copyright (C) 2008 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later
<http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show
copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu"...
(gdb) run -f
Starting program: /usr/sbin/freeradius -f
[Thread debugging using libthread_db enabled]
[New Thread 0x7f9ba2eeaae0 (LWP 14420)]
[New Thread 0x41313950 (LWP 14423)]
[New Thread 0x4271a950 (LWP 14424)]
[New Thread 0x42f1b950 (LWP 14425)]
[New Thread 0x4371c950 (LWP 14426)]
[New Thread 0x43f1d950 (LWP 14427)]

Program received signal SIGTERM, Terminated.
[Switching to Thread 0x7f9ba2eeaae0 (LWP 14420)]
0x00007f9ba171e1c7 in kill () from /lib/libc.so.6
(gdb) bt full
#0  0x00007f9ba171e1c7 in kill () from /lib/libc.so.6
No symbol table info available.
#1  0x00000000004228d9 in main (argc=2, argv=0x7fffaaef61c8) at radiusd.c:419
        rcode = 0
        argval = -1
        spawn_flag = 1
        dont_fork = 1
        flag = 0
        act = {__sigaction_handler = {sa_handler = 0x422ab0 <sig_fatal>, sa_sigaction = 0x422ab0 <sig_fatal>}, sa_mask = {__val = {0 <repeats 16 times>}}, sa_flags = 0,   sa_restorer = 0}
(gdb) where
#0  0x00007f9ba171e1c7 in kill () from /lib/libc.so.6
#1  0x00000000004228d9 in main (argc=2, argv=0x7fffaaef61c8) at radiusd.c:419
(gdb)

(gdb) run -f
The program being debugged has been started already.
Start it from the beginning? (y or n) y
Starting program: /usr/sbin/freeradius -f
[Thread debugging using libthread_db enabled]
[New Thread 0x7f0874b2bae0 (LWP 14731)]
[New Thread 0x40d60950 (LWP 14732)]
[New Thread 0x41561950 (LWP 14733)]
[New Thread 0x41d62950 (LWP 14734)]
[New Thread 0x42563950 (LWP 14735)]
[New Thread 0x42d64950 (LWP 14736)]

Program received signal SIGTERM, Terminated.
[Switching to Thread 0x7f0874b2bae0 (LWP 14731)]
0x00007f087335f1c7 in kill () from /lib/libc.so.6
(gdb) bt full
#0  0x00007f087335f1c7 in kill () from /lib/libc.so.6
No symbol table info available.
#1  0x00000000004228d9 in main (argc=2, argv=0x7fff7cb36e08) at
radiusd.c:419
        rcode = 0
        argval = -1
        spawn_flag = 1
        dont_fork = 1
        flag = 0
        act = {__sigaction_handler = {sa_handler = 0x422ab0 <sig_fatal>, sa_sigaction = 0x422ab0 <sig_fatal>}, sa_mask = {__val = {0 <repeats 16 times>}}, sa_flags = 0,   sa_restorer = 0}
(gdb) where
#0  0x00007f087335f1c7 in kill () from /lib/libc.so.6
#1  0x00000000004228d9 in main (argc=2, argv=0x7fff7cb36e08) at radiusd.c:419
----

Happens about twice a day....completely unrelated to the load on the
server.  Quick 'fix' is to back up to commit
9261f3e0026323b2c397af13d02fbc5780908143.

Cheers

--
Alexander Clouter
.sigmonster says: You are the only person to ever get this message.

-
List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html

[Craig Campbell]

Hi Alexander.

  Thanks for the update - I was concluding I'd have to wait for the release
of 2.1.8 to pursue this.  I am currently in a situation where I can help
debug 2.1.8, since the 'new' systems aren't yet in production.

Looking at your debug output (and I am in no way an expert at that) it seems
as though the process received a signal?
I am running a 'custom' module (event.c as I recall) from Alan that resolves
an issue with hung children (very exciting!), and I followed Alan's
instructions to get to this point.  I would really like to try to 'give
back' if I can and assist in identifying the cause of the program exiting
(assuming it is a new and as of yet unidentified bug).

Would copying the steps you have below on my two redhat systems be a good
way to proceed?

Let me know,
-craig
----- Original Message -----
From: "Alexander Clouter" <alex@...>
To: <freeradius-users@...>
Sent: Wednesday, November 04, 2009 11:43 AM
Subject: Re: Unexpected "Exiting normally" 2.1.8?

> Craig Campbell <craig@...> wrote:
>>
>> I'm running an unreleased '"development?" version of freeradius (2.1.8?).
>>
> "me too", I get exactly what you are getting.  If you are always
> fiddling with FreeRADIUS I recommend you always run it in gdb as then
> you can get things fixed easily.
>
> I usually build FreeRADIUS (under Debian stable) with:
> ----
> git clone http://git.freeradius.org/freeradius-server.git
> cd freeradius-server
> git checkout release_2_1_7
> git checkout -b soas
>
> git cherry-pick c7a9d2aa1b3fa91591ce95f19aa5ba42c102f4f7
> git cherry-pick fbdc02ad699b9bc4d410daaa54f76df7141d2f64
> git cherry-pick fa0e98d1056e22fa413078dbd8c3fe0d85532826
> git cherry-pick 92ab5fef40320d1dbc3fe59db82cb20a3ec69249
> git cherry-pick 4ca219b1f1ab68fc8434072e51a8e4b95cf37c16
> git cherry-pick 52880d0020b7b900ae8383b142b08e4e11cde639
> git cherry-pick 137e3759b2ffc0c4f99064dadbd7461d3e86ae2a
> git cherry-pick 9491d6eb7b963532855ccc8a63a523a2a1e3af2b
> git cherry-pick 4baebf8202d7db372a9ad2ce5026ec6c986f0de7
> git cherry-pick 382b6c2223ba1a233ca9f4d248beb888a0123f3e
> git cherry-pick 751e9a39b2221a2623001a4611021a8e01cf4375
> git cherry-pick 1013e94b66064f24170e394e63ba4f093c141d74
> git cherry-pick 1628ef2387d9f7a89b3c2ff8945f49777eb135f1
> git cherry-pick 83c2cd412b1208e67381372baa73c779cd2848f6
> git cherry-pick f6e2dba8a7e4dd31d36d5b8ee434d21600e3f99f
> git cherry-pick 64700e41098a874581d683c8606c94f9ad23079d
> git cherry-pick e69be18535bd8b9a2cfb50a9df7cb44e3129ab4c
> git cherry-pick 9261f3e0026323b2c397af13d02fbc5780908143
>
> DEB_BUILD_OPTIONS='debug nostrip noopt' CFLAGS='-DIE_LIBTOOL_DIE'
> debuild -us -b
> ----
>
>
> It's when I add (I am pretty sure it's the in the first 8 or so
> patches) the following I get the same problem with FreeRADIUS:
> ----
> git cherry-pick 12ead56dffca9b3ecddc8a7860a1ef5b5361b374
> git cherry-pick d711a368ebf0e057e54596d22584ca2ce37e209c
> git cherry-pick 057c7ac764a4639f715edcbde7dc22491b79be62
> git cherry-pick a4202aeb848174ed430fd29573e3dd2db78ae2a1
> git cherry-pick a92700b3fb88239ccb0db9f5ece68dd430937df3
> git cherry-pick b1e815d0b4bec01f9721d4b92786960b2218f149
> git cherry-pick 30adbf8230730a7503f5e3654df90c5c2a38a8ed
> git cherry-pick f2d96581f98990d24991c99a681d018a3df85e92
> git cherry-pick 5aa01c58d91063b5bbbf5aef941137d7cf638bbe
> git cherry-pick 9b70af0c517daad7d374f4cc948488429d3a9cc0
> git cherry-pick 98b22609015439b16cc62cf45e4472a14377da2a
> git cherry-pick 092f0ea30cdfc2d669afe47061fafb9407269641
> git cherry-pick b853a84e6c4ccd5d9e2c4ad9da2c421a234e887f
> git cherry-pick d9dd62aae7baa5346f19236cead4414c03546d45
> git cherry-pick 1700127c8a7150f57056495a2980fd132dafdb92
> git cherry-pick 9dbc8974fdd2300a70293eda9c62bce20a3c9165
> ----
>
> I guess at this point I am going to be told to be a good boy and run off
> and use git bisect? :)
>
> Looking through the patches normally I cannot see what could have caused
> the graceful exit...which is exactly what I am getting:
> ----
> garibaldi:/usr/src# gdb freeradius
> GNU gdb 6.8-debian
> Copyright (C) 2008 Free Software Foundation, Inc.
> License GPLv3+: GNU GPL version 3 or later
> <http://gnu.org/licenses/gpl.html>
> This is free software: you are free to change and redistribute it.
> There is NO WARRANTY, to the extent permitted by law.  Type "show
> copying"
> and "show warranty" for details.
> This GDB was configured as "x86_64-linux-gnu"...
> (gdb) run -f
> Starting program: /usr/sbin/freeradius -f
> [Thread debugging using libthread_db enabled]
> [New Thread 0x7f9ba2eeaae0 (LWP 14420)]
> [New Thread 0x41313950 (LWP 14423)]
> [New Thread 0x4271a950 (LWP 14424)]
> [New Thread 0x42f1b950 (LWP 14425)]
> [New Thread 0x4371c950 (LWP 14426)]
> [New Thread 0x43f1d950 (LWP 14427)]
>
> Program received signal SIGTERM, Terminated.
> [Switching to Thread 0x7f9ba2eeaae0 (LWP 14420)]
> 0x00007f9ba171e1c7 in kill () from /lib/libc.so.6
> (gdb) bt full
> #0  0x00007f9ba171e1c7 in kill () from /lib/libc.so.6
> No symbol table info available.
> #1  0x00000000004228d9 in main (argc=2, argv=0x7fffaaef61c8) at
> radiusd.c:419
>        rcode = 0
>        argval = -1
>        spawn_flag = 1
>        dont_fork = 1
>        flag = 0
>        act = {__sigaction_handler = {sa_handler = 0x422ab0 <sig_fatal>,
> sa_sigaction = 0x422ab0 <sig_fatal>}, sa_mask = {__val = {0 <repeats 16
> times>}}, sa_flags = 0,   sa_restorer = 0}
> (gdb) where
> #0  0x00007f9ba171e1c7 in kill () from /lib/libc.so.6
> #1  0x00000000004228d9 in main (argc=2, argv=0x7fffaaef61c8) at
> radiusd.c:419
> (gdb)
>
> (gdb) run -f
> The program being debugged has been started already.
> Start it from the beginning? (y or n) y
> Starting program: /usr/sbin/freeradius -f
> [Thread debugging using libthread_db enabled]
> [New Thread 0x7f0874b2bae0 (LWP 14731)]
> [New Thread 0x40d60950 (LWP 14732)]
> [New Thread 0x41561950 (LWP 14733)]
> [New Thread 0x41d62950 (LWP 14734)]
> [New Thread 0x42563950 (LWP 14735)]
> [New Thread 0x42d64950 (LWP 14736)]
>
> Program received signal SIGTERM, Terminated.
> [Switching to Thread 0x7f0874b2bae0 (LWP 14731)]
> 0x00007f087335f1c7 in kill () from /lib/libc.so.6
> (gdb) bt full
> #0  0x00007f087335f1c7 in kill () from /lib/libc.so.6
> No symbol table info available.
> #1  0x00000000004228d9 in main (argc=2, argv=0x7fff7cb36e08) at
> radiusd.c:419
>        rcode = 0
>        argval = -1
>        spawn_flag = 1
>        dont_fork = 1
>        flag = 0
>        act = {__sigaction_handler = {sa_handler = 0x422ab0 <sig_fatal>,
> sa_sigaction = 0x422ab0 <sig_fatal>}, sa_mask = {__val = {0 <repeats 16
> times>}}, sa_flags = 0,   sa_restorer = 0}
> (gdb) where
> #0  0x00007f087335f1c7 in kill () from /lib/libc.so.6
> #1  0x00000000004228d9 in main (argc=2, argv=0x7fff7cb36e08) at
> radiusd.c:419
> ----
>
> Happens about twice a day....completely unrelated to the load on the
> server.  Quick 'fix' is to back up to commit
> 9261f3e0026323b2c397af13d02fbc5780908143.
>
> Cheers
>
> --
> Alexander Clouter
> .sigmonster says: You are the only person to ever get this message.
>
> -
> List info/subscribe/unsubscribe? See
> http://www.freeradius.org/list/users.html
>
> __________ Information from ESET Smart Security, version of virus
> signature database 4573 (20091104) __________
>
> The message was checked by ESET Smart Security.
>
> http://www.eset.com
>
>
>

__________ Information from ESET Smart Security, version of virus signature database 4573 (20091104) __________

The message was checked by ESET Smart Security.

http://www.eset.com



-
List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html

[Alan DeKok-2]

Alexander Clouter wrote:
> It's when I add (I am pretty sure it's the in the first 8 or so
> patches) the following I get the same problem with FreeRADIUS:
...
> I guess at this point I am going to be told to be a good boy and run off
> and use git bisect? :)

 Pretty much, sorry.

> Looking through the patches normally I cannot see what could have caused
> the graceful exit...which is exactly what I am getting:
...
> #1  0x00000000004228d9 in main (argc=2, argv=0x7fffaaef61c8) at radiusd.c:419

  That just means that the main event loop exited, and the server is
telling all child threads to stop.

  It looks like the server received a TERM, QUIT, or INT signal.  Why, I
don't know.

  But yes, "git bisect" would be tremendously useful.  I'm traveling for
the next week, so I'll have limited time to look at it myself.

  Alan DeKok.
-
List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html

[Alexander Clouter]

Alan DeKok <aland@...> wrote:

>
> Alexander Clouter wrote:
>> It's when I add (I am pretty sure it's the in the first 8 or so
>> patches) the following I get the same problem with FreeRADIUS:
> ...
>> I guess at this point I am going to be told to be a good boy and run off
>> and use git bisect? :)
>
> Pretty much, sorry.
>

It really is bug week for me.  Cisco (x4), FreeRADIUS (x1), Linux (x2),
etc etc.

Say, I do the git bisect, you will let my ldap xlat dn patch[1] go in,
I have been patient and waited two years? :)
 

>> Looking through the patches normally I cannot see what could have caused
>> the graceful exit...which is exactly what I am getting:
> ...
>> #1  0x00000000004228d9 in main (argc=2, argv=0x7fffaaef61c8) at radiusd.c:419
>
>  That just means that the main event loop exited, and the server is
> telling all child threads to stop.
>
>  It looks like the server received a TERM, QUIT, or INT signal.  Why, I
> don't know.
>

Yep, that was my take too.  As far as I can tell it just decided to
gracefully close down which is why when I nosey through the applied
patches I was hunting for a change in logic flow or something.

>  But yes, "git bisect" would be tremendously useful.  I'm traveling for
> the next week, so I'll have limited time to look at it myself.
>
Sure thing.  I'll try to find the time tomorrow, however it could take
a week or so to pin down as I'll need to run for two days to be sure it
is 'okay'.

Cheers

[1] http://stuff.digriz.org.uk/0001-support-to-get-DN-in-ldap_xlat.patch

--
Alexander Clouter
.sigmonster says: Be careful!  Is it classified?

-
List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html

[Alexander Clouter]

Craig Campbell <craig@...> wrote:
>
>  Thanks for the update - I was concluding I'd have to wait for the release
> of 2.1.8 to pursue this.  I am currently in a situation where I can help
> debug 2.1.8, since the 'new' systems aren't yet in production.
>
Well I can see no reason to run FreeRADIUS no in a debugger all the
time, even when in production.  However my nickname is "Rambo Clouter"
so maybe you do not want to follow my advice. :)

When you compile FreeRADIUS you simply make sure you leave
debugging symbols in and turn off compiler optimisations (so your CFLAGS
should be '-O0 -g'.  You probably can do this by running configure as
follows:
----
CFLAGS='-O0 -g' ./configure --all-your-usual-options-that-you-want
----

> Looking at your debug output (and I am in no way an expert at that) it seems
> as though the process received a signal?
>
Well FreeRADIUS is sending it to herself according to gdb:
---- src/main/radiusd.c line 419 ----
        /*
         *      Send a TERM signal to all
         *      associated processes
         *      (including us, which gets
         *      ignored.)
         */
#ifndef __MINGW32__
        if (spawn_flag) kill(-radius_pid, SIGTERM);
#endif  
----

For whatever reason, it is not getting ignored.  At first I thought it
was because I run my FreeRADIUS (even in production) in gdb, but as you
do not I am wondering what is actually going on.

To run it in the debugger just run 'gdb freeradius' and you will get the
gdb prompt.  There you want to type 'run -f' and wait for it to puke.  
When it does you could type 'where' for it to tell you what happened,
but we know what is happening, we want to find which patch is doing it
:)  Oh familise yourself with screen[2] if you do not know it already,
you should run the debugger in a screen'd session so you can return to
it later without having to remain logged in.

> I am running a 'custom' module (event.c as I recall) from Alan that resolves
> an issue with hung children (very exciting!), and I followed Alan's
> instructions to get to this point.  I would really like to try to 'give
> back' if I can and assist in identifying the cause of the program exiting
> (assuming it is a new and as of yet unidentified bug).
>
> Would copying the steps you have below on my two redhat systems be a good
> way to proceed?
>
Pretty much follow:

http://www.reactivated.net/weblog/archives/2006/01/using-git-bisect-to-find-buggy-kernel-patches/

I had been running with the cherry-pick'ed patches for weeks and had no
problems up to 9261f3e0026323b2c397af13d02fbc5780908143, so I am certain
that the issue is the result of the patches between
12ead56dffca9b3ecddc8a7860a1ef5b5361b374 and
9dbc8974fdd2300a70293eda9c62bce20a3c9165.  The problem is you *have* to
apply my listed cherry-picks, as if you add *any* of the TCP related
code Alan has been working on, it all stops compiling[1]

Cheers

[1] I am pretty sure Alan has stashed a number of patches that he has
        not put into the publically available GIT trees as things like
        the jumbo socket clean up patch
        (e04b62f1bd257489bd92ccc584b0886c7e2011e8) refer to
        my_ipaddr/my_port which is not in any header files I have or
        found in 'master'
[2] http://blogamundo.net/code/screen/
--
Alexander Clouter
.sigmonster says: Simplicity does not precede complexity, but follows it.

-
List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html

[Alan DeKok-2]

Alexander Clouter wrote:
>  The problem is you *have* to
> apply my listed cherry-picks, as if you add *any* of the TCP related
> code Alan has been working on, it all stops compiling[1]

  *Please* use the git "stable" branch.  The "master" branch has a whole
whack of other changes in it which may or may not get into a stable release.

  Much of the work in "stable" has been merged into "master".  But...
the TCP work hasn't.  This is because the re-work in "master" that moves
sockets into loadable modules conflicts with the TCP changes.

  I haven't had the time to go integrate the changes.  And since the
"stable" branch works, it's a low priority.

  Alan DeKok.
-
List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html

[Craig Campbell]

Ok,
 I performed the following on two servers that relay receive accounting
packets only and relay these to each other (if it matters).

This is the same test code the Alan had me build, with his custom event.c.
(I really do not know how to use git)
I deleted the existing installation, and then performed the following steps.

    CFLAGS='-O0 -g' ./configure
    make clean
    make
    sudo make install

I then ran the code from within gdb as Alexander described (within screen -
thanks!)

    gdb radiusd
    (gdb) run -f
    <messages program runs until...>
    Detaching after fork from child process 14277.
    Detaching after fork from child process 14334.
    Detaching after fork from child process 14364.
    Detaching after fork from child process 14394.
    Detaching after fork from child process 14424.

    Program received signal SIGTERM, Terminated.
    0x0000003acf4306a7 in kill () from /lib64/libc.so.6
    (gdb) where
    #0  0x0000003acf4306a7 in kill () from /lib64/libc.so.6
    #1  0x0000000000424186 in main (argc=2, argv=0x7fff67ce72d8) at
radiusd.c:419
    (gdb)

The gdb output is identical for the other radius server.

I'm afraid  I haven't helped much.  The source I have was acquired Oct 20
15:05 (EST) if that helps in any way.  Perhaps I have fewer of the suspected
patches in my build than you have in yours?  Is there a way we could
compare?

Is there any way I can get more information out of these test runs?

Thanks,
-craig

----- Original Message -----
From: "Alexander Clouter" <alex@...>
To: <freeradius-users@...>
Sent: Wednesday, November 04, 2009 3:47 PM
Subject: Re: Unexpected "Exiting normally" 2.1.8?

> Craig Campbell <craig@...> wrote:
>>
>>  Thanks for the update - I was concluding I'd have to wait for the
>> release
>> of 2.1.8 to pursue this.  I am currently in a situation where I can help
>> debug 2.1.8, since the 'new' systems aren't yet in production.
>>
> Well I can see no reason to run FreeRADIUS no in a debugger all the
> time, even when in production.  However my nickname is "Rambo Clouter"
> so maybe you do not want to follow my advice. :)
>
> When you compile FreeRADIUS you simply make sure you leave
> debugging symbols in and turn off compiler optimisations (so your CFLAGS
> should be '-O0 -g'.  You probably can do this by running configure as
> follows:
> ----
> CFLAGS='-O0 -g' ./configure --all-your-usual-options-that-you-want
> ----
>
>> Looking at your debug output (and I am in no way an expert at that) it
>> seems
>> as though the process received a signal?
>>
> Well FreeRADIUS is sending it to herself according to gdb:
> ---- src/main/radiusd.c line 419 ----
>        /*
>         *      Send a TERM signal to all
>         *      associated processes
>         *      (including us, which gets
>         *      ignored.)
>         */
> #ifndef __MINGW32__
>        if (spawn_flag) kill(-radius_pid, SIGTERM);
> #endif
> ----
>
> For whatever reason, it is not getting ignored.  At first I thought it
> was because I run my FreeRADIUS (even in production) in gdb, but as you
> do not I am wondering what is actually going on.
>
> To run it in the debugger just run 'gdb freeradius' and you will get the
> gdb prompt.  There you want to type 'run -f' and wait for it to puke.
> When it does you could type 'where' for it to tell you what happened,
> but we know what is happening, we want to find which patch is doing it
> :)  Oh familise yourself with screen[2] if you do not know it already,
> you should run the debugger in a screen'd session so you can return to
> it later without having to remain logged in.
>
>> I am running a 'custom' module (event.c as I recall) from Alan that
>> resolves
>> an issue with hung children (very exciting!), and I followed Alan's
>> instructions to get to this point.  I would really like to try to 'give
>> back' if I can and assist in identifying the cause of the program exiting
>> (assuming it is a new and as of yet unidentified bug).
>>
>> Would copying the steps you have below on my two redhat systems be a good
>> way to proceed?
>>
> Pretty much follow:
>
> http://www.reactivated.net/weblog/archives/2006/01/using-git-bisect-to-find-buggy-kernel-patches/
>
> I had been running with the cherry-pick'ed patches for weeks and had no
> problems up to 9261f3e0026323b2c397af13d02fbc5780908143, so I am certain
> that the issue is the result of the patches between
> 12ead56dffca9b3ecddc8a7860a1ef5b5361b374 and
> 9dbc8974fdd2300a70293eda9c62bce20a3c9165.  The problem is you *have* to
> apply my listed cherry-picks, as if you add *any* of the TCP related
> code Alan has been working on, it all stops compiling[1]
>
> Cheers
>
> [1] I am pretty sure Alan has stashed a number of patches that he has
> not put into the publically available GIT trees as things like
> the jumbo socket clean up patch
> (e04b62f1bd257489bd92ccc584b0886c7e2011e8) refer to
> my_ipaddr/my_port which is not in any header files I have or
> found in 'master'
> [2] http://blogamundo.net/code/screen/
> --
> Alexander Clouter
> .sigmonster says: Simplicity does not precede complexity, but follows it.
>
> -
> List info/subscribe/unsubscribe? See
> http://www.freeradius.org/list/users.html
>
> __________ Information from ESET Smart Security, version of virus
> signature database 4574 (20091104) __________
>
> The message was checked by ESET Smart Security.
>
> http://www.eset.com
>
>
>

__________ Information from ESET Smart Security, version of virus signature database 4578 (20091106) __________

The message was checked by ESET Smart Security.

http://www.eset.com



-
List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html

[Alexander Clouter]

Craig Campbell <craig@...> wrote:

>
> [snipped]
>
> I then ran the code from within gdb as Alexander described (within screen -
> thanks!)
>
>    gdb radiusd
>    (gdb) run -f
>    <messages program runs until...>
>    Detaching after fork from child process 14277.
>    Detaching after fork from child process 14334.
>    Detaching after fork from child process 14364.
>    Detaching after fork from child process 14394.
>    Detaching after fork from child process 14424.
>
>    Program received signal SIGTERM, Terminated.
>    0x0000003acf4306a7 in kill () from /lib64/libc.so.6
>    (gdb) where
>    #0  0x0000003acf4306a7 in kill () from /lib64/libc.so.6
>    #1  0x0000000000424186 in main (argc=2, argv=0x7fff67ce72d8) at
> radiusd.c:419
>    (gdb)
>
> The gdb output is identical for the other radius server.
>
> I'm afraid  I haven't helped much.  The source I have was acquired Oct 20
> 15:05 (EST) if that helps in any way.  Perhaps I have fewer of the suspected
> patches in my build than you have in yours?  Is there a way we could
> compare?
>
> Is there any way I can get more information out of these test runs?
>

Afraid that is all there is to see.  If this was a more 'interesting'
bug you would get a lot more, but the nature of this particular bug
means you do not get much.

Now you have the compiling of FreeRADIUS working and got a successful
round of GDB cooking, you need to play with 'git bisect' to pin it down
to which patch broke things.

Cheers

--
Alexander Clouter
.sigmonster says: You have many friends and very few living enemies.

-
List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html

[Craig Campbell]

Regarding git bisect.  I know the current version is "bad", but I have no
idea if any 2.1.8 version is 'Good'.

If there a way to use git bisect without identifying 'good'?  How can we
find the first version?  Can we assume it was 'good'?

Sorry to be so non knowledgeable about git.

-craig
----- Original Message -----
From: "Alexander Clouter" <alex@...>
To: <freeradius-users@...>
Sent: Friday, November 06, 2009 8:21 AM
Subject: Re: Unexpected "Exiting normally" 2.1.8?

> Craig Campbell <craig@...> wrote:
>>
>> [snipped]
>>
>> I then ran the code from within gdb as Alexander described (within
>> screen -
>> thanks!)
>>
>>    gdb radiusd
>>    (gdb) run -f
>>    <messages program runs until...>
>>    Detaching after fork from child process 14277.
>>    Detaching after fork from child process 14334.
>>    Detaching after fork from child process 14364.
>>    Detaching after fork from child process 14394.
>>    Detaching after fork from child process 14424.
>>
>>    Program received signal SIGTERM, Terminated.
>>    0x0000003acf4306a7 in kill () from /lib64/libc.so.6
>>    (gdb) where
>>    #0  0x0000003acf4306a7 in kill () from /lib64/libc.so.6
>>    #1  0x0000000000424186 in main (argc=2, argv=0x7fff67ce72d8) at
>> radiusd.c:419
>>    (gdb)
>>
>> The gdb output is identical for the other radius server.
>>
>> I'm afraid  I haven't helped much.  The source I have was acquired Oct 20
>> 15:05 (EST) if that helps in any way.  Perhaps I have fewer of the
>> suspected
>> patches in my build than you have in yours?  Is there a way we could
>> compare?
>>
>> Is there any way I can get more information out of these test runs?
>>
> Afraid that is all there is to see.  If this was a more 'interesting'
> bug you would get a lot more, but the nature of this particular bug
> means you do not get much.
>
> Now you have the compiling of FreeRADIUS working and got a successful
> round of GDB cooking, you need to play with 'git bisect' to pin it down
> to which patch broke things.
>
> Cheers
>
> --
> Alexander Clouter
> .sigmonster says: You have many friends and very few living enemies.
>
> -
> List info/subscribe/unsubscribe? See
> http://www.freeradius.org/list/users.html
>
> __________ Information from ESET Smart Security, version of virus
> signature database 4578 (20091106) __________
>
> The message was checked by ESET Smart Security.
>
> http://www.eset.com
>
>
>

__________ Information from ESET Smart Security, version of virus signature database 4579 (20091106) __________

The message was checked by ESET Smart Security.

http://www.eset.com



-
List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html

[Alexander Clouter]

Craig Campbell <craig@...> wrote:
>
> Regarding git bisect.  I know the current version is "bad", but I have no
> idea if any 2.1.8 version is 'Good'.
>
> If there a way to use git bisect without identifying 'good'?  How can we
> find the first version?  Can we assume it was 'good'?
>
> Sorry to be so non knowledgeable about git.
>
Please read my original email listing all the git cherry picks that I
used.

Cheers

--
Alexander Clouter
.sigmonster says: Logic is a pretty flower that smells bad.

-
List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html

[Craig Campbell]

I'm afraid I did something incorrect.

I ended up with,

gcc -o .libs/radmin .libs/radmin.o /usr/lib/libreadline.so
/usr/lib/libtermcap.so .libs/util.o .libs/log.o .libs/conffile.o
/home/craig/src/freeradius/freeradius-server/src/lib/.libs/libfreeradius-radius.so
 -lnsl -lresolv -lpthread -lreadline -ltermcap  -Wl,--rpath -Wl,/usr/local/lib
/usr/lib/libreadline.so: could not read symbols: File in wrong format
collect2: ld returned 1 exit status

Which I suspect is result of my git bisect attempts....

 $git bisect start
 $git bisect bad
 $git bisect good 9261f3e0026323b2c397af13d02fbc5780908143

 $git bisect loggit bisect start
# bad: [1489f5ffa5835e001499ee4f7a5f50a0da375613] Conf for debugging
git bisect bad 1489f5ffa5835e001499ee4f7a5f50a0da375613
# good: [9261f3e0026323b2c397af13d02fbc5780908143] Ensure that there is a
cleanup event for proxied packets
git bisect good 9261f3e0026323b2c397af13d02fbc5780908143
# good: [9261f3e0026323b2c397af13d02fbc5780908143] Ensure that there is a
cleanup event for proxied packets
git bisect good 9261f3e0026323b2c397af13d02fbc5780908143

$git bisect next
$CFLAGS='-O0 -g' ./configure
$make clean
$make
.....
gcc -o .libs/radmin .libs/radmin.o /usr/lib/libreadline.so
/usr/lib/libtermcap.so .libs/util.o .libs/log.o .libs/conffile.o
/home/craig/src/freeradius/freeradius-server/src/lib/.libs/libfreeradius-radius.so
 -lnsl -lresolv -lpthread -lreadline -ltermcap  -Wl,--rpath -Wl,/usr/local/lib
/usr/lib/libreadline.so: could not read symbols: File in wrong format
collect2: ld returned 1 exit status


----- Original Message -----
From: "Alexander Clouter" <alex@...>
To: <freeradius-users@...>
Sent: Friday, November 06, 2009 10:15 AM
Subject: Re: Unexpected "Exiting normally" 2.1.8?

> Craig Campbell <craig@...> wrote:
>>
>> Regarding git bisect.  I know the current version is "bad", but I have no
>> idea if any 2.1.8 version is 'Good'.
>>
>> If there a way to use git bisect without identifying 'good'?  How can we
>> find the first version?  Can we assume it was 'good'?
>>
>> Sorry to be so non knowledgeable about git.
>>
> Please read my original email listing all the git cherry picks that I
> used.
>
> Cheers
>
> --
> Alexander Clouter
> .sigmonster says: Logic is a pretty flower that smells bad.
>
> -
> List info/subscribe/unsubscribe? See
> http://www.freeradius.org/list/users.html
>
> __________ Information from ESET Smart Security, version of virus
> signature database 4579 (20091106) __________
>
> The message was checked by ESET Smart Security.
>
> http://www.eset.com
>
>
>

__________ Information from ESET Smart Security, version of virus signature database 4579 (20091106) __________

The message was checked by ESET Smart Security.

http://www.eset.com



-
List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html

[Craig Campbell]

I was able to get some bisect runs (I think).  However, I am encountering a
different error in these.

If radiusd is run in multithreaded mode, it hangs shortly after beginning.
This particular error has already been fixed (later).

Do you know if the Signal/Exit error depends upon multi threading?  i.e will
it happen if run with the -s option?

I'm trying to determine a strategy to find close in on the error we are
interested in.

Thanks,
-craig

----- Original Message -----
From: "Craig Campbell" <craig@...>
To: "FreeRadius users mailing list" <freeradius-users@...>
Sent: Friday, November 06, 2009 11:43 AM
Subject: Re: Unexpected "Exiting normally" 2.1.8?

> I'm afraid I did something incorrect.
>
> I ended up with,
>
> gcc -o .libs/radmin .libs/radmin.o /usr/lib/libreadline.so
> /usr/lib/libtermcap.so .libs/util.o .libs/log.o .libs/conffile.o
> /home/craig/src/freeradius/freeradius-server/src/lib/.libs/libfreeradius-radius.so
>  -lnsl -lresolv -lpthread -lreadline -ltermcap  -Wl,--rpath -Wl,/usr/local/lib
> /usr/lib/libreadline.so: could not read symbols: File in wrong format
> collect2: ld returned 1 exit status
>
> Which I suspect is result of my git bisect attempts....
>
> $git bisect start
> $git bisect bad
> $git bisect good 9261f3e0026323b2c397af13d02fbc5780908143
>
> $git bisect loggit bisect start
> # bad: [1489f5ffa5835e001499ee4f7a5f50a0da375613] Conf for debugging
> git bisect bad 1489f5ffa5835e001499ee4f7a5f50a0da375613
> # good: [9261f3e0026323b2c397af13d02fbc5780908143] Ensure that there is a
> cleanup event for proxied packets
> git bisect good 9261f3e0026323b2c397af13d02fbc5780908143
> # good: [9261f3e0026323b2c397af13d02fbc5780908143] Ensure that there is a
> cleanup event for proxied packets
> git bisect good 9261f3e0026323b2c397af13d02fbc5780908143
>
> $git bisect next
> $CFLAGS='-O0 -g' ./configure
> $make clean
> $make
> .....
> gcc -o .libs/radmin .libs/radmin.o /usr/lib/libreadline.so
> /usr/lib/libtermcap.so .libs/util.o .libs/log.o .libs/conffile.o
> /home/craig/src/freeradius/freeradius-server/src/lib/.libs/libfreeradius-radius.so
>  -lnsl -lresolv -lpthread -lreadline -ltermcap  -Wl,--rpath -Wl,/usr/local/lib
> /usr/lib/libreadline.so: could not read symbols: File in wrong format
> collect2: ld returned 1 exit status
>
>
> ----- Original Message -----
> From: "Alexander Clouter" <alex@...>
> To: <freeradius-users@...>
> Sent: Friday, November 06, 2009 10:15 AM
> Subject: Re: Unexpected "Exiting normally" 2.1.8?
>
>
>> Craig Campbell <craig@...> wrote:
>>>
>>> Regarding git bisect.  I know the current version is "bad", but I have
>>> no
>>> idea if any 2.1.8 version is 'Good'.
>>>
>>> If there a way to use git bisect without identifying 'good'?  How can we
>>> find the first version?  Can we assume it was 'good'?
>>>
>>> Sorry to be so non knowledgeable about git.
>>>
>> Please read my original email listing all the git cherry picks that I
>> used.
>>
>> Cheers
>>
>> --
>> Alexander Clouter
>> .sigmonster says: Logic is a pretty flower that smells bad.
>>
>> -
>> List info/subscribe/unsubscribe? See
>> http://www.freeradius.org/list/users.html
>>
>> __________ Information from ESET Smart Security, version of virus
>> signature database 4579 (20091106) __________
>>
>> The message was checked by ESET Smart Security.
>>
>> http://www.eset.com
>>
>>
>>
>
>
> __________ Information from ESET Smart Security, version of virus
> signature database 4579 (20091106) __________
>
> The message was checked by ESET Smart Security.
>
> http://www.eset.com
>
>
>
> -
> List info/subscribe/unsubscribe? See
> http://www.freeradius.org/list/users.html
>
> __________ Information from ESET Smart Security, version of virus
> signature database 4579 (20091106) __________
>
> The message was checked by ESET Smart Security.
>
> http://www.eset.com
>
>
>

__________ Information from ESET Smart Security, version of virus signature database 4580 (20091106) __________

The message was checked by ESET Smart Security.

http://www.eset.com



-
List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html

[Alan DeKok-2]

Craig Campbell wrote:
> I was able to get some bisect runs (I think).  However, I am
> encountering a different error in these.
>
> If radiusd is run in multithreaded mode, it hangs shortly after
> beginning. This particular error has already been fixed (later).

  Use a system that supports recursive mutexes.

> Do you know if the Signal/Exit error depends upon multi threading?  i.e
> will it happen if run with the -s option?

  It depends on multithreading.

  Alan DeKok.
-
List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html

[Craig Campbell]

Still running tests with bisect.

successful runs take some time to identify (a day).

Please let me know if the bug is identified, otherwise I'll keep plugging
away.

Thanks,
-craig

----- Original Message -----
From: "Alan DeKok" <aland@...>
To: "FreeRadius users mailing list" <freeradius-users@...>
Sent: Friday, November 06, 2009 5:04 PM
Subject: Re: Unexpected "Exiting normally" 2.1.8?

> Craig Campbell wrote:
>> I was able to get some bisect runs (I think).  However, I am
>> encountering a different error in these.
>>
>> If radiusd is run in multithreaded mode, it hangs shortly after
>> beginning. This particular error has already been fixed (later).
>
>  Use a system that supports recursive mutexes.
>
>> Do you know if the Signal/Exit error depends upon multi threading?  i.e
>> will it happen if run with the -s option?
>
>  It depends on multithreading.
>
>  Alan DeKok.
> -
> List info/subscribe/unsubscribe? See
> http://www.freeradius.org/list/users.html
>
> __________ Information from ESET Smart Security, version of virus
> signature database 4580 (20091106) __________
>
> The message was checked by ESET Smart Security.
>
> http://www.eset.com
>
>
>

__________ Information from ESET Smart Security, version of virus signature database 4611 (20091116) __________

The message was checked by ESET Smart Security.

http://www.eset.com



-
List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html

[Alan DeKok-2]

Craig Campbell wrote:
> Still running tests with bisect.
>
> successful runs take some time to identify (a day).
>
> Please let me know if the bug is identified, otherwise I'll keep
> plugging away.

  Thanks.  Once we know the commit, the fix should hopefully be easy.

  Alan DeKok.
-
List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html

[Craig Campbell]

Ok,
    I hope this is helpful.  Below please find the git bisect log.
There were a number of iterations with make errors which I then skipped.  I
suspect the errors were OS specific and were clearly fixed in later
iterations.

-bash-3.2$ git bisect log
git bisect start
# bad: [9dbc8974fdd2300a70293eda9c62bce20a3c9165] errormsg may be NULL
git bisect bad 9dbc8974fdd2300a70293eda9c62bce20a3c9165
# good: [321c0ae58641f709d115526bb564cbd8c4dab71d] Fix typos
git bisect good 321c0ae58641f709d115526bb564cbd8c4dab71d
# skip: [44119cccba8278fad9599969ec458e880dc25415] Check value of
Fall-Through, too
git bisect skip 44119cccba8278fad9599969ec458e880dc25415
# skip: [c0b41a30bff50e7b8d207a9397e5df2a25dc5e64] Replace references to
<ltdl.h> with <freeradius-devel/modpriv.h>
git bisect skip c0b41a30bff50e7b8d207a9397e5df2a25dc5e64
# good: [4f02141e5ee092f0f9a4a8b8b364897ad273d2a3] RFC 5580 and dictionary
git bisect good 4f02141e5ee092f0f9a4a8b8b364897ad273d2a3
# skip: [ada7bae4e68c3181759bbc5ab70a1dc6770c3857] Add scaffolding for proxy
listeners.
git bisect skip ada7bae4e68c3181759bbc5ab70a1dc6770c3857
# skip: [c7a9d2aa1b3fa91591ce95f19aa5ba42c102f4f7] Stop processing packets
when the socket is closed.
git bisect skip c7a9d2aa1b3fa91591ce95f19aa5ba42c102f4f7
# good: [f9302474d9bd38242d13e13f37043a745f460bf0] Fix values as note on
list
git bisect good f9302474d9bd38242d13e13f37043a745f460bf0
# skip: [0d8afaf03ef7b5df7556304f2664ac43dbe822f7] Clean up state machine so
it's more forgiving
git bisect skip 0d8afaf03ef7b5df7556304f2664ac43dbe822f7
# skip: [1a176810b6786c78ba744e1839d808cc6804788e] Check for NOOP from
opendir.c
git bisect skip 1a176810b6786c78ba744e1839d808cc6804788e
# skip: [f2273694594b65174b30680bef077485c9372f92] Forgot to include this...
git bisect skip f2273694594b65174b30680bef077485c9372f92
# skip: [7c638d0134c07c2481d0e7c866c1f8dd9d346048] 64-bit fixes.
git bisect skip 7c638d0134c07c2481d0e7c866c1f8dd9d346048
# skip: [9261f3e0026323b2c397af13d02fbc5780908143] Ensure that there is a
cleanup event for proxied packets
git bisect skip 9261f3e0026323b2c397af13d02fbc5780908143
# skip: [92ab5fef40320d1dbc3fe59db82cb20a3ec69249] Fixed compile error
git bisect skip 92ab5fef40320d1dbc3fe59db82cb20a3ec69249
# skip: [83b08deebddf31adcc4229359df905673f2b1703] Allow the packet API to
auto-discover TCP
git bisect skip 83b08deebddf31adcc4229359df905673f2b1703
# skip: [5fe2ab70a782fc1389748897f8d7838e8b63efc2] Removed unnecessary line
git bisect skip 5fe2ab70a782fc1389748897f8d7838e8b63efc2
# skip: [f687388c0f7b9bdc81db3482e319e684231bca8e] Document TCP options for
clients and home servers.
git bisect skip f687388c0f7b9bdc81db3482e319e684231bca8e
# skip: [4ca219b1f1ab68fc8434072e51a8e4b95cf37c16] Be more flexible about
parsing detail files
git bisect skip 4ca219b1f1ab68fc8434072e51a8e4b95cf37c16
# skip: [ecf751a2a662d8749f45fa77f8b023b37b01056c] Set broadcast &&
reuseaddr before binding to socket
git bisect skip ecf751a2a662d8749f45fa77f8b023b37b01056c
# skip: [52880d0020b7b900ae8383b142b08e4e11cde639] Fixed typo && include
attrs.access_challenge in build
git bisect skip 52880d0020b7b900ae8383b142b08e4e11cde639
# skip: [05b63f38b036995164a7d3f5cbe5d81676058d95] Track the number of
outstanding packets on a TCP connection.
git bisect skip 05b63f38b036995164a7d3f5cbe5d81676058d95
# good: [4d73838c46c713ac427f7da6b5c40fe2a87bd457] As posted to the list
git bisect good 4d73838c46c713ac427f7da6b5c40fe2a87bd457
# skip: [545ed1e65f5852a3c1fb2001221531f76b6abb27] Allow outgoing TCP
connections to home servers.
git bisect skip 545ed1e65f5852a3c1fb2001221531f76b6abb27
# skip: [6840a546d793271a36cdf331a1492a49573c11ee] Reference $(INCLTDL)
instead of fixed link
git bisect skip 6840a546d793271a36cdf331a1492a49573c11ee
# skip: [1013e94b66064f24170e394e63ba4f093c141d74] Start simplifying the
code that encodes attributes
git bisect skip 1013e94b66064f24170e394e63ba4f093c141d74
# skip: [b853a84e6c4ccd5d9e2c4ad9da2c421a234e887f] Fix openssl checks
git bisect skip b853a84e6c4ccd5d9e2c4ad9da2c421a234e887f
# skip: [e6c108b5e68f0ebe1f8a5d4a1c08500bbadcdab4] Fix passwords to have
even length
git bisect skip e6c108b5e68f0ebe1f8a5d4a1c08500bbadcdab4
# skip: [f0697780861ace0e52ba8fd0526454097caf567f] No need to include
modules.h twice
git bisect skip f0697780861ace0e52ba8fd0526454097caf567f
# skip: [751e9a39b2221a2623001a4611021a8e01cf4375] Increase max_sessions
git bisect skip 751e9a39b2221a2623001a4611021a8e01cf4375
# skip: [828ef4d40c8b43977b48fb11d02cc69263f3ce0f] Be less forgiving about
the allowed operators.
git bisect skip 828ef4d40c8b43977b48fb11d02cc69263f3ce0f
# skip: [e237107e1dca922dab291c5b011468ee24b768c2] event.c frees the
listener, so we don't need to
git bisect skip e237107e1dca922dab291c5b011468ee24b768c2
# skip: [9b70af0c517daad7d374f4cc948488429d3a9cc0] Print env vars in parent,
not child
git bisect skip 9b70af0c517daad7d374f4cc948488429d3a9cc0
# skip: [66b8e171eb8f7f67d09032aed4f541ed88523f9a] Fix arguments to
client_find
git bisect skip 66b8e171eb8f7f67d09032aed4f541ed88523f9a
# skip: [58ee226a502cf2fb2e33ba48c46e9a78b73b1f9c] Added sample configs for
MySQL cluster
git bisect skip 58ee226a502cf2fb2e33ba48c46e9a78b73b1f9c
# skip: [2065201e7df32961cc870d7e862ca9b2f9ae59f7] Moved illegal attributes
to the new dictionary
git bisect skip 2065201e7df32961cc870d7e862ca9b2f9ae59f7
# skip: [f2d96581f98990d24991c99a681d018a3df85e92] Define names
git bisect skip f2d96581f98990d24991c99a681d018a3df85e92
# skip: [4baebf8202d7db372a9ad2ce5026ec6c986f0de7] Allow old-style
dictionary formats, too
git bisect skip 4baebf8202d7db372a9ad2ce5026ec6c986f0de7
# skip: [b1e815d0b4bec01f9721d4b92786960b2218f149] Write the PID file as
late as possible
git bisect skip b1e815d0b4bec01f9721d4b92786960b2218f149
# skip: [e04b62f1bd257489bd92ccc584b0886c7e2011e8] Jumbo patch to clean up
socket handling
git bisect skip e04b62f1bd257489bd92ccc584b0886c7e2011e8
# skip: [a92700b3fb88239ccb0db9f5ece68dd430937df3] Fix typo
git bisect skip a92700b3fb88239ccb0db9f5ece68dd430937df3
# skip: [28d8dd3de3e4251b252a6b2846bfc4079df09c66] Get private key
passphrase from keychain using certadmin command.
git bisect skip 28d8dd3de3e4251b252a6b2846bfc4079df09c66
# good: [d047c4beb2130c196d266d39ba6974bc2ecac10a] Move "set state" to
before log message
git bisect good d047c4beb2130c196d266d39ba6974bc2ecac10a
# skip: [731db3b088a482726c765b56f46a33e2a4e45522] More plumbing to get the
server to listen on TCP sockets.
git bisect skip 731db3b088a482726c765b56f46a33e2a4e45522
# skip: [9491d6eb7b963532855ccc8a63a523a2a1e3af2b] Use packet codes from
libradius
git bisect skip 9491d6eb7b963532855ccc8a63a523a2a1e3af2b
# skip: [057c7ac764a4639f715edcbde7dc22491b79be62] Don't use source IP for
EAP packets.
git bisect skip 057c7ac764a4639f715edcbde7dc22491b79be62
# skip: [137e3759b2ffc0c4f99064dadbd7461d3e86ae2a] Moved Ascends illegal
attributes to their own file
git bisect skip 137e3759b2ffc0c4f99064dadbd7461d3e86ae2a
# skip: [fbdc02ad699b9bc4d410daaa54f76df7141d2f64] Ensure that cached SSL
sessions have data
git bisect skip fbdc02ad699b9bc4d410daaa54f76df7141d2f64
# skip: [1fef8c64bf31668808bb9c2a67c480d9d0a7f2d6] Assign variable before
using it
git bisect skip 1fef8c64bf31668808bb9c2a67c480d9d0a7f2d6
# skip: [fa0e98d1056e22fa413078dbd8c3fe0d85532826] Changed order of code to
avoid race conditions
git bisect skip fa0e98d1056e22fa413078dbd8c3fe0d85532826
# good: [7d8c35a78970319b948ef3356d1c5504d6c015c2] Print out a little more
information
git bisect good 7d8c35a78970319b948ef3356d1c5504d6c015c2
# skip: [12ead56dffca9b3ecddc8a7860a1ef5b5361b374] Return rather than use
the same ptr twice
git bisect skip 12ead56dffca9b3ecddc8a7860a1ef5b5361b374
# skip: [1700127c8a7150f57056495a2980fd132dafdb92] As posted to the list
git bisect skip 1700127c8a7150f57056495a2980fd132dafdb92
# skip: [215cb16a373f3cdb88a6c196c7cb53ef69b5fb4c] 64-bit fixes and return
NOOP for AD users.
git bisect skip 215cb16a373f3cdb88a6c196c7cb53ef69b5fb4c
# skip: [d9dd62aae7baa5346f19236cead4414c03546d45] Conf for debugging
git bisect skip d9dd62aae7baa5346f19236cead4414c03546d45
# skip: [6e45792b73caf67c01e4776065c52acc62d28d2b] Mark home server dead if
it doesn't respond to pings
git bisect skip 6e45792b73caf67c01e4776065c52acc62d28d2b
# good: [18987ef653986dc1647a7c43144c51dddaa96671] Allow home servers to use
TCP
git bisect good 18987ef653986dc1647a7c43144c51dddaa96671
# skip: [f6e2dba8a7e4dd31d36d5b8ee434d21600e3f99f] Simplify the code
git bisect skip f6e2dba8a7e4dd31d36d5b8ee434d21600e3f99f
# skip: [a6e46823d1756782b974257184e35d4e796c98a3] Always initialize proto
git bisect skip a6e46823d1756782b974257184e35d4e796c98a3
# skip: [81f6cad3846b61227fd4c5f92959306a8b7e140c] Include proto in API, no
matter what build options
git bisect skip 81f6cad3846b61227fd4c5f92959306a8b7e140c
# skip: [0ce61f286385679608bbb82967e371641c47d381] More ifdef's and
assertions for checkign TCP != UDP
git bisect skip 0ce61f286385679608bbb82967e371641c47d381
# skip: [83c2cd412b1208e67381372baa73c779cd2848f6] More detailed debugging
for detail
git bisect skip 83c2cd412b1208e67381372baa73c779cd2848f6
# skip: [092f0ea30cdfc2d669afe47061fafb9407269641] Initialize via attr
git bisect skip 092f0ea30cdfc2d669afe47061fafb9407269641
# skip: [2db0b3fe16a9d460efc63cf76e38584419dcdfe0] Use new API
git bisect skip 2db0b3fe16a9d460efc63cf76e38584419dcdfe0
# skip: [98b22609015439b16cc62cf45e4472a14377da2a] Retry if there was no
response to the packet.
git bisect skip 98b22609015439b16cc62cf45e4472a14377da2a
# skip: [1628ef2387d9f7a89b3c2ff8945f49777eb135f1] Be more restrictive on
bad input
git bisect skip 1628ef2387d9f7a89b3c2ff8945f49777eb135f1
# skip: [572d4fd5f3f735e17bce2980af72fed12376fb62] Scaffolding to make it
build.
git bisect skip 572d4fd5f3f735e17bce2980af72fed12376fb62
# skip: [64700e41098a874581d683c8606c94f9ad23079d] Check for undefined
types, too
git bisect skip 64700e41098a874581d683c8606c94f9ad23079d
# skip: [f4dd3a6e803219b61f3ec1d1b7f3767ee54e8eec] Free tcp structure, too
git bisect skip f4dd3a6e803219b61f3ec1d1b7f3767ee54e8eec
# skip: [382b6c2223ba1a233ca9f4d248beb888a0123f3e] Print more descriptive
error message for too many EAP sessions
git bisect skip 382b6c2223ba1a233ca9f4d248beb888a0123f3e
# skip: [5aa01c58d91063b5bbbf5aef941137d7cf638bbe] Changed stop packet msg
to debug rather than error
git bisect skip 5aa01c58d91063b5bbbf5aef941137d7cf638bbe
# skip: [e69be18535bd8b9a2cfb50a9df7cb44e3129ab4c] Added more debugging
messages
git bisect skip e69be18535bd8b9a2cfb50a9df7cb44e3129ab4c
# skip: [817e64f14df0e5816d87784f995e8fc4a240e048] Initialize proto for
old-style realms
git bisect skip 817e64f14df0e5816d87784f995e8fc4a240e048
# skip: [d711a368ebf0e057e54596d22584ca2ce37e209c] Make
client/port/key-balance more like fail-over
git bisect skip d711a368ebf0e057e54596d22584ca2ce37e209c
# skip: [ff89e4cac7f2a9256c7d360b1d53a1eb69a28f40] More plumbing to get to
home servers via TCP
git bisect skip ff89e4cac7f2a9256c7d360b1d53a1eb69a28f40
# skip: [fe4bf0a8d6d7e168e0c6729115df1315abbe5e20] Fix typo
git bisect skip fe4bf0a8d6d7e168e0c6729115df1315abbe5e20
# skip: [732917380982c0aa5ff862ffa2d901fbe52dac36] Allow radclient to
send/receive RADIUS over TCP
git bisect skip 732917380982c0aa5ff862ffa2d901fbe52dac36
# skip: [a4202aeb848174ed430fd29573e3dd2db78ae2a1] fix debian/rules to
honour CFLAGS
git bisect skip a4202aeb848174ed430fd29573e3dd2db78ae2a1
# skip: [6a6d2b450fd7ddff65e9f73bbe96ba3f5f914f08] Check src_port, not
dst_port
git bisect skip 6a6d2b450fd7ddff65e9f73bbe96ba3f5f914f08
# skip: [30adbf8230730a7503f5e3654df90c5c2a38a8ed] Call detach only if
function exists
git bisect skip 30adbf8230730a7503f5e3654df90c5c2a38a8ed
# skip: [8fa1a08726aad4f379c7bcc6df608f8d79594a34] Removed recursive
mutexes.
git bisect skip 8fa1a08726aad4f379c7bcc6df608f8d79594a34
# skip: [ce2a48e678fd80199b886aeda654ed2f94340c19] Allow clients to use TCP
git bisect skip ce2a48e678fd80199b886aeda654ed2f94340c19
-bash-3.2$
----- Original Message -----
From: "Alan DeKok" <aland@...>
To: "FreeRadius users mailing list" <freeradius-users@...>
Sent: Monday, November 16, 2009 11:02 AM
Subject: Re: Unexpected "Exiting normally" 2.1.8?

> Craig Campbell wrote:
>> Still running tests with bisect.
>>
>> successful runs take some time to identify (a day).
>>
>> Please let me know if the bug is identified, otherwise I'll keep
>> plugging away.
>
>  Thanks.  Once we know the commit, the fix should hopefully be easy.
>
>  Alan DeKok.
> -
> List info/subscribe/unsubscribe? See
> http://www.freeradius.org/list/users.html
>
> __________ Information from ESET Smart Security, version of virus
> signature database 4612 (20091116) __________
>
> The message was checked by ESET Smart Security.
>
> http://www.eset.com
>
>
>

__________ Information from ESET Smart Security, version of virus signature database 4617 (20091118) __________

The message was checked by ESET Smart Security.

http://www.eset.com



-
List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html

[Alan DeKok-2]

Craig Campbell wrote:
> Ok,
>    I hope this is helpful.  Below please find the git bisect log.
> There were a number of iterations with make errors which I then
> skipped.  I suspect the errors were OS specific and were clearly fixed
> in later iterations.
>
> -bash-3.2$ git bisect log
> git bisect start
> # bad: [9dbc8974fdd2300a70293eda9c62bce20a3c9165] errormsg may be NULL

  Huh...  Since that commit doesn't help the reported bug, it's likely
best to just revert it.  Oh well.

  Alan DeKok.
-
List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html

[Craig Campbell]

Once you have another version (reverted), I can test again...

I am really unfamiliar with git, so I may need a hint as to getting  the
correct version for testing.

Thanks,
-craig
----- Original Message -----
From: "Alan DeKok" <aland@...>
To: "FreeRadius users mailing list" <freeradius-users@...>
Sent: Wednesday, November 18, 2009 12:31 PM
Subject: Re: Unexpected "Exiting normally" 2.1.8?

> Craig Campbell wrote:
>> Ok,
>>    I hope this is helpful.  Below please find the git bisect log.
>> There were a number of iterations with make errors which I then
>> skipped.  I suspect the errors were OS specific and were clearly fixed
>> in later iterations.
>>
>> -bash-3.2$ git bisect log
>> git bisect start
>> # bad: [9dbc8974fdd2300a70293eda9c62bce20a3c9165] errormsg may be NULL
>
>  Huh...  Since that commit doesn't help the reported bug, it's likely
> best to just revert it.  Oh well.
>
>  Alan DeKok.
> -
> List info/subscribe/unsubscribe? See
> http://www.freeradius.org/list/users.html
>
> __________ Information from ESET Smart Security, version of virus
> signature database 4618 (20091118) __________
>
> The message was checked by ESET Smart Security.
>
> http://www.eset.com
>
>
>

__________ Information from ESET Smart Security, version of virus signature database 4618 (20091118) __________

The message was checked by ESET Smart Security.

http://www.eset.com



-
List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html

[Alan DeKok-2]

Craig Campbell wrote:
> Once you have another version (reverted), I can test again...
>
> I am really unfamiliar with git, so I may need a hint as to getting  the
> correct version for testing.

  I've reverted the problem commit.  It doesn't fix the PostgreSQL
issue, and it causes other problems.

  The fix is now in the "stable" branch.

  Alan DeKok.
-
List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html