Dropping packet maximum number of childs

[Bert Colemont]

Hey all,

 

We have a radius server running on 4 cpu machine with 4GB ram.

 

Now frequently the machine gets a load of 20, of radiusd, and in the log the message : Dropping packet : Maximum number of children active

 

I have tried to increase the number of child processes, but then radius crashes (if I go above 20) frequently.

I have looked, and io is it also not.

Our network is also not the problem.

I tried so rais the ulimit -n but also no resolution.

Date and time are also correct synced

Iptables is open on the needed ports

 

Really I don't know what to do next, can anyone please help me?

 

We are running it on CentOS 5.2 with all updates installed

Radius version is 1.5

 

 

_______________________________________________
Bug-gnu-radius mailing list
Bug-gnu-radius@...
http://lists.gnu.org/mailman/listinfo/bug-gnu-radius

[Sergey Poznyakoff-2]

Bert Colemont <bert.colemont@...> ha escrit:

>    Now frequently the machine gets a load of 20, of radiusd, and in the
>    log the message : Dropping packet : Maximum number of children active

There is not enough information to correctly diagnose your problem:

1. What is your configuration? In particular, files radd/config, users.
hints, huntgroups. If you use SQL, then raddb/sqlserver also.
2. What command options are used when starting daemon?
2. What do you get in your logs?

Regards,
Sergey


_______________________________________________
Bug-gnu-radius mailing list
Bug-gnu-radius@...
http://lists.gnu.org/mailman/listinfo/bug-gnu-radius

[Bert Colemont]

Hello Sergey,

We use a SQL server (separated machine)

the config file looks like this :

# For detailed description, run:
#       info Radius config

option {
        radiusd-user radius;
        max-requests 1024;
        resolve no;
};


logging {
        prefix-hook "default_log_prefix";
        channel default {
                file "radius.log";
                print-category yes;
                print-level yes;
        };
        category auth {
                print-auth yes;
                print-failed-pass yes;
                channel default;
        };
        category info {
                channel default;
        };
        category debug {
                channel default;
                #level radiusd=1,files,sql=99,auth,input;
        };
        category * {
                channel default;
        };
};


auth {
        listen 193.109.184.73:1812,193.109.184.74:1812,192.168.3.2:1812;
        max-requests 127;
        request-cleanup-delay 2;
        detail yes;
        strip-names yes;
        checkrad-assume-logged yes;
};

acct {
        listen 193.109.184.73:1813,193.109.184.74:1813,192.168.3.2:1813;
        max-requests 127;
        request-cleanup-delay 2;
};

rewrite {
        load "checknas.rw";
        load "log-hook.rw";
};

# snmp {
#       listen no;
# };


our radius.log file is filled with

Jan 05 11:40:33 Auth.notice: (Access-Request 172.28.3.158 27 "dsl28543" CLID=*BAS-ANTWERPEN*19/0*247*487): Dropping packet: Maximum number of children active
Jan 05 11:40:33 Auth.notice: (Access-Request 172.28.192.233 11 "dsl35220" CLID=*MSR02MAR1*13/1*213*84): Dropping packet: Maximum number of children active
Jan 05 11:40:34 Acct.notice: (Accounting-Request 172.28.3.158 3 "dsl29126" CLID=*BAS-ANTWERPEN*21/1*242*304): Dropping packet: Maximum number of children active
Jan 05 11:40:34 Acct.notice: (Accounting-Request 172.28.3.190 2 "dsl37161" CLID=*BAS-LEUVEN*21/1*245*386): Dropping packet: Maximum number of children active
Jan 05 11:40:34 Acct.notice: (Accounting-Request 172.28.3.10 2 "dsl31163" CLID=*BAS-ROGIER*27/1*241*1846): Dropping packet: Maximum number of children active
Jan 05 11:40:34 Auth.notice: (Access-Request 172.28.3.10 96 "dsl31163" CLID=*BAS-ROGIER*27/1*241*1846): Dropping packet: Maximum number of children active
Jan 05 11:40:34 Acct.notice: (Accounting-Request 172.28.3.182 8 "dsl43156" CLID=*BAS-HASSELT*23/0*210*372): Dropping packet: Maximum number of children active
Jan 05 11:40:34 Acct.notice: (Accounting-Request 172.28.3.158 5 "dsl31350" CLID=*BAS-ANTWERPEN*27/1*29*377): Dropping packet: Maximum number of children active
Jan 05 11:40:34 Acct.notice: (Accounting-Request 172.28.3.158 9 "dsl43067" CLID=*BAS-ANTWERPEN*19/0*246*421): Dropping packet: Maximum number of children active
Jan 05 11:40:34 Acct.notice: (Accounting-Request 172.28.3.198 40 "dsl30089" CLID=*BAS-LIEGE*23/1*217*128): Dropping packet: Maximum number of children active
Jan 05 11:40:34 Acct.notice: (Accounting-Request 172.28.18.98 3 "dsl43597" CLID=*BAS-BERCHEM*23/1*204*898): Dropping packet: Maximum number of children active
Jan 05 11:40:34 Auth.notice: (Access-Request 172.28.192.234 30 "dsl20464" CLID=*MSR02STR1*14/1*248*99): Dropping packet: Maximum number of children active
Jan 05 11:40:34 Auth.notice: (Access-Request 172.28.192.238 26 "dsl45173" CLID=*MSR11HAS1*13/1*225*310): Dropping packet: Maximum number of children active
Jan 05 11:40:35 Auth.notice: (Access-Request 172.28.192.240 2 "dsl18008" CLID=*MSR15MES1*11/1*229*1305): Dropping packet: Maximum number of children active
Jan 05 11:40:35 Acct.notice: (Accounting-Request 172.28.3.10 8 "dsl31163" CLID=*BAS-ROGIER*27/1*241*1846): Dropping packet: Maximum number of children active


on a previous machine running on CentOS with kernel 2.4.25 running with gnu radius 1.2 has never got the problem.

This machine is a CentOS 5.2 with gnu radius 1.4 or 1.5 (tried both)

Hope you can help me?

2009/1/3 Sergey Poznyakoff <gray@...>

Bert Colemont <bert.colemont@...> ha escrit:

>    Now frequently the machine gets a load of 20, of radiusd, and in the
>    log the message : Dropping packet : Maximum number of children active

There is not enough information to correctly diagnose your problem:

1. What is your configuration? In particular, files radd/config, users.
hints, huntgroups. If you use SQL, then raddb/sqlserver also.
2. What command options are used when starting daemon?
2. What do you get in your logs?

Regards,
Sergey

_______________________________________________
Bug-gnu-radius mailing list
Bug-gnu-radius@...
http://lists.gnu.org/mailman/listinfo/bug-gnu-radius

[Bert Colemont]

Hello Sergey,

We use a SQL server (separated machine)

the config file looks like this :

# For detailed description, run:
#       info Radius config

option {
        radiusd-user radius;
        max-requests 1024;
        resolve no;
};


logging {
        prefix-hook "default_log_prefix";
        channel default {
                file "radius.log";
                print-category yes;
                print-level yes;
        };
        category auth {
                print-auth yes;
                print-failed-pass yes;
                channel default;
        };
        category info {
                channel default;
        };
        category debug {
                channel default;
                #level radiusd=1,files,sql=99,auth,input;
        };
        category * {
                channel default;
        };
};


auth {
        listen 193.109.184.73:1812,193.109.184.74:1812,192.168.3.2:1812;
        max-requests 127;
        request-cleanup-delay 2;
        detail yes;
        strip-names yes;
        checkrad-assume-logged yes;
};

acct {
        listen 193.109.184.73:1813,193.109.184.74:1813,192.168.3.2:1813;
        max-requests 127;
        request-cleanup-delay 2;
};

rewrite {
        load "checknas.rw";
        load "log-hook.rw";
};

# snmp {
#       listen no;
# };


our radius.log file is filled with

Jan 05 11:40:33 Auth.notice: (Access-Request 172.28.3.158 27 "dsl28543" CLID=*BAS-ANTWERPEN*19/0*247*487): Dropping packet: Maximum number of children active
Jan 05 11:40:33 Auth.notice: (Access-Request 172.28.192.233 11 "dsl35220" CLID=*MSR02MAR1*13/1*213*84): Dropping packet: Maximum number of children active
Jan 05 11:40:34 Acct.notice: (Accounting-Request 172.28.3.158 3 "dsl29126" CLID=*BAS-ANTWERPEN*21/1*242*304): Dropping packet: Maximum number of children active
Jan 05 11:40:34 Acct.notice: (Accounting-Request 172.28.3.190 2 "dsl37161" CLID=*BAS-LEUVEN*21/1*245*386): Dropping packet: Maximum number of children active
Jan 05 11:40:34 Acct.notice: (Accounting-Request 172.28.3.10 2 "dsl31163" CLID=*BAS-ROGIER*27/1*241*1846): Dropping packet: Maximum number of children active
Jan 05 11:40:34 Auth.notice: (Access-Request 172.28.3.10 96 "dsl31163" CLID=*BAS-ROGIER*27/1*241*1846): Dropping packet: Maximum number of children active
Jan 05 11:40:34 Acct.notice: (Accounting-Request 172.28.3.182 8 "dsl43156" CLID=*BAS-HASSELT*23/0*210*372): Dropping packet: Maximum number of children active
Jan 05 11:40:34 Acct.notice: (Accounting-Request 172.28.3.158 5 "dsl31350" CLID=*BAS-ANTWERPEN*27/1*29*377): Dropping packet: Maximum number of children active
Jan 05 11:40:34 Acct.notice: (Accounting-Request 172.28.3.158 9 "dsl43067" CLID=*BAS-ANTWERPEN*19/0*246*421): Dropping packet: Maximum number of children active
Jan 05 11:40:34 Acct.notice: (Accounting-Request 172.28.3.198 40 "dsl30089" CLID=*BAS-LIEGE*23/1*217*128): Dropping packet: Maximum number of children active
Jan 05 11:40:34 Acct.notice: (Accounting-Request 172.28.18.98 3 "dsl43597" CLID=*BAS-BERCHEM*23/1*204*898): Dropping packet: Maximum number of children active
Jan 05 11:40:34 Auth.notice: (Access-Request 172.28.192.234 30 "dsl20464" CLID=*MSR02STR1*14/1*248*99): Dropping packet: Maximum number of children active
Jan 05 11:40:34 Auth.notice: (Access-Request 172.28.192.238 26 "dsl45173" CLID=*MSR11HAS1*13/1*225*310): Dropping packet: Maximum number of children active
Jan 05 11:40:35 Auth.notice: (Access-Request 172.28.192.240 2 "dsl18008" CLID=*MSR15MES1*11/1*229*1305): Dropping packet: Maximum number of children active
Jan 05 11:40:35 Acct.notice: (Accounting-Request 172.28.3.10 8 "dsl31163" CLID=*BAS-ROGIER*27/1*241*1846): Dropping packet: Maximum number of children active


on a previous machine running on CentOS with kernel 2.4.25 running with gnu radius 1.2 has never got the problem.

This machine is a CentOS 5.2 with gnu radius 1.4 or 1.5 (tried both)

Hope you can help me?

2009/1/3 Sergey Poznyakoff <gray@...>

Bert Colemont <bert.colemont@...> ha escrit:

>    Now frequently the machine gets a load of 20, of radiusd, and in the
>    log the message : Dropping packet : Maximum number of children active

There is not enough information to correctly diagnose your problem:

1. What is your configuration? In particular, files radd/config, users.
hints, huntgroups. If you use SQL, then raddb/sqlserver also.
2. What command options are used when starting daemon?
2. What do you get in your logs?

Regards,
Sergey

_______________________________________________
Bug-gnu-radius mailing list
Bug-gnu-radius@...
http://lists.gnu.org/mailman/listinfo/bug-gnu-radius

[Sergey Poznyakoff-2]

Bert Colemont <bert.colemont@...> ha escrit:

>    our radius.log file is filled with
>    Jan 05 11:40:33 Auth.notice: (Access-Request 172.28.3.158 27 "dsl28543"
>    CLID=*BAS-ANTWERPEN*19/0*247*487): Dropping packet: Maximum number of
>    children active

>    We use a SQL server (separated machine)

Is radius able to connect to your SQL database? Do you see any
connections in the DB logs? If so, how long does it take to authenticate
a user (i.e. to perform all authentication queries from your
raddb/sqlserver) on a DB?

Regards,
Sergey


_______________________________________________
Bug-gnu-radius mailing list
Bug-gnu-radius@...
http://lists.gnu.org/mailman/listinfo/bug-gnu-radius

[Bert Colemont]

Hello Sergey,

Yes, as somethimes we see in the log also

Jan 05 17:25:00 Acct.notice: (Accounting-Request 172.28.0.10 2 "dsl36941" CLID=*BAS-MECHELEN*21/0*211*153): Dropping packet: Maximum number of children active
Jan 05 17:25:00 Auth.notice: (Access-Request 172.28.192.240 26 "dsl18008" CLID=*MSR15MES1*11/1*229*1305): Dropping packet: Maximum number of children active
Jan 05 17:25:00 Auth.notice: (Access-Request 172.28.192.234 16 "dsl20464" CLID=*MSR02STR1*14/1*248*99): Dropping packet: Maximum number of children active
Jan 05 17:25:00 Auth.notice: (Access-Request 172.28.18.106 77 "dsl38588" CLID=*BAS-ASSEBROEK*15/0*211*1007): Dropping packet: Maximum number of children active
Jan 05 17:25:00 Acct.notice: (Accounting-Request 172.30.38.102 12 "dsl28436" CLID=*BAS-GRETRY2*19/1*229*554): Dropping packet: Maximum number of children active
Jan 05 17:25:01 Auth.notice: (Access-Request 172.28.3.158 6 "dsl24392" CLID=*BAS-ANTWERPEN*21/1*24*876): No such user [dsl24392]
Jan 05 17:25:03 Auth.notice: (Access-Request 172.28.192.233 12 "dsl23266" CLID=*MSR02MAR1*13/1*211*1054): Login OK [dsl23266]

I have also tested this from home, and sometimes I can connect in a second, sometimes it takes me up to 5 min :(

Really, I don't know what to do next, I tried all version of gnu radius that compile with gcc-4 (1.4, 1.5, 1.6)
I also don't see any differance with our old server who never had that problem.

Also when the Dropping packets increase, I see (via uptime) the load of the machine sometimes rising to even 30!

[root@anubis raddb]# uptime
 17:27:36 up 60 days,  4:58,  1 user,  load average: 18.97, 16.91, 13.50

2009/1/5 Sergey Poznyakoff <gray@...>

Bert Colemont <bert.colemont@...> ha escrit:

>    our radius.log file is filled with
>    Jan 05 11:40:33 Auth.notice: (Access-Request 172.28.3.158 27 "dsl28543"
>    CLID=*BAS-ANTWERPEN*19/0*247*487): Dropping packet: Maximum number of
>    children active

>    We use a SQL server (separated machine)

Is radius able to connect to your SQL database? Do you see any
connections in the DB logs? If so, how long does it take to authenticate
a user (i.e. to perform all authentication queries from your
raddb/sqlserver) on a DB?

Regards,
Sergey

_______________________________________________
Bug-gnu-radius mailing list
Bug-gnu-radius@...
http://lists.gnu.org/mailman/listinfo/bug-gnu-radius

[Sergey Poznyakoff-2]

Bert Colemont <bert.colemont@...> ha escrit:

>    I have also tested this from home, and sometimes I can connect in a
>    second, sometimes it takes me up to 5 min :(

That's not what I meant. How long does it take for the *SQL*
authentication queries to complete? You can easily measure it by looking
into your mysql.log files.

Regards,
Sergey


_______________________________________________
Bug-gnu-radius mailing list
Bug-gnu-radius@...
http://lists.gnu.org/mailman/listinfo/bug-gnu-radius

[Bert Colemont]

You mean this or so?

INSERT INTO connections VALUES(1,'username','2009-01-05 18:00:11','10.10.10.10',3908692444,'E8F9E9DC-49624A2F',0,0,0,0,0,0,'198.198.198.198','','*BAS-SOMEWHERE*19/0*243*72','0');

2009/1/5 Sergey Poznyakoff <gray@...>

Bert Colemont <bert.colemont@...> ha escrit:

>    I have also tested this from home, and sometimes I can connect in a
>    second, sometimes it takes me up to 5 min :(

That's not what I meant. How long does it take for the *SQL*
authentication queries to complete? You can easily measure it by looking
into your mysql.log files.

Regards,
Sergey

_______________________________________________
Bug-gnu-radius mailing list
Bug-gnu-radius@...
http://lists.gnu.org/mailman/listinfo/bug-gnu-radius

[Sergey Poznyakoff-2]

Bert Colemont <bert.colemont@...> ha escrit:

>    You mean this or so?
>    INSERT INTO connections VALUES(1,'username','2009-01-05
>    18:00:11','10.10.10.10',3908692444,'E8F9E9DC-49624A2F',0,0,0,0,0,0,'198
>    .198.198.198','','*BAS-SOMEWHERE*19/0*243*72','0');

No, Bert, that's accounting, not authentication. It is not relevant in
your case, at least for now. Your problems are related to
authentication. What happens is that authentication sub-processes take
too long to reply to requests. This can happen due to the following two
reasons:

1. Request time-to-live is too short[1].

This cannot be the reason, because your raddb/config does not override
time-to-live, so its default value of 60 seconds remains in effect, and,
I suppose, this time is more than enough for any real-life application.

2. Authentication scheme takes too long to complete.

This is the most probable reason for your problem. I don't know how
exactly you authenticate your users (you didn't supply your
raddb/users, nor huntgroups, nor hints), so I can only make guesses. If
you are using *only* SQL authentication, it can be the reason of the slowdown.
To verify if this is so, you have to: 1. measure average time of connecting
to the SQL database, 2. measure average time of completing SQL
authentication. Then, you'll obtain average time of authentication
bu adding these two values.

To measure (2), either look in your SQL logs for a series of
authentication queries, or execute these queries manually from a mysql
console. I don't know how you configured SQL authentication (because you
didn't supply your raddb/sqlserver either), so I can't tell exactly what
queries you should be looking for/executing. You will find a detailed
description of SQL authentication and authorization process in [2] and
[3], which in conjunction with your actual raddb/sqlserver will help you
determine the exact sequence of queries.

Regards,
Sergey

[1]. http://www.gnu.org/software/radius/manual/html_node/auth.html
[2]. http://www.gnu.org/software/radius/manual/html_node/Authentication-Server-
Parameters.html
[3]. http://www.gnu.org/software/radius/manual/html_node/Authorization-Paramete
rs.html


_______________________________________________
Bug-gnu-radius mailing list
Bug-gnu-radius@...
http://lists.gnu.org/mailman/listinfo/bug-gnu-radius