1024 file descriptors is good

[Mariel Sebedio]

Hi, I have a RHEL 5.4 with squid3.0STABLE19 and have a performance
problems...

My cache.log not report warning

When I see in cachemgr.cgi I just have a 1024 File descriptors...

My ulimit -n is 1024, I need to modificated this and configure another
time or I have a another options to increase the File descriptor for
Squid3.0.
I only fount diferente options for squid 2.7 or less

Thanks

--
Lic. Mariel Sebedio
Division Computos y Sistemas
Tel [phone] int 2307
INVAP S.E. - www.invap.com.ar

[Leonardo Rodrigues Magalhães]

Mariel Sebedio escreveu:
> Hi, I have a RHEL 5.4 with squid3.0STABLE19 and have a performance
> problems...
>
> My cache.log not report warning
>
> When I see in cachemgr.cgi I just have a 1024 File descriptors...
>

    if you're not getting the famous WARNING in your cache.log

WARNING! Your cache is running out of filedescriptors

    then you really dont need to worry about 1024 FDs. That's now too
much, but that's pretty enough for having a good number of simultaneos
clients.

    Filedescriptors problems (running low on them) could give you some
problems, but in any case you would see the warning on your logs. If
you're not seeing it, then problem is not filedescriptor related. And if
that's not filedescriptor related, raising it wont change anything.

    your performance problem is somewhere else .....




--


        Atenciosamente / Sincerily,
        Leonardo Rodrigues
        Solutti Tecnologia
        http://www.solutti.com.br

        Minha armadilha de SPAM, NÃO mandem email
        gertrudes@...
        My SPAMTRAP, do not email it

[Mariel Sebedio]

In this server I have a

# cat /proc/sys/net/ipv4/netfilter/ip_conntrack_count
2086

Here is the stadistic, could you help me to find anything wrong?

Some times, this page is slow for the access too...

Squid Object Cache: Version 3.0.STABLE19

Start Time: Thu, 15 Oct 2009 16:00:32 GMT
Current Time: Wed, 21 Oct 2009 11:44:52 GMT


Connection information for squid:
        Number of clients accessing cache: 251
        Number of HTTP requests received: 3320854
        Number of ICP messages received: 0
        Number of ICP messages sent: 0
        Number of queued ICP replies: 0
        Number of HTCP messages received: 0
        Number of HTCP messages sent: 0
        Request failure ratio: 0.00
        Average HTTP requests per minute since start: 396.1
        Average ICP messages per minute since start: 0.0
        Select loop called: 294394197 times, 1.709 ms avg
Cache information for squid:
        Hits as % of all requests: 5min: 24.4%, 60min: 36.3%
        Hits as % of bytes sent: 5min: 12.3%, 60min: 17.8%
        Memory hits as % of hit requests: 5min: 3.9%, 60min: 7.2%
        Disk hits as % of hit requests: 5min: 41.2%, 60min: 40.6%
        Storage Swap size: 13868820 KB
        Storage Swap capacity: 16.9% used, 83.1% free
        Storage Mem size: 32744 KB
        Storage Mem capacity: 99.9% used,  0.1% free
        Mean Object Size: 16.74 KB
        Requests given to unlinkd: 210485
Median Service Times (seconds)  5 min    60 min:
        HTTP Requests (All):   0.37825  0.22004
        Cache Misses:          0.49576  0.49576
        Cache Hits:            0.00000  0.00000
        Near Hits:             0.19742  0.18699
        Not-Modified Replies:  0.00000  0.00000
        DNS Lookups:           0.04237  0.03868
        ICP Queries:           0.00000  0.00000
Resource usage for squid:
        UP Time: 503059.131 seconds
        CPU Time: 3957.634 seconds
        CPU Usage: 0.79%
        CPU Usage, 5 minute avg: 2.42%
        CPU Usage, 60 minute avg: 2.18%
        Process Data Segment Size via sbrk(): 70028 KB
        Maximum Resident Size: 0 KB
        Page faults with physical i/o: 9
Memory usage for squid via mallinfo():
        Total space in arena:   70296 KB
        Ordinary blocks:        68545 KB   1041 blks
        Small blocks:               0 KB      0 blks
        Holding blocks:         71596 KB    367 blks
        Free Small blocks:          0 KB
        Free Ordinary blocks:    1750 KB
        Total in use:          140141 KB 99%
        Total free:              1750 KB 1%
        Total size:            141892 KB
Memory accounted for:
        Total accounted:       118614 KB  84%
        memPool accounted:     118614 KB  84%
        memPool unaccounted:    23277 KB  16%
        memPoolAlloc calls: 798802270
        memPoolFree calls:  796049501
File descriptor usage for squid:
        Maximum number of file descriptors:   1024
        Largest file desc currently in use:    611
        Number of file desc currently in use:  561
        Files queued for open:                   0
        Available number of file descriptors:  463
        Reserved number of file descriptors:   100
        Store Disk files open:                   0
Internal Data Structures:
        828817 StoreEntries
          6768 StoreEntries with MemObjects
          6737 Hot Object Cache Items
        828531 on-disk objects


I see the DNS Statistics and I don't find anything...

Internal DNS Statistics:

The Queue:
                       DELAY SINCE
  ID   SIZE SENDS FIRST SEND LAST SEND
------ ---- ----- ---------- ---------

Nameservers:
IP ADDRESS      # QUERIES # REPLIES
--------------- --------- ---------
200.0.243.10        22696     22454
200.0.243.11          282       271

Rcode Matrix:
RCODE ATTEMPT1 ATTEMPT2 ATTEMPT3
    0   154413       31        1
    1        0        0        0
    2      314      266      261
    3      764       17        0
    4        0        0        0
    5        0        0        0

Search list:
invap.com.ar


Thanks a lot, Mariel


Leonardo Rodrigues wrote:

> > Mariel Sebedio escreveu:
>  
>> >> Hi, I have a RHEL 5.4 with squid3.0STABLE19 and have a performance
>> >> problems...
>> >>
>> >> My cache.log not report warning
>> >>
>> >> When I see in cachemgr.cgi I just have a 1024 File descriptors...
>> >>
>>    
> >
> >    if you're not getting the famous WARNING in your cache.log
> >
> > WARNING! Your cache is running out of filedescriptors
> >
> >    then you really dont need to worry about 1024 FDs. That's now too
> > much, but that's pretty enough for having a good number of simultaneos
> > clients.
> >
> >    Filedescriptors problems (running low on them) could give you some
> > problems, but in any case you would see the warning on your logs. If
> > you're not seeing it, then problem is not filedescriptor related. And
> > if that's not filedescriptor related, raising it wont change anything.
> >
> >    your performance problem is somewhere else .....
> >
> >
> >
> >
>  

-- Lic. Mariel Sebedio Division Computos y Sistemas Tel [phone]
int 2307

[Henrik Nordstrom-5]

ons 2009-10-21 klockan 09:46 -0200 skrev Mariel Sebedio:
> In this server I have a
>
> # cat /proc/sys/net/ipv4/netfilter/ip_conntrack_count
> 2086

and ip_conntrack_max?


Regards
Henrik

[George Herbert]

On the other hand - used as outbound caching proxies, for typical ISP
users, 1024 may be too small.  Former client of mine had it turned to
--with-maxfd=8192

Also note - when compiling on RHEL 5.x (and some other systems) you
need to have ulimit -n *of the configure and build environment* set to
at least the --with-maxfd value as well.

We used a wrapper on the configure which essentially did this:
--
export 'CFLAGS=-g -O -march=opteron -DNUMTHREADS=120
-DBUILDID=SQUID3.x-CUSTOMER-DATE'

echo "setting max open files hard/soft limits to 32k"
ulimit -HSn 32768
printenv
./configure (long list of configure options read from a separate file)
--

If you didn't do that, the actual maxfd limit it built with was less
than the --with-maxfd


-george

On Tue, Oct 20, 2009 at 1:04 PM, Leonardo Rodrigues
<leolistas@...> wrote:

> Mariel Sebedio escreveu:
>>
>> Hi, I have a RHEL 5.4 with squid3.0STABLE19 and have a performance
>> problems...
>>
>> My cache.log not report warning
>>
>> When I see in cachemgr.cgi I just have a 1024 File descriptors...
>>
>
>   if you're not getting the famous WARNING in your cache.log
>
> WARNING! Your cache is running out of filedescriptors
>
>   then you really dont need to worry about 1024 FDs. That's now too much,
> but that's pretty enough for having a good number of simultaneos clients.
>
>   Filedescriptors problems (running low on them) could give you some
> problems, but in any case you would see the warning on your logs. If you're
> not seeing it, then problem is not filedescriptor related. And if that's not
> filedescriptor related, raising it wont change anything.
>
>   your performance problem is somewhere else .....
>
>
>
>
> --
>
>
>        Atenciosamente / Sincerily,
>        Leonardo Rodrigues
>        Solutti Tecnologia
>        http://www.solutti.com.br
>
>        Minha armadilha de SPAM, NÃO mandem email
>        gertrudes@...
>        My SPAMTRAP, do not email it
>
>
>
>
>

--
-george william herbert
george.herbert@...

[Luis Daniel Lucio Quiroz-2]

Le mardi 20 octobre 2009 16:04:16, Leonardo Rodrigues a écrit :

> Mariel Sebedio escreveu:
> > Hi, I have a RHEL 5.4 with squid3.0STABLE19 and have a performance
> > problems...
> >
> > My cache.log not report warning
> >
> > When I see in cachemgr.cgi I just have a 1024 File descriptors...
>
>     if you're not getting the famous WARNING in your cache.log
>
> WARNING! Your cache is running out of filedescriptors
>
>     then you really dont need to worry about 1024 FDs. That's now too
> much, but that's pretty enough for having a good number of simultaneos
> clients.
>
>     Filedescriptors problems (running low on them) could give you some
> problems, but in any case you would see the warning on your logs. If
> you're not seeing it, then problem is not filedescriptor related. And if
> that's not filedescriptor related, raising it wont change anything.
>
>     your performance problem is somewhere else .....
>

I did fix that with this method:
/etc/security/limits.conf:
*               -       nofile          131072

and configure with --with-filedescriptors=8192

numbers are just a try, but you must set both of them higher than 1024.  After
that I get this error rid.

LD

[Mariel Sebedio]

Thanks I found it this solution and in this moment works very well for
increase the max-descriptors.

The original problem was in my firewall, in this moment I have the proxy
with Squid 3.0STABLE19 on RHEL5.4 works Ok.

Thanks for all.

Luis Daniel Lucio Quiroz wrote:

> Le mardi 20 octobre 2009 16:04:16, Leonardo Rodrigues a écrit :
>  
>> Mariel Sebedio escreveu:
>>    
>>> Hi, I have a RHEL 5.4 with squid3.0STABLE19 and have a performance
>>> problems...
>>>
>>> My cache.log not report warning
>>>
>>> When I see in cachemgr.cgi I just have a 1024 File descriptors...
>>>      
>>     if you're not getting the famous WARNING in your cache.log
>>
>> WARNING! Your cache is running out of filedescriptors
>>
>>     then you really dont need to worry about 1024 FDs. That's now too
>> much, but that's pretty enough for having a good number of simultaneos
>> clients.
>>
>>     Filedescriptors problems (running low on them) could give you some
>> problems, but in any case you would see the warning on your logs. If
>> you're not seeing it, then problem is not filedescriptor related. And if
>> that's not filedescriptor related, raising it wont change anything.
>>
>>     your performance problem is somewhere else .....
>>
>>    
>
> I did fix that with this method:
> /etc/security/limits.conf:
> *               -       nofile          131072
>
> and configure with --with-filedescriptors=8192
>
> numbers are just a try, but you must set both of them higher than 1024.  After
> that I get this error rid.
>
> LD
>
>
>  

--
Lic. Mariel Sebedio
Division Computos y Sistemas
Tel [phone] int 2307
INVAP S.E. - www.invap.com.ar