A little help in configuring a better solution for ZEO deployment

[Войнаровський Тарас]

Good day comunity.

My project is a ZOPE3/BlueBream project, that runs mainly of ZODB using ZEO. I use wsgi to get multiple instances(4) of my project and connect to ZEO Server from them. 

The database is around 3 GB in unpacked state and around 800 MB in packed.

I use 4 processes with 7 threads each for serving data

WSGIDaemonProcess doba.ua user=www group=www processes=4 threads=7 maximum-requests=2000000 

I have little space on /tmp partition, so the cache of ZEO is only 40 MB

    pool-size 30

  <zeoclient>

    server localhost:8200

    storage 1

    # ZEO client cache, in bytes

    cache-size 40MB

    # Uncomment to have a persistent disk cache

    #client zeo1

  </zeoclient>

The server has those specs:

8GB ram. 8 CPU 3.2 Ghz.  Raid-2 in a mirror mode.

The load of instances:

1 GB per instance - 8 GB total. Other apps consum up to 1.5 GB. So there is 2.5 GB in inactive mode all the time.

The problem, that I have:

The server has a Raid-2 in a mirror mode, so I only have 1 hard drive, which contain all my data. After a while of serving data the load gets highter and this hard drive just starts having 100% of disk usage. Then everything starts to fall. I have problems accessing the server and even restarting the project. 

I can't find any ther explanation, that ZEO does not handle the load well enough.

Does someone see any way to ease the load? 

_______________________________________________
For more information about ZODB, see http://zodb.org/

ZODB-Dev mailing list  -  ZODB-Dev@...
https://mail.zope.org/mailman/listinfo/zodb-dev

[Alan Runyan-3]

> I use 4 processes with 7 threads each for serving data

Any reason for 7 threads?  Try 1 thread?  Is it a classical web application? Or
you creating threads in your application?

> WSGIDaemonProcess doba.ua user=www group=www processes=4 threads=7
> maximum-requests=2000000
>
> I have little space on /tmp partition, so the cache of ZEO is only 40 MB

You can change the tmp location in Python by setting TMPDIR environment variable
for your process.

I would advise against persistent disk cache and increase the ZODB
cache-size. If you
are using latest ZODB you can use the cache-size-bytes which mostly
works.  You can
set it so zodb cache for each process will not exceed, say, 512MB.


> I can't find any ther explanation, that ZEO does not handle the load well
> enough.

If your ZEO server process is not bottlenecking -  it is your application.

> Does someone see any way to ease the load?

dont know your application but it is doubtful that ZODB is your bottleneck.

--
Alan Runyan

Skype/Twitter:: runyaga
Office:: 713.942.2377 ext 111
http://ploud.com/  Plone site in less than 10 seconds
_______________________________________________
For more information about ZODB, see http://zodb.org/

ZODB-Dev mailing list  -  ZODB-Dev@...
https://mail.zope.org/mailman/listinfo/zodb-dev