Nabble - PostgreSQL - performance

Slow select

2009-12-16T05:01:24Z

I have a table with column of character varying(100). There are about 150.000.000 rows in a table. Index was created as

CREATE INDEX idx_stringv
ON bn_stringvalue
USING btree
(lower(value::text));

I'm trying to execute queries like 'select * from stringvalue where value=lower(?)'. Making 1000 selects takes about 4-5 min. I did vacuum and analyze on this table and checked that query plan uses index. What can I do to make it faster?

Thanks in advance, Yulia

Re: Parallel Function calls using multiple processes

2009-12-16T00:38:22Z

2009/12/16 Vishal Gupta <vishal.g@...>:

> Dear Pavel,
>
> Thanks for quick response.
>
> Yes I am using explicit locks but only at row-level, not at table level,
> will this cause sequential execution?
> select ...on primary key..... for update;
> My objective is to execute this function in parallel for different PK
> entities, for same primary key value, sequential is fine.
>
> Also I have noted that though the function execution after the code calls is
> taking more than 2-3 minutes but ,
> when I have put in notice statements for start/end of function it has only
> taken a second in pg_log.
> Code is definitely not the problem here, as earlier function logic was
> getting executed as multiple queries
> from the code and were working fine for even higher load before I moved to
> function for the same. This problem started after moving to function only.
>
> I doubt that query is the reason for slowness, and it seems to be more
> related to database load at times, because it happens 3-4 times out of every
> 4000 transactions.
> pg_stat doesn't show any locks at the moment, but might not reflect the
> actual scenario as currently the transactions are working fine.

it should be a different problem - in bgwriter configuration. There
could help migration on 8.3, maybe.

http://old.nabble.com/Checkpoint-tuning-on-8.2.4-td17685494.html

Regards
Pavel Stehule

>
> Regards,
> Vishal Gupta - 9910991635
>
>
> ----- Original Message -----
> From: "Pavel Stehule" <pavel.stehule@...>
> To: "Vishal Gupta" <vishal.g@...>
> Cc: pgsql-performance@...
> Sent: Wednesday, December 16, 2009 1:04:48 PM GMT +05:30 Chennai, Kolkata,
> Mumbai, New Delhi
> Subject: Re: [PERFORM] Parallel Function calls using multiple processes
>
> 2009/12/16 Vishal Gupta <vishal.g@...>:
>> Problem: Function call typically takes 2-3 millisecond but at times
>> exceeding 2-3 Minutes.
>> =====================================================
>> From the DB logs it seems when multiple processes are trying to execute
>> the
>> function,
>> execution takes sequentially rather than parallel, which means Nth thread
>> will have to wait for (N-1)*ExecutionTime before getting its turn
>
> it's depend - if there are some locks then yes.
>
> but reason could be a slow query inside procedure too - look on
> pg_stat_activity table, if there are processes waiting for lock.
>
> see
> http://old.nabble.com/Query-is-slow-when-executing-in-procedure-td26490782.html
>
> Regards
> Pavel Stehule
>
>
>>
>> Is my observation correct? If yes then what is the solution for this? If
>> not
>> where/how to find the exact cause of the above problem?
>> =====================================================
>> DB Version: 8.2
>> Function Details:
>> --returns numeric, takes 10 parameters
>> --select query to validate data
>> --row level lock for select and validate
>> --bare minimum business logic
>> --update data
>> --couple of inserts for transaction logs/account management
>> --also note that few of the tables have audit triggers causing the row to
>> be
>> inserted in audit table with the action (only Update/Insert/Delete)
>>
>>
>

--
Sent via pgsql-performance mailing list (pgsql-performance@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance

Re: Parallel Function calls using multiple processes

2009-12-16T00:17:01Z

Dear Pavel,

Thanks for quick response.

Yes I am using explicit locks but only at row-level, not at table level, will this cause sequential execution?
select ...on primary key..... for update;
My objective is to execute this function in parallel for different PK entities, for same primary key value, sequential is fine.

Also I have noted that though the function execution after the code calls is taking more than 2-3 minutes but ,
when I have put in notice statements for start/end of function it has only taken a second in pg_log.
Code is definitely not the problem here, as earlier function logic was getting executed as multiple queries
from the code and were working fine for even higher load before I moved to function for the same. This problem started after moving to function only.

I doubt that query is the reason for slowness, and it seems to be more related to database load at times, because it happens 3-4 times out of every 4000 transactions.
pg_stat doesn't show any locks at the moment, but might not reflect the actual scenario as currently the transactions are working fine.

Regards,
Vishal Gupta - 9910991635

----- Original Message -----
From: "Pavel Stehule" <pavel.stehule@...>
To: "Vishal Gupta" <vishal.g@...>
Cc: pgsql-performance@...
Sent: Wednesday, December 16, 2009 1:04:48 PM GMT +05:30 Chennai, Kolkata, Mumbai, New Delhi
Subject: Re: [PERFORM] Parallel Function calls using multiple processes

2009/12/16 Vishal Gupta <vishal.g@...>:
> Problem: Function call typically takes 2-3 millisecond but at times
> exceeding 2-3 Minutes.
> =====================================================
> From the DB logs it seems when multiple processes are trying to execute the
> function,
> execution takes sequentially rather than parallel, which means Nth thread
> will have to wait for (N-1)*ExecutionTime before getting its turn

it's depend - if there are some locks then yes.

but reason could be a slow query inside procedure too - look on
pg_stat_activity table, if there are processes waiting for lock.

see http://old.nabble.com/Query-is-slow-when-executing-in-procedure-td26490782.html

Regards
Pavel Stehule

>
> Is my observation correct? If yes then what is the solution for this? If not
> where/how to find the exact cause of the above problem?
> =====================================================
> DB Version: 8.2
> Function Details:
> --returns numeric, takes 10 parameters
> --select query to validate data
> --row level lock for select and validate
> --bare minimum business logic
> --update data
> --couple of inserts for transaction logs/account management
> --also note that few of the tables have audit triggers causing the row to be
> inserted in audit table with the action (only Update/Insert/Delete)
>
>

Re: Parallel Function calls using multiple processes

2009-12-15T23:34:48Z

2009/12/16 Vishal Gupta <vishal.g@...>:
> Problem: Function call typically takes 2-3 millisecond but at times
> exceeding 2-3 Minutes.
> =====================================================
> From the DB logs it seems when multiple processes are trying to execute the
> function,
> execution takes sequentially rather than parallel, which means Nth thread
> will have to wait for (N-1)*ExecutionTime before getting its turn

it's depend - if there are some locks then yes.

but reason could be a slow query inside procedure too - look on
pg_stat_activity table, if there are processes waiting for lock.

see http://old.nabble.com/Query-is-slow-when-executing-in-procedure-td26490782.html

Regards
Pavel Stehule

--
Sent via pgsql-performance mailing list (pgsql-performance@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance

Parallel Function calls using multiple processes

2009-12-15T22:46:36Z

Problem: Function call typically takes 2-3 millisecond but at times exceeding 2-3 Minutes.
=====================================================
From the DB logs it seems when multiple processes are trying to execute the function,
execution takes sequentially rather than parallel, which means Nth thread will have to wait for (N-1)*ExecutionTime before getting its turn

Is my observation correct? If yes then what is the solution for this? If not where/how to find the exact cause of the above problem?
=====================================================
DB Version: 8.2
Function Details:
--returns numeric, takes 10 parameters
--select query to validate data
--row level lock for select and validate
--bare minimum business logic
--update data
--couple of inserts for transaction logs/account management
--also note that few of the tables have audit triggers causing the row to be inserted in audit table with the action (only Update/Insert/Delete)

Re: big select is resulting in a large amount of disk writing by kjournald

2009-12-15T17:28:21Z

On 12/10/09 8:41 AM, "Kevin Grittner" <Kevin.Grittner@...> wrote:

>
> There has been some talk about possibly writing tuples in a frozen
> state with the hint bits already set if they are loaded in the same
> database transaction which creates the table, but I'm not aware of
> anyone currently working on this.
>

Wow, that would be nice. That would cut in half the time it takes to
restore a several TB db (3 days to 1.5 here).

I assume this would help a lot of "CREATE TABLE AS SELECT ..." use cases
too. That is often the fastest way to do a large update on a table, but it
can still be annoyingly write intensive.

> -Kevin
>
> --
> Sent via pgsql-performance mailing list (pgsql-performance@...)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-performance
>

--
Sent via pgsql-performance mailing list (pgsql-performance@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance

Re: 8.4.1 ubuntu karmic slow createdb

2009-12-12T19:56:42Z

On Fri, Dec 11, 2009 at 5:12 PM, Scott Marlowe <scott.marlowe@...> wrote:

> On Fri, Dec 11, 2009 at 2:59 PM, Scott Mead
> <scott.lists@...> wrote:
>> On Fri, Dec 11, 2009 at 4:39 PM, Nikolas Everett <nik9000@...> wrote:
>>>
>>>
>>>
>>> Fair enough. I'm of the opinion that developers need to have their unit
>>> tests run fast. If they aren't fast then your just not going to test as
>>> much as you should. If your unit tests *have* to createdb then you have to
>>> do whatever you have to do to get it fast. It'd probably be better if unit
>>> tests don't create databases or alter tables at all though.
>>>
>>> Regardless of what is going on on your dev box you really should leave
>>> fsync on on your continuous integration, integration test, and QA machines.
>>> They're what your really modeling your production on anyway.
>>
>>
>> The other common issue is that developers running with something like
>> 'fsync=off' means that they have completely unrealistic expectations of the
>> performance surrounding something. If your developers see that when fsync
>> is on, createdb takes x seconds vs. when it's off, then they'll know that
>> basing their entire process on that probably isn't a good idea. When
>> developers think something is lightning, they tend to base lots of stuff on
>> it, whether it's production ready or not.
>
> Yeah, it's a huge mistake to give development super fast servers to
> test on. Keep in mind production may need to handle 10k requests a
> minute / second whatever. Developers cannot generate that kind of
> load by just pointing and clicking. Our main production is on a
> cluster of 8 and 12 core machines with scads of memory and RAID-10
> arrays all over the place. Development gets a 4 core machine with 8G
> ram and an 8 drive RAID-6. It ain't slow, but it ain't really that
> fast either.

My development box at work is an 1.8 Ghz Celeron with 256K of CPU
cache, 1 GB of memory, and a single IDE drive... I don't have too
many slow queries in there.

...Robert

--
Sent via pgsql-performance mailing list (pgsql-performance@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance

Re: 8.4.1 ubuntu karmic slow createdb

2009-12-12T12:38:41Z

On Saturday 12 December 2009 21:36:27 Michael Clemmons wrote:
> If ppl think its worth it I'll create a ticket
Thanks, no need. I will post a patch tomorrow or so.

Andres

--
Sent via pgsql-performance mailing list (pgsql-performance@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance

Re: 8.4.1 ubuntu karmic slow createdb

2009-12-12T12:36:27Z

If ppl think its worth it I'll create a ticket

On Sat, Dec 12, 2009 at 6:09 AM, Hannu Krosing <hannu@...> wrote:

On Sat, 2009-12-12 at 01:19 +0100, Andres Freund wrote:
> Hi,
>
> On Saturday 12 December 2009 00:59:13 Scott Marlowe wrote:
> > On Fri, Dec 11, 2009 at 3:52 PM, Michael Clemmons
> > > Createdb takes
> > > 12secs on my system(9.10 pg8.4 and ext4) which is impossibly slow for
> > > running 200unittests.
> > > Fsync got it to .2secs or so which is blazing but
> > > also the speed I expected being used to 8.3 and xfs. This dev box is my
> > > laptop and the data is litterally unimportant and doesn't exist longer
> > > than 20sec but Im all about good practices. Will definately try
> > > synchronous commit tonight once Im done working for the day. I've got
> > > some massive copying todo later though so this will probably help in the
> > > future as well.
> > Yeah, I'd probably resort to fsync off in that circumstance too
> > especially if syn commit off didn't help that much.
>
> How should syn commit help with creating databases?

It does not help here. Tested ;)

> The problem with 8.4 and creating databases is that the number of files
> increased hugely because of the introduction of relation forks.

Plus the fact that fsync on ext4 is really slow. some info here:
http://ldn.linuxfoundation.org/article/filesystems-data-preservation-fsync-and-benchmarks-pt-3

> It probably wouldnt be that hard to copy all files first, then reopen and fsync
> them. Actually that should be a patch doable in an hour or two.

Probably something worth doing, as it will speed this up on all
filesystems, and doubly so on ext4 and xfs.

--
Hannu Krosing http://www.2ndQuadrant.com
PostgreSQL Scalability and Availability
Services, Consulting and Training

Re: 8.4.1 ubuntu karmic slow createdb

2009-12-11T16:19:38Z

Hi,

On Saturday 12 December 2009 00:59:13 Scott Marlowe wrote:

> On Fri, Dec 11, 2009 at 3:52 PM, Michael Clemmons
> > Createdb takes
> > 12secs on my system(9.10 pg8.4 and ext4) which is impossibly slow for
> > running 200unittests.
> > Fsync got it to .2secs or so which is blazing but
> > also the speed I expected being used to 8.3 and xfs. This dev box is my
> > laptop and the data is litterally unimportant and doesn't exist longer
> > than 20sec but Im all about good practices. Will definately try
> > synchronous commit tonight once Im done working for the day. I've got
> > some massive copying todo later though so this will probably help in the
> > future as well.
> Yeah, I'd probably resort to fsync off in that circumstance too
> especially if syn commit off didn't help that much.

How should syn commit help with creating databases?

The problem with 8.4 and creating databases is that the number of files
increased hugely because of the introduction of relation forks.
It probably wouldnt be that hard to copy all files first, then reopen and fsync
them. Actually that should be a patch doable in an hour or two.

Andres

--
Sent via pgsql-performance mailing list (pgsql-performance@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance

Re: 8.4.1 ubuntu karmic slow createdb

2009-12-11T15:59:13Z

On Fri, Dec 11, 2009 at 3:52 PM, Michael Clemmons
<glassresistor@...> wrote:
> Thanks all this has been a good help.
> I don't have control(or easy control) over unit tests creating/deleting
> databases since Im using the django framework for this job.

Reminds of the issues we had with Ruby on Rails and it's (at the time)
very mysql-centric tools that made us take a fork to large portions of
its brain to get things like this working. Worked with a developer
for a day or two fixing most of the worst mysqlisms in RoR at the time
to just get this kind of stuff working.

> Createdb takes
> 12secs on my system(9.10 pg8.4 and ext4) which is impossibly slow for
> running 200unittests.

Wait, so each unit test createdbs by itself? Wow...

> Fsync got it to .2secs or so which is blazing but
> also the speed I expected being used to 8.3 and xfs. This dev box is my
> laptop and the data is litterally unimportant and doesn't exist longer than
> 20sec but Im all about good practices. Will definately try synchronous
> commit tonight once Im done working for the day. I've got some massive
> copying todo later though so this will probably help in the future as well.

Yeah, I'd probably resort to fsync off in that circumstance too
especially if syn commit off didn't help that much.

--
Sent via pgsql-performance mailing list (pgsql-performance@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance

Re: 8.4.1 ubuntu karmic slow createdb

2009-12-11T14:52:01Z

Thanks all this has been a good help.
I don't have control(or easy control) over unit tests creating/deleting databases since Im using the django framework for this job. Createdb takes 12secs on my system(9.10 pg8.4 and ext4) which is impossibly slow for running 200unittests. Fsync got it to .2secs or so which is blazing but also the speed I expected being used to 8.3 and xfs. This dev box is my laptop and the data is litterally unimportant and doesn't exist longer than 20sec but Im all about good practices. Will definately try synchronous commit tonight once Im done working for the day. I've got some massive copying todo later though so this will probably help in the future as well.

On Fri, Dec 11, 2009 at 5:39 PM, Greg Smith <greg@...> wrote:

Scott Mead wrote:

The other common issue is that developers running with something like 'fsync=off' means that they have completely unrealistic expectations of the performance surrounding something.

Right, but the flip side here is that often the production server will have hardware such as a caching RAID card that vastly improves performance in this area. There's some room to cheat in order to accelerate the dev systems lack of such things, while still not giving a completely unrealistic view of performance.

As far as I'm concerned, using "fsync=off" is almost never excusable if you're running 8.3 or later where "synchronous_commit=off" is a possibility. If you use that, it will usually improve the worst part of commit issues substantially. And it happens in a way that's actually quite similar to how a caching write production server will run: small writes happen instantly, but eventually bigger ones will end up bottlenecked at the disks anyway.

It would improve the average safety of our community members if anytime someone suggests "fsync=off", we strongly suggest "synchronous_commit=off" and potentially tuning its interval instead as a middle ground, while still helping people who need to speed their systems up. Saying "never turn fsync off" without suggesting this alternative is counter-productive. If you're in the sort of position where fsync is killing your performance you'll do anything to speed things up (I've seen a 100:1 speed improvement) no matter how risky. I've ran a production system of 8.2 with fsync off, a TB of data, and no safety net if a crash introduced corruption beyond a ZFS snapshot. It wasn't fun, but it was the only possibility to get bulk loading (there was an ETL step in the middle after COPY) to happen fast enough. Using async commit instead is a much better approach now that it's available.

--
Greg Smith 2ndQuadrant Baltimore, MD
PostgreSQL Training, Services and Support
greg@... www.2ndQuadrant.com

Re: 8.4.1 ubuntu karmic slow createdb

2009-12-11T14:39:54Z

Scott Mead wrote:
> The other common issue is that developers running with something like
> 'fsync=off' means that they have completely unrealistic expectations
> of the performance surrounding something.
Right, but the flip side here is that often the production server will
have hardware such as a caching RAID card that vastly improves
performance in this area. There's some room to cheat in order to
accelerate the dev systems lack of such things, while still not giving a
completely unrealistic view of performance.

As far as I'm concerned, using "fsync=off" is almost never excusable if
you're running 8.3 or later where "synchronous_commit=off" is a
possibility. If you use that, it will usually improve the worst part of
commit issues substantially. And it happens in a way that's actually
quite similar to how a caching write production server will run: small
writes happen instantly, but eventually bigger ones will end up
bottlenecked at the disks anyway.

It would improve the average safety of our community members if anytime
someone suggests "fsync=off", we strongly suggest
"synchronous_commit=off" and potentially tuning its interval instead as
a middle ground, while still helping people who need to speed their
systems up. Saying "never turn fsync off" without suggesting this
alternative is counter-productive. If you're in the sort of position
where fsync is killing your performance you'll do anything to speed
things up (I've seen a 100:1 speed improvement) no matter how risky.
I've ran a production system of 8.2 with fsync off, a TB of data, and no
safety net if a crash introduced corruption beyond a ZFS snapshot. It
wasn't fun, but it was the only possibility to get bulk loading (there
was an ETL step in the middle after COPY) to happen fast enough. Using
async commit instead is a much better approach now that it's available.

--
Greg Smith 2ndQuadrant Baltimore, MD
PostgreSQL Training, Services and Support
greg@... www.2ndQuadrant.com

--
Sent via pgsql-performance mailing list (pgsql-performance@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance

Re: 8.4.1 ubuntu karmic slow createdb

2009-12-11T14:19:05Z

On Fri, Dec 11, 2009 at 3:12 PM, Scott Carey <scott@...> wrote:

>
> On 12/11/09 1:57 PM, "Scott Marlowe" <scott.marlowe@...> wrote:
>
>>
>> This is my big issue. dropping / creating databases for unit tests is
>> overkill. Running any DDL at all for a unit test seems wrong to me
>> too. Insert a row if you need it, MAYBE. Unit tests should work with
>> a test database that HAS the structure and database already in place.
>>
>> What happens if your unit tests get lose in production and drop a
>> database, or a table. Not good.
>>
>
> Production should not have a db with the same username/pw combination as dev
> boxes and unit tests . . .
>
> Unfortunately, unit-like (often more than a 'unit') tests can't always rely
> on a test db being already set up. If one leaves any cruft around, it might
> break later tests later on non-deterministically. Automated tests that
> insert data are absolutely required somewhere if the application inserts
> data.
>
> The best way to do this in postgres is to create a template database from
> scratch with whatever DDL is needed at the start of the run, and then create
> and drop db's as copies of that template per test or test suite.

Debateable. Last job we had 44k or so unit tests, and we gave each
dev their own db made from the main qa / unit testing db that they
could refresh at any time, and run the unit tests locally before
committing code. Actual failures like the one you mention were very
rare because of this approach. A simple ant refresh-db and they were
ready to test their code before committing it to the continuous
testing farm.

--
Sent via pgsql-performance mailing list (pgsql-performance@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance

Re: 8.4.1 ubuntu karmic slow createdb

2009-12-11T14:12:47Z

On Fri, Dec 11, 2009 at 2:59 PM, Scott Mead
<scott.lists@...> wrote:

> On Fri, Dec 11, 2009 at 4:39 PM, Nikolas Everett <nik9000@...> wrote:
>>
>>
>>
>> Fair enough. I'm of the opinion that developers need to have their unit
>> tests run fast. If they aren't fast then your just not going to test as
>> much as you should. If your unit tests *have* to createdb then you have to
>> do whatever you have to do to get it fast. It'd probably be better if unit
>> tests don't create databases or alter tables at all though.
>>
>> Regardless of what is going on on your dev box you really should leave
>> fsync on on your continuous integration, integration test, and QA machines.
>> They're what your really modeling your production on anyway.
>
>
> The other common issue is that developers running with something like
> 'fsync=off' means that they have completely unrealistic expectations of the
> performance surrounding something. If your developers see that when fsync
> is on, createdb takes x seconds vs. when it's off, then they'll know that
> basing their entire process on that probably isn't a good idea. When
> developers think something is lightning, they tend to base lots of stuff on
> it, whether it's production ready or not.

Yeah, it's a huge mistake to give development super fast servers to
test on. Keep in mind production may need to handle 10k requests a
minute / second whatever. Developers cannot generate that kind of
load by just pointing and clicking. Our main production is on a
cluster of 8 and 12 core machines with scads of memory and RAID-10
arrays all over the place. Development gets a 4 core machine with 8G
ram and an 8 drive RAID-6. It ain't slow, but it ain't really that
fast either.

--
Sent via pgsql-performance mailing list (pgsql-performance@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance

Re: 8.4.1 ubuntu karmic slow createdb

2009-12-11T14:12:45Z

On 12/11/09 1:57 PM, "Scott Marlowe" <scott.marlowe@...> wrote:

>
> This is my big issue. dropping / creating databases for unit tests is
> overkill. Running any DDL at all for a unit test seems wrong to me
> too. Insert a row if you need it, MAYBE. Unit tests should work with
> a test database that HAS the structure and database already in place.
>
> What happens if your unit tests get lose in production and drop a
> database, or a table. Not good.
>

Production should not have a db with the same username/pw combination as dev
boxes and unit tests . . .

Unfortunately, unit-like (often more than a 'unit') tests can't always rely
on a test db being already set up. If one leaves any cruft around, it might
break later tests later on non-deterministically. Automated tests that
insert data are absolutely required somewhere if the application inserts
data.

The best way to do this in postgres is to create a template database from
scratch with whatever DDL is needed at the start of the run, and then create
and drop db's as copies of that template per test or test suite.

So no, its not overkill at all IMO. I do wish to avoid it, and ideally all
tests clean up after themselves, but in practice this does not happen and
results in hard to track down issues where test X fails because of something
that any one of tests A to W did (all of which pass), often wasting time of
the most valuable developers -- those who know the majority of the system
well enough to track down such issues across the whole system.

One thing to consider, is putting this temp database in a RAMFS, or ramdisk
since postgres does a lot of file creates and fsyncs when cloning a db from
a template. For almost all such test db's the actual data is small, but the
# of tables is large.

> --
> Sent via pgsql-performance mailing list (pgsql-performance@...)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-performance
>

--
Sent via pgsql-performance mailing list (pgsql-performance@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance

Re: 8.4.1 ubuntu karmic slow createdb

2009-12-11T13:59:43Z

On Fri, Dec 11, 2009 at 4:39 PM, Nikolas Everett <nik9000@...> wrote:

Fair enough. I'm of the opinion that developers need to have their unit tests run fast. If they aren't fast then your just not going to test as much as you should. If your unit tests *have* to createdb then you have to do whatever you have to do to get it fast. It'd probably be better if unit tests don't create databases or alter tables at all though.

Regardless of what is going on on your dev box you really should leave fsync on on your continuous integration, integration test, and QA machines. They're what your really modeling your production on anyway.

The other common issue is that developers running with something like 'fsync=off' means that they have completely unrealistic expectations of the performance surrounding something. If your developers see that when fsync is on, createdb takes x seconds vs. when it's off, then they'll know that basing their entire process on that probably isn't a good idea. When developers think something is lightning, they tend to base lots of stuff on it, whether it's production ready or not.

--Scott

Re: 8.4.1 ubuntu karmic slow createdb

2009-12-11T13:57:56Z

On Fri, Dec 11, 2009 at 2:39 PM, Nikolas Everett <nik9000@...> wrote:

>
>
> On Fri, Dec 11, 2009 at 3:50 PM, Joshua D. Drake <jd@...>
> wrote:
>>
>> On Fri, 2009-12-11 at 15:43 -0500, Nikolas Everett wrote:
>> > Turning fsync off on a dev database is a bad idea? Sure you might
>> > kill it and have to start over, but thats kind of the point in a dev
>> > database.
>>
>> My experience is that bad dev practices turn into bad production
>> practices, whether intentionally or not.
>
> Fair enough. I'm of the opinion that developers need to have their unit
> tests run fast. If they aren't fast then your just not going to test as
> much as you should. If your unit tests *have* to createdb then you have to
> do whatever you have to do to get it fast. It'd probably be better if unit
> tests don't create databases or alter tables at all though.

This is my big issue. dropping / creating databases for unit tests is
overkill. Running any DDL at all for a unit test seems wrong to me
too. Insert a row if you need it, MAYBE. Unit tests should work with
a test database that HAS the structure and database already in place.

What happens if your unit tests get lose in production and drop a
database, or a table. Not good.

--
Sent via pgsql-performance mailing list (pgsql-performance@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance

Re: 8.4.1 ubuntu karmic slow createdb

2009-12-11T13:39:34Z

On Fri, Dec 11, 2009 at 3:50 PM, Joshua D. Drake <jd@...> wrote:

On Fri, 2009-12-11 at 15:43 -0500, Nikolas Everett wrote:
> Turning fsync off on a dev database is a bad idea? Sure you might
> kill it and have to start over, but thats kind of the point in a dev
> database.

My experience is that bad dev practices turn into bad production
practices, whether intentionally or not.

Fair enough. I'm of the opinion that developers need to have their unit tests run fast. If they aren't fast then your just not going to test as much as you should. If your unit tests *have* to createdb then you have to do whatever you have to do to get it fast. It'd probably be better if unit tests don't create databases or alter tables at all though.

Regardless of what is going on on your dev box you really should leave fsync on on your continuous integration, integration test, and QA machines. They're what your really modeling your production on anyway.

Re: 8.4.1 ubuntu karmic slow createdb

2009-12-11T12:50:10Z

On Fri, 2009-12-11 at 15:43 -0500, Nikolas Everett wrote:
> Turning fsync off on a dev database is a bad idea? Sure you might
> kill it and have to start over, but thats kind of the point in a dev
> database.

My experience is that bad dev practices turn into bad production
practices, whether intentionally or not.

Joshua D. Drake

--
PostgreSQL.org Major Contributor
Command Prompt, Inc: http://www.commandprompt.com/ - 503.667.4564
Consulting, Training, Support, Custom Development, Engineering
If the world pushes look it in the eye and GRR. Then push back harder. - Salamander

--
Sent via pgsql-performance mailing list (pgsql-performance@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance

Re: 8.4.1 ubuntu karmic slow createdb

2009-12-11T12:43:59Z

Turning fsync off on a dev database is a bad idea? Sure you might kill it and have to start over, but thats kind of the point in a dev database.

On Fri, Dec 11, 2009 at 12:58 PM, Joshua D. Drake <jd@...> wrote:

On Thu, 2009-12-10 at 20:38 -0500, Nikolas Everett wrote:
> In my limited experience ext4 as presented by Karmic is not db
> friendly. I had to carve my swap partition into a swap partition and
> an xfs partition to get better db performance. Try fsync=off first,
> but if that doesn't work then try a mini xfs.

Do not turn fsync off. That is bad advice. I would not suggest ext4 at
this point for database operations. Use ext3. It is backward compatible.

Joshua D. Drake

--
PostgreSQL.org Major Contributor
Command Prompt, Inc: http://www.commandprompt.com/ - 503.667.4564
Consulting, Training, Support, Custom Development, Engineering
If the world pushes look it in the eye and GRR. Then push back harder. - Salamander

Re: 8.4.1 ubuntu karmic slow createdb

2009-12-11T09:58:39Z

On Thu, 2009-12-10 at 20:38 -0500, Nikolas Everett wrote:
> In my limited experience ext4 as presented by Karmic is not db
> friendly. I had to carve my swap partition into a swap partition and
> an xfs partition to get better db performance. Try fsync=off first,
> but if that doesn't work then try a mini xfs.

Do not turn fsync off. That is bad advice. I would not suggest ext4 at
this point for database operations. Use ext3. It is backward compatible.

Joshua D. Drake

--
PostgreSQL.org Major Contributor
Command Prompt, Inc: http://www.commandprompt.com/ - 503.667.4564
Consulting, Training, Support, Custom Development, Engineering
If the world pushes look it in the eye and GRR. Then push back harder. - Salamander

--
Sent via pgsql-performance mailing list (pgsql-performance@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance

Re: Load experimentation

2009-12-10T18:37:16Z

On 12/10/09 3:29 PM, "Scott Carey" <scott@...> wrote:

> On 12/7/09 11:12 AM, "Ben Brehmer" <benbrehmer@...> wrote:
>
>> Thanks for the quick responses. I will respond to all questions in one email:
>>
>> COPY command: Unfortunately I'm stuck with INSERTS due to the nature this
>> data
>> was generated (Hadoop/MapReduce).
>
> If you have control over the MapReduce output, you can have that output
> result files in a format that COPY likes.
>
> If you don't have any control over that its more complicated. I use a final
> pass Hadoop Map only job to go over the output and insert into postgres
> directly from the job, using the :
>
> INSERT INTO <table> VALUES (val1, val2, ... ), (val1, val2, ...), ...
> Insert style from Java with about 80 rows per insert statement and a single
> transaction for about a thousand of these. This was faster than batch
> inserts .
>

I should mention that the above is a bit off. There is an important caveat
that each of these individual tasks might run twice in Hadoop (only one will
finish -- speculative execution and retry on error). To deal with this you
can run each job inside a single transaction so that a failure will
rollback, and likely want to turn off speculative execution.

Another option is to run only one map job, with no reduce for this sort of
work in order to ensure duplicate data is not inserted. We are inserting
into a temp table named uniquely per chunk first (sometimes in parallel).
Then while holding a posstgres advisory lock we do a SELECT * FROM <temp>
INTO <destination> type operation, which is fast.

>
>>
>> On 07/12/2009 10:39 AM, Thom Brown wrote:
>>>
>>> 2009/12/7 Kevin Grittner <Kevin.Grittner@...>
>>>
>>>>
>>>> Ben Brehmer <benbrehmer@...> wrote:
>>>>
>>>>> -7.5 GB memory
>>>>> -4 EC2 Compute Units (2 virtual cores with 2 EC2 Compute Units
>>>>> each)
>>>>> -64-bit platform
>>>>
>>>>
>>>> What OS?
>>>>
>>>>> (PostgreSQL 8.1.3)
>>>>
>>>> Why use such an antiquated, buggy version? Newer versions are
>>>> faster.
>>>>
>>>> -Kevin
>>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> I'd agree with trying to use the latest version you can.
>>>
>>>
>>>
>>>
>>> How are you loading this data? I'd make sure you haven't got any indices,
>>> primary keys, triggers or constraints on your tables before you begin the
>>> initial load, just add them after. Also use either the COPY command for
>>> loading, or prepared transactions. Individual insert commands will just
>>> take
>>> way too long.
>>>
>>>
>>>
>>>
>>> Regards
>>>
>>>
>>>
>>>
>>> Thom
>>
>
>
> --
> Sent via pgsql-performance mailing list (pgsql-performance@...)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-performance
>

--
Sent via pgsql-performance mailing list (pgsql-performance@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance

Re: 8.4.1 ubuntu karmic slow createdb

2009-12-10T17:38:25Z

In my limited experience ext4 as presented by Karmic is not db friendly. I had to carve my swap partition into a swap partition and an xfs partition to get better db performance. Try fsync=off first, but if that doesn't work then try a mini xfs.

On Thu, Dec 10, 2009 at 5:09 PM, Andres Freund <andres@...> wrote:

Hi,

On Thursday 10 December 2009 23:01:08 Michael Clemmons wrote:
> Im not sure what that means ppl in my office with slower hd speeds using
> 8.4 can create a db in 2s vs my 8-12s.

- Possibly their config is different - they could have disabled the "fsync"
parameter which turns the database to be not crashsafe anymore but much faster
in some circumstances.

- Possibly you have much data in your template1 database?
You could check whether

CREATE DATABASE speedtest TEMPLATE template1; takes more time than
CREATE DATABASE speedtest TEMPLATE template0;.

You should issue both multiple times to ensure caching on the template
database doesnt play a role.

> Could using md5 instead of ident do it?

Seems unlikely.
Is starting psql near-instantaneus? Are you using "createdb" or are you
issuing "CREATE DATABASE ..."?

Andres

--
Sent via pgsql-performance mailing list (pgsql-performance@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance

Re: Load experimentation

2009-12-10T15:29:59Z

On 12/7/09 11:12 AM, "Ben Brehmer" <benbrehmer@...> wrote:

> Thanks for the quick responses. I will respond to all questions in one email:
>
> COPY command: Unfortunately I'm stuck with INSERTS due to the nature this data
> was generated (Hadoop/MapReduce).

If you have control over the MapReduce output, you can have that output
result files in a format that COPY likes.

If you don't have any control over that its more complicated. I use a final
pass Hadoop Map only job to go over the output and insert into postgres
directly from the job, using the :

INSERT INTO <table> VALUES (val1, val2, ... ), (val1, val2, ...), ...
Insert style from Java with about 80 rows per insert statement and a single
transaction for about a thousand of these. This was faster than batch
inserts .

>
> On 07/12/2009 10:39 AM, Thom Brown wrote:
>>
>> 2009/12/7 Kevin Grittner <Kevin.Grittner@...>
>>
>>>
>>> Ben Brehmer <benbrehmer@...> wrote:
>>>
>>>> -7.5 GB memory
>>>> -4 EC2 Compute Units (2 virtual cores with 2 EC2 Compute Units
>>>> each)
>>>> -64-bit platform
>>>
>>>
>>> What OS?
>>>
>>>> (PostgreSQL 8.1.3)
>>>
>>> Why use such an antiquated, buggy version? Newer versions are
>>> faster.
>>>
>>> -Kevin
>>>
>>
>>
>>
>>
>>
>>
>> I'd agree with trying to use the latest version you can.
>>
>>
>>
>>
>> How are you loading this data? I'd make sure you haven't got any indices,
>> primary keys, triggers or constraints on your tables before you begin the
>> initial load, just add them after. Also use either the COPY command for
>> loading, or prepared transactions. Individual insert commands will just take
>> way too long.
>>
>>
>>
>>
>> Regards
>>
>>
>>
>>
>> Thom
>

--
Sent via pgsql-performance mailing list (pgsql-performance@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance

Re: 8.4.1 ubuntu karmic slow createdb

2009-12-10T14:09:03Z

Hi,

On Thursday 10 December 2009 23:01:08 Michael Clemmons wrote:
> Im not sure what that means ppl in my office with slower hd speeds using
> 8.4 can create a db in 2s vs my 8-12s.
- Possibly their config is different - they could have disabled the "fsync"
parameter which turns the database to be not crashsafe anymore but much faster
in some circumstances.

- Possibly you have much data in your template1 database?
You could check whether

CREATE DATABASE speedtest TEMPLATE template1; takes more time than
CREATE DATABASE speedtest TEMPLATE template0;.

You should issue both multiple times to ensure caching on the template
database doesnt play a role.

> Could using md5 instead of ident do it?
Seems unlikely.
Is starting psql near-instantaneus? Are you using "createdb" or are you
issuing "CREATE DATABASE ..."?

Andres

--
Sent via pgsql-performance mailing list (pgsql-performance@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance

Re: 8.4.1 ubuntu karmic slow createdb

2009-12-10T14:01:08Z

Im not sure what that means ppl in my office with slower hd speeds using 8.4 can create a db in 2s vs my 8-12s. Could using md5 instead of ident do it?

On Thu, Dec 10, 2009 at 4:56 PM, Andres Freund <andres@...> wrote:

On Thursday 10 December 2009 21:41:08 Michael Clemmons wrote:
> Hey,
> I've got a computer which runs but 8.3 and 8.4. To create a db it takes 4s
> for 8.3 and 9s for 8.4. I have many unit tests which create databases all
> of the time and now run much slower than 8.3 but it seems to be much longer
> as I remember at one point creating databases I considered an instantaneous
> thing. Does any on the list know why this is true and if I can get it back
> to normal.

Possibly you had fsync=off at the time?

Andres

Re: 8.4.1 ubuntu karmic slow createdb

2009-12-10T13:56:59Z

On Thursday 10 December 2009 21:41:08 Michael Clemmons wrote:
> Hey,
> I've got a computer which runs but 8.3 and 8.4. To create a db it takes 4s
> for 8.3 and 9s for 8.4. I have many unit tests which create databases all
> of the time and now run much slower than 8.3 but it seems to be much longer
> as I remember at one point creating databases I considered an instantaneous
> thing. Does any on the list know why this is true and if I can get it back
> to normal.
Possibly you had fsync=off at the time?

Andres

--
Sent via pgsql-performance mailing list (pgsql-performance@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance

8.4.1 ubuntu karmic slow createdb

2009-12-10T12:41:08Z

Hey,
I've got a computer which runs but 8.3 and 8.4. To create a db it takes 4s for 8.3 and 9s for 8.4. I have many unit tests which create databases all of the time and now run much slower than 8.3 but it seems to be much longer as I remember at one point creating databases I considered an instantaneous thing. Does any on the list know why this is true and if I can get it back to normal.
-Michael

Re: Load experimentation

2009-12-10T12:24:06Z

Hi Andy,

Load is chugging along. We've optimized our postgres conf as much as
possible but are seeing the inevitable I/O bottleneck. I had the same
thought as you (converting inserts into copy's) a while back but
unfortunately each file has many inserts into many different tables.
Potentially I could rip through this with a little MapReduce job on
50-100 nodes, which is still something I might do.

One thought we are playing with was taking advantage of 4 x 414GB EBS
devices in a RAID0 configuration. This would spread disk writes across 4
block devices.

Right now I'm wrapping about 1500 inserts in a transaction block. Since
its an I/O bottlenecks, COPY statements might not give me much advantage.

Its definitely a work in progress :)

Ben

On 09/12/2009 5:31 AM, Andy Colson wrote:

> On 12/07/2009 12:12 PM, Ben Brehmer wrote:
>> Hello All,
>>
>> I'm in the process of loading a massive amount of data (500 GB). After
>> some initial timings, I'm looking at 260 hours to load the entire 500GB.
>> 10 days seems like an awfully long time so I'm searching for ways to
>> speed this up. The load is happening in the Amazon cloud (EC2), on a
>> m1.large instance:
>> -7.5 GB memory
>> -4 EC2 Compute Units (2 virtual cores with 2 EC2 Compute Units each)
>> -64-bit platform
>>
>>
>> So far I have modified my postgresql.conf file (PostgreSQL 8.1.3). The
>> modifications I have made are as follows:
>>
>> shared_buffers = 786432
>> work_mem = 10240
>> maintenance_work_mem = 6291456
>> max_fsm_pages = 3000000
>> wal_buffers = 2048
>> checkpoint_segments = 200
>> checkpoint_timeout = 300
>> checkpoint_warning = 30
>> autovacuum = off
>>
>>
>> There are a variety of instance types available in the Amazon cloud
>> (http://aws.amazon.com/ec2/instance-types/), including high memory and
>> high CPU. High memory instance types come with 34GB or 68GB of memory.
>> High CPU instance types have a lot less memory (7GB max) but up to 8
>> virtual cores. I am more than willing to change to any of the other
>> instance types.
>>
>> Also, there is nothing else happening on the loading server. It is
>> completely dedicated to the load.
>>
>> Any advice would be greatly appreciated.
>>
>> Thanks,
>>
>> Ben
>>
>
> I'm kind of curious, how goes the load? Is it done yet? Still
> looking at days'n'days to finish?
>
> I was thinking... If the .sql files are really nicely formatted, it
> would not be too hard to whip up a perl script to run as a filter to
> change the statements into copy's.
>
> Each file would have to only fill one table, and only contain inserts,
> and all the insert statements would have to set the same fields. (And
> I'm sure there could be other problems).
>
> Also, just for the load, did you disable fsync?
>
> -Andy
>

--
Sent via pgsql-performance mailing list (pgsql-performance@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance

Re: Fw: Help me put 2 Gigs of RAM to use

2009-12-10T09:19:44Z

On Thu, Dec 10, 2009 at 11:45 AM, Greg Smith <greg@...> wrote:
> So your shared buffers setting is 8192 * 8K = 64MB
> effective_cache_size is 8MB
> work_mem is 4MB.
>
> The first and last of those are reasonable but on the small side, the last
> is...not.

I believe that the second instance of the word "last" in that sentence
should have been "middle", referring to effective_cache_size. Small
values discourage the planner from using indices in certain
situations.

...Robert

--
Sent via pgsql-performance mailing list (pgsql-performance@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance

Re: Fw: Help me put 2 Gigs of RAM to use

2009-12-10T08:45:30Z

Mark Stosberg wrote:

> I find the file a bit hard to read because of the lack of units in
> the examples, but perhaps that's already been addressed in future
> versions.
>
> max_connections = 400 # Seems to be enough us
> shared_buffers = 8192
> effective_cache_size = 1000
> work_mem = 4096
> maintenance_work_mem = 160MB
>

It's already addressed in 8.2, as you can note by the fact that
"maintenance_work_mem" is in there with an easy to read format.
Guessing that someone either pulled in settings from an older version,
or used some outdated web guide to get starter settings.

To convert the rest of them, you need to know what the units for each
parameter is. You can find that out like this:

gsmith=# select name,setting,unit from pg_settings where name in
('shared_buffers','effective_cache_size','work_mem');

name | setting | unit
----------------------+---------+------
effective_cache_size | 16384 | 8kB
shared_buffers | 4096 | 8kB
work_mem | 1024 | kB

So your shared buffers setting is 8192 * 8K = 64MB
effective_cache_size is 8MB
work_mem is 4MB.

The first and last of those are reasonable but on the small side, the
last is...not. Increasing it won't actually use more memory on your
server though, it will just change query plans--so you want to be
careful about increasing it too much in one shot.

The next set of stuff you need to know about general guidelines for
server sizing is at
http://wiki.postgresql.org/wiki/Tuning_Your_PostgreSQL_Server

You'd probably want to put shared_buffers at a higher level based on the
amount of RAM on your server, but I'd suggest you tune the checkpoint
parameters along with that--just increasing the buffer space along can
cause problems rather than solve them if you're having checkpoints all
the time.

--
Greg Smith 2ndQuadrant Baltimore, MD
PostgreSQL Training, Services and Support
greg@... www.2ndQuadrant.com

--
Sent via pgsql-performance mailing list (pgsql-performance@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance

Re: Help me put 2 Gigs of RAM to use

2009-12-10T08:44:32Z

Thanks for the response, Matthew.

> On Thu, 10 Dec 2009, Mark Stosberg wrote:
> > What I'm noticing is that the while the FreeBSD server has 4 Gigs of
> > memory, there are rarely every more than 2 in use-- the memory use
> > graphs as being rather constant. My goal is to make good use of those 2
> > Gigs of memory to improve performance and reduce the CPU usage.
>
> I think you'll find that the RAM is already being used quite effectively
> as disc cache by the OS. It sounds like the server is actually set up
> pretty well. You may get slightly better performance by tweaking a thing
> here or there, but the server needs some OS disc cache to perform well.

As part of reviewing this status, I it appears that the OS is only
addresses 3 of the 4 Gigs of memory. We'll work on our FreeBSD setup to
cure that.

Here's how "top" reports the memory breakdown:

Mem: 513M Active, 2246M Inact, 249M Wired, 163M Cache, 112M Buf, 7176K
Free Swap: 9216M Total, 1052K Used, 9215M Free

So perhaps the OS disc cache is represented in the "Inactive" memory
statistic? I suppose once we have the 4th Gig of memory actually
available, that would all be doing to the disk cache.

> > (We currently run 8.2, but are planning an upgrade to 8.4 "soon").
>
> Highly recommended.

For performance improvements in particular?

Mark

--
. . . . . . . . . . . . . . . . . . . . . . . . . . .
Mark Stosberg Principal Developer
mark@... Summersault, LLC
765-939-9301 ext 202 database driven websites
. . . . . http://www.summersault.com/ . . . . . . . .

--
Sent via pgsql-performance mailing list (pgsql-performance@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance

Re: big select is resulting in a large amount of disk writing by kjournald

2009-12-10T08:41:53Z

Joseph S <jks@...> wrote:

> I just installed a shiny new database server with pg 8.4.1 running
> on CentOS 5.4. After using slony to replicate over my database I
> decided to do some basic performance tests to see how spiffy my
> shiny new server is. This machine has 32G ram, over 31 of which
> is used for the system file cache.
>
> So I run "select count(*) from large_table" and I see in xosview a
> solid block of write activity. Runtime is 28125.644 ms for the
> first run. The second run does not show a block of write activity
> and takes 3327.441 ms

As others have mentioned, this is due to hint bit updates, and doing
an explicit VACUUM after the load and before you start using the
database will avoid run-time issues. You also need statistics, so
be sure to do VACUUM ANALYZE.

There is one other sneaky surprise awaiting you, however. Since
this stuff was all loaded with a narrow range of transaction IDs,
they will all need to be frozen at about the same time; so somewhere
down the road, either during a routine database vacuum or possibly
in the middle of normal operations, all of these rows will need to
be rewritten *again* to change the transaction IDs used for managing
MVCC to the special "frozen" value. We routinely follow a load with
VACUUM FREEZE ANALYZE of the database to combine the update to
freeze the tuples with the update to set the hint bits and avoid
this problem.

There has been some talk about possibly writing tuples in a frozen
state with the hint bits already set if they are loaded in the same
database transaction which creates the table, but I'm not aware of
anyone currently working on this.

-Kevin

--
Sent via pgsql-performance mailing list (pgsql-performance@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance

Re: Fw: Help me put 2 Gigs of RAM to use

2009-12-10T08:03:11Z

On Thu, 10 Dec 2009, Mark Stosberg wrote:
> What I'm noticing is that the while the FreeBSD server has 4 Gigs of
> memory, there are rarely every more than 2 in use-- the memory use
> graphs as being rather constant. My goal is to make good use of those 2
> Gigs of memory to improve performance and reduce the CPU usage.

I think you'll find that the RAM is already being used quite effectively
as disc cache by the OS. It sounds like the server is actually set up
pretty well. You may get slightly better performance by tweaking a thing
here or there, but the server needs some OS disc cache to perform well.

> (We currently run 8.2, but are planning an upgrade to 8.4 "soon").

Highly recommended.

> [I tried to post this yesterday but didn't see it come through. This
> message is a second attempt.)

The mailing list server will silently chuck any message whose subject
starts with the word "help", just in case you're asking for help about
managing the mailing list. The default behaviour is not to inform you that
it has done so. It is highly annoying - could a list admin please consider
changing this?

Matthew

--
I would like to think that in this day and age people would know better than
to open executables in an e-mail. I'd also like to be able to flap my arms
and fly to the moon. -- Tim Mullen

--
Sent via pgsql-performance mailing list (pgsql-performance@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance