Nabble - PostgreSQL - general

Re: Justifying a PG over MySQL approach to a project

2009-12-17T19:28:44Z

On Thu, Dec 17, 2009 at 7:51 PM, David Boreham <david_list@...> wrote:

> Scott Marlowe wrote:
>>
>> I would recommend using a traffic shaping router (like the one built
>> into the linux kernel and controlled by tc / iptables) to simulate a
>> long distance connection and testing this yourself to see which
>> replication engine will work best for you.
>>
>
> Netem :
> http://www.linuxfoundation.org/collaborate/workgroups/networking/netem
> We used this to make a test rig for Directory Server replication, to verify
> a
> re-design that added pipelining to the replication protocol.
> It's already in the modern Linuxes--just needs to be configured.

Wow, everytime I turn around someone's built something cool from a set
of small sharp tools. Thanks!

--
Sent via pgsql-general mailing list (pgsql-general@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

Re: Justifying a PG over MySQL approach to a project

2009-12-17T18:51:08Z

Scott Marlowe wrote:
> I would recommend using a traffic shaping router (like the one built
> into the linux kernel and controlled by tc / iptables) to simulate a
> long distance connection and testing this yourself to see which
> replication engine will work best for you.
>
Netem :
http://www.linuxfoundation.org/collaborate/workgroups/networking/netem
We used this to make a test rig for Directory Server replication, to
verify a
re-design that added pipelining to the replication protocol.
It's already in the modern Linuxes--just needs to be configured.

--
Sent via pgsql-general mailing list (pgsql-general@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

Re: Justifying a PG over MySQL approach to a project

2009-12-17T18:47:18Z

-----BEGIN PGP SIGNED MESSAGE-----
Hash: RIPEMD160

> One concern I have about these trigger based replication systems is
> that I fear it may ping the slave for each and every DML statement
> separately in time and in a transaction. My slave will literally be
> 1400 miles away and all replication communications will be over the
> net. If I have a transaction which has 1000 DML statements in it, is
> this thing going to update the slave 1000 times separately over the
> net ? (I may not live long enough to see it finish) Or will it be
> smart enough to wait until I "commit" then send over a single bundle
> of 1000 DML? The time diff will be more than significant.

Both Slony and Bucardo are transaction based, meaning all 1000 changes
will happen at once, and only when the transaction has committed. In
practice, you'll find that both are surprisingly quick.

With Bucardo, the number of changes may be much less than 1000, as it
is data based, not statement based. Thus, as an extreme example, if
the 1000 statements are all updating a single row, only a single
update (actually a delete/insert) is done on the slave.

- --
Greg Sabino Mullane greg@...
End Point Corporation http://www.endpoint.com/
PGP Key: 0x14964AC8 200912172144
http://biglumber.com/x/web?pk=2529DF6AB8F79407E94445B4BC9B906714964AC8
-----BEGIN PGP SIGNATURE-----

iEYEAREDAAYFAksq7RMACgkQvJuQZxSWSsgvNQCg0Og/Ilg7rO9UjBecCTVwf04x
E1cAn05wdiJSh+VN3XpZrsFhNJYUTYjG
=VLf6
-----END PGP SIGNATURE-----

--
Sent via pgsql-general mailing list (pgsql-general@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

Re: Justifying a PG over MySQL approach to a project

2009-12-17T18:40:39Z

I would recommend using a traffic shaping router (like the one built
into the linux kernel and controlled by tc / iptables) to simulate a
long distance connection and testing this yourself to see which
replication engine will work best for you.

On Thu, Dec 17, 2009 at 7:35 PM, Gauthier, Dave <dave.gauthier@...> wrote:

> One concern I have about these trigger based replication systems is that I
> fear it may ping the slave for each and every DML statement separately in
> time and in a transaction. My slave will literally be 1400 miles away and
> all replication communications will be over the net. If I have a
> transaction which has 1000 DML statements in it, is this thing going to
> update the slave 1000 times separately over the net ? (I may not live long
> enough to see it finish) Or will it be smart enough to wait until I
> "commit" then send over a single bundle of 1000 DML? The time diff will be
> more than significant.
>
> Thanks for all the great input on this!
>
> -----Original Message-----
> From: pgsql-general-owner@...
> [mailto:pgsql-general-owner@...] On Behalf Of Greg Sabino Mullane
> Sent: Thursday, December 17, 2009 11:58 AM
> To: pgsql-general@...
> Subject: Re: [GENERAL] Justifying a PG over MySQL approach to a project
>
>
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: RIPEMD160
>
>
>> How difficult is it to switch the master's hat from one DB instance
>> to another? Let's say the master in a master-slave scenario goes
>> down but the slave is fine. Can I designate the slave as being the
>> new master, use it for read/write, and then just call the broken
>> master the new slave once it comes back to life (something like that)?
>
> Sure. Bucardo slaves are not changed at all, so they are already
> read/write and don't need anything special done to "unslave" them.
>
> One possible way to handle the scenario is:
>
> Assuming three servers:
> * A (master) sends changes to B, receives read/write queries
> * B (slave) has transaction_read_only set to true, receives read queries
> * C has the Bucardo database and daemon
>
> Box A goes down suddenly.
>
> * Stop Bucardo on box C
> * Flip the boxes around in the bucardo.db table
> * Do a 'bucardo_ctl validate sync all'
> (This will create the needed triggers on B)
> * Set B's transaction_read_only to false
> * Point your apps at B instead of A for read/write queries
>
> When A comes back up:
>
> * DROP SCHEMA bucardo CASCADE; (drops all triggers)
> * Set transaction_read_only to true
> * Start Bucardo on C
> * Once caught up, point read-only queries to A
>
> If you are in a rush, you point things to B immediately after A fails,
> but you'll have to recopy the entire table data to the slave, as the
> triggers won't be in place yet.
>
> - --
> Greg Sabino Mullane greg@...
> End Point Corporation http://www.endpoint.com/
> PGP Key: 0x14964AC8 200912171153
> http://biglumber.com/x/web?pk=2529DF6AB8F79407E94445B4BC9B906714964AC8
> -----BEGIN PGP SIGNATURE-----
>
> iEYEAREDAAYFAksqYqUACgkQvJuQZxSWSsjZtQCfTwbI3f9W0z+82IU7lL+2LwNK
> aUYAnj3AMjRDOeFIuHDee4JJemneArie
> =75Ho
> -----END PGP SIGNATURE-----
>
>
>
> --
> Sent via pgsql-general mailing list (pgsql-general@...)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-general
>

--
When fascism comes to America, it will be intolerance sold as diversity.

--
Sent via pgsql-general mailing list (pgsql-general@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

Re: pg_dump and ON DELETE CASCADE problem

2009-12-17T18:39:45Z

--- On Thu, 12/17/09, Adrian Klaver <aklaver@...> wrote:

>
> Would it be possible to see the table schemas and indices
> ?
>
> >

Sure (you asked for it!!) :

CREATE TABLE packet
(
id integer NOT NULL DEFAULT nextval('packet_id_seq'::regclass),
packet_uuid uniqueidentifier NOT NULL DEFAULT newid(),
username character varying(50) NOT NULL DEFAULT ''::character varying,
pgroup_uuid uniqueidentifier DEFAULT newid(),
orig_trans_uuid uniqueidentifier,
user_reference_id character varying(50) DEFAULT ''::character varying,
trans_data character varying(100) NOT NULL DEFAULT ''::character varying,
trans_type character varying(50) NOT NULL DEFAULT 'unknown'::character varying,
trans_date timestamp with time zone DEFAULT ('now'::text)::timestamp(6) with time zone,
processor character varying(10),
service character varying(10),
CONSTRAINT packet_pkey PRIMARY KEY (id)
)
WITH (
OIDS=TRUE
);
ALTER TABLE packet OWNER TO postgres;
GRANT ALL ON TABLE packet TO postgres;
GRANT ALL ON TABLE packet TO adduser;

CREATE INDEX packet_otuuid_idx
ON packet
USING btree
(orig_trans_uuid);

CREATE INDEX packet_pgroup_uuid_idx
ON packet
USING btree
(pgroup_uuid);

CREATE INDEX packet_puuid_hash_uniq
ON packet
USING hash
(packet_uuid);

CREATE UNIQUE INDEX packet_puuid_idx
ON packet
USING btree
(packet_uuid);

CREATE INDEX packet_trans_date_idx
ON packet
USING btree
(trans_date);

CREATE INDEX packet_user_idx
ON packet
USING btree
(username);

CREATE INDEX packet_user_puuid_idx
ON packet
USING btree
(username, packet_uuid);

CREATE OR REPLACE RULE packet_delete_rule AS
ON DELETE TO packet DO INSERT INTO removed_packet (id, packet_uuid, username, pgroup_uuid, orig_trans_uuid, user_reference_id, trans_data, trans_type, trans_date, processor, service) SELECT packet.id, packet.packet_uuid, packet.username, packet.pgroup_uuid, packet.orig_trans_uuid, packet.user_reference_id, packet.trans_data, packet.trans_type, packet.trans_date, packet.processor, packet.service
FROM packet
WHERE packet.id = old.id;

CREATE TRIGGER packet_count_delete_trig
BEFORE DELETE
ON packet
FOR EACH ROW
EXECUTE PROCEDURE letter_count_trig();

CREATE TRIGGER packet_count_insert_trig
AFTER INSERT
ON packet
FOR EACH ROW
EXECUTE PROCEDURE letter_count_trig();

CREATE TRIGGER packet_delete_trig
BEFORE DELETE
ON packet
FOR EACH ROW
EXECUTE PROCEDURE packet_datalink_status_trig();

CREATE TRIGGER packet_insert_trig
AFTER INSERT
ON packet
FOR EACH ROW
EXECUTE PROCEDURE packet_ins_trig();

CREATE TABLE dpo.packet_search_trigram
(
id integer NOT NULL DEFAULT nextval('packet_search_trigram_id_seq'::regclass),
packet_uuid uniqueidentifier NOT NULL,
trigram_vector tsvector NOT NULL,
CONSTRAINT packet_search_trigram_id_pkey PRIMARY KEY (id),
CONSTRAINT packet_search_trigram_puuid_fkey FOREIGN KEY (packet_uuid)
REFERENCES dpo.packet (packet_uuid) MATCH SIMPLE
ON UPDATE CASCADE ON DELETE CASCADE
)
WITH (
OIDS=FALSE
);
ALTER TABLE dpo.packet_search_trigram OWNER TO postgres;
GRANT ALL ON TABLE dpo.packet_search_trigram TO postgres WITH GRANT OPTION;
GRANT ALL ON TABLE dpo.packet_search_trigram TO addgroup;

CREATE INDEX packet_search_trigram_packet_uuid_idx
ON dpo.packet_search_trigram
USING hash
(packet_uuid);

CREATE INDEX packet_search_trigram_trigram_vector_idx
ON dpo.packet_search_trigram
USING gin
(trigram_vector);

--
Sent via pgsql-general mailing list (pgsql-general@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

Re: Justifying a PG over MySQL approach to a project

2009-12-17T18:35:35Z

One concern I have about these trigger based replication systems is that I fear it may ping the slave for each and every DML statement separately in time and in a transaction. My slave will literally be 1400 miles away and all replication communications will be over the net. If I have a transaction which has 1000 DML statements in it, is this thing going to update the slave 1000 times separately over the net ? (I may not live long enough to see it finish) Or will it be smart enough to wait until I "commit" then send over a single bundle of 1000 DML? The time diff will be more than significant.

Thanks for all the great input on this!

-----Original Message-----
From: pgsql-general-owner@... [pgsql-general-owner@...] On Behalf Of Greg Sabino Mullane
Sent: Thursday, December 17, 2009 11:58 AM
To: pgsql-general@...
Subject: Re: [GENERAL] Justifying a PG over MySQL approach to a project

-----BEGIN PGP SIGNED MESSAGE-----

Hash: RIPEMD160

> How difficult is it to switch the master's hat from one DB instance

> to another? Let's say the master in a master-slave scenario goes

> down but the slave is fine. Can I designate the slave as being the

> new master, use it for read/write, and then just call the broken

> master the new slave once it comes back to life (something like that)?

Sure. Bucardo slaves are not changed at all, so they are already

read/write and don't need anything special done to "unslave" them.

One possible way to handle the scenario is:

Assuming three servers:

* A (master) sends changes to B, receives read/write queries

* B (slave) has transaction_read_only set to true, receives read queries

* C has the Bucardo database and daemon

Box A goes down suddenly.

* Stop Bucardo on box C

* Flip the boxes around in the bucardo.db table

* Do a 'bucardo_ctl validate sync all'

(This will create the needed triggers on B)

* Set B's transaction_read_only to false

* Point your apps at B instead of A for read/write queries

When A comes back up:

* DROP SCHEMA bucardo CASCADE; (drops all triggers)

* Set transaction_read_only to true

* Start Bucardo on C

* Once caught up, point read-only queries to A

If you are in a rush, you point things to B immediately after A fails,

but you'll have to recopy the entire table data to the slave, as the

triggers won't be in place yet.

- --

Greg Sabino Mullane greg@...

End Point Corporation http://www.endpoint.com/

PGP Key: 0x14964AC8 200912171153

http://biglumber.com/x/web?pk=2529DF6AB8F79407E94445B4BC9B906714964AC8

-----BEGIN PGP SIGNATURE-----

iEYEAREDAAYFAksqYqUACgkQvJuQZxSWSsjZtQCfTwbI3f9W0z+82IU7lL+2LwNK

aUYAnj3AMjRDOeFIuHDee4JJemneArie

=75Ho

-----END PGP SIGNATURE-----

Sent via pgsql-general mailing list (pgsql-general@...)

To make changes to your subscription:

http://www.postgresql.org/mailpref/pgsql-general

Re: Server Requirements

2009-12-17T18:16:15Z

On Wed, Dec 16, 2009 at 4:21 PM, Christine Penner
<christine@...> wrote:
> Hi,
>
> If we have clients that are going to buy new computers or upgrade current
> ones, what we can recommend to them for optimal system performance to run
> Postgres. These can be servers or desktop PCs. We can have from 1-10 users
> in at a time. At this point all of our database's are small but that can
> change of course.

Like Craig said, your description is pretty vague, so it's hard to say
what you need most to make your pgsql server run fast.

Generally, the order of things to do to make it fast are: Add memory,
add hard drives, add battery backed caching RAID controllers, add more
CPU horsepower.

But it depends largely on your usage patterns. So, what are you doing
with your db?

--
Sent via pgsql-general mailing list (pgsql-general@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

Re: Justifying a PG over MySQL approach to a project

2009-12-17T18:06:35Z

On Thu, Dec 17, 2009 at 12:35 PM, Madison Kelly <linux@...> wrote:

> Gauthier, Dave wrote:
>>
>> Hi Everyone:
>>
>>
>> Tomorrow, I will need to present to a group of managers (who know nothing
>> about DBs) why I chose to use PG over MySQL in a project, MySQL being the
>> more popular DB choice with other engineers, and managers fearing things
>> that are “different” (risk). I have a few hard tecnical reasons (check
>> constraint, deferred constraint checking, array data type), but I’m looking
>> for a “it’s more reliable” reasons. Again, the audience is managers. Is
>> there an impartial, 3^rd party evaluation of the 2 DBs out there that
>> identifies PG as being more reliable? It might mention things like fewer
>> incidences of corrupt tables/indexes, fewer deamon crashes, better recovery
>> after system crashes, etc... ?
>>
>>
>> Thanks !
>
> There is a current question about the survivability of MySQL right now with
> the potential sale of MySQL. I would not bank on MySQL for any long-term
> project. I am sure that MySQL will live in the long run, but they may well
> be turbulent times ahead if whomever comes to own MySQL decides to neglect
> or kill it and the source gets forked.

It's important to separate out MySQL AB the company, owned by Sun,
soon to be owned by Oracle, from MySQL the GPL licensed database,
which may or may not allow you to distribute your own commercial code
without buying a license. Given the OSS License loophole, and the
fact that many of those OSS licenses do NOT require the release of
source code, there's every possibiliy you could release your
commercial code under the BSD code to yourself, and then give only the
compiled code to customers and you'd technically be safe distributing
MySQL with it.

There are several companies in the MySQL biosphere that are releasing
their own forks of MySQL with lots of bug fixes, and I have no doubt
that MySQL the GPL database will continue to be available as a GPL
product for quite some time. However, the availability of
commercially licensed MySQL may or may not continue based on the
business needs of Oracle.

For in house use only, this is a non-issue, as the GPL only affects
distribution of MySQL, not internal usage.

The biggest argument I'd use against MySQL in general is that
PostgreSQL is a better database for any time your data and its
integrity are important.

--
Sent via pgsql-general mailing list (pgsql-general@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

Re: Extended Query using the Frontend/Backend Protocol 3.0

2009-12-17T17:26:21Z

On Dec 17, 2009, at 11:13 AM, Raimon Fernandez wrote:

> I'm trying to integrate the extended query protocol with my libraries.
>
> I'm sending a simple SELECT to validate the method, but I'm getting an Invalid Message Format.

I did not add up your byte count, but maybe this will help:

(write-byte p stream)
(write-int32 (+ int32-length (length name) 1 (length sql-string) 1 int16-length (* int32-length param-count)) stream)
(write-cstring name stream)
(write-cstring sql-string stream)
(write-int16 param-count stream)

John DeSoi, Ph.D.

--
Sent via pgsql-general mailing list (pgsql-general@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

Re: alter table performance

2009-12-17T15:49:08Z

Op 17-12-09 23:46, Tom Lane schreef:
> This is just one of many many things that could be improved by getting
> off of 7.3. In general, complaining about performance (or features)
> of a seven-year-old, long since EOL'd release is not a productive use of
> anybody's time.
>

I'm sorry, didn't mean to.

I was just checking if I did it the right way, or if it was supposed to
be that slow.

I know we should upgrade the client machines, but in some cases we just
can't.

Anyway, thanks for giving me the answer.

Antonio

--
Sent via pgsql-general mailing list (pgsql-general@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

Re: Table Partitioning Advice Request

2009-12-17T15:41:04Z

On Thu, Dec 17, 2009 at 8:29 AM, Vincenzo Romano
<vincenzo.romano@...> wrote:
> 2009/12/17 Sam Jas <samjas33@...>
>>
>> Rule is not advisable, Trigger is the best solution.
>
> Does the trigger solution need the TABLE CHECK constraint?
> It looks to me it won't.

The table check constraint is used during selects to pick just the
child tables needed to deliver the data you are asking for, assuming
the key value is in the where clause.

The main reason to avoid rules is that they're much less efficient
than triggers. We use partitioning at work for our stats db, and
partition by day, and we have about 2 years worth of stats data, so
our insert trigger has about 700 if / elseif / else conditions in it,
and the trigger is still very very fast. I tried it with rules before
and it was way too slow.

Note that the absolute fastest way to insert to the right child table
is to do it from the application side, choosing the right child table
there.

We automated our trigger creation. While the trigger is huge in terms
of number of lines, speed wise the creation of the new trigger each
night at midnight is measured in milliseconds

--
Sent via pgsql-general mailing list (pgsql-general@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

Re: Installing PL/pgSQL by default

2009-12-17T15:39:55Z

Bruce Momjian wrote:

> Tom Lane wrote:
> > Bruce Momjian <bruce@...> writes:
> > > I installed PL/pgSQL by default via initdb with the attached patch. The
> > > only problem is that pg_dump still dumps out the language creation:
> > > CREATE PROCEDURAL LANGUAGE plpgsql;
> > > ALTER PROCEDURAL LANGUAGE plpgsql OWNER TO postgres;
> > > What is odd is that I used the same process that initdb uses to create
> > > other objects. Does anyone know why this is happening?
> >
> > I think pg_dump pays attention to what schema the objects are in,
> > and that's most likely creating them in PUBLIC. Try adding
> > "set search_path = pg_catalog".
> >
> > It's not impossible that we'll have to tweak pg_dump a bit; it's
> > never had to deal with languages that shouldn't be dumped ...
>
> I found that pg_dump tests for pg_language.lanispl == true, which is
> true for all the stored procedure languages. I can easily special case
> plpgsql, or check for FirstNormalObjectId, though I don't see that used
> in pg_dump currently.
>
> A more difficult issue is whether we should preserve the fact that
> plpgsql was _removed_ in the pg_dump output, i.e, if someone removes
> plpgsql from a database, do we issue a DROP LANGUAGE in pg_dump? I
> don't remember us having to deal with anything like this before.

OK, the attached patch installs plpgsql by default from initdb, and
supresses the dumping of CREATE LANGUAGE in 8.5 and in 8.3/8.4 if binary
upgrade is used (because you know you are upgrading to a release that
has plpgsql installed by default). The 8.3/8.4 is necessary so the
schema load doesn't generate any errors and cause pg_migrator to exit.

--
Bruce Momjian <bruce@...> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

Index: doc/src/sgml/installation.sgml
===================================================================
RCS file: /cvsroot/pgsql/doc/src/sgml/installation.sgml,v
retrieving revision 1.333
diff -c -c -r1.333 installation.sgml
*** doc/src/sgml/installation.sgml 15 Dec 2009 22:59:53 -0000 1.333
--- doc/src/sgml/installation.sgml 17 Dec 2009 23:35:36 -0000
***************
*** 2266,2279 ****
is <command>createlang</command> failing with unusual errors.
For example, running as the owner of the PostgreSQL installation:
<screen>
! -bash-3.00$ createlang plpgsql template1
! createlang: language installation failed: ERROR: could not load library "/opt/dbs/pgsql748/lib/plpgsql.so": A memory address is not in the address space for the process.
</screen>
Running as a non-owner in the group posessing the PostgreSQL
installation:
<screen>
! -bash-3.00$ createlang plpgsql template1
! createlang: language installation failed: ERROR: could not load library "/opt/dbs/pgsql748/lib/plpgsql.so": Bad address
</screen>
Another example is out of memory errors in the PostgreSQL server
logs, with every memory allocation near or greater than 256 MB
--- 2266,2279 ----
is <command>createlang</command> failing with unusual errors.
For example, running as the owner of the PostgreSQL installation:
<screen>
! -bash-3.00$ createlang plperl template1
! createlang: language installation failed: ERROR: could not load library "/opt/dbs/pgsql748/lib/plperl.so": A memory address is not in the address space for the process.
</screen>
Running as a non-owner in the group posessing the PostgreSQL
installation:
<screen>
! -bash-3.00$ createlang plperl template1
! createlang: language installation failed: ERROR: could not load library "/opt/dbs/pgsql748/lib/plperl.so": Bad address
</screen>
Another example is out of memory errors in the PostgreSQL server
logs, with every memory allocation near or greater than 256 MB
Index: src/bin/initdb/initdb.c
===================================================================
RCS file: /cvsroot/pgsql/src/bin/initdb/initdb.c,v
retrieving revision 1.178
diff -c -c -r1.178 initdb.c
*** src/bin/initdb/initdb.c 11 Dec 2009 03:34:56 -0000 1.178
--- src/bin/initdb/initdb.c 17 Dec 2009 23:35:36 -0000
***************
*** 176,181 ****
--- 176,182 ----
static void setup_privileges(void);
static void set_info_version(void);
static void setup_schema(void);
+ static void load_plpgsql(void);
static void vacuum_db(void);
static void make_template0(void);
static void make_postgres(void);
***************
*** 1894,1899 ****
--- 1895,1925 ----
}

/*
+ * load PL/pgsql server-side language
+ */
+ static void
+ load_plpgsql(void)
+ {
+ PG_CMD_DECL;
+
+ fputs(_("loading PL/pgSQL server-side language ... "), stdout);
+ fflush(stdout);
+
+ snprintf(cmd, sizeof(cmd),
+ "\"%s\" %s template1 >%s",
+ backend_exec, backend_options,
+ DEVNULL);
+
+ PG_CMD_OPEN;
+
+ PG_CMD_PUTS("CREATE LANGUAGE plpgsql;\n");
+
+ PG_CMD_CLOSE;
+
+ check_ok();
+ }
+
+ /*
* clean everything up in template1
*/
static void
***************
*** 3126,3131 ****
--- 3152,3159 ----

setup_schema();

+ load_plpgsql();
+
vacuum_db();

make_template0();
Index: src/bin/pg_dump/pg_dump.c
===================================================================
RCS file: /cvsroot/pgsql/src/bin/pg_dump/pg_dump.c,v
retrieving revision 1.556
diff -c -c -r1.556 pg_dump.c
*** src/bin/pg_dump/pg_dump.c 14 Dec 2009 00:39:11 -0000 1.556
--- src/bin/pg_dump/pg_dump.c 17 Dec 2009 23:35:36 -0000
***************
*** 32,37 ****
--- 32,38 ----

#include "access/attnum.h"
#include "access/sysattr.h"
+ #include "access/transam.h"
#include "catalog/pg_cast.h"
#include "catalog/pg_class.h"
#include "catalog/pg_default_acl.h"
***************
*** 4599,4606 ****
"(%s lanowner) AS lanowner "
"FROM pg_language "
"WHERE lanispl "
"ORDER BY oid",
! username_subquery);
}
else if (g_fout->remoteVersion >= 80300)
{
--- 4600,4609 ----
"(%s lanowner) AS lanowner "
"FROM pg_language "
"WHERE lanispl "
+ /* do not dump initdb-installed languages */
+ "AND oid >= %u "
"ORDER BY oid",
! username_subquery, FirstNormalObjectId);
}
else if (g_fout->remoteVersion >= 80300)
{
***************
*** 4610,4618 ****
"lanvalidator, lanacl, "
"(%s lanowner) AS lanowner "
"FROM pg_language "
! "WHERE lanispl "
"ORDER BY oid",
! username_subquery);
}
else if (g_fout->remoteVersion >= 80100)
{
--- 4613,4622 ----
"lanvalidator, lanacl, "
"(%s lanowner) AS lanowner "
"FROM pg_language "
! "WHERE lanispl%s"
"ORDER BY oid",
! username_subquery,
! binary_upgrade ? "\nAND lanname != 'plpgsql'" : "");
}
else if (g_fout->remoteVersion >= 80100)
{
Index: src/test/regress/GNUmakefile
===================================================================
RCS file: /cvsroot/pgsql/src/test/regress/GNUmakefile,v
retrieving revision 1.79
diff -c -c -r1.79 GNUmakefile
*** src/test/regress/GNUmakefile 26 Oct 2009 21:11:22 -0000 1.79
--- src/test/regress/GNUmakefile 17 Dec 2009 23:35:38 -0000
***************
*** 138,144 ****
## Run tests
##

! pg_regress_call = ./pg_regress --inputdir=$(srcdir) --dlpath=. --multibyte=$(MULTIBYTE) --load-language=plpgsql $(NOLOCALE)

check: all
$(pg_regress_call) --temp-install=./tmp_check --top-builddir=$(top_builddir) --schedule=$(srcdir)/parallel_schedule $(MAXCONNOPT) $(TEMP_CONF)
--- 138,144 ----
## Run tests
##

! pg_regress_call = ./pg_regress --inputdir=$(srcdir) --dlpath=. --multibyte=$(MULTIBYTE) $(NOLOCALE)

check: all
$(pg_regress_call) --temp-install=./tmp_check --top-builddir=$(top_builddir) --schedule=$(srcdir)/parallel_schedule $(MAXCONNOPT) $(TEMP_CONF)

--
Sent via pgsql-general mailing list (pgsql-general@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

Re: alter table performance

2009-12-17T14:46:03Z

Thomas Kellerer <spam_eater@...> writes:
> What's wrong with:
> alter table person add column address varchar(64) not null default '';

This:

regression=# alter table person add column address varchar(64) not null default '';
ERROR: Adding columns with defaults is not implemented.
Add the column, then use ALTER TABLE SET DEFAULT.
regression=# \q

This is just one of many many things that could be improved by getting
off of 7.3. In general, complaining about performance (or features)
of a seven-year-old, long since EOL'd release is not a productive use of
anybody's time.

regards, tom lane

--
Sent via pgsql-general mailing list (pgsql-general@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

Re: feature request: create table with unique constraint

2009-12-17T14:39:39Z

<wumpus@...> writes:
> , unique ( a_id , ( case when actual then 0 else id end ) )

> Why can i not define the unique constraint in the
> create table?

The syntax for a unique constraint in CREATE TABLE is defined by the SQL
standard, and it doesn't include expressions. More than the syntax, a
unique constraint also ought to show up in the information_schema views,
and those don't have the ability to handle expressions. So we have the
capability, but it's only available via CREATE INDEX which is outside
the standard.

regards, tom lane

--
Sent via pgsql-general mailing list (pgsql-general@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

Re: pg_dump and ON DELETE CASCADE problem

2009-12-17T14:39:20Z

----- "CG" <cgg007@...> wrote:

> --- On Tue, 12/15/09, Adrian Klaver <aklaver@...> wrote:
>
> > From: Adrian Klaver <aklaver@...>
> > Subject: Re: [GENERAL] pg_dump and ON DELETE CASCADE problem
> > To: cgg007@...
> > Cc: "postgresql listserv" <pgsql-general@...>, "Craig
> Ringer" <craig@...>, "Scott Marlowe"
> <scott.marlowe@...>
> > Date: Tuesday, December 15, 2009, 6:53 PM
> > On Tuesday 15 December 2009 2:33:39
> > pm CG wrote:
> >
> > >
> > > Bingo. Showed right up. I did a reindex, and now it
> > shows up searching via
> > > sequential scan or index scan.
> > >
> > > So that's pretty scary to have a corrupted index. Once
> > I reindexed, I'm
> > > able to see /a lot/ of data I couldn't before. This is
> > the first time in 9
> > > years that I've been bitten by PostgreSQL, and this
> > one HURT.
> > >
> > > PostgreSQL didn't crash, so there was no indication of
> > failure until the
> > > demp-reload. To quote from the masters: Although in
> > theory this should
> > > never happen, in practice indexes may become corrupted
> > due to software bugs
> > > or hardware failures. I'm reasonably certain that the
> > hardware for the
> > > server is sound. No crashes, no alarms... That leaves
> > sofware bugs.
> > >
> > > We're running PostgreSQL 8.4.1. I don't see any
> > smoking gun bugfixes in
> > > 8.4.2, but we'll upgrade ASAP anyway...
> > >
> > > What are your suggestions for how to proceed?
> >
> > Interesting, though something is still bothering me. To
> > quote from one of your
> > posts upstream;
> >
> > "That was the same failure I got the previous night. I go
> > to the live database
> > and rows with that key are /not/ in either one of those
> > tables. They /were/ in
> > the tables at one point. I have an ON DELETE trigger that
> > copies deleted rows
> > into another table, so I can see that a row with that key
> > once existed in those
> > tables."
> >
> > Would seem that the rows where deleted and should not be
> > there when the table
> > was reindexed. Are the 'new' rows you are seeing also in
> > the delete table?
> >
>
> select foo from bar where baz = 'key';
>
> I was mistaken when I said that the row was not in the table. If I had
> an index on baz, and the index was corrupted, that SQL would return 0
> rows leading me to believe that there were no rows in the table with
> that key.
>
> And, the reason for that row remaining in the database after its
> foreign keyed parent row was deleted was because the delete operation
> was depending on the index to find the rows to delete, and that index
> was corrupt.
>
> Of course, I had no idea that the index was corrupt when I made my
> first post.
>
> On the table that has the "martian" row, there is no delete storage.
> Since the data in the table is trigger-generated for FTI searches,
> there's no reason to keep that data around.

Would it be possible to see the table schemas and indices ?

>
> I'm still faced with the unpleasant and unresolved issue of why the
> index was corrupted in the first place.
>
>

Adrian Klaver
aklaver@...

--
Sent via pgsql-general mailing list (pgsql-general@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

Re: alter table performance

2009-12-17T14:30:28Z

Antonio Goméz Soto wrote on 17.12.2009 22:26:

> Hi,
>
> I am regularly altering tables, adding columns setting default values etc.
> This very often takes a very long time and is very disk intensive, and this
> gets pretty annoying.
>
> Things are hampered by the fact that some of our servers run PG 7.3
>
> Suppose I have a table and I want to add a non NULL column with a
> default value.
> What I normally do is:
>
> alter table person add column address varchar(64);
> update person set address = '' where address IS NULL;
> alter table person alter column address set not NULL;
> alter table person alter column address set default '';
>
> When the table contains millions of records this takes forever.
>
> Am I doing something wrong? Do other people have the same problems?

What's wrong with:

alter table person add column address varchar(64) not null default '';

Although I don't know if such a pre-historic version like 7.3 would support that.

It works for 8.4 and I believe this was working with 8.3 and 8.2 as well

Thomas

--
Sent via pgsql-general mailing list (pgsql-general@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

Re: alter table performance

2009-12-17T14:27:03Z

On Thursday 17 December 2009, Antonio Goméz Soto
<antonio.gomez.soto@...> wrote:

> Hi,
>
> I am regularly altering tables, adding columns setting default values
> etc. This very often takes a very long time and is very disk intensive,
> and this gets pretty annoying.
>
> Things are hampered by the fact that some of our servers run PG 7.3
>
> Suppose I have a table and I want to add a non NULL column with a default
> value. What I normally do is:
>
> alter table person add column address varchar(64);
> update person set address = '' where address IS NULL;
> alter table person alter column address set not NULL;
> alter table person alter column address set default '';
>
> When the table contains millions of records this takes forever.
>
> Am I doing something wrong? Do other people have the same problems?
>
> Thanks,
> Antonio

You could speed it up:

- drop all indexes on the table
- alter table person add column address varchar(64) not null default ''
- recreate the indexes

It will require exclusive access to the table for the duration, but it'll be
a lot faster and result in a lot less bloat than what you're doing now. It
still has to rewrite the whole table, but it's a lot faster than UPDATE.

(I have no idea if this works on 7.3).

--
"No animals were harmed in the recording of this episode. We tried but that
damn monkey was just too fast."

--
Sent via pgsql-general mailing list (pgsql-general@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

alter table performance

2009-12-17T13:26:53Z

Hi,

I am regularly altering tables, adding columns setting default values etc.
This very often takes a very long time and is very disk intensive, and this
gets pretty annoying.

Things are hampered by the fact that some of our servers run PG 7.3

Suppose I have a table and I want to add a non NULL column with a default value.
What I normally do is:

alter table person add column address varchar(64);
update person set address = '' where address IS NULL;
alter table person alter column address set not NULL;
alter table person alter column address set default '';

When the table contains millions of records this takes forever.

Am I doing something wrong? Do other people have the same problems?

Thanks,
Antonio

--
Sent via pgsql-general mailing list (pgsql-general@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

feature request: create table with unique constraint

2009-12-17T12:51:07Z

begin ;

create table a (
id serial primary key
, info text not null -- not interested here
) ;

create table b (
id serial primary key
, a_id int not null references a
, info text not null -- not interested here
, actual bool not null
) ;

create unique index b_actual on b
( a_id , ( case when actual then 0 else id end ) )
;

create table c (
id serial primary key
, a_id int not null references a
, info text not null -- not interested here
, actual bool not null
, unique ( a_id , ( case when actual then 0 else id end ) )
) ;

Why can i not define the unique constraint in the
create table? I know this is described in the manual but
why this is.

thanks
Andreas

btw Why must i give the index a name. The name should
be created automatically if none is specified as i
define a unique constraint in the create table.

--
Sent via pgsql-general mailing list (pgsql-general@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

Re: How to remove non-UTF values from a table?

2009-12-17T11:54:45Z

Bruce Momjian wrote:

> pg_migrator (not pg_upgrade) has been used by many people to migrate
> from 8.3 to 8.4. I just helped someone yesterday with a migration.
> pg_migrator threw an error because they had reindexed pg_largeobject,
> and pg_migrator was not prepared to handle that. They also had an old
> cluster with a higher autovacuum_freeze_max_age than the new cluster,
> causing autovacuum freeze to run during the upgrade and remove clog
> files. These are unusual cases, so I assume there are many people who
> successfully use pg_migrator that I never hear from.
>
> Both of these issues are fixed in pg_migrator 8.4.6, released yesterday.
> I also talked to Alvaro about using fixed oids to remove these migration
> restrictions in 8.5:
>
> o a user-defined composite data type
> o a user-defined array data type
> o a user-defined enum data type
>
> So, pg_migrator is a work in progress. It current requires migration
> testing and should be used by experienced Postgres users. Someday
> pg_migrator will be easier to use and more reliable.

Oh, and pg_migrator 8.4.6 works for migrations from 8.4 to 8.5 CVS.

--
Bruce Momjian <bruce@...> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

--
Sent via pgsql-general mailing list (pgsql-general@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

Re: How to remove non-UTF values from a table?

2009-12-17T11:51:24Z

Tom Lane wrote:

> "Raymond O'Donnell" <rod@...> writes:
> > On 16/12/2009 15:01, Richard Broersma wrote:
> >> It looks like the future 8.5 release will be able to
> >> preform an in-place upgrade on 8.4.
>
> > Really? That would be *wonderful*. I know it's impossible to be
> > definitive, but how likely would you say this is?
>
> It's wishful thinking. We tried to have that for 8.4, and failed: there
> are so many constraints on pg_upgrade that it's not a usable mainstream
> solution. So far as I've seen there has been no work done that would
> make it any more reliable for 8.5.

Some people just enjoy discouraging people.

pg_migrator (not pg_upgrade) has been used by many people to migrate
from 8.3 to 8.4. I just helped someone yesterday with a migration.
pg_migrator threw an error because they had reindexed pg_largeobject,
and pg_migrator was not prepared to handle that. They also had an old
cluster with a higher autovacuum_freeze_max_age than the new cluster,
causing autovacuum freeze to run during the upgrade and remove clog
files. These are unusual cases, so I assume there are many people who
successfully use pg_migrator that I never hear from.

Both of these issues are fixed in pg_migrator 8.4.6, released yesterday.
I also talked to Alvaro about using fixed oids to remove these migration
restrictions in 8.5:

o a user-defined composite data type
o a user-defined array data type
o a user-defined enum data type

So, pg_migrator is a work in progress. It current requires migration
testing and should be used by experienced Postgres users. Someday
pg_migrator will be easier to use and more reliable.

--
Bruce Momjian <bruce@...> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

--
Sent via pgsql-general mailing list (pgsql-general@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

Re: Justifying a PG over MySQL approach to a project

2009-12-17T11:35:10Z

Gauthier, Dave wrote:

> Hi Everyone:
>
>
>
> Tomorrow, I will need to present to a group of managers (who know
> nothing about DBs) why I chose to use PG over MySQL in a project, MySQL
> being the more popular DB choice with other engineers, and managers
> fearing things that are “different” (risk). I have a few hard tecnical
> reasons (check constraint, deferred constraint checking, array data
> type), but I’m looking for a “it’s more reliable” reasons. Again, the
> audience is managers. Is there an impartial, 3^rd party evaluation of
> the 2 DBs out there that identifies PG as being more reliable? It might
> mention things like fewer incidences of corrupt tables/indexes, fewer
> deamon crashes, better recovery after system crashes, etc... ?
>
>
>
> Thanks !

There is a current question about the survivability of MySQL right now
with the potential sale of MySQL. I would not bank on MySQL for any
long-term project. I am sure that MySQL will live in the long run, but
they may well be turbulent times ahead if whomever comes to own MySQL
decides to neglect or kill it and the source gets forked.

Madi

--
Sent via pgsql-general mailing list (pgsql-general@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

Re: Server Requirements

2009-12-17T11:30:30Z

Christine Penner wrote:
> Hi,
>
> If we have clients that are going to buy new computers or upgrade
> current ones, what we can recommend to them for optimal system
> performance to run Postgres. These can be servers or desktop PCs. We can
> have from 1-10 users in at a time. At this point all of our database's
> are small but that can change of course.

Hi Christine,

The problem with this question is that it is far too vague to be able
to answer in any meaningful way. You need to add some information to
your request. Things like:

- Data set size; How many tables, how many columns, how fast will it
grow, what kind of data are in the columns?
- Performance; are you using triggers, functions, a lot of complex or
simple queries, lots of UPDATEs, INSERTs and DELETEs?
- Redundancy; How do you plan to backup the data? What performance
criteria do you have? What's your acceptable down time in the case of a
failure?
- Interface; Users is one thing, but how many transactions will these
users incur?
- Budget; How much is your client willing to invest? What about
long-term maintenance or support contracts?
- Environment; What operating system will postgres run on?

Answer these questions and you will find the hardware requirements
will likely begin to become self-evident. If not, ask here again with
this info and we'll be much more able to help. :)

Madi

--
Sent via pgsql-general mailing list (pgsql-general@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

Re: How to remove non-UTF values from a table?

2009-12-17T11:29:27Z

Tom Lane wrote:

I do not share Tom's conclusions.

--
Bruce Momjian <bruce@...> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

--
Sent via pgsql-general mailing list (pgsql-general@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

Re: postgres 8.2.4 cores on SUN

2009-12-17T10:42:18Z

Abraham, Danny wrote:

>
> “postgres” cores on SUN 2.9. Disk is in 100% IO.
>
> (dbx) where
>
> [1] 0xffffffff65eacdd4(0xffffffff7fffa36c, 0x400, 0x10032d4e8, 0x1,
> 0x48, 0xd6), at 0xffffffff65eacdd4
>
> [2] XLogFileInit(0x48, 0xd6, 0xffffffff7fffa85d, 0x1, 0x488000, 0x0),
> at 0x10007955c
>
> ...
>
> Any idea?
>

now without a lot more input.

where did this postgres 8.2.4 come from, how is it configured, what
hardware is it running on, what operation(s) are you doing when it
'cores', what is the disk doing thats 10)% busy (eg, what file(s), etc)
was this system stable prior to this happening, etc etc etc.

--
Sent via pgsql-general mailing list (pgsql-general@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

Re: Trigger function language

2009-12-17T10:40:10Z

Vincenzo Romano <vincenzo.romano@...> writes:
> Is there any performance difference in writing trigger function in SQL
> rather than PL/PgSQL?

SQL functions don't support being called as triggers, so performance
questions are a bit beside the point ...

regards, tom lane

--
Sent via pgsql-general mailing list (pgsql-general@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

Re: Justifying a PG over MySQL approach to a project

2009-12-17T10:38:19Z

Gauthier, Dave wrote:

>
> Hi Everyone:
>
> Tomorrow, I will need to present to a group of managers (who know
> nothing about DBs) why I chose to use PG over MySQL in a project,
> MySQL being the more popular DB choice with other engineers, and
> managers fearing things that are “different” (risk). I have a few hard
> tecnical reasons (check constraint, deferred constraint checking,
> array data type), but I’m looking for a “it’s more reliable” reasons.
> Again, the audience is managers. Is there an impartial, 3^rd party
> evaluation of the 2 DBs out there that identifies PG as being more
> reliable? It might mention things like fewer incidences of corrupt
> tables/indexes, fewer deamon crashes, better recovery after system
> crashes, etc... ?
>

let me just say this one word about that.

ORACLE

i think its a pretty safe assumption that Oracle will not be good for MySQL.

--
Sent via pgsql-general mailing list (pgsql-general@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

Trigger function language

2009-12-17T10:31:10Z

Hi all.
Is there any performance difference in writing trigger function in SQL
rather than PL/PgSQL?
In my case the body would actually be 1 line.

--
Vincenzo Romano
NON QVIETIS MARIBVS NAVTA PERITVS

--
Sent via pgsql-general mailing list (pgsql-general@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

Re: pg_dump and ON DELETE CASCADE problem

2009-12-17T09:29:05Z

--- On Tue, 12/15/09, Adrian Klaver <aklaver@...> wrote:

> From: Adrian Klaver <aklaver@...>
> Subject: Re: [GENERAL] pg_dump and ON DELETE CASCADE problem
> To: cgg007@...
> Cc: "postgresql listserv" <pgsql-general@...>, "Craig Ringer" <craig@...>, "Scott Marlowe" <scott.marlowe@...>
> Date: Tuesday, December 15, 2009, 6:53 PM
> On Tuesday 15 December 2009 2:33:39
> pm CG wrote:
>
> >
> > Bingo. Showed right up. I did a reindex, and now it
> shows up searching via
> > sequential scan or index scan.
> >
> > So that's pretty scary to have a corrupted index. Once
> I reindexed, I'm
> > able to see /a lot/ of data I couldn't before. This is
> the first time in 9
> > years that I've been bitten by PostgreSQL, and this
> one HURT.
> >
> > PostgreSQL didn't crash, so there was no indication of
> failure until the
> > demp-reload. To quote from the masters: Although in
> theory this should
> > never happen, in practice indexes may become corrupted
> due to software bugs
> > or hardware failures. I'm reasonably certain that the
> hardware for the
> > server is sound. No crashes, no alarms... That leaves
> sofware bugs.
> >
> > We're running PostgreSQL 8.4.1. I don't see any
> smoking gun bugfixes in
> > 8.4.2, but we'll upgrade ASAP anyway...
> >
> > What are your suggestions for how to proceed?
>
> Interesting, though something is still bothering me. To
> quote from one of your
> posts upstream;
>
> "That was the same failure I got the previous night. I go
> to the live database
> and rows with that key are /not/ in either one of those
> tables. They /were/ in
> the tables at one point. I have an ON DELETE trigger that
> copies deleted rows
> into another table, so I can see that a row with that key
> once existed in those
> tables."
>
> Would seem that the rows where deleted and should not be
> there when the table
> was reindexed. Are the 'new' rows you are seeing also in
> the delete table?
>

select foo from bar where baz = 'key';

I was mistaken when I said that the row was not in the table. If I had an index on baz, and the index was corrupted, that SQL would return 0 rows leading me to believe that there were no rows in the table with that key.

And, the reason for that row remaining in the database after its foreign keyed parent row was deleted was because the delete operation was depending on the index to find the rows to delete, and that index was corrupt.

Of course, I had no idea that the index was corrupt when I made my first post.

On the table that has the "martian" row, there is no delete storage. Since the data in the table is trigger-generated for FTI searches, there's no reason to keep that data around.

I'm still faced with the unpleasant and unresolved issue of why the index was corrupted in the first place.

--
Sent via pgsql-general mailing list (pgsql-general@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

Re: Justifying a PG over MySQL approach to a project

2009-12-17T08:58:36Z

On Thursday 17 December 2009 09:51:19 Richard Broersma wrote:
> On Thu, Dec 17, 2009 at 8:23 AM, Gauthier, Dave <dave.gauthier@...>
wrote:
> > How difficult is it to switch the master's hat from one DB instance to
> > another? Let's say the master in a master-slave scenario goes down but
> > the slave is fine. Can I designate the slave as being the new master,
> > use it for read/write, and then just call the broken master the new slave
> > once it comes back to life (something like that)?

This is very easy with SLONY. If the master is truly 'dead' you can run a
SLONY 'failover' command like this (note these are slonik commands where node
1 is the 'dead' master and node 2 is the current slave):

echo 'Preparing to failover (set id = 1, backup node = 2)';
failover (id = 1, backup node = 2);
echo 'Done';
echo 'Preparing to drop node (set id = 1, event node = 2)';
drop node (id = 1, event node = 2);
echo 'Done';
echo 'Failover complete';

at this point the dead master node is no longer part of the replication
cluster and the slave is the new master

Once the dead node is rebuilt then you simply add it to the replication
cluster as a new slave node

Also, if you just wanted to "move" the master, changing the existing master
into a slave in the process you can do this:

lock set (id = 1, origin = 1);
wait for event (origin = 1, confirmed = 2);
echo 'set locked';
move set (id = 1, old origin = 1, new origin = 2);
wait for event (origin = 1, confirmed = 2);
echo 'switchover complete';

>
> I know someone that uses a revolving Sony Master-Slave setup between
> China and the US. During the US working hour the US server is the
> master, during the working hours of China it becomes the Master.
>
> Of course the person how constructed this system mentioned it was
> woefully challenging. In his case, not only was the slony
> configuration difficult but also finding and keeping stable
> communication path-ways between China and the US.
>

--
Sent via pgsql-general mailing list (pgsql-general@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

Re: Justifying a PG over MySQL approach to a project

2009-12-17T08:57:40Z

-----BEGIN PGP SIGNED MESSAGE-----
Hash: RIPEMD160

> How difficult is it to switch the master's hat from one DB instance
> to another? Let's say the master in a master-slave scenario goes
> down but the slave is fine. Can I designate the slave as being the
> new master, use it for read/write, and then just call the broken
> master the new slave once it comes back to life (something like that)?

Sure. Bucardo slaves are not changed at all, so they are already
read/write and don't need anything special done to "unslave" them.

One possible way to handle the scenario is:

Assuming three servers:
* A (master) sends changes to B, receives read/write queries
* B (slave) has transaction_read_only set to true, receives read queries
* C has the Bucardo database and daemon

Box A goes down suddenly.

* Stop Bucardo on box C
* Flip the boxes around in the bucardo.db table
* Do a 'bucardo_ctl validate sync all'
(This will create the needed triggers on B)
* Set B's transaction_read_only to false
* Point your apps at B instead of A for read/write queries

When A comes back up:

* DROP SCHEMA bucardo CASCADE; (drops all triggers)
* Set transaction_read_only to true
* Start Bucardo on C
* Once caught up, point read-only queries to A

If you are in a rush, you point things to B immediately after A fails,
but you'll have to recopy the entire table data to the slave, as the
triggers won't be in place yet.

- --
Greg Sabino Mullane greg@...
End Point Corporation http://www.endpoint.com/
PGP Key: 0x14964AC8 200912171153
http://biglumber.com/x/web?pk=2529DF6AB8F79407E94445B4BC9B906714964AC8
-----BEGIN PGP SIGNATURE-----

iEYEAREDAAYFAksqYqUACgkQvJuQZxSWSsjZtQCfTwbI3f9W0z+82IU7lL+2LwNK
aUYAnj3AMjRDOeFIuHDee4JJemneArie
=75Ho
-----END PGP SIGNATURE-----

--
Sent via pgsql-general mailing list (pgsql-general@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

Re: Justifying a PG over MySQL approach to a project

2009-12-17T08:56:03Z

"Gauthier, Dave" <dave.gauthier@...> writes:

> I am more concerned with getting a robust DB replication system up and
> running. Bucardo looks pretty good, but I've just started looking at
> the options. Any suggestions?

Master Slave replication? Meaning no writes on the "sister site".

If yes, consider Londiste from Skytools. Easy to setup and maintain, and
robust.
http://wiki.postgresql.org/wiki/Skytools
http://wiki.postgresql.org/wiki/Londiste_Tutorial

Regards,
--
dim

--
Sent via pgsql-general mailing list (pgsql-general@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

Re: Justifying a PG over MySQL approach to a project

2009-12-17T08:51:19Z

On Thu, Dec 17, 2009 at 8:23 AM, Gauthier, Dave <dave.gauthier@...> wrote:
> How difficult is it to switch the master's hat from one DB instance to another? Let's say the master in a master-slave scenario goes down but the slave is fine. Can I designate the slave as being the new master, use it for read/write, and then just call the broken master the new slave once it comes back to life (something like that)?

I know someone that uses a revolving Sony Master-Slave setup between
China and the US. During the US working hour the US server is the
master, during the working hours of China it becomes the Master.

Of course the person how constructed this system mentioned it was
woefully challenging. In his case, not only was the slony
configuration difficult but also finding and keeping stable
communication path-ways between China and the US.

--
Regards,
Richard Broersma Jr.

Visit the Los Angeles PostgreSQL Users Group (LAPUG)
http://pugs.postgresql.org/lapug

--
Sent via pgsql-general mailing list (pgsql-general@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

Re: Justifying a PG over MySQL approach to a project

2009-12-17T08:23:48Z

How difficult is it to switch the master's hat from one DB instance to another? Let's say the master in a master-slave scenario goes down but the slave is fine. Can I designate the slave as being the new master, use it for read/write, and then just call the broken master the new slave once it comes back to life (something like that)?

-----Original Message-----
From: pgsql-general-owner@... [mailto:pgsql-general-owner@...] On Behalf Of Greg Sabino Mullane
Sent: Thursday, December 17, 2009 9:28 AM
To: pgsql-general@...
Subject: Re: [GENERAL] Justifying a PG over MySQL approach to a project

-----BEGIN PGP SIGNED MESSAGE-----
Hash: RIPEMD160

> - MySQL is horizontally scalable via clustering and multi-master
> replication (though you must beware of numerous gotchas). PostgreSQL can
> be used with read-only slaves via Slony/Bucardo/etc replication, but is
> limited to a single authoriative master.
>
> (There's work ongoing to enable readonly hot standby slaves with
> failover, but no multi-master is on the horizion).

Well that's refreshing: usually Bucardo is mistaken for a system that
only does master-master and not master-slave, rather than vice-versa. :)
You can have two authoritative masters with Bucardo, in addition to
any number of slaves radiating from one or both of those (as well as
just simple master->slaves).

> - It's a cool tool when you want to query and integrate data from all
> sorts of disparate sources, thanks to its support for pluggable storage
> engines. If you want something for data analysis and integration rather
> than safe storage it's well worth looking at.

What sort of sources? I'm curious here to find areas we can improve upon.

- --
Greg Sabino Mullane greg@...
End Point Corporation http://www.endpoint.com/
PGP Key: 0x14964AC8 200912170927
http://biglumber.com/x/web?pk=2529DF6AB8F79407E94445B4BC9B906714964AC8
-----BEGIN PGP SIGNATURE-----

iEYEAREDAAYFAksqP9kACgkQvJuQZxSWSshbUQCg3CfvpeivDi6gg2bkr74I17Qe
RKAAnRu3GTUQ3Bg3R2Fq3eOsgK4N0xd1
=5r9R
-----END PGP SIGNATURE-----

--
Sent via pgsql-general mailing list (pgsql-general@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

--
Sent via pgsql-general mailing list (pgsql-general@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

Re: Justifying a PG over MySQL approach to a project

2009-12-17T08:19:05Z

Actually, the DB I'm working on is rather small but has a somewhat complex system of constraints and triggers that maintain the data. Queries will outnumber writes (20x at least). And the DB has to be mirrorred at a sister site a couple thousand miles away, so I'm looking for a robust DB replication system for that.

These are the key points they will be worried about...
- DB up time (most important), including recovery time after disasters (e.g. power outages)
- Data integrity. I'm addressing this with constraints and using triggers to populate columns with derived data.
- Data Quality. NO CORRUPT TABLES / INDEXES
- Retrofitting existing apps to work with PG. Perl/DBI is a subtle change in the DBD designation. Some Tcl-MySQL code is tougher. I'm proposing changing everything to go through ODBC as a standard now, and for the future.
- Cost of maintainence. Do I have to babysit this DB 4 hours every day, or does it run by itself? Is this like Oracle where we have to hire professional 24x7 DBAs, or is this hands-off? That kind of question.

I have a DB up and working. Runs great, no problems, but very lightly loaded and/or used at this time. Having worked with PG in the past, I'm not worried about this piece.

I am more concerned with getting a robust DB replication system up and running. Bucardo looks pretty good, but I've just started looking at the options. Any suggestions?

Thanks!

-----Original Message-----
From: Erik Jones [mailto:ejones@...]
Sent: Thursday, December 17, 2009 4:42 AM
To: Craig Ringer
Cc: Gauthier, Dave; pgsql-general@...
Subject: Re: [GENERAL] Justifying a PG over MySQL approach to a project

On Dec 16, 2009, at 10:30 PM, Craig Ringer wrote:

> - If you don't care about your data, MySQL used with MyISAM is *crazy* fast for lots of small simple queries.

This one causes me no end of grief as too often it's simply touted as "MyISAM is fast(er)" while leaving of the bit about "for lots of small, simple queries". Developers then pick MySQL with MyISAM storage and then scratch their heads saying, "But! I heard it was faster...," when I tell them the reason their app is crawling is because they have even moderately complex reads or writes starving out the rest of their app thanks to the table locks required by MyISAM. As you mentioned, for the type of active workloads that MyISAM is good for, you might as well just use memcache over something more reliable and/or concurrent, or even a simple key-value or document store if you really don't need transactions.

Erik Jones, Database Administrator
Engine Yard
Support, Scalability, Reliability
866.518.9273 x 260
Location: US/Pacific
IRC: mage2k

--
Sent via pgsql-general mailing list (pgsql-general@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general