Nabble - PostgreSQL - sql

Re: Re: constants in 2-column foreign keys or how to design a storage for text-groups ?

2009-12-11T02:20:47Z

Jasen Betts schrieb:

> On 2009-12-09, Andreas <maps.on@...> wrote:
> ...
>
>> stupid example:
>> ---------------
>> color: red, green, blue
>> size: tiny, little, big, giant
>> structure: hard, soft, floppy
>> How would I solve the rather common text storage issue?
>>
>
> have you considered using enumerated types instead?

Yes, but I need those texts in the GUI to show them as listboxes or
comboboxes.
There might be changes too when I later need to add or drop an option of
a group.

E.g. there are questionnaires to model.
Lets say 10 questions where each has a couple of predefined answers
where one should be selected.
So I've got to store every group of possible answers to a question
either in a seperate table or in a kind of repository all within one big
table (row_id, questionnare_id, question_id, answer_nr, answer)
The row_id so I just need to store 1 value per answer.

Now I've got to make sure that it is impossible that accidentally there
gets a question-17 answer connected to a question-42 and above all, that
the questions dont get mixed between the questionnaires.

I can do this with the frontend and enough hope that nothing bad will
happen.
Still I'd rather hardwire the integrity into the table design.

--
Sent via pgsql-sql mailing list (pgsql-sql@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-sql

Re: FIND_IN_SET

2009-12-11T00:48:33Z

2009/12/11 Michael Eshom <oldiesmann@...>:

> I am on the marketing team for a popular forum system, and am also the
> primary PostgreSQL tester/bugfixer. Currently our forum system treats
> MySQL's FIND_IN_SET() as a boolean (eg whether or not the specified value
> was found in the given set), which is fine since MySQL will treat any
> integer greater than 0 as boolean true and 0 as boolean false. I have
> already managed to write a FIND_IN_SET() function for Postgres that behaves
> as boolean. However, we would like to be able to use the true functionality
> of this function (so it will return an integer instead of boolean).
>
> This is where I've run into a problem. The mysqlcompat package has a
> FIND_IN_SET() in it, but it requires plpgsql, and I'd rather not require
> something that a regular user can't install themselves, regardless of how
> simple it is for the host to add it.
>
> I did find another version of FIND_IN_SET() on a blog with several other
> MySQL-compatible functions, and while it uses regular SQL, it requires the
> generate_subscripts() function which isn't available in Postgres 8.1 - the
> latest version officially supported by CentOS.
>
> Is there a way to do this without requiring plpgsql or generate_subscripts?

Hello

you can define own generate_subscripts function

CREATE OR REPLACE FUNCTION find_in_set(str text, strlist text)
RETURNS int AS $$
SELECT i
FROM generate_series(string_to_array($2,','),1) g(i)
WHERE (string_to_array($2, ','))[i] = $1
UNION ALL
SELECT 0
LIMIT 1
$$ LANGUAGE sql STRICT;

CREATE OR REPLACE generate_subscripts(anyarray, int)
RETURNS SETOF int AS $$
SELECT generate_series(array_lower($1,$2), array_upper($1,$2))
$$ LANGUAGE sql;

Regards
Pavel Stehule

> --
> Michael "Oldiesmann" Eshom
> Christian Oldies Fan
> Cincinnati, Ohio

--
Sent via pgsql-sql mailing list (pgsql-sql@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-sql

Re: constants in 2-column foreign keys or how to design a storage for text-groups ?

2009-12-11T00:39:47Z

On 2009-12-09, Andreas <maps.on@...> wrote:
> Hi,

...
> stupid example:
> ---------------
> color: red, green, blue
> size: tiny, little, big, giant
> structure: hard, soft, floppy
>

> How would I solve the rather common text storage issue?

have you considered using enumerated types instead?

--
Sent via pgsql-sql mailing list (pgsql-sql@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-sql

FIND_IN_SET

2009-12-10T22:22:02Z

I am on the marketing team for a popular forum system, and am also the primary PostgreSQL tester/bugfixer. Currently our forum system treats MySQL's FIND_IN_SET() as a boolean (eg whether or not the specified value was found in the given set), which is fine since MySQL will treat any integer greater than 0 as boolean true and 0 as boolean false. I have already managed to write a FIND_IN_SET() function for Postgres that behaves as boolean. However, we would like to be able to use the true functionality of this function (so it will return an integer instead of boolean).

This is where I've run into a problem. The mysqlcompat package has a FIND_IN_SET() in it, but it requires plpgsql, and I'd rather not require something that a regular user can't install themselves, regardless of how simple it is for the host to add it.

I did find another version of FIND_IN_SET() on a blog with several other MySQL-compatible functions, and while it uses regular SQL, it requires the generate_subscripts() function which isn't available in Postgres 8.1 - the latest version officially supported by CentOS.

Is there a way to do this without requiring plpgsql or generate_subscripts?

--
Michael "Oldiesmann" Eshom
Christian Oldies Fan
Cincinnati, Ohio

Re: Window function trouble

2009-12-10T15:08:07Z

Harald Fuchs <hari.fuchs@...> writes:
> That being said, I still think that PostgreSQL could do better - how
> about naming expression columns so that they are distinct from column
> names?

Even though the rules we use are pretty arbitrary, I'm hesitant to make
changes in them; it seems more likely to break existing applications
than to do anyone any good.

regards, tom lane

--
Sent via pgsql-sql mailing list (pgsql-sql@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-sql

Re: Window function trouble

2009-12-10T13:02:00Z

In article <25983.1260468559@...>,
Tom Lane <tgl@...> writes:

> The SQL standard says the default name for any output column other
> than a simple column reference is implementation-dependent. I think
> our implementation involves looking at the default value for a CASE.

Thanks for the clarification, Tom - so it's not a PostgreSQL bug, but
a misfeature of the SQL standard.

That being said, I still think that PostgreSQL could do better - how
about naming expression columns so that they are distinct from column
names? The current implementation throws an error e.g. for

SELECT adr,
CASE lag(adr) OVER (ORDER BY adr)
WHEN adr THEN NULL
ELSE adr
END,
usr, count(*)
FROM tbl
WHERE host = 'h1'
GROUP BY adr, usr
ORDER BY adr, usr

namely `ORDER BY "adr" is ambiguous' which I find somewhat confusing.

--
Sent via pgsql-sql mailing list (pgsql-sql@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-sql

Re: Window function trouble

2009-12-10T10:09:19Z

Harald Fuchs <hari.fuchs@...> writes:
> Apparently the CASE expression is named "adr" unless named
> otherwise, and the result set is sorted by this expression instead of
> the column named "adr". Does anyone know what the SQL standard says
> about that?

The SQL standard says the default name for any output column other
than a simple column reference is implementation-dependent. I think
our implementation involves looking at the default value for a CASE.

regards, tom lane

--
Sent via pgsql-sql mailing list (pgsql-sql@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-sql

Re: Window function trouble

2009-12-10T09:49:57Z

In article <pud42mahnr.fsf@...>,
Harald Fuchs <hari.fuchs@...> writes:

> The other users for one address are gone. Does anyone know why?

Update: they are not gone, but they've moved to the end of the result
set. Apparently the CASE expression is named "adr" unless named
otherwise, and the result set is sorted by this expression instead of
the column named "adr". Does anyone know what the SQL standard says
about that?

--
Sent via pgsql-sql mailing list (pgsql-sql@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-sql

Window function trouble

2009-12-10T09:00:08Z

I have a table like this:

CREATE TABLE tbl (
host text NOT NULL,
adr ip4 NOT NULL,
usr text NOT NULL
);

(ip4 is from the ip4r contrib module)
and I want the number of entries per address and per user:

SELECT adr, usr, count(*)
FROM tbl
WHERE host = ?
AND adr <<= ?
GROUP BY adr, usr
ORDER BY adr, usr

That's pretty basic stuff and returns something like this:

adr1 usr1_1 cnt1_1
adr1 usr1_2 cnt1_2
adr1 usr1_3 cnt1_3
adr2 usr2_1 cnt2_1
...

But I want the address to be NULL if it's the same as the address of
the previous row. I came up with this:

SELECT CASE lag(adr) OVER (ORDER BY adr)
WHEN adr THEN NULL
ELSE adr
END AS myaddr,
usr, count(*)
FROM tbl
WHERE host = ?
AND adr <<= ?
GROUP BY adr, usr
ORDER BY adr, usr

This returns something like

adr1 usr1_1 cnt1_1
NULL usr1_2 cnt1_2
NULL usr1_3 cnt1_3
adr2 usr2_1 cnt2_1
...

what's exactly what I want. But when I don't name the CASE expression
(i.e. I delete "AS myaddr"), I get the following:

adr1 usr1_1 cnt1_1
adr2 usr2_1 cnt2_1
...

The other users for one address are gone. Does anyone know why?

--
Sent via pgsql-sql mailing list (pgsql-sql@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-sql

Re: SQL state 58P01 triggered by a database script execution

2009-12-09T05:46:23Z

In response to aymen marouani :

> Thanks for the help,
>
> In deed I found those lines in my script
>
> "--
> -- TOC entry 25 (class 1255 OID 16409)
> -- Dependencies: 6
> -- Name: bt_metap(text); Type: FUNCTION; Schema: public; Owner: postgres
> --
>
> CREATE FUNCTION bt_metap(relname text, OUT magic integer, OUT version integer,
> OUT root integer, OUT level integer, OUT fastroot integer, OUT fastlevel
> integer) RETURNS record
> AS '$libdir/pageinspect', 'bt_metap'
> LANGUAGE c STRICT;
> "
>
> Can I comment those lines to remove the error ?

Sure. But that don't solve your problem if you call that function
later...

Regards, Andreas
--
Andreas Kretschmer
Kontakt: Heynitz: 035242/47150, D1: 0160/7141639 (mehr: -> Header)
GnuPG: 0x31720C99, 1006 CCB4 A326 1D42 6431 2EB0 389D 1DC2 3172 0C99

--
Sent via pgsql-sql mailing list (pgsql-sql@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-sql

Re: SQL state 58P01 triggered by a database script execution

2009-12-09T05:33:47Z

Thanks for the help,

In deed I found those lines in my script

"--

-- TOC entry 25 (class 1255 OID 16409)

-- Dependencies: 6

-- Name: bt_metap(text); Type: FUNCTION; Schema: public; Owner: postgres

CREATE FUNCTION bt_metap(relname text, OUT magic integer, OUT version integer, OUT root integer, OUT level integer, OUT fastroot integer, OUT fastlevel integer) RETURNS record

AS '$libdir/pageinspect', 'bt_metap'

LANGUAGE c STRICT;

Can I comment those lines to remove the error ?

2009/12/9 A. Kretschmer <andreas.kretschmer@...>

In response to aymen marouani :

> Hi,
>
> I have a database under Postgres v8.3 and I generated its backups script using
> PGAdmin III.
> I executed the same script under Postgres v8.2 in order to create the same
> database but I get the following error
>
> " ERROR: could not access file "$libdir/pageinspect": No such file or directory
> SQL state :58P01"
>

Pageinspect is a contrib-modul, so i think, you should install that. But
i don't know if this available for 8.2.

Andreas
--
Andreas Kretschmer
Kontakt: Heynitz: 035242/47150, D1: 0160/7141639 (mehr: -> Header)
GnuPG: 0x31720C99, 1006 CCB4 A326 1D42 6431 2EB0 389D 1DC2 3172 0C99

--
Sent via pgsql-sql mailing list (pgsql-sql@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-sql

Re: SQL state 58P01 triggered by a database script execution

2009-12-09T05:24:54Z

In response to aymen marouani :

> Hi,
>
> I have a database under Postgres v8.3 and I generated its backups script using
> PGAdmin III.
> I executed the same script under Postgres v8.2 in order to create the same
> database but I get the following error
>
> " ERROR: could not access file "$libdir/pageinspect": No such file or directory
> SQL state :58P01"
>

Pageinspect is a contrib-modul, so i think, you should install that. But
i don't know if this available for 8.2.

Andreas
--
Andreas Kretschmer
Kontakt: Heynitz: 035242/47150, D1: 0160/7141639 (mehr: -> Header)
GnuPG: 0x31720C99, 1006 CCB4 A326 1D42 6431 2EB0 389D 1DC2 3172 0C99

--
Sent via pgsql-sql mailing list (pgsql-sql@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-sql

SQL state 58P01 triggered by a database script execution

2009-12-09T05:10:36Z

Hi,

I have a database under Postgres v8.3 and I generated its backups script using PGAdmin III.

I executed the same script under Postgres v8.2 in order to create the same database but I get the following error

" ERROR: could not access file "$libdir/pageinspect": No such file or directory

SQL state :58P01"

Is this error related to the difference of versions ?

Thanks in advance.

Re: Stalled post to pgsql-sql

2009-12-09T02:33:54Z

hello
postgres=# create table foo(a timestamp(0));
CREATE TABLE
Time: 111,105 ms
postgres=# insert into foo values(now());
INSERT 0 1
Time: 1,292 ms
postgres=# select now(), a from foo;
now | a
-------------------------------+---------------------
2009-12-09 11:33:22.746217+01 | 2009-12-09 11:33:12
(1 row)

Time: 21,010 ms

regards
Pavel Stehule

2009/12/9 sergey kapustin <kapustin.sergey@...>:
> Hello,
> I need a type that will store date and time, but without seconds and
> microseconds (like timestamp does). At least without microseconds.
> I also need all the operators to work.
> Is this possible with one of existing date/time types or i need to create my
> own?
>
> Thank you!
>

--
Sent via pgsql-sql mailing list (pgsql-sql@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-sql

Re: Stalled post to pgsql-sql

2009-12-09T02:30:44Z

Hello,
I need a type that will store date and time, but without seconds and microseconds (like timestamp does). At least without microseconds.
I also need all the operators to work.
Is this possible with one of existing date/time types or i need to create my own?

Thank you!

constants in 2-column foreign keys or how to design a storage for text-groups ?

2009-12-08T16:20:26Z

Hi,
my frontend has a lot of combo- and listboxes where one can chose a
textsnippet that represents a key-number which is stored in several
tables as foreign-key attributes.
Those textsnippets are usually semantically grouped in 2-10 strings that
belong together somehow.

stupid example:
---------------
color: red, green, blue
size: tiny, little, big, giant
structure: hard, soft, floppy

now I'd like to build tables like
thing( color_fk foreign key to color, size_fk foreign key to size,
structure_fk foreign key to structure, sometext, atimestamp ...)
so far no problems.

With time those little text-list-tables clutter up the database so I'm
thinking about one big text-storage that has the groups represented by a
number like:
snippets (snippet_id, snippet_group_nr, snippet)
(100, 1, red), (101, 1, green), (102, 1, blue), (200, 2, tiny), (201, 2,
little), ...

Simple foreign-keys still work nicely but they cant prohibit that I
store id-values from wrong groups.
Here color_fk would only be correct if the id is out of group 1. The
foreign key doesnt catch it if I put a group-3-id into color_fk.

Id be cool to be able to have constants in 2-column foreign keys like
color_fk integer not null default 0
FOREIGN KEY (color_fk, 1 ) REFERENCES snippets (snippet_id,
snippet_group_nr)

This throws an error. So this approach might be not advisable.
I could add an additional column for every foreign-key that stores
constant group-ids then I can have 2-column-fk but this looks bloated
since those extra columns would hold eternally the same number in every row.

How would I solve the rather common text storage issue?

--
Sent via pgsql-sql mailing list (pgsql-sql@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-sql

Re: Query Performance

2009-12-07T23:55:09Z

Tomas: Thanks for your replay.

I got it working within 3 minutes because of hash join by rearranging the query.

explain analyze select *, subs from (select user_name,A.user_id, dnd_window_start, dnd_window_stop from users A ,subs_feed B where A.user_id=B.user_id and b.feed_id=1413 and f_sms='t') as foo left outer join user_subscriptions u on (foo.user_id=u.user_id);

Hash Left Join (cost=1472105.16..2775728.42 rows=957624 width=75) (actual time=86322.190..192125.402 rows=1340957 loops=1)
   Hash Cond: (a.user_id = u.user_id)
   -> Hash Join (cost=1347862.70..2638877.52 rows=957624 width=26) (actual time=79407.144..181327.848 rows=1340957 loops=1)
         Hash Cond: (b.user_id = a.user_id)
         -> Bitmap Heap Scan on subs_feed b (cost=34518.45..1127550.37 rows=957624 width=4) (actual time=8791.019..72889.040 rows=1340957 loops=1)
               Recheck Cond: (feed_id = 1413)
               Filter: f_sms
               -> Bitmap Index Scan on feed_user_id (cost=0.00..34279.04 rows=1442930 width=0) (actual time=8492.067..8492.067 rows=1352180 loops=1)
                     Index Cond: (feed_id = 1413)
         -> Hash (cost=916273.00..916273.00 rows=19548100 width=26) (actual time=70605.366..70605.366 rows=19548560 loops=1)
               -> Seq Scan on users a (cost=0.00..916273.00 rows=19548100 width=26) (actual time=10.405..55676.884 rows=19548560 loops=1)
   -> Hash (cost=77415.54..77415.54 rows=3746154 width=49) (actual time=6909.632..6909.632 rows=3758304 loops=1)
         -> Seq Scan on user_subscriptions u (cost=0.00..77415.54 rows=3746154 width=49) (actual time=3.624..4265.114 rows=3758304 loops=1)
Total runtime: 192585.437 ms

From: "tv@..." <tv@...>
To: Postgre Novice <postgrenovice@...>
Cc: pgsql-sql@...
Sent: Mon, December 7, 2009 7:23:19 PM
Subject: Re: [SQL] Query Performance

Yes, the problem is the nested loop scan - it's scanning users 609070
times, which is awful.

Could you provide explain plan that executed fast? Was it executed with
the same parameter values or did the parameters change (maybe it's slow
for some parameters values only)?

Have you tried to rewrite the subselect to a join? I.e. something like this

select user_name,A.user_id, dnd_window_start, dnd_window_stop, B.subs as
subs, B.city_id as city_id, B.source_type as source_type from
users A left join user_subscriptions B on (A.user_id=B.user_id)
join subs_feed C ON (A.user_id = C.user_id)
where feed_id=1411 and f_sms='t'

But I guess it won't solve the issue (it seems PostgreSQL did this rewrite
on it's own).

Tomas

> Hello List,
>
> I have a query which use to run very fast now has turn into show stopper .
>
> PostgreSQL:8.2
>
> explain analyze select user_name,A.user_id, dnd_window_start,
> dnd_window_stop, B.subs as subs, B.city_id as city_id, B.source_type as
> source_type from
> users A left join user_subscriptions B on (A.user_id=B.user_id)
> where A.user_id in (select user_id from subs_feed where feed_id=1411 and
> f_sms='t')
> ;
>
> Nested Loop Left Join (cost=986325.88..1094601.47 rows=11148
> width=55) (actual time=132635.994..1590487.280 rows=609070
> loops=1)
> -> Nested Loop (cost=986325.88..1062280.53 rows=11148 width=26)
> (actual time=132630.057..1398299.117 rows=609070 loops=1)
> -> HashAggregate (cost=986325.88..986437.36 rows=11148 width=4)
> (actual time=132591.648..133386.651 rows=609070 loops=1)
> -> Bitmap Heap Scan on subs_feed
> (cost=16316.71..985194.44 rows=452576 width=4) (actual
> time=20199.571..131566.494 rows=609070 loops=1)
> Recheck Cond: (feed_id = 1411)
> Filter: f_sms
> -> Bitmap Index Scan on feed_user_id
> (cost=0.00..16203.57 rows=681933 width=0) (actual
> time=19919.512..19919.512 rows=616900 loops=1)
> Index Cond: (feed_id = 1411)
> -> Index Scan using users_pkey on users a (cost=0.00..6.79
> rows=1 width=26) (actual time=2.073..2.074 rows=1 loops=609070)
> Index Cond: (a.user_id = subs_feed.user_id)
> -> Index Scan using user_subscriptions_user_id_pk on
> user_subscriptions b (cost=0.00..2.89 rows=1 width=33) (actual
> time=0.312..0.313 rows=1 loops=609070)
> Index Cond: (a.user_id = b.user_id)
> Total runtime: 1590755.918 ms
> (13 rows)
>
>
>
> This query runs almost half an hour. It is evident that nested loop is
> taking most of the time (approx 27 minutes).
>
> Any tips would be very useful.
>
> Also these table have below count:
>
> select relname,reltuples from pg_class where relname in
> ('users','user_subscriptions','subs_feed');
> relname | reltuples
> --------------------+-------------
> user_subscriptions | 3758304
> users | 1.95481e+07
> subs_feed | 2.96492e+07
>
>
> select n_tup_ins,n_tup_upd,n_tup_del,last_vacuum,last_analyze from
> pg_stat_user_tables where relname='user_subscriptions';
> n_tup_ins | n_tup_upd | n_tup_del | last_vacuum |
> last_analyze
> -----------+-----------+-----------+----------------------------------+----------------------------------
> 86371397 | 25865942 | 0 | 2009-12-06 23:00:36.355251+05:30 |
> 2009-12-06 23:00:36.355251+05:30
>
>
>
> Thanks in advance for help ...
>
>
>

--
Sent via pgsql-sql mailing list (pgsql-sql@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-sql

Re: problems with changing the case of turkish characters

2009-12-07T13:28:03Z

to clarify. just trying to ensure I understand what PG 8.4 and 8.3 provide where you may have data from alternate languages where ideally you'd like them in the same table in the same database etc.

create table T ( c1 char( ...), c2 char (...) ... ) where c1 may contain thai, c2 korean, c3 turkish etc names

create table TKO (c1 char(...)), create table TTH (c1 char (...))

different databases etc

On Mon, Dec 7, 2009 at 4:20 PM, the6campbells <the6campbells@...> wrote:

so where would I define something akin to what I can do in DB2 LUW where collate using system means to sort by the codeset. ie. without english, united states in LC_COLLATE.

USING CODESET UTF-8 TERRITORY US COLLATE USING SYSTEM

On Mon, Dec 7, 2009 at 12:28 PM, Tom Lane <tgl@...> wrote:

the6campbells <the6campbells@...> writes:
> Just want to clarify if there is something I've overlooked or if this is a
> known issue in PG 8.4 and 8.3

> CREATE DATABASE test
> WITH OWNER = postgres
> ENCODING = 'UTF8'
> LC_COLLATE = 'English, United States, UTF-8'
> LC_CTYPE = 'English, United States, UTF-8'

Not sure why you'd be expecting an English locale to follow Turkish
case-changing rules.

regards, tom lane

Re: problems with changing the case of turkish characters

2009-12-07T13:20:26Z

so where would I define something akin to what I can do in DB2 LUW where collate using system means to sort by the codeset. ie. without english, united states in LC_COLLATE.

USING CODESET UTF-8 TERRITORY US COLLATE USING SYSTEM

On Mon, Dec 7, 2009 at 12:28 PM, Tom Lane <tgl@...> wrote:

the6campbells <the6campbells@...> writes:
> Just want to clarify if there is something I've overlooked or if this is a
> known issue in PG 8.4 and 8.3

> CREATE DATABASE test
> WITH OWNER = postgres
> ENCODING = 'UTF8'
> LC_COLLATE = 'English, United States, UTF-8'
> LC_CTYPE = 'English, United States, UTF-8'

Not sure why you'd be expecting an English locale to follow Turkish
case-changing rules.

regards, tom lane

Re: problems with changing the case of turkish characters

2009-12-07T09:28:08Z

the6campbells <the6campbells@...> writes:
> Just want to clarify if there is something I've overlooked or if this is a
> known issue in PG 8.4 and 8.3

> CREATE DATABASE test
> WITH OWNER = postgres
> ENCODING = 'UTF8'
> LC_COLLATE = 'English, United States, UTF-8'
> LC_CTYPE = 'English, United States, UTF-8'

Not sure why you'd be expecting an English locale to follow Turkish
case-changing rules.

regards, tom lane

--
Sent via pgsql-sql mailing list (pgsql-sql@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-sql

problems with changing the case of turkish characters

2009-12-07T09:12:51Z

Just want to clarify if there is something I've overlooked or if this is a known issue in PG 8.4 and 8.3

CREATE DATABASE test
WITH OWNER = postgres
       ENCODING = 'UTF8'
       LC_COLLATE = 'English, United States, UTF-8'
       LC_CTYPE = 'English, United States, UTF-8'

select ('İsteği') from tversion
"İsteği" was expecting "isteği"

select upper('iışğüçö') from tversion

"IıŞĞÜÇÖ"

select lower ('İIŞĞÜÇÖ') from tversion

"İişğüçö"

Re: Query Performance

2009-12-07T05:53:19Z

Yes, the problem is the nested loop scan - it's scanning users 609070
times, which is awful.

Could you provide explain plan that executed fast? Was it executed with
the same parameter values or did the parameters change (maybe it's slow
for some parameters values only)?

Have you tried to rewrite the subselect to a join? I.e. something like this

select user_name,A.user_id, dnd_window_start, dnd_window_stop, B.subs as
subs, B.city_id as city_id, B.source_type as source_type from
users A left join user_subscriptions B on (A.user_id=B.user_id)
join subs_feed C ON (A.user_id = C.user_id)
where feed_id=1411 and f_sms='t'

But I guess it won't solve the issue (it seems PostgreSQL did this rewrite
on it's own).

Tomas

--
Sent via pgsql-sql mailing list (pgsql-sql@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-sql

Query Performance

2009-12-07T05:00:58Z

Hello List,

I have a query which use to run very fast now has turn into show stopper .

PostgreSQL:8.2

explain analyze select user_name,A.user_id, dnd_window_start, dnd_window_stop, B.subs as subs, B.city_id as city_id, B.source_type as source_type from
users A left join user_subscriptions B on (A.user_id=B.user_id)
where A.user_id in (select user_id from subs_feed where feed_id=1411 and f_sms='t')
;

           Nested Loop Left Join (cost=986325.88..1094601.47 rows=11148 width=55) (actual time=132635.994..1590487.280 rows=609070 loops=1)
   -> Nested Loop (cost=986325.88..1062280.53 rows=11148 width=26) (actual time=132630.057..1398299.117 rows=609070 loops=1)
         -> HashAggregate (cost=986325.88..986437.36 rows=11148 width=4) (actual time=132591.648..133386.651 rows=609070 loops=1)
               -> Bitmap Heap Scan on subs_feed (cost=16316.71..985194.44 rows=452576 width=4) (actual time=20199.571..131566.494 rows=609070 loops=1)
                     Recheck Cond: (feed_id = 1411)
                     Filter: f_sms
                     -> Bitmap Index Scan on feed_user_id (cost=0.00..16203.57 rows=681933 width=0) (actual time=19919.512..19919.512 rows=616900 loops=1)
                           Index Cond: (feed_id = 1411)
         -> Index Scan using users_pkey on users a (cost=0.00..6.79 rows=1 width=26) (actual time=2.073..2.074 rows=1 loops=609070)
               Index Cond: (a.user_id = subs_feed.user_id)
   -> Index Scan using user_subscriptions_user_id_pk on user_subscriptions b (cost=0.00..2.89 rows=1 width=33) (actual time=0.312..0.313 rows=1 loops=609070)
         Index Cond: (a.user_id = b.user_id)
Total runtime: 1590755.918 ms
(13 rows)

This query runs almost half an hour. It is evident that nested loop is taking most of the time (approx 27 minutes).

Any tips would be very useful.

Also these table have below count:

select relname,reltuples from pg_class where relname in ('users','user_subscriptions','subs_feed');
      relname       | reltuples
--------------------+-------------
user_subscriptions |          3758304
users              | 1.95481e+07
subs_feed          | 2.96492e+07

select n_tup_ins,n_tup_upd,n_tup_del,last_vacuum,last_analyze from pg_stat_user_tables where relname='user_subscriptions';
n_tup_ins | n_tup_upd | n_tup_del |           last_vacuum            |           last_analyze
-----------+-----------+-----------+----------------------------------+----------------------------------
86371397 | 25865942 |         0 | 2009-12-06 23:00:36.355251+05:30 | 2009-12-06 23:00:36.355251+05:30

Thanks in advance for help ...

Re: Getting more than one row in UNIQUE fields

2009-12-06T00:55:43Z

On 2009-12-04, Another Trad <anothertrad@...> wrote:
> --001485f94df095921c0479ea62fd
> Content-Type: text/plain; charset=ISO-8859-1
>
> My table 'client' has the unique 'client_id'.
> My test server, when I try:
> select * from client where client_id = 12
> My server returns 3 rows!!!!

I had similar with a client on friday, windows event log shouwed disk
errors, chkdsk showed damaged database files.

I passed the problem back to their IT guy.

--
Sent via pgsql-sql mailing list (pgsql-sql@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-sql

Re: get sequence name from table name

2009-12-04T11:24:34Z

Uwe Maiwald wrote:
> how to get the name of the sequence that is responsible for setting
> the autoincrement value of a tables primary key column?

You can use the pg_get_serial_sequence() function. You need the name of
the column in addition to the table name though.

--
Alvaro Herrera http://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.

--
Sent via pgsql-sql mailing list (pgsql-sql@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-sql

Re: get sequence name from table name

2009-12-04T10:02:37Z

Uwe Maiwald wrote:

> how to get the name of the sequence that is responsible for setting the
> autoincrement value of a tables primary key column?
>
> i only have the name of the table and need to have an appropiate sql
> statement.
>
>
> i need this to write a program that loops through all tables of a
> database and then ajusts the start values of the sequencees in case the
> table has an automatic id value (serial/bigserial)
>
>
> thanks,
> Uwe
>

The information schema provides what you need.

test=# create table testing (id serial);
NOTICE: CREATE TABLE will create implicit sequence "testing_id_seq" for
serial column "testing.id"

test=# SELECT table_name, column_name, column_default from
information_schema.columns where table_name='testing';
table_name | column_name | column_default
------------+-------------+-------------------------------------
testing | id | nextval('testing_id_seq'::regclass)
(1 row)

You may need an extra work finding out which the primary keys are, look
at the information schema docs [0]. Maybe you will also need the help of
the system catalogs [1].

[0] http://www.postgresql.org/docs/8.3/static/information-schema.html
[1] http://www.postgresql.org/docs/8.3/static/catalogs.html

(as you can see, this docs are from the 8.3 version. Check yours)

HTH
Gerardo

--
Sent via pgsql-sql mailing list (pgsql-sql@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-sql

Re: Getting more than one row in UNIQUE fields

2009-12-04T09:36:54Z

Another Trad <anothertrad@...> writes:
> My table 'client' has the unique 'client_id'.
> My test server, when I try:
> select * from client where client_id = 12
> My server returns 3 rows!!!!

We've seen bugs of that sort in the past ... what PG version is this,
and on what platform?

regards, tom lane

--
Sent via pgsql-sql mailing list (pgsql-sql@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-sql

Getting more than one row in UNIQUE fields

2009-12-04T09:24:49Z

My table 'client' has the unique 'client_id'.
My test server, when I try:
select * from client where client_id = 12
My server returns 3 rows!!!!
When I try to update this client, his name for example,using my framework (SQLAlchemy), obviously returns the error:
"Updated rowcount 3 does not match number of objects updated 1"
But the field client_id is unique and has a sequence to auto increment.

Anyone ever pass to this?

get sequence name from table name

2009-12-04T08:53:34Z

how to get the name of the sequence that is responsible for setting the
autoincrement value of a tables primary key column?

i only have the name of the table and need to have an appropiate sql
statement.

i need this to write a program that loops through all tables of a
database and then ajusts the start values of the sequencees in case the
table has an automatic id value (serial/bigserial)

thanks,
Uwe

--
Sent via pgsql-sql mailing list (pgsql-sql@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-sql

Re: create user xxx with md5 string?

2009-12-03T11:38:55Z

Sorry for my prev post. I mean md5(password || username).

'test10' is a username on which i checked. :) Sorry!

2009/12/3 Dmitriy Igrishin <dmitigr@...>

Hello all.
Tom right. Encrypted MD5 passwords computed as md5(password || test10). I checked.

Regards, Dmitriy Igrishin.

2009/12/3 Tom Lane <tgl@...>

Gerardo Herzig <gherzig@...> writes:
> Hi all. Im triyng, with no success, to create a user, and set the
> password with a md5 string.

The correct method for computing the md5'd password includes the
username. I think it's 'md5' || md5sum(password || username)
but look at the code or docs.

regards, tom lane

--
Sent via pgsql-sql mailing list (pgsql-sql@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-sql

Re: create user xxx with md5 string?

2009-12-03T11:36:58Z

Hello all.

Tom right. Encrypted MD5 passwords computed as md5(password || test10). I checked.

Regards, Dmitriy Igrishin.

2009/12/3 Tom Lane <tgl@...>

Gerardo Herzig <gherzig@...> writes:
> Hi all. Im triyng, with no success, to create a user, and set the
> password with a md5 string.

The correct method for computing the md5'd password includes the
username. I think it's 'md5' || md5sum(password || username)
but look at the code or docs.

regards, tom lane

--
Sent via pgsql-sql mailing list (pgsql-sql@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-sql

Re: create user xxx with md5 string?

2009-12-03T11:21:19Z

Gerardo Herzig <gherzig@...> writes:
> Hi all. Im triyng, with no success, to create a user, and set the
> password with a md5 string.

The correct method for computing the md5'd password includes the
username. I think it's 'md5' || md5sum(password || username)
but look at the code or docs.

regards, tom lane

--
Sent via pgsql-sql mailing list (pgsql-sql@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-sql

Re: create user xxx with md5 string?

2009-12-03T11:02:58Z

Tom Lane wrote:

> Gerardo Herzig <gherzig@...> writes:
>> Hi all. Im triyng, with no success, to create a user, and set the
>> password with a md5 string.
>
> The correct method for computing the md5'd password includes the
> username. I think it's 'md5' || md5sum(password || username)
> but look at the code or docs.
>
> regards, tom lane
>

Perfect! I could not find that in the docs.
It works. Thanks!

Gerardo

--
Sent via pgsql-sql mailing list (pgsql-sql@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-sql

create user xxx with md5 string?

2009-12-03T10:05:02Z

Hi all. Im triyng, with no success, to create a user, and set the
password with a md5 string.

I did this:

comechingon:~ # echo -n 123 | md5sum
202cb962ac59075b964b07152d234b70 -

Ok, so then
test=# CREATE USER foobar ENCRYPTED PASSWORD
'md5202cb962ac59075b964b07152d234b70'; --I have to add 'md5' at the
begging of the string
CREATE ROLE

test=# SELECT * from pg_shadow where usename='foobar';
usename | usesysid | usecreatedb | usesuper | usecatupd |
passwd | valuntil | useconfig
---------+----------+-------------+----------+-----------+-------------------------------------+----------+-----------
foobar | 3250592 | f | f | f |
md5202cb962ac59075b964b07152d234b70 | |
(1 row)

test=# \c test foobar
Password for user foobar: [123, off course]
FATAL: password authentication failed for user "foobar"
Previous connection kept

test=# SHOW server_version;
server_version
----------------
8.3.6

Im wondering which my mystake is:
Using md5sum?

I do have 'md5' as validation mechanism in pg_hba.conf

Any hints?
Thanks!

Gerardo

--
Sent via pgsql-sql mailing list (pgsql-sql@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-sql

Re: pg_restore followed by immediate VACUUM ANALYZE

2009-11-30T20:04:29Z

Bryce Nesbitt <bryce2@...> writes:
>> Correct. Autovacuum might fix them eventually, but usually it's
>> worth forcing the issue once you've completed your data loading.
>> (This might involve multiple steps, which is why pg_restore
>> doesn't try to force it for you.)</pre>

> Hmm. It seems like pg_restore would already know 90% of what it needs
> to in this case.<br>

But of course pg_restore doesn't know what else you might want to do.

> Of course I'm just trying to cut down on the 4+ hour VACUUM ANALYZE.

You could try just ANALYZE without the VACUUM.

regards, tom lane

--
Sent via pgsql-sql mailing list (pgsql-sql@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-sql