Nabble - PostgreSQL - novice

Question when to use BEGIN / END

2009-11-30T01:32:18Z

I have a case where I want to forbid a race condition. Basically, I have this:

$q = "CREATE VIEW foo AS ( SELECT * FROM foo WHERE [...different
conditions every call ...] )";
@pg_query($db, $q);
$q = "SELECT * FROM foo";
$res = pg_query($db, $q);
$row = pg_fetch_object($res);
... change table foo...
$q = "DROP VIEW foo";
@pg_query($db, $q);

It is of utmost importance that there is not another client doing the
same thing while the above is running. When the DROP VIEW is done,
another client can start its work but not until then.

I'm reading about BEGIN/END on
http://www.postgresql.org/docs/7.4/interactive/sql-begin.html but I'm
not sure this is the solution. What I'm wondering is this: If I
prepend BEGIN on CREATE VIEW above, and append END after DROP VIEW,
can I still access the table foo in between? Or is BEGIN/END only
supposed to be used on queries that do INSERT or UPDATE, i.e. no
data-reading?

--
- Rikard

--
Sent via pgsql-novice mailing list (pgsql-novice@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-novice

Re: bytea and text

2009-11-29T06:30:57Z

Kris Kewley a écrit :

> Some interesting points were made. Depending on the roadmap for you
> application consideration should be given to storing the image files
> outside of the db.
>
> IMO the driving factors here are:
> 1. Scalability
> 2. Version control of images
> 3. Backup
> 4. Performance
>
> Including images in your db will accerlate it's growth. You noted legal
> concerns, version control is typically an important aspect of this. Back
> up and restore time will increase as db size is larger, and will be more
> costly (assume db on more expensive disk than file server). Finally
> performance.. Blobs typically increase disk io on queries (at least they
> do in oracle) so should be used judicially in heavily accessed tables.

These tables are not heavily accessed: they're linked to heavily accessed
other tables.
They only contains an id, a RI, a pic_nb and a bytea.
Pictures are practically never upgraded and jettisoned when obsolete.

> Realize that I have made some wild assumptions about your app here, but
> this is my experience.
>
> Consider a link in your db and using cvs for storing the images, or
> filesystem depending on the extent of to your needs.

NO, I want only one backup solution, not one for each case; and perfs are
only needed for other tables.

> I too stand to be corrected :-)

I see, you're doing BDSM :D

JY
--
Leave no stone unturned.
-- Euripides

--
Sent via pgsql-novice mailing list (pgsql-novice@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-novice

Re: bytea and text

2009-11-29T05:41:58Z

Some interesting points were made. Depending on the roadmap for you
application consideration should be given to storing the image files
outside of the db.

IMO the driving factors here are:
1. Scalability
2. Version control of images
3. Backup
4. Performance

Including images in your db will accerlate it's growth. You noted
legal concerns, version control is typically an important aspect of
this. Back up and restore time will increase as db size is larger, and
will be more costly (assume db on more expensive disk than file
server). Finally performance.. Blobs typically increase disk io on
queries (at least they do in oracle) so should be used judicially in
heavily accessed tables.

Realize that I have made some wild assumptions about your app here,
but this is my experience.

Consider a link in your db and using cvs for storing the images, or
filesystem depending on the extent of to your needs.

I too stand to be corrected :-)
Kris

On 29-Nov-09, at 3:30, richard terry <rterry@...> wrote:

> On Sunday 29 November 2009 17:50:10 Didier Gasser-Morlay wrote:
>> I hope you won't mind if I had my 2 cents to this conversation;
>> aside from
>> the actual format, this question comes up with regularity on various
>> database lists.
>>
>> I have never understood the need for storing images inside a
>> database (and
>> to some extend blob data) because:
>> - being a blog you do not query it just store and retrieve, you
>> rarely
>> update, so the need for a fancy SQL and DB engine is somewhat
>> limited;
>>
>> - storing images is heavy, makes the database grow, thus makes
>> backup/restore that bit more painful and possibly that bit less
>> stable. ..
>>
>> For years I designed system for picture libraries where numbers
>> like 10s of
>> thousands to millions of images are common place, we would never have
>> contemplated to store images on anything more complex than a file
>> system.
>> using the database to store a URL to that resource.
>>
>> Hope this helps
>>
>> Didier
>>
> this is **mission critical** for me (alone perhaps) as the images
> (usually
> very small (eg many could be 15K for a small photo, 3k - 16K for a
> small
> diagram of a body part) are part of some medical record software I'm
> writing
> for myself, and sit in the progress notes when displayed. Just
> alteration of
> a line drawn on a body part for an injured patient could end you up
> in court
> or de-registered.
>
> Medico-legal issues abound.
>
> Yes, one can easily backup files in a directory, but my current
> feeling is its
> easier to do an entire DB dump - could be wrong, stand to be
> corrected.
>
> Regards
>
> Richard
>
> --
> Sent via pgsql-novice mailing list (pgsql-novice@...)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-novice

--
Sent via pgsql-novice mailing list (pgsql-novice@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-novice

Fwd: Re: bytea and text

2009-11-29T03:29:51Z

sorry, just have faulty memory from doing it months ago with both derbydb and
postgresql as
alternative drivers. Derbydb was the one that preferred intermediate files for
bulk loading of
binary stuff as hex I think but there was no delimiters apart from comma for
that , whereas
postgresql allows you to escape every character as a slash followed by 1-3 digit
octal or 1-2 digit hex preceded
by a slash and a x , so that was the point of the question, whether is it better
to be paranoid that a
bulk load is going to fail and escape every character, or can you be optimistic
an just escape
newline carriage return , backslash itself and the current delimiter character ,
but is there an end of file
sequence ? and doing you have to escape the null sequence ? \N if it happens to
occur in your raw binary data.
If a 3 gig bulk load fails because of one character, it's annoying.
I remember doing the same thing as individual pgdb api calls a few years ago,
and having no such problem in python with binary data , or at least you could
fail one image and not have it affect the rest of the rows by having autocommit on.

On Sat 28/11/09 13:19 , Bob McConnell rmcconne@... sent:

> Syan Tan wrote:
> > here's a quote from the postgresql
> documentation>
> > Backslash characters (\) can be used in the COPY
> data to quote data> characters that might otherwise be taken as row
> or column delimiters.> In particular, the following characters must be
> preceded by a> backslash if they appear as part of a column
> value: backslash itself,> newline, carriage return, and the current
> delimiter character.>
> > So if I scan a field value ( say a big buffer to
> hold an entire> image), and have a second buffer to copy an
> escaped image string to ,> and I read one of the 4 characters mentioned
> above, and insert a> backslash into the destination buffer, and then
> continue copying into> the second buffer, I should end up with a string
> that I can put> inside a comma-separated text file ( so comma =
> the current delimiter> character ), and then I can bulk load a huge
> text file containing> thousands of images without problems using
> COPY..FROM , is that > correct ?
> >
> >
> > On Sat 28/11/09 01:29 , "Tom Lane" tgl@...
> .pa.us sent:>> richard terry
> writes:> tom, I wonder if>> you could give us a sample of using client
> side lo_creat , > insert>> functions to insert a blob into postgres -
> in an sql statement. >> There's an example program in the docs:
> >> http://www.postgresql.org/docs/8.4/static/lo-examplesect.html >> regards,

tom lane

>
> It is my understanding that the current SQL standard defines the single
> quote as the official escape character. While the DB engine accepts
> others, wouldn't it be best to use the official character?
>
> Bob McConnell
> N2SPP
>
>
> --
> Sent via pgsql-novice mailing list (p
> gsql-novice@...)To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-novice
>
>

--
Sent via pgsql-novice mailing list (pgsql-novice@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-novice

Re: bytea and text

2009-11-29T00:34:28Z

On 2009-11-29, Didier Gasser-Morlay <didiergm@...> wrote:

> --0016e6dab093a91c6704797cefd5
> Content-Type: text/plain; charset=ISO-8859-1
>
> I hope you won't mind if I had my 2 cents to this conversation; aside from
> the actual format, this question comes up with regularity on various
> database lists.
>
> I have never understood the need for storing images inside a database (and
> to some extend blob data) because:
> - being a blog you do not query it just store and retrieve, you rarely
> update, so the need for a fancy SQL and DB engine is somewhat limited;
>
> - storing images is heavy, makes the database grow, thus makes
> backup/restore that bit more painful and possibly that bit less stable. ..
>
> For years I designed system for picture libraries where numbers like 10s of
> thousands to millions of images are common place, we would never have
> contemplated to store images on anything more complex than a file system.
> using the database to store a URL to that resource.

ON DELETE CASCADE doesn't work for disk files.

all the applications that need the images need to ba able to
contact the server for the files.

only need to backup one thing.

--
Sent via pgsql-novice mailing list (pgsql-novice@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-novice

Re: bytea and text

2009-11-29T00:30:51Z

On Sunday 29 November 2009 17:50:10 Didier Gasser-Morlay wrote:

> I hope you won't mind if I had my 2 cents to this conversation; aside from
> the actual format, this question comes up with regularity on various
> database lists.
>
> I have never understood the need for storing images inside a database (and
> to some extend blob data) because:
> - being a blog you do not query it just store and retrieve, you rarely
> update, so the need for a fancy SQL and DB engine is somewhat limited;
>
> - storing images is heavy, makes the database grow, thus makes
> backup/restore that bit more painful and possibly that bit less stable. ..
>
> For years I designed system for picture libraries where numbers like 10s of
> thousands to millions of images are common place, we would never have
> contemplated to store images on anything more complex than a file system.
> using the database to store a URL to that resource.
>
> Hope this helps
>
> Didier
>

this is **mission critical** for me (alone perhaps) as the images (usually
very small (eg many could be 15K for a small photo, 3k - 16K for a small
diagram of a body part) are part of some medical record software I'm writing
for myself, and sit in the progress notes when displayed. Just alteration of
a line drawn on a body part for an injured patient could end you up in court
or de-registered.

Medico-legal issues abound.

Yes, one can easily backup files in a directory, but my current feeling is its
easier to do an entire DB dump - could be wrong, stand to be corrected.

Regards

Richard

--
Sent via pgsql-novice mailing list (pgsql-novice@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-novice

Re: Problem loading data from Intro to Postgis

2009-11-28T23:24:16Z

Thanks Tom, I will repost on the postgis list.

Tim

Tim Clark
Department of Zoology
University of Hawaii

--- On Sat, 11/28/09, Tom Lane <tgl@...> wrote:

> From: Tom Lane <tgl@...>
> Subject: Re: [NOVICE] Problem loading data from Intro to Postgis
> To: "Tim Clark" <mudiver1200@...>
> Cc: pgsql-novice@...
> Date: Saturday, November 28, 2009, 7:10 PM
> Tim Clark <mudiver1200@...>
> writes:
> > I just installed postgis and am trying to work through
> the Introduction to Postgis workhop at
> > http://2007.foss4g.org/workshops/W-04/PostGIS%20Workshop.doc#_Toc177458966
>
> You'd probably be best off to ask about that on the postgis
> mailing
> lists. I'm not sure any of that crowd read
> pgsql-novice.
>
>
> regards, tom lane
>

--
Sent via pgsql-novice mailing list (pgsql-novice@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-novice

Re: bytea and text

2009-11-28T23:23:31Z

> this is not a blog, but pictures aren't updated
Sorry I meant "blob", I guess it was not obvious,

>hmmmm, just a one shot backup for "just users".
hmmmm never heard of backup scripts ? 


>I'm not going to feed this troll, this is MY choice.
Not trolling, trying to help by sharing experience, which I thought was the purpose of this list.
I do apologize for interrupting your thought process

Re: bytea and text

2009-11-28T22:55:40Z

Didier Gasser-Morlay a écrit :
> I hope you won't mind if I had my 2 cents to this conversation; aside
> from the actual format, this question comes up with regularity on
> various database lists.
>
> I have never understood the need for storing images inside a database
> (and to some extend blob data) because:
> - being a blog you do not query it just store and retrieve, you rarely
> update, so the need for a fancy SQL and DB engine is somewhat limited;

this is not a blog, but pictures aren't updated

> - storing images is heavy, makes the database grow, thus makes
> backup/restore that bit more painful and possibly that bit less stable. ..
^^^^^^^^^^^^^^^^^^^^
?

> For years I designed system for picture libraries where numbers like 10s
> of thousands to millions of images are common place, we would never have
> contemplated to store images on anything more complex than a file
> system. using the database to store a URL to that resource.

hmmmm, just a one shot backup for "just users".

I'm not going to feed this troll, this is MY choice.

--
Being stoned on marijuana isn't very different from being stoned on gin.
-- Ralph Nader

--
Sent via pgsql-novice mailing list (pgsql-novice@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-novice

Re: bytea and text

2009-11-28T22:50:10Z

I hope you won't mind if I had my 2 cents to this conversation; aside from the actual format, this question comes up with regularity on various database lists.

I have never understood the need for storing images inside a database (and to some extend blob data) because:
- being a blog you do not query it just store and retrieve, you rarely update, so the need for a fancy SQL and DB engine is somewhat limited;

- storing images is heavy, makes the database grow, thus makes backup/restore that bit more painful and possibly that bit less stable. ..

For years I designed system for picture libraries where numbers like 10s of thousands to millions of images are common place, we would never have contemplated to store images on anything more complex than a file system. using the database to store a URL to that resource.

Hope this helps

Didier

Re: Problem loading data from Intro to Postgis

2009-11-28T21:10:12Z

Tim Clark <mudiver1200@...> writes:
> I just installed postgis and am trying to work through the Introduction to Postgis workhop at
> http://2007.foss4g.org/workshops/W-04/PostGIS%20Workshop.doc#_Toc177458966

You'd probably be best off to ask about that on the postgis mailing
lists. I'm not sure any of that crowd read pgsql-novice.

regards, tom lane

--
Sent via pgsql-novice mailing list (pgsql-novice@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-novice

Problem loading data from Intro to Postgis

2009-11-28T20:57:15Z

Dear List,

I just installed postgis and am trying to work through the Introduction to Postgis workhop at
http://2007.foss4g.org/workshops/W-04/PostGIS%20Workshop.doc#_Toc177458966

I have postgis and postgresql installed on a Dell Inspiron 5150 running Windows XP. My problem is that I can't get any data into the database. I have gotten to page 26-27, loading shapefiles. I have run the pg_setenv.bat file at the windows command prompt and it seems to have set up the path correctly. I have looked in the postgresql bin directory and the proper commands are in the folder. However, when I run the given shp2pgsql or psql commands I get an error that the command "is not recognized as an internal or external command, operable program or batch file."

I would appreciate any help or suggestions you could give to gelp me past page 27!

Thanks,

Tim

C:\postgis_workshop\data>dir
Volume in drive C has no label.
Volume Serial Number is 6071-6A25

Directory of C:\postgis_workshop\data

11/28/2009 05:55 PM <DIR> .
11/28/2009 05:55 PM <DIR> ..
06/26/2007 11:15 AM 31,260 bc_border.dbf
06/26/2007 11:15 AM 170 bc_border.prj
06/26/2007 11:15 AM 1,196,124 bc_border.shp
06/26/2007 11:15 AM 41,692 bc_border.shx
06/26/2007 11:15 AM 3,298 bc_hospitals.dbf
06/26/2007 11:15 AM 431 bc_hospitals.prj
06/26/2007 11:15 AM 1,332 bc_hospitals.shp
06/26/2007 11:15 AM 452 bc_hospitals.shx
06/26/2007 11:15 AM 7,388 bc_municipality.dbf
06/26/2007 11:15 AM 431 bc_municipality.prj
06/26/2007 11:15 AM 278,184 bc_municipality.shp
06/26/2007 11:15 AM 1,396 bc_municipality.shx
06/26/2007 11:15 AM 63,193 bc_pubs.dbf
06/26/2007 11:15 AM 431 bc_pubs.prj
06/26/2007 11:15 AM 10,576 bc_pubs.shp
06/26/2007 11:15 AM 3,436 bc_pubs.shx
06/26/2007 11:15 AM 6,690,323 bc_roads.dbf
06/26/2007 11:15 AM 431 bc_roads.prj
06/26/2007 11:15 AM 19,687,612 bc_roads.shp
06/26/2007 11:15 AM 1,305,516 bc_roads.shx
06/26/2007 11:15 AM 1,544 bc_roads.sld
08/23/2007 08:01 PM 1,206,304 bc_voting_areas.dbf
06/26/2007 11:15 AM 431 bc_voting_areas.prj
08/23/2007 08:01 PM 12,586,104 bc_voting_areas.shp
08/23/2007 08:01 PM 63,988 bc_voting_areas.shx
06/26/2007 11:15 AM 532,252 cities.dbf
06/26/2007 11:15 AM 142 cities.prj
06/26/2007 11:15 AM 71,024 cities.shp
06/26/2007 11:15 AM 141,281 cities.shp.xml
06/26/2007 11:15 AM 20,364 cities.shx
06/26/2007 11:15 AM 14,551 countries.dbf
06/26/2007 11:15 AM 142 countries.prj
06/26/2007 11:15 AM 1,537,144 countries.shp
06/26/2007 11:15 AM 1,772 countries.shx
06/26/2007 11:15 AM 6,263 countries.sld
06/26/2007 11:15 AM 48,914 newyork_census.dbf
06/26/2007 11:15 AM 69,652 newyork_census.htm
06/26/2007 11:15 AM 570 newyork_census.prj
06/26/2007 11:15 AM 3,662,668 newyork_census.shp
06/26/2007 11:15 AM 17,828 newyork_census.shx
06/26/2007 11:15 AM 122,367 nnewyork_census.xml
08/22/2007 01:16 PM 55 pg_setenv.bat
08/22/2007 01:16 PM 402 pg_shpsql.bat
06/26/2007 11:15 AM 332 timezone.dbf
06/26/2007 11:15 AM 142 timezone.prj
06/26/2007 11:15 AM 466,308 timezone.shp
06/26/2007 11:15 AM 136,399 timezone.shp.xml
06/26/2007 11:15 AM 404 timezone.shx
06/26/2007 11:15 AM 1,165 timezone.sld
06/26/2007 11:15 AM 1,327,200 usa_counties.dbf
06/26/2007 11:15 AM 164 usa_counties.prj
06/26/2007 11:15 AM 1,518,248 usa_counties.shp
06/26/2007 11:15 AM 25,228 usa_counties.shx
53 File(s) 52,905,028 bytes
2 Dir(s) 18,336,935,936 bytes free

C:\postgis_workshop\data>pg_setenv.bat

C:\postgis_workshop\data>SET PATH="C:\Program Files\PostgreSQL\8.2\bin";C:\Progr
am Files\MiKTeX 2.7\miktex\bin;C:\Perl\site\bin;C:\Perl\bin;C:\WINDOWS\system32;
C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\MATLAB7\bin\win32;C:\PROGRA~1\AUTOSI~1.7;
C:\Program Files\QuickTime\QTSystem\

C:\postgis_workshop\data>dir *.shp
Volume in drive C has no label.
Volume Serial Number is 6071-6A25

Directory of C:\postgis_workshop\data

06/26/2007 11:15 AM 1,196,124 bc_border.shp
06/26/2007 11:15 AM 1,332 bc_hospitals.shp
06/26/2007 11:15 AM 278,184 bc_municipality.shp
06/26/2007 11:15 AM 10,576 bc_pubs.shp
06/26/2007 11:15 AM 19,687,612 bc_roads.shp
08/23/2007 08:01 PM 12,586,104 bc_voting_areas.shp
06/26/2007 11:15 AM 71,024 cities.shp
06/26/2007 11:15 AM 1,537,144 countries.shp
06/26/2007 11:15 AM 3,662,668 newyork_census.shp
06/26/2007 11:15 AM 466,308 timezone.shp
06/26/2007 11:15 AM 1,518,248 usa_counties.shp
11 File(s) 41,015,324 bytes
0 Dir(s) 18,336,935,936 bytes free

C:\postgis_workshop\data>shp2pgsql -i -D -s 3005 bc_pubs.shp bc_pubs > bc_pubs.s
ql
'shp2pgsql' is not recognized as an internal or external command,
operable program or batch file.

C:\postgis_workshop\data>psql -U postgres -f bc_pubs.sql -d postgis
'psql' is not recognized as an internal or external command,
operable program or batch file.

C:\postgis_workshop\data>psql
'psql' is not recognized as an internal or external command,
operable program or batch file.

Tim Clark
Department of Zoology
University of Hawaii

--
Sent via pgsql-novice mailing list (pgsql-novice@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-novice

Re: bytea and text

2009-11-28T05:19:15Z

Syan Tan wrote:

> here's a quote from the postgresql documentation
>
> Backslash characters (\) can be used in the COPY data to quote data
> characters that might otherwise be taken as row or column delimiters.
> In particular, the following characters must be preceded by a
> backslash if they appear as part of a column value: backslash itself,
> newline, carriage return, and the current delimiter character.
>
> So if I scan a field value ( say a big buffer to hold an entire
> image), and have a second buffer to copy an escaped image string to ,
> and I read one of the 4 characters mentioned above, and insert a
> backslash into the destination buffer, and then continue copying into
> the second buffer, I should end up with a string that I can put
> inside a comma-separated text file ( so comma = the current delimiter
> character ), and then I can bulk load a huge text file containing
> thousands of images without problems using COPY..FROM , is that
> correct ?
>
>
> On Sat 28/11/09 01:29 , "Tom Lane" tgl@... sent:
>> richard terry <rterry@ pacific.net.au> writes:> tom, I wonder if
>> you could give us a sample of using client side lo_creat , > insert
>> functions to insert a blob into postgres - in an sql statement.
>> There's an example program in the docs:
>> http://www.postgresql.org/docs/8.4/static/lo-examplesect.html
>> regards, tom lane

It is my understanding that the current SQL standard defines the single
quote as the official escape character. While the DB engine accepts
others, wouldn't it be best to use the official character?

Bob McConnell
N2SPP

--
Sent via pgsql-novice mailing list (pgsql-novice@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-novice

Re: bytea and text

2009-11-27T22:24:15Z

here's a quote from the postgresql documentation

Backslash characters (\) can be used in the COPY data to quote data characters that might otherwise be taken as
row or column delimiters. In particular, the following characters must be preceded by a backslash if they appear
as part of a column value: backslash itself, newline, carriage return, and the current delimiter character.

So if I scan a field value ( say a big buffer to hold an entire image), and have a second buffer to copy
an escaped image string to , and I read one of the 4 characters mentioned above, and insert a backslash
into the destination buffer, and then continue copying into the second buffer, I should end up with a string
that I can put inside a comma-separated text file ( so comma = the current delimiter character ), and then
I can bulk load a huge text file containing thousands of images without problems using COPY..FROM , is that
correct ?

On Sat 28/11/09 01:29 , "Tom Lane" tgl@... sent:

> richard terry <rterry@
> pacific.net.au> writes:> tom, I wonder if you could give us a sample of
> using client side lo_creat , > insert functions to insert a blob into postgres
> - in an sql statement.
> There's an example program in the docs:
> http://www.postgresql.org/docs/8.4/static/lo-examplesect.html
> regards, tom lane
>
> --
> Sent via pgsql-novice mailing list (p
> gsql-novice@...)To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-novice
>
>

--
Sent via pgsql-novice mailing list (pgsql-novice@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-novice

Re: bytea and text

2009-11-27T21:30:10Z

Is there a gambas specific wrapper for the lo_export/lo_import function ?

On Sat 28/11/09 01:29 , "Tom Lane" tgl@... sent:

--
Sent via pgsql-novice mailing list (pgsql-novice@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-novice

Re: bytea and text

2009-11-27T17:29:52Z

richard terry <rterry@...> writes:
> tom, I wonder if you could give us a sample of using client side lo_creat ,
> insert functions to insert a blob into postgres - in an sql statement.

There's an example program in the docs:
http://www.postgresql.org/docs/8.4/static/lo-examplesect.html

regards, tom lane

--
Sent via pgsql-novice mailing list (pgsql-novice@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-novice

Re: bytea and text

2009-11-27T17:05:12Z

this is a wrapper for one parameter 'base64' , I thought what he wanted was a simple
way of uploading files by filename into a postgresql database field via a gambase
client interface.

On Sat 28/11/09 00:08 , "Jean-Yves F. Barbier" 12ukwn@... sent:

> richard terry a écrit :
> ...
> > tom, I wonder if you could give us a sample of
> using client side lo_creat , > insert functions to insert a blob into postgres
> - in an sql statement.>
> > Despite reading the docs's I'm totally in the
> dark and can't understand the > syntax.
>
> Hi Richard, I modified the functions I sent you: they now store into
> BINARY (BYTEA) format, not anymore into BASE64:
>
> CREATE OR REPLACE FUNCTION ucommon.testbytea_ins(Pstring TEXT) RETURNS oid
> AS $$DECLARE
> NewId OID;
> NewBytea BYTEA;
> BEGIN
> NewBytea = decode(Pstring, 'base64');
> INSERT INTO common.testbytea VALUES(default, NewBytea);
> SELECT id INTO NewId FROM common.testbytea WHERE id = (SELECT
> currval('testbytea_id_seq'));RETURN NewId;
> END;
> $$ LANGUAGE PLPGSQL STRICT SECURITY DEFINER;
> REVOKE ALL ON FUNCTION ucommon.testbytea_ins(TEXT) FROM PUBLIC;
> ---------------------------------------------------
> CREATE OR REPLACE FUNCTION ucommon.testbytea_sel(Pid OID) RETURNS TEXT AS
> $$DECLARE
> MyPic BYTEA;
> MyString TEXT;
> BEGIN
> SELECT pic INTO MyPic FROM common.testbytea WHERE id = Pid;
> MyString = encode(MyPic, 'base64');
> RETURN MyString;
> END;
> $$ LANGUAGE PLPGSQL STRICT SECURITY DEFINER;
> REVOKE ALL ON FUNCTION ucommon.testbytea_sel(OID) FROM PUBLIC;
>
> HIWH
>
> JY
> --
> Objects in mirror may be closer than they appear.
>
> --
> Sent via pgsql-novice mailing list (p
> gsql-novice@...)To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-novice
>
>

--
Sent via pgsql-novice mailing list (pgsql-novice@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-novice

Re: bytea and text

2009-11-27T16:08:31Z

richard terry a écrit :
...
> tom, I wonder if you could give us a sample of using client side lo_creat ,
> insert functions to insert a blob into postgres - in an sql statement.
>
> Despite reading the docs's I'm totally in the dark and can't understand the
> syntax.

Hi Richard, I modified the functions I sent you: they now store into
BINARY (BYTEA) format, not anymore into BASE64:

CREATE OR REPLACE FUNCTION ucommon.testbytea_ins(Pstring TEXT) RETURNS oid AS $$
DECLARE
NewId OID;
NewBytea BYTEA;
BEGIN
NewBytea = decode(Pstring, 'base64');
INSERT INTO common.testbytea VALUES(default, NewBytea);
SELECT id INTO NewId FROM common.testbytea WHERE id = (SELECT currval('testbytea_id_seq'));
RETURN NewId;
END;
$$ LANGUAGE PLPGSQL STRICT SECURITY DEFINER;
REVOKE ALL ON FUNCTION ucommon.testbytea_ins(TEXT) FROM PUBLIC;
---------------------------------------------------
CREATE OR REPLACE FUNCTION ucommon.testbytea_sel(Pid OID) RETURNS TEXT AS $$
DECLARE
MyPic BYTEA;
MyString TEXT;
BEGIN
SELECT pic INTO MyPic FROM common.testbytea WHERE id = Pid;
MyString = encode(MyPic, 'base64');
RETURN MyString;
END;
$$ LANGUAGE PLPGSQL STRICT SECURITY DEFINER;
REVOKE ALL ON FUNCTION ucommon.testbytea_sel(OID) FROM PUBLIC;

HIWH

JY
--
Objects in mirror may be closer than they appear.

--
Sent via pgsql-novice mailing list (pgsql-novice@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-novice

Re: bytea and text

2009-11-27T16:00:06Z

Not sure if decode works for COPY statements though, I thought copy is faster
than insert for bulk loading lots of data
than using a client call in whatever language postgres driver. But then its only
one field for binary data which is a lot bigger
than the other fields in the tuple, so I'm not sure what is the rate-limiting
factor here.

On Fri 27/11/09 14:18 , "Jean-Yves F. Barbier" 12ukwn@... sent:

> Jasen Betts a écrit :
> > On 2009-11-26, Jean-Yves F. Barbier <12ukwn@gmail
> .com> wrote:>> Hi list,
> >>
> >> I'm asking myself what solution for storing
> pictures is the best:>>
> >> * using a BYTEA column type, and having no
> intrinsic compression gain >> because a picture it almost every time
> already compressed,>>
> >> * using a TEXT column type, and store a
> Base64(picture) in it; it should>> take benefits of intrinsic compression
> (?).>
> > use bytea and convert to/from base64 if you
> can't use the binary> interface and find the text interface
> inconvenient.
> that's what I do now
>
> > this will make the data over 1/3
> bigger.
> uuu, na: it comes as Base64 (to avoid a 0x89 unknown utf8 char error), but
> itis converted to BYTEA as I now use decode('inparm', 'base64') before
> insert,as back to Base64 when I extract it (with encode).
>
> --
> * JHM wonders what Joey did to earn "I'd just like to say, for the
> record,that Joey rules."
> -- Seen on #Debian
>
> --
> Sent via pgsql-novice mailing list (p
> gsql-novice@...)To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-novice
>
>

--
Sent via pgsql-novice mailing list (pgsql-novice@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-novice

Re: bytea and text

2009-11-27T15:50:44Z

On Friday 27 November 2009 02:58:01 Tom Lane wrote:

> "Jean-Yves F. Barbier" <12ukwn@...> writes:
> > I'm asking myself what solution for storing pictures is the best:
> >
> > * using a BYTEA column type, and having no intrinsic compression gain
> > because a picture it almost every time already compressed,
> >
> > * using a TEXT column type, and store a Base64(picture) in it; it should
> > take benefits of intrinsic compression (?).
>
> At best, the compression would get rid of the overhead you added by
> converting to base64. It probably wouldn't completely succeed at that,
> though, meaning the second alternative is always a loser.
>
> regards, tom lane
>

tom, I wonder if you could give us a sample of using client side lo_creat ,
insert functions to insert a blob into postgres - in an sql statement.

Despite reading the docs's I'm totally in the dark and can't understand the
syntax.

Regards

richard (using gambas basic)

--
Sent via pgsql-novice mailing list (pgsql-novice@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-novice

Extracting special case from a player table without union

2009-11-27T11:05:15Z

I'm running a hockey player database and I'm building up statistics
about player penalties. I extract the penalties like this:

SELECT P.playerid, T.number, count(P.playerid) AS num_penalties,
sum(P.length) AS length_penalties
FROM period_penalties P, troops T
WHERE T.origin=1 AND T.season=2009 AND T.playerid=P.playerid AND NOT
P.opponent AND P.period_id IN (...id_list...)
GROUP BY P.playerid, T.number

and get a result like this:

playerid | number | num_penalties | length_penalties
----------+--------+---------------+------------------
236 | 89 | 9 | 26
267 | 20 | 8 | 37
214 | 14 | 1 | 2
...

This is what I want.

However, there is a special case which I need to fetch as well. The
special case is lines in the table with playerid = 0, which means it's
a team penalty instead of a player penalty. Id 0 does not exist in the
player table (called "troops" in the query above), and that's why that
row does not get included in the result.

The only way I found out how to include the special case row is by
doing a union-query, like this:

SELECT ...as above...
UNION
SELECT P.playerid, Null, count(P.playerid) AS num_penalties,
sum(P.length) AS length_penalties
FROM period_penalties P
WHERE playerid=0 AND NOT P.opponent AND P.period_id IN (...period_ids...)
GROUP BY P.playerid
ORDER BY length_penalties DESC, num_penalties DESC

and the result is like this:

playerid | number | num_penalties | length_penalties
----------+--------+---------------+------------------
20 | 6 | 15 | 103
28 | 4 | 11 | 46
267 | 20 | 8 | 37
265 | 2 | 5 | 31
0 | | 4 | 29
122 | 11 | 4 | 29
...

Which is the actual result I want. However, I find the union-query
horrible and in a way somewhat redundant. But since my JOIN-skills are
too low I am unable to find a better solution than this.

If anyone can shed some light on how to approach this problem better
than my solution I'm all ears.

--
- Rikard - http://bos.hack.org/cv/

--
Sent via pgsql-novice mailing list (pgsql-novice@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-novice

views

2009-11-27T07:18:55Z

Hi,

Knowing all fields can be selected, is there a difference making
a view that references all fields one by one and a view with a "select *"?

--
"I don't mind going nowhere as long as it's an interesting path."
-- Ronald Mabbitt

--
Sent via pgsql-novice mailing list (pgsql-novice@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-novice

Re: How best to index foreign key for inner join

2009-11-27T07:16:23Z

2009/11/27 Nathaniel Trellice <naptrel@...>

On 27 Nov 2009, at 11:12, Thom Brown <thombrown@...> wrote:

Since for each row of table1 you'll be looking for multiple records on table2, I imagine you'd want to index table2.table1_id, and table2.t since that's what you're filtering on. If table1 is particularly big, you might benefit from also indexing table1.name.

I personally prefer to use INNER JOIN syntax as it's clearer, although the query planner will probably be identical as it's clever that way:

SELECT table2.x, table2.y, table2.t
FROM table1
INNER JOIN table2 ON table1.id = table2.table1_id
AND table1.name = 'some_name'
WHERE table2.t BETWEEN some_t AND some_other_t;

Thanks for the help Thom.

Do you think it's possible to phrase this query in such a way that the planner would be able to exploit any benefits from a mutli-column index on table2 on either (table1_id, t) or (t, table1_t)?

If you imagine setting up a view that encapsulates the inner join, creating a 'virtual' table with columns:

name text
x real
y real

t timestampz

Then the manual (11.3) suggests that ANDed queries on 'name' and 't' will be improved by using a multi-column index (name, t).

But the join confuses me. When the rule system breaks down both the query and the view's join, will the benefits of the multi-column index still be realised?

Nathaniel

Again, it really depends on the size of your tables. If, for example, table2 contains 5 million rows and 99% of them have the same t value, and index probably won't be used by the planner since it's value distribution in the statistics data for that table (gathered by ANALYZE) would suggest a sequential scan is just as efficient. If you have a lot of varying combinations between table2.table1_id and table2.t, a multi-column index will probably benefit you.

A good way to tell how much work is involved in running a query is to put EXPLAIN ANALYZE before it to see where the more expensive parts of the query are.

There are many instances where indexes won't be used by the planner purely because it's statistics tell it that's it's more or equally efficient to a sequential scan. You need to know your data well and take the size of the table into account.

I'm hoping someone with better query-planner knowledge can chime in here with more solid recommendations though.

Thom

Re: index speed-up and automatic tables/procedures creation

2009-11-27T07:04:44Z

Tom Lane a écrit :
...

> I think you're wasting your time. What you are setting out to do here
> is manually emulate the top layer or so of a large index. Unless you
> have very specific (and unusual) data access patterns that you know in
> considerable detail, this is not a game you are going to win. Just go
> with the one big table and one index, you'll be happier. (Note that
> "several million rows" is not big, it's barely enough to notice.)
>
> You will see a lot of discussion about partitioning of tables if you
> look around the list archives, but this is not done with the idea that
> it makes access to any one row faster. The biggest motivation usually
> is to allow dropping ranges of data cheaply, like throwing away a month's
> or year's worth of old data at once.

Just to make sure I understood the spirit:

* I keep a large table,

* As my join tables have just (pkey=pkeys from each side), I also make indexes
on each foreign pkey,

* (May be?) I also make partial indexes, in order to have ie a faster retrieve
of not-sold items instead of excluding sold items in the query

JY
--
-- I have seen the FUN --

--
Sent via pgsql-novice mailing list (pgsql-novice@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-novice

Re: How best to index foreign key for inner join

2009-11-27T06:44:53Z

On 27 Nov 2009, at 11:12, Thom Brown <thombrown@...> wrote:

Since for each row of table1 you'll be looking for multiple records on table2, I imagine you'd want to index table2.table1_id, and table2.t since that's what you're filtering on. If table1 is particularly big, you might benefit from also indexing table1.name.

I personally prefer to use INNER JOIN syntax as it's clearer, although the query planner will probably be identical as it's clever that way:

SELECT table2.x, table2.y, table2.t
FROM table1
INNER JOIN table2 ON table1.id = table2.table1_id
AND table1.name = 'some_name'
WHERE table2.t BETWEEN some_t AND some_other_t;

Thanks for the help Thom.

Do you think it's possible to phrase this query in such a way that the planner would be able to exploit any benefits from a mutli-column index on table2 on either (table1_id, t) or (t, table1_t)?

If you imagine setting up a view that encapsulates the inner join, creating a 'virtual' table with columns:

name text

x real

y real

t timestampz

Then the manual (11.3) suggests that ANDed queries on 'name' and 't' will be improved by using a multi-column index (name, t).

But the join confuses me. When the rule system breaks down both the query and the view's join, will the benefits of the multi-column index still be realised?

Nathaniel

Re: index speed-up and automatic tables/procedures creation

2009-11-27T06:21:03Z

Tom Lane a écrit :

> "Jean-Yves F. Barbier" <12ukwn@...> writes:
>> 1)- I'd like to keep a table in one piece, but it'll be huge (several millions rows
>> and growing); can a segmentation of indexes (all indexes that are used for
>> searching) speed-up this table scans enough to keep it as responsive to queries as
>> multiple tables? And what can I do about the primary key index, which is monolitic?
>> (I can't use inheritance as there are some integrity references into it.)
>
> I think you're wasting your time. What you are setting out to do here
> is manually emulate the top layer or so of a large index. Unless you
> have very specific (and unusual) data access patterns that you know in
> considerable detail, this is not a game you are going to win. Just go
> with the one big table and one index, you'll be happier. (Note that
> "several million rows" is not big, it's barely enough to notice.)
>
> You will see a lot of discussion about partitioning of tables if you
> look around the list archives, but this is not done with the idea that
> it makes access to any one row faster. The biggest motivation usually
> is to allow dropping ranges of data cheaply, like throwing away a month's
> or year's worth of old data at once.
>
> regards, tom lane

That's a *very* clear answer, thanks Tom!

JY
--
If a thing's worth having, it's worth cheating for.
-- W. C. Fields

--
Sent via pgsql-novice mailing list (pgsql-novice@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-novice

Re: bytea and text

2009-11-27T06:18:54Z

Jasen Betts a écrit :

> On 2009-11-26, Jean-Yves F. Barbier <12ukwn@...> wrote:
>> Hi list,
>>
>> I'm asking myself what solution for storing pictures is the best:
>>
>> * using a BYTEA column type, and having no intrinsic compression gain
>> because a picture it almost every time already compressed,
>>
>> * using a TEXT column type, and store a Base64(picture) in it; it should
>> take benefits of intrinsic compression (?).
>
> use bytea and convert to/from base64 if you can't use the binary
> interface and find the text interface inconvenient.

that's what I do now

> this will make the data over 1/3 bigger.

uuu, na: it comes as Base64 (to avoid a 0x89 unknown utf8 char error), but it
is converted to BYTEA as I now use decode('inparm', 'base64') before insert,
as back to Base64 when I extract it (with encode).

--
* JHM wonders what Joey did to earn "I'd just like to say, for the record,
that Joey rules."
-- Seen on #Debian

--
Sent via pgsql-novice mailing list (pgsql-novice@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-novice

Re: How best to index foreign key for inner join

2009-11-27T03:12:09Z

2009/11/27 Nathaniel Trellice <naptrel@...>

Hello,

I'd appreciate advice anyone can offer on this simple, newbie
question. It really is a straightforward question, so please bare with
me if I don't explain it succinctly below.

I want to search a table based on criteria on both a column within
that table and a column within a joined table, and I'm trying to
figure out how to index things for my particular form of query. The
postgres manual is excellent, but it doesn't appear to answer this
question explicitly through a matching example, and I'm not very good
with indexing, so I'm seeking advice here (yet again).

E.g. Suppose I have 2 tables defined:

CREATE TABLE table1 (
id SERIAL PRIMARY KEY,
name text
);

CREATE TABLE table2 (
x real,
y real,
t timestampz,
table1_id integer REFERENCES table1 ON DELETE CASCADE
);

My queries upon these tables are, almost exclusively, asking for all
the records with a particular 'name' between a range of times 't', and
so are of the form:

SELECT table2.x, table2.y, table2.t
FROM table2, table1
WHERE
table2.table1_id = table1.id -- inner join
AND table1.name = 'some_name' -- criterion on table1
AND table2.t BETWEEN some_t AND some_other_t; --criterion on table2

What columns should I index in table2, and how: multi-column on
(table1_id, t), say, or multiple single column indexes, or only one
on 't'?

I guess the query might benefit from an index on table1's 'name'
column, but since there are, typically, only tens or hundreds of
records in table1, it might not be worth it. There are millions of
records in table2.

Thanks,

Nathaniel

Since for each row of table1 you'll be looking for multiple records on table2, I imagine you'd want to index table2.table1_id, and table2.t since that's what you're filtering on. If table1 is particularly big, you might benefit from also indexing table1.name.

I personally prefer to use INNER JOIN syntax as it's clearer, although the query planner will probably be identical as it's clever that way:

SELECT table2.x, table2.y, table2.t
FROM table1
INNER JOIN table2 ON table1.id = table2.table1_id
AND table1.name = 'some_name'
WHERE table2.t BETWEEN some_t AND some_other_t;

Regards

Thom

How best to index foreign key for inner join

2009-11-27T03:04:21Z

Hello,

I'd appreciate advice anyone can offer on this simple, newbie
question. It really is a straightforward question, so please bare with
me if I don't explain it succinctly below.

I want to search a table based on criteria on both a column within
that table and a column within a joined table, and I'm trying to
figure out how to index things for my particular form of query. The
postgres manual is excellent, but it doesn't appear to answer this
question explicitly through a matching example, and I'm not very good
with indexing, so I'm seeking advice here (yet again).

E.g. Suppose I have 2 tables defined:

CREATE TABLE table1 (
id SERIAL PRIMARY KEY,
name text
);

CREATE TABLE table2 (
x real,
y real,
t timestampz,
table1_id integer REFERENCES table1 ON DELETE CASCADE
);

My queries upon these tables are, almost exclusively, asking for all
the records with a particular 'name' between a range of times 't', and
so are of the form:

SELECT table2.x, table2.y, table2.t
FROM table2, table1
WHERE
table2.table1_id = table1.id -- inner join
AND table1.name = 'some_name' -- criterion on table1
AND table2.t BETWEEN some_t AND some_other_t; --criterion on table2

What columns should I index in table2, and how: multi-column on
(table1_id, t), say, or multiple single column indexes, or only one
on 't'?

I guess the query might benefit from an index on table1's 'name'
column, but since there are, typically, only tens or hundreds of
records in table1, it might not be worth it. There are millions of
records in table2.

Thanks,

Nathaniel

--
Sent via pgsql-novice mailing list (pgsql-novice@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-novice

Re: bytea and text

2009-11-26T23:57:18Z

On 2009-11-26, Jean-Yves F. Barbier <12ukwn@...> wrote:
> Hi list,
>
> I'm asking myself what solution for storing pictures is the best:
>
> * using a BYTEA column type, and having no intrinsic compression gain
> because a picture it almost every time already compressed,
>
> * using a TEXT column type, and store a Base64(picture) in it; it should
> take benefits of intrinsic compression (?).

use bytea and convert to/from base64 if you can't use the binary
interface and find the text interface inconvenient.

this will make the data over 1/3 bigger.

--
Sent via pgsql-novice mailing list (pgsql-novice@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-novice

Re: index speed-up and automatic tables/procedures creation

2009-11-26T22:49:39Z

"Jean-Yves F. Barbier" <12ukwn@...> writes:
> 1)- I'd like to keep a table in one piece, but it'll be huge (several millions rows
> and growing); can a segmentation of indexes (all indexes that are used for
> searching) speed-up this table scans enough to keep it as responsive to queries as
> multiple tables? And what can I do about the primary key index, which is monolitic?
> (I can't use inheritance as there are some integrity references into it.)

I think you're wasting your time. What you are setting out to do here
is manually emulate the top layer or so of a large index. Unless you
have very specific (and unusual) data access patterns that you know in
considerable detail, this is not a game you are going to win. Just go
with the one big table and one index, you'll be happier. (Note that
"several million rows" is not big, it's barely enough to notice.)

You will see a lot of discussion about partitioning of tables if you
look around the list archives, but this is not done with the idea that
it makes access to any one row faster. The biggest motivation usually
is to allow dropping ranges of data cheaply, like throwing away a month's
or year's worth of old data at once.

regards, tom lane

--
Sent via pgsql-novice mailing list (pgsql-novice@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-novice

Re: index speed-up and automatic tables/procedures creation

2009-11-26T20:47:17Z

Greg Stark a écrit :
> On Fri, Nov 27, 2009 at 3:15 AM, Jean-Yves F. Barbier <12ukwn@...> wrote:
>> Sooo, I guess the answer is: cut your table by pieces=year?
>>
>
> Well there's no one-size-fits-all solution. It'll depend on your
> priorities and your situation.
>

Ok, this one will be the items' table, as there can be many types of
links between an item and different documents, links will be done
by tables containing item(id),doc(id) - one for each type of doc.

May be, I'm not sure at the moment, when items will be out of warranty
and/or documents won't need to be legally kept online, I'll move
them to itema (for Archive) - and the same for docs.

--
Very few profundities can be expressed in less than 80 characters.

--
Sent via pgsql-novice mailing list (pgsql-novice@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-novice

Re: index speed-up and automatic tables/procedures creation

2009-11-26T19:58:22Z

On Fri, Nov 27, 2009 at 3:15 AM, Jean-Yves F. Barbier <12ukwn@...> wrote:
> Sooo, I guess the answer is: cut your table by pieces=year?
>

Well there's no one-size-fits-all solution. It'll depend on your
priorities and your situation.

--
greg

--
Sent via pgsql-novice mailing list (pgsql-novice@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-novice

Re: index speed-up and automatic tables/procedures creation

2009-11-26T19:15:44Z

Greg Stark a écrit :

> On Thu, Nov 26, 2009 at 10:19 PM, Jean-Yves F. Barbier <12ukwn@...> wrote:
>> 1)- I'd like to keep a table in one piece, but it'll be huge (several millions rows
>> and growing); can a segmentation of indexes (all indexes that are used for
>> searching) speed-up this table scans enough to keep it as responsive to queries as
>> multiple tables? And what can I do about the primary key index, which is monolitic?
>> (I can't use inheritance as there are some integrity references into it.)
>
> There are plenty of people with tables with many more than several
> million records. How big that is depends on how wide those rows are,
> but still, it's not necessarily a problem. Indexed access speed should
> scale fine.
>
> The real problem that partitioning addresses is routine maintenance.
> When it comes time to dump this table or create a new index or even
> just scan a large section of the table for a report you may find the
> jobs taking impracticably long.

Sooo, I guess the answer is: cut your table by pieces=year?

--
<Overfiend> ltd: Fine, go through life just pointing and grunting at
what you mean. Works for Mac users.

--
Sent via pgsql-novice mailing list (pgsql-novice@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-novice

Re: index speed-up and automatic tables/procedures creation

2009-11-26T16:04:01Z

On Thu, Nov 26, 2009 at 10:19 PM, Jean-Yves F. Barbier <12ukwn@...> wrote:
> 1)- I'd like to keep a table in one piece, but it'll be huge (several millions rows
> and growing); can a segmentation of indexes (all indexes that are used for
> searching) speed-up this table scans enough to keep it as responsive to queries as
> multiple tables? And what can I do about the primary key index, which is monolitic?
> (I can't use inheritance as there are some integrity references into it.)

There are plenty of people with tables with many more than several
million records. How big that is depends on how wide those rows are,
but still, it's not necessarily a problem. Indexed access speed should
scale fine.

The real problem that partitioning addresses is routine maintenance.
When it comes time to dump this table or create a new index or even
just scan a large section of the table for a report you may find the
jobs taking impracticably long.

--
greg

--
Sent via pgsql-novice mailing list (pgsql-novice@...)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-novice