|
View:
New views
19 Messages
—
Rating Filter:
Alert me
|
|
|
Updated shapefile to update postgres databaseHello!
I would like some help updating a postgres database. This is the problem: We created the database using shp2pgsql to convert a shapefile that contains a city's street details (centerline), after we created the database we also created an aditional column to classify the street types (eg: street, avenues, highways, etc.), now we are working on the original shapefile, updating new roads, changes made to highways due to constructions modifications, etc., and we want to upload the "updated" shapefile, without afecting the actual database, another words, we would like to convert to postgres the updated shapefile with the new information (it cuold be the whole shapefile) but to keep the added clasiffication column intact. How can we do that? Thanks |
|
|
Re: Updated shapefile to update postgres databasePresumably what you mean by "update" is that you want to replace old
versions of roads already in PostGIS with new versions of the same roads from shapefiles, and you want to keep the old street type in conjunction with the updated geometry? If that's the case, your problem is really determining which old roads are coincident with which new roads. One approach might be to buffer the new road by a metre or two, and then select the old road whose length within the buffer is the maximum of all old roads that intersect the buffer. This specific old road's street type could be transferred to the new road, and the old road could be deleted. The above is probably an oversimplification that will crash and burn when confronted by real live roads whose updates are not particularly coincident with their older avatars. SenTnel wrote: > Hello! > > I would like some help updating a postgres database. This is the problem: We > created the database using shp2pgsql to convert a shapefile that contains a > city's street details (centerline), after we created the database we also > created an aditional column to classify the street types (eg: street, > avenues, highways, etc.), now we are working on the original shapefile, > updating new roads, changes made to highways due to constructions > modifications, etc., and we want to upload the "updated" shapefile, without > afecting the actual database, another words, we would like to convert to > postgres the updated shapefile with the new information (it cuold be the > whole shapefile) but to keep the added clasiffication column intact. How can > we do that? > > Thanks > -- Regards, Chris Hermansen mailto:chris.hermansen@... tel+1.604.714.2878 · fax+1.604.733.0631 · mob+1.778.232.0644 Timberline Natural Resource Group · http://www.timberline.ca 401 · 958 West 8th Avenue · Vancouver BC · Canada · V5Z 1E5 _______________________________________________ postgis-users mailing list postgis-users@... http://postgis.refractions.net/mailman/listinfo/postgis-users |
|
|
RE: Updated shapefile to update postgres databaseOgr2Ogr at http://www.gdal.org/ogr/ogr2ogr.html has append and update
options. As long as your new columns were added to the end of the table it will probably will work fine. Best regards, Martin -----Original Message----- From: postgis-users-bounces@... [mailto:postgis-users-bounces@...] On Behalf Of SenTnel Sent: Tuesday, May 06, 2008 4:40 PM To: postgis-users@... Subject: [postgis-users] Updated shapefile to update postgres database Hello! I would like some help updating a postgres database. This is the problem: We created the database using shp2pgsql to convert a shapefile that contains a city's street details (centerline), after we created the database we also created an aditional column to classify the street types (eg: street, avenues, highways, etc.), now we are working on the original shapefile, updating new roads, changes made to highways due to constructions modifications, etc., and we want to upload the "updated" shapefile, without afecting the actual database, another words, we would like to convert to postgres the updated shapefile with the new information (it cuold be the whole shapefile) but to keep the added clasiffication column intact. How can we do that? Thanks -- View this message in context: http://www.nabble.com/Updated-shapefile-to-update-postgres-database-tp170535 62p17053562.html Sent from the PostGIS - User mailing list archive at Nabble.com. _______________________________________________ postgis-users mailing list postgis-users@... http://postgis.refractions.net/mailman/listinfo/postgis-users _______________________________________________ postgis-users mailing list postgis-users@... http://postgis.refractions.net/mailman/listinfo/postgis-users |
|
|
Re: Updated shapefile to update postgres databaseI was interested in a similar thing once, but the ogr2ogr option names
are misleading. "-update" means add to an existing data source (ie the database in Postgres). Otherwise ogr2ogr will try to create a new database, and fail if it exists. "-append" means add to an existing layer (or, the table within the database) within a data source. Otherwise it will try to create a new table, and fail if it exists. Update will not replace any attributes or geometry for a feature that already exists with the same FID, but will stop with a duplicate key error. On May 6, 2008, at 6:06 PM, Martin Chapman wrote: > Ogr2Ogr at http://www.gdal.org/ogr/ogr2ogr.html has append and update > options. As long as your new columns were added to the end of the > table it > will probably will work fine. > > Best regards, > Martin > > > -----Original Message----- > From: postgis-users-bounces@... > [mailto:postgis-users-bounces@...] On Behalf Of > SenTnel > Sent: Tuesday, May 06, 2008 4:40 PM > To: postgis-users@... > Subject: [postgis-users] Updated shapefile to update postgres database > > > Hello! > > I would like some help updating a postgres database. This is the > problem: We > created the database using shp2pgsql to convert a shapefile that > contains a > city's street details (centerline), after we created the database we > also > created an aditional column to classify the street types (eg: street, > avenues, highways, etc.), now we are working on the original > shapefile, > updating new roads, changes made to highways due to constructions > modifications, etc., and we want to upload the "updated" shapefile, > without > afecting the actual database, another words, we would like to > convert to > postgres the updated shapefile with the new information (it cuold be > the > whole shapefile) but to keep the added clasiffication column intact. > How can > we do that? > > Thanks > -- > View this message in context: > http://www.nabble.com/Updated-shapefile-to-update-postgres-database-tp170535 > 62p17053562.html > Sent from the PostGIS - User mailing list archive at Nabble.com. > > _______________________________________________ > postgis-users mailing list > postgis-users@... > http://postgis.refractions.net/mailman/listinfo/postgis-users > > _______________________________________________ > postgis-users mailing list > postgis-users@... > http://postgis.refractions.net/mailman/listinfo/postgis-users ----- William Kyngesburye <kyngchaos*at*kyngchaos*dot*com> http://www.kyngchaos.com/ The equator is so long, it could encircle the earth completely once. _______________________________________________ postgis-users mailing list postgis-users@... http://postgis.refractions.net/mailman/listinfo/postgis-users |
|
|
Re: Updated shapefile to update postgres databaseSenTnel wrote:
> Hello! > > I would like some help updating a postgres database. This is the problem: We > created the database using shp2pgsql to convert a shapefile that contains a > city's street details (centerline), after we created the database we also > created an aditional column to classify the street types (eg: street, > avenues, highways, etc.), now we are working on the original shapefile, > updating new roads, changes made to highways due to constructions > modifications, etc., and we want to upload the "updated" shapefile, without > afecting the actual database, another words, we would like to convert to > postgres the updated shapefile with the new information (it cuold be the > whole shapefile) but to keep the added clasiffication column intact. How can > we do that? > > Thanks Well hopefully you have some key field for match the old data to the new data. I would load the new data into a new table and then add a classification column to the new table and then update it based on the common key values. update new_table set class='highway' where uid in ((select uid from old_table where class='highway')); and repeat for each class type you have. There is probably a better way to fo this with a join of some kind, but I'll let other chime in on that :) if the new_table looks good you can rename it in a transaction like: begin; alter table old_table rename to archive_table; alter table new_table rename to old_table; commit; and you will be using the new table. -Steve W. _______________________________________________ postgis-users mailing list postgis-users@... http://postgis.refractions.net/mailman/listinfo/postgis-users |
|
|
Re: Updated shapefile to update postgres databaseThanks! Thats exactly what I want to do, but just don't know how to. I don't pretend for you to give me a step by step indication but Ill apreciate some instructions since Im really new to this and im eager to learn more about it. Thanks again!
|
|
|
RE: Updated shapefile to update postgres databaseSenTnel,
To add to what Chris and Steve W. said there are a couple of ways to do this. 1) If you have some uniquely identifiable attribute fields you can use, then that is the best approach 2) If you are basing simply on geometry, then you will need to decide how different enough geometries can be to be considered the same road. Basic approach 1) Import new shape file into a temp table, put indexes on identifying attributes 2) update existing roads 3) add new roads The attribute update way - you can do as Steve W. had mentioned or you can do with a 2 table update statement So lets say you have 2 fields road_name, road_dir that uniquely identifies a road, then you can do something like --update existing roads UPDATE currenttable SET the_geom = t.the_geom, field2 = t.field2, field2 = t.field3 , etc.. FROM temptable As t WHERE t.road_name = currenttable.road_name AND t.road_dir = currenttable.road_dir To add new records, you would do --For the below this only works if you insure you have no nulls in road_name of currenttable (and ideally you have a unique index on the field or set of fields you will use for identification INSERT INTO currenttable(the_geom, road_name, road_dir, field2,field3, ...etc) SELECT t.the_geom, t.road_name, t.road_dir, t.field2, t.field3, .... FROM temptable As t LEFT JOIN currenttable c ON (t.road_name = c.road_name AND t.road_dir = c.road_dir) WHERE c.road_name IS NULL To do it based on geometry alone, that is kind of tricky and involves too much guess work e.g. using ST_Difference, ST_Intersection, ST_Distance etc. and comparing how different the lengths, areas are based on some tolerance. So I would suggest coming up with some scheme of uniquely tagging your roads if you don't have one already. Make sure to vacuum analyze currenttable after done - since this will create a ton of junk for many records. Hope that helps, Regina -----Original Message----- From: postgis-users-bounces@... [mailto:postgis-users-bounces@...] On Behalf Of SenTnel Sent: Wednesday, May 07, 2008 1:50 AM To: postgis-users@... Subject: Re: [postgis-users] Updated shapefile to update postgres database Thanks! Thats exactly what I want to do, but just don't know how to. I don't pretend for you to give me a step by step indication but Ill apreciate some instructions since Im really new to this and im eager to learn more about it. Thanks again! Chris Hermansen wrote: > > Presumably what you mean by "update" is that you want to replace old > versions of roads already in PostGIS with new versions of the same roads > from shapefiles, and you want to keep the old street type in conjunction > with the updated geometry? > > SenTnel wrote: >> Hello! >> >> I would like some help updating a postgres database. This is the problem: >> We >> created the database using shp2pgsql to convert a shapefile that contains >> a >> city's street details (centerline), after we created the database we also >> created an aditional column to classify the street types (eg: street, >> avenues, highways, etc.), now we are working on the original shapefile, >> updating new roads, changes made to highways due to constructions >> modifications, etc., and we want to upload the "updated" shapefile, >> without >> afecting the actual database, another words, we would like to convert to >> postgres the updated shapefile with the new information (it cuold be the >> whole shapefile) but to keep the added clasiffication column intact. How >> can >> we do that? >> >> Thanks >> > > > -- > Regards, > > Chris Hermansen mailto:chris.hermansen@... > tel+1.604.714.2878 * fax+1.604.733.0631 * mob+1.778.232.0644 > Timberline Natural Resource Group * http://www.timberline.ca > 401 * 958 West 8th Avenue * Vancouver BC * Canada * V5Z 1E5 > > _______________________________________________ > postgis-users mailing list > postgis-users@... > http://postgis.refractions.net/mailman/listinfo/postgis-users > > -- View this message in context: http://www.nabble.com/Updated-shapefile-to-update-postgres-database-tp17 053562p17097758.html Sent from the PostGIS - User mailing list archive at Nabble.com. _______________________________________________ postgis-users mailing list postgis-users@... http://postgis.refractions.net/mailman/listinfo/postgis-users ----------------------------------------- The substance of this message, including any attachments, may be confidential, legally privileged and/or exempt from disclosure pursuant to Massachusetts law. It is intended solely for the addressee. If you received this in error, please contact the sender and delete the material from any computer. _______________________________________________ postgis-users mailing list postgis-users@... http://postgis.refractions.net/mailman/listinfo/postgis-users |
|
|
Re: Updated shapefile to update postgres databaseWith respect to Regina's comments on the spatial approach, I think you
should try to answer another question as well. Is this a one-time exercise or will you be doing this regularly? Again, pursuing the spatial approach, the key thing is to determine how congruent the old and new road geometries are. If they are the same, except for the updates, this process should be more straightforward. However, if they are different - let's say the old ones are mapped with a zoom transfer scope or from satellite and the new ones are GPSed, then I think you'll have centrelines that are near each other at best. In either case, buffering the old roads (which have the street type attribute) and intersecting the new roads is the key to the process. What you are hoping is that the new road falls generally within the old road's buffer (which is only a hope, and you may find some cases where that doesn't occur or where you have to make the buffer so large that it grabs other roads as well). When you do an st_intersects() to find the new roads that fall within the old road buffers you will get multiple roads that intersect. You will need to return st_intersection() of these new roads to get the part of their geometries that fall within the buffer. You will need to return st_length() of these partial geometries to get the length. So now it looks something like this: select new.road_id,st_length(st_intersection(new.the_geom)) as new_int_length, old.road_id,old.st_length(old.the_geom) as old_length,old.road_type from old,new where st_intersects(st_buffer(old.the_geom,10.0)); You'll have to experiment with that buffer distance (10.0 in my example above). If your street network is mostly orthogonal I would think you would get pretty good results with your buffer distance being less than half the average block length for example. You might want to put the results of the above query in a table. Again, if you're lucky you should be able to just select out the rows where the new_int_length is longest, or where the absolute value of new_int_length - old_length is minimum. Obe, Regina wrote: > SenTnel, > > To add to what Chris and Steve W. said there are a couple of ways to do > this. > > 1) If you have some uniquely identifiable attribute fields you can use, > then that is the best approach > 2) If you are basing simply on geometry, then you will need to decide > how different enough geometries can be to be considered the same road. > > Basic approach > 1) Import new shape file into a temp table, put indexes on identifying > attributes > 2) update existing roads > 3) add new roads > > The attribute update way - you can do as Steve W. had mentioned or you > can do with a 2 table update statement > > So lets say you have 2 fields road_name, road_dir that uniquely > identifies a road, then you can do something like > --update existing roads > UPDATE currenttable > SET the_geom = t.the_geom, field2 = t.field2, field2 = t.field3 > , etc.. > FROM temptable As t > WHERE t.road_name = currenttable.road_name AND t.road_dir = > currenttable.road_dir > > To add new records, you would do > --For the below this only works if you insure you have no nulls in > road_name of currenttable (and ideally you have a unique index on the > field or set of > fields you will use for identification > > INSERT INTO currenttable(the_geom, road_name, road_dir, field2,field3, > ...etc) > SELECT t.the_geom, t.road_name, t.road_dir, t.field2, t.field3, .... > FROM temptable As t LEFT JOIN currenttable c ON (t.road_name = > c.road_name AND t.road_dir = c.road_dir) > WHERE c.road_name IS NULL > > To do it based on geometry alone, that is kind of tricky and involves > too much guess work e.g. using ST_Difference, ST_Intersection, > ST_Distance etc. and comparing how different the lengths, areas are > based on some tolerance. So I would suggest coming up with some > scheme of uniquely tagging your roads if you don't have one already. > > Make sure to > > vacuum analyze currenttable > > after done - since this will create a ton of junk for many records. > > Hope that helps, > Regina > > -----Original Message----- > From: postgis-users-bounces@... > [mailto:postgis-users-bounces@...] On Behalf Of > SenTnel > Sent: Wednesday, May 07, 2008 1:50 AM > To: postgis-users@... > Subject: Re: [postgis-users] Updated shapefile to update postgres > database > > > Thanks! Thats exactly what I want to do, but just don't know how to. I > don't > pretend for you to give me a step by step indication but Ill apreciate > some > instructions since Im really new to this and im eager to learn more > about > it. Thanks again! > > > Chris Hermansen wrote: > >> Presumably what you mean by "update" is that you want to replace old >> versions of roads already in PostGIS with new versions of the same >> > roads > >> from shapefiles, and you want to keep the old street type in >> > conjunction > >> with the updated geometry? >> >> SenTnel wrote: >> >>> Hello! >>> >>> I would like some help updating a postgres database. This is the >>> > problem: > >>> We >>> created the database using shp2pgsql to convert a shapefile that >>> > contains > >>> a >>> city's street details (centerline), after we created the database we >>> > also > >>> created an aditional column to classify the street types (eg: street, >>> avenues, highways, etc.), now we are working on the original >>> > shapefile, > >>> updating new roads, changes made to highways due to constructions >>> modifications, etc., and we want to upload the "updated" shapefile, >>> without >>> afecting the actual database, another words, we would like to convert >>> > to > >>> postgres the updated shapefile with the new information (it cuold be >>> > the > >>> whole shapefile) but to keep the added clasiffication column intact. >>> > How > >>> can >>> we do that? >>> >>> Thanks >>> >>> >> -- >> Regards, >> >> Chris Hermansen mailto:chris.hermansen@... >> tel+1.604.714.2878 * fax+1.604.733.0631 * mob+1.778.232.0644 >> Timberline Natural Resource Group * http://www.timberline.ca >> 401 * 958 West 8th Avenue * Vancouver BC * Canada * V5Z 1E5 >> >> _______________________________________________ >> postgis-users mailing list >> postgis-users@... >> http://postgis.refractions.net/mailman/listinfo/postgis-users >> >> >> > > -- Regards, Chris Hermansen mailto:chris.hermansen@... tel+1.604.714.2878 · fax+1.604.733.0631 · mob+1.778.232.0644 Timberline Natural Resource Group · http://www.timberline.ca 401 · 958 West 8th Avenue · Vancouver BC · Canada · V5Z 1E5 _______________________________________________ postgis-users mailing list postgis-users@... http://postgis.refractions.net/mailman/listinfo/postgis-users |
|
|
RE: Updated shapefile to update postgres databaseThanks for that info William.
Best regards, Martin -----Original Message----- From: William Kyngesburye [mailto:woklist@...] Sent: Tuesday, May 06, 2008 5:56 PM To: PostGIS Users Discussion Cc: Martin Chapman; SenTnel Subject: Re: [postgis-users] Updated shapefile to update postgres database I was interested in a similar thing once, but the ogr2ogr option names are misleading. "-update" means add to an existing data source (ie the database in Postgres). Otherwise ogr2ogr will try to create a new database, and fail if it exists. "-append" means add to an existing layer (or, the table within the database) within a data source. Otherwise it will try to create a new table, and fail if it exists. Update will not replace any attributes or geometry for a feature that already exists with the same FID, but will stop with a duplicate key error. On May 6, 2008, at 6:06 PM, Martin Chapman wrote: > Ogr2Ogr at http://www.gdal.org/ogr/ogr2ogr.html has append and update > options. As long as your new columns were added to the end of the > table it > will probably will work fine. > > Best regards, > Martin > > > -----Original Message----- > From: postgis-users-bounces@... > [mailto:postgis-users-bounces@...] On Behalf Of > SenTnel > Sent: Tuesday, May 06, 2008 4:40 PM > To: postgis-users@... > Subject: [postgis-users] Updated shapefile to update postgres database > > > Hello! > > I would like some help updating a postgres database. This is the > problem: We > created the database using shp2pgsql to convert a shapefile that > contains a > city's street details (centerline), after we created the database we > also > created an aditional column to classify the street types (eg: street, > avenues, highways, etc.), now we are working on the original > shapefile, > updating new roads, changes made to highways due to constructions > modifications, etc., and we want to upload the "updated" shapefile, > without > afecting the actual database, another words, we would like to > convert to > postgres the updated shapefile with the new information (it cuold be > the > whole shapefile) but to keep the added clasiffication column intact. > How can > we do that? > > Thanks > -- > View this message in context: > > 62p17053562.html > Sent from the PostGIS - User mailing list archive at Nabble.com. > > _______________________________________________ > postgis-users mailing list > postgis-users@... > http://postgis.refractions.net/mailman/listinfo/postgis-users > > _______________________________________________ > postgis-users mailing list > postgis-users@... > http://postgis.refractions.net/mailman/listinfo/postgis-users ----- William Kyngesburye <kyngchaos*at*kyngchaos*dot*com> http://www.kyngchaos.com/ The equator is so long, it could encircle the earth completely once. _______________________________________________ postgis-users mailing list postgis-users@... http://postgis.refractions.net/mailman/listinfo/postgis-users |
|
|
Re: Updated shapefile to update postgres databaseThanks Regina, Chris and all of you for your help. Im truly newby but with such help Im ready to start the procedures, Ill keep you posted of my progress or else if some issue comes along that may require some light from you, Thanks again and I hope my problem/solution may be of help to many others,
Thanks again!
|
|
|
Re: Updated shapefile to update postgres databaseHello! Im confused about this issue:
I have a street (polyline) duplicated, one on top of the other, exact same features, same "the_geom", but one bears the street name (column:"street") "main street" and the other bears "N/A" as street name. Now my maps shows the street but the name that shows is "N/A", and I would like to delete the "N/A" polyline, how can i do basically this: delete from mystreets_table street = 'N/A' where 'N/A' = street 'main street'.the_geom (delete from mystreets table the street named 'N/A' where the_geom in 'N/A' equals the_geom in street "main street") another words, delete only the N/A streets that the_geom is same as "main street", I hope you understand, Thanks!
|
|
|
RE: Updated shapefile to update postgres databaseHere is a thought, but not sure how slow this will be
DELETE FROM mystreets_table WHERE street = 'N/A' AND EXISTS (SELECT n.street FROM mystreets_table n WHERE (n.street <> 'N/A' AND n.street > '') AND ST_Equals(n.the_geom, mystreets_table.the_geom)) The st_equals might be better to replace with ~= if you want to match exact vertices. I presume ~= is actually more efficient Hope that helps, Regina -----Original Message----- From: postgis-users-bounces@... [mailto:postgis-users-bounces@...] On Behalf Of SenTnel Sent: Saturday, June 07, 2008 2:25 AM To: postgis-users@... Subject: Re: [postgis-users] Updated shapefile to update postgres database Hello! Im confused about this issue: I have a street (polyline) duplicated, one on top of the other, exact same features, same "the_geom", but one bears the street name (column:"street") "main street" and the other bears "N/A" as street name. Now my maps shows the street but the name that shows is "N/A", and I would like to delete the "N/A" polyline, how can i do basically this: delete from mystreets_table street = 'N/A' where 'N/A' = street 'main street'.the_geom (delete from mystreets table the street named 'N/A' where the_geom in 'N/A' equals the_geom in street "main street") another words, delete only the N/A streets that the_geom is same as "main street", I hope you understand, Thanks! SenTnel wrote: > > Thanks Regina, Chris and all of you for your help. Im truly newby but > with such help Im ready to start the procedures, Ill keep you posted > of my progress or else if some issue comes along that may require some > light from you, Thanks again and I hope my problem/solution may be of > help to many others, > > Thanks again! > > > > SenTnel wrote: >> >> Hello! >> >> I would like some help updating a postgres database. This is the problem: >> We created the database using shp2pgsql to convert a shapefile that >> contains a city's street details (centerline), after we created the >> database we also created an aditional column to classify the street >> types >> (eg: street, avenues, highways, etc.), now we are working on the >> original shapefile, updating new roads, changes made to highways due >> to constructions modifications, etc., and we want to upload the "updated" >> shapefile, without afecting the actual database, another words, we >> would like to convert to postgres the updated shapefile with the new >> information (it cuold be the whole shapefile) but to keep the added >> clasiffication column intact. How can we do that? >> >> Thanks >> > > -- View this message in context: http://www.nabble.com/Updated-shapefile-to-update-postgres-database-tp170535 62p17705777.html Sent from the PostGIS - User mailing list archive at Nabble.com. _______________________________________________ postgis-users mailing list postgis-users@... http://postgis.refractions.net/mailman/listinfo/postgis-users _______________________________________________ postgis-users mailing list postgis-users@... http://postgis.refractions.net/mailman/listinfo/postgis-users |
|
|
RE: Updated shapefile to update postgres databaseHello Regina! Thanks for your help, but when I try running it I got this error message:
ERROR: function st_equals(geometry, geometry) does not exist HINT: No function matches the given name and argument types. You may need to add explicit type casts. ![]() Thanks!
|
|
|
RE: Updated shapefile to update postgres databaseI wonder if its just called Equals in the old version.
Try doing just ~=. If your geometries are exactly the same that will probably be better anyway DELETE FROM mystreets_table WHERE street = 'N/A' AND EXISTS (SELECT n.street FROM mystreets_table n WHERE n.street <> 'N/A' AND n.street > '' AND n.the_geom ~= mystreets_table.the_geom) Hope that helps. Regina -----Original Message----- From: postgis-users-bounces@... [mailto:postgis-users-bounces@...] On Behalf Of SenTnel Sent: Saturday, June 07, 2008 4:36 PM To: postgis-users@... Subject: RE: [postgis-users] Updated shapefile to update postgres database Hello Regina! Thanks for your help, but when I try running it I got this error message: ERROR: function st_equals(geometry, geometry) does not exist HINT: No function matches the given name and argument types. You may need to add explicit type casts. :confused: Thanks! Paragon Corporation-2 wrote: > > Here is a thought, but not sure how slow this will be > > DELETE FROM mystreets_table > WHERE street = 'N/A' AND > EXISTS (SELECT n.street FROM mystreets_table n WHERE (n.street <> 'N/A' > AND > n.street > '') AND ST_Equals(n.the_geom, mystreets_table.the_geom)) > > The st_equals might be better to replace with ~= if you want to match > exact vertices. I presume ~= is actually more efficient > > Hope that helps, > Regina > > -----Original Message----- > From: postgis-users-bounces@... > [mailto:postgis-users-bounces@...] On Behalf Of > SenTnel > Sent: Saturday, June 07, 2008 2:25 AM > To: postgis-users@... > Subject: Re: [postgis-users] Updated shapefile to update postgres > database > > > Hello! Im confused about this issue: > > I have a street (polyline) duplicated, one on top of the other, exact > same features, same "the_geom", but one bears the street name > (column:"street") "main street" and the other bears "N/A" as street > name. Now my maps shows the street but the name that shows is "N/A", > and I would like to delete the "N/A" polyline, how can i do basically > this: > > delete from mystreets_table street = 'N/A' where 'N/A' = street 'main > street'.the_geom (delete from mystreets table the street named 'N/A' > where the_geom in 'N/A' > equals the_geom in street "main street") another words, delete only > the N/A streets that the_geom is same as "main street", I hope you > understand, > > Thanks! > > > > > SenTnel wrote: >> >> Thanks Regina, Chris and all of you for your help. Im truly newby but >> with such help Im ready to start the procedures, Ill keep you posted >> of my progress or else if some issue comes along that may require >> some light from you, Thanks again and I hope my problem/solution may >> be of help to many others, >> >> Thanks again! >> >> >> >> SenTnel wrote: >>> >>> Hello! >>> >>> I would like some help updating a postgres database. This is the >>> problem: >>> We created the database using shp2pgsql to convert a shapefile that >>> contains a city's street details (centerline), after we created the >>> database we also created an aditional column to classify the street >>> types >>> (eg: street, avenues, highways, etc.), now we are working on the >>> original shapefile, updating new roads, changes made to highways due >>> to constructions modifications, etc., and we want to upload the >>> "updated" >>> shapefile, without afecting the actual database, another words, we >>> would like to convert to postgres the updated shapefile with the new >>> information (it cuold be the whole shapefile) but to keep the added >>> clasiffication column intact. How can we do that? >>> >>> Thanks >>> >> >> > > -- > View this message in context: > http://www.nabble.com/Updated-shapefile-to-update-postgres-database-tp > 170535 > 62p17705777.html > Sent from the PostGIS - User mailing list archive at Nabble.com. > > _______________________________________________ > postgis-users mailing list > postgis-users@... > http://postgis.refractions.net/mailman/listinfo/postgis-users > > > > _______________________________________________ > postgis-users mailing list > postgis-users@... > http://postgis.refractions.net/mailman/listinfo/postgis-users > > -- View this message in context: http://www.nabble.com/Updated-shapefile-to-update-postgres-database-tp170535 62p17712835.html Sent from the PostGIS - User mailing list archive at Nabble.com. _______________________________________________ postgis-users mailing list postgis-users@... http://postgis.refractions.net/mailman/listinfo/postgis-users _______________________________________________ postgis-users mailing list postgis-users@... http://postgis.refractions.net/mailman/listinfo/postgis-users |
|
|
RE: Updated shapefile to update postgres databaseHi Regina!
Sorry took me so long to reply to this post, crazy week around here, but thank god there's always a weekend to look forward! This week I tried this method twice but finally had to stop it. First time I stopped the query after 4 hours thinking there was something wrong. Two nights later I tried again, left it running all night, next day, almost 10 hours later the query was running still. My table has 123,000+ rows, and just want to ask you if for such size is normal to take that long, if so I'll try again, if not, if you don't think it should take that long then you have any other suggestion? Thanks again!
|
|
|
RE: Updated shapefile to update postgres databaseIt shouldn't take that long I don't think. 123,000 records is not a lot.
Granted this is a suboptimal delete. First thing to verify 1) Make sure to have a btree index on street 2) I assume you have a gist index on your the_geom field already. If not that could be the culprit. 3) What are your postgresql.conf settings like for work_mem etc. If these are at there defuats, then that would be a problem too. The other alternative is to do with an IN instead of an EXISTS. That would at least not be correlated, but I'm not clear if an the_geom IN (SELECT ....) Would match up correctly since I suspect it may use = for compare Hope that helps, Regina -----Original Message----- From: postgis-users-bounces@... [mailto:postgis-users-bounces@...] On Behalf Of SenTnel Sent: Sunday, June 15, 2008 1:34 AM To: postgis-users@... Subject: RE: [postgis-users] Updated shapefile to update postgres database Hi Regina! Sorry took me so long to reply to this post, crazy week around here, but thank god there's always a weekend to look forward! This week I tried this method twice but finally had to stop it. First time I stopped the query after 4 hours thinking there was something wrong. Two nights later I tried again, left it running all night, next day, almost 10 hours later the query was running still. My table has 123,000+ rows, and just want to ask you if for such size is normal to take that long, if so I'll try again, if not, if you don't think it should take that long then you have any other suggestion? Thanks again! Paragon Corporation-2 wrote: > > I wonder if its just called Equals in the old version. > > Try doing just ~=. If your geometries are exactly the same that will > probably be better anyway > > DELETE FROM mystreets_table > WHERE street = 'N/A' AND > EXISTS (SELECT n.street FROM mystreets_table n WHERE n.street <> 'N/A' > AND n.street > '' AND n.the_geom ~= mystreets_table.the_geom) > > Hope that helps. > Regina > > -----Original Message----- > From: postgis-users-bounces@... > [mailto:postgis-users-bounces@...] On Behalf Of > SenTnel > Sent: Saturday, June 07, 2008 4:36 PM > To: postgis-users@... > Subject: RE: [postgis-users] Updated shapefile to update postgres > database > > > Hello Regina! Thanks for your help, but when I try running it I got > this error message: > > > ERROR: function st_equals(geometry, geometry) does not exist > HINT: No function matches the given name and argument types. You may > need to add explicit type casts. > > :confused: > Thanks! > > > > Paragon Corporation-2 wrote: >> >> Here is a thought, but not sure how slow this will be >> >> DELETE FROM mystreets_table >> WHERE street = 'N/A' AND >> EXISTS (SELECT n.street FROM mystreets_table n WHERE (n.street <> 'N/A' >> AND >> n.street > '') AND ST_Equals(n.the_geom, mystreets_table.the_geom)) >> >> The st_equals might be better to replace with ~= if you want to >> match exact vertices. I presume ~= is actually more efficient >> >> Hope that helps, >> Regina >> >> -----Original Message----- >> From: postgis-users-bounces@... >> [mailto:postgis-users-bounces@...] On Behalf Of >> SenTnel >> Sent: Saturday, June 07, 2008 2:25 AM >> To: postgis-users@... >> Subject: Re: [postgis-users] Updated shapefile to update postgres >> database >> >> >> Hello! Im confused about this issue: >> >> I have a street (polyline) duplicated, one on top of the other, exact >> same features, same "the_geom", but one bears the street name >> (column:"street") "main street" and the other bears "N/A" as street >> name. Now my maps shows the street but the name that shows is "N/A", >> and I would like to delete the "N/A" polyline, how can i do basically >> this: >> >> delete from mystreets_table street = 'N/A' where 'N/A' = street 'main >> street'.the_geom (delete from mystreets table the street named 'N/A' >> where the_geom in 'N/A' >> equals the_geom in street "main street") another words, delete only >> the N/A streets that the_geom is same as "main street", I hope you >> understand, >> >> Thanks! >> >> >> >> >> SenTnel wrote: >>> >>> Thanks Regina, Chris and all of you for your help. Im truly newby >>> but with such help Im ready to start the procedures, Ill keep you >>> posted of my progress or else if some issue comes along that may >>> require some light from you, Thanks again and I hope my >>> problem/solution may be of help to many others, >>> >>> Thanks again! >>> >>> >>> >>> SenTnel wrote: >>>> >>>> Hello! >>>> >>>> I would like some help updating a postgres database. This is the >>>> problem: >>>> We created the database using shp2pgsql to convert a shapefile that >>>> contains a city's street details (centerline), after we created the >>>> database we also created an aditional column to classify the street >>>> types >>>> (eg: street, avenues, highways, etc.), now we are working on the >>>> original shapefile, updating new roads, changes made to highways >>>> due to constructions modifications, etc., and we want to upload the >>>> "updated" >>>> shapefile, without afecting the actual database, another words, we >>>> would like to convert to postgres the updated shapefile with the >>>> new information (it cuold be the whole shapefile) but to keep the >>>> added clasiffication column intact. How can we do that? >>>> >>>> Thanks >>>> >>> >>> >> >> -- >> View this message in context: >> http://www.nabble.com/Updated-shapefile-to-update-postgres-database-t >> p >> 170535 >> 62p17705777.html >> Sent from the PostGIS - User mailing list archive at Nabble.com. >> >> _______________________________________________ >> postgis-users mailing list >> postgis-users@... >> http://postgis.refractions.net/mailman/listinfo/postgis-users >> >> >> >> _______________________________________________ >> postgis-users mailing list >> postgis-users@... >> http://postgis.refractions.net/mailman/listinfo/postgis-users >> >> > > -- > View this message in context: > http://www.nabble.com/Updated-shapefile-to-update-postgres-database-tp > 170535 > 62p17712835.html > Sent from the PostGIS - User mailing list archive at Nabble.com. > > _______________________________________________ > postgis-users mailing list > postgis-users@... > http://postgis.refractions.net/mailman/listinfo/postgis-users > > > > _______________________________________________ > postgis-users mailing list > postgis-users@... > http://postgis.refractions.net/mailman/listinfo/postgis-users > > -- View this message in context: http://www.nabble.com/Updated-shapefile-to-update-postgres-database-tp170535 62p17846638.html Sent from the PostGIS - User mailing list archive at Nabble.com. _______________________________________________ postgis-users mailing list postgis-users@... http://postgis.refractions.net/mailman/listinfo/postgis-users _______________________________________________ postgis-users mailing list postgis-users@... http://postgis.refractions.net/mailman/listinfo/postgis-users |
|
|
RE: Updated shapefile to update postgres databaseForgot make sure you
Vacuum analyze yourtable After you create indexes and load lots of data. -----Original Message----- From: postgis-users-bounces@... [mailto:postgis-users-bounces@...] On Behalf Of SenTnel Sent: Sunday, June 15, 2008 1:34 AM To: postgis-users@... Subject: RE: [postgis-users] Updated shapefile to update postgres database Hi Regina! Sorry took me so long to reply to this post, crazy week around here, but thank god there's always a weekend to look forward! This week I tried this method twice but finally had to stop it. First time I stopped the query after 4 hours thinking there was something wrong. Two nights later I tried again, left it running all night, next day, almost 10 hours later the query was running still. My table has 123,000+ rows, and just want to ask you if for such size is normal to take that long, if so I'll try again, if not, if you don't think it should take that long then you have any other suggestion? Thanks again! Paragon Corporation-2 wrote: > > I wonder if its just called Equals in the old version. > > Try doing just ~=. If your geometries are exactly the same that will > probably be better anyway > > DELETE FROM mystreets_table > WHERE street = 'N/A' AND > EXISTS (SELECT n.street FROM mystreets_table n WHERE n.street <> 'N/A' > AND n.street > '' AND n.the_geom ~= mystreets_table.the_geom) > > Hope that helps. > Regina > > -----Original Message----- > From: postgis-users-bounces@... > [mailto:postgis-users-bounces@...] On Behalf Of > SenTnel > Sent: Saturday, June 07, 2008 4:36 PM > To: postgis-users@... > Subject: RE: [postgis-users] Updated shapefile to update postgres > database > > > Hello Regina! Thanks for your help, but when I try running it I got > this error message: > > > ERROR: function st_equals(geometry, geometry) does not exist > HINT: No function matches the given name and argument types. You may > need to add explicit type casts. > > :confused: > Thanks! > > > > Paragon Corporation-2 wrote: >> >> Here is a thought, but not sure how slow this will be >> >> DELETE FROM mystreets_table >> WHERE street = 'N/A' AND >> EXISTS (SELECT n.street FROM mystreets_table n WHERE (n.street <> 'N/A' >> AND >> n.street > '') AND ST_Equals(n.the_geom, mystreets_table.the_geom)) >> >> The st_equals might be better to replace with ~= if you want to >> match exact vertices. I presume ~= is actually more efficient >> >> Hope that helps, >> Regina >> >> -----Original Message----- >> From: postgis-users-bounces@... >> [mailto:postgis-users-bounces@...] On Behalf Of >> SenTnel >> Sent: Saturday, June 07, 2008 2:25 AM >> To: postgis-users@... >> Subject: Re: [postgis-users] Updated shapefile to update postgres >> database >> >> >> Hello! Im confused about this issue: >> >> I have a street (polyline) duplicated, one on top of the other, exact >> same features, same "the_geom", but one bears the street name >> (column:"street") "main street" and the other bears "N/A" as street >> name. Now my maps shows the street but the name that shows is "N/A", >> and I would like to delete the "N/A" polyline, how can i do basically >> this: >> >> delete from mystreets_table street = 'N/A' where 'N/A' = street 'main >> street'.the_geom (delete from mystreets table the street named 'N/A' >> where the_geom in 'N/A' >> equals the_geom in street "main street") another words, delete only >> the N/A streets that the_geom is same as "main street", I hope you >> understand, >> >> Thanks! >> >> >> >> >> SenTnel wrote: >>> >>> Thanks Regina, Chris and all of you for your help. Im truly newby >>> but with such help Im ready to start the procedures, Ill keep you >>> posted of my progress or else if some issue comes along that may >>> require some light from you, Thanks again and I hope my >>> problem/solution may be of help to many others, >>> >>> Thanks again! >>> >>> >>> >>> SenTnel wrote: >>>> >>>> Hello! >>>> >>>> I would like some help updating a postgres database. This is the >>>> problem: >>>> We created the database using shp2pgsql to convert a shapefile that >>>> contains a city's street details (centerline), after we created the >>>> database we also created an aditional column to classify the street >>>> types >>>> (eg: street, avenues, highways, etc.), now we are working on the >>>> original shapefile, updating new roads, changes made to highways >>>> due to constructions modifications, etc., and we want to upload the >>>> "updated" >>>> shapefile, without afecting the actual database, another words, we >>>> would like to convert to postgres the updated shapefile with the >>>> new information (it cuold be the whole shapefile) but to keep the >>>> added clasiffication column intact. How can we do that? >>>> >>>> Thanks >>>> >>> >>> >> >> -- >> View this message in context: >> http://www.nabble.com/Updated-shapefile-to-update-postgres-database-t >> p >> 170535 >> 62p17705777.html >> Sent from the PostGIS - User mailing list archive at Nabble.com. >> >> _______________________________________________ >> postgis-users mailing list >> postgis-users@... >> http://postgis.refractions.net/mailman/listinfo/postgis-users >> >> >> >> _______________________________________________ >> postgis-users mailing list >> postgis-users@... >> http://postgis.refractions.net/mailman/listinfo/postgis-users >> >> > > -- > View this message in context: > http://www.nabble.com/Updated-shapefile-to-update-postgres-database-tp > 170535 > 62p17712835.html > Sent from the PostGIS - User mailing list archive at Nabble.com. > > _______________________________________________ > postgis-users mailing list > postgis-users@... > http://postgis.refractions.net/mailman/listinfo/postgis-users > > > > _______________________________________________ > postgis-users mailing list > postgis-users@... > http://postgis.refractions.net/mailman/listinfo/postgis-users > > -- View this message in context: http://www.nabble.com/Updated-shapefile-to-update-postgres-database-tp170535 62p17846638.html Sent from the PostGIS - User mailing list archive at Nabble.com. _______________________________________________ postgis-users mailing list postgis-users@... http://postgis.refractions.net/mailman/listinfo/postgis-users _______________________________________________ postgis-users mailing list postgis-users@... http://postgis.refractions.net/mailman/listinfo/postgis-users |
|
|
RE: Updated shapefile to update postgres databaseThanks Regina for your help! I must tell you that the worst part of this is not knowing what most of what you tell me means!
Im trying to become familiar with this but don't know much... Totally ignore btree, gist index, but I'll google it to see what I can find. As far as the configuration it is the installation's default, suggestions are welcome!
... again... Thanks!
|
|
|
RE: Updated shapefile to update postgres databaseTake a look at our article on the subject if you haven't already.
It covers how to create gist and btree indexes http://www.bostongis.com/PrinterFriendly.aspx?content_name=postgis_tut01 Samples exist in the postgis docs too. -----Original Message----- From: postgis-users-bounces@... [mailto:postgis-users-bounces@...] On Behalf Of SenTnel Sent: Sunday, June 15, 2008 4:37 AM To: postgis-users@... Subject: RE: [postgis-users] Updated shapefile to update postgres database Thanks Regina for your help! I must tell you that the worst part of this is not knowing what most of what you tell me means! :confused: Im trying to become familiar with this but don't know much... Totally ignore btree, gist index, but I'll google it to see what I can find. As far as the configuration it is the installation's default, suggestions are welcome! =^D ... again... Thanks! Paragon Corporation-2 wrote: > > It shouldn't take that long I don't think. 123,000 records is not a lot. > Granted this is a suboptimal delete. > > First thing to verify > > 1) Make sure to have a btree index on street > 2) I assume you have a gist index on your the_geom field already. If > not that could be the culprit. > 3) What are your postgresql.conf settings like for work_mem etc. If > these are at there defuats, then that would be a problem too. > > The other alternative is to do with an IN instead of an EXISTS. That > would at least not be correlated, but I'm not clear if an > > the_geom IN (SELECT ....) > > Would match up correctly since I suspect it may use = for compare > > Hope that helps, > Regina > > > > -----Original Message----- > From: postgis-users-bounces@... > [mailto:postgis-users-bounces@...] On Behalf Of > SenTnel > Sent: Sunday, June 15, 2008 1:34 AM > To: postgis-users@... > Subject: RE: [postgis-users] Updated shapefile to update postgres > database > > > Hi Regina! > > Sorry took me so long to reply to this post, crazy week around here, > but thank god there's always a weekend to look forward! > > This week I tried this method twice but finally had to stop it. First > time I stopped the query after 4 hours thinking there was something > wrong. Two nights later I tried again, left it running all night, next > day, almost 10 hours later the query was running still. My table has > 123,000+ rows, and just want to ask you if for such size is normal to > take that long, if so I'll try again, if not, if you don't think it > should take that long then you have any other suggestion? > > Thanks again! > > > > Paragon Corporation-2 wrote: >> >> I wonder if its just called Equals in the old version. >> >> Try doing just ~=. If your geometries are exactly the same that will >> probably be better anyway >> >> DELETE FROM mystreets_table >> WHERE street = 'N/A' AND >> EXISTS (SELECT n.street FROM mystreets_table n WHERE n.street <> 'N/A' >> AND n.street > '' AND n.the_geom ~= mystreets_table.the_geom) >> >> Hope that helps. >> Regina >> >> -----Original Message----- >> From: postgis-users-bounces@... >> [mailto:postgis-users-bounces@...] On Behalf Of >> SenTnel >> Sent: Saturday, June 07, 2008 4:36 PM >> To: postgis-users@... >> Subject: RE: [postgis-users] Updated shapefile to update postgres >> database >> >> >> Hello Regina! Thanks for your help, but when I try running it I got >> this error message: >> >> >> ERROR: function st_equals(geometry, geometry) does not exist >> HINT: No function matches the given name and argument types. You may >> need to add explicit type casts. >> >> :confused: >> Thanks! >> >> >> >> Paragon Corporation-2 wrote: >>> >>> Here is a thought, but not sure how slow this will be >>> >>> DELETE FROM mystreets_table >>> WHERE street = 'N/A' AND >>> EXISTS (SELECT n.street FROM mystreets_table n WHERE (n.street <> 'N/A' >>> AND >>> n.street > '') AND ST_Equals(n.the_geom, mystreets_table.the_geom)) >>> >>> The st_equals might be better to replace with ~= if you want to >>> match exact vertices. I presume ~= is actually more efficient >>> >>> Hope that helps, >>> Regina >>> >>> -----Original Message----- >>> From: postgis-users-bounces@... >>> [mailto:postgis-users-bounces@...] On Behalf Of >>> SenTnel >>> Sent: Saturday, June 07, 2008 2:25 AM >>> To: postgis-users@... >>> Subject: Re: [postgis-users] Updated shapefile to update postgres >>> database >>> >>> >>> Hello! Im confused about this issue: >>> >>> I have a street (polyline) duplicated, one on top of the other, >>> exact same features, same "the_geom", but one bears the street name >>> (column:"street") "main street" and the other bears "N/A" as street >>> name. Now my maps shows the street but the name that shows is "N/A", >>> and I would like to delete the "N/A" polyline, how can i do >>> basically >>> this: >>> >>> delete from mystreets_table street = 'N/A' where 'N/A' = street >>> 'main street'.the_geom (delete from mystreets table the street named >>> where the_geom in 'N/A' >>> equals the_geom in street "main street") another words, delete only >>> the N/A streets that the_geom is same as "main street", I hope you >>> understand, >>> >>> Thanks! >>> >>> >>> >>> >>> SenTnel wrote: >>>> >>>> Thanks Regina, Chris and all of you for your help. Im truly newby >>>> but with such help Im ready to start the procedures, Ill keep you >>>> posted of my progress or else if some issue comes along that may >>>> require some light from you, Thanks again and I hope my >>>> problem/solution may be of help to many others, >>>> >>>> Thanks again! >>>> >>>> >>>> >>>> SenTnel wrote: >>>>> >>>>> Hello! >>>>> >>>>> I would like some help updating a postgres database. This is the >>>>> problem: >>>>> We created the database using shp2pgsql to convert a shapefile >>>>> that contains a city's street details (centerline), after we >>>>> created the database we also created an aditional column to >>>>> classify the street types >>>>> (eg: street, avenues, highways, etc.), now we are working on the >>>>> original shapefile, updating new roads, changes made to highways >>>>> due to constructions modifications, etc., and we want to upload >>>>> the "updated" >>>>> shapefile, without afecting the actual database, another words, we >>>>> would like to convert to postgres the updated shapefile with the >>>>> new information (it cuold be the whole shapefile) but to keep the >>>>> added clasiffication column intact. How can we do that? >>>>> >>>>> Thanks >>>>> >>>> >>>> >>> >>> -- >>> View this message in context: >>> http://www.nabble.com/Updated-shapefile-to-update-postgres-database- >>> t >>> p >>> 170535 >>> 62p17705777.html >>> Sent from the PostGIS - User mailing list archive at Nabble.com. >>> >>> _______________________________________________ >>> postgis-users mailing list >>> postgis-users@... >>> http://postgis.refractions.net/mailman/listinfo/postgis-users >>> >>> >>> >>> _______________________________________________ >>> postgis-users mailing list >>> postgis-users@... >>> http://postgis.refractions.net/mailman/listinfo/postgis-users >>> >>> >> >> -- >> View this message in context: >> http://www.nabble.com/Updated-shapefile-to-update-postgres-database-t >> p >> 170535 >> 62p17712835.html >> Sent from the PostGIS - User mailing list archive at Nabble.com. >> >> _______________________________________________ >> postgis-users mailing list >> postgis-users@... >> http://postgis.refractions.net/mailman/listinfo/postgis-users >> >> >> >> _______________________________________________ >> postgis-users mailing list >> postgis-users@... >> http://postgis.refractions.net/mailman/listinfo/postgis-users >> >> > > -- > View this message in context: > http://www.nabble.com/Updated-shapefile-to-update-postgres-database-tp > 170535 > 62p17846638.html > Sent from the PostGIS - User mailing list archive at Nabble.com. > > _______________________________________________ > postgis-users mailing list > postgis-users@... > http://postgis.refractions.net/mailman/listinfo/postgis-users > > > _______________________________________________ > postgis-users mailing list > postgis-users@... > http://postgis.refractions.net/mailman/listinfo/postgis-users > > -- View this message in context: http://www.nabble.com/Updated-shapefile-to-update-postgres-database-tp170535 62p17847480.html Sent from the PostGIS - User mailing list archive at Nabble.com. _______________________________________________ postgis-users mailing list postgis-users@... http://postgis.refractions.net/mailman/listinfo/postgis-users _______________________________________________ postgis-users mailing list postgis-users@... http://postgis.refractions.net/mailman/listinfo/postgis-users |
| Free embeddable forum powered by Nabble | Forum Help |