Updated shapefile to update postgres database

View: New views
19 Messages — Rating Filter:   Alert me  

Updated shapefile to update postgres database

by SenTnel :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Hello!

I would like some help updating a postgres database. This is the problem: We created the database using shp2pgsql to convert a shapefile that contains a city's street details (centerline), after we created the database we also created an aditional column to classify the street types (eg: street, avenues, highways, etc.), now we are working on the original shapefile, updating new roads, changes made to highways due to constructions modifications, etc., and we want to upload the "updated" shapefile, without afecting the actual database, another words, we would like to convert to postgres the updated shapefile with the new information (it cuold be the whole shapefile) but to keep the added clasiffication column intact. How can we do that?

Thanks

Re: Updated shapefile to update postgres database

by Chris Hermansen :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Presumably what you mean by "update" is that you want to replace old
versions of roads already in PostGIS with new versions of the same roads
from shapefiles, and you want to keep the old street type in conjunction
with the updated geometry?

If that's the case, your problem is really determining which old roads
are coincident with which new roads.  One approach might be to buffer
the new road by a metre or two, and then select the old road whose
length within the buffer is the maximum of all old roads that intersect
the buffer.  This specific old road's street type could be transferred
to the new road, and the old road could be deleted.

The above is probably an oversimplification that will crash and burn
when confronted by real live roads whose updates are not particularly
coincident with their older avatars.

SenTnel wrote:

> Hello!
>
> I would like some help updating a postgres database. This is the problem: We
> created the database using shp2pgsql to convert a shapefile that contains a
> city's street details (centerline), after we created the database we also
> created an aditional column to classify the street types (eg: street,
> avenues, highways, etc.), now we are working on the original shapefile,
> updating new roads, changes made to highways due to constructions
> modifications, etc., and we want to upload the "updated" shapefile, without
> afecting the actual database, another words, we would like to convert to
> postgres the updated shapefile with the new information (it cuold be the
> whole shapefile) but to keep the added clasiffication column intact. How can
> we do that?
>
> Thanks
>  


--
Regards,

Chris Hermansen         mailto:chris.hermansen@...
tel+1.604.714.2878 · fax+1.604.733.0631 · mob+1.778.232.0644
Timberline Natural Resource Group · http://www.timberline.ca
401 · 958 West 8th Avenue  · Vancouver BC · Canada · V5Z 1E5

_______________________________________________
postgis-users mailing list
postgis-users@...
http://postgis.refractions.net/mailman/listinfo/postgis-users

RE: Updated shapefile to update postgres database

by Martin Chapman-2 :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Ogr2Ogr at http://www.gdal.org/ogr/ogr2ogr.html has append and update
options.  As long as your new columns were added to the end of the table it
will probably will work fine.

Best regards,
Martin


-----Original Message-----
From: postgis-users-bounces@...
[mailto:postgis-users-bounces@...] On Behalf Of SenTnel
Sent: Tuesday, May 06, 2008 4:40 PM
To: postgis-users@...
Subject: [postgis-users] Updated shapefile to update postgres database


Hello!

I would like some help updating a postgres database. This is the problem: We
created the database using shp2pgsql to convert a shapefile that contains a
city's street details (centerline), after we created the database we also
created an aditional column to classify the street types (eg: street,
avenues, highways, etc.), now we are working on the original shapefile,
updating new roads, changes made to highways due to constructions
modifications, etc., and we want to upload the "updated" shapefile, without
afecting the actual database, another words, we would like to convert to
postgres the updated shapefile with the new information (it cuold be the
whole shapefile) but to keep the added clasiffication column intact. How can
we do that?

Thanks
--
View this message in context:
http://www.nabble.com/Updated-shapefile-to-update-postgres-database-tp170535
62p17053562.html
Sent from the PostGIS - User mailing list archive at Nabble.com.

_______________________________________________
postgis-users mailing list
postgis-users@...
http://postgis.refractions.net/mailman/listinfo/postgis-users

_______________________________________________
postgis-users mailing list
postgis-users@...
http://postgis.refractions.net/mailman/listinfo/postgis-users

Re: Updated shapefile to update postgres database

by William Kyngesburye :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

I was interested in a similar thing once, but the ogr2ogr option names  
are misleading.

"-update" means add to an existing data source (ie the database in  
Postgres).  Otherwise ogr2ogr will try to create a new database, and  
fail if it exists.

"-append" means add to an existing layer (or, the table within the  
database) within a data source.  Otherwise it will try to create a new  
table, and fail if it exists.

Update will not replace any attributes or geometry for a feature that  
already exists with the same FID, but will stop with a duplicate key  
error.


On May 6, 2008, at 6:06 PM, Martin Chapman wrote:

> Ogr2Ogr at http://www.gdal.org/ogr/ogr2ogr.html has append and update
> options.  As long as your new columns were added to the end of the  
> table it
> will probably will work fine.
>
> Best regards,
> Martin
>
>
> -----Original Message-----
> From: postgis-users-bounces@...
> [mailto:postgis-users-bounces@...] On Behalf Of  
> SenTnel
> Sent: Tuesday, May 06, 2008 4:40 PM
> To: postgis-users@...
> Subject: [postgis-users] Updated shapefile to update postgres database
>
>
> Hello!
>
> I would like some help updating a postgres database. This is the  
> problem: We
> created the database using shp2pgsql to convert a shapefile that  
> contains a
> city's street details (centerline), after we created the database we  
> also
> created an aditional column to classify the street types (eg: street,
> avenues, highways, etc.), now we are working on the original  
> shapefile,
> updating new roads, changes made to highways due to constructions
> modifications, etc., and we want to upload the "updated" shapefile,  
> without
> afecting the actual database, another words, we would like to  
> convert to
> postgres the updated shapefile with the new information (it cuold be  
> the
> whole shapefile) but to keep the added clasiffication column intact.  
> How can
> we do that?
>
> Thanks
> --
> View this message in context:
> http://www.nabble.com/Updated-shapefile-to-update-postgres-database-tp170535
> 62p17053562.html
> Sent from the PostGIS - User mailing list archive at Nabble.com.
>
> _______________________________________________
> postgis-users mailing list
> postgis-users@...
> http://postgis.refractions.net/mailman/listinfo/postgis-users
>
> _______________________________________________
> postgis-users mailing list
> postgis-users@...
> http://postgis.refractions.net/mailman/listinfo/postgis-users

-----
William Kyngesburye <kyngchaos*at*kyngchaos*dot*com>
http://www.kyngchaos.com/

The equator is so long, it could encircle the earth completely once.

_______________________________________________
postgis-users mailing list
postgis-users@...
http://postgis.refractions.net/mailman/listinfo/postgis-users

Re: Updated shapefile to update postgres database

by Stephen Woodbridge :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

SenTnel wrote:

> Hello!
>
> I would like some help updating a postgres database. This is the problem: We
> created the database using shp2pgsql to convert a shapefile that contains a
> city's street details (centerline), after we created the database we also
> created an aditional column to classify the street types (eg: street,
> avenues, highways, etc.), now we are working on the original shapefile,
> updating new roads, changes made to highways due to constructions
> modifications, etc., and we want to upload the "updated" shapefile, without
> afecting the actual database, another words, we would like to convert to
> postgres the updated shapefile with the new information (it cuold be the
> whole shapefile) but to keep the added clasiffication column intact. How can
> we do that?
>
> Thanks

Well hopefully you have some key field for match the old data to the new
data.

I would load the new data into a new table and then add a classification
column to the new table and then update it based on the common key values.

update new_table set class='highway' where uid in ((select uid from
old_table where class='highway'));

and repeat for each class type you have.

There is probably a better way to fo this with a join of some kind, but
I'll let other chime in on that :)

if the new_table looks good you can rename it in a transaction like:

begin;
alter table old_table rename to archive_table;
alter table new_table rename to old_table;
commit;

and you will be using the new table.

-Steve W.
_______________________________________________
postgis-users mailing list
postgis-users@...
http://postgis.refractions.net/mailman/listinfo/postgis-users

Re: Updated shapefile to update postgres database

by SenTnel :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Thanks! Thats exactly what I want to do, but just don't know how to. I don't pretend for you to give me a step by step indication but Ill apreciate some instructions since Im really new to this and im eager to learn more about it. Thanks again!

Chris Hermansen wrote:
Presumably what you mean by "update" is that you want to replace old
versions of roads already in PostGIS with new versions of the same roads
from shapefiles, and you want to keep the old street type in conjunction
with the updated geometry?

SenTnel wrote:
> Hello!
>
> I would like some help updating a postgres database. This is the problem: We
> created the database using shp2pgsql to convert a shapefile that contains a
> city's street details (centerline), after we created the database we also
> created an aditional column to classify the street types (eg: street,
> avenues, highways, etc.), now we are working on the original shapefile,
> updating new roads, changes made to highways due to constructions
> modifications, etc., and we want to upload the "updated" shapefile, without
> afecting the actual database, another words, we would like to convert to
> postgres the updated shapefile with the new information (it cuold be the
> whole shapefile) but to keep the added clasiffication column intact. How can
> we do that?
>
> Thanks
>  


--
Regards,

Chris Hermansen         mailto:chris.hermansen@timberline.ca
tel+1.604.714.2878 · fax+1.604.733.0631 · mob+1.778.232.0644
Timberline Natural Resource Group · http://www.timberline.ca
401 · 958 West 8th Avenue  · Vancouver BC · Canada · V5Z 1E5

_______________________________________________
postgis-users mailing list
postgis-users@postgis.refractions.net
http://postgis.refractions.net/mailman/listinfo/postgis-users

RE: Updated shapefile to update postgres database

by Regina Obe :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

SenTnel,

To add to what Chris and Steve W. said there are a couple of ways to do
this.

1) If you have some uniquely identifiable attribute fields you can use,
then that is the best approach
2) If you are basing simply on geometry, then you will need to decide
how different enough geometries can be to be considered the same road.

Basic approach
1) Import new shape file into a temp table, put indexes on identifying
attributes
2) update existing roads
3) add new roads

The attribute update way - you can do as Steve W. had mentioned or you
can do with a 2 table update statement

So lets say you have 2 fields road_name, road_dir that uniquely
identifies a road, then you can do something like
--update existing roads
UPDATE currenttable
        SET the_geom = t.the_geom, field2 = t.field2, field2 = t.field3
, etc..
FROM temptable As t
WHERE t.road_name = currenttable.road_name AND t.road_dir =
currenttable.road_dir

To add new records, you would do
--For the below this only works if you insure you have no nulls in
road_name of currenttable (and ideally you have a unique index on the
field or set of
fields you will use for identification

INSERT INTO currenttable(the_geom, road_name, road_dir, field2,field3,
...etc)
SELECT t.the_geom, t.road_name, t.road_dir, t.field2, t.field3, ....
FROM temptable As t LEFT JOIN currenttable c ON (t.road_name =
c.road_name AND t.road_dir = c.road_dir)
WHERE c.road_name IS NULL
         
To do it based on geometry alone, that is kind of tricky and involves
too much guess work e.g. using ST_Difference, ST_Intersection,
ST_Distance etc. and comparing how different the lengths, areas are
based on some tolerance.  So I would suggest coming up with some
scheme of uniquely tagging your roads if you don't have one already.

Make sure to

vacuum analyze currenttable

after done - since this will create a ton of junk for many records.

Hope that helps,
Regina

-----Original Message-----
From: postgis-users-bounces@...
[mailto:postgis-users-bounces@...] On Behalf Of
SenTnel
Sent: Wednesday, May 07, 2008 1:50 AM
To: postgis-users@...
Subject: Re: [postgis-users] Updated shapefile to update postgres
database


Thanks! Thats exactly what I want to do, but just don't know how to. I
don't
pretend for you to give me a step by step indication but Ill apreciate
some
instructions since Im really new to this and im eager to learn more
about
it. Thanks again!


Chris Hermansen wrote:
>
> Presumably what you mean by "update" is that you want to replace old
> versions of roads already in PostGIS with new versions of the same
roads
> from shapefiles, and you want to keep the old street type in
conjunction
> with the updated geometry?
>
> SenTnel wrote:
>> Hello!
>>
>> I would like some help updating a postgres database. This is the
problem:
>> We
>> created the database using shp2pgsql to convert a shapefile that
contains
>> a
>> city's street details (centerline), after we created the database we
also
>> created an aditional column to classify the street types (eg: street,
>> avenues, highways, etc.), now we are working on the original
shapefile,
>> updating new roads, changes made to highways due to constructions
>> modifications, etc., and we want to upload the "updated" shapefile,
>> without
>> afecting the actual database, another words, we would like to convert
to
>> postgres the updated shapefile with the new information (it cuold be
the
>> whole shapefile) but to keep the added clasiffication column intact.
How

>> can
>> we do that?
>>
>> Thanks
>>  
>
>
> --
> Regards,
>
> Chris Hermansen         mailto:chris.hermansen@...
> tel+1.604.714.2878 * fax+1.604.733.0631 * mob+1.778.232.0644
> Timberline Natural Resource Group * http://www.timberline.ca
> 401 * 958 West 8th Avenue  * Vancouver BC * Canada * V5Z 1E5
>
> _______________________________________________
> postgis-users mailing list
> postgis-users@...
> http://postgis.refractions.net/mailman/listinfo/postgis-users
>
>

--
View this message in context:
http://www.nabble.com/Updated-shapefile-to-update-postgres-database-tp17
053562p17097758.html
Sent from the PostGIS - User mailing list archive at Nabble.com.

_______________________________________________
postgis-users mailing list
postgis-users@...
http://postgis.refractions.net/mailman/listinfo/postgis-users
-----------------------------------------
The substance of this message, including any attachments, may be
confidential, legally privileged and/or exempt from disclosure
pursuant to Massachusetts law. It is intended
solely for the addressee. If you received this in error, please
contact the sender and delete the material from any computer.

_______________________________________________
postgis-users mailing list
postgis-users@...
http://postgis.refractions.net/mailman/listinfo/postgis-users

Re: Updated shapefile to update postgres database

by Chris Hermansen :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

With respect to Regina's comments on the spatial approach, I think you
should try to answer another question as well.  Is this a one-time
exercise or will you be doing this regularly?

Again, pursuing the spatial approach, the key thing is to determine how
congruent the old and new road geometries are.  If they are the same,
except for the updates, this process should be more straightforward.  
However, if they are different - let's say the old ones are mapped with
a zoom transfer scope or from satellite and the new ones are GPSed, then
I think you'll have centrelines that are near each other at best.

In either case, buffering the old roads (which have the street type
attribute) and intersecting the new roads is the key to the process.  
What you are hoping is that the new road falls generally within the old
road's buffer (which is only a hope, and you may find some cases where
that doesn't occur or where you have to make the buffer so large that it
grabs other roads as well).

When you do an st_intersects() to find the new roads that fall within
the old road buffers you will get multiple roads that intersect.  You
will need to return st_intersection() of these new roads to get the part
of their geometries that fall within the buffer.  You will need to
return st_length() of these partial geometries to get the length.  So
now it looks something like this:

select
    new.road_id,st_length(st_intersection(new.the_geom)) as new_int_length,
    old.road_id,old.st_length(old.the_geom) as old_length,old.road_type
from old,new
where st_intersects(st_buffer(old.the_geom,10.0));

You'll have to experiment with that buffer distance (10.0 in my example
above).  If your street network is mostly orthogonal I would think you
would get pretty good results with your buffer distance being less than
half the average block length for example.

You might want to put the results of the above query in a table.

Again, if you're lucky you should be able to just select out the rows
where the new_int_length is longest, or where the absolute value of
new_int_length - old_length is minimum.

Obe, Regina wrote:

> SenTnel,
>
> To add to what Chris and Steve W. said there are a couple of ways to do
> this.
>
> 1) If you have some uniquely identifiable attribute fields you can use,
> then that is the best approach
> 2) If you are basing simply on geometry, then you will need to decide
> how different enough geometries can be to be considered the same road.
>
> Basic approach
> 1) Import new shape file into a temp table, put indexes on identifying
> attributes
> 2) update existing roads
> 3) add new roads
>
> The attribute update way - you can do as Steve W. had mentioned or you
> can do with a 2 table update statement
>
> So lets say you have 2 fields road_name, road_dir that uniquely
> identifies a road, then you can do something like
> --update existing roads
> UPDATE currenttable
> SET the_geom = t.the_geom, field2 = t.field2, field2 = t.field3
> , etc..
> FROM temptable As t
> WHERE t.road_name = currenttable.road_name AND t.road_dir =
> currenttable.road_dir
>
> To add new records, you would do
> --For the below this only works if you insure you have no nulls in
> road_name of currenttable (and ideally you have a unique index on the
> field or set of
> fields you will use for identification
>
> INSERT INTO currenttable(the_geom, road_name, road_dir, field2,field3,
> ...etc)
> SELECT t.the_geom, t.road_name, t.road_dir, t.field2, t.field3, ....
> FROM temptable As t LEFT JOIN currenttable c ON (t.road_name =
> c.road_name AND t.road_dir = c.road_dir)
> WHERE c.road_name IS NULL
>
> To do it based on geometry alone, that is kind of tricky and involves
> too much guess work e.g. using ST_Difference, ST_Intersection,
> ST_Distance etc. and comparing how different the lengths, areas are
> based on some tolerance.  So I would suggest coming up with some
> scheme of uniquely tagging your roads if you don't have one already.
>
> Make sure to
>
> vacuum analyze currenttable
>
> after done - since this will create a ton of junk for many records.
>
> Hope that helps,
> Regina
>
> -----Original Message-----
> From: postgis-users-bounces@...
> [mailto:postgis-users-bounces@...] On Behalf Of
> SenTnel
> Sent: Wednesday, May 07, 2008 1:50 AM
> To: postgis-users@...
> Subject: Re: [postgis-users] Updated shapefile to update postgres
> database
>
>
> Thanks! Thats exactly what I want to do, but just don't know how to. I
> don't
> pretend for you to give me a step by step indication but Ill apreciate
> some
> instructions since Im really new to this and im eager to learn more
> about
> it. Thanks again!
>
>
> Chris Hermansen wrote:
>  
>> Presumably what you mean by "update" is that you want to replace old
>> versions of roads already in PostGIS with new versions of the same
>>    
> roads
>  
>> from shapefiles, and you want to keep the old street type in
>>    
> conjunction
>  
>> with the updated geometry?
>>
>> SenTnel wrote:
>>    
>>> Hello!
>>>
>>> I would like some help updating a postgres database. This is the
>>>      
> problem:
>  
>>> We
>>> created the database using shp2pgsql to convert a shapefile that
>>>      
> contains
>  
>>> a
>>> city's street details (centerline), after we created the database we
>>>      
> also
>  
>>> created an aditional column to classify the street types (eg: street,
>>> avenues, highways, etc.), now we are working on the original
>>>      
> shapefile,
>  
>>> updating new roads, changes made to highways due to constructions
>>> modifications, etc., and we want to upload the "updated" shapefile,
>>> without
>>> afecting the actual database, another words, we would like to convert
>>>      
> to
>  
>>> postgres the updated shapefile with the new information (it cuold be
>>>      
> the
>  
>>> whole shapefile) but to keep the added clasiffication column intact.
>>>      
> How
>  
>>> can
>>> we do that?
>>>
>>> Thanks
>>>  
>>>      
>> --
>> Regards,
>>
>> Chris Hermansen         mailto:chris.hermansen@...
>> tel+1.604.714.2878 * fax+1.604.733.0631 * mob+1.778.232.0644
>> Timberline Natural Resource Group * http://www.timberline.ca
>> 401 * 958 West 8th Avenue  * Vancouver BC * Canada * V5Z 1E5
>>
>> _______________________________________________
>> postgis-users mailing list
>> postgis-users@...
>> http://postgis.refractions.net/mailman/listinfo/postgis-users
>>
>>
>>    
>
>  


--
Regards,

Chris Hermansen         mailto:chris.hermansen@...
tel+1.604.714.2878 · fax+1.604.733.0631 · mob+1.778.232.0644
Timberline Natural Resource Group · http://www.timberline.ca
401 · 958 West 8th Avenue  · Vancouver BC · Canada · V5Z 1E5

_______________________________________________
postgis-users mailing list
postgis-users@...
http://postgis.refractions.net/mailman/listinfo/postgis-users

RE: Updated shapefile to update postgres database

by Martin Chapman-2 :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Thanks for that info William.  

Best regards,
Martin

-----Original Message-----
From: William Kyngesburye [mailto:woklist@...]
Sent: Tuesday, May 06, 2008 5:56 PM
To: PostGIS Users Discussion
Cc: Martin Chapman; SenTnel
Subject: Re: [postgis-users] Updated shapefile to update postgres database

I was interested in a similar thing once, but the ogr2ogr option names  
are misleading.

"-update" means add to an existing data source (ie the database in  
Postgres).  Otherwise ogr2ogr will try to create a new database, and  
fail if it exists.

"-append" means add to an existing layer (or, the table within the  
database) within a data source.  Otherwise it will try to create a new  
table, and fail if it exists.

Update will not replace any attributes or geometry for a feature that  
already exists with the same FID, but will stop with a duplicate key  
error.


On May 6, 2008, at 6:06 PM, Martin Chapman wrote:

> Ogr2Ogr at http://www.gdal.org/ogr/ogr2ogr.html has append and update
> options.  As long as your new columns were added to the end of the  
> table it
> will probably will work fine.
>
> Best regards,
> Martin
>
>
> -----Original Message-----
> From: postgis-users-bounces@...
> [mailto:postgis-users-bounces@...] On Behalf Of  
> SenTnel
> Sent: Tuesday, May 06, 2008 4:40 PM
> To: postgis-users@...
> Subject: [postgis-users] Updated shapefile to update postgres database
>
>
> Hello!
>
> I would like some help updating a postgres database. This is the  
> problem: We
> created the database using shp2pgsql to convert a shapefile that  
> contains a
> city's street details (centerline), after we created the database we  
> also
> created an aditional column to classify the street types (eg: street,
> avenues, highways, etc.), now we are working on the original  
> shapefile,
> updating new roads, changes made to highways due to constructions
> modifications, etc., and we want to upload the "updated" shapefile,  
> without
> afecting the actual database, another words, we would like to  
> convert to
> postgres the updated shapefile with the new information (it cuold be  
> the
> whole shapefile) but to keep the added clasiffication column intact.  
> How can
> we do that?
>
> Thanks
> --
> View this message in context:
>
http://www.nabble.com/Updated-shapefile-to-update-postgres-database-tp170535

> 62p17053562.html
> Sent from the PostGIS - User mailing list archive at Nabble.com.
>
> _______________________________________________
> postgis-users mailing list
> postgis-users@...
> http://postgis.refractions.net/mailman/listinfo/postgis-users
>
> _______________________________________________
> postgis-users mailing list
> postgis-users@...
> http://postgis.refractions.net/mailman/listinfo/postgis-users

-----
William Kyngesburye <kyngchaos*at*kyngchaos*dot*com>
http://www.kyngchaos.com/

The equator is so long, it could encircle the earth completely once.


_______________________________________________
postgis-users mailing list
postgis-users@...
http://postgis.refractions.net/mailman/listinfo/postgis-users

Re: Updated shapefile to update postgres database

by SenTnel :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Thanks Regina, Chris and all of you for your help. Im truly newby but with such help Im ready to start the procedures, Ill keep you posted of my progress or else if some issue comes along that may require some light from you, Thanks again and I hope my problem/solution may be of help to many others,

Thanks again!


SenTnel wrote:
Hello!

I would like some help updating a postgres database. This is the problem: We created the database using shp2pgsql to convert a shapefile that contains a city's street details (centerline), after we created the database we also created an aditional column to classify the street types (eg: street, avenues, highways, etc.), now we are working on the original shapefile, updating new roads, changes made to highways due to constructions modifications, etc., and we want to upload the "updated" shapefile, without afecting the actual database, another words, we would like to convert to postgres the updated shapefile with the new information (it cuold be the whole shapefile) but to keep the added clasiffication column intact. How can we do that?

Thanks

Re: Updated shapefile to update postgres database

by SenTnel :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Hello! Im confused about this issue:

I have a street (polyline) duplicated, one on top of the other, exact same features, same "the_geom", but one bears the street name (column:"street") "main street" and the other bears "N/A" as street name. Now my maps shows the street but the name that shows is "N/A", and I would like to delete the "N/A" polyline, how can i do basically this:

delete from mystreets_table street = 'N/A' where 'N/A' = street 'main street'.the_geom
(delete from mystreets table the street named 'N/A' where the_geom in 'N/A' equals the_geom in street "main street") another words, delete only the N/A streets that the_geom is same as "main street",
I hope you understand,

Thanks!




Thanks Regina, Chris and all of you for your help. Im truly newby but with such help Im ready to start the procedures, Ill keep you posted of my progress or else if some issue comes along that may require some light from you, Thanks again and I hope my problem/solution may be of help to many others,

Thanks again!


SenTnel wrote:
Hello!

I would like some help updating a postgres database. This is the problem: We created the database using shp2pgsql to convert a shapefile that contains a city's street details (centerline), after we created the database we also created an aditional column to classify the street types (eg: street, avenues, highways, etc.), now we are working on the original shapefile, updating new roads, changes made to highways due to constructions modifications, etc., and we want to upload the "updated" shapefile, without afecting the actual database, another words, we would like to convert to postgres the updated shapefile with the new information (it cuold be the whole shapefile) but to keep the added clasiffication column intact. How can we do that?

Thanks


RE: Updated shapefile to update postgres database

by Paragon Corporation-2 :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Here is a thought, but not sure how slow this will be

DELETE FROM mystreets_table
WHERE street = 'N/A' AND
EXISTS (SELECT n.street FROM mystreets_table n  WHERE (n.street <> 'N/A' AND
n.street > '')  AND ST_Equals(n.the_geom, mystreets_table.the_geom))

The st_equals might be better to replace with ~=  if you want to match exact
vertices.  I presume ~= is actually more efficient

Hope that helps,
Regina

-----Original Message-----
From: postgis-users-bounces@...
[mailto:postgis-users-bounces@...] On Behalf Of SenTnel
Sent: Saturday, June 07, 2008 2:25 AM
To: postgis-users@...
Subject: Re: [postgis-users] Updated shapefile to update postgres database


Hello! Im confused about this issue:

I have a street (polyline) duplicated, one on top of the other, exact same
features, same "the_geom", but one bears the street name (column:"street")
"main street" and the other bears "N/A" as street name. Now my maps shows
the street but the name that shows is "N/A", and I would like to delete the
"N/A" polyline, how can i do basically this:

delete from mystreets_table street = 'N/A' where 'N/A' = street 'main
street'.the_geom (delete from mystreets table the street named 'N/A' where
the_geom in 'N/A'
equals the_geom in street "main street") another words, delete only the N/A
streets that the_geom is same as "main street", I hope you understand,

Thanks!




SenTnel wrote:

>
> Thanks Regina, Chris and all of you for your help. Im truly newby but
> with such help Im ready to start the procedures, Ill keep you posted
> of my progress or else if some issue comes along that may require some
> light from you, Thanks again and I hope my problem/solution may be of
> help to many others,
>
> Thanks again!
>
>
>
> SenTnel wrote:
>>
>> Hello!
>>
>> I would like some help updating a postgres database. This is the problem:
>> We created the database using shp2pgsql to convert a shapefile that
>> contains a city's street details (centerline), after we created the
>> database we also created an aditional column to classify the street
>> types
>> (eg: street, avenues, highways, etc.), now we are working on the
>> original shapefile, updating new roads, changes made to highways due
>> to constructions modifications, etc., and we want to upload the "updated"
>> shapefile, without afecting the actual database, another words, we
>> would like to convert to postgres the updated shapefile with the new
>> information (it cuold be the whole shapefile) but to keep the added
>> clasiffication column intact. How can we do that?
>>
>> Thanks
>>
>
>

--
View this message in context:
http://www.nabble.com/Updated-shapefile-to-update-postgres-database-tp170535
62p17705777.html
Sent from the PostGIS - User mailing list archive at Nabble.com.

_______________________________________________
postgis-users mailing list
postgis-users@...
http://postgis.refractions.net/mailman/listinfo/postgis-users



_______________________________________________
postgis-users mailing list
postgis-users@...
http://postgis.refractions.net/mailman/listinfo/postgis-users

RE: Updated shapefile to update postgres database

by SenTnel :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Hello Regina! Thanks for your help, but when I try running it I got this error message:


ERROR:  function st_equals(geometry, geometry) does not exist
HINT:  No function matches the given name and argument types. You may need to add explicit type casts.


Thanks!


Paragon Corporation-2 wrote:
Here is a thought, but not sure how slow this will be

DELETE FROM mystreets_table
WHERE street = 'N/A' AND
EXISTS (SELECT n.street FROM mystreets_table n  WHERE (n.street <> 'N/A' AND
n.street > '')  AND ST_Equals(n.the_geom, mystreets_table.the_geom))

The st_equals might be better to replace with ~=  if you want to match exact
vertices.  I presume ~= is actually more efficient

Hope that helps,
Regina

-----Original Message-----
From: postgis-users-bounces@postgis.refractions.net
[mailto:postgis-users-bounces@postgis.refractions.net] On Behalf Of SenTnel
Sent: Saturday, June 07, 2008 2:25 AM
To: postgis-users@postgis.refractions.net
Subject: Re: [postgis-users] Updated shapefile to update postgres database


Hello! Im confused about this issue:

I have a street (polyline) duplicated, one on top of the other, exact same
features, same "the_geom", but one bears the street name (column:"street")
"main street" and the other bears "N/A" as street name. Now my maps shows
the street but the name that shows is "N/A", and I would like to delete the
"N/A" polyline, how can i do basically this:

delete from mystreets_table street = 'N/A' where 'N/A' = street 'main
street'.the_geom (delete from mystreets table the street named 'N/A' where
the_geom in 'N/A'
equals the_geom in street "main street") another words, delete only the N/A
streets that the_geom is same as "main street", I hope you understand,

Thanks!




SenTnel wrote:
>
> Thanks Regina, Chris and all of you for your help. Im truly newby but
> with such help Im ready to start the procedures, Ill keep you posted
> of my progress or else if some issue comes along that may require some
> light from you, Thanks again and I hope my problem/solution may be of
> help to many others,
>
> Thanks again!
>
>
>
> SenTnel wrote:
>>
>> Hello!
>>
>> I would like some help updating a postgres database. This is the problem:
>> We created the database using shp2pgsql to convert a shapefile that
>> contains a city's street details (centerline), after we created the
>> database we also created an aditional column to classify the street
>> types
>> (eg: street, avenues, highways, etc.), now we are working on the
>> original shapefile, updating new roads, changes made to highways due
>> to constructions modifications, etc., and we want to upload the "updated"
>> shapefile, without afecting the actual database, another words, we
>> would like to convert to postgres the updated shapefile with the new
>> information (it cuold be the whole shapefile) but to keep the added
>> clasiffication column intact. How can we do that?
>>
>> Thanks
>>
>
>

--
View this message in context:
http://www.nabble.com/Updated-shapefile-to-update-postgres-database-tp170535
62p17705777.html
Sent from the PostGIS - User mailing list archive at Nabble.com.

_______________________________________________
postgis-users mailing list
postgis-users@postgis.refractions.net
http://postgis.refractions.net/mailman/listinfo/postgis-users



_______________________________________________
postgis-users mailing list
postgis-users@postgis.refractions.net
http://postgis.refractions.net/mailman/listinfo/postgis-users

RE: Updated shapefile to update postgres database

by Paragon Corporation-2 :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

I wonder if its just called Equals in the old version.

Try doing just ~=. If your geometries are exactly the same that will
probably be better anyway

 DELETE FROM mystreets_table
 WHERE street = 'N/A' AND
 EXISTS (SELECT n.street FROM mystreets_table n  WHERE n.street <> 'N/A'
 AND n.street > ''  AND n.the_geom ~= mystreets_table.the_geom)

Hope that helps.
Regina

-----Original Message-----
From: postgis-users-bounces@...
[mailto:postgis-users-bounces@...] On Behalf Of SenTnel
Sent: Saturday, June 07, 2008 4:36 PM
To: postgis-users@...
Subject: RE: [postgis-users] Updated shapefile to update postgres database


Hello Regina! Thanks for your help, but when I try running it I got this
error message:


ERROR:  function st_equals(geometry, geometry) does not exist
HINT:  No function matches the given name and argument types. You may need
to add explicit type casts.

:confused:
Thanks!



Paragon Corporation-2 wrote:

>
> Here is a thought, but not sure how slow this will be
>
> DELETE FROM mystreets_table
> WHERE street = 'N/A' AND
> EXISTS (SELECT n.street FROM mystreets_table n  WHERE (n.street <> 'N/A'
> AND
> n.street > '')  AND ST_Equals(n.the_geom, mystreets_table.the_geom))
>
> The st_equals might be better to replace with ~=  if you want to match
> exact vertices.  I presume ~= is actually more efficient
>
> Hope that helps,
> Regina
>
> -----Original Message-----
> From: postgis-users-bounces@...
> [mailto:postgis-users-bounces@...] On Behalf Of
> SenTnel
> Sent: Saturday, June 07, 2008 2:25 AM
> To: postgis-users@...
> Subject: Re: [postgis-users] Updated shapefile to update postgres
> database
>
>
> Hello! Im confused about this issue:
>
> I have a street (polyline) duplicated, one on top of the other, exact
> same features, same "the_geom", but one bears the street name
> (column:"street") "main street" and the other bears "N/A" as street
> name. Now my maps shows the street but the name that shows is "N/A",
> and I would like to delete the "N/A" polyline, how can i do basically
> this:
>
> delete from mystreets_table street = 'N/A' where 'N/A' = street 'main
> street'.the_geom (delete from mystreets table the street named 'N/A'
> where the_geom in 'N/A'
> equals the_geom in street "main street") another words, delete only
> the N/A streets that the_geom is same as "main street", I hope you
> understand,
>
> Thanks!
>
>
>
>
> SenTnel wrote:
>>
>> Thanks Regina, Chris and all of you for your help. Im truly newby but
>> with such help Im ready to start the procedures, Ill keep you posted
>> of my progress or else if some issue comes along that may require
>> some light from you, Thanks again and I hope my problem/solution may
>> be of help to many others,
>>
>> Thanks again!
>>
>>
>>
>> SenTnel wrote:
>>>
>>> Hello!
>>>
>>> I would like some help updating a postgres database. This is the
>>> problem:
>>> We created the database using shp2pgsql to convert a shapefile that
>>> contains a city's street details (centerline), after we created the
>>> database we also created an aditional column to classify the street
>>> types
>>> (eg: street, avenues, highways, etc.), now we are working on the
>>> original shapefile, updating new roads, changes made to highways due
>>> to constructions modifications, etc., and we want to upload the
>>> "updated"
>>> shapefile, without afecting the actual database, another words, we
>>> would like to convert to postgres the updated shapefile with the new
>>> information (it cuold be the whole shapefile) but to keep the added
>>> clasiffication column intact. How can we do that?
>>>
>>> Thanks
>>>
>>
>>
>
> --
> View this message in context:
> http://www.nabble.com/Updated-shapefile-to-update-postgres-database-tp
> 170535
> 62p17705777.html
> Sent from the PostGIS - User mailing list archive at Nabble.com.
>
> _______________________________________________
> postgis-users mailing list
> postgis-users@...
> http://postgis.refractions.net/mailman/listinfo/postgis-users
>
>
>
> _______________________________________________
> postgis-users mailing list
> postgis-users@...
> http://postgis.refractions.net/mailman/listinfo/postgis-users
>
>

--
View this message in context:
http://www.nabble.com/Updated-shapefile-to-update-postgres-database-tp170535
62p17712835.html
Sent from the PostGIS - User mailing list archive at Nabble.com.

_______________________________________________
postgis-users mailing list
postgis-users@...
http://postgis.refractions.net/mailman/listinfo/postgis-users



_______________________________________________
postgis-users mailing list
postgis-users@...
http://postgis.refractions.net/mailman/listinfo/postgis-users

RE: Updated shapefile to update postgres database

by SenTnel :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Hi Regina!

Sorry took me so long to reply to this post, crazy week around here, but thank god there's always a weekend to look forward!

This week I tried this method twice but finally had to stop it. First time I stopped the query after 4 hours thinking there was something wrong. Two nights later I tried again, left it running all night, next day, almost 10 hours later the query was running still. My table has 123,000+ rows, and just want to ask you if for such size is normal to take that long, if so I'll try again, if not, if you don't think it should take that long then you have any other suggestion?

Thanks again!


Paragon Corporation-2 wrote:
I wonder if its just called Equals in the old version.

Try doing just ~=. If your geometries are exactly the same that will
probably be better anyway

 DELETE FROM mystreets_table
 WHERE street = 'N/A' AND
 EXISTS (SELECT n.street FROM mystreets_table n  WHERE n.street <> 'N/A'
 AND n.street > ''  AND n.the_geom ~= mystreets_table.the_geom)

Hope that helps.
Regina

-----Original Message-----
From: postgis-users-bounces@postgis.refractions.net
[mailto:postgis-users-bounces@postgis.refractions.net] On Behalf Of SenTnel
Sent: Saturday, June 07, 2008 4:36 PM
To: postgis-users@postgis.refractions.net
Subject: RE: [postgis-users] Updated shapefile to update postgres database


Hello Regina! Thanks for your help, but when I try running it I got this
error message:


ERROR:  function st_equals(geometry, geometry) does not exist
HINT:  No function matches the given name and argument types. You may need
to add explicit type casts.

:confused:
Thanks!



Paragon Corporation-2 wrote:
>
> Here is a thought, but not sure how slow this will be
>
> DELETE FROM mystreets_table
> WHERE street = 'N/A' AND
> EXISTS (SELECT n.street FROM mystreets_table n  WHERE (n.street <> 'N/A'
> AND
> n.street > '')  AND ST_Equals(n.the_geom, mystreets_table.the_geom))
>
> The st_equals might be better to replace with ~=  if you want to match
> exact vertices.  I presume ~= is actually more efficient
>
> Hope that helps,
> Regina
>
> -----Original Message-----
> From: postgis-users-bounces@postgis.refractions.net
> [mailto:postgis-users-bounces@postgis.refractions.net] On Behalf Of
> SenTnel
> Sent: Saturday, June 07, 2008 2:25 AM
> To: postgis-users@postgis.refractions.net
> Subject: Re: [postgis-users] Updated shapefile to update postgres
> database
>
>
> Hello! Im confused about this issue:
>
> I have a street (polyline) duplicated, one on top of the other, exact
> same features, same "the_geom", but one bears the street name
> (column:"street") "main street" and the other bears "N/A" as street
> name. Now my maps shows the street but the name that shows is "N/A",
> and I would like to delete the "N/A" polyline, how can i do basically
> this:
>
> delete from mystreets_table street = 'N/A' where 'N/A' = street 'main
> street'.the_geom (delete from mystreets table the street named 'N/A'
> where the_geom in 'N/A'
> equals the_geom in street "main street") another words, delete only
> the N/A streets that the_geom is same as "main street", I hope you
> understand,
>
> Thanks!
>
>
>
>
> SenTnel wrote:
>>
>> Thanks Regina, Chris and all of you for your help. Im truly newby but
>> with such help Im ready to start the procedures, Ill keep you posted
>> of my progress or else if some issue comes along that may require
>> some light from you, Thanks again and I hope my problem/solution may
>> be of help to many others,
>>
>> Thanks again!
>>
>>
>>
>> SenTnel wrote:
>>>
>>> Hello!
>>>
>>> I would like some help updating a postgres database. This is the
>>> problem:
>>> We created the database using shp2pgsql to convert a shapefile that
>>> contains a city's street details (centerline), after we created the
>>> database we also created an aditional column to classify the street
>>> types
>>> (eg: street, avenues, highways, etc.), now we are working on the
>>> original shapefile, updating new roads, changes made to highways due
>>> to constructions modifications, etc., and we want to upload the
>>> "updated"
>>> shapefile, without afecting the actual database, another words, we
>>> would like to convert to postgres the updated shapefile with the new
>>> information (it cuold be the whole shapefile) but to keep the added
>>> clasiffication column intact. How can we do that?
>>>
>>> Thanks
>>>
>>
>>
>
> --
> View this message in context:
> http://www.nabble.com/Updated-shapefile-to-update-postgres-database-tp
> 170535
> 62p17705777.html
> Sent from the PostGIS - User mailing list archive at Nabble.com.
>
> _______________________________________________
> postgis-users mailing list
> postgis-users@postgis.refractions.net
> http://postgis.refractions.net/mailman/listinfo/postgis-users
>
>
>
> _______________________________________________
> postgis-users mailing list
> postgis-users@postgis.refractions.net
> http://postgis.refractions.net/mailman/listinfo/postgis-users
>
>

--
View this message in context:
http://www.nabble.com/Updated-shapefile-to-update-postgres-database-tp170535
62p17712835.html
Sent from the PostGIS - User mailing list archive at Nabble.com.

_______________________________________________
postgis-users mailing list
postgis-users@postgis.refractions.net
http://postgis.refractions.net/mailman/listinfo/postgis-users



_______________________________________________
postgis-users mailing list
postgis-users@postgis.refractions.net
http://postgis.refractions.net/mailman/listinfo/postgis-users

RE: Updated shapefile to update postgres database

by Paragon Corporation-2 :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

It shouldn't take that long I don't think.  123,000 records is not a lot.
Granted this is a suboptimal delete.

First thing to verify

1) Make sure to have a btree index on street
2) I assume you have a gist index on your the_geom field already.  If not
that could be the culprit.
3) What are your postgresql.conf settings like for work_mem etc.  If these
are at there defuats, then that would be a problem too.

The other alternative is to do with an IN instead of an EXISTS.  That would
at least not be correlated, but I'm not clear if an

the_geom IN (SELECT ....)

Would match up correctly since I suspect it may use = for compare

Hope that helps,
Regina



-----Original Message-----
From: postgis-users-bounces@...
[mailto:postgis-users-bounces@...] On Behalf Of SenTnel
Sent: Sunday, June 15, 2008 1:34 AM
To: postgis-users@...
Subject: RE: [postgis-users] Updated shapefile to update postgres database


Hi Regina!

Sorry took me so long to reply to this post, crazy week around here, but
thank god there's always a weekend to look forward!

This week I tried this method twice but finally had to stop it. First time I
stopped the query after 4 hours thinking there was something wrong. Two
nights later I tried again, left it running all night, next day, almost 10
hours later the query was running still. My table has 123,000+ rows, and
just want to ask you if for such size is normal to take that long, if so
I'll try again, if not, if you don't think it should take that long then you
have any other suggestion?

Thanks again!



Paragon Corporation-2 wrote:

>
> I wonder if its just called Equals in the old version.
>
> Try doing just ~=. If your geometries are exactly the same that will
> probably be better anyway
>
>  DELETE FROM mystreets_table
>  WHERE street = 'N/A' AND
>  EXISTS (SELECT n.street FROM mystreets_table n  WHERE n.street <> 'N/A'
>  AND n.street > ''  AND n.the_geom ~= mystreets_table.the_geom)
>
> Hope that helps.
> Regina
>
> -----Original Message-----
> From: postgis-users-bounces@...
> [mailto:postgis-users-bounces@...] On Behalf Of
> SenTnel
> Sent: Saturday, June 07, 2008 4:36 PM
> To: postgis-users@...
> Subject: RE: [postgis-users] Updated shapefile to update postgres
> database
>
>
> Hello Regina! Thanks for your help, but when I try running it I got
> this error message:
>
>
> ERROR:  function st_equals(geometry, geometry) does not exist
> HINT:  No function matches the given name and argument types. You may
> need to add explicit type casts.
>
> :confused:
> Thanks!
>
>
>
> Paragon Corporation-2 wrote:
>>
>> Here is a thought, but not sure how slow this will be
>>
>> DELETE FROM mystreets_table
>> WHERE street = 'N/A' AND
>> EXISTS (SELECT n.street FROM mystreets_table n  WHERE (n.street <> 'N/A'
>> AND
>> n.street > '')  AND ST_Equals(n.the_geom, mystreets_table.the_geom))
>>
>> The st_equals might be better to replace with ~=  if you want to
>> match exact vertices.  I presume ~= is actually more efficient
>>
>> Hope that helps,
>> Regina
>>
>> -----Original Message-----
>> From: postgis-users-bounces@...
>> [mailto:postgis-users-bounces@...] On Behalf Of
>> SenTnel
>> Sent: Saturday, June 07, 2008 2:25 AM
>> To: postgis-users@...
>> Subject: Re: [postgis-users] Updated shapefile to update postgres
>> database
>>
>>
>> Hello! Im confused about this issue:
>>
>> I have a street (polyline) duplicated, one on top of the other, exact
>> same features, same "the_geom", but one bears the street name
>> (column:"street") "main street" and the other bears "N/A" as street
>> name. Now my maps shows the street but the name that shows is "N/A",
>> and I would like to delete the "N/A" polyline, how can i do basically
>> this:
>>
>> delete from mystreets_table street = 'N/A' where 'N/A' = street 'main
>> street'.the_geom (delete from mystreets table the street named 'N/A'
>> where the_geom in 'N/A'
>> equals the_geom in street "main street") another words, delete only
>> the N/A streets that the_geom is same as "main street", I hope you
>> understand,
>>
>> Thanks!
>>
>>
>>
>>
>> SenTnel wrote:
>>>
>>> Thanks Regina, Chris and all of you for your help. Im truly newby
>>> but with such help Im ready to start the procedures, Ill keep you
>>> posted of my progress or else if some issue comes along that may
>>> require some light from you, Thanks again and I hope my
>>> problem/solution may be of help to many others,
>>>
>>> Thanks again!
>>>
>>>
>>>
>>> SenTnel wrote:
>>>>
>>>> Hello!
>>>>
>>>> I would like some help updating a postgres database. This is the
>>>> problem:
>>>> We created the database using shp2pgsql to convert a shapefile that
>>>> contains a city's street details (centerline), after we created the
>>>> database we also created an aditional column to classify the street
>>>> types
>>>> (eg: street, avenues, highways, etc.), now we are working on the
>>>> original shapefile, updating new roads, changes made to highways
>>>> due to constructions modifications, etc., and we want to upload the
>>>> "updated"
>>>> shapefile, without afecting the actual database, another words, we
>>>> would like to convert to postgres the updated shapefile with the
>>>> new information (it cuold be the whole shapefile) but to keep the
>>>> added clasiffication column intact. How can we do that?
>>>>
>>>> Thanks
>>>>
>>>
>>>
>>
>> --
>> View this message in context:
>> http://www.nabble.com/Updated-shapefile-to-update-postgres-database-t
>> p
>> 170535
>> 62p17705777.html
>> Sent from the PostGIS - User mailing list archive at Nabble.com.
>>
>> _______________________________________________
>> postgis-users mailing list
>> postgis-users@...
>> http://postgis.refractions.net/mailman/listinfo/postgis-users
>>
>>
>>
>> _______________________________________________
>> postgis-users mailing list
>> postgis-users@...
>> http://postgis.refractions.net/mailman/listinfo/postgis-users
>>
>>
>
> --
> View this message in context:
> http://www.nabble.com/Updated-shapefile-to-update-postgres-database-tp
> 170535
> 62p17712835.html
> Sent from the PostGIS - User mailing list archive at Nabble.com.
>
> _______________________________________________
> postgis-users mailing list
> postgis-users@...
> http://postgis.refractions.net/mailman/listinfo/postgis-users
>
>
>
> _______________________________________________
> postgis-users mailing list
> postgis-users@...
> http://postgis.refractions.net/mailman/listinfo/postgis-users
>
>

--
View this message in context:
http://www.nabble.com/Updated-shapefile-to-update-postgres-database-tp170535
62p17846638.html
Sent from the PostGIS - User mailing list archive at Nabble.com.

_______________________________________________
postgis-users mailing list
postgis-users@...
http://postgis.refractions.net/mailman/listinfo/postgis-users


_______________________________________________
postgis-users mailing list
postgis-users@...
http://postgis.refractions.net/mailman/listinfo/postgis-users

RE: Updated shapefile to update postgres database

by Paragon Corporation-2 :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Forgot make sure you

Vacuum analyze yourtable

After you create indexes and load lots of data.

-----Original Message-----
From: postgis-users-bounces@...
[mailto:postgis-users-bounces@...] On Behalf Of SenTnel
Sent: Sunday, June 15, 2008 1:34 AM
To: postgis-users@...
Subject: RE: [postgis-users] Updated shapefile to update postgres database


Hi Regina!

Sorry took me so long to reply to this post, crazy week around here, but
thank god there's always a weekend to look forward!

This week I tried this method twice but finally had to stop it. First time I
stopped the query after 4 hours thinking there was something wrong. Two
nights later I tried again, left it running all night, next day, almost 10
hours later the query was running still. My table has 123,000+ rows, and
just want to ask you if for such size is normal to take that long, if so
I'll try again, if not, if you don't think it should take that long then you
have any other suggestion?

Thanks again!



Paragon Corporation-2 wrote:

>
> I wonder if its just called Equals in the old version.
>
> Try doing just ~=. If your geometries are exactly the same that will
> probably be better anyway
>
>  DELETE FROM mystreets_table
>  WHERE street = 'N/A' AND
>  EXISTS (SELECT n.street FROM mystreets_table n  WHERE n.street <> 'N/A'
>  AND n.street > ''  AND n.the_geom ~= mystreets_table.the_geom)
>
> Hope that helps.
> Regina
>
> -----Original Message-----
> From: postgis-users-bounces@...
> [mailto:postgis-users-bounces@...] On Behalf Of
> SenTnel
> Sent: Saturday, June 07, 2008 4:36 PM
> To: postgis-users@...
> Subject: RE: [postgis-users] Updated shapefile to update postgres
> database
>
>
> Hello Regina! Thanks for your help, but when I try running it I got
> this error message:
>
>
> ERROR:  function st_equals(geometry, geometry) does not exist
> HINT:  No function matches the given name and argument types. You may
> need to add explicit type casts.
>
> :confused:
> Thanks!
>
>
>
> Paragon Corporation-2 wrote:
>>
>> Here is a thought, but not sure how slow this will be
>>
>> DELETE FROM mystreets_table
>> WHERE street = 'N/A' AND
>> EXISTS (SELECT n.street FROM mystreets_table n  WHERE (n.street <> 'N/A'
>> AND
>> n.street > '')  AND ST_Equals(n.the_geom, mystreets_table.the_geom))
>>
>> The st_equals might be better to replace with ~=  if you want to
>> match exact vertices.  I presume ~= is actually more efficient
>>
>> Hope that helps,
>> Regina
>>
>> -----Original Message-----
>> From: postgis-users-bounces@...
>> [mailto:postgis-users-bounces@...] On Behalf Of
>> SenTnel
>> Sent: Saturday, June 07, 2008 2:25 AM
>> To: postgis-users@...
>> Subject: Re: [postgis-users] Updated shapefile to update postgres
>> database
>>
>>
>> Hello! Im confused about this issue:
>>
>> I have a street (polyline) duplicated, one on top of the other, exact
>> same features, same "the_geom", but one bears the street name
>> (column:"street") "main street" and the other bears "N/A" as street
>> name. Now my maps shows the street but the name that shows is "N/A",
>> and I would like to delete the "N/A" polyline, how can i do basically
>> this:
>>
>> delete from mystreets_table street = 'N/A' where 'N/A' = street 'main
>> street'.the_geom (delete from mystreets table the street named 'N/A'
>> where the_geom in 'N/A'
>> equals the_geom in street "main street") another words, delete only
>> the N/A streets that the_geom is same as "main street", I hope you
>> understand,
>>
>> Thanks!
>>
>>
>>
>>
>> SenTnel wrote:
>>>
>>> Thanks Regina, Chris and all of you for your help. Im truly newby
>>> but with such help Im ready to start the procedures, Ill keep you
>>> posted of my progress or else if some issue comes along that may
>>> require some light from you, Thanks again and I hope my
>>> problem/solution may be of help to many others,
>>>
>>> Thanks again!
>>>
>>>
>>>
>>> SenTnel wrote:
>>>>
>>>> Hello!
>>>>
>>>> I would like some help updating a postgres database. This is the
>>>> problem:
>>>> We created the database using shp2pgsql to convert a shapefile that
>>>> contains a city's street details (centerline), after we created the
>>>> database we also created an aditional column to classify the street
>>>> types
>>>> (eg: street, avenues, highways, etc.), now we are working on the
>>>> original shapefile, updating new roads, changes made to highways
>>>> due to constructions modifications, etc., and we want to upload the
>>>> "updated"
>>>> shapefile, without afecting the actual database, another words, we
>>>> would like to convert to postgres the updated shapefile with the
>>>> new information (it cuold be the whole shapefile) but to keep the
>>>> added clasiffication column intact. How can we do that?
>>>>
>>>> Thanks
>>>>
>>>
>>>
>>
>> --
>> View this message in context:
>> http://www.nabble.com/Updated-shapefile-to-update-postgres-database-t
>> p
>> 170535
>> 62p17705777.html
>> Sent from the PostGIS - User mailing list archive at Nabble.com.
>>
>> _______________________________________________
>> postgis-users mailing list
>> postgis-users@...
>> http://postgis.refractions.net/mailman/listinfo/postgis-users
>>
>>
>>
>> _______________________________________________
>> postgis-users mailing list
>> postgis-users@...
>> http://postgis.refractions.net/mailman/listinfo/postgis-users
>>
>>
>
> --
> View this message in context:
> http://www.nabble.com/Updated-shapefile-to-update-postgres-database-tp
> 170535
> 62p17712835.html
> Sent from the PostGIS - User mailing list archive at Nabble.com.
>
> _______________________________________________
> postgis-users mailing list
> postgis-users@...
> http://postgis.refractions.net/mailman/listinfo/postgis-users
>
>
>
> _______________________________________________
> postgis-users mailing list
> postgis-users@...
> http://postgis.refractions.net/mailman/listinfo/postgis-users
>
>

--
View this message in context:
http://www.nabble.com/Updated-shapefile-to-update-postgres-database-tp170535
62p17846638.html
Sent from the PostGIS - User mailing list archive at Nabble.com.

_______________________________________________
postgis-users mailing list
postgis-users@...
http://postgis.refractions.net/mailman/listinfo/postgis-users


_______________________________________________
postgis-users mailing list
postgis-users@...
http://postgis.refractions.net/mailman/listinfo/postgis-users

RE: Updated shapefile to update postgres database

by SenTnel :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Thanks Regina for your help! I must tell you that the worst part of this is not knowing what most of what you tell me means!  Im trying to become familiar with this but don't know much...  Totally ignore btree, gist index, but I'll google it to see what I can find. As far as the configuration it is the installation's default, suggestions are welcome!
 

... again... Thanks!

Paragon Corporation-2 wrote:
It shouldn't take that long I don't think.  123,000 records is not a lot.
Granted this is a suboptimal delete.

First thing to verify

1) Make sure to have a btree index on street
2) I assume you have a gist index on your the_geom field already.  If not
that could be the culprit.
3) What are your postgresql.conf settings like for work_mem etc.  If these
are at there defuats, then that would be a problem too.

The other alternative is to do with an IN instead of an EXISTS.  That would
at least not be correlated, but I'm not clear if an

the_geom IN (SELECT ....)

Would match up correctly since I suspect it may use = for compare

Hope that helps,
Regina



-----Original Message-----
From: postgis-users-bounces@postgis.refractions.net
[mailto:postgis-users-bounces@postgis.refractions.net] On Behalf Of SenTnel
Sent: Sunday, June 15, 2008 1:34 AM
To: postgis-users@postgis.refractions.net
Subject: RE: [postgis-users] Updated shapefile to update postgres database


Hi Regina!

Sorry took me so long to reply to this post, crazy week around here, but
thank god there's always a weekend to look forward!

This week I tried this method twice but finally had to stop it. First time I
stopped the query after 4 hours thinking there was something wrong. Two
nights later I tried again, left it running all night, next day, almost 10
hours later the query was running still. My table has 123,000+ rows, and
just want to ask you if for such size is normal to take that long, if so
I'll try again, if not, if you don't think it should take that long then you
have any other suggestion?

Thanks again!



Paragon Corporation-2 wrote:
>
> I wonder if its just called Equals in the old version.
>
> Try doing just ~=. If your geometries are exactly the same that will
> probably be better anyway
>
>  DELETE FROM mystreets_table
>  WHERE street = 'N/A' AND
>  EXISTS (SELECT n.street FROM mystreets_table n  WHERE n.street <> 'N/A'
>  AND n.street > ''  AND n.the_geom ~= mystreets_table.the_geom)
>
> Hope that helps.
> Regina
>
> -----Original Message-----
> From: postgis-users-bounces@postgis.refractions.net
> [mailto:postgis-users-bounces@postgis.refractions.net] On Behalf Of
> SenTnel
> Sent: Saturday, June 07, 2008 4:36 PM
> To: postgis-users@postgis.refractions.net
> Subject: RE: [postgis-users] Updated shapefile to update postgres
> database
>
>
> Hello Regina! Thanks for your help, but when I try running it I got
> this error message:
>
>
> ERROR:  function st_equals(geometry, geometry) does not exist
> HINT:  No function matches the given name and argument types. You may
> need to add explicit type casts.
>
> :confused:
> Thanks!
>
>
>
> Paragon Corporation-2 wrote:
>>
>> Here is a thought, but not sure how slow this will be
>>
>> DELETE FROM mystreets_table
>> WHERE street = 'N/A' AND
>> EXISTS (SELECT n.street FROM mystreets_table n  WHERE (n.street <> 'N/A'
>> AND
>> n.street > '')  AND ST_Equals(n.the_geom, mystreets_table.the_geom))
>>
>> The st_equals might be better to replace with ~=  if you want to
>> match exact vertices.  I presume ~= is actually more efficient
>>
>> Hope that helps,
>> Regina
>>
>> -----Original Message-----
>> From: postgis-users-bounces@postgis.refractions.net
>> [mailto:postgis-users-bounces@postgis.refractions.net] On Behalf Of
>> SenTnel
>> Sent: Saturday, June 07, 2008 2:25 AM
>> To: postgis-users@postgis.refractions.net
>> Subject: Re: [postgis-users] Updated shapefile to update postgres
>> database
>>
>>
>> Hello! Im confused about this issue:
>>
>> I have a street (polyline) duplicated, one on top of the other, exact
>> same features, same "the_geom", but one bears the street name
>> (column:"street") "main street" and the other bears "N/A" as street
>> name. Now my maps shows the street but the name that shows is "N/A",
>> and I would like to delete the "N/A" polyline, how can i do basically
>> this:
>>
>> delete from mystreets_table street = 'N/A' where 'N/A' = street 'main
>> street'.the_geom (delete from mystreets table the street named 'N/A'
>> where the_geom in 'N/A'
>> equals the_geom in street "main street") another words, delete only
>> the N/A streets that the_geom is same as "main street", I hope you
>> understand,
>>
>> Thanks!
>>
>>
>>
>>
>> SenTnel wrote:
>>>
>>> Thanks Regina, Chris and all of you for your help. Im truly newby
>>> but with such help Im ready to start the procedures, Ill keep you
>>> posted of my progress or else if some issue comes along that may
>>> require some light from you, Thanks again and I hope my
>>> problem/solution may be of help to many others,
>>>
>>> Thanks again!
>>>
>>>
>>>
>>> SenTnel wrote:
>>>>
>>>> Hello!
>>>>
>>>> I would like some help updating a postgres database. This is the
>>>> problem:
>>>> We created the database using shp2pgsql to convert a shapefile that
>>>> contains a city's street details (centerline), after we created the
>>>> database we also created an aditional column to classify the street
>>>> types
>>>> (eg: street, avenues, highways, etc.), now we are working on the
>>>> original shapefile, updating new roads, changes made to highways
>>>> due to constructions modifications, etc., and we want to upload the
>>>> "updated"
>>>> shapefile, without afecting the actual database, another words, we
>>>> would like to convert to postgres the updated shapefile with the
>>>> new information (it cuold be the whole shapefile) but to keep the
>>>> added clasiffication column intact. How can we do that?
>>>>
>>>> Thanks
>>>>
>>>
>>>
>>
>> --
>> View this message in context:
>> http://www.nabble.com/Updated-shapefile-to-update-postgres-database-t
>> p
>> 170535
>> 62p17705777.html
>> Sent from the PostGIS - User mailing list archive at Nabble.com.
>>
>> _______________________________________________
>> postgis-users mailing list
>> postgis-users@postgis.refractions.net
>> http://postgis.refractions.net/mailman/listinfo/postgis-users
>>
>>
>>
>> _______________________________________________
>> postgis-users mailing list
>> postgis-users@postgis.refractions.net
>> http://postgis.refractions.net/mailman/listinfo/postgis-users
>>
>>
>
> --
> View this message in context:
> http://www.nabble.com/Updated-shapefile-to-update-postgres-database-tp
> 170535
> 62p17712835.html
> Sent from the PostGIS - User mailing list archive at Nabble.com.
>
> _______________________________________________
> postgis-users mailing list
> postgis-users@postgis.refractions.net
> http://postgis.refractions.net/mailman/listinfo/postgis-users
>
>
>
> _______________________________________________
> postgis-users mailing list
> postgis-users@postgis.refractions.net
> http://postgis.refractions.net/mailman/listinfo/postgis-users
>
>

--
View this message in context:
http://www.nabble.com/Updated-shapefile-to-update-postgres-database-tp170535
62p17846638.html
Sent from the PostGIS - User mailing list archive at Nabble.com.

_______________________________________________
postgis-users mailing list
postgis-users@postgis.refractions.net
http://postgis.refractions.net/mailman/listinfo/postgis-users


_______________________________________________
postgis-users mailing list
postgis-users@postgis.refractions.net
http://postgis.refractions.net/mailman/listinfo/postgis-users

RE: Updated shapefile to update postgres database

by Paragon Corporation-2 :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Take a look at our article on the subject if you haven't already.

It covers how to create gist and btree indexes

http://www.bostongis.com/PrinterFriendly.aspx?content_name=postgis_tut01

Samples exist in the postgis docs too.



-----Original Message-----
From: postgis-users-bounces@...
[mailto:postgis-users-bounces@...] On Behalf Of SenTnel
Sent: Sunday, June 15, 2008 4:37 AM
To: postgis-users@...
Subject: RE: [postgis-users] Updated shapefile to update postgres database


Thanks Regina for your help! I must tell you that the worst part of this is
not knowing what most of what you tell me means! :confused: Im trying to
become familiar with this but don't know much...  Totally ignore btree, gist
index, but I'll google it to see what I can find. As far as the
configuration it is the installation's default, suggestions are welcome!
=^D

... again... Thanks!


Paragon Corporation-2 wrote:

>
> It shouldn't take that long I don't think.  123,000 records is not a lot.
> Granted this is a suboptimal delete.
>
> First thing to verify
>
> 1) Make sure to have a btree index on street
> 2) I assume you have a gist index on your the_geom field already.  If
> not that could be the culprit.
> 3) What are your postgresql.conf settings like for work_mem etc.  If
> these are at there defuats, then that would be a problem too.
>
> The other alternative is to do with an IN instead of an EXISTS.  That
> would at least not be correlated, but I'm not clear if an
>
> the_geom IN (SELECT ....)
>
> Would match up correctly since I suspect it may use = for compare
>
> Hope that helps,
> Regina
>
>
>
> -----Original Message-----
> From: postgis-users-bounces@...
> [mailto:postgis-users-bounces@...] On Behalf Of
> SenTnel
> Sent: Sunday, June 15, 2008 1:34 AM
> To: postgis-users@...
> Subject: RE: [postgis-users] Updated shapefile to update postgres
> database
>
>
> Hi Regina!
>
> Sorry took me so long to reply to this post, crazy week around here,
> but thank god there's always a weekend to look forward!
>
> This week I tried this method twice but finally had to stop it. First
> time I stopped the query after 4 hours thinking there was something
> wrong. Two nights later I tried again, left it running all night, next
> day, almost 10 hours later the query was running still. My table has
> 123,000+ rows, and just want to ask you if for such size is normal to
> take that long, if so I'll try again, if not, if you don't think it
> should take that long then you have any other suggestion?
>
> Thanks again!
>
>
>
> Paragon Corporation-2 wrote:
>>
>> I wonder if its just called Equals in the old version.
>>
>> Try doing just ~=. If your geometries are exactly the same that will
>> probably be better anyway
>>
>>  DELETE FROM mystreets_table
>>  WHERE street = 'N/A' AND
>>  EXISTS (SELECT n.street FROM mystreets_table n  WHERE n.street <> 'N/A'
>>  AND n.street > ''  AND n.the_geom ~= mystreets_table.the_geom)
>>
>> Hope that helps.
>> Regina
>>
>> -----Original Message-----
>> From: postgis-users-bounces@...
>> [mailto:postgis-users-bounces@...] On Behalf Of
>> SenTnel
>> Sent: Saturday, June 07, 2008 4:36 PM
>> To: postgis-users@...
>> Subject: RE: [postgis-users] Updated shapefile to update postgres
>> database
>>
>>
>> Hello Regina! Thanks for your help, but when I try running it I got
>> this error message:
>>
>>
>> ERROR:  function st_equals(geometry, geometry) does not exist
>> HINT:  No function matches the given name and argument types. You may
>> need to add explicit type casts.
>>
>> :confused:
>> Thanks!
>>
>>
>>
>> Paragon Corporation-2 wrote:
>>>
>>> Here is a thought, but not sure how slow this will be
>>>
>>> DELETE FROM mystreets_table
>>> WHERE street = 'N/A' AND
>>> EXISTS (SELECT n.street FROM mystreets_table n  WHERE (n.street <> 'N/A'
>>> AND
>>> n.street > '')  AND ST_Equals(n.the_geom, mystreets_table.the_geom))
>>>
>>> The st_equals might be better to replace with ~=  if you want to
>>> match exact vertices.  I presume ~= is actually more efficient
>>>
>>> Hope that helps,
>>> Regina
>>>
>>> -----Original Message-----
>>> From: postgis-users-bounces@...
>>> [mailto:postgis-users-bounces@...] On Behalf Of
>>> SenTnel
>>> Sent: Saturday, June 07, 2008 2:25 AM
>>> To: postgis-users@...
>>> Subject: Re: [postgis-users] Updated shapefile to update postgres
>>> database
>>>
>>>
>>> Hello! Im confused about this issue:
>>>
>>> I have a street (polyline) duplicated, one on top of the other,
>>> exact same features, same "the_geom", but one bears the street name
>>> (column:"street") "main street" and the other bears "N/A" as street
>>> name. Now my maps shows the street but the name that shows is "N/A",
>>> and I would like to delete the "N/A" polyline, how can i do
>>> basically
>>> this:
>>>
>>> delete from mystreets_table street = 'N/A' where 'N/A' = street
>>> 'main street'.the_geom (delete from mystreets table the street named
'N/A'

>>> where the_geom in 'N/A'
>>> equals the_geom in street "main street") another words, delete only
>>> the N/A streets that the_geom is same as "main street", I hope you
>>> understand,
>>>
>>> Thanks!
>>>
>>>
>>>
>>>
>>> SenTnel wrote:
>>>>
>>>> Thanks Regina, Chris and all of you for your help. Im truly newby
>>>> but with such help Im ready to start the procedures, Ill keep you
>>>> posted of my progress or else if some issue comes along that may
>>>> require some light from you, Thanks again and I hope my
>>>> problem/solution may be of help to many others,
>>>>
>>>> Thanks again!
>>>>
>>>>
>>>>
>>>> SenTnel wrote:
>>>>>
>>>>> Hello!
>>>>>
>>>>> I would like some help updating a postgres database. This is the
>>>>> problem:
>>>>> We created the database using shp2pgsql to convert a shapefile
>>>>> that contains a city's street details (centerline), after we
>>>>> created the database we also created an aditional column to
>>>>> classify the street types
>>>>> (eg: street, avenues, highways, etc.), now we are working on the
>>>>> original shapefile, updating new roads, changes made to highways
>>>>> due to constructions modifications, etc., and we want to upload
>>>>> the "updated"
>>>>> shapefile, without afecting the actual database, another words, we
>>>>> would like to convert to postgres the updated shapefile with the
>>>>> new information (it cuold be the whole shapefile) but to keep the
>>>>> added clasiffication column intact. How can we do that?
>>>>>
>>>>> Thanks
>>>>>
>>>>
>>>>
>>>
>>> --
>>> View this message in context:
>>> http://www.nabble.com/Updated-shapefile-to-update-postgres-database-
>>> t
>>> p
>>> 170535
>>> 62p17705777.html
>>> Sent from the PostGIS - User mailing list archive at Nabble.com.
>>>
>>> _______________________________________________
>>> postgis-users mailing list
>>> postgis-users@...
>>> http://postgis.refractions.net/mailman/listinfo/postgis-users
>>>
>>>
>>>
>>> _______________________________________________
>>> postgis-users mailing list
>>> postgis-users@...
>>> http://postgis.refractions.net/mailman/listinfo/postgis-users
>>>
>>>
>>
>> --
>> View this message in context:
>> http://www.nabble.com/Updated-shapefile-to-update-postgres-database-t
>> p
>> 170535
>> 62p17712835.html
>> Sent from the PostGIS - User mailing list archive at Nabble.com.
>>
>> _______________________________________________
>> postgis-users mailing list
>> postgis-users@...
>> http://postgis.refractions.net/mailman/listinfo/postgis-users
>>
>>
>>
>> _______________________________________________
>> postgis-users mailing list
>> postgis-users@...
>> http://postgis.refractions.net/mailman/listinfo/postgis-users
>>
>>
>
> --
> View this message in context:
> http://www.nabble.com/Updated-shapefile-to-update-postgres-database-tp
> 170535
> 62p17846638.html
> Sent from the PostGIS - User mailing list archive at Nabble.com.
>
> _______________________________________________
> postgis-users mailing list
> postgis-users@...
> http://postgis.refractions.net/mailman/listinfo/postgis-users
>
>
> _______________________________________________
> postgis-users mailing list
> postgis-users@...
> http://postgis.refractions.net/mailman/listinfo/postgis-users
>
>

--
View this message in context:
http://www.nabble.com/Updated-shapefile-to-update-postgres-database-tp170535
62p17847480.html
Sent from the PostGIS - User mailing list archive at Nabble.com.

_______________________________________________
postgis-users mailing list
postgis-users@...
http://postgis.refractions.net/mailman/listinfo/postgis-users


_______________________________________________
postgis-users mailing list
postgis-users@...
http://postgis.refractions.net/mailman/listinfo/postgis-users