Shapely and polygon simplifications

[Helge Jensen-2]

I am doing simplification of polygons for storage and calculation
efficiency. and would like to do polygon-simplification.

Currently I use the postgis function ST_SimplifyPreserveTopology to do the
simplification, however the dataset is growing large and I would like to
move the pre-processing away from the database which should be busy
answering real-time queries.

I am already doing pre-processing (parsing from list(x,y,level),
rasterisation and polygonization) in python (and distributed too, thanks to
the new multiprocessing module).

So, I spotted the shapely bindings for GEOS, and the thread
http://www.mail-archive.com/community@.../msg00510.htmlwhich
indicated that a Douglas-Peucker variant was easily accessible for me
and my pre-processing code.

However I have checked out the trunk and not been able to "from
shapely.geos.ops import simplify", or even find the shapely.geos.ops module
_______________________________________________
Community mailing list
Community@...
http://lists.gispython.org/mailman/listinfo/community

[Sean Gillies-2]

On Nov 9, 2009, at 7:43 PM, Helge Jensen wrote:

> I am doing simplification of polygons for storage and calculation
> efficiency. and would like to do polygon-simplification.
>
> Currently I use the postgis function ST_SimplifyPreserveTopology to  
> do the
> simplification, however the dataset is growing large and I would  
> like to
> move the pre-processing away from the database which should be busy
> answering real-time queries.
>
> I am already doing pre-processing (parsing from list(x,y,level),
> rasterisation and polygonization) in python (and distributed too,  
> thanks to
> the new multiprocessing module).
>
> So, I spotted the shapely bindings for GEOS, and the thread
> http://www.mail-archive.com/community@.../msg00510.htmlwhich
> indicated that a Douglas-Peucker variant was easily accessible for me
> and my pre-processing code.
>
> However I have checked out the trunk and not been able to "from
> shapely.geos.ops import simplify", or even find the shapely.geos.ops  
> module

Hi,

In that email, I'd suggested that in the trunk I *might* move the  
simplify method to a function in an ops module, or provide a  
complementary function. The work hasn't been done yet.

Instead of using the trunk, I strongly recommend trying the 1.2  
branch. Aron Bierbaum has put a lot of work into it, tests are  
passing, and I think we're pretty close to where we can start to  
consider a preview release. A geometry simplify method or its  
functional counterpart isn't implemented there yet, but wouldn't be  
much work, and would be a lot closer to being deployable than the  
Shapely trunk.

http://svn.gispython.org/svn/gispy/Shapely/branches/1.2

Aron, what do you think about simplification for 1.2?

--
Sean Gillies
Programmer
Institute for the Study of the Ancient World
New York University

_______________________________________________
Community mailing list
Community@...
http://lists.gispython.org/mailman/listinfo/community

[Aron Bierbaum]

Sounds good to me. I have added two methods, simplify and
topology_preserve_simplify to the base geometry class in r1493. I
wasn't sure about the naming of the methods. The second method is a
little long, and I would normally use camel case, but I think this is
the Shapely standard. Please correct me if this is wrong though. I
copied a few of the tests from GEOS and they seem to work fine. Is
this what you were thinking for support in 1.2? I would agree that in
2.0 we could add something better since it might not be a OGC
standard. This should allow people to start using it though.

-Aron

On Mon, Nov 9, 2009 at 2:13 PM, Sean Gillies <sean.gillies@...> wrote:

> On Nov 9, 2009, at 7:43 PM, Helge Jensen wrote:
>
>> I am doing simplification of polygons for storage and calculation
>> efficiency. and would like to do polygon-simplification.
>>
>> Currently I use the postgis function ST_SimplifyPreserveTopology to
>> do the
>> simplification, however the dataset is growing large and I would
>> like to
>> move the pre-processing away from the database which should be busy
>> answering real-time queries.
>>
>> I am already doing pre-processing (parsing from list(x,y,level),
>> rasterisation and polygonization) in python (and distributed too,
>> thanks to
>> the new multiprocessing module).
>>
>> So, I spotted the shapely bindings for GEOS, and the thread
>> http://www.mail-archive.com/community@.../msg00510.htmlwhich
>> indicated that a Douglas-Peucker variant was easily accessible for me
>> and my pre-processing code.
>>
>> However I have checked out the trunk and not been able to "from
>> shapely.geos.ops import simplify", or even find the shapely.geos.ops
>> module
>
> Hi,
>
> In that email, I'd suggested that in the trunk I *might* move the
> simplify method to a function in an ops module, or provide a
> complementary function. The work hasn't been done yet.
>
> Instead of using the trunk, I strongly recommend trying the 1.2
> branch. Aron Bierbaum has put a lot of work into it, tests are
> passing, and I think we're pretty close to where we can start to
> consider a preview release. A geometry simplify method or its
> functional counterpart isn't implemented there yet, but wouldn't be
> much work, and would be a lot closer to being deployable than the
> Shapely trunk.
>
> http://svn.gispython.org/svn/gispy/Shapely/branches/1.2
>
> Aron, what do you think about simplification for 1.2?
>
> --
> Sean Gillies
> Programmer
> Institute for the Study of the Ancient World
> New York University
>
> _______________________________________________
> Community mailing list
> Community@...
> http://lists.gispython.org/mailman/listinfo/community
>

_______________________________________________
Community mailing list
Community@...
http://lists.gispython.org/mailman/listinfo/community

[Helge Jensen-2]

Thanks for a very resolution!

Based on your commit i can see that I was doing somewhat the right thing but should have guessed that the geom_factory was the thing to use on the results.

Hopefully I will have time to use it later today and report back (I can see you have added tests, so I expect it to work :)

--
Helge Jensen
Partner & Senior Software Developer



_______________________________________________
Community mailing list
Community@...
http://lists.gispython.org/mailman/listinfo/community

[Helge Jensen-2]

Excellent,

It seems to be working on my GDAL polygonized data.

It will be nice to move the simplification off the database. I process around 20k multipolygons covering roughly 50x50 km. each with roughly 10-20k polygons each having 100s of points in the outer-rings :)

On Tue, Nov 10, 2009 at 8:22 AM, Helge Jensen <hej@...> wrote:

Thanks for a very resolution!

Based on your commit i can see that I was doing somewhat the right thing but should have guessed that the geom_factory was the thing to use on the results.

Hopefully I will have time to use it later today and report back (I can see you have added tests, so I expect it to work :)

--
Helge Jensen
Partner & Senior Software Developer

--
Helge Jensen
Partner & Senior Software Developer



_______________________________________________
Community mailing list
Community@...
http://lists.gispython.org/mailman/listinfo/community

[Sean Gillies-2]

Aron,

I like it. We could consider collecting the methods into simplify with  
a preserve keyword option?

   def simplify(self, tolerance, preserve_topology=False):
       if preserve_topology:
           ...
       else:
           ...

The question then is whether the default is to preserve or not.

On Nov 10, 2009, at 5:26 AM, Aron Bierbaum wrote:

> Sounds good to me. I have added two methods, simplify and
> topology_preserve_simplify to the base geometry class in r1493. I
> wasn't sure about the naming of the methods. The second method is a
> little long, and I would normally use camel case, but I think this is
> the Shapely standard. Please correct me if this is wrong though. I
> copied a few of the tests from GEOS and they seem to work fine. Is
> this what you were thinking for support in 1.2? I would agree that in
> 2.0 we could add something better since it might not be a OGC
> standard. This should allow people to start using it though.
>
> -Aron
>
> On Mon, Nov 9, 2009 at 2:13 PM, Sean Gillies  
> <sean.gillies@...> wrote:
>> On Nov 9, 2009, at 7:43 PM, Helge Jensen wrote:
>>
>>> I am doing simplification of polygons for storage and calculation
>>> efficiency. and would like to do polygon-simplification.
>>>
>>> Currently I use the postgis function ST_SimplifyPreserveTopology to
>>> do the
>>> simplification, however the dataset is growing large and I would
>>> like to
>>> move the pre-processing away from the database which should be busy
>>> answering real-time queries.
>>>
>>> I am already doing pre-processing (parsing from list(x,y,level),
>>> rasterisation and polygonization) in python (and distributed too,
>>> thanks to
>>> the new multiprocessing module).
>>>
>>> So, I spotted the shapely bindings for GEOS, and the thread
>>> http://www.mail-archive.com/community@.../msg00510.htmlwhich
>>> indicated that a Douglas-Peucker variant was easily accessible for  
>>> me
>>> and my pre-processing code.
>>>
>>> However I have checked out the trunk and not been able to "from
>>> shapely.geos.ops import simplify", or even find the shapely.geos.ops
>>> module
>>
>> Hi,
>>
>> In that email, I'd suggested that in the trunk I *might* move the
>> simplify method to a function in an ops module, or provide a
>> complementary function. The work hasn't been done yet.
>>
>> Instead of using the trunk, I strongly recommend trying the 1.2
>> branch. Aron Bierbaum has put a lot of work into it, tests are
>> passing, and I think we're pretty close to where we can start to
>> consider a preview release. A geometry simplify method or its
>> functional counterpart isn't implemented there yet, but wouldn't be
>> much work, and would be a lot closer to being deployable than the
>> Shapely trunk.
>>
>> http://svn.gispython.org/svn/gispy/Shapely/branches/1.2
>>
>> Aron, what do you think about simplification for 1.2?
>>
>> --
>> Sean Gillies
>> Programmer
>> Institute for the Study of the Ancient World
>> New York University
>>
>> _______________________________________________
>> Community mailing list
>> Community@...
>> http://lists.gispython.org/mailman/listinfo/community
>>
> _______________________________________________
> Community mailing list
> Community@...
> http://lists.gispython.org/mailman/listinfo/community

--
Sean

_______________________________________________
Community mailing list
Community@...
http://lists.gispython.org/mailman/listinfo/community

[Kristian Thy]

On Tue, Nov 10, Sean Gillies wrote:

> I like it. We could consider collecting the methods into simplify with  
> a preserve keyword option?
>
>    def simplify(self, tolerance, preserve_topology=False):
>        if preserve_topology:
>            ...
>        else:
>            ...
>
> The question then is whether the default is to preserve or not.

I would expect the default to preserve topology to avoid surprises.

\\kristian
--
... et nemo ex vobis interrogat me: »Quo vadis?«
_______________________________________________
Community mailing list
Community@...
http://lists.gispython.org/mailman/listinfo/community

[Helge Jensen-2]

By the principle of "least-surprise" preserve_topology should probably default to True.

Powerusers will probably know that preserve_topology=True is more costly. Perhaps documentation could reflect this?

There is some documentation in the GEOS headers (DouglasPeuckerSimplifier.h and TopologyPreservingSimplifier.h) that could be repeated for the simplify function.


--
Helge Jensen
Partner & Senior Software Developer



_______________________________________________
Community mailing list
Community@...
http://lists.gispython.org/mailman/listinfo/community

[Aron Bierbaum]

I had considered adding something like that, but I didn't know if  
Shapley was meant to mirror the GEOS API more closely. I completely  
agree that it makes more sense as a single method. It appears that  
people agree that by default we should preserve topology too.

I also tried to find a default value for the tolerance. Any ideas  
opinions on a default for this? As far as I could tell there was no  
default in GEOS?

-Aron

Aron Bierbaum
Software Engineer
Priority 5

On Nov 10, 2009, at 4:40 AM, Helge Jensen <hej@...> wrote:

> By the principle of "least-surprise" preserve_topology should  
> probably default to True.
>
> Powerusers will probably know that preserve_topology=True is more  
> costly. Perhaps documentation could reflect this?
>
> There is some documentation in the GEOS headers  
> (DouglasPeuckerSimplifier.h and TopologyPreservingSimplifier.h) that  
> could be repeated for the simplify function.
>
>
> --
> Helge Jensen
> Partner & Senior Software Developer
>
>
> _______________________________________________
> Community mailing list
> Community@...
> http://lists.gispython.org/mailman/listinfo/community

_______________________________________________
Community mailing list
Community@...
http://lists.gispython.org/mailman/listinfo/community

[Helge Jensen-2]

With regards to the choice between mirroring the GEOS api or a more "pythonic" approach, please take my input with a grain of salt as I have only just started using shapely :)

However you already have pythonic wrapping of the geometries and have simplify(*) as member-functions, not free functions, so my personal feeling about it is that you are already quite a long way down the road of a pythonic library and not a mirror of the geos API.

Shapely doesn't "feel" like GEOS, nor is it limited to GEOS, and frankly, it is nice with a pythonic python-gis library for a change (as opposed to, the GDAL bindings for example).

I don't think there are any strong candidates for a default tolerance.

One could argue (very weakly, in my eyes) for either 0.0, float-epsilon or 1.0 as the default tolerance, but I think those are all rather sought. I would personally prefer that there was no default.

On Tue, Nov 10, 2009 at 1:36 PM, Aron Bierbaum <aronbierbaum@...> wrote:

I had considered adding something like that, but I didn't know if
Shapley was meant to mirror the GEOS API more closely. I completely
agree that it makes more sense as a single method. It appears that
people agree that by default we should preserve topology too.

I also tried to find a default value for the tolerance. Any ideas
opinions on a default for this? As far as I could tell there was no
default in GEOS?

-Aron

Aron Bierbaum
Software Engineer
Priority 5

On Nov 10, 2009, at 4:40 AM, Helge Jensen <hej@...> wrote:

> By the principle of "least-surprise" preserve_topology should
> probably default to True.
>
> Powerusers will probably know that preserve_topology=True is more
> costly. Perhaps documentation could reflect this?
>
> There is some documentation in the GEOS headers
> (DouglasPeuckerSimplifier.h and TopologyPreservingSimplifier.h) that
> could be repeated for the simplify function.
>
>
> --
> Helge Jensen
> Partner & Senior Software Developer
>
>

> _______________________________________________
> Community mailing list
> Community@...
> http://lists.gispython.org/mailman/listinfo/community
_______________________________________________
Community mailing list
Community@...
http://lists.gispython.org/mailman/listinfo/community

--
Helge Jensen
Partner & Senior Software Developer



_______________________________________________
Community mailing list
Community@...
http://lists.gispython.org/mailman/listinfo/community

[Sean Gillies-2]

Agreed. A default tolerance must not be tolerated ;)

On Nov 10, 2009, at 7:02 PM, Helge Jensen wrote:

> With regards to the choice between mirroring the GEOS api or a more  
> "pythonic" approach, please take my input with a grain of salt as I  
> have only just started using shapely :)
>
> However you already have pythonic wrapping of the geometries and  
> have simplify(*) as member-functions, not free functions, so my  
> personal feeling about it is that you are already quite a long way  
> down the road of a pythonic library and not a mirror of the geos API.
>
> Shapely doesn't "feel" like GEOS, nor is it limited to GEOS, and  
> frankly, it is nice with a pythonic python-gis library for a change  
> (as opposed to, the GDAL bindings for example).
>
> I don't think there are any strong candidates for a default tolerance.
>
> One could argue (very weakly, in my eyes) for either 0.0, float-
> epsilon or 1.0 as the default tolerance, but I think those are all  
> rather sought. I would personally prefer that there was no default.
>
> On Tue, Nov 10, 2009 at 1:36 PM, Aron Bierbaum  
> <aronbierbaum@...> wrote:
> I had considered adding something like that, but I didn't know if
> Shapley was meant to mirror the GEOS API more closely. I completely
> agree that it makes more sense as a single method. It appears that
> people agree that by default we should preserve topology too.
>
> I also tried to find a default value for the tolerance. Any ideas
> opinions on a default for this? As far as I could tell there was no
> default in GEOS?
>
> -Aron
>
> Aron Bierbaum
> Software Engineer
> Priority 5
>
> On Nov 10, 2009, at 4:40 AM, Helge Jensen <hej@...> wrote:
>
> > By the principle of "least-surprise" preserve_topology should
> > probably default to True.
> >
> > Powerusers will probably know that preserve_topology=True is more
> > costly. Perhaps documentation could reflect this?
> >
> > There is some documentation in the GEOS headers
> > (DouglasPeuckerSimplifier.h and TopologyPreservingSimplifier.h) that
> > could be repeated for the simplify function.
> >
> >
> > --
> > Helge Jensen
> > Partner & Senior Software Developer
> >
> >
> > _______________________________________________
> > Community mailing list
> > Community@...
> > http://lists.gispython.org/mailman/listinfo/community
> _______________________________________________
> Community mailing list
> Community@...
> http://lists.gispython.org/mailman/listinfo/community
>
>
>
> --
> Helge Jensen
> Partner & Senior Software Developer
>
>
> _______________________________________________
> Community mailing list
> Community@...
> http://lists.gispython.org/mailman/listinfo/community

--
Sean Gillies
Programmer
Institute for the Study of the Ancient World
New York University



--
Sean

_______________________________________________
Community mailing list
Community@...
http://lists.gispython.org/mailman/listinfo/community