Buildmap-osm progress

View: New views
4 Messages — Rating Filter:   Alert me  

Buildmap-osm progress

by Danny Backx :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Hi,

Yesterday's change gets buildmap_osm to work on the UK map for about 810
minutes, and then die :

../buildmap_osm -i iso-uk.osm -o iso-uk.rdm
-- Dictionary statistics:
--    attributes: 14 items, 3 hits, 199 bytes
                     14 references, 1 nodes
--    prefix: 1 items, 0 hits, 13 bytes
                     1 references, 1 nodes
--    street: 64476 items, 76643 hits, 1986970 bytes
                     130011 references, 65536 nodes
--    type: 1 items, 0 hits, 13 bytes
                     1 references, 1 nodes
--    suffix: 1 items, 0 hits, 13 bytes
                     1 references, 1 nodes
--    city: 1405 items, 16 hits, 32039 bytes
                     2233 references, 829 nodes
-- totals: 65898 items, 76662 hits, 2019247 bytes, 132261 references,
66369 nodes
## dictionary tree full

This is an improvement over the previous state :-) I'll try to get this
fixed too.

        Danny
--
Danny Backx ; danny.backx - at - scarlet.be ; http://danny.backx.info


------------------------------------------------------------------------------
The NEW KODAK i700 Series Scanners deliver under ANY circumstances! Your
production scanning environment may not be a perfect world - but thanks to
Kodak, there's a perfect scanner to get the job done! With the NEW KODAK i700
Series Scanner you'll get full speed at 300 dpi even with all image
processing features enabled. http://p.sf.net/sfu/kodak-com
_______________________________________________
Roadmap-general mailing list
Roadmap-general@...
https://lists.sourceforge.net/lists/listinfo/roadmap-general

Re: Buildmap-osm progress

by Paul Fox-3 :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

danny wrote:
 > Hi,
 >
 > Yesterday's change gets buildmap_osm to work on the UK map for about 810
 > minutes, and then die :

i think at some point we're going to have to figure out how to
split the OSM-derived country maps into chunks.  doing whole
countries at once will always cause memory issues when building
the maps _and_ when rendering them.  the county level of granularity
is more than just an inconvenience imposed by Tiger data -- it gives
roadmap a reasonable "working set" to use, on devices with moderate
memory and cpu capacity.  when i did the quadtiles, i chose the default
sizes of the tiles to be roughly "county"-sized -- i.e. under 50 miles
across.

paul

 >
 > ../buildmap_osm -i iso-uk.osm -o iso-uk.rdm
 > -- Dictionary statistics:
 > --    attributes: 14 items, 3 hits, 199 bytes
 >                      14 references, 1 nodes
 > --    prefix: 1 items, 0 hits, 13 bytes
 >                      1 references, 1 nodes
 > --    street: 64476 items, 76643 hits, 1986970 bytes
 >                      130011 references, 65536 nodes
 > --    type: 1 items, 0 hits, 13 bytes
 >                      1 references, 1 nodes
 > --    suffix: 1 items, 0 hits, 13 bytes
 >                      1 references, 1 nodes
 > --    city: 1405 items, 16 hits, 32039 bytes
 >                      2233 references, 829 nodes
 > -- totals: 65898 items, 76662 hits, 2019247 bytes, 132261 references,
 > 66369 nodes
 > ## dictionary tree full
 >
 > This is an improvement over the previous state :-) I'll try to get this
 > fixed too.
 >
 > Danny
 > --
 > Danny Backx ; danny.backx - at - scarlet.be ; http://danny.backx.info
 >
 >
 > ------------------------------------------------------------------------------
 > The NEW KODAK i700 Series Scanners deliver under ANY circumstances! Your
 > production scanning environment may not be a perfect world - but thanks to
 > Kodak, there's a perfect scanner to get the job done! With the NEW KODAK i700
 > Series Scanner you'll get full speed at 300 dpi even with all image
 > processing features enabled. http://p.sf.net/sfu/kodak-com
 > _______________________________________________
 > Roadmap-general mailing list
 > Roadmap-general@...
 > https://lists.sourceforge.net/lists/listinfo/roadmap-general

=---------------------
 paul fox, pgf@... (arlington, ma, where it's 59.9 degrees)

------------------------------------------------------------------------------
The NEW KODAK i700 Series Scanners deliver under ANY circumstances! Your
production scanning environment may not be a perfect world - but thanks to
Kodak, there's a perfect scanner to get the job done! With the NEW KODAK i700
Series Scanner you'll get full speed at 300 dpi even with all image
processing features enabled. http://p.sf.net/sfu/kodak-com
_______________________________________________
Roadmap-general mailing list
Roadmap-general@...
https://lists.sourceforge.net/lists/listinfo/roadmap-general

Re: Buildmap-osm progress

by Dodi :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

> -----Original Message-----
> From: Paul Fox [mailto:pgf@...]
> Sent: Wednesday, May 13, 2009 4:08 PM
> To: General RoadMap discussion
> Subject: Re: [Roadmap-general] Buildmap-osm progress
>
> danny wrote:
>  > Hi,
>  >
>  > Yesterday's change gets buildmap_osm to work on the UK map for about
> 810
>  > minutes, and then die :
>
> i think at some point we're going to have to figure out how to
> split the OSM-derived country maps into chunks.  doing whole
> countries at once will always cause memory issues when building
> the maps _and_ when rendering them.  
>

... you can use  osmosis as a preprocessor for spliting countries or whole
continents into smaller chunks ...
http://wiki.openstreetmap.org/wiki/Osmosis

or osmccut   http://wiki.openstreetmap.org/wiki/Osmcut

or splitter http://wiki.openstreetmap.org/wiki/Splitter - variable sized
tiles (dimension is viariable, size is constant)


Dodi

> paul
>


------------------------------------------------------------------------------
The NEW KODAK i700 Series Scanners deliver under ANY circumstances! Your
production scanning environment may not be a perfect world - but thanks to
Kodak, there's a perfect scanner to get the job done! With the NEW KODAK i700
Series Scanner you'll get full speed at 300 dpi even with all image
processing features enabled. http://p.sf.net/sfu/kodak-com
_______________________________________________
Roadmap-general mailing list
Roadmap-general@...
https://lists.sourceforge.net/lists/listinfo/roadmap-general

Re: Buildmap-osm progress

by Danny Backx :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Yeah, I'm not trying to get a map of the whole world to build. There
must be practical limits, I have no interest in pushing these.

That said, we did have a bug that got reported a couple of times, and it
looks like I finally found it.

        Danny

On Wed, 2009-05-13 at 10:08 -0400, Paul Fox wrote:

> danny wrote:
>  > Hi,
>  >
>  > Yesterday's change gets buildmap_osm to work on the UK map for about 810
>  > minutes, and then die :
>
> i think at some point we're going to have to figure out how to
> split the OSM-derived country maps into chunks.  doing whole
> countries at once will always cause memory issues when building
> the maps _and_ when rendering them.  the county level of granularity
> is more than just an inconvenience imposed by Tiger data -- it gives
> roadmap a reasonable "working set" to use, on devices with moderate
> memory and cpu capacity.  when i did the quadtiles, i chose the default
> sizes of the tiles to be roughly "county"-sized -- i.e. under 50 miles
> across.
>
> paul
>
>  >
>  > ../buildmap_osm -i iso-uk.osm -o iso-uk.rdm
>  > -- Dictionary statistics:
>  > --    attributes: 14 items, 3 hits, 199 bytes
>  >                      14 references, 1 nodes
>  > --    prefix: 1 items, 0 hits, 13 bytes
>  >                      1 references, 1 nodes
>  > --    street: 64476 items, 76643 hits, 1986970 bytes
>  >                      130011 references, 65536 nodes
>  > --    type: 1 items, 0 hits, 13 bytes
>  >                      1 references, 1 nodes
>  > --    suffix: 1 items, 0 hits, 13 bytes
>  >                      1 references, 1 nodes
>  > --    city: 1405 items, 16 hits, 32039 bytes
>  >                      2233 references, 829 nodes
>  > -- totals: 65898 items, 76662 hits, 2019247 bytes, 132261 references,
>  > 66369 nodes
>  > ## dictionary tree full
>  >
>  > This is an improvement over the previous state :-) I'll try to get this
>  > fixed too.
>  >
>  > Danny
>  > --
>  > Danny Backx ; danny.backx - at - scarlet.be ; http://danny.backx.info
>  >
>  >
>  > ------------------------------------------------------------------------------
>  > The NEW KODAK i700 Series Scanners deliver under ANY circumstances! Your
>  > production scanning environment may not be a perfect world - but thanks to
>  > Kodak, there's a perfect scanner to get the job done! With the NEW KODAK i700
>  > Series Scanner you'll get full speed at 300 dpi even with all image
>  > processing features enabled. http://p.sf.net/sfu/kodak-com
>  > _______________________________________________
>  > Roadmap-general mailing list
>  > Roadmap-general@...
>  > https://lists.sourceforge.net/lists/listinfo/roadmap-general
>
> =---------------------
>  paul fox, pgf@... (arlington, ma, where it's 59.9 degrees)
>
> ------------------------------------------------------------------------------
> The NEW KODAK i700 Series Scanners deliver under ANY circumstances! Your
> production scanning environment may not be a perfect world - but thanks to
> Kodak, there's a perfect scanner to get the job done! With the NEW KODAK i700
> Series Scanner you'll get full speed at 300 dpi even with all image
> processing features enabled. http://p.sf.net/sfu/kodak-com
> _______________________________________________
> Roadmap-general mailing list
> Roadmap-general@...
> https://lists.sourceforge.net/lists/listinfo/roadmap-general
>
--
Danny Backx ; danny.backx - at - scarlet.be ; http://danny.backx.info


------------------------------------------------------------------------------
The NEW KODAK i700 Series Scanners deliver under ANY circumstances! Your
production scanning environment may not be a perfect world - but thanks to
Kodak, there's a perfect scanner to get the job done! With the NEW KODAK i700
Series Scanner you'll get full speed at 300 dpi even with all image
processing features enabled. http://p.sf.net/sfu/kodak-com
_______________________________________________
Roadmap-general mailing list
Roadmap-general@...
https://lists.sourceforge.net/lists/listinfo/roadmap-general