Re: Convert from KML to gdb bug

### Eclipse Workspace Patch 1.0
#P gpsbabel
Index: kml.c
===================================================================
RCS file: /cvsroot/gpsbabel/gpsbabel/kml.c,v
retrieving revision 1.111
diff -u -r1.111 kml.c
--- kml.c 25 Jun 2011 22:03:46 -0000 1.111
+++ kml.c 29 Nov 2011 00:41:33 -0000
@@ -277,7 +277,7 @@
  }
  track_add_head(trk_head);
 
- while ((n = sscanf(args, "%lf,%lf,%lf %n", &lon, &lat, &alt, &consumed)) > 0) {
+ while ((n = sscanf(args, "%lf,%lf,%lf%n", &lon, &lat, &alt, &consumed)) > 0) {
 
  trkpt = waypt_new();
  trkpt->latitude = lat;

------------------------------------------------------------------------------
All the data continuously generated in your IT infrastructure
contains a definitive record of customers, application performance,
security threats, fraudulent activity, and more. Splunk takes this
data and makes sense of it. IT sense. And common sense.
http://p.sf.net/sfu/splunk-novd2d
_______________________________________________
Gpsbabel-misc mailing list http://www.gpsbabel.org
Gpsbabel-misc@...
To unsubscribe, change list options, or see archives, visit:
https://lists.sourceforge.net/lists/listinfo/gpsbabel-misc

Nice work Dave. 
The source code is now under subversion here http://gpsbabel.googlecode.com/svn/trunk/gpsbabel
It seems the one character solution may be one too few.  Lines 306 and 314 both have the blank before the %n. 

There does seem to be some confusion/history about the %n directive and the return value, and the applicability of the assignment suppression character.

man page:

       n      Nothing  is  expected;  instead,  the  number  of characters consumed thus far from the input is stored through the next pointer, which must be a
              pointer to int.  This is not a conversion, although it can be suppressed with the * assignment-suppression character.  The C standard says: "Exe-
              cution of a %n directive does not increment the assignment count returned at the completion of execution" but the Corrigendum seems to contradict
              this. Probably it is wise not to make any assumptions on the effect of %n conversions on the return value.

WG14/N1256 Committee Draft — Septermber 7, 2007 ISO/IEC 9899:TC3:

n No input is consumed. The corresponding argument shall be a pointer to
signed integer into which is to be written the number of characters read from
the input stream so far by this call to the fscanf function. Execution of a
%n directive does not increment the assignment count returned at the
completion of execution of the fscanf function. No argument is converted,
but one is consumed. If the conversion specification includes an assignmentsuppressing
character or a field width, the behavior is undefined.

21 EXAMPLE 4 In:
#include <stdio.h>
/* ... */
int d1, d2, n1, n2, i;
i = sscanf("123", "%d%n%n%d", &d1, &n1, &n2, &d2);
the value 123 is assigned to d1 and the value 3 to n1. Because %n can never get an input failure the value
of 3 is also assigned to n2. The value of d2 is not affected. The value 1 is assigned to i.

    293 void trk_coord(const char* args, const char** attrv)
    294 {
    295   int consumed = 0;
    296   double lat, lon, alt;
    297   waypoint* trkpt;
    298   int n = 0;
    299
    300   route_head* trk_head = route_head_alloc();
    301   if (wpt_tmp->shortname) {
    302     trk_head->rte_name  = xstrdup(wpt_tmp->shortname);
    303   }
    304   track_add_head(trk_head);
    305
    306   while ((n = sscanf(args, "%lf,%lf,%lf %n", &lon, &lat, &alt, &consumed)) > 0) {
    307
    308     trkpt = waypt_new();
    309     trkpt->latitude = lat;
    310     trkpt->longitude = lon;
    311
    312     // Line malformed or two-arg format without alt .  Rescan.
    313     if (2 == n) {
    314       sscanf(args, "%lf,%lf %n", &lon, &lat, &consumed);
    315     }
    316
    317     if (3 == n) {
    318       trkpt->altitude = alt;
    319     }
    320
    321     track_add_wpt(trk_head, trkpt);
    322
    323     args += consumed;
    324   }
    325 }

Best Regards,
Steve

On 11/28/2011 6:25 PM, Eccleston, Dave wrote:

I have isolated the problem down to a particular function: trk_coord in kml.c (line 267).  Apparently there is a difference in the implementation of the %n format specifier for sscanf between Windows and Linux.  This problem is documented in this post (from 1998!): http://cygwin.com/ml/cygwin/1998-06/msg00463.html.  I have attached a patch that fixes the problem.

 

Detailed Explanation: The problem occurs because the format string for the sscanf in this function specifies a space followed by %n.  A %n in the format is supposed to assign the number of characters read.  The gpsbabel trk_coord function uses it to populate a variable ‘consumed’, which is then used to increment a string pointer to args.  Because the last coordinates in a linestring often don’t have a space following them, the sscanf stops scanning at the space and does not continue to execute the %n.  This then fails to change the ‘consumed’ variable, which is left with the value from the previous time through the loop (instead of 0 as you might expect).  When the too-big consumed variable is used to increment the args pointer, it is incremented past the end of the string into uncharted memory.  Results from this point on will vary depending on memory contents.  Debug printf’s show it’s actually happening to me at the end of every linestring, but only a couple of the tracks were followed by memory that looked like data to sscanf: in these cases it continued to parse and add garbage to my tracks.

 

This also explains why Steven couldn’t reproduce the problem: symptoms of the problem will depend on both OS, c compiler, and memory contents, so a VM system could easily be different in one of these ways.

 

Solution: There are several ways to fix the problem.  In my opinion, the easiest is to remove the space from the sscanf format on line 280 (see attached patch).  The space is not necessary, as sscanf will automatically skip any whitespace before or after the comma-separated values.  Whew, all that for a one character solution!

 

Thanks,

Dave

 

p.s. Yikes, C!  I’ve been a java developer for the past 12 years, it took a while to get back into the swing of running and debugging C.

 

 

From: tsteven4 [tsteven4@...]
Sent: Tuesday, November 22, 2011 9:35 PM
To: Eccleston, Dave
Cc: gpsbabel-misc@...
Subject: Re: [Gpsbabel-misc] Convert from KML to gdb bug

 

I generated this on a centos 5.7 guest running under a windows 7 64bit host with virtual box.

You could try to set the debug level:
    -D level         Set debug level [0]

Did you use the gui or command line?

I will send you the gpx file.

On 11/22/2011 7:20 PM, Eccleston, Dave wrote:

I’m not really familiar with gpsbabel, is there any place to look for debug/error output?  Are you on the same OS as me? I’m on Windows 7 64bit.   

 

Also, would you mind sending me the gpx output file you created? That would be a big help to me for the project I’m working on.

 

Thanks for your help,

Dave

 

 

From: tsteven4 [tsteven4@...]
Sent: Tuesday, November 22, 2011 9:02 PM
To: Eccleston, Dave
Cc: gpsbabel-misc@...
Subject: Re: [Gpsbabel-misc] Convert from KML to gdb bug

 

Dave,

I tried to convert your invalid document anyway.  I could not reproduce your problem.  I used gpsbabel 1.4.2.  gpsbabel produced valid output despite the invalid input. 

gpsbabel142 -i kml -f doc.kml -o gpx -F doc.gpx

<trkpt lat="38.738311000" lon="-79.699404000">
  <ele>0.000000</ele>
</trkpt>
<trkpt lat="38.736838000" lon="-79.701963000">
  <ele>0.000000</ele>
</trkpt>
<trkpt lat="38.736838000" lon="-79.701963000">
  <ele>0.000000</ele>
</trkpt>
<trkpt lat="38.736834000" lon="-79.701959000">
  <ele>0.000000</ele>
</trkpt>
</trkseg>
</trk>
<trk>
  <name>Laurel River Trail</name>

On 11/22/2011 3:27 PM, Eccleston, Dave wrote:

Thanks for your quick response.  Unfortunately, I have similar issues with gpx format.  However, the nice thing about gpx is that you can see where it goes wrong.  There are two trails that are not converted correctly: North Fork Mountain Trail and Laurel River Trail.  If you look at the gpx output you can see a point where it stops reporting elevation and the longitude goes to zero (see snippet below of Laurel Trail).  The original kml has valid data all the way through.

 

I think this points to a problem with gpsbabel since it’s the only tool in the process from kml to gpx.  I may be able to hand repair the file at this point, so thanks for the gpx suggestion.  Let me know if you need any more info about the issue though, I’d be interested if there’s a fix.

 

Dave

 

<trkpt lat="38.736838000" lon="-79.701963000">

  <ele>0.000000</ele>

</trkpt>

<trkpt lat="38.736838000" lon="-79.701963000">

  <ele>0.000000</ele>

</trkpt>

<trkpt lat="38.736834000" lon="-79.701959000">

  <ele>0.000000</ele>

</trkpt>

<trkpt lat="38.736834000" lon="0.000000000">

</trkpt>

<trkpt lat="38.736834000" lon="0.000000000">

</trkpt>

<trkpt lat="38.736834000" lon="0.000000000">

</trkpt>

<trkpt lat="38.736834000" lon="0.000000000">

</trkpt>

 

-----Original Message-----
From: Maarten Sneep [maarten.sneep@...]
Sent: Tuesday, November 22, 2011 3:07 PM
To: Eccleston, Dave
Cc: gpsbabel-misc@...
Subject: Re: [Gpsbabel-misc] Convert from KML to gdb bug

 

 

On 22 nov. 2011, at 19:43, Eccleston, Dave wrote:

 

> The link below is a KML representation of trails in Monongahela National Forest. Downloading this link and viewing in Google Earth works without issue.  However, when I convert using GPSBabel to gdb, the resulting file (when viewed in MapSource) has a few points that leave the United States.  It seems to be the result of lat/lon values being incorrectly converted to 0 values.  There are two trails in particular that seem to have the issue: Moore Run Trail and Burner Mountain Trail.

> 

> I do the following process:

> ·         Rename the file to be .zip.

> ·         Extract the doc.kml from the zip.

> ·         Run GPSBabel on doc.kml, using source format Google Earth (Keyhole) Markup Language and destination format Garmin MapSource – gdb.

> 

> The very odd thing about this case is that if I edit the kml file to only contain the trouble files (Moore and Burner Mountain Trail) then both trails display correctly after conversion.

> 

> Software versions I am using are: GPS Babel 1.4.2., and MapSource 6.16.3 (I checked and both appear to be up-to-date versions).

>

 

gdb is as far as I understand it a reverse engineered format, that is almost completely supported, but may have issues. I tried a conversion to gpx (which MapSource can read as well), and did not encounter any issues. Note that I used RoadTrip as I'm on a Mac.

 

Best,

 

Maarten

 

------------------------------------------------------------------------------

All the data continuously generated in your IT infrastructure 

contains a definitive record of customers, application performance, 

security threats, fraudulent activity, and more. Splunk takes this 

data and makes sense of it. IT sense. And common sense.

http://p.sf.net/sfu/splunk-novd2d

_______________________________________________

Gpsbabel-misc mailing list http://www.gpsbabel.org

Gpsbabel-misc@...

To unsubscribe, change list options, or see archives, visit:

https://lists.sourceforge.net/lists/listinfo/gpsbabel-misc

------------------------------------------------------------------------------
All the data continuously generated in your IT infrastructure
contains a definitive record of customers, application performance,
security threats, fraudulent activity, and more. Splunk takes this
data and makes sense of it. IT sense. And common sense.
http://p.sf.net/sfu/splunk-novd2d
_______________________________________________
Gpsbabel-misc mailing list http://www.gpsbabel.org
Gpsbabel-misc@...
To unsubscribe, change list options, or see archives, visit:
https://lists.sourceforge.net/lists/listinfo/gpsbabel-misc

------------------------------------------------------------------------------
All the data continuously generated in your IT infrastructure
contains a definitive record of customers, application performance,
security threats, fraudulent activity, and more. Splunk takes this
data and makes sense of it. IT sense. And common sense.
http://p.sf.net/sfu/splunk-novd2d
_______________________________________________
Gpsbabel-misc mailing list http://www.gpsbabel.org
Gpsbabel-misc@...
To unsubscribe, change list options, or see archives, visit:
https://lists.sourceforge.net/lists/listinfo/gpsbabel-misc