Esri shapefile size constraint

[Keenan ()]

Hi,

     I wanted to read a shapefile using java and its successfully working using geotools.
     I am using the program given in this link
http://docs.codehaus.org/display/GEOTDOC/04+How+to+Read+a+Shapefile

    This program works shapefiles in size of KB .It works upto 4 MB sized .shp files, though it takes time to read, but thats fine. Anyhoo when the size goes above 4.5MB(approx) it gives this error.


Mar 17, 2008 4:00:03 PM org.geotools.data.db2.DB2DataStoreFactory isAvailable
INFO: DB2 driver found: false
java.lang.IllegalArgumentException: Duplicate AttributeTypes DefaultAttributeType [name=BCCONT40_C , type=class java.lang.Long , nillable=true, min=0, max=0]
        at org.geotools.feature.FeatureTypeBuilder.check(FeatureTypeBuilder.java:607)
        at org.geotools.feature.FeatureTypeBuilder.addType(FeatureTypeBuilder.java:354)
        at org.geotools.feature.FeatureTypeBuilder.addTypes(FeatureTypeBuilder.java:334)
        at org.geotools.feature.FeatureTypes.newFeatureType(FeatureTypes.java:266)
        at org.geotools.feature.FeatureTypes.newFeatureType(FeatureTypes.java:216)
        at org.geotools.data.shapefile.indexed.IndexedShapefileDataStore.getSchema(IndexedShapefileDataStore.java:912)
        at org.geotools.data.shapefile.indexed.IndexedShapefileDataStore.getSchema(IndexedShapefileDataStore.java:891)
        at org.geotools.data.shapefile.indexed.IndexedShapefileDataStore.getFeatureSource(IndexedShapefileDataStore.java:1001)
        at src.ShapefileRead.main(ShapefileRead.java:61)

Is this a problem with the ESRI or woth the geotools code?

TIA
KEENAn

[Jody Garnett]

Yes the shapefile reading code actually does not *read* anything until
you open up an iterator(); and then it only keeps the file open as you
call next(), .. hasNext(), ... next() ... etc...

So it streams the content into your application as you read; it does NOT
load it into memory allowing you to work with massive files.
Note GIS data is almost always big; so this approach is needed.

You can find large shapefiles on the web; have a look at GeoNetwork and
the uDig data page.
Jody

> Hi,
>
>      I wanted to read a shapefile using java and its successfully working
> using geotools.
>      I am using the program given in this link
> http://docs.codehaus.org/display/GEOTDOC/04+How+to+Read+a+Shapefile
>
>     This program works shapefiles in size of KB . i want to know if the
> program can read files exceeding in sizes that is files going upto a several
> MB's.
> PS :  I am asking this question because i am not able to find any shapefile
> with size in MB.
>
> TIA
> KEENAn
>
>  

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

[Keenan]

Yes, i am using an iterator to stream through the contents of the shapefile  like i am extracting details like geometry, attributes,fid etc. But what is the error all about if its not the size that matters? Could you please explain that and if possible provide a solution? I have breaking my head over this for sometime now.
I have posted the error in my first message

Thanks a ton
Keenan

Jody Garnett wrote:

Yes the shapefile reading code actually does not *read* anything until
you open up an iterator(); and then it only keeps the file open as you
call next(), .. hasNext(), ... next() ... etc...

So it streams the content into your application as you read; it does NOT
load it into memory allowing you to work with massive files.
Note GIS data is almost always big; so this approach is needed.

You can find large shapefiles on the web; have a look at GeoNetwork and
the uDig data page.
Jody
> Hi,
>
>      I wanted to read a shapefile using java and its successfully working
> using geotools.
>      I am using the program given in this link
> http://docs.codehaus.org/display/GEOTDOC/04+How+to+Read+a+Shapefile
>
>     This program works shapefiles in size of KB . i want to know if the
> program can read files exceeding in sizes that is files going upto a several
> MB's.
> PS :  I am asking this question because i am not able to find any shapefile
> with size in MB.
>
> TIA
> KEENAn
>
>  


---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

[Jody Garnett]

You never mentioned an error; even in your first message... we should
actually use the geotools - user list to talk over the use of the library.
Jody
Keenan wrote:

> Yes, i am using an iterator to stream through the contents of the shapefile
> like i am extracting details like geometry, attributes,fid etc. But what is
> the error all about if its not the size that matters? Could you please
> explain that and if possible provide a solution? I have breaking my head
> over this for sometime now.
> I have posted the error in my first message
>
> Thanks a ton
> Keenan
>
> Jody Garnett wrote:
>  
>> Yes the shapefile reading code actually does not *read* anything until
>> you open up an iterator(); and then it only keeps the file open as you
>> call next(), .. hasNext(), ... next() ... etc...
>>
>> So it streams the content into your application as you read; it does NOT
>> load it into memory allowing you to work with massive files.
>> Note GIS data is almost always big; so this approach is needed.
>>
>> You can find large shapefiles on the web; have a look at GeoNetwork and
>> the uDig data page.
>> Jody
>>    

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email