Re: SQLite on PocketBook

[D. Richard Hipp]

On Nov 4, 2009, at 4:53 AM, Alexander Drozd wrote:

>
>  My name is Alexander. I am working on an open-source  spaced-
> repetition software project  (http://code.google.com/p/pbanki/). My  
> software relies on SQLite library. I came across some bug-like  
> problems with running SQLite on a low-memory e-ink reader device. I  
> am very  sorry to bother you, but I tried to submit my  problem to  
> the bugtracker at the SQLite site, and for some reason anonymous  
> login failed.
>
>  The problem appears at the point of reading real values from an  
> SQLite database. I created a simple database
>
> CREATE TABLE cards (
>    text TEXT NOT NULL,
>    value REAL NOT NULL
> );
>
> I also tried to use NUMERIC and FLOAT instead of REAL. Then I  
> inserted a few values:
>
> INSERT INTO cards VALUES('second',100.1);
> INSERT INTO cards VALUES('first', 100.0);
>
> Then I execute "select * from cards order by due" query with sample  
> code from http://www.sqlite.org/quickstart.html It works perfectly  
> when compiled on desktop computer, but fails on target device. The  
> device is PocketBook301+ (http://pocketbook.com.ua/). Unfortunately  
> their site does not have an English version. This device is based on  
> Samsung S3C2440 AL-40 CPU. It runs under open-source firmware called  
> pocketbookfree, that is based on Linux 2.6.18.2 armv4tl.
>
> The above query run on pocketbook returns corrupted values for  
> floats if they have a non-zero fractional part:
>
> text = first
> val = 100.0
>
> text = second
> val = 1.90359837350824e+185
>
> Sorting by columns containing float numbers also fails when  
> specified with ORDER BY. I am not sure whether this is an issue with  
> SQLite or with cross-compiler for PocketBook, but I would greatly  
> appreciate any suggestions on how to treat this problem.

I'm guessing that your hardward does not implement IEEE 754 floating  
point correctly.  We've seen that sort of thing before, especially  
with GCC.  There are some options to GCC (which escape my memory right  
now) that can force it to use strict IEEE 754 floating point rather  
than its preferred, speedier but non-standard alternative.


D. Richard Hipp
drh@...



_______________________________________________
sqlite-users mailing list
sqlite-users@...
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

[O'Neill, Owen]

Correct, ARM's emulate hardware floating point using software.
And it's really slow too since it forces a context switch as the
processor spots it's been given a FP operation to do and has to jump
into the fp emulation.

If you are lucky enough to have control over it & depending on the level
of accuracy you need, I'd recommend storing everything as an integer
with a separate scalar.

Owen.



-----Original Message-----
From: sqlite-users-bounces@...
[mailto:sqlite-users-bounces@...] On Behalf Of D. Richard Hipp
Sent: Wednesday, November 04, 2009 3:26 PM
To: General Discussion of SQLite Database
Cc: Alexander Drozd
Subject: Re: [sqlite] SQLite on PocketBook


On Nov 4, 2009, at 4:53 AM, Alexander Drozd wrote:

>
>  My name is Alexander. I am working on an open-source  spaced-
> repetition software project  (http://code.google.com/p/pbanki/). My  
> software relies on SQLite library. I came across some bug-like  
> problems with running SQLite on a low-memory e-ink reader device. I  
> am very  sorry to bother you, but I tried to submit my  problem to  
> the bugtracker at the SQLite site, and for some reason anonymous  
> login failed.
>
>  The problem appears at the point of reading real values from an  
> SQLite database. I created a simple database
>
> CREATE TABLE cards (
>    text TEXT NOT NULL,
>    value REAL NOT NULL
> );
>
> I also tried to use NUMERIC and FLOAT instead of REAL. Then I  
> inserted a few values:
>
> INSERT INTO cards VALUES('second',100.1);
> INSERT INTO cards VALUES('first', 100.0);
>
> Then I execute "select * from cards order by due" query with sample  
> code from http://www.sqlite.org/quickstart.html It works perfectly  
> when compiled on desktop computer, but fails on target device. The  
> device is PocketBook301+ (http://pocketbook.com.ua/). Unfortunately  
> their site does not have an English version. This device is based on  
> Samsung S3C2440 AL-40 CPU. It runs under open-source firmware called  
> pocketbookfree, that is based on Linux 2.6.18.2 armv4tl.
>
> The above query run on pocketbook returns corrupted values for  
> floats if they have a non-zero fractional part:
>
> text = first
> val = 100.0
>
> text = second
> val = 1.90359837350824e+185
>
> Sorting by columns containing float numbers also fails when  
> specified with ORDER BY. I am not sure whether this is an issue with  
> SQLite or with cross-compiler for PocketBook, but I would greatly  
> appreciate any suggestions on how to treat this problem.

I'm guessing that your hardward does not implement IEEE 754 floating  
point correctly.  We've seen that sort of thing before, especially  
with GCC.  There are some options to GCC (which escape my memory right  
now) that can force it to use strict IEEE 754 floating point rather  
than its preferred, speedier but non-standard alternative.


D. Richard Hipp
drh@...



_______________________________________________
sqlite-users mailing list
sqlite-users@...
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
_______________________________________________
sqlite-users mailing list
sqlite-users@...
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

[Beau Wilkinson]

>I'm guessing that your hardward does not implement IEEE 754 floating
>point correctly.  We've seen that sort of thing before, especially
>with GCC.  There are some options to GCC (which escape my memory right
>now) that can force it to use strict IEEE 754 floating point rather
>than its preferred, speedier but non-standard alternative.

What he's getting back is so far from correct, though, that I would tend to blame a run-of-the-mill bug rather than some point-of-detail. Non-IEEE floating point often sacrifices things like the distinctions between "Not-a-Number" and "Infinity," or the difference between positive and negative zero, and so on. Perhaps in some cases the rounding of the last bit is wrong. But no FP system should give results that are flat out wrong, especially when doing arithmetic. (I can see where series of operations involving exponents &c. might get way out-of-line but I don't think Sqlite is doing any of these things.)

What he is getting back looks like Double.MaxVal... is there a divide-by-zero somewhere? That is the sort of thing that different FP specs will legimately handle differently.
________________________________________
From: sqlite-users-bounces@... [sqlite-users-bounces@...] On Behalf Of D. Richard Hipp [drh@...]
Sent: Wednesday, November 04, 2009 9:26 AM
To: General Discussion of SQLite Database
Cc: Alexander Drozd
Subject: Re: [sqlite] SQLite on PocketBook

On Nov 4, 2009, at 4:53 AM, Alexander Drozd wrote:

>
>  My name is Alexander. I am working on an open-source  spaced-
> repetition software project  (http://code.google.com/p/pbanki/). My
> software relies on SQLite library. I came across some bug-like
> problems with running SQLite on a low-memory e-ink reader device. I
> am very  sorry to bother you, but I tried to submit my  problem to
> the bugtracker at the SQLite site, and for some reason anonymous
> login failed.
>
>  The problem appears at the point of reading real values from an
> SQLite database. I created a simple database
>
> CREATE TABLE cards (
>    text TEXT NOT NULL,
>    value REAL NOT NULL
> );
>
> I also tried to use NUMERIC and FLOAT instead of REAL. Then I
> inserted a few values:
>
> INSERT INTO cards VALUES('second',100.1);
> INSERT INTO cards VALUES('first', 100.0);
>
> Then I execute "select * from cards order by due" query with sample
> code from http://www.sqlite.org/quickstart.html It works perfectly
> when compiled on desktop computer, but fails on target device. The
> device is PocketBook301+ (http://pocketbook.com.ua/). Unfortunately
> their site does not have an English version. This device is based on
> Samsung S3C2440 AL-40 CPU. It runs under open-source firmware called
> pocketbookfree, that is based on Linux 2.6.18.2 armv4tl.
>
> The above query run on pocketbook returns corrupted values for
> floats if they have a non-zero fractional part:
>
> text = first
> val = 100.0
>
> text = second
> val = 1.90359837350824e+185
>
> Sorting by columns containing float numbers also fails when
> specified with ORDER BY. I am not sure whether this is an issue with
> SQLite or with cross-compiler for PocketBook, but I would greatly
> appreciate any suggestions on how to treat this problem.

I'm guessing that your hardward does not implement IEEE 754 floating
point correctly.  We've seen that sort of thing before, especially
with GCC.  There are some options to GCC (which escape my memory right
now) that can force it to use strict IEEE 754 floating point rather
than its preferred, speedier but non-standard alternative.


D. Richard Hipp
drh@...



_______________________________________________
sqlite-users mailing list
sqlite-users@...
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

The information contained in this e-mail is privileged and confidential information intended only for the use of the individual or entity named.  If you are not the intended recipient, or the employee or agent responsible for delivering this message to the intended recipient, you are hereby notified that any disclosure, dissemination, distribution, or copying of this communication is strictly prohibited.  If you have received this e-mail in error, please immediately notify the sender and delete any copies from your system.
_______________________________________________
sqlite-users mailing list
sqlite-users@...
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

[Beau Wilkinson]

> Correct, ARM's emulate hardware floating point using software.

That doesn't really say anything about compliance with standards/

________________________________________
From: sqlite-users-bounces@... [sqlite-users-bounces@...] On Behalf Of O'Neill, Owen [OONeill@...]
Sent: Wednesday, November 04, 2009 9:36 AM
To: General Discussion of SQLite Database
Cc: Alexander Drozd
Subject: Re: [sqlite] SQLite on PocketBook

Correct, ARM's emulate hardware floating point using software.
And it's really slow too since it forces a context switch as the
processor spots it's been given a FP operation to do and has to jump
into the fp emulation.

If you are lucky enough to have control over it & depending on the level
of accuracy you need, I'd recommend storing everything as an integer
with a separate scalar.

Owen.



-----Original Message-----
From: sqlite-users-bounces@...
[mailto:sqlite-users-bounces@...] On Behalf Of D. Richard Hipp
Sent: Wednesday, November 04, 2009 3:26 PM
To: General Discussion of SQLite Database
Cc: Alexander Drozd
Subject: Re: [sqlite] SQLite on PocketBook


On Nov 4, 2009, at 4:53 AM, Alexander Drozd wrote:

>
>  My name is Alexander. I am working on an open-source  spaced-
> repetition software project  (http://code.google.com/p/pbanki/). My
> software relies on SQLite library. I came across some bug-like
> problems with running SQLite on a low-memory e-ink reader device. I
> am very  sorry to bother you, but I tried to submit my  problem to
> the bugtracker at the SQLite site, and for some reason anonymous
> login failed.
>
>  The problem appears at the point of reading real values from an
> SQLite database. I created a simple database
>
> CREATE TABLE cards (
>    text TEXT NOT NULL,
>    value REAL NOT NULL
> );
>
> I also tried to use NUMERIC and FLOAT instead of REAL. Then I
> inserted a few values:
>
> INSERT INTO cards VALUES('second',100.1);
> INSERT INTO cards VALUES('first', 100.0);
>
> Then I execute "select * from cards order by due" query with sample
> code from http://www.sqlite.org/quickstart.html It works perfectly
> when compiled on desktop computer, but fails on target device. The
> device is PocketBook301+ (http://pocketbook.com.ua/). Unfortunately
> their site does not have an English version. This device is based on
> Samsung S3C2440 AL-40 CPU. It runs under open-source firmware called
> pocketbookfree, that is based on Linux 2.6.18.2 armv4tl.
>
> The above query run on pocketbook returns corrupted values for
> floats if they have a non-zero fractional part:
>
> text = first
> val = 100.0
>
> text = second
> val = 1.90359837350824e+185
>
> Sorting by columns containing float numbers also fails when
> specified with ORDER BY. I am not sure whether this is an issue with
> SQLite or with cross-compiler for PocketBook, but I would greatly
> appreciate any suggestions on how to treat this problem.

I'm guessing that your hardward does not implement IEEE 754 floating
point correctly.  We've seen that sort of thing before, especially
with GCC.  There are some options to GCC (which escape my memory right
now) that can force it to use strict IEEE 754 floating point rather
than its preferred, speedier but non-standard alternative.


D. Richard Hipp
drh@...



_______________________________________________
sqlite-users mailing list
sqlite-users@...
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
_______________________________________________
sqlite-users mailing list
sqlite-users@...
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

The information contained in this e-mail is privileged and confidential information intended only for the use of the individual or entity named.  If you are not the intended recipient, or the employee or agent responsible for delivering this message to the intended recipient, you are hereby notified that any disclosure, dissemination, distribution, or copying of this communication is strictly prohibited.  If you have received this e-mail in error, please immediately notify the sender and delete any copies from your system.
_______________________________________________
sqlite-users mailing list
sqlite-users@...
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

[Beau Wilkinson]

I really think this warrants further discussion. Perhaps the correct answer (that ARMs implement a non-standard FP type which is incompatible with Sqlite) is already out there, but I think the issues I raised with that answer should at least be addressed.

Assuming (and perhaps this is the rub...) that Sqlite is built around C++ "float" and "double,"  then I fail to see how any FP system that is even plausibly useful could give the results cited by Mr Drozd. If I put (for example) the value 100.0 into a "double," and then transport or store/retrieve the binary representation somehow, and then take those bits and once more treat them as a "double," then I ought to get 100 (or at least something very, very close). These are the sorts of things that Sqlite should, to my mind at least, be doing with real number data, and it ought not to matter what the underlying representation is.

And yet it has been put forth in this forum that such is not the case. Rather, the underlying representation must comply with the IEEE FP standard, or even basic operations will not work. And this is so certain, well-known, and reasonable that discussion amongst the plebians is not warranted.

How is this possible architecturally? The only explanation I can fathom is that Sqlite depends on the underlying representation following the IEEE standard at the bit level. For example, when doing sorts, maybe Sqlite is assuming the mantissae and exponents are in the bit ranges specified by IEEE, and that they are represented in the specified format (e.g. excess vs. complement notation) as well.

If this is indeed the case, I think this is a very misguided architecture. Depending on the bit-level representation is bad. It's a brittle design. Of course, it's easy for you all to intimidate anyone who has a problem with this architecture... the complainer is "not in compliance with the IEEE standard" and is thus clearly worthy of your speedy dismissal. Bah.

Ultimately, I think this is an easy excuse for a bad design. Types like "float" and "double" are intended by their designers to abstract over many FP implementations. They are not just convenient macros from IEEE FP, nor should they be.

I could go on to take issue with the IEEE standard itself. The allocation of bits to exponent-versus-mantissa is rigid, for example. IEEE makes no allowance (that I know of) for allowing a tradeoff between precision and dynamic range, which is a major oversight for such a widely-used standard. Until very recently IEEE FP included no support for 16-bit (half precision) data. IEEE was also designed by committee so it includes all sorts of nice-to-have pet features (two zeros, distinct error and condition codes, etc.) which may or may not be worthwhile on any given real-world system. (I tend to lean toward the "may not" direction).

But whether IEEE is bad or good or indifferent makes no difference- the standard should not, in my opinion, be built into Sqlite. Basic software engineering sense must still trump even the best standard.

Forgive me if I have missed something here, but this seems like what I would call "Standardizationism" run amok.

________________________________________
From: Beau Wilkinson
Sent: Wednesday, November 04, 2009 9:39 AM
To: General Discussion of SQLite Database
Cc: Alexander Drozd
Subject: RE: [sqlite] SQLite on PocketBook

>I'm guessing that your hardward does not implement IEEE 754 floating
>point correctly.  We've seen that sort of thing before, especially
>with GCC.  There are some options to GCC (which escape my memory right
>now) that can force it to use strict IEEE 754 floating point rather
>than its preferred, speedier but non-standard alternative.

What he's getting back is so far from correct, though, that I would tend to blame a run-of-the-mill bug rather than some point-of-detail. Non-IEEE floating point often sacrifices things like the distinctions between "Not-a-Number" and "Infinity," or the difference between positive and negative zero, and so on. Perhaps in some cases the rounding of the last bit is wrong. But no FP system should give results that are flat out wrong, especially when doing arithmetic. (I can see where series of operations involving exponents &c. might get way out-of-line but I don't think Sqlite is doing any of these things.)

What he is getting back looks like Double.MaxVal... is there a divide-by-zero somewhere? That is the sort of thing that different FP specs will legimately handle differently.
________________________________________
From: sqlite-users-bounces@... [sqlite-users-bounces@...] On Behalf Of D. Richard Hipp [drh@...]
Sent: Wednesday, November 04, 2009 9:26 AM
To: General Discussion of SQLite Database
Cc: Alexander Drozd
Subject: Re: [sqlite] SQLite on PocketBook

On Nov 4, 2009, at 4:53 AM, Alexander Drozd wrote:

>
>  My name is Alexander. I am working on an open-source  spaced-
> repetition software project  (http://code.google.com/p/pbanki/). My
> software relies on SQLite library. I came across some bug-like
> problems with running SQLite on a low-memory e-ink reader device. I
> am very  sorry to bother you, but I tried to submit my  problem to
> the bugtracker at the SQLite site, and for some reason anonymous
> login failed.
>
>  The problem appears at the point of reading real values from an
> SQLite database. I created a simple database
>
> CREATE TABLE cards (
>    text TEXT NOT NULL,
>    value REAL NOT NULL
> );
>
> I also tried to use NUMERIC and FLOAT instead of REAL. Then I
> inserted a few values:
>
> INSERT INTO cards VALUES('second',100.1);
> INSERT INTO cards VALUES('first', 100.0);
>
> Then I execute "select * from cards order by due" query with sample
> code from http://www.sqlite.org/quickstart.html It works perfectly
> when compiled on desktop computer, but fails on target device. The
> device is PocketBook301+ (http://pocketbook.com.ua/). Unfortunately
> their site does not have an English version. This device is based on
> Samsung S3C2440 AL-40 CPU. It runs under open-source firmware called
> pocketbookfree, that is based on Linux 2.6.18.2 armv4tl.
>
> The above query run on pocketbook returns corrupted values for
> floats if they have a non-zero fractional part:
>
> text = first
> val = 100.0
>
> text = second
> val = 1.90359837350824e+185
>
> Sorting by columns containing float numbers also fails when
> specified with ORDER BY. I am not sure whether this is an issue with
> SQLite or with cross-compiler for PocketBook, but I would greatly
> appreciate any suggestions on how to treat this problem.

I'm guessing that your hardward does not implement IEEE 754 floating
point correctly.  We've seen that sort of thing before, especially
with GCC.  There are some options to GCC (which escape my memory right
now) that can force it to use strict IEEE 754 floating point rather
than its preferred, speedier but non-standard alternative.


D. Richard Hipp
drh@...



_______________________________________________
sqlite-users mailing list
sqlite-users@...
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

The information contained in this e-mail is privileged and confidential information intended only for the use of the individual or entity named.  If you are not the intended recipient, or the employee or agent responsible for delivering this message to the intended recipient, you are hereby notified that any disclosure, dissemination, distribution, or copying of this communication is strictly prohibited.  If you have received this e-mail in error, please immediately notify the sender and delete any copies from your system.
_______________________________________________
sqlite-users mailing list
sqlite-users@...
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

[Kavita Raghunathan]

I saw a presentation on sqlite by Dr Hipp that mentioned that anytime I'm
storing data in structures or tables, I should be thinking about using
sqlite instead.

Would it be more efficient to use the sqlite database to store a table that
Looks like this: where lets say I'm looking for the word "auto-align". Would
the query be quicker than searching through this table in a "for" or while
loop? Assume the table has about 200 entries. I want to know if the
performance will be better and if I should consider storing these constants
in the database.

.
.
  {"giants",                e_sf_attr_pm_ethernet_giants},
  {"last_time_cleared",     e_sf_attr_pm_ethernet_last_time_cleared},
  {"port_counters_start",   e_sf_attr_pm_ethernet_port_counters_start},
  {"port_counters_end",     e_sf_attr_pm_ethernet_port_counters_end},
  {"mac_rcv_unicast",       e_sf_attr_pm_ethernet_mac_rcv_unicast},
  {"mac_rcv_multicast",     e_sf_attr_pm_ethernet_mac_rcv_multicast},
  {"mac_rcv_broadcase",     e_sf_attr_pm_ethernet_mac_rcv_broadcast},
  {"mac_xmit_unicast",      e_sf_attr_pm_ethernet_mac_xmit_unicast},
  {"mac_xmit_multicast",    e_sf_attr_pm_ethernet_mac_xmit_multicast},
  {"mac_xmit_broadcast",    e_sf_attr_pm_ethernet_mac_xmit_broadcast},
  {"mac_rcv_octet",         e_sf_attr_pm_ethernet_mac_rcv_octet},
  {"mac_xmit_octet",        e_sf_attr_pm_ethernet_mac_xmit_octet},
  {"mac_delay_exceed",      e_sf_attr_pm_ethernet_mac_delay_exceed},
  {"mac_mtu_exceed",        e_sf_attr_pm_ethernet_mac_mtu_exceed},
  {"mac_in_discard",        e_sf_attr_pm_ethernet_mac_in_discard},
  {"mac_out_discard",       e_sf_attr_pm_ethernet_mac_out_discard},
  {"mac_last_time_cleared", e_sf_attr_pm_ethernet_mac_last_time_cleared},
  {"manual_align",           e_sf_attr_pm_manual_alig},
  {"auto_align",             e_sf_attr_pm_auto_align},
  {"initial_align",          e_sf_attr_pm_initial_align},
  {"seconds_on_align",       e_sf_attr_pm_seconds_on_align},
  {"align_start_time",       e_sf_attr_pm_last_align_start_time},
  {"align_start_trigger",    e_sf_attr_pm_last_align_start_trigger},
  {"align_start_azimuth",    e_sf_attr_pm_last_align_start_azimuth},
  {"align_start_elevation",  e_sf_attr_pm_last_align_start_elevation},
  {"align_start_rssi",       e_sf_attr_pm_last_align_start_rssi},
  {"align_start_ber",        e_sf_attr_pm_last_align_start_ber},
  {"align_end_time",         e_sf_attr_pm_last_align_end_time},
.
.


On 11/5/09 4:15 PM, "Beau Wilkinson" <beau@...> wrote:

> I really think this warrants further discussion. Perhaps the correct answer
> (that ARMs implement a non-standard FP type which is incompatible with Sqlite)
> is already out there, but I think the issues I raised with that answer should
> at least be addressed.
>
> Assuming (and perhaps this is the rub...) that Sqlite is built around C++
> "float" and "double,"  then I fail to see how any FP system that is even
> plausibly useful could give the results cited by Mr Drozd. If I put (for
> example) the value 100.0 into a "double," and then transport or store/retrieve
> the binary representation somehow, and then take those bits and once more
> treat them as a "double," then I ought to get 100 (or at least something very,
> very close). These are the sorts of things that Sqlite should, to my mind at
> least, be doing with real number data, and it ought not to matter what the
> underlying representation is.
>
> And yet it has been put forth in this forum that such is not the case. Rather,
> the underlying representation must comply with the IEEE FP standard, or even
> basic operations will not work. And this is so certain, well-known, and
> reasonable that discussion amongst the plebians is not warranted.
>
> How is this possible architecturally? The only explanation I can fathom is
> that Sqlite depends on the underlying representation following the IEEE
> standard at the bit level. For example, when doing sorts, maybe Sqlite is
> assuming the mantissae and exponents are in the bit ranges specified by IEEE,
> and that they are represented in the specified format (e.g. excess vs.
> complement notation) as well.
>
> If this is indeed the case, I think this is a very misguided architecture.
> Depending on the bit-level representation is bad. It's a brittle design. Of
> course, it's easy for you all to intimidate anyone who has a problem with this
> architecture... the complainer is "not in compliance with the IEEE standard"
> and is thus clearly worthy of your speedy dismissal. Bah.
>
> Ultimately, I think this is an easy excuse for a bad design. Types like
> "float" and "double" are intended by their designers to abstract over many FP
> implementations. They are not just convenient macros from IEEE FP, nor should
> they be.
>
> I could go on to take issue with the IEEE standard itself. The allocation of
> bits to exponent-versus-mantissa is rigid, for example. IEEE makes no
> allowance (that I know of) for allowing a tradeoff between precision and
> dynamic range, which is a major oversight for such a widely-used standard.
> Until very recently IEEE FP included no support for 16-bit (half precision)
> data. IEEE was also designed by committee so it includes all sorts of
> nice-to-have pet features (two zeros, distinct error and condition codes,
> etc.) which may or may not be worthwhile on any given real-world system. (I
> tend to lean toward the "may not" direction).
>
> But whether IEEE is bad or good or indifferent makes no difference- the
> standard should not, in my opinion, be built into Sqlite. Basic software
> engineering sense must still trump even the best standard.
>
> Forgive me if I have missed something here, but this seems like what I would
> call "Standardizationism" run amok.
>
> ________________________________________
> From: Beau Wilkinson
> Sent: Wednesday, November 04, 2009 9:39 AM
> To: General Discussion of SQLite Database
> Cc: Alexander Drozd
> Subject: RE: [sqlite] SQLite on PocketBook
>
>> I'm guessing that your hardward does not implement IEEE 754 floating
>> point correctly.  We've seen that sort of thing before, especially
>> with GCC.  There are some options to GCC (which escape my memory right
>> now) that can force it to use strict IEEE 754 floating point rather
>> than its preferred, speedier but non-standard alternative.
>
> What he's getting back is so far from correct, though, that I would tend to
> blame a run-of-the-mill bug rather than some point-of-detail. Non-IEEE
> floating point often sacrifices things like the distinctions between
> "Not-a-Number" and "Infinity," or the difference between positive and negative
> zero, and so on. Perhaps in some cases the rounding of the last bit is wrong.
> But no FP system should give results that are flat out wrong, especially when
> doing arithmetic. (I can see where series of operations involving exponents
> &c. might get way out-of-line but I don't think Sqlite is doing any of these
> things.)
>
> What he is getting back looks like Double.MaxVal... is there a divide-by-zero
> somewhere? That is the sort of thing that different FP specs will legimately
> handle differently.
> ________________________________________
> From: sqlite-users-bounces@... [sqlite-users-bounces@...] On
> Behalf Of D. Richard Hipp [drh@...]
> Sent: Wednesday, November 04, 2009 9:26 AM
> To: General Discussion of SQLite Database
> Cc: Alexander Drozd
> Subject: Re: [sqlite] SQLite on PocketBook
>
> On Nov 4, 2009, at 4:53 AM, Alexander Drozd wrote:
>>
>>  My name is Alexander. I am working on an open-source  spaced-
>> repetition software project  (http://code.google.com/p/pbanki/). My
>> software relies on SQLite library. I came across some bug-like
>> problems with running SQLite on a low-memory e-ink reader device. I
>> am very  sorry to bother you, but I tried to submit my  problem to
>> the bugtracker at the SQLite site, and for some reason anonymous
>> login failed.
>>
>>  The problem appears at the point of reading real values from an
>> SQLite database. I created a simple database
>>
>> CREATE TABLE cards (
>>    text TEXT NOT NULL,
>>    value REAL NOT NULL
>> );
>>
>> I also tried to use NUMERIC and FLOAT instead of REAL. Then I
>> inserted a few values:
>>
>> INSERT INTO cards VALUES('second',100.1);
>> INSERT INTO cards VALUES('first', 100.0);
>>
>> Then I execute "select * from cards order by due" query with sample
>> code from http://www.sqlite.org/quickstart.html It works perfectly
>> when compiled on desktop computer, but fails on target device. The
>> device is PocketBook301+ (http://pocketbook.com.ua/). Unfortunately
>> their site does not have an English version. This device is based on
>> Samsung S3C2440 AL-40 CPU. It runs under open-source firmware called
>> pocketbookfree, that is based on Linux 2.6.18.2 armv4tl.
>>
>> The above query run on pocketbook returns corrupted values for
>> floats if they have a non-zero fractional part:
>>
>> text = first
>> val = 100.0
>>
>> text = second
>> val = 1.90359837350824e+185
>>
>> Sorting by columns containing float numbers also fails when
>> specified with ORDER BY. I am not sure whether this is an issue with
>> SQLite or with cross-compiler for PocketBook, but I would greatly
>> appreciate any suggestions on how to treat this problem.
>
>
> I'm guessing that your hardward does not implement IEEE 754 floating
> point correctly.  We've seen that sort of thing before, especially
> with GCC.  There are some options to GCC (which escape my memory right
> now) that can force it to use strict IEEE 754 floating point rather
> than its preferred, speedier but non-standard alternative.
>
>
> D. Richard Hipp
> drh@...
>
>
>
> _______________________________________________
> sqlite-users mailing list
> sqlite-users@...
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>
> The information contained in this e-mail is privileged and confidential
> information intended only for the use of the individual or entity named.  If
> you are not the intended recipient, or the employee or agent responsible for
> delivering this message to the intended recipient, you are hereby notified
> that any disclosure, dissemination, distribution, or copying of this
> communication is strictly prohibited.  If you have received this e-mail in
> error, please immediately notify the sender and delete any copies from your
> system.
> _______________________________________________
> sqlite-users mailing list
> sqlite-users@...
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

_______________________________________________
sqlite-users mailing list
sqlite-users@...
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

[D. Richard Hipp]

On Nov 5, 2009, at 5:15 PM, Beau Wilkinson wrote:

>
> Assuming (and perhaps this is the rub...) that Sqlite is built  
> around C++ "float" and "double,"  then I fail to see how any FP  
> system that is even plausibly useful could give the results cited by  
> Mr Drozd. If I put (for example) the value 100.0 into a "double,"  
> and then transport or store/retrieve the binary representation  
> somehow, and then take those bits and once more treat them as a  
> "double," then I ought to get 100 (or at least something very, very  
> close). These are the sorts of things that Sqlite should, to my mind  
> at least, be doing with real number data, and it ought not to matter  
> what the underlying representation is.
>

This is exactly what SQLite does.  It uses the C (not C++) double type  
for floating point operations.  It assumes that double is represented  
by 64 contiguous bits in a format that can be understood by different  
architectures and it blindly copies those bits to disk.  Later, it  
reads the same bits back from disk and copies them into another double.

If you write a double into an SQLite database on a machine that has  
IEEE FP, then transfer that database to another machine with a  
different floating point format, then read the double out of the  
database, the bits won't be right and you'll get a weird answer.  This  
comes up a lot on ARM because of the use of mixed-endian  
representation - neither big-endian or little-endian. I thought we had  
worked around that problem, but perhaps you have found a way to defeat  
our work-around.

We've also seen problems with GCC's floating point optimizer going  
overboard and simply generating bad code when the --fast-math opton is  
specified.  We recommend that you not use --fast-math.

I'm sorry that you are having difficulty and that SQLite is not  
working to your satisfaction.  But the fact remains that it works  
great on every platform that I have access to.  If you are able to  
find a bug in SQLite we will be happy to look into it for you.  But we  
have no capability of helping you with your ARM problems.

D. Richard Hipp
drh@...



_______________________________________________
sqlite-users mailing list
sqlite-users@...
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

[P Kishor-3]

First, you shouldn't hijack a thread. If you have a new subject, start
a new email thread instead of using an existing one.


On Thu, Nov 5, 2009 at 4:43 PM, Kavita Raghunathan
<kavita.raghunathan@...> wrote:

>
> I saw a presentation on sqlite by Dr Hipp that mentioned that anytime I'm
> storing data in structures or tables, I should be thinking about using
> sqlite instead.
>
> Would it be more efficient to use the sqlite database to store a table that
> Looks like this: where lets say I'm looking for the word "auto-align". Would
> the query be quicker than searching through this table in a "for" or while
> loop? Assume the table has about 200 entries. I want to know if the
> performance will be better and if I should consider storing these constants
> in the database.
>
> .

Seriously, with 200 entries, you couldn't even begin to tell the
difference in speed. You could store 200 lines in a text file, grep
through it and find your answer just as quickly as you would any other
way.

On the other hand, if you had 200,000 or 2,000,000 entries, then
storing the data in a table, indexing it, and searching it would
probably be more useful in terms of speed.

A long time ago I remember meeting with someone who wanted to know if
Oracle would be suitable for storing their departmental address book.
Apparently the address book was rather large, with several hundred
employees. Go figure. I gently told him that yes, Oracle would be ok,
but testing would be required to back the decision.

> .
>  {"giants",                e_sf_attr_pm_ethernet_giants},
>  {"last_time_cleared",     e_sf_attr_pm_ethernet_last_time_cleared},
>  {"port_counters_start",   e_sf_attr_pm_ethernet_port_counters_start},
>  {"port_counters_end",     e_sf_attr_pm_ethernet_port_counters_end},
>  {"mac_rcv_unicast",       e_sf_attr_pm_ethernet_mac_rcv_unicast},
>  {"mac_rcv_multicast",     e_sf_attr_pm_ethernet_mac_rcv_multicast},
>  {"mac_rcv_broadcase",     e_sf_attr_pm_ethernet_mac_rcv_broadcast},
>  {"mac_xmit_unicast",      e_sf_attr_pm_ethernet_mac_xmit_unicast},
>  {"mac_xmit_multicast",    e_sf_attr_pm_ethernet_mac_xmit_multicast},
>  {"mac_xmit_broadcast",    e_sf_attr_pm_ethernet_mac_xmit_broadcast},
>  {"mac_rcv_octet",         e_sf_attr_pm_ethernet_mac_rcv_octet},
>  {"mac_xmit_octet",        e_sf_attr_pm_ethernet_mac_xmit_octet},
>  {"mac_delay_exceed",      e_sf_attr_pm_ethernet_mac_delay_exceed},
>  {"mac_mtu_exceed",        e_sf_attr_pm_ethernet_mac_mtu_exceed},
>  {"mac_in_discard",        e_sf_attr_pm_ethernet_mac_in_discard},
>  {"mac_out_discard",       e_sf_attr_pm_ethernet_mac_out_discard},
>  {"mac_last_time_cleared", e_sf_attr_pm_ethernet_mac_last_time_cleared},
>  {"manual_align",           e_sf_attr_pm_manual_alig},
>  {"auto_align",             e_sf_attr_pm_auto_align},
>  {"initial_align",          e_sf_attr_pm_initial_align},
>  {"seconds_on_align",       e_sf_attr_pm_seconds_on_align},
>  {"align_start_time",       e_sf_attr_pm_last_align_start_time},
>  {"align_start_trigger",    e_sf_attr_pm_last_align_start_trigger},
>  {"align_start_azimuth",    e_sf_attr_pm_last_align_start_azimuth},
>  {"align_start_elevation",  e_sf_attr_pm_last_align_start_elevation},
>  {"align_start_rssi",       e_sf_attr_pm_last_align_start_rssi},
>  {"align_start_ber",        e_sf_attr_pm_last_align_start_ber},
>  {"align_end_time",         e_sf_attr_pm_last_align_end_time},
> .
> .
>
>
> On 11/5/09 4:15 PM, "Beau Wilkinson" <beau@...> wrote:
>
>> I really think this warrants further discussion. Perhaps the correct answer
>> (that ARMs implement a non-standard FP type which is incompatible with Sqlite)
>> is already out there, but I think the issues I raised with that answer should
>> at least be addressed.
>>
>> Assuming (and perhaps this is the rub...) that Sqlite is built around C++
>> "float" and "double,"  then I fail to see how any FP system that is even
>> plausibly useful could give the results cited by Mr Drozd. If I put (for
>> example) the value 100.0 into a "double," and then transport or store/retrieve
>> the binary representation somehow, and then take those bits and once more
>> treat them as a "double," then I ought to get 100 (or at least something very,
>> very close). These are the sorts of things that Sqlite should, to my mind at
>> least, be doing with real number data, and it ought not to matter what the
>> underlying representation is.
>>
>> And yet it has been put forth in this forum that such is not the case. Rather,
>> the underlying representation must comply with the IEEE FP standard, or even
>> basic operations will not work. And this is so certain, well-known, and
>> reasonable that discussion amongst the plebians is not warranted.
>>
>> How is this possible architecturally? The only explanation I can fathom is
>> that Sqlite depends on the underlying representation following the IEEE
>> standard at the bit level. For example, when doing sorts, maybe Sqlite is
>> assuming the mantissae and exponents are in the bit ranges specified by IEEE,
>> and that they are represented in the specified format (e.g. excess vs.
>> complement notation) as well.
>>
>> If this is indeed the case, I think this is a very misguided architecture.
>> Depending on the bit-level representation is bad. It's a brittle design. Of
>> course, it's easy for you all to intimidate anyone who has a problem with this
>> architecture... the complainer is "not in compliance with the IEEE standard"
>> and is thus clearly worthy of your speedy dismissal. Bah.
>>
>> Ultimately, I think this is an easy excuse for a bad design. Types like
>> "float" and "double" are intended by their designers to abstract over many FP
>> implementations. They are not just convenient macros from IEEE FP, nor should
>> they be.
>>
>> I could go on to take issue with the IEEE standard itself. The allocation of
>> bits to exponent-versus-mantissa is rigid, for example. IEEE makes no
>> allowance (that I know of) for allowing a tradeoff between precision and
>> dynamic range, which is a major oversight for such a widely-used standard.
>> Until very recently IEEE FP included no support for 16-bit (half precision)
>> data. IEEE was also designed by committee so it includes all sorts of
>> nice-to-have pet features (two zeros, distinct error and condition codes,
>> etc.) which may or may not be worthwhile on any given real-world system. (I
>> tend to lean toward the "may not" direction).
>>
>> But whether IEEE is bad or good or indifferent makes no difference- the
>> standard should not, in my opinion, be built into Sqlite. Basic software
>> engineering sense must still trump even the best standard.
>>
>> Forgive me if I have missed something here, but this seems like what I would
>> call "Standardizationism" run amok.
>>
>> ________________________________________
>> From: Beau Wilkinson
>> Sent: Wednesday, November 04, 2009 9:39 AM
>> To: General Discussion of SQLite Database
>> Cc: Alexander Drozd
>> Subject: RE: [sqlite] SQLite on PocketBook
>>
>>> I'm guessing that your hardward does not implement IEEE 754 floating
>>> point correctly.  We've seen that sort of thing before, especially
>>> with GCC.  There are some options to GCC (which escape my memory right
>>> now) that can force it to use strict IEEE 754 floating point rather
>>> than its preferred, speedier but non-standard alternative.
>>
>> What he's getting back is so far from correct, though, that I would tend to
>> blame a run-of-the-mill bug rather than some point-of-detail. Non-IEEE
>> floating point often sacrifices things like the distinctions between
>> "Not-a-Number" and "Infinity," or the difference between positive and negative
>> zero, and so on. Perhaps in some cases the rounding of the last bit is wrong.
>> But no FP system should give results that are flat out wrong, especially when
>> doing arithmetic. (I can see where series of operations involving exponents
>> &c. might get way out-of-line but I don't think Sqlite is doing any of these
>> things.)
>>
>> What he is getting back looks like Double.MaxVal... is there a divide-by-zero
>> somewhere? That is the sort of thing that different FP specs will legimately
>> handle differently.
>> ________________________________________
>> From: sqlite-users-bounces@... [sqlite-users-bounces@...] On
>> Behalf Of D. Richard Hipp [drh@...]
>> Sent: Wednesday, November 04, 2009 9:26 AM
>> To: General Discussion of SQLite Database
>> Cc: Alexander Drozd
>> Subject: Re: [sqlite] SQLite on PocketBook
>>
>> On Nov 4, 2009, at 4:53 AM, Alexander Drozd wrote:
>>>
>>>  My name is Alexander. I am working on an open-source  spaced-
>>> repetition software project  (http://code.google.com/p/pbanki/). My
>>> software relies on SQLite library. I came across some bug-like
>>> problems with running SQLite on a low-memory e-ink reader device. I
>>> am very  sorry to bother you, but I tried to submit my  problem to
>>> the bugtracker at the SQLite site, and for some reason anonymous
>>> login failed.
>>>
>>>  The problem appears at the point of reading real values from an
>>> SQLite database. I created a simple database
>>>
>>> CREATE TABLE cards (
>>>    text TEXT NOT NULL,
>>>    value REAL NOT NULL
>>> );
>>>
>>> I also tried to use NUMERIC and FLOAT instead of REAL. Then I
>>> inserted a few values:
>>>
>>> INSERT INTO cards VALUES('second',100.1);
>>> INSERT INTO cards VALUES('first', 100.0);
>>>
>>> Then I execute "select * from cards order by due" query with sample
>>> code from http://www.sqlite.org/quickstart.html It works perfectly
>>> when compiled on desktop computer, but fails on target device. The
>>> device is PocketBook301+ (http://pocketbook.com.ua/). Unfortunately
>>> their site does not have an English version. This device is based on
>>> Samsung S3C2440 AL-40 CPU. It runs under open-source firmware called
>>> pocketbookfree, that is based on Linux 2.6.18.2 armv4tl.
>>>
>>> The above query run on pocketbook returns corrupted values for
>>> floats if they have a non-zero fractional part:
>>>
>>> text = first
>>> val = 100.0
>>>
>>> text = second
>>> val = 1.90359837350824e+185
>>>
>>> Sorting by columns containing float numbers also fails when
>>> specified with ORDER BY. I am not sure whether this is an issue with
>>> SQLite or with cross-compiler for PocketBook, but I would greatly
>>> appreciate any suggestions on how to treat this problem.
>>
>>
>> I'm guessing that your hardward does not implement IEEE 754 floating
>> point correctly.  We've seen that sort of thing before, especially
>> with GCC.  There are some options to GCC (which escape my memory right
>> now) that can force it to use strict IEEE 754 floating point rather
>> than its preferred, speedier but non-standard alternative.
>>
>>
>> D. Richard Hipp
>> drh@...
>>
>>
>>
>> _______________________________________________
>> sqlite-users mailing list
>> sqlite-users@...
>> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>>
>> The information contained in this e-mail is privileged and confidential
>> information intended only for the use of the individual or entity named.  If
>> you are not the intended recipient, or the employee or agent responsible for
>> delivering this message to the intended recipient, you are hereby notified
>> that any disclosure, dissemination, distribution, or copying of this
>> communication is strictly prohibited.  If you have received this e-mail in
>> error, please immediately notify the sender and delete any copies from your
>> system.
>> _______________________________________________
>> sqlite-users mailing list
>> sqlite-users@...
>> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>
> _______________________________________________
> sqlite-users mailing list
> sqlite-users@...
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>

--
Puneet Kishor http://www.punkish.org
Carbon Model http://carbonmodel.org
Charter Member, Open Source Geospatial Foundation http://www.osgeo.org
Science Commons Fellow, http://sciencecommons.org/about/whoweare/kishor
Nelson Institute, UW-Madison http://www.nelson.wisc.edu
-----------------------------------------------------------------------
Assertions are politics; backing up assertions with evidence is science
=======================================================================
Sent from Madison, Wisconsin, United States
_______________________________________________
sqlite-users mailing list
sqlite-users@...
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

[D. Richard Hipp]

On Nov 5, 2009, at 5:43 PM, Kavita Raghunathan wrote:

>
> I saw a presentation on sqlite by Dr Hipp that mentioned that  
> anytime I'm
> storing data in structures or tables, I should be thinking about using
> sqlite instead.
>
> Would it be more efficient to use the sqlite database to store a  
> table that
> Looks like this: where lets say I'm looking for the word "auto-
> align". Would
> the query be quicker than searching through this table in a "for" or  
> while
> loop? Assume the table has about 200 entries. I want to know if the
> performance will be better and if I should consider storing these  
> constants
> in the database.

Raw C code will be faster.

But is speed really the criteria here?  What about correctness?  Or  
ease of maintenance and enhancement?  How fast does your program  
really need to go?  If it is absolutely critical that you find your  
answer in nanoseconds, then by all means use C code.  But if you are  
operating on human timescales (tens or hundreds of milliseconds) then  
SQLite will be fast enough and, depending on what you are trying to  
accomplish, might be less error prone and easier to expand and enhance  
down the road.

D. Richard Hipp
drh@...



_______________________________________________
sqlite-users mailing list
sqlite-users@...
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

[Kavita Raghunathan]

Ok, sqlite group, please accept my apologies. It was thoughtless of me.


On 11/5/09 4:54 PM, "P Kishor" <punk.kish@...> wrote:

> First, you shouldn't hijack a thread. If you have a new subject, start
> a new email thread instead of using an existing one.
>
>
> On Thu, Nov 5, 2009 at 4:43 PM, Kavita Raghunathan
> <kavita.raghunathan@...> wrote:
>>
>> I saw a presentation on sqlite by Dr Hipp that mentioned that anytime I'm
>> storing data in structures or tables, I should be thinking about using
>> sqlite instead.
>>
>> Would it be more efficient to use the sqlite database to store a table that
>> Looks like this: where lets say I'm looking for the word "auto-align". Would
>> the query be quicker than searching through this table in a "for" or while
>> loop? Assume the table has about 200 entries. I want to know if the
>> performance will be better and if I should consider storing these constants
>> in the database.
>>
>> .
>
> Seriously, with 200 entries, you couldn't even begin to tell the
> difference in speed. You could store 200 lines in a text file, grep
> through it and find your answer just as quickly as you would any other
> way.
>
> On the other hand, if you had 200,000 or 2,000,000 entries, then
> storing the data in a table, indexing it, and searching it would
> probably be more useful in terms of speed.
>
> A long time ago I remember meeting with someone who wanted to know if
> Oracle would be suitable for storing their departmental address book.
> Apparently the address book was rather large, with several hundred
> employees. Go figure. I gently told him that yes, Oracle would be ok,
> but testing would be required to back the decision.
>
>
>
>> .
>>  {"giants",                e_sf_attr_pm_ethernet_giants},
>>  {"last_time_cleared",     e_sf_attr_pm_ethernet_last_time_cleared},
>>  {"port_counters_start",   e_sf_attr_pm_ethernet_port_counters_start},
>>  {"port_counters_end",     e_sf_attr_pm_ethernet_port_counters_end},
>>  {"mac_rcv_unicast",       e_sf_attr_pm_ethernet_mac_rcv_unicast},
>>  {"mac_rcv_multicast",     e_sf_attr_pm_ethernet_mac_rcv_multicast},
>>  {"mac_rcv_broadcase",     e_sf_attr_pm_ethernet_mac_rcv_broadcast},
>>  {"mac_xmit_unicast",      e_sf_attr_pm_ethernet_mac_xmit_unicast},
>>  {"mac_xmit_multicast",    e_sf_attr_pm_ethernet_mac_xmit_multicast},
>>  {"mac_xmit_broadcast",    e_sf_attr_pm_ethernet_mac_xmit_broadcast},
>>  {"mac_rcv_octet",         e_sf_attr_pm_ethernet_mac_rcv_octet},
>>  {"mac_xmit_octet",        e_sf_attr_pm_ethernet_mac_xmit_octet},
>>  {"mac_delay_exceed",      e_sf_attr_pm_ethernet_mac_delay_exceed},
>>  {"mac_mtu_exceed",        e_sf_attr_pm_ethernet_mac_mtu_exceed},
>>  {"mac_in_discard",        e_sf_attr_pm_ethernet_mac_in_discard},
>>  {"mac_out_discard",       e_sf_attr_pm_ethernet_mac_out_discard},
>>  {"mac_last_time_cleared", e_sf_attr_pm_ethernet_mac_last_time_cleared},
>>  {"manual_align",           e_sf_attr_pm_manual_alig},
>>  {"auto_align",             e_sf_attr_pm_auto_align},
>>  {"initial_align",          e_sf_attr_pm_initial_align},
>>  {"seconds_on_align",       e_sf_attr_pm_seconds_on_align},
>>  {"align_start_time",       e_sf_attr_pm_last_align_start_time},
>>  {"align_start_trigger",    e_sf_attr_pm_last_align_start_trigger},
>>  {"align_start_azimuth",    e_sf_attr_pm_last_align_start_azimuth},
>>  {"align_start_elevation",  e_sf_attr_pm_last_align_start_elevation},
>>  {"align_start_rssi",       e_sf_attr_pm_last_align_start_rssi},
>>  {"align_start_ber",        e_sf_attr_pm_last_align_start_ber},
>>  {"align_end_time",         e_sf_attr_pm_last_align_end_time},
>> .
>> .
>>
>>
>> On 11/5/09 4:15 PM, "Beau Wilkinson" <beau@...> wrote:
>>
>>> I really think this warrants further discussion. Perhaps the correct answer
>>> (that ARMs implement a non-standard FP type which is incompatible with
>>> Sqlite)
>>> is already out there, but I think the issues I raised with that answer
>>> should
>>> at least be addressed.
>>>
>>> Assuming (and perhaps this is the rub...) that Sqlite is built around C++
>>> "float" and "double,"  then I fail to see how any FP system that is even
>>> plausibly useful could give the results cited by Mr Drozd. If I put (for
>>> example) the value 100.0 into a "double," and then transport or
>>> store/retrieve
>>> the binary representation somehow, and then take those bits and once more
>>> treat them as a "double," then I ought to get 100 (or at least something
>>> very,
>>> very close). These are the sorts of things that Sqlite should, to my mind at
>>> least, be doing with real number data, and it ought not to matter what the
>>> underlying representation is.
>>>
>>> And yet it has been put forth in this forum that such is not the case.
>>> Rather,
>>> the underlying representation must comply with the IEEE FP standard, or even
>>> basic operations will not work. And this is so certain, well-known, and
>>> reasonable that discussion amongst the plebians is not warranted.
>>>
>>> How is this possible architecturally? The only explanation I can fathom is
>>> that Sqlite depends on the underlying representation following the IEEE
>>> standard at the bit level. For example, when doing sorts, maybe Sqlite is
>>> assuming the mantissae and exponents are in the bit ranges specified by
>>> IEEE,
>>> and that they are represented in the specified format (e.g. excess vs.
>>> complement notation) as well.
>>>
>>> If this is indeed the case, I think this is a very misguided architecture.
>>> Depending on the bit-level representation is bad. It's a brittle design. Of
>>> course, it's easy for you all to intimidate anyone who has a problem with
>>> this
>>> architecture... the complainer is "not in compliance with the IEEE standard"
>>> and is thus clearly worthy of your speedy dismissal. Bah.
>>>
>>> Ultimately, I think this is an easy excuse for a bad design. Types like
>>> "float" and "double" are intended by their designers to abstract over many
>>> FP
>>> implementations. They are not just convenient macros from IEEE FP, nor
>>> should
>>> they be.
>>>
>>> I could go on to take issue with the IEEE standard itself. The allocation of
>>> bits to exponent-versus-mantissa is rigid, for example. IEEE makes no
>>> allowance (that I know of) for allowing a tradeoff between precision and
>>> dynamic range, which is a major oversight for such a widely-used standard.
>>> Until very recently IEEE FP included no support for 16-bit (half precision)
>>> data. IEEE was also designed by committee so it includes all sorts of
>>> nice-to-have pet features (two zeros, distinct error and condition codes,
>>> etc.) which may or may not be worthwhile on any given real-world system. (I
>>> tend to lean toward the "may not" direction).
>>>
>>> But whether IEEE is bad or good or indifferent makes no difference- the
>>> standard should not, in my opinion, be built into Sqlite. Basic software
>>> engineering sense must still trump even the best standard.
>>>
>>> Forgive me if I have missed something here, but this seems like what I would
>>> call "Standardizationism" run amok.
>>>
>>> ________________________________________
>>> From: Beau Wilkinson
>>> Sent: Wednesday, November 04, 2009 9:39 AM
>>> To: General Discussion of SQLite Database
>>> Cc: Alexander Drozd
>>> Subject: RE: [sqlite] SQLite on PocketBook
>>>
>>>> I'm guessing that your hardward does not implement IEEE 754 floating
>>>> point correctly.  We've seen that sort of thing before, especially
>>>> with GCC.  There are some options to GCC (which escape my memory right
>>>> now) that can force it to use strict IEEE 754 floating point rather
>>>> than its preferred, speedier but non-standard alternative.
>>>
>>> What he's getting back is so far from correct, though, that I would tend to
>>> blame a run-of-the-mill bug rather than some point-of-detail. Non-IEEE
>>> floating point often sacrifices things like the distinctions between
>>> "Not-a-Number" and "Infinity," or the difference between positive and
>>> negative
>>> zero, and so on. Perhaps in some cases the rounding of the last bit is
>>> wrong.
>>> But no FP system should give results that are flat out wrong, especially
>>> when
>>> doing arithmetic. (I can see where series of operations involving exponents
>>> &c. might get way out-of-line but I don't think Sqlite is doing any of these
>>> things.)
>>>
>>> What he is getting back looks like Double.MaxVal... is there a
>>> divide-by-zero
>>> somewhere? That is the sort of thing that different FP specs will legimately
>>> handle differently.
>>> ________________________________________
>>> From: sqlite-users-bounces@... [sqlite-users-bounces@...] On
>>> Behalf Of D. Richard Hipp [drh@...]
>>> Sent: Wednesday, November 04, 2009 9:26 AM
>>> To: General Discussion of SQLite Database
>>> Cc: Alexander Drozd
>>> Subject: Re: [sqlite] SQLite on PocketBook
>>>
>>> On Nov 4, 2009, at 4:53 AM, Alexander Drozd wrote:
>>>>
>>>>  My name is Alexander. I am working on an open-source  spaced-
>>>> repetition software project  (http://code.google.com/p/pbanki/). My
>>>> software relies on SQLite library. I came across some bug-like
>>>> problems with running SQLite on a low-memory e-ink reader device. I
>>>> am very  sorry to bother you, but I tried to submit my  problem to
>>>> the bugtracker at the SQLite site, and for some reason anonymous
>>>> login failed.
>>>>
>>>>  The problem appears at the point of reading real values from an
>>>> SQLite database. I created a simple database
>>>>
>>>> CREATE TABLE cards (
>>>>    text TEXT NOT NULL,
>>>>    value REAL NOT NULL
>>>> );
>>>>
>>>> I also tried to use NUMERIC and FLOAT instead of REAL. Then I
>>>> inserted a few values:
>>>>
>>>> INSERT INTO cards VALUES('second',100.1);
>>>> INSERT INTO cards VALUES('first', 100.0);
>>>>
>>>> Then I execute "select * from cards order by due" query with sample
>>>> code from http://www.sqlite.org/quickstart.html It works perfectly
>>>> when compiled on desktop computer, but fails on target device. The
>>>> device is PocketBook301+ (http://pocketbook.com.ua/). Unfortunately
>>>> their site does not have an English version. This device is based on
>>>> Samsung S3C2440 AL-40 CPU. It runs under open-source firmware called
>>>> pocketbookfree, that is based on Linux 2.6.18.2 armv4tl.
>>>>
>>>> The above query run on pocketbook returns corrupted values for
>>>> floats if they have a non-zero fractional part:
>>>>
>>>> text = first
>>>> val = 100.0
>>>>
>>>> text = second
>>>> val = 1.90359837350824e+185
>>>>
>>>> Sorting by columns containing float numbers also fails when
>>>> specified with ORDER BY. I am not sure whether this is an issue with
>>>> SQLite or with cross-compiler for PocketBook, but I would greatly
>>>> appreciate any suggestions on how to treat this problem.
>>>
>>>
>>> I'm guessing that your hardward does not implement IEEE 754 floating
>>> point correctly.  We've seen that sort of thing before, especially
>>> with GCC.  There are some options to GCC (which escape my memory right
>>> now) that can force it to use strict IEEE 754 floating point rather
>>> than its preferred, speedier but non-standard alternative.
>>>
>>>
>>> D. Richard Hipp
>>> drh@...
>>>
>>>
>>>
>>> _______________________________________________
>>> sqlite-users mailing list
>>> sqlite-users@...
>>> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>>>
>>> The information contained in this e-mail is privileged and confidential
>>> information intended only for the use of the individual or entity named.  If
>>> you are not the intended recipient, or the employee or agent responsible for
>>> delivering this message to the intended recipient, you are hereby notified
>>> that any disclosure, dissemination, distribution, or copying of this
>>> communication is strictly prohibited.  If you have received this e-mail in
>>> error, please immediately notify the sender and delete any copies from your
>>> system.
>>> _______________________________________________
>>> sqlite-users mailing list
>>> sqlite-users@...
>>> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>>
>> _______________________________________________
>> sqlite-users mailing list
>> sqlite-users@...
>> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>>
>
>
>
> --
> Puneet Kishor http://www.punkish.org
> Carbon Model http://carbonmodel.org
> Charter Member, Open Source Geospatial Foundation http://www.osgeo.org
> Science Commons Fellow, http://sciencecommons.org/about/whoweare/kishor
> Nelson Institute, UW-Madison http://www.nelson.wisc.edu
> -----------------------------------------------------------------------
> Assertions are politics; backing up assertions with evidence is science
> =======================================================================
> Sent from Madison, Wisconsin, United States
> _______________________________________________
> sqlite-users mailing list
> sqlite-users@...
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

_______________________________________________
sqlite-users mailing list
sqlite-users@...
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

[Simon Slavin-3]

On 5 Nov 2009, at 10:45pm, D. Richard Hipp wrote:

> I'm sorry that you are having difficulty and that SQLite is not
> working to your satisfaction.  But the fact remains that it works
> great on every platform that I have access to.  If you are able to
> find a bug in SQLite we will be happy to look into it for you.  But we
> have no capability of helping you with your ARM problems.

Just to supplement what DRH wrote, iPhones have an ARM processor, and  
SQLite is prevalent throughout the iPhone system software -- not just  
Apps written by the public but in the standard system operations like  
the list of recent calls, and how SMS messages work.  I'm not saying  
that the iPhone is completely bug free, but if there were significant  
bugs in the iPhone version of SQLite, five million users would  
probably have discovered it.  So if there's a problem with how SQLite  
is working on your platform, this is not something that affects all  
ARM devices.  But you'll need someone who understands the low level of  
SQLite better than I do to figure out what's going wrong.

Simon.
_______________________________________________
sqlite-users mailing list
sqlite-users@...
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

[John Crenshaw-2]

SQLite will have some overhead, no questions asked. If you aren't using
indexes, a raw loop and raw data is going to be faster. A raw file will
also always be smaller. That said, Dr. Hipp is right. If you are STORING
the data, you should strongly consider using SQLite rather than
proprietary file formats. SQLite is far more flexible, and the moment
you need to adjust the schema, or and an index, SQLite will immediately
become worth it.

I wouldn't use SQLite for most in memory data that never needs to be
stored on disk, but I STRONGLY recommend SQLite for persistent data.

John

-----Original Message-----
From: sqlite-users-bounces@...
[mailto:sqlite-users-bounces@...] On Behalf Of Kavita Raghunathan
Sent: Thursday, November 05, 2009 5:44 PM
To: General Discussion of SQLite Database
Subject: [sqlite] Local data structures vs sqlite


I saw a presentation on sqlite by Dr Hipp that mentioned that anytime
I'm
storing data in structures or tables, I should be thinking about using
sqlite instead.

Would it be more efficient to use the sqlite database to store a table
that
Looks like this: where lets say I'm looking for the word "auto-align".
Would
the query be quicker than searching through this table in a "for" or
while
loop? Assume the table has about 200 entries. I want to know if the
performance will be better and if I should consider storing these
constants
in the database.

.
.
  {"giants",                e_sf_attr_pm_ethernet_giants},
  {"last_time_cleared",     e_sf_attr_pm_ethernet_last_time_cleared},
  {"port_counters_start",   e_sf_attr_pm_ethernet_port_counters_start},
  {"port_counters_end",     e_sf_attr_pm_ethernet_port_counters_end},
  {"mac_rcv_unicast",       e_sf_attr_pm_ethernet_mac_rcv_unicast},
  {"mac_rcv_multicast",     e_sf_attr_pm_ethernet_mac_rcv_multicast},
  {"mac_rcv_broadcase",     e_sf_attr_pm_ethernet_mac_rcv_broadcast},
  {"mac_xmit_unicast",      e_sf_attr_pm_ethernet_mac_xmit_unicast},
  {"mac_xmit_multicast",    e_sf_attr_pm_ethernet_mac_xmit_multicast},
  {"mac_xmit_broadcast",    e_sf_attr_pm_ethernet_mac_xmit_broadcast},
  {"mac_rcv_octet",         e_sf_attr_pm_ethernet_mac_rcv_octet},
  {"mac_xmit_octet",        e_sf_attr_pm_ethernet_mac_xmit_octet},
  {"mac_delay_exceed",      e_sf_attr_pm_ethernet_mac_delay_exceed},
  {"mac_mtu_exceed",        e_sf_attr_pm_ethernet_mac_mtu_exceed},
  {"mac_in_discard",        e_sf_attr_pm_ethernet_mac_in_discard},
  {"mac_out_discard",       e_sf_attr_pm_ethernet_mac_out_discard},
  {"mac_last_time_cleared",
e_sf_attr_pm_ethernet_mac_last_time_cleared},
  {"manual_align",           e_sf_attr_pm_manual_alig},
  {"auto_align",             e_sf_attr_pm_auto_align},
  {"initial_align",          e_sf_attr_pm_initial_align},
  {"seconds_on_align",       e_sf_attr_pm_seconds_on_align},
  {"align_start_time",       e_sf_attr_pm_last_align_start_time},
  {"align_start_trigger",    e_sf_attr_pm_last_align_start_trigger},
  {"align_start_azimuth",    e_sf_attr_pm_last_align_start_azimuth},
  {"align_start_elevation",  e_sf_attr_pm_last_align_start_elevation},
  {"align_start_rssi",       e_sf_attr_pm_last_align_start_rssi},
  {"align_start_ber",        e_sf_attr_pm_last_align_start_ber},
  {"align_end_time",         e_sf_attr_pm_last_align_end_time},
.
.


On 11/5/09 4:15 PM, "Beau Wilkinson" <beau@...> wrote:

> I really think this warrants further discussion. Perhaps the correct
answer
> (that ARMs implement a non-standard FP type which is incompatible with
Sqlite)
> is already out there, but I think the issues I raised with that answer
should
> at least be addressed.
>
> Assuming (and perhaps this is the rub...) that Sqlite is built around
C++
> "float" and "double,"  then I fail to see how any FP system that is
even
> plausibly useful could give the results cited by Mr Drozd. If I put
(for
> example) the value 100.0 into a "double," and then transport or
store/retrieve
> the binary representation somehow, and then take those bits and once
more
> treat them as a "double," then I ought to get 100 (or at least
something very,
> very close). These are the sorts of things that Sqlite should, to my
mind at
> least, be doing with real number data, and it ought not to matter what
the
> underlying representation is.
>
> And yet it has been put forth in this forum that such is not the case.
Rather,
> the underlying representation must comply with the IEEE FP standard,
or even
> basic operations will not work. And this is so certain, well-known,
and
> reasonable that discussion amongst the plebians is not warranted.
>
> How is this possible architecturally? The only explanation I can
fathom is
> that Sqlite depends on the underlying representation following the
IEEE
> standard at the bit level. For example, when doing sorts, maybe Sqlite
is
> assuming the mantissae and exponents are in the bit ranges specified
by IEEE,
> and that they are represented in the specified format (e.g. excess vs.
> complement notation) as well.
>
> If this is indeed the case, I think this is a very misguided
architecture.
> Depending on the bit-level representation is bad. It's a brittle
design. Of
> course, it's easy for you all to intimidate anyone who has a problem
with this
> architecture... the complainer is "not in compliance with the IEEE
standard"
> and is thus clearly worthy of your speedy dismissal. Bah.
>
> Ultimately, I think this is an easy excuse for a bad design. Types
like
> "float" and "double" are intended by their designers to abstract over
many FP
> implementations. They are not just convenient macros from IEEE FP, nor
should
> they be.
>
> I could go on to take issue with the IEEE standard itself. The
allocation of
> bits to exponent-versus-mantissa is rigid, for example. IEEE makes no
> allowance (that I know of) for allowing a tradeoff between precision
and
> dynamic range, which is a major oversight for such a widely-used
standard.
> Until very recently IEEE FP included no support for 16-bit (half
precision)
> data. IEEE was also designed by committee so it includes all sorts of
> nice-to-have pet features (two zeros, distinct error and condition
codes,
> etc.) which may or may not be worthwhile on any given real-world
system. (I
> tend to lean toward the "may not" direction).
>
> But whether IEEE is bad or good or indifferent makes no difference-
the
> standard should not, in my opinion, be built into Sqlite. Basic
software
> engineering sense must still trump even the best standard.
>
> Forgive me if I have missed something here, but this seems like what I
would

> call "Standardizationism" run amok.
>
> ________________________________________
> From: Beau Wilkinson
> Sent: Wednesday, November 04, 2009 9:39 AM
> To: General Discussion of SQLite Database
> Cc: Alexander Drozd
> Subject: RE: [sqlite] SQLite on PocketBook
>
>> I'm guessing that your hardward does not implement IEEE 754 floating
>> point correctly.  We've seen that sort of thing before, especially
>> with GCC.  There are some options to GCC (which escape my memory

right
>> now) that can force it to use strict IEEE 754 floating point rather
>> than its preferred, speedier but non-standard alternative.
>
> What he's getting back is so far from correct, though, that I would
tend to
> blame a run-of-the-mill bug rather than some point-of-detail. Non-IEEE
> floating point often sacrifices things like the distinctions between
> "Not-a-Number" and "Infinity," or the difference between positive and
negative
> zero, and so on. Perhaps in some cases the rounding of the last bit is
wrong.
> But no FP system should give results that are flat out wrong,
especially when
> doing arithmetic. (I can see where series of operations involving
exponents
> &c. might get way out-of-line but I don't think Sqlite is doing any of
these
> things.)
>
> What he is getting back looks like Double.MaxVal... is there a
divide-by-zero
> somewhere? That is the sort of thing that different FP specs will
legimately
> handle differently.
> ________________________________________
> From: sqlite-users-bounces@...
[sqlite-users-bounces@...] On

> Behalf Of D. Richard Hipp [drh@...]
> Sent: Wednesday, November 04, 2009 9:26 AM
> To: General Discussion of SQLite Database
> Cc: Alexander Drozd
> Subject: Re: [sqlite] SQLite on PocketBook
>
> On Nov 4, 2009, at 4:53 AM, Alexander Drozd wrote:
>>
>>  My name is Alexander. I am working on an open-source  spaced-
>> repetition software project  (http://code.google.com/p/pbanki/). My
>> software relies on SQLite library. I came across some bug-like
>> problems with running SQLite on a low-memory e-ink reader device. I
>> am very  sorry to bother you, but I tried to submit my  problem to
>> the bugtracker at the SQLite site, and for some reason anonymous
>> login failed.
>>
>>  The problem appears at the point of reading real values from an
>> SQLite database. I created a simple database
>>
>> CREATE TABLE cards (
>>    text TEXT NOT NULL,
>>    value REAL NOT NULL
>> );
>>
>> I also tried to use NUMERIC and FLOAT instead of REAL. Then I
>> inserted a few values:
>>
>> INSERT INTO cards VALUES('second',100.1);
>> INSERT INTO cards VALUES('first', 100.0);
>>
>> Then I execute "select * from cards order by due" query with sample
>> code from http://www.sqlite.org/quickstart.html It works perfectly
>> when compiled on desktop computer, but fails on target device. The
>> device is PocketBook301+ (http://pocketbook.com.ua/). Unfortunately
>> their site does not have an English version. This device is based on
>> Samsung S3C2440 AL-40 CPU. It runs under open-source firmware called
>> pocketbookfree, that is based on Linux 2.6.18.2 armv4tl.
>>
>> The above query run on pocketbook returns corrupted values for
>> floats if they have a non-zero fractional part:
>>
>> text = first
>> val = 100.0
>>
>> text = second
>> val = 1.90359837350824e+185
>>
>> Sorting by columns containing float numbers also fails when
>> specified with ORDER BY. I am not sure whether this is an issue with
>> SQLite or with cross-compiler for PocketBook, but I would greatly
>> appreciate any suggestions on how to treat this problem.
>
>
> I'm guessing that your hardward does not implement IEEE 754 floating
> point correctly.  We've seen that sort of thing before, especially
> with GCC.  There are some options to GCC (which escape my memory right
> now) that can force it to use strict IEEE 754 floating point rather
> than its preferred, speedier but non-standard alternative.
>
>
> D. Richard Hipp
> drh@...
>
>
>
> _______________________________________________
> sqlite-users mailing list
> sqlite-users@...
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>
> The information contained in this e-mail is privileged and

confidential
> information intended only for the use of the individual or entity
named.  If
> you are not the intended recipient, or the employee or agent
responsible for
> delivering this message to the intended recipient, you are hereby
notified
> that any disclosure, dissemination, distribution, or copying of this
> communication is strictly prohibited.  If you have received this
e-mail in
> error, please immediately notify the sender and delete any copies from
your
> system.
> _______________________________________________
> sqlite-users mailing list
> sqlite-users@...
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

_______________________________________________
sqlite-users mailing list
sqlite-users@...
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
_______________________________________________
sqlite-users mailing list
sqlite-users@...
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

[Jean-Christophe Deschamps]

Hi John,

>I wouldn't use SQLite for most in memory data that never needs to be
>stored on disk

Even this depends entirely on your context.  Of course if only a simple
lookup in a table is more or less all you ever need, there is little
point in SQLite.  But if or when your requirements get more complex and
things shift away from what basic data structures can do, SQLite can be
something worst considering.

For instance, if the language you use doesn't offer native support for
flexible data structure and fancy access to such data, you can feel
much more comfortable using memory DB(s) than a collection of
third-party libraries, not always consistent between each other, to
achieve the same effect as SQLite can offer in minutes.

I use AutoIt (a very complete Basic-like scripting language for
Windows) to develop most applications I need.  It doesn't offer
structures, nor unions, nor objects nor, say, associative arrays.  But
it only takes a dozen lines of code to have associative arrays
available using an SQLite memory database, with more flexibility in
usage than most AA implementations rigidly built into many fashionable
languages.

For instance I use memory DBs to store all the (volatile obviously)
settings and readings of a series of Lan-enabled (NI Visa) GPIB
measurement instruments and I can perform sophisticated calibration
sequences accross several of them with only a handful of SQLite
statements, using the available AutoIt Visa support.  Doing the same
with native variants and arrays of AutoIt and proprietary structures
and code would be very difficult and error-prone: hundreds of
parameters and a fairly large set of constraints between them.  Doing
the same --even in C[++]-- without SQLite would turn this into a _much_
bigger development.

OTOH I also use the same setup to develop applications for my wife's
business back office and there again I've finaly found SQLite memory
DBs extremely simple to use and powerful.

In the two cases, I don't care how much memory is needed for data, nor
which size the executable and shared libraries sum up to, nor the
overhead these layers have: all this is faster than human brain, so I
consider myself lucky.

The picture obviously blurs in situations where resources and cycles
are less abundant.  SQLtiny for MicroChip controllers: anyone? ;-))



_______________________________________________
sqlite-users mailing list
sqlite-users@...
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

[Jean-Christophe Deschamps]

You read 'worse' instead of 'worst', of course!



_______________________________________________
sqlite-users mailing list
sqlite-users@...
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

[bfr00]

On Nov 5, 2009, at 5:44 PM, Jean-Christophe Deschamps wrote:

> Hi John,
>
>> I wouldn't use SQLite for most in memory data that never needs to be
>> stored on disk
>
> Even this depends entirely on your context.  Of course if only a  
> simple
> lookup in a table is more or less all you ever need, there is little
> point in SQLite.  But if or when your requirements get more complex  
> and
> things shift away from what basic data structures can do, SQLite can  
> be
> something worst considering.

Nice Freudian slip!
_______________________________________________
sqlite-users mailing list
sqlite-users@...
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

[Mohd Radzi Ibrahim]

I think "worth" should be more accurate.


----- Original Message -----
From: "Jean-Christophe Deschamps" <jcd@...>
To: "General Discussion of SQLite Database" <sqlite-users@...>
Sent: Friday, November 06, 2009 9:47 AM
Subject: Re: [sqlite] Local data structures vs sqlite


> You read 'worse' instead of 'worst', of course!
>
>
>
> _______________________________________________
> sqlite-users mailing list
> sqlite-users@...
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>

_______________________________________________
sqlite-users mailing list
sqlite-users@...
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

[Doug Currie-2]

On Nov 5, 2009, at 5:15 PM, Beau Wilkinson wrote:

> I really think this warrants further discussion. Perhaps the correct  
> answer (that ARMs implement a non-standard FP type which is  
> incompatible with Sqlite) is already out there, but I think the  
> issues I raised with that answer should at least be addressed.

I don't know if this is the problem on PocketBook, but...

We have successfully used the SQLite compile option  
SQLITE_MIXED_ENDIAN_64BIT_FLOAT when building SQLite for ARM to get  
interoperability of databases between ARM Linux and other platforms  
such as x86 Linux, MacOSX, and Windows. Some compilers and runtimes  
for ARM use a format wherein the two 32-bit halves of a double are  
swapped relative to other platforms (the two 32-bit words are in big-
endian order, whereas the bytes in the words are in little endian  
order, hence the rationale for the MIXED nomenclature in the option  
name).

e

_______________________________________________
sqlite-users mailing list
sqlite-users@...
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

[Jean-Christophe Deschamps]

>I think "worth" should be more accurate.

Yes, rotfl! I realize this while hitting Send and didn't dare one more
post.

You know, I'm a victim of rogue cosmic rays and badly need one of those
stock ECC brains.  Sorry for my French accent btw!



_______________________________________________
sqlite-users mailing list
sqlite-users@...
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

[Jean-Christophe Deschamps]

>Nice Freudian slip!

Again rotfl, when you know that slip is French word for men underwear...



_______________________________________________
sqlite-users mailing list
sqlite-users@...
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

[John Crenshaw-2]

>>I wouldn't use SQLite for most in memory data that never needs to be
>>stored on disk
>
>Even this depends entirely on your context.  Of course if only a simple

>lookup in a table is more or less all you ever need, there is little
>point in SQLite.  But if or when your requirements get more complex and

>things shift away from what basic data structures can do, SQLite can be

>something worst considering.

Yes. The operative word in my statement was "most". In other words, the
vast majority of the time, data and structure needs are simple. Most
data doesn't need to be stored and accessed in a way that makes SQL
useful. Occasionally memory databases/tables are valuable, but it is the
exception rather than the rule. I just don't want anyone saying "but
this one dude told me to replace all my structs with SQLite memory
tables."

>For instance, if the language you use doesn't offer native support for
>flexible data structure and fancy access to such data, you can feel
>much more comfortable using memory DB(s) than a collection of
>third-party libraries, not always consistent between each other, to
>achieve the same effect as SQLite can offer in minutes.

I think the best plan in this case would be to get a real language (the
exception being if the language in question is assembly, in which case
you didn't want SQLite anyway).

>I use AutoIt (a very complete Basic-like scripting language for
>Windows) to develop most applications I need.  It doesn't offer
>structures, nor unions, nor objects nor, say, associative arrays.  But
>it only takes a dozen lines of code to have associative arrays
>available using an SQLite memory database, with more flexibility in
>usage than most AA implementations rigidly built into many fashionable
>languages.

I would go crazy if I had to use a language without any structures,
objects, or arrays. How could a man LIVE without arrays? Heck, even
assembly language has arrays and/or structures (allocate your memory,
and use offsets).

John
_______________________________________________
sqlite-users mailing list
sqlite-users@...
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users