Major data loss: Calc formatting numbers

[Lars Nooden]

Dotan Cohen wrote:

> I cannot reproduce the rounding incident...

That in itself may be an indication of the cause of the problem...

It would be of great help if you did manage to stumble across a way to
reproduce the problem.

-Lars

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...

[Per-13]

Dotan Cohen skrev:

> When I'm done I will then have to convert about
> 30 spreadsheets and I haven't even looked at how many word processor
> documents, and convert then to MSO 2003's native format, then check
> again for formatting and dataloss.
Phu, many hours with "fun" work, ´cause the lack of a proper backup
routine..

> OOo will not be used in the end,
> this whole OOo switch has been a headache for all involved and while
> I'm not sorry that it happened as I have learned a lot about what a
> real organization needs from an office suite,
don´t forget the hunt for the problem that caused this formatting-thing....
it´s when there is a problem you have to solve that all your "techie"
and computer skills are tested to the limit and you are learning about
the software under this process to..
but trying to pinpoint the problem could be a real pain..   :-)
>
> And what of backups? Some files, but not all due to space
> considerations, will simply be put in a BackMeUp directory that will
> be copied to DVD whenever someone feels like it.
>  

Funny that the organization don´t see the value with backups, even after
this incident.... :-\


Best of luck with the converting of the files...


// Per

[Eike Rathke]

Hi Dotan,

On Friday, 2009-07-03 09:22:29 +0300, Dotan Cohen wrote:

> There were other dealbreakers as well, such as the American dates
> issue when editing a date in Calc. That was and continued to be a huge
> issue, and has caused dataloss in the past.

But you are aware that the editing date format, as well as all default
display formats if nothing differently applied to a cell, depends on the
locale used on the system, and the locale can be overridden in
Tools->Options->LanguageSettings->Languages?

  Eike

--
 OOo/SO Calc core developer. Number formatter stricken i18n transpositionizer.
 SunSign   0x87F8D412 : 2F58 5236 DB02 F335 8304  7D6C 65C9 F9B5 87F8 D412
 OpenOffice.org Engineering at Sun: http://blogs.sun.com/GullFOSS
 Please don't send personal mail to the erl@... account, which I use for
 mailing lists only and don't read from outside Sun. Use erack@... Thanks.

[Dave Post]

On Jul 3, 2009, at 2:56 AM, Lars Nooden wrote:

> Dotan Cohen wrote:
>
>> I cannot reproduce the rounding incident...
>
> That in itself may be an indication of the cause of the problem...

Good point. A simple "glitch" on the host machine -- which may have  
had nothing to do with Calc (e.g., an OS error, memory or disk fault,  
or power fluctuation) -- could have caused the corruption and would  
never be reproducible. The lesson that should be taken from this  
experience is that a critical database shouldn't be hosted on a system  
that lacks a UPS, ECC-memory, and a suitable RAID plus automatic  
backup system.

Given that the problem doesn't seem to have occurred for other users,  
it makes no sense to me for your management to blame OO.

On Jul 3, 2009, at 2:22 AM, Dotan Cohen wrote:

> Why is the least significant figure (days) in the American date  
> format in the middle?!? Do American didgital watches use the  
> HH:SS:MM format by any chance?

Don't know but, being American, I find the Euro format just as  
confusing as they find ours. Similarly, their preference for  
expressing fuel efficiency as volume/distance baffles me. It's just a  
matter of which convention one is accustomed to. As for digital  
watches, every one I've seen uses hr:min:sec, although there are  
surely a few novelty models that differ.

Dave

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...

[dotancohen]

>> I cannot reproduce the rounding incident...
>
> That in itself may be an indication of the cause of the problem...
>
> It would be of great help if you did manage to stumble across a way to
> reproduce the problem.
>

I suspect a version-change had something to do with it, and I am not
about to go through five OOo versions to test!


>> And what of backups? Some files, but not all due to space
>> considerations, will simply be put in a BackMeUp directory that will
>> be copied to DVD whenever someone feels like it.
>
> Funny that the organization don´t see the value with backups, even after
> this incident.... :-\
>

I think that this is a common stupidity.

2009/7/3 Eike Rathke <erl@...>:

> Hi Dotan,
>
> On Friday, 2009-07-03 09:22:29 +0300, Dotan Cohen wrote:
>
>> There were other dealbreakers as well, such as the American dates
>> issue when editing a date in Calc. That was and continued to be a huge
>> issue, and has caused dataloss in the past.
>
> But you are aware that the editing date format, as well as all default
> display formats if nothing differently applied to a cell, depends on the
> locale used on the system, and the locale can be overridden in
> Tools->Options->LanguageSettings->Languages?
>

On my home machine, the system setting is to use the yyyy-mm-dd format
and some applications, such as Thunderbird, respect that. Calc does
not, I just tested. Kubuntu 9.04, OOo 3.1. Tell me where I err, not
the LC_TIME is en_DK.utf8 which forces Thunderbird to give me
yyy-mm-dd dates. Calc still does not.

$ locale
LANG=en_US.UTF-8
LANGUAGE=
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME=en_DK.utf8
LC_COLLATE="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_PAPER="en_US.UTF-8"
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT="en_US.UTF-8"
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=


--
Dotan Cohen

http://what-is-what.com
http://gibberish.co.il

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...

[Robin Laing]

Dotan Cohen wrote:

>>> I cannot reproduce the rounding incident...
>> That in itself may be an indication of the cause of the problem...
>>
>> It would be of great help if you did manage to stumble across a way to
>> reproduce the problem.
>>
>
> I suspect a version-change had something to do with it, and I am not
> about to go through five OOo versions to test!
>
>
>>> And what of backups? Some files, but not all due to space
>>> considerations, will simply be put in a BackMeUp directory that will
>>> be copied to DVD whenever someone feels like it.
>> Funny that the organization don´t see the value with backups, even after
>> this incident.... :-\
>>
>
> I think that this is a common stupidity.
>
> 2009/7/3 Eike Rathke <erl@...>:
>> Hi Dotan,
>>
>> On Friday, 2009-07-03 09:22:29 +0300, Dotan Cohen wrote:
>>
>>> There were other dealbreakers as well, such as the American dates
>>> issue when editing a date in Calc. That was and continued to be a huge
>>> issue, and has caused dataloss in the past.
>> But you are aware that the editing date format, as well as all default
>> display formats if nothing differently applied to a cell, depends on the
>> locale used on the system, and the locale can be overridden in
>> Tools->Options->LanguageSettings->Languages?
>>
>
> On my home machine, the system setting is to use the yyyy-mm-dd format
> and some applications, such as Thunderbird, respect that. Calc does
> not, I just tested. Kubuntu 9.04, OOo 3.1. Tell me where I err, not
> the LC_TIME is en_DK.utf8 which forces Thunderbird to give me
> yyy-mm-dd dates. Calc still does not.
>
> $ locale
> LANG=en_US.UTF-8
> LANGUAGE=
> LC_CTYPE="en_US.UTF-8"
> LC_NUMERIC="en_US.UTF-8"
> LC_TIME=en_DK.utf8
> LC_COLLATE="en_US.UTF-8"
> LC_MONETARY="en_US.UTF-8"
> LC_MESSAGES="en_US.UTF-8"
> LC_PAPER="en_US.UTF-8"
> LC_NAME="en_US.UTF-8"
> LC_ADDRESS="en_US.UTF-8"
> LC_TELEPHONE="en_US.UTF-8"
> LC_MEASUREMENT="en_US.UTF-8"
> LC_IDENTIFICATION="en_US.UTF-8"
> LC_ALL=
>
>

I will second that on Fedora 10.  I have yet to try Fedora 11 as I had
other issues with changing LC_TIME.

--
Robin Laing

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...

[Robin Laing]

Dotan Cohen wrote:

> A new OOo user is using Calc to store information on volunteers in an
> organization. Israeli ID numbers are 9 digits in length, such as this:
> 317907977
>
> However, Calc has reformated them to this:
> 317908000
>
> This is a major dataloss for the organization. The person in charge is
> pissed, and both myself and OOo are to blame. OOo is already coming
> off their machines (14 desktops, and I think a few laptops as well)
> and I am tasked with recovering the missing data. Is this possible?
>

I have used OOo since before it was OOo.  I have not lost data due to
the software in any case.

As for data in spreadsheets.  I don't agree with using information on
users being stored in a spreadsheet.  It isn't what it was designed for.
  It makes it to easy for a simple user error to make a mistake and
corrupt the data.

Backups are for recovering missing data.



--
Robin Laing

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...

[dotancohen]

> I have used OOo since before it was OOo.  I have not lost data due to the
> software in any case.
>

Probably because of:


> As for data in spreadsheets.  I don't agree with using information on users
> being stored in a spreadsheet.  It isn't what it was designed for.  It makes
> it to easy for a simple user error to make a mistake and corrupt the data.
>

That is what this organization does. They have pretty good reasons for
not using Base or Access. This is not the place to discuss their
strategies, however, rather how we can help them achieve the goals
that they have set for themselves.


> Backups are for recovering missing data.
>

I am satisfied with the partial backup plan now in place. It may not
be ideal, but it is good enough. They now have an "Important Files"
folder which will periodically be backed up.


--
Dotan Cohen

http://what-is-what.com
http://gibberish.co.il

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...

[Graham P Davis-3]

Robin Laing wrote:

> As for data in spreadsheets.  I don't agree with using information on
> users being stored in a spreadsheet.  It isn't what it was designed for.
>   It makes it to easy for a simple user error to make a mistake and
> corrupt the data.

I use Calc for storing genealogical data because it's so easy It may not be
what it was designed for but it works. I just want to make the initial
storage in a table and I know of nothing better to use.

--
Graham P Davis, Bracknell, Berks., UK.  E-mail: newsman not newsboy
"I wear the cheese. It does not wear me."


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...

[NoOp-4]

On 07/02/2009 11:22 PM, Dotan Cohen wrote:
....
>
> I cannot reproduce the rounding incident, but from my familiarity with
> the project and the person operating the machine, I do not suspect
> user error. I suspect that moving from OOo 2.2 (as it turns out, this
> whole thing started off as an experiment over two years ago!) to 2.3
> to 2.4 to 3.0 to 3.1 had something to do with the loss. I simply don't
> have the time to perform the upgrades and test so thoroughly, but if
> anyone wnats to try I can provide information.

Send away. I'll be happy to try and reproduce if you'd like. I don't
think I have anything earlier than 2.4.1 around (well except for my
original StarOffice 5.1 CD), but Ubuntu has files going back to 2.0.2
that I can load.

>
> There were other dealbreakers as well, such as the American dates
> issue when editing a date in Calc. That was and continued to be a huge
> issue, and has caused dataloss in the past. Why is the least
> significant figure (days) in the American date format in the middle?!?
> Do American didgital watches use the HH:SS:MM format by any chance?

Only if you buy one from Wal-Mart. But dates are locale & language
dependent... as they are in MSO.

>
> And what of backups? Some files, but not all due to space
> considerations, will simply be put in a BackMeUp directory that will
> be copied to DVD whenever someone feels like it.
>

Well... you already know the answer to that one.

Anyway, if you'd like to send to me directly I'll be happy to both
maintain the confidentiality of the file(s) and see if I can assist.


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...

[dotancohen]

> Send away. I'll be happy to try and reproduce if you'd like. I don't
> think I have anything earlier than 2.4.1 around (well except for my
> original StarOffice 5.1 CD), but Ubuntu has files going back to 2.0.2
> that I can load.
>

I cannot send to you the original file, as it contains sensitive
information, but in 2.x try entering this into a cell:
317907977. Open it and modify in a later version of 2.x then in 3.0
then 3.1. See if it rounds up.


>> There were other dealbreakers as well, such as the American dates
>> issue when editing a date in Calc. That was and continued to be a huge
>> issue, and has caused dataloss in the past. Why is the least
>> significant figure (days) in the American date format in the middle?!?
>> Do American didgital watches use the HH:SS:MM format by any chance?
>
> Only if you buy one from Wal-Mart. But dates are locale & language
> dependent... as they are in MSO.
>

Correct, but OOo ignores the system setting:
http://www.openoffice.org/issues/show_bug.cgi?id=5556

Here was another date-related bug in OOo that caused the same
organization grief:
http://www.openoffice.org/issues/show_bug.cgi?id=85362


>> And what of backups? Some files, but not all due to space
>> considerations, will simply be put in a BackMeUp directory that will
>> be copied to DVD whenever someone feels like it.
>>
>
> Well... you already know the answer to that one.
>

42?

Thanks!


--
Dotan Cohen

http://what-is-what.com
http://gibberish.co.il

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...

[NoOp-4]

On 07/03/2009 05:52 PM, Dotan Cohen wrote:

>> Send away. I'll be happy to try and reproduce if you'd like. I don't
>> think I have anything earlier than 2.4.1 around (well except for my
>> original StarOffice 5.1 CD), but Ubuntu has files going back to 2.0.2
>> that I can load.
>>
>
> I cannot send to you the original file, as it contains sensitive
> information, but in 2.x try entering this into a cell:
> 317907977. Open it and modify in a later version of 2.x then in 3.0
> then 3.1. See if it rounds up.

I assume that you want the test all using Ubuntu versions, or would you
like standard OOo and Windows versions as well? I do have copies of Win
(S)OOo 2.4.1, (S)OOo 3.0.0rc4, (S)OOo 3.0.1, and (S)OOo 3.1.1, and
(S)OOo 3.2.0(Dev)

Note: no issue found between (U)OOo 2.4.1 ==> (U)3.0.1 ==> (U)3.1.0 ==>
(S)3.2.0(Dev). It will take me awhile to fire up the earlier &
in-between versions.
....
>>> And what of backups? Some files, but not all due to space
>>> considerations, will simply be put in a BackMeUp directory that will
>>> be copied to DVD whenever someone feels like it.
>>>
>>
>> Well... you already know the answer to that one.
>>
>
> 42?

43... you forgot to round up.

>
> Thanks!
>
>



---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...

[dotancohen]

> I assume that you want the test all using Ubuntu versions, or would you
> like standard OOo and Windows versions as well? I do have copies of Win
> (S)OOo 2.4.1, (S)OOo 3.0.0rc4, (S)OOo 3.0.1, and (S)OOo 3.1.1, and
> (S)OOo 3.2.0(Dev)
>
> Note: no issue found between (U)OOo 2.4.1 ==> (U)3.0.1 ==> (U)3.1.0 ==>
> (S)3.2.0(Dev). It will take me awhile to fire up the earlier &
> in-between versions.

Actually, they were using the Windows versions.

>> 42?
>
> 43... you forgot to round up.
>

Hehehe.

--
Dotan Cohen

http://what-is-what.com
http://gibberish.co.il

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...

[NoOp-4]

On 07/04/2009 06:01 AM, Dotan Cohen wrote:

>> I assume that you want the test all using Ubuntu versions, or would you
>> like standard OOo and Windows versions as well? I do have copies of Win
>> (S)OOo 2.4.1, (S)OOo 3.0.0rc4, (S)OOo 3.0.1, and (S)OOo 3.1.1, and
>> (S)OOo 3.2.0(Dev)
>>
>> Note: no issue found between (U)OOo 2.4.1 ==> (U)3.0.1 ==> (U)3.1.0 ==>
>> (S)3.2.0(Dev). It will take me awhile to fire up the earlier &
>> in-between versions.
>
> Actually, they were using the Windows versions.

Is there a particular version they were using (between x and y) that
they noticed the issue?  I can only test from 2.4.1 to current.


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...

[NoOp-4]

On 07/05/2009 02:03 PM, NoOp wrote:

> On 07/04/2009 06:01 AM, Dotan Cohen wrote:
>>> I assume that you want the test all using Ubuntu versions, or would you
>>> like standard OOo and Windows versions as well? I do have copies of Win
>>> (S)OOo 2.4.1, (S)OOo 3.0.0rc4, (S)OOo 3.0.1, and (S)OOo 3.1.1, and
>>> (S)OOo 3.2.0(Dev)
>>>
>>> Note: no issue found between (U)OOo 2.4.1 ==> (U)3.0.1 ==> (U)3.1.0 ==>
>>> (S)3.2.0(Dev). It will take me awhile to fire up the earlier &
>>> in-between versions.
>>
>> Actually, they were using the Windows versions.
>
> Is there a particular version they were using (between x and y) that
> they noticed the issue?  I can only test from 2.4.1 to current.

OK - done. I suspect operator error and/or some other problem with your
rounding issue. I have tested (as you suggested:

<quote>
in 2.x try entering this into a cell:
317907977. Open it and modify in a later version of 2.x then in 3.0
then 3.1. See if it rounds up.
</quote>

with WinXPPro:

2.4.1 ==> 2.4.2 ==> 3.0.0 ==> 3.1.0

No rounding errors, no issues.

I created a file for each version & can send to you directly if you wish.




---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...

[dotancohen]

> Is there a particular version they were using (between x and y) that
> they noticed the issue?  I can only test from 2.4.1 to current.
>

Thanks, Gary. The problem was just noticed last week, already on 3.1.

I should note that I insist that this is an OOo issue because I have
had a similar problem, also on a spreadsheet that was created in OOo
2.x. However, I do have backups so when I get to my mother-in-law's
place (I only keep the latest backup on-site) I intend to copy those
backups and see exactly where the problem occured. When I know more I
will file an issue and report back.

Thank you!

--
Dotan Cohen

http://what-is-what.com
http://gibberish.co.il

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...

[dotancohen]

> OK - done. I suspect operator error and/or some other problem with your
> rounding issue. I have tested (as you suggested:
>
> <quote>
> in 2.x try entering this into a cell:
> 317907977. Open it and modify in a later version of 2.x then in 3.0
> then 3.1. See if it rounds up.
> </quote>
>
> with WinXPPro:
>
> 2.4.1 ==> 2.4.2 ==> 3.0.0 ==> 3.1.0
>
> No rounding errors, no issues.
>
> I created a file for each version & can send to you directly if you wish.
>

Thanks. Send to me the latest, I will see if it rounds here and at the
Latet office.

--
Dotan Cohen

http://what-is-what.com
http://gibberish.co.il

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...

[Richard Detwiler-3]

Robin Laing wrote:
>
> I have used OOo since before it was OOo.  I have not lost data due to
> the software in any case.
>
> As for data in spreadsheets.  I don't agree with using information on
> users being stored in a spreadsheet.  It isn't what it was designed
> for.  It makes it to easy for a simple user error to make a mistake
> and corrupt the data.
>

I'm a bit confused by the statement about how information on users
shouldn't be stored in a spreadsheet, because it is so easy to corrupt
the data.

Does this suggest that the only information that should be put in a
spreadsheet is data where it doesn't matter if it gets corrupted?

If so, that would significantly reduce the usefulness of spreadsheets.

I assume there is some key point that I'm missing, so please enlighten
.... thanks.

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...

[Gene Young]

Richard Detwiler wrote:

> Robin Laing wrote:
>>
>> I have used OOo since before it was OOo.  I have not lost data due to
>> the software in any case.
>>
>> As for data in spreadsheets.  I don't agree with using information on
>> users being stored in a spreadsheet.  It isn't what it was designed
>> for.  It makes it to easy for a simple user error to make a mistake
>> and corrupt the data.
>>
>
> I'm a bit confused by the statement about how information on users
> shouldn't be stored in a spreadsheet, because it is so easy to corrupt
> the data.
>
> Does this suggest that the only information that should be put in a
> spreadsheet is data where it doesn't matter if it gets corrupted?
>
> If so, that would significantly reduce the usefulness of spreadsheets.
>
> I assume there is some key point that I'm missing, so please enlighten
> .... thanks.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@...
> For additional commands, e-mail: users-help@...
>
>

Basically a database is for storing and retrieving information.  A
spreadsheet is primarily for performing calculations on data.  In common
usage, people have come to use spreadsheets to store information because
the learning curve is much shallower than learning to work with a database.
--
Gene Y.

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...

[Robert Holtzman]

On Mon, 6 Jul 2009, Gene Young wrote:

> Richard Detwiler wrote:
>>
>> I'm a bit confused by the statement about how information on users
>> shouldn't be stored in a spreadsheet, because it is so easy to corrupt the
>> data.
>>
>> Does this suggest that the only information that should be put in a
>> spreadsheet is data where it doesn't matter if it gets corrupted?
>>
>> If so, that would significantly reduce the usefulness of spreadsheets.
>>
>> I assume there is some key point that I'm missing, so please enlighten ....
>> thanks.

> Basically a database is for storing and retrieving information.  A
> spreadsheet is primarily for performing calculations on data.  In common
> usage, people have come to use spreadsheets to store information because the
> learning curve is much shallower than learning to work with a database.

The answer you stated addresses why people tend to use spreadsheets to
store data. What I would like to know is why you think information in a
spreadsheet can be corrupted more easily than that in a data base. As
the previous said: "I assume there is some key point that I'm missing,
so please enlighten ...."

--
Bob Holtzman
AF9D 8760 0CFA F95A 6C77  E125 BF90 580F 8D54 9279
"If you think you're getting free lunch,
  check the price of the beer"

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...