Not constant sha1sum

[LERTI - Paul Vidonne]

Hello all !

Does smb  met the following issue : several hash for an
unique file ? Of course a big one (4 GB). OS is Linux
Fedora. File system EXT3 mounted on a SATA RAID-5 on Adaptec
card

Could you enlighten me ?

Exemple :
[root@spica acquisit]# sha1sum -b 07667-SDH-dd.001
fe8195547af6d7ce76cd2e44160e06310a964063 *07667-SDH-dd.001

[root@spica acquisit]# sha1sum -b 07667-SDH-dd.001
e8dde55722ed1f2424fd7bb6246163120c561927 *07667-SDH-dd.001

[root@spica acquisit]# sha1sum -b 07667-SDH-dd.001
65f5eb98d33f7ccb1a8a82b0e6d916921c9d97b9 *07667-SDH-dd.001

The best is that the second hash is the good one !

Truly yours,

Paul Vidonne
--
LERTI - Laboratoire d'Expertise et de
  Recherche de Traces Informatiques
http://www.lerti.fr +33.4 76 90 54 21

[Isaac Perez Moncho]

If hash changes the file had changed.
Hash should be the same for the same file (if not changed) always, if
not is useless.
What are you doing with this file? You are writing it while hashing?
You can control the access to the file?
Maybe a hide program is accessing it.

En/na LERTI - Paul Vidonne ha escrit:

> Hello all !
>
> Does smb  met the following issue : several hash for an
> unique file ? Of course a big one (4 GB). OS is Linux
> Fedora. File system EXT3 mounted on a SATA RAID-5 on Adaptec
> card
>
> Could you enlighten me ?
>
> Exemple :
> [root@spica acquisit]# sha1sum -b 07667-SDH-dd.001
> fe8195547af6d7ce76cd2e44160e06310a964063 *07667-SDH-dd.001
>
> [root@spica acquisit]# sha1sum -b 07667-SDH-dd.001
> e8dde55722ed1f2424fd7bb6246163120c561927 *07667-SDH-dd.001
>
> [root@spica acquisit]# sha1sum -b 07667-SDH-dd.001
> 65f5eb98d33f7ccb1a8a82b0e6d916921c9d97b9 *07667-SDH-dd.001
>
> The best is that the second hash is the good one !
>
> Truly yours,
>
> Paul Vidonne
> --
> LERTI - Laboratoire d'Expertise et de
>   Recherche de Traces Informatiques
> http://www.lerti.fr +33.4 76 90 54 21
>
>
>
>  

--
Isaac Perez Moncho
GSEC, SSP-GHD, SSP-MPA, Microsoft MCP.
JPL TSolucio S.L
www.tsolucio.com

[Valdis.Kletnieks]

On Sat, 08 Sep 2007 10:33:38 +0200, LERTI - Paul Vidonne said:
> Hello all !
>
> Does smb  met the following issue : several hash for an
> unique file ? Of course a big one (4 GB). OS is Linux
> Fedora. File system EXT3 mounted on a SATA RAID-5 on Adaptec
> card
>
> Could you enlighten me ?

Most likely, you have marginal hardware, and you only *think* you're
sha1sum'ing the exact same data every time.  With a 4G file, it wouldn't
be at all surprising if a single-bit error was creeping in once in a while.

Have you run memtest86 or similar on the box in question?

[Morgan Reed]

On 9/8/07, LERTI - Paul Vidonne <paul.vidonne@...> wrote:
> Does smb  met the following issue : several hash for an
> unique file ? Of course a big one (4 GB). OS is Linux
> Fedora. File system EXT3 mounted on a SATA RAID-5 on Adaptec
> card
>
> Could you enlighten me ?

I gather you mean you're trying to SHA1 sum a large file on an SMB share?

I've seen similar tee'ing a dd image through md5sum/sha1sum over a
network, and also large files on slow media (CF card).

I suspect the only way you will get consistent results is if you run
them on local data (i.e. copy the image locally and run it or
alternatively run it on the machine which stores the data)

Best Regards,

Morgan

[Jon Hill-2]

If I'm reading your post correctly, you have an entirely Linux system (NOT mounting a remote SMB filesystem) that when you perform a SHA1Sum on a file you're getting different hashes? There are only a few things I can think of that would cause this:

1.) The file is changing but it's unlikely given your second hash matches a previously generated hash (I'm assuming from your comment).

2.) The SHA1Sum app isn't reading data in a consistent manner. This can be caused by a number of systems-level issues related to reading data:

 *) The Adaptec RAID driver,
 *) a bad implementation of the "read" call in the SHA1Sum app,
 *) failing disk cache,
 *) pick your random SysAdmin issue...

Linux doesn't complain as loudly as Windows does  with these kinds of issues. I'd recommend copying the image off to another storage volume and trying to run the hash again. That would at least tell you if it's storage subsystem related. As a note, I'd recommend using dcfldd to copy it...

[Greg Freemyer]

On 9/8/07, LERTI - Paul Vidonne <paul.vidonne@...> wrote:

> Hello all !
>
> Does smb  met the following issue : several hash for an
> unique file ? Of course a big one (4 GB). OS is Linux
> Fedora. File system EXT3 mounted on a SATA RAID-5 on Adaptec
> card
>
> Could you enlighten me ?
>
> Exemple :
> [root@spica acquisit]# sha1sum -b 07667-SDH-dd.001
> fe8195547af6d7ce76cd2e44160e06310a964063 *07667-SDH-dd.001
>
> [root@spica acquisit]# sha1sum -b 07667-SDH-dd.001
> e8dde55722ed1f2424fd7bb6246163120c561927 *07667-SDH-dd.001
>
> [root@spica acquisit]# sha1sum -b 07667-SDH-dd.001
> 65f5eb98d33f7ccb1a8a82b0e6d916921c9d97b9 *07667-SDH-dd.001
>

The only times I've seen inconsistent hashes, there were disk errors
reported in /var/log/warn.

ie. Some sectors were failing the CRC check repeatedly and returning
different values each time they were read.

Greg
--
Greg Freemyer
Litigation Triage Solutions Specialist
http://www.linkedin.com/in/gregfreemyer

The Norcross Group
The Intersection of Evidence & Technology
http://www.norcrossgroup.com

[Paul Vidonne]

Hello All,

Thanks for all your good advices. The issue is probably a bad hardware.

File are stored on a EXT3 partition on a server running Linux Fedora.
When I compute with Linux (through ssh, then with server resources)
I have a bad result.
When I compute these same files situated on the same server with Windows
(through Samba, then with desktop resources) I have a good result.

As a conclusion I would say "Never trust one hash only. Forensic good
practice require two equal hash coming from two different ways"

At 10:33 08/09/07 +0200, LERTI - Paul Vidonne wrote:

>Hello all !
>
>Does smb  met the following issue : several hash for an
>unique file ? Of course a big one (4 GB). OS is Linux
>Fedora. File system EXT3 mounted on a SATA RAID-5 on Adaptec
>card
>
>Could you enlighten me ?
>
>Exemple :
>[root@spica acquisit]# sha1sum -b 07667-SDH-dd.001
>fe8195547af6d7ce76cd2e44160e06310a964063 *07667-SDH-dd.001
>
>[root@spica acquisit]# sha1sum -b 07667-SDH-dd.001
>e8dde55722ed1f2424fd7bb6246163120c561927 *07667-SDH-dd.001
>
>[root@spica acquisit]# sha1sum -b 07667-SDH-dd.001
>65f5eb98d33f7ccb1a8a82b0e6d916921c9d97b9 *07667-SDH-dd.001
>
>The best is that the second hash is the good one !
>
>Truly yours,
>
>Paul Vidonne
>--
>LERTI - Laboratoire d'Expertise et de
>  Recherche de Traces Informatiques
>http://www.lerti.fr +33.4 76 90 54 21

--
Paul Vidonne Consultant
16, chemin de Malacher
38240 Meylan
Tel : [phone]
http://www.vidonne.fr