delete deleted data

[new_guy]

Marco Peereboom wrote:

bullshit.

I decided to put my money where my mouth is :)

I bought a 80GB, Western Digital IDE hard drive. $60 USD. Attached it to a Windows XP laptop (usb-ide bridge), initialized it, created one (1) primary partition, formatted it NTFS and copied an older subversion repository to it. I documented and screen-shot the entire process.

I then booted the laptop with an OpenBSD 4.2 install CD and selected the 's' option and ran dd like this on the hard drive:

dd if=/dev/zero of=/dev/rsd0c

I called three (3) well-known data recovery companies. Two of them said recovery was not possible after the dd procedure, one of them said they'd be willing to try so long as no other data recovery company had opened the HDD case and offered to do a free analysis in one of their ISO certified labs. I'm sending the drive off tomorrow, I'll let you know in a few weeks how it turns out.

Brad

[Marco Peereboom]

Great.  The companies I worked with charged $500 per megabyte.  I am
sure you'll spend that to prove whatever point you are trying to make.

Let me repeat this one more time; I have worked with several of these
companies in the past who have recovered data from flooded and burnt
hard disks.  I also worked with incidental formats and such.  I am not
making this up.

On Thu, Jan 03, 2008 at 11:55:16AM -0800, new_guy wrote:

> Marco Peereboom wrote:
> >
> > bullshit.
> >
>
> I decided to put my money where my mouth is :)
>
> I bought a 80GB, Western Digital IDE hard drive. $60 USD. Attached it to a
> Windows XP laptop (usb-ide bridge), initialized it, created one (1) primary
> partition, formatted it NTFS and copied an older subversion repository to
> it. I documented and screen-shot the entire process.
>
> I then booted the laptop with an OpenBSD 4.2 install CD and selected the 's'
> option and ran dd like this on the hard drive:
>
> dd if=/dev/zero of=/dev/rsd0c
>
> I called three (3) well-known data recovery companies. Two of them said
> recovery was not possible after the dd procedure, one of them said they'd be
> willing to try so long as no other data recovery company had opened the HDD
> case and offered to do a free analysis in one of their ISO certified labs.
> I'm sending the drive off tomorrow, I'll let you know in a few weeks how it
> turns out.
>
> Brad
>
> --
> View this message in context: http://www.nabble.com/delete-deleted-data-tp14560809p14604134.html
> Sent from the openbsd user - misc mailing list archive at Nabble.com.

[Eric Furman-3]

On Thu, 3 Jan 2008 11:55:16 -0800 (PST), "new_guy" <byte8bits@...>
said:

> Marco Peereboom wrote:
> >
> > bullshit.
> >
>
> I decided to put my money where my mouth is :)
>
> I bought a 80GB, Western Digital IDE hard drive. $60 USD. Attached it to
> a
> Windows XP laptop (usb-ide bridge), initialized it, created one (1)
> primary
> partition, formatted it NTFS and copied an older subversion repository to
> it. I documented and screen-shot the entire process.
>
> I then booted the laptop with an OpenBSD 4.2 install CD and selected the
> 's'
> option and ran dd like this on the hard drive:
>
> dd if=/dev/zero of=/dev/rsd0c
>
> I called three (3) well-known data recovery companies. Two of them said
> recovery was not possible after the dd procedure, one of them said they'd
> be
> willing to try so long as no other data recovery company had opened the
> HDD
> case and offered to do a free analysis in one of their ISO certified
> labs.
> I'm sending the drive off tomorrow, I'll let you know in a few weeks how
> it
> turns out.

It can't be done. it's an urban legend, AFAICT.
http://www.nber.org/sys-admin/overwritten-data-guttman.html
Which references Gutmann's paper which started all this...

[Unix Fan]

I'm sorry Marco, but I think what you've said is "bullshit", as well contacted several "so called" data recovery organizations, after admitting to have "zeroed" the drive contents - They said recovery wasn't possible..



While it "might" be possible to get miscellaneous data off of a drive, it would likely be cost prohibitive (if even possible..).



But let's see how new_guy(aka Brad)'s quest goes.. perhaps he can post any documents/paper would returned by the company..



-Nix Fan.

[new_guy]

> It can't be done. it's an urban legend, AFAICT.

Yes I know. That's the whole point of this. It would have been better
to donate a 100 bucks to OpenBSD. I'm just fed-up with the stupid
drivel about needing to burn, grind, overwrite, and nuke drives... and
even after all of that there's still a chance (albeit small) that the
NSA can recover all data from the non-existent drive... out of the
ether I guess

/dev/zero is all you need :)

[new_guy]

On Jan 3, 2008 3:35 PM, Marco Peereboom <slash@...> wrote:
> Great.  The companies I worked with charged $500 per megabyte.  I am
> sure you'll spend that to prove whatever point you are trying to make.

Free analysis. I pay shipping. The drive cost 60 bucks. I'll probably
have a total of 100 bucks in it at most... cause they ain't gonna
recover jack... even in their ISO certified labs. We need to put a
stop to the notion that mulitiple overwrites and grinding and burning
and nuking drives is *required*... it's silly and wasteful. One pass
from /dev/zero is more than enough for all cases.

[Diana Eichert]

You can locate data from formatted and "wiped" hard drive, if you have the
resources behind you.

A place I knows has a large capacity degausser, which a number of drives
get thrown into for a rinse cycle, then they get tossed into a pile.  When
the pile gets big enough the industrial metal shredder is turned on and
the pile gets turned in to little metal pieces which then get sold as
scrap metal.

Pesonally my favorite is using old hard drives for target practice, with
.223 or .50 rounds.

diana

[Diana Eichert]

On Thu, 3 Jan 2008, Brad Tilley wrote:
SNIP
> and nuking drives is *required*... it's silly and wasteful. One pass
> from /dev/zero is more than enough for all cases.

HaHaHa, I wish my day job employer would let me take the drugs you're on.

diana

[new_guy]

On Jan 3, 2008 4:52 PM, Marco Peereboom <slash@...> wrote:
> Really man don't waste your money.  You are talking to a single vendor.
> I have dealt with this in the past.

As have I... on multiple occasions. I am dealing with 3 vendors. 2 of
which declined to review the drive because of the dd overwrite
performed from the OpenBSD 4.2 install CD. I'll send it to the one
'ISO Certified' company that agreed to examine it. If they cannot
recover any data (and they will not be able to). I'll send it to you
and you can recover it yourself...  if you can even tell me the *name*
of one file that was one the drive, I'll give you the drive + 40 USD
bucks (a 100 dollar value!)

I'll put up a website with all the details and pictures... I'll call
it 'Put Up Or Shut Up' Anyone who wants a crack at recovering data
from the drive may do so (as long as they pay the shipping charges
both ways). If they can name one file that existed on the drive before
the dd overwrite from an OpenBSD install CD, then they can keep the
drive and be crowned king of data recovery and get $40 USD. Come on,
let's actually *do* and not just *talk*, OK?

Saying "bullshit" or "hahaha" only goes so far... know what I mean?

[Mark Rolen]

Diana Eichert wrote:
> You can locate data from formatted and "wiped" hard drive, if you have
> the resources behind you.

Can you point to an actual instance you know of where this has
happened?  I don't mean that in an aggressive or challenging way, I'm
sincerely interested after reading that rebuttal of Guttman's paper.  
I've also always subscribed to the "complete destruction" idea.

I don't mean recovery of data where someone accidentally issued a "del"
or "rm" command and the file is pieced back together, or recovery of
some data after filesystem corruption, etc.  I'm wondering if someone
has truly recovered data from a drive where every single bit of data has
been overwritten with zeroes/random data/whatever.

Regards,
Mark

[Marco S Hyman]

"Brad Tilley" writes:
 > performed from the OpenBSD 4.2 install CD. I'll send it to the one
 > 'ISO Certified' company that agreed to examine it. If they cannot

You keep throwing around the 'ISO Certified' tag as if it had some
special meaning.  Certified to what standard?  It makes a difference.
If they are certified to the 9001 standard, for example, all it means
is that they have written procedures and they follow them.  That's
all it means.

ISO 9001 certification is actually pretty easy to get.  The companies
that fail to get it are trying to hard.  They come up with procedures
that sound great but are impossible to follow.  That's not what
certification means.

If I have a software company and write up a procedure that says
"all code will be developed on a laptop while sitting in a Starbucks"
and actually follow that procedure, then I can be an "ISO Certified"
company.

As for disk destruction... I don't know nor pretend to know what can
and can not be recovered.  Take a look at

https://www.dss.mil/portal/ShowBinary/BEA%20Repository/new_dss_internet/isp/odaa/documents/clear_n_san_matrix_06282007_rev_11122007.pdf

The DSS (Defense Security Service, part of the DoD) calls what you have
done "clearing" the disk.   It does not "sanitize" the disk.  To sanitize
you need to either degauss or destroy the disk.

// marc

[new_guy]

Marco S Hyman wrote:

"Brad Tilley" writes:
 > performed from the OpenBSD 4.2 install CD. I'll send it to the one
 > 'ISO Certified' company that agreed to examine it. If they cannot

You keep throwing around the 'ISO Certified' tag as if it had some
special meaning.  Certified to what standard?  

I'm just parroting the *one* data recover company's marketing hype that agreed to take the drive. They make this claim:

"ISO 9001 - 2000 certified"

I'm working on putting a website up now where I'll fully disclose the details. Lots of pictures and details. I will attribute the dd used to OpenBSD (the best OS on the planet bar none... although the dd on the install CD did not support the conv option... I would have liked to have done conv=noerror,sync). I plan to ship the drive off tomorrow. I plan to put this myth to rest... where it belongs.

[Harpalus a Como]

Myth? Why are you so upset about this? It's not myth.

The techniques involved in recovering data in the manner Marco and the NSA,
DoD, and many others describe isn't a matter of running a simple software
tool. It's a long, slow, annoying process that is also costly. But it is
possible. Not every company or person in the forensics industry is a master
at their job. If they say it's not possible, perhaps it's just "not
something their software package does for them?" (I'm not trying to be
derogatory, but I do know a guy who does computer forensics work, and the
software/hardware he uses is about all he knows. He just goes through the
motions. Doesn't know all that much about filesystems or disks.)

Why are you so hellbent on proving everybody wrong, to the point of actually
shipping your drive off? It's by no means a myth. If it is, there are a
number of companies and government institutions interesting in how they
recover data in this fashion if it's "not possible." I'm having a hard time
believing
On Jan 3, 2008 7:54 PM, new_guy <byte8bits@...> wrote:

> Marco S Hyman wrote:
> >
> > "Brad Tilley" writes:
> >  > performed from the OpenBSD 4.2 install CD. I'll send it to the one
> >  > 'ISO Certified' company that agreed to examine it. If they cannot
> >
> > You keep throwing around the 'ISO Certified' tag as if it had some
> > special meaning.  Certified to what standard?
> >
>
> I'm just parroting the *one* data recover company's marketing hype that
> agreed to take the drive. They make this claim:
>
> "ISO 9001 - 2000 certified"
>
> I'm working on putting a website up now where I'll fully disclose the
> details. Lots of pictures and details. I will attribute the dd used to
> OpenBSD (the best OS on the planet bar none... although the dd on the
> install CD did not support the conv option... I would have liked to have
> done conv=noerror,sync). I plan to ship the drive off tomorrow. I plan to
> put this myth to rest... where it belongs.
> --
> View this message in context:
> http://www.nabble.com/delete-deleted-data-tp14560809p14608861.html
> Sent from the openbsd user - misc mailing list archive at Nabble.com.

[Douglas A. Tutty]

On Thu, Jan 03, 2008 at 04:08:08PM -0800, Marco S Hyman wrote:
 
> As for disk destruction... I don't know nor pretend to know what can
> and can not be recovered.  Take a look at
>
> https://www.dss.mil/portal/ShowBinary/BEA%20Repository/new_dss_internet/isp/odaa/documents/clear_n_san_matrix_06282007_rev_11122007.pdf
>
> The DSS (Defense Security Service, part of the DoD) calls what you have
> done "clearing" the disk.   It does not "sanitize" the disk.  To sanitize
> you need to either degauss or destroy the disk.
>

The NIST article that (I think) started this thread says that it (the
document) applies to commercial-grade privacy but not to
government-grade classified material.  In other words, there's an
implied difference between the ability of a commercial data recovery
company and a major government.  

So, you have to look at who your adversary is and the value of the data.
If the value is less than the drive, then clear the disk and sell it.
If you are keeping the disk in-house but just re-allocating it, then
clear the disk and re-use it.  However, if the agency you wish to not be
able to read the disk has the backing of a major government:

1: distroy the disk
2: distroy the computer (the document actually says this re RAM
                chips)
3: re-evaluate the whole concept of using a computer at all,
        expecially if the hardware is at risk of being "stolen" (seized,
        confiscated, etc).

If the data on the drive has always been in encrypted form, then you
have to evaluate the strength of the encryption vs. the strength of the
adversary.  

JM2c

Doug.

[Greg Thomas-3]

On Jan 3, 2008 5:21 PM, Harpalus a Como <harpalus.como@...> wrote:
> Myth?

Have you read this:
http://www.nber.org/sys-admin/overwritten-data-guttman.html?

> Why are you so upset about this?

Myth's that compel people to waste time and energy should be destroyed.

> It's not myth.

Have you read this or any of the papers referenced here:
http://www.nber.org/sys-admin/overwritten-data-guttman.html?

Greg
--
Up mesons in the low desert:  http://lodesertprotosites.org

Dethink to survive - Mclusky

[Unix Fan]

new_guy wrote:

> I'm working on putting a website up now where I'll fully disclose the

> details. Lots of pictures and details. I will attribute the dd used to

> OpenBSD (the best OS on the planet bar none... although the dd on the

> install CD did not support the conv option... I would have liked to have

> done conv=noerror,sync). I plan to ship the drive off tomorrow. I plan to

> put this myth to rest... where it belongs.



Awesome :)



I totally like the idea, I'd like to put the "1 pass is not enough" myth to rest as well... I've been in the computing industry for 20 years, and I still refuse to submit to the "Americas NSA knows all" dogmai.



(I like the "Put Up or Shut Up" slogan as well!!)



Keep us informed Brad :)



-Nix Fan.

[Kennith Mann III]

On 3 Jan 2008 18:55:14 -0800, Unix Fan <unixfan@...> wrote:
> (I like the "Put Up or Shut Up" slogan as well!!)

The problem is that none of us have the funds that the NSA has to
aquire an answer that will actually silence this thread.
The reality is: Who are you trying to protect it against?
That question also allows you to guess what level of funding they have.

If you are talking about $random_person_from_ebay, then sure probably
/dev/zero is Good Enough (TM), however if you are talking about
someone who can assign a dedicated team and spend months on it, with
well over a million dollar budget then you will need to spend an equal
amount to see that answer. This is precisely who some on this list are
saying "that isn't enough" while others are going "it's good enough"
because I suspect one is thinking on on a different budget level.

While I haven't read every single message in this thread, I haven't
seen anyone mention who they are trying to hide the data from.

[Diana Eichert]

On Thu, 3 Jan 2008, Mark Rolen wrote:

> Diana Eichert wrote:
>> You can locate data from formatted and "wiped" hard drive, if you have the
>> resources behind you.
>
> Can you point to an actual instance you know of where this has happened?  I
> don't mean that in an aggressive or challenging way, I'm sincerely interested
> after reading that rebuttal of Guttman's paper.  I've also always subscribed
> to the "complete destruction" idea.

I'm sorry, I can't point to any particular instance.  I know a lot of
people don't believe it and think it is all "black helicopter" stuff et al.
I am also not saying it's any one particular gov't TLA nor which
nation(s)'s intelligence organization.

> I don't mean recovery of data where someone accidentally issued a "del" or
> "rm" command and the file is pieced back together, or recovery of some data
> after filesystem corruption, etc.  I'm wondering if someone has truly
> recovered data from a drive where every single bit of data has been
> overwritten with zeroes/random data/whatever.
>
> Regards,
> Mark

diana

[Steve Shockley]

Eric Furman wrote:
> It can't be done. it's an urban legend, AFAICT.
> http://www.nber.org/sys-admin/overwritten-data-guttman.html
> Which references Gutmann's paper which started all this...

Of course I'm sure a tax analyst (http://www.nber.org/vitae/vita184.htm)
knows more about data recovery than a security researcher with a history
of researching overwritten-data-retrieval
(http://www.cs.auckland.ac.nz/~pgut001/).

[Ted Unangst-2]

On 1/3/08, new_guy <byte8bits@...> wrote:
> I'm working on putting a website up now where I'll fully disclose the
> details. Lots of pictures and details. I will attribute the dd used to
> OpenBSD (the best OS on the planet bar none... although the dd on the
> install CD did not support the conv option... I would have liked to have
> done conv=noerror,sync). I plan to ship the drive off tomorrow. I plan to
> put this myth to rest... where it belongs.

you are not proving that data cannot be recovered.  you are proving
that it cannot be recovered at a cost of $100.  if you have not spent
$1 million, you cannot claim that someone with $1 million cannot
recover the data.  that's just how things work.

but this has, as ever when it comes up, gone terribly far astray.

the first rule of data recovery:  in order for your data to be
recovered, there has to be someone willing to do (pay for) the
recovery.

the original question was about overwriting a file in such a manner
that the drive could still be used.  melting or shredding the drive
does not result in a usable drive.  if i never see another chucklehead
recommending "use thermite" it will be too soon.

overwriting the disk with /dev/zero or any other pattern does result
in a usable drive, but not a usable filesystem.  so now we're down to
just scorched earth, but we won't salt the fields.  again, not
helping.

so we come back to rm -P.  this comes pretty damn close.  it kills the
targeted file with no collateral damage.  it even does a reasonable
job of overwriting more than once, though not with the "guttman
superpattern", but that's a load of crap anyway.  but there are three
things rm -P does not delete.

it doesn't delete any temporary files created when the original was
being edited.
it doesn't delete any blocks or fragments that ffs may have rearranged
in a cluster op.
it doesn't delete any bad blocks that the disk itself moved around.
it also doesn't delete any data that's been posted on youtube, but
that's a whole nother issue.

the first is an application issue, but it can be very difficult to
control.  this even applies to the thermite people.  if /tmp is on a
different disk than /home, nuking one won't destroy all the data.

you have basically no control over what ffs does.  this also applies
if you have ever truncated the file down.  rm -P can only overwrite
the current file.

you can solve both these issues by writing a tool that overwrites all
unused disk blocks.  the code for fsck would be a good place to start
writing such a tool.  then you can run it periodically and know that
whatever free space is on your filesystem is clean.

as for the disk relocating blocks, there's nothing you can do
programatically.  by the same token, however, it's not so trivial to
recover and it depends on your secret data having been relocated.  for
most people, doing a cost/benefit risk analysis here should come up
somewhere short of vaporization.

the solution nobody ever comes up with but which is so totally obvious
is to prevent the data from being stored on the hard drive in the
first place.  holy cryptographic kryptonite batman.  if you encrypt
the data, you don't have to worry about somebody reading it even if
you don't delete it at all.

if you are giving away a hard drive and intend for the recipient to
use it, wiping it is the best you can do.  in most cases though, hard
drives are cheap, so you're not likely to give away a disk without
data.  instead, you want to dispose of the disk and data, permanently.
 in that case, a quick whack with a hammer to a control chip and a
connector and tossing into an anonymous dumpster is even faster than
wiping.