Improve recovery speed by chopping up the drive

View: New views
6 Messages — Rating Filter:   Alert me  

Improve recovery speed by chopping up the drive

by James Watts :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Hello all,

I am in the early stages of recovering deleted files from a 500GB external drive. So the task at hand is imaging the drive, then data carving all the lost files. Unfortunately, only about 20GB of the 500 was used. So I've got an easter egg hunt on my hands. I'm imaging the 500GB damaged external drive to a 1.5TB external drive. I've noticed that after writing the first 30-50GB, the data transmission rate drops from ~10MB/s to ~500KB/s (sometimes dipping below 100KB/s. Both drives are only a couple months old, and perfectly healthy.

I think the reason for the slow down has nothing to do with either drive. Instead, I think once the image file reaches such a large size, the transmission rate necessarily drops so the recovery drive can write the file and manage it. I have no proof, just a hunch. And I don't know the technical reasons either. So instead of trying to create a single gigantic 500GB image file, I'd like to create ten 50GB files. Since I'm doing data carving, I figured why not?

How do you do this in ddrescue?

I started with the following command:
sudo ddrescue -v -n /dev/sdb /media/recoverydrive/01.img /media/recoverydrive/01.log
and stopped the process once 01.img reached 50GB

Then, I used the following command:
sudo ddrescue -v -n -i 50GB /dev/sdb /media/recoverydrive/02.img /media/recoverydrive/02.log

This certainly seems to address the speed problem as I'm back up to ~10MB/s, however the 02.img file begins as a 50GB file, then increases from there. I expected it to start at size 0. What did I do wrong?

Again, I just want to chop-up the 500GB into more manageable chunks of 50GB. How would you do this?

Please advise,

James

_______________________________________________
Bug-ddrescue mailing list
Bug-ddrescue@...
http://lists.gnu.org/mailman/listinfo/bug-ddrescue

Re: Improve recovery speed by chopping up the drive

by Bugzilla from ant_diaz@teleline.es :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

James Watts wrote:
> I've noticed that after writing the first 30-50GB, the data transmission rate drops from ~10MB/s to ~500KB/s

What is the filesystem in your destination drive (/media/recoverydrive)?


> I'd like to create ten 50GB files. How do you do this in ddrescue?

sudo ddrescue -vn -i0 -o0 /dev/sdb /media/recoverydrive/01.img
/media/recoverydrive/01.log
sudo ddrescue -vn -i50GiB -o0 /dev/sdb /media/recoverydrive/02.img
/media/recoverydrive/02.log
sudo ddrescue -vn -i100GiB -o0 /dev/sdb /media/recoverydrive/03.img
/media/recoverydrive/03.log
etc...


Regards,
Antonio.


_______________________________________________
Bug-ddrescue mailing list
Bug-ddrescue@...
http://lists.gnu.org/mailman/listinfo/bug-ddrescue

Re: Improve recovery speed by chopping up the drive

by Bugzilla from ant_diaz@teleline.es :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

James Watts wrote:
> file, I'd like to create ten 50GB files. How do you do this in ddrescue?

Ooops! I forgot the size. Here are the correct commands:

sudo ddrescue -vn -i0 -o0 -s50GiB /dev/sdb /media/recoverydrive/01.img
/media/recoverydrive/01.log

sudo ddrescue -vn -i50GiB -o0 -s50GiB /dev/sdb
/media/recoverydrive/02.img /media/recoverydrive/02.log

sudo ddrescue -vn -i100GiB -o0 -s50GiB /dev/sdb
/media/recoverydrive/03.img /media/recoverydrive/03.log

etc...


Regards,
Antonio.


_______________________________________________
Bug-ddrescue mailing list
Bug-ddrescue@...
http://lists.gnu.org/mailman/listinfo/bug-ddrescue

Re: Improve recovery speed by chopping up the drive

by James Watts :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Thank you! I will try the commands you suggested.

Antonio Diaz Diaz <ant_diaz@...> wrote:
What is the filesystem in your destination drive (/media/recoverydrive)?

Both drives use a single, NTFS partition.

James


On Thu, Aug 13, 2009 at 3:44 AM, Antonio Diaz Diaz <ant_diaz@...> wrote:
James Watts wrote:
file, I'd like to create ten 50GB files. How do you do this in ddrescue?

Ooops! I forgot the size. Here are the correct commands:

sudo ddrescue -vn -i0 -o0 -s50GiB /dev/sdb /media/recoverydrive/01.img /media/recoverydrive/01.log

sudo ddrescue -vn -i50GiB -o0 -s50GiB /dev/sdb /media/recoverydrive/02.img /media/recoverydrive/02.log

sudo ddrescue -vn -i100GiB -o0 -s50GiB /dev/sdb /media/recoverydrive/03.img /media/recoverydrive/03.log

etc...


Regards,
Antonio.


_______________________________________________
Bug-ddrescue mailing list
Bug-ddrescue@...
http://lists.gnu.org/mailman/listinfo/bug-ddrescue

Re: Improve recovery speed by chopping up the drive

by Bugzilla from ant_diaz@teleline.es :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

James Watts wrote:
>> What is the filesystem in your destination drive (/media/recoverydrive)?
>
> Both drives use a single, NTFS partition.

NTFS is probably the reason of the slowness. See this message (NTFS-3g
eating 100%. Solved by switching to ext3)
http://lists.gnu.org/archive/html/bug-ddrescue/2009-06/msg00004.html


Regards,
Antonio.



_______________________________________________
Bug-ddrescue mailing list
Bug-ddrescue@...
http://lists.gnu.org/mailman/listinfo/bug-ddrescue

Re: Improve recovery speed by chopping up the drive

by James Watts :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Hello all,

To close the loop...

I will echo the results described in the message link Antonio sent below. Wow, what a difference the file system choice makes! Following advice from Antonio and another contributor to this buglist, I switched the recovery drive to EXT3 and imaged ~300 of 500GB overnight. The data rate averaged around 20MB/s. Amazing!

There is clearly some flaw in NTFS-3G. Using the same setup with both drives formatted as NTFS, the data rate started high (~20MB/s) and gradually decayed to under 100KB/s, never increased, and only imaged ~30 of 500GB overnight. Does anyone know if this issue has been reported to the NTFS-3G team? If not, I'd like to log a bug or at least report results to them.

Finally, I'd like to thank the ddrescue author and community for all the informative responses and suggestions. A true testament to the strength of open source community. Thank you very much for helping me speed things up!

Cheers,

James



On Thu, Aug 13, 2009 at 11:30 AM, Antonio Diaz Diaz <ant_diaz@...> wrote:
James Watts wrote:
What is the filesystem in your destination drive (/media/recoverydrive)?

Both drives use a single, NTFS partition.

NTFS is probably the reason of the slowness. See this message (NTFS-3g eating 100%. Solved by switching to ext3) http://lists.gnu.org/archive/html/bug-ddrescue/2009-06/msg00004.html


Regards,
Antonio.



_______________________________________________
Bug-ddrescue mailing list
Bug-ddrescue@...
http://lists.gnu.org/mailman/listinfo/bug-ddrescue