ZFS disk replacement questions

[Derrick Ryalls]

Hello list,

I plan on rebuilding my home fileserver next month with FreeBSD 8.0
x64 and will be using 4x 2TB drives in an external eSATA hotswap
enclosure using RAIDZ.  I have played around with FreeBSD in
VirtualBox just to see how easy it is to deal with ZFS, but a few
questions have come up for this configuration.

1) In the event of a disk failure, how do I trace back the name such
as adX to a physical drive in the enclosure?  Is there a way to take
the drive offline then use atacontrol to spin it down or something so
it is easy to identify?

2) In the event that I do experience a failure and manage to replace a
working disk instead of the failed disk, am I now completely hosed or
can I put the working disk back in and try pulling another drive which
is hopefully the bad drive?

3) Finally, does the extra load of resilvering the replaced drive
'age' the good drives causing them to fail quicker?


Any tips would be greatly appreciated.

Thanks.
_______________________________________________
freebsd-questions@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscribe@..."

[Steve Polyack]

Derrick Ryalls wrote:
> 1) In the event of a disk failure, how do I trace back the name such
> as adX to a physical drive in the enclosure?  Is there a way to take
> the drive offline then use atacontrol to spin it down or something so
> it is easy to identify?
>  
In my opinion you are best off using glabel(8) to give names to the
disks.  This way you can name them in a way that makes sense to you.  
Additionally, when you create the ZFS pool you will use the glabel'd
names.  This means that the pool will still come up properly if
something causes your devices to be numbered differently (i.e. a drive
dies and you happen to reboot the system).

Depending on your controller it is also probably worth it to use one of
the SATA-specific drivers in FreeBSD 8 - these are ones like ahci(4) and
siis(4).  While the generic ata(4) driver will work for pretty much
everything, the updated AHCI drivers can take advantage of some more
features.  Enable the modules at boot to use them.
> Any tips would be greatly appreciated.
>
>  


_______________________________________________
freebsd-questions@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscribe@..."

[Derrick Ryalls]

On Tue, Nov 3, 2009 at 10:21 AM, Steve Polyack <korvus@...> wrote:

> Derrick Ryalls wrote:
>>
>> 1) In the event of a disk failure, how do I trace back the name such
>> as adX to a physical drive in the enclosure?  Is there a way to take
>> the drive offline then use atacontrol to spin it down or something so
>> it is easy to identify?
>>
>
> In my opinion you are best off using glabel(8) to give names to the disks.
>  This way you can name them in a way that makes sense to you.  Additionally,
> when you create the ZFS pool you will use the glabel'd names.  This means
> that the pool will still come up properly if something causes your devices
> to be numbered differently (i.e. a drive dies and you happen to reboot the
> system).

I believe ZFS does this automatically.  Supposedly, if you take a
working set of RAIDZ drives from one machine and put it in another,
ZFS will figure out the drives since they get labelled by ZFS
internally.  My question concerns how to identify the physical disk in
question based on the adX or glabel name?  Different name in software
is fine, but if the drive fails I want to make sure I pull the correct
drive.

>
> Depending on your controller it is also probably worth it to use one of the
> SATA-specific drivers in FreeBSD 8 - these are ones like ahci(4) and
> siis(4).  While the generic ata(4) driver will work for pretty much
> everything, the updated AHCI drivers can take advantage of some more
> features.  Enable the modules at boot to use them.

I will look into it, thanks.  The machine in question is 2 year old
hardware currently with a 3ware raid card.  I will be going software
raid only, but FreeBSD already recognizes the eSATA drive I have
attached as a backup device so I know the O/S can at least talk to
sata drives attached to the mobo.

>>
>> Any tips would be greatly appreciated.
>>
>>
>
>
>
_______________________________________________
freebsd-questions@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscribe@..."

[Steve Polyack]

Derrick Ryalls wrote:

> On Tue, Nov 3, 2009 at 10:21 AM, Steve Polyack <korvus@...> wrote:
>  
>> Derrick Ryalls wrote:
>>    
>>> 1) In the event of a disk failure, how do I trace back the name such
>>> as adX to a physical drive in the enclosure?  Is there a way to take
>>> the drive offline then use atacontrol to spin it down or something so
>>> it is easy to identify?
>>>
>>>      
>> In my opinion you are best off using glabel(8) to give names to the disks.
>>  This way you can name them in a way that makes sense to you.  Additionally,
>> when you create the ZFS pool you will use the glabel'd names.  This means
>> that the pool will still come up properly if something causes your devices
>> to be numbered differently (i.e. a drive dies and you happen to reboot the
>> system).
>>    
>
> I believe ZFS does this automatically.  Supposedly, if you take a
> working set of RAIDZ drives from one machine and put it in another,
> ZFS will figure out the drives since they get labelled by ZFS
> internally.  My question concerns how to identify the physical disk in
> question based on the adX or glabel name?  Different name in software
> is fine, but if the drive fails I want to make sure I pull the correct
> drive.
>
>  

This is possible, but I don't remember reading that ZFS handles this
anywhere, and I've seen glabel(8) recommended elsewhere for the same
reason.

Either way, you can add your drives one-by-one and label them on the
enclosure "arraydrive00" and then glabel the individual disks with the
same name.  This way when ZFS tells you "arraydrive03" is dead/offline,
you can look at your enclosure and pull the drive with the arraydrive03
label.

>> Depending on your controller it is also probably worth it to use one of the
>> SATA-specific drivers in FreeBSD 8 - these are ones like ahci(4) and
>> siis(4).  While the generic ata(4) driver will work for pretty much
>> everything, the updated AHCI drivers can take advantage of some more
>> features.  Enable the modules at boot to use them.
>>    
>
> I will look into it, thanks.  The machine in question is 2 year old
> hardware currently with a 3ware raid card.  I will be going software
> raid only, but FreeBSD already recognizes the eSATA drive I have
> attached as a backup device so I know the O/S can at least talk to
> sata drives attached to the mobo.
>
>  

_______________________________________________
freebsd-questions@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscribe@..."

[krad-2]

2009/11/3 Steve Polyack <korvus@...>

> Derrick Ryalls wrote:
>
>> On Tue, Nov 3, 2009 at 10:21 AM, Steve Polyack <korvus@...>
>> wrote:
>>
>>
>>> Derrick Ryalls wrote:
>>>
>>>
>>>> 1) In the event of a disk failure, how do I trace back the name such
>>>> as adX to a physical drive in the enclosure?  Is there a way to take
>>>> the drive offline then use atacontrol to spin it down or something so
>>>> it is easy to identify?
>>>>
>>>>
>>>>
>>> In my opinion you are best off using glabel(8) to give names to the
>>> disks.
>>>  This way you can name them in a way that makes sense to you.
>>>  Additionally,
>>> when you create the ZFS pool you will use the glabel'd names.  This means
>>> that the pool will still come up properly if something causes your
>>> devices
>>> to be numbered differently (i.e. a drive dies and you happen to reboot
>>> the
>>> system).
>>>
>>>
>>
>> I believe ZFS does this automatically.  Supposedly, if you take a
>> working set of RAIDZ drives from one machine and put it in another,
>> ZFS will figure out the drives since they get labelled by ZFS
>> internally.  My question concerns how to identify the physical disk in
>> question based on the adX or glabel name?  Different name in software
>> is fine, but if the drive fails I want to make sure I pull the correct
>> drive.
>>
>>
>>
> This is possible, but I don't remember reading that ZFS handles this
> anywhere, and I've seen glabel(8) recommended elsewhere for the same reason.
>
> Either way, you can add your drives one-by-one and label them on the
> enclosure "arraydrive00" and then glabel the individual disks with the same
> name.  This way when ZFS tells you "arraydrive03" is dead/offline, you can
> look at your enclosure and pull the drive with the arraydrive03 label.
>
>  Depending on your controller it is also probably worth it to use one of
>>> the
>>> SATA-specific drivers in FreeBSD 8 - these are ones like ahci(4) and
>>> siis(4).  While the generic ata(4) driver will work for pretty much
>>> everything, the updated AHCI drivers can take advantage of some more
>>> features.  Enable the modules at boot to use them.
>>>
>>>
>>
>> I will look into it, thanks.  The machine in question is 2 year old
>> hardware currently with a 3ware raid card.  I will be going software
>> raid only, but FreeBSD already recognizes the eSATA drive I have
>> attached as a backup device so I know the O/S can at least talk to
>> sata drives attached to the mobo.
>>
>>
>>
>
> _______________________________________________
> freebsd-questions@... mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-questions
> To unsubscribe, send any mail to "
> freebsd-questions-unsubscribe@..."
>

One thing to note about resilvering; unlike most raid systems zfs knows what
is going on at the filesystem level as well as block level. Therefore when a
drive has to be resilvered, only the data on the drive is rebuilt rather
than every block as with most other raid subsystems. eg if you have a 1TB hd
but only have 20 Gig of data, only 20 gig is copied/rebuilt rather than 1 TB
of data if you were using gvinum/gmirror. This massively speeds up rebuild
times and stress on the other drives. However the fuller the drive the less
the benefits
_______________________________________________
freebsd-questions@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscribe@..."

[Graeme Dargie]

-----Original Message-----
From: krad [mailto:kraduk@...]
Sent: 04 November 2009 09:19
To: Steve Polyack
Cc: Derrick Ryalls; FreeBSD Questions
Subject: Re: ZFS disk replacement questions

2009/11/3 Steve Polyack <korvus@...>

> Derrick Ryalls wrote:
>
>> On Tue, Nov 3, 2009 at 10:21 AM, Steve Polyack <korvus@...>
>> wrote:
>>
>>
>>> Derrick Ryalls wrote:
>>>
>>>
>>>> 1) In the event of a disk failure, how do I trace back the name

such
>>>> as adX to a physical drive in the enclosure?  Is there a way to
take
>>>> the drive offline then use atacontrol to spin it down or something
so
>>>> it is easy to identify?
>>>>
>>>>
>>>>
>>> In my opinion you are best off using glabel(8) to give names to the
>>> disks.
>>>  This way you can name them in a way that makes sense to you.
>>>  Additionally,
>>> when you create the ZFS pool you will use the glabel'd names.  This
means
>>> that the pool will still come up properly if something causes your
>>> devices
>>> to be numbered differently (i.e. a drive dies and you happen to
reboot
>>> the
>>> system).
>>>
>>>
>>
>> I believe ZFS does this automatically.  Supposedly, if you take a
>> working set of RAIDZ drives from one machine and put it in another,
>> ZFS will figure out the drives since they get labelled by ZFS
>> internally.  My question concerns how to identify the physical disk
in
>> question based on the adX or glabel name?  Different name in software
>> is fine, but if the drive fails I want to make sure I pull the
correct
>> drive.
>>
>>
>>
> This is possible, but I don't remember reading that ZFS handles this
> anywhere, and I've seen glabel(8) recommended elsewhere for the same
reason.
>
> Either way, you can add your drives one-by-one and label them on the
> enclosure "arraydrive00" and then glabel the individual disks with the
same
> name.  This way when ZFS tells you "arraydrive03" is dead/offline, you
can
> look at your enclosure and pull the drive with the arraydrive03 label.
>
>  Depending on your controller it is also probably worth it to use one
of

>>> the
>>> SATA-specific drivers in FreeBSD 8 - these are ones like ahci(4) and
>>> siis(4).  While the generic ata(4) driver will work for pretty much
>>> everything, the updated AHCI drivers can take advantage of some more
>>> features.  Enable the modules at boot to use them.
>>>
>>>
>>
>> I will look into it, thanks.  The machine in question is 2 year old
>> hardware currently with a 3ware raid card.  I will be going software
>> raid only, but FreeBSD already recognizes the eSATA drive I have
>> attached as a backup device so I know the O/S can at least talk to
>> sata drives attached to the mobo.
>>
>>
>>
>
> _______________________________________________
> freebsd-questions@... mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-questions
> To unsubscribe, send any mail to "
> freebsd-questions-unsubscribe@..."
>

One thing to note about resilvering; unlike most raid systems zfs knows
what
is going on at the filesystem level as well as block level. Therefore
when a
drive has to be resilvered, only the data on the drive is rebuilt rather
than every block as with most other raid subsystems. eg if you have a
1TB hd
but only have 20 Gig of data, only 20 gig is copied/rebuilt rather than
1 TB
of data if you were using gvinum/gmirror. This massively speeds up
rebuild
times and stress on the other drives. However the fuller the drive the
less
the benefits
_______________________________________________
freebsd-questions@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to
"freebsd-questions-unsubscribe@..."


Hi All

Sorry to jump in on someone else's question / answer but I have a
related query. I notice the previous answer mentioned specific achi(4)
driver and Freebsd 8.0 are these available in 7.2 ? Will the achi(4)
driver work happily along side the ata driver. I just replaced every
drive in my raidz array the dirty way as I could not see away to make
the replacement drive show up without doing a reboot, would the achi(4)
driver allow me to hot swap the disks in the future ?

Regards

Graeme

_______________________________________________
freebsd-questions@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscribe@..."

[Mahlon E. Smith]

On Tue, Nov 03, 2009, Derrick Ryalls wrote:

> On Tue, Nov 3, 2009 at 10:21 AM, Steve Polyack <korvus@...> wrote:
> > Derrick Ryalls wrote:
> >>
> >> 1) In the event of a disk failure, how do I trace back the name such
> >> as adX to a physical drive in the enclosure?  Is there a way to take
> >> the drive offline then use atacontrol to spin it down or something so
> >> it is easy to identify?
> >>
> >
> > In my opinion you are best off using glabel(8) to give names to the disks.
> >  This way you can name them in a way that makes sense to you.  Additionally,
> > when you create the ZFS pool you will use the glabel'd names.  This means
> > that the pool will still come up properly if something causes your devices
> > to be numbered differently (i.e. a drive dies and you happen to reboot the
> > system).
>
> I believe ZFS does this automatically.  Supposedly, if you take a
> working set of RAIDZ drives from one machine and put it in another,
> ZFS will figure out the drives since they get labelled by ZFS
> internally.  My question concerns how to identify the physical disk in
> question based on the adX or glabel name?  Different name in software
> is fine, but if the drive fails I want to make sure I pull the correct
> drive.

This is only true if the metadata on the drives is re-read -- if your
pool loses a drive and the device numbers shuffle, your pool will be
FAILED on the next boot.

You can, however, force the metadata to be re-read via a 'zfs export
POOL', and a subsequent 'zfs import POOL'.

However, using glabel avoid that step entirely, as ZFS will always see
the 'right' devices in the right places, regardless of where they are
physically.

-Mahlon

--
Mahlon E. Smith  
http://www.martini.nu/contact.html