|
Remote NASsing Filesystem
I have an HTPC, which records terabytes to my RAID10 array. The data is too large to back up any more, so I've decided that I want to move one side of that mirror into the garage, in case of theft or fire. I plan to build a little headless box which will have a small mobo, four hot-swap disk drives, and GbEthernet, as my remote storage in the garage.
I'll need to communicate with it quickly and effectively, and so far it looks like iSCSI is the ticket, but FUSE is so amazing that I thought I'd check and see whether there are any solutions here. My main focus is to make this a RAID10 array (mirrored & striped), with one side of the mirror in the HTPC and the other in the garage attached by GbEthernet.
The only FUSE modules that look close are Gluster, Starfish, and Moose. Any suggestions?
Also I am looking for a *high*quality* enclosure and cage, similar to this preassembled (and expensive) one:
http://qnap.com/pro_detail_feature.asp?p_id=110Any suggestions?
------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference_______________________________________________
fuse-devel mailing list
fuse-devel@...
https://lists.sourceforge.net/lists/listinfo/fuse-devel
|
|
Re: Remote NASsing Filesystem
What you want is ZFS, possibly on Solaris but doable on Linux. You will
create one pool in the HTPC with your four disks inside that case there. Then
you will export the four disks in your garage using NBD or iSCSI, and import
these block devices in your HTPC. Then you will add each remote disk as a
mirror to the corresponding local disk in your pool.
this config
locald1 - remoted1 MIRROR
locald2 - remoted2 MIRROR
locald3 - remoted3 MIRROR
locald4 - remoted4 MIRROR
This is a zpool composed of four RAID1 disks, one in your house, one outside
it. You have redundancy, checksumming (VERY IMPORTANT), the ability to check
the entire pool with a scrub (which is proportional to the size of the data
you have) AND you can do backups by snapshotting.
:-)
See you on the ZFS-FUSE list!
El Jueves 08 Octubre 2009, adfas asd escribió:
> I have an HTPC, which records terabytes to my RAID10 array. The data is
> too large to back up any more, so I've decided that I want to move one side
> of that mirror into the garage, in case of theft or fire. I plan to build
> a little headless box which will have a small mobo, four hot-swap disk
> drives, and GbEthernet, as my remote storage in the garage.
>
> I'll need to communicate with it quickly and effectively, and so far it
> looks like iSCSI is the ticket, but FUSE is so amazing that I thought I'd
> check and see whether there are any solutions here. My main focus is to
> make this a RAID10 array (mirrored & striped), with one side of the mirror
> in the HTPC and the other in the garage attached by GbEthernet.
>
> The only FUSE modules that look close are Gluster, Starfish, and Moose.
> Any suggestions?
>
> Also I am looking for a *high*quality* enclosure and cage, similar to this
> preassembled (and expensive) one:
> http://qnap.com/pro_detail_feature.asp?p_id=110> Any suggestions?
>
>
>
>
> ---------------------------------------------------------------------------
>--- Come build with us! The BlackBerry(R) Developer Conference in SF, CA is
> the only developer event you need to attend this year. Jumpstart your
> developing skills, take BlackBerry mobile applications to market and stay
> ahead of the curve. Join us from November 9 - 12, 2009. Register now!
> http://p.sf.net/sfu/devconference> _______________________________________________
> fuse-devel mailing list
> fuse-devel@...
> https://lists.sourceforge.net/lists/listinfo/fuse-devel------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference_______________________________________________
fuse-devel mailing list
fuse-devel@...
https://lists.sourceforge.net/lists/listinfo/fuse-devel
|
|
Re: Remote NASsing Filesystem
What you want is ZFS, possibly on Solaris but doable on Linux. You will
create one pool in the HTPC with your four disks inside that case there. Then
you will export the four disks in your garage using NBD or iSCSI, and import
these block devices in your HTPC. Then you will add each remote disk as a
mirror to the corresponding local disk in your pool.
this config
locald1 - remoted1 MIRROR
locald2 - remoted2 MIRROR
locald3 - remoted3 MIRROR
locald4 - remoted4 MIRROR
This is a zpool composed of four RAID1 disks, one in your house, one outside
it. You have redundancy, checksumming (VERY IMPORTANT), the ability to check
the entire pool with a scrub (which is proportional to the size of the data
you have) AND you can do backups by snapshotting.
:-)
See you on the ZFS-FUSE list!
El Jueves 08 Octubre 2009, adfas asd escribió:
> I have an HTPC, which records terabytes to my RAID10 array. The data is
> too large to back up any more, so I've decided that I want to move one side
> of that mirror into the garage, in case of theft or fire. I plan to build
> a little headless box which will have a small mobo, four hot-swap disk
> drives, and GbEthernet, as my remote storage in the garage.
>
> I'll need to communicate with it quickly and effectively, and so far it
> looks like iSCSI is the ticket, but FUSE is so amazing that I thought I'd
> check and see whether there are any solutions here. My main focus is to
> make this a RAID10 array (mirrored & striped), with one side of the mirror
> in the HTPC and the other in the garage attached by GbEthernet.
>
> The only FUSE modules that look close are Gluster, Starfish, and Moose.
> Any suggestions?
>
> Also I am looking for a *high*quality* enclosure and cage, similar to this
> preassembled (and expensive) one:
> http://qnap.com/pro_detail_feature.asp?p_id=110> Any suggestions?
>
>
>
>
> ---------------------------------------------------------------------------
>--- Come build with us! The BlackBerry(R) Developer Conference in SF, CA is
> the only developer event you need to attend this year. Jumpstart your
> developing skills, take BlackBerry mobile applications to market and stay
> ahead of the curve. Join us from November 9 - 12, 2009. Register now!
> http://p.sf.net/sfu/devconference> _______________________________________________
> fuse-devel mailing list
> fuse-devel@...
> https://lists.sourceforge.net/lists/listinfo/fuse-devel------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference_______________________________________________
fuse-devel mailing list
fuse-devel@...
https://lists.sourceforge.net/lists/listinfo/fuse-devel
|
|
Re: Remote NASsing Filesystem
I wouldn't consider another OS, so it'd be Debian. I see I would need to compile the FUSE module.
What would this mean from an operational standpoint? Once the FUSE module is in place would ZFS be just like another filesystem, with mkzfs, etc? Would ZFS be available on boot, if I made say /home ZFS? Would I set up a regular boot drive, say ext3, then set up /home as raw disks with ZFS mirrored? Has anyone done performance testing, to compare it with other filesystems/ RAID paradigms?
So this is RAID1 through ZFS (as opposed to mdadm)? Is it proven? Is there a way to stripe -and- mirror, as with RAID10? Is it possible to RAID with only 2 drives, as it is with RAID10?
I see that development of the FUSE module is rather slow. Does it have enough ZFS features? Compression? Checksumming? Why is checksumming important when the drive does automatic bad block replacement?
I understand that BTRFS is the 2nd generation ZFS for Linux. Is that ready yet/ recommendable for use?
Which is more recommendable: iSCSI, NBD, ENBD, or DRBD?
--- On Thu, 10/8/09, Manuel Amador (Rudd-O) < rudd-o@...> wrote:
> What you want is ZFS, possibly on
> Solaris but doable on Linux. You will
> create one pool in the HTPC with your four disks inside
> that case there. Then
> you will export the four disks in your garage using NBD or
> iSCSI, and import
> these block devices in your HTPC. Then you will add
> each remote disk as a
> mirror to the corresponding local disk in your pool.
------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference_______________________________________________
fuse-devel mailing list
fuse-devel@...
https://lists.sourceforge.net/lists/listinfo/fuse-devel
|
|
Re: Remote NASsing Filesystem
ZFS talks to the kernel via FUSE (/dev/fuse). Every kernel already comes with
FUSE, so there is nothing to compile.
ZFS is feature complete and much, much better than BTRFS. There is
checksumming and it is important because drives FAIL, sometimes deliver bad
data, sometimes the cable sends bad data and corrupts data, sometimes sectors
go bad and they take the contained data with them (happens all the time).
End-to-end checksumming is the best thing because it lets you detect problems
and fix them. It also has compression.
El Viernes 09 Octubre 2009, adfas asd escribió:
> I wouldn't consider another OS, so it'd be Debian. I see I would need to
> compile the FUSE module.
>
> What would this mean from an operational standpoint? Once the FUSE module
> is in place would ZFS be just like another filesystem, with mkzfs, etc?
> Would ZFS be available on boot, if I made say /home ZFS? Would I set up a
> regular boot drive, say ext3, then set up /home as raw disks with ZFS
> mirrored? Has anyone done performance testing, to compare it with other
> filesystems/ RAID paradigms?
>
> So this is RAID1 through ZFS (as opposed to mdadm)? Is it proven? Is
> there a way to stripe -and- mirror, as with RAID10? Is it possible to
> RAID with only 2 drives, as it is with RAID10?
>
> I see that development of the FUSE module is rather slow. Does it have
> enough ZFS features? Compression? Checksumming? Why is checksumming
> important when the drive does automatic bad block replacement?
>
> I understand that BTRFS is the 2nd generation ZFS for Linux. Is that ready
> yet/ recommendable for use?
>
> Which is more recommendable: iSCSI, NBD, ENBD, or DRBD?
>
> --- On Thu, 10/8/09, Manuel Amador (Rudd-O) < rudd-o@...> wrote:
> > What you want is ZFS, possibly on
> > Solaris but doable on Linux. You will
> > create one pool in the HTPC with your four disks inside
> > that case there. Then
> > you will export the four disks in your garage using NBD or
> > iSCSI, and import
> > these block devices in your HTPC. Then you will add
> > each remote disk as a
> > mirror to the corresponding local disk in your pool.
>
> ---------------------------------------------------------------------------
> --- Come build with us! The BlackBerry(R) Developer Conference in SF, CA is
> the only developer event you need to attend this year. Jumpstart your
> developing skills, take BlackBerry mobile applications to market and stay
> ahead of the curve. Join us from November 9 - 12, 2009. Register now!
> http://p.sf.net/sfu/devconference> _______________________________________________
> fuse-devel mailing list
> fuse-devel@...
> https://lists.sourceforge.net/lists/listinfo/fuse-devel>
------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference_______________________________________________
fuse-devel mailing list
fuse-devel@...
https://lists.sourceforge.net/lists/listinfo/fuse-devel
|
|
Re: Remote NASsing Filesystem
That's not what my questions are about. I don't doubt the value of ZFS. I'm concerned about the qualities of the ZFS-FUSE module, its interaction with FUSE and the kernel, and whether there could be any instability or slowdown in this vital link of the chain. ...And my other questions below.
--- On Sun, 10/11/09, Manuel Amador (Rudd-O) < rudd-o@...> wrote:
> ZFS talks to the kernel via FUSE
> (/dev/fuse). Every kernel already comes with
> FUSE, so there is nothing to compile.
>
> ZFS is feature complete and much, much better than
> BTRFS. There is
> checksumming and it is important because drives FAIL,
> sometimes deliver bad
> data, sometimes the cable sends bad data and corrupts data,
> sometimes sectors
> go bad and they take the contained data with them (happens
> all the time).
> End-to-end checksumming is the best thing because it lets
> you detect problems
> and fix them. It also has compression.
>
> El Viernes 09 Octubre 2009, adfas asd escribió:
> > I wouldn't consider another OS, so it'd be
> Debian. I see I would need to
> > compile the FUSE module.
> >
> > What would this mean from an operational
> standpoint? Once the FUSE module
> > is in place would ZFS be just like another
> filesystem, with mkzfs, etc?
> > Would ZFS be available on boot, if I made say
> /home ZFS? Would I set up a
> > regular boot drive, say ext3, then set up /home
> as raw disks with ZFS
> > mirrored? Has anyone done performance
> testing, to compare it with other
> > filesystems/ RAID paradigms?
> >
> > So this is RAID1 through ZFS (as opposed to
> mdadm)? Is it proven? Is
> > there a way to stripe -and- mirror, as with
> RAID10? Is it possible to
> > RAID with only 2 drives, as it is with RAID10?
> >
> > I see that development of the FUSE module is rather
> slow. Does it have
> > enough ZFS features? Compression?
> Checksumming? Why is checksumming
> > important when the drive does automatic bad
> block replacement?
> >
> > I understand that BTRFS is the 2nd generation ZFS for
> Linux. Is that ready
> > yet/ recommendable for use?
> >
> > Which is more recommendable: iSCSI, NBD, ENBD, or
> DRBD?
> >
> > --- On Thu, 10/8/09, Manuel Amador (Rudd-O) < rudd-o@...>
> wrote:
> > > What you want is ZFS, possibly on
> > > Solaris but doable on Linux. You will
> > > create one pool in the HTPC with your four disks
> inside
> > > that case there. Then
> > > you will export the four disks in your garage
> using NBD or
> > > iSCSI, and import
> > > these block devices in your HTPC. Then you
> will add
> > > each remote disk as a
> > > mirror to the corresponding local disk in your
> pool.
> >
> >
> ---------------------------------------------------------------------------
> > --- Come build with us! The BlackBerry(R) Developer
> Conference in SF, CA is
> > the only developer event you need to attend this
> year. Jumpstart your
> > developing skills, take BlackBerry mobile
> applications to market and stay
> > ahead of the curve. Join us from November 9 -
> 12, 2009. Register now!
> > http://p.sf.net/sfu/devconference> > _______________________________________________
> > fuse-devel mailing list
> > fuse-devel@...
> > https://lists.sourceforge.net/lists/listinfo/fuse-devel> >
>
>
> -----Inline Attachment Follows-----
>
> ------------------------------------------------------------------------------
> Come build with us! The BlackBerry(R) Developer Conference
> in SF, CA
> is the only developer event you need to attend this year.
> Jumpstart your
> developing skills, take BlackBerry mobile applications to
> market and stay
> ahead of the curve. Join us from November 9 - 12, 2009.
> Register now!
> http://p.sf.net/sfu/devconference> -----Inline Attachment Follows-----
>
> _______________________________________________
> fuse-devel mailing list
> fuse-devel@...
> https://lists.sourceforge.net/lists/listinfo/fuse-devel>
------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference_______________________________________________
fuse-devel mailing list
fuse-devel@...
https://lists.sourceforge.net/lists/listinfo/fuse-devel
|
|
Re: Remote NASsing Filesystem
> I'm concerned about the qualities of the ZFS-FUSE module, its interaction
> with FUSE and the kernel, and whether there could be any instability or
> slowdown in this vital link of the chain. ...And my other questions
> below.
Sir, there is NO zfs-fuse module. ZFS is a USER SPACE PROGRAM. Read what I
wrote again.
>
> --- On Sun, 10/11/09, Manuel Amador (Rudd-O) < rudd-o@...> wrote:
> > ZFS talks to the kernel via FUSE
> > (/dev/fuse). Every kernel already comes with
> > FUSE, so there is nothing to compile.
> >
> > ZFS is feature complete and much, much better than
> > BTRFS. There is
> > checksumming and it is important because drives FAIL,
> > sometimes deliver bad
> > data, sometimes the cable sends bad data and corrupts data,
> > sometimes sectors
> > go bad and they take the contained data with them (happens
> > all the time).
> > End-to-end checksumming is the best thing because it lets
> > you detect problems
> > and fix them. It also has compression.
> >
> > El Viernes 09 Octubre 2009, adfas asd escribió:
> > > I wouldn't consider another OS, so it'd be
> >
> > Debian. I see I would need to
> >
> > > compile the FUSE module.
> > >
> > > What would this mean from an operational
> >
> > standpoint? Once the FUSE module
> >
> > > is in place would ZFS be just like another
> >
> > filesystem, with mkzfs, etc?
> >
> > > Would ZFS be available on boot, if I made say
> >
> > /home ZFS? Would I set up a
> >
> > > regular boot drive, say ext3, then set up /home
> >
> > as raw disks with ZFS
> >
> > > mirrored? Has anyone done performance
> >
> > testing, to compare it with other
> >
> > > filesystems/ RAID paradigms?
> > >
> > > So this is RAID1 through ZFS (as opposed to
> >
> > mdadm)? Is it proven? Is
> >
> > > there a way to stripe -and- mirror, as with
> >
> > RAID10? Is it possible to
> >
> > > RAID with only 2 drives, as it is with RAID10?
> > >
> > > I see that development of the FUSE module is rather
> >
> > slow. Does it have
> >
> > > enough ZFS features? Compression?
> >
> > Checksumming? Why is checksumming
> >
> > > important when the drive does automatic bad
> >
> > block replacement?
> >
> > > I understand that BTRFS is the 2nd generation ZFS for
> >
> > Linux. Is that ready
> >
> > > yet/ recommendable for use?
> > >
> > > Which is more recommendable: iSCSI, NBD, ENBD, or
> >
> > DRBD?
> >
> > > --- On Thu, 10/8/09, Manuel Amador (Rudd-O) < rudd-o@...>
> >
> > wrote:
> > > > What you want is ZFS, possibly on
> > > > Solaris but doable on Linux. You will
> > > > create one pool in the HTPC with your four disks
> >
> > inside
> >
> > > > that case there. Then
> > > > you will export the four disks in your garage
> >
> > using NBD or
> >
> > > > iSCSI, and import
> > > > these block devices in your HTPC. Then you
> >
> > will add
> >
> > > > each remote disk as a
> > > > mirror to the corresponding local disk in your
> >
> > pool.
> >
> >
> > -------------------------------------------------------------------------
> >--
> >
> > > --- Come build with us! The BlackBerry(R) Developer
> >
> > Conference in SF, CA is
> >
> > > the only developer event you need to attend this
> >
> > year. Jumpstart your
> >
> > > developing skills, take BlackBerry mobile
> >
> > applications to market and stay
> >
> > > ahead of the curve. Join us from November 9 -
> >
> > 12, 2009. Register now!
> >
> > > http://p.sf.net/sfu/devconference> > > _______________________________________________
> > > fuse-devel mailing list
> > > fuse-devel@...
> > > https://lists.sourceforge.net/lists/listinfo/fuse-devel> >
> > -----Inline Attachment Follows-----
> >
> > -------------------------------------------------------------------------
> >----- Come build with us! The BlackBerry(R) Developer Conference
> > in SF, CA
> > is the only developer event you need to attend this year.
> > Jumpstart your
> > developing skills, take BlackBerry mobile applications to
> > market and stay
> > ahead of the curve. Join us from November 9 - 12, 2009.
> > Register now!
> > http://p.sf.net/sfu/devconference> > -----Inline Attachment Follows-----
> >
> > _______________________________________________
> > fuse-devel mailing list
> > fuse-devel@...
> > https://lists.sourceforge.net/lists/listinfo/fuse-devel>
> ---------------------------------------------------------------------------
> --- Come build with us! The BlackBerry(R) Developer Conference in SF, CA is
> the only developer event you need to attend this year. Jumpstart your
> developing skills, take BlackBerry mobile applications to market and stay
> ahead of the curve. Join us from November 9 - 12, 2009. Register now!
> http://p.sf.net/sfu/devconference> _______________________________________________
> fuse-devel mailing list
> fuse-devel@...
> https://lists.sourceforge.net/lists/listinfo/fuse-devel>
------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference_______________________________________________
fuse-devel mailing list
fuse-devel@...
https://lists.sourceforge.net/lists/listinfo/fuse-devel
|
|
Re: Remote NASsing Filesystem
On Sun, Oct 11, 2009 at 10:22:35PM -0500, Manuel Amador (Rudd-O) wrote:
> ZFS talks to the kernel via FUSE (/dev/fuse). Every kernel already comes with
> FUSE, so there is nothing to compile.
>
> ZFS is feature complete and much, much better than BTRFS. There is
> checksumming and it is important because drives FAIL, sometimes deliver bad
> data, sometimes the cable sends bad data and corrupts data, sometimes sectors
> go bad and they take the contained data with them (happens all the time).
> End-to-end checksumming is the best thing because it lets you detect problems
> and fix them. It also has compression.
Both of which btrfs has. OTOH, btrfs can shrink its volumes, thing that
ZFS still lacks, and many other things. Oh, BTW, even people that did work
on zfs development say btrfs is better.
But that's OT on the fuse list.
Mike
------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference_______________________________________________
fuse-devel mailing list
fuse-devel@...
https://lists.sourceforge.net/lists/listinfo/fuse-devel
|
|
Re: Remote NASsing Filesystem
Oh. I didn't understand that.
So for all my prior questions, please replace the term 'module' with 'user-space program'.
--- On Sun, 10/11/09, Manuel Amador (Rudd-O) < rudd-o@...> wrote:
> Sir, there is NO zfs-fuse module. ZFS is a USER SPACE
> PROGRAM.
------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference_______________________________________________
fuse-devel mailing list
fuse-devel@...
https://lists.sourceforge.net/lists/listinfo/fuse-devel
|
|
Re: Remote NASsing Filesystem
I actually used BTRFS a couple months ago and depended on it for a month or so, when it destroyed all my data in a power fail. My data is videos recorded by MythTV, which are simply too large to back up.
I lost all confidence in it at that time, and I see now that they're still on 0.19. They are screaming that it's experimental, and you had better believe it.
--- On Mon, 10/12/09, Mike Hommey < mh@...> wrote:
> Both of which btrfs has. OTOH, btrfs can shrink its
> volumes, thing that
> ZFS still lacks, and many other things. Oh, BTW, even
> people that did work
> on zfs development say btrfs is better.
------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference_______________________________________________
fuse-devel mailing list
fuse-devel@...
https://lists.sourceforge.net/lists/listinfo/fuse-devel
|
|
Re: Remote NASsing Filesystem
On Mon, Oct 12, 2009 at 05:30:40AM -0700, adfas asd wrote:
> I actually used BTRFS a couple months ago and depended on it for a month or so, when it destroyed all my data in a power fail.
There is absolutely no filesystem that will prevent random data
destruction in a power fail. The best thing you can do to prevent them
from happening is to have an array controller with battery, that will
keep caches in such cases. Failing that, there is nothing else than
backups.
Mike
------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference_______________________________________________
fuse-devel mailing list
fuse-devel@...
https://lists.sourceforge.net/lists/listinfo/fuse-devel
|
|
Re: Remote NASsing Filesystem
On Mon, Oct 12, 2009 at 11:34 PM, Mike Hommey < mh@...> wrote:
> On Mon, Oct 12, 2009 at 05:30:40AM -0700, adfas asd wrote:
>> I actually used BTRFS a couple months ago and depended on it for a month or so, when it destroyed all my data in a power fail.
>
> There is absolutely no filesystem that will prevent random data
> destruction in a power fail. The best thing you can do to prevent them
> from happening is to have an array controller with battery, that will
> keep caches in such cases. Failing that, there is nothing else than
> backups.
There's a difference between a data destruction and a partial data loss.
In a presence of a disk controller that doesn't reorder writes, ZFS guarantees
that you data will always be consistent, but not always up-to-date.
Thanks,
Roman.
P.S. Full disclaimer: I used to work for Sun.
------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference_______________________________________________
fuse-devel mailing list
fuse-devel@...
https://lists.sourceforge.net/lists/listinfo/fuse-devel
|
|
Re: Remote NASsing Filesystem
On 13/10/09 07:55, Roman Shaposhnik wrote:
> On Mon, Oct 12, 2009 at 11:34 PM, Mike Hommey< mh@...> wrote:
>
>> On Mon, Oct 12, 2009 at 05:30:40AM -0700, adfas asd wrote:
>>
>>> I actually used BTRFS a couple months ago and depended on it for a month or so, when it destroyed all my data in a power fail.
>>>
>> There is absolutely no filesystem that will prevent random data
>> destruction in a power fail. The best thing you can do to prevent them
>> from happening is to have an array controller with battery, that will
>> keep caches in such cases. Failing that, there is nothing else than
>> backups.
>>
> There's a difference between a data destruction and a partial data loss.
> In a presence of a disk controller that doesn't reorder writes, ZFS guarantees
> that you data will always be consistent, but not always up-to-date.
>
Neither of those statements are strictly true. There are filesystems
that guarantee that the data is in secure storage at certain
well-defined times. Making those filesystems fast enough to be usable
is another matter entirely.
The ZFS guarantees require a barrier: "at this well-defined point the
disk subsystem will have put data somewhere secure". Mostly, disk
controllers will honour that requirement, but if, for example, you're
using LVM for your underlying storage then you've lost the barrier.
Yes, I know, you shouldn't be using LVM and ZFS, it's just an illustration.
Btrfs has designed into it all those safeguards that ZFS has: checksums
all over the place, storing data in more than one place, consistency
guarantees, proper barriers, etc etc. Notice that these things are
_designed_ in, they don't necessarily have the bugs worked out yet. ZFS
is a mature product, it's been well-used and well-tested for several
years; btrfs is relatively new and it's marked "experimental" because
it's still not up to production quality. Even Fedora, which is not
averse to experimental technology, doesn't enable btrfs unless you
invoke the right magic word.
I do worry when I see people who say that they depend on btrfs -- you
really shouldn't be doing that yet. If you want to use btrfs then make
sure that you don't mind losing data if things go badly wrong. That
either means making sure you have good backups or that you only use it
for ephemeral data -- stuff you don't mind losing.
Btrfs has a lot going for it, probably more that zfs -- don't write it
off just yet.
I work for Oracle but I have no vested interest in btrfs or zfs and, of
course, these opinions are mine, not the company's (I'm not paid enough
for that).
jch
------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference_______________________________________________
fuse-devel mailing list
fuse-devel@...
https://lists.sourceforge.net/lists/listinfo/fuse-devel
|
|
Re: Remote NASsing Filesystem
On Tue, Oct 13, 2009 at 09:00:22AM +0100, John Haxby wrote:
> On 13/10/09 07:55, Roman Shaposhnik wrote:
> > On Mon, Oct 12, 2009 at 11:34 PM, Mike Hommey< mh@...> wrote:
> >
> >> On Mon, Oct 12, 2009 at 05:30:40AM -0700, adfas asd wrote:
> >>
> >>> I actually used BTRFS a couple months ago and depended on it for a month or so, when it destroyed all my data in a power fail.
> >>>
> >> There is absolutely no filesystem that will prevent random data
> >> destruction in a power fail. The best thing you can do to prevent them
> >> from happening is to have an array controller with battery, that will
> >> keep caches in such cases. Failing that, there is nothing else than
> >> backups.
> >>
> > There's a difference between a data destruction and a partial data loss.
> > In a presence of a disk controller that doesn't reorder writes, ZFS guarantees
> > that you data will always be consistent, but not always up-to-date.
> >
>
> Neither of those statements are strictly true. There are filesystems
> that guarantee that the data is in secure storage at certain
> well-defined times.
My point was: the are no such guarantees in all cases, especially on
low-end hardware, which I guess is what "adfas asd" was using when he
lost data to btrfs. My point is that his loss in power fail would have
most probably happened with any other fs, except if he has the right
hardware.
Mike
------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference_______________________________________________
fuse-devel mailing list
fuse-devel@...
https://lists.sourceforge.net/lists/listinfo/fuse-devel
|
|
Re: Remote NASsing Filesystem
El Martes 13 Octubre 2009, Mike Hommey escribió:
> On Mon, Oct 12, 2009 at 05:30:40AM -0700, adfas asd wrote:
> > I actually used BTRFS a couple months ago and depended on it for a month
> > or so, when it destroyed all my data in a power fail.
>
> There is absolutely no filesystem that will prevent random data
> destruction in a power fail. The best thing you can do to prevent them
> from happening is to have an array controller with battery, that will
> keep caches in such cases. Failing that, there is nothing else than
> backups.
Sorry for being adamant, but that is NONSENSE. ZFS has been explicitly
engineered to survive power failures, and has withstood over one million
forced crashes.
On ZFS, even if you lose caches due to power outages and the whole transaction
write does not make it or makes it inconsistently across disk vdevs, the use
of barriers on modern SATA drives will ensure that the transaction will be
committed completely, or won't be committed at all. The data loss will
absolutely NOT BE RANDOM, but will be rather limited to the very last
transaction if at all. And even if one of your devices in your file system
gets completely TRASHED, ZFS can scrub and repair data on the failed device by
using checksummed and verified data from the other device, AND letting you
know which files went inaccessible. I know all of this from experience.
Really, this mindset of "battery backed RAID arrays"... it's so nineties that
Vanilla Ice would like it back. Don't spread FUD.
>
> Mike
>
> ---------------------------------------------------------------------------
> --- Come build with us! The BlackBerry(R) Developer Conference in SF, CA is
> the only developer event you need to attend this year. Jumpstart your
> developing skills, take BlackBerry mobile applications to market and stay
> ahead of the curve. Join us from November 9 - 12, 2009. Register now!
> http://p.sf.net/sfu/devconference> _______________________________________________
> fuse-devel mailing list
> fuse-devel@...
> https://lists.sourceforge.net/lists/listinfo/fuse-devel>
------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference_______________________________________________
fuse-devel mailing list
fuse-devel@...
https://lists.sourceforge.net/lists/listinfo/fuse-devel
|
|
Re: Remote NASsing Filesystem
Sorry again.
> > There's a difference between a data destruction and a partial data loss.
> > In a presence of a disk controller that doesn't reorder writes, ZFS
> > guarantees that you data will always be consistent, but not always
> > up-to-date.
>
> Neither of those statements are strictly true. There are filesystems
> that guarantee that the data is in secure storage at certain
> well-defined times. Making those filesystems fast enough to be usable
> is another matter entirely.
>
But the statement your interlocutor said is strictly true to my understanding.
You just reworded it -- perhaps in an attempt to evade conceding a very
obvious point that he was making. And ZFS gives you platter speed.
> The ZFS guarantees require a barrier: "at this well-defined point the
> disk subsystem will have put data somewhere secure". Mostly, disk
> controllers will honour that requirement, but if, for example, you're
> using LVM for your underlying storage then you've lost the barrier.
> Yes, I know, you shouldn't be using LVM and ZFS, it's just an illustration.
If you are using ZFS and LVM, you deserve what you get. ZFS is designed to
make do away with LVM-type solutions.
> Btrfs has designed into it all those safeguards that ZFS has: checksums
> all over the place, storing data in more than one place, consistency
> guarantees, proper barriers, etc etc.
BTRFS, to the extent of my knowledge, does not implement ditto blocks.
Important blocks in the file system are not duplicated, unlike ZFS that indeed
does duplicate important blocks.
> Btrfs has a lot going for it, probably more that zfs -- don't write it
> off just yet.
OK then, let's wait two or three years and we'll use ZFS in the meantime?
------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference_______________________________________________
fuse-devel mailing list
fuse-devel@...
https://lists.sourceforge.net/lists/listinfo/fuse-devel
|
|
Re: Remote NASsing Filesystem
> My point was: the are no such guarantees in all cases, especially on
> low-end hardware, which I guess is what "adfas asd" was using when he
> lost data to btrfs. My point is that his loss in power fail would have
> most probably happened with any other fs, except if he has the right
> hardware.
This is JUST NOT TRUE. Every modern SATA disk has write barriers that prevent
write reordering (this is the *cornerstone* of transactional commit to disk),
and ZFS does indeed make use of those write barriers. The "it would have most
probably happened with any other FS", I can understand from people running
ext2 or FAT32 filesystems -- not from those running ZFS.
Go to our list and ask who has experienced catastrophic data loss with ZFS.
We have had our share of bugs, impediments, faults, but none that would cause
data loss so far.
>
> Mike
>
> ---------------------------------------------------------------------------
> --- Come build with us! The BlackBerry(R) Developer Conference in SF, CA is
> the only developer event you need to attend this year. Jumpstart your
> developing skills, take BlackBerry mobile applications to market and stay
> ahead of the curve. Join us from November 9 - 12, 2009. Register now!
> http://p.sf.net/sfu/devconference> _______________________________________________
> fuse-devel mailing list
> fuse-devel@...
> https://lists.sourceforge.net/lists/listinfo/fuse-devel>
------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference_______________________________________________
fuse-devel mailing list
fuse-devel@...
https://lists.sourceforge.net/lists/listinfo/fuse-devel
|
|
Re: Remote NASsing Filesystem
Manuel,
I just want to make sure. Are you talking about ZFS in Solaris or zfs-fuse ?
Do zfs-fuse implement barriers ?
Is it possible to implement barriers on a fuse filesystem ?
If it is, how would one do that ?
Regards
Paul
Manuel Amador (Rudd-O) wrote:
>> My point was: the are no such guarantees in all cases, especially on
>> low-end hardware, which I guess is what "adfas asd" was using when he
>> lost data to btrfs. My point is that his loss in power fail would have
>> most probably happened with any other fs, except if he has the right
>> hardware.
>>
>
> This is JUST NOT TRUE. Every modern SATA disk has write barriers that prevent
> write reordering (this is the *cornerstone* of transactional commit to disk),
> and ZFS does indeed make use of those write barriers. The "it would have most
> probably happened with any other FS", I can understand from people running
> ext2 or FAT32 filesystems -- not from those running ZFS.
>
> Go to our list and ask who has experienced catastrophic data loss with ZFS.
> We have had our share of bugs, impediments, faults, but none that would cause
> data loss so far.
>
>
>> Mike
>>
>> ---------------------------------------------------------------------------
>> --- Come build with us! The BlackBerry(R) Developer Conference in SF, CA is
>> the only developer event you need to attend this year. Jumpstart your
>> developing skills, take BlackBerry mobile applications to market and stay
>> ahead of the curve. Join us from November 9 - 12, 2009. Register now!
>> http://p.sf.net/sfu/devconference>> _______________________________________________
>> fuse-devel mailing list
>> fuse-devel@...
>> https://lists.sourceforge.net/lists/listinfo/fuse-devel>>
>>
>
>
> ------------------------------------------------------------------------
>
> ------------------------------------------------------------------------------
> Come build with us! The BlackBerry(R) Developer Conference in SF, CA
> is the only developer event you need to attend this year. Jumpstart your
> developing skills, take BlackBerry mobile applications to market and stay
> ahead of the curve. Join us from November 9 - 12, 2009. Register now!
> http://p.sf.net/sfu/devconference> ------------------------------------------------------------------------
>
> _______________________________________________
> fuse-devel mailing list
> fuse-devel@...
> https://lists.sourceforge.net/lists/listinfo/fuse-devel>
------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference_______________________________________________
fuse-devel mailing list
fuse-devel@...
https://lists.sourceforge.net/lists/listinfo/fuse-devel
|
|
Re: Remote NASsing Filesystem
On Tue, Oct 13, 2009 at 05:28:24AM -0500, Manuel Amador (Rudd-O) wrote:
> El Martes 13 Octubre 2009, Mike Hommey escribió:
> > On Mon, Oct 12, 2009 at 05:30:40AM -0700, adfas asd wrote:
> > > I actually used BTRFS a couple months ago and depended on it for a month
> > > or so, when it destroyed all my data in a power fail.
> >
> > There is absolutely no filesystem that will prevent random data
> > destruction in a power fail. The best thing you can do to prevent them
> > from happening is to have an array controller with battery, that will
> > keep caches in such cases. Failing that, there is nothing else than
> > backups.
>
> Sorry for being adamant, but that is NONSENSE. ZFS has been explicitly
> engineered to survive power failures, and has withstood over one million
> forced crashes.
>
> On ZFS, even if you lose caches due to power outages and the whole transaction
> write does not make it or makes it inconsistently across disk vdevs, the use
> of barriers on modern SATA drives will ensure that the transaction will be
> committed completely, or won't be committed at all. The data loss will
> absolutely NOT BE RANDOM, but will be rather limited to the very last
> transaction if at all. And even if one of your devices in your file system
> gets completely TRASHED, ZFS can scrub and repair data on the failed device by
> using checksummed and verified data from the other device, AND letting you
> know which files went inaccessible. I know all of this from experience.
>
> Really, this mindset of "battery backed RAID arrays"... it's so nineties that
> Vanilla Ice would like it back. Don't spread FUD.
You're not contradicting my thoughts, we're just not on the same tracks.
What I meant to say is that while filesystems can guarantee levels of
consistency, they won't *ever* guarantee that *everything* you write has
been written and won't ever be lost after a power failure. And in some
cases, "the very last transation" can be very important data, too. And
that will be lost, unless the hardware prevents that by some means. And
depending on when the power loss happens, and how the data has been put
physically to disk or not, you don't know in advance what is going to
have been destroyed. And there are various ways to lose write barriers
(like using lvm, as has been reminded in this thread).
That said, some filesystems help prevent some disasters, but they can't
do miracles either.
Mike
------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference_______________________________________________
fuse-devel mailing list
fuse-devel@...
https://lists.sourceforge.net/lists/listinfo/fuse-devel
|
|
Re: Remote NASsing Filesystem
On 13/10/09 11:32, Manuel Amador (Rudd-O) wrote:
>
>> The ZFS guarantees require a barrier: "at this well-defined point the
>> disk subsystem will have put data somewhere secure". Mostly, disk
>> controllers will honour that requirement, but if, for example, you're
>> using LVM for your underlying storage then you've lost the barrier.
>> Yes, I know, you shouldn't be using LVM and ZFS, it's just an illustration.
>>
> If you are using ZFS and LVM, you deserve what you get. ZFS is designed to
> make do away with LVM-type solutions.
>
>
I did say it was an illustration. I wouldn't be in the least surprised
if one of the cheap NAS boxes that also offers iSCSI doesn't honour
barriers. Or any other cheap hardware for that matter. And expensive
hardware failing that way would only raise my eyebrows.
>> Btrfs has designed into it all those safeguards that ZFS has: checksums
>> all over the place, storing data in more than one place, consistency
>> guarantees, proper barriers, etc etc.
>>
> BTRFS, to the extent of my knowledge, does not implement ditto blocks.
> Important blocks in the file system are not duplicated, unlike ZFS that indeed
> does duplicate important blocks.
>
>
Yes, it does. It doesn't call them ditto blocks but btrfs will
duplicate blocks. Indeed, the per-block checksums allow it to silently
recover from bad disk blocks (and dead disks): you just fetch the duplicate.
>> Btrfs has a lot going for it, probably more that zfs -- don't write it
>> off just yet.
>>
> OK then, let's wait two or three years and we'll use ZFS in the meantime?
>
You probably should. Btrfs is very new; the reason is made it into the
mainline kernel so early on is that it holds considerable promise and
the more exposure it gets from people willing to test it and fix bugs
the quicker it will become stable enough for the enterprise.
There is one insuperable problem with zfs, and the reason that zfs-fuse
exists in the first place: CDDL is incompatible with the GPL. It's a
shame, but without that limitation this entire thread probably wouldn't
exist. In some ways I'm glad that limitation does exist though: it
seems to me that btrfs has learned from all its predecessors, zfs and
hardware technology changes included, and will wind up being much, much
better as a result. Yes, being able to shrink a volume is _that_
important.
jch
------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference_______________________________________________
fuse-devel mailing list
fuse-devel@...
https://lists.sourceforge.net/lists/listinfo/fuse-devel
|
signature.asc (197 bytes) 