Re: HSM/ILM for Netapp

Babar> IHAC who is searching for a product that can provide HSM/ILM
Babar> function for massive NAS environment.

It's not an easy solution space at all.  Esp once you start thinking
about Backups and restores.  We've been down this route with a couple
of vendors/solutions and found limitations in all of them.

In our case, we're purely interested in NFS clients and servers.
CIFS, iSCSI is a minor part of our operation.

Babar> This product can be either a hardware appliance, a software, or
Babar> combination of both that can provide a total transparent
Babar> solution for our user (i.e. regardless where is the physical
Babar> data location, user can remain using it old logical path to
Babar> access the same data) with tiered storage.

These are going to be *tough* to reconcile together.  Having a single
mount point on client systems, which can have data spread across
multiple backend systems and be dynamically moved around isn't simple
to accomplish.

Babar> So far I can only identify two products one hardware Acopia and
Babar> the other one is software approach CommVault.  Both have it
Babar> pros and cons but the biggest challenges for both products are
Babar> not able to manage snapshot and do not go down to qtree level.

We've used both.  Acopia is a *neat* idea, but the problem with it is
backups.

You don't want to backup through the acopia, since it's a big
bottleneck, so you backup the Filers directly.  But then you need to
manage and track *where* your files really are stored, and that
becomes a nightmare to deal with.  

We also ran into some problems with Acopia's and filesystems with
large numbers of files (on the order of 10+ million) but those bugs
were fixed relatively quickly, and we haven't had major problems since
then.

The other issue is .snapshot/ dirs, since those are just so convenient
for user's to use and access to do their own restores.  We ended up
exporting snapshots to another mount path user's could access and
giving them direction on how to access snapshots via the alternative
path.  Not very ideal, and yet another thing to manually manage.

We've now also migrated to CommVault as our backup software, partly
because Legato was expensive to bring current, esp with NDMP
licensing, etc.  We also have been intrigued by the integrated HSM
features of CommVault as well.  

Note, that CV requires you have CIFS licenses, and a dedicated Windows
box (MediaAgent) which handles all the scanning of the
filesystem(s) for file(s) to migrate from one tier to another.  So if
you're an NFS shop, you'll find that you now need CIFS licenses as
well from Netapp, which can be a hidden gotcha if you're not careful.

In our preliminary testing, the HSM aspect has worked pretty well.  We
can stage files to disk/tape, they get recalled automatically and life
is good.  We can even do a backup of the stub file, move it to another
filer/volume, and have access just work.  We're still in the initial
deployment phase, but we're planning on rolling this out to all our
sites.  

This is all using CV 7.0, they now have 8.0 out and we might upgrade
to that before we roll out HSM.  But it can be tricky.  

For us, the integration of HSM and backups is the *key* thing.  Having
a single mountpoint for user data, which doens't change isn't as
important in the grand scheme of things.  So the Acopia handles the
transparent migration of data between backend storage nicely, but
impacts backups and .snapshot access.  Using CV, we get integrated
backups and HSM and regular .snapshots, but not transparent shuffling
of data.  

To me, the big issue I want to see addressed is the size of NetApp
Aggregates.  16Tb aggregates are *stupid* esp since they have Raid
Groups.  

Some way to span aggregates with volumes, or move volumes live between
aggregates would be a godsend, but just bumping the size to 32Tb would
be a win too.  

Hope this helps.
John
    John Stoffel - Senior Staff Systems Administrator - System LSI Group
  Toshiba America Electronic Components, Inc. - http://www.toshiba.com/taec
             john.stoffel@... - 508-486-1087