Anyone using Amazon S3 for backup?

View: New views
18 Messages — Rating Filter:   Alert me  

Anyone using Amazon S3 for backup?

by Cliff Hirsch :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Anyone using Amazon S3 for backup? Is anyone using Amazon S3 for backup? Instead of getting another independent server, setting up rsync, etc. this seems like a good solution for backing up a server or sql database. Sure, my ISP does daily backups, but having more control by backing up the database off-site would make me sleep better.

Anyone have any experience? Good, bad, suggestions, ideas? Any other alternatives worth looking at?

Cliff

_______________________________________________
New York PHP Community Talk Mailing List
http://lists.nyphp.org/mailman/listinfo/talk

NYPHPCon 2006 Presentations Online
http://www.nyphpcon.com

Show Your Participation in New York PHP
http://www.nyphp.org/show_participation.php

Re: Anyone using Amazon S3 for backup?

by Steve Manes :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Cliff Hirsch wrote:
> Is anyone using Amazon S3 for backup? Instead of getting another
> independent server, setting up rsync, etc. this seems like a good
> solution for backing up a server or sql database. Sure, my ISP does
> daily backups, but having more control by backing up the database
> off-site would make me sleep better.
>
> Anyone have any experience? Good, bad, suggestions, ideas? Any other
> alternatives worth looking at?

It's not an rsync solution (although there are some like
http://s3sync.net/wiki) but a friend of mine who owns a consulting
company employing several telecommuters is using S3 via a $20 piece of
software called Jungle Disk (http://jungledisk.com/) to back up his
laptops in the field.  He's been very happy with it.  I'm trying to get
one of my clients to consider it for his computers in the wild.
_______________________________________________
New York PHP Community Talk Mailing List
http://lists.nyphp.org/mailman/listinfo/talk

NYPHPCon 2006 Presentations Online
http://www.nyphpcon.com

Show Your Participation in New York PHP
http://www.nyphp.org/show_participation.php

RE: Anyone using Amazon S3 for backup?

by Hans Zaunere :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

> Cliff Hirsch wrote:
> > Is anyone using Amazon S3 for backup? Instead of getting another
> > independent server, setting up rsync, etc. this seems like a good
> > solution for backing up a server or sql database. Sure, my ISP does
> > daily backups, but having more control by backing up the database
> > off-site would make me sleep better.

For servers:

rsync and www.raidarray.net


For laptops:

http://www.2brightsparks.com/syncback/sbse.html

(and then FTP/SMB/rsync/etc)

As far as my opinion on S3 - interesting, but not where I want to put my
most valuable data.

H


_______________________________________________
New York PHP Community Talk Mailing List
http://lists.nyphp.org/mailman/listinfo/talk

NYPHPCon 2006 Presentations Online
http://www.nyphpcon.com

Show Your Participation in New York PHP
http://www.nyphp.org/show_participation.php

Re: Anyone using Amazon S3 for backup?

by Cliff Hirsch :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

> For servers:
> www.raidarray.net

This looks very interesting. Surprising the the std. service is ftp, which
is not secure, instead of sftp. rsync over ssh looks good though.

> As far as my opinion on S3 - interesting, but not where I want to put my
> most valuable data.

Why? Because of security or reliability concerns?


_______________________________________________
New York PHP Community Talk Mailing List
http://lists.nyphp.org/mailman/listinfo/talk

NYPHPCon 2006 Presentations Online
http://www.nyphpcon.com

Show Your Participation in New York PHP
http://www.nyphp.org/show_participation.php

RE: Anyone using Amazon S3 for backup?

by Hans Zaunere :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

 
> > For servers:
> > www.raidarray.net
>
> This looks very interesting. Surprising the the std. service is ftp, which
> is not secure, instead of sftp. rsync over ssh looks good though.

Yeah, they're great.  I of course have sftp/smb/rsync enabled and only allow
FTP access when through my VPN.  Their service is well firewalled, so you
have to request holes, which they're happy to customize.

> > As far as my opinion on S3 - interesting, but not where I want to put my
> > most valuable data.
>
> Why? Because of security or reliability concerns?

Yeah, both.  I wasn't impressed with the SLA, and the access methods don't
appeal to me.  It feels a lot more like a CDN, than a secure remote storage
solution - then again, maybe I'm just old fashioned.

H


_______________________________________________
New York PHP Community Talk Mailing List
http://lists.nyphp.org/mailman/listinfo/talk

NYPHPCon 2006 Presentations Online
http://www.nyphpcon.com

Show Your Participation in New York PHP
http://www.nyphp.org/show_participation.php

Re: Anyone using Amazon S3 for backup?

by Chris Snyder-6 :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

On Jan 18, 2008 1:35 PM, Cliff Hirsch <cliff@...> wrote:
>
>  Is anyone using Amazon S3 for backup? Instead of getting another
> independent server, setting up rsync, etc. this seems like a good solution
> for backing up a server or sql database. Sure, my ISP does daily backups,
> but having more control by backing up the database off-site would make me
> sleep better.
>

I keep wanting to do this, but several things are holding me back:

1) How redundant/safe is S3, really?

I trust Amazon more than some other large providers, but S3 is a
high-value target and the SLA is for service uptime only. It doesn't
mention the possibility of data loss, corruption, or breach of
privacy... why not?

2) Lack of S3 support in existing tools/kernels

The weird http interface and naming conventions have made it difficult
to use S3 as general-purpose storage. I find it odd that Amazon's
engineers haven't developed a FUSE plugin for the service. There have
been efforts in this direction, but Jungle Disk seems to be the only
one that is decently organized and production-ready.

3) Physical storage gets cheaper while S3 remains the same

...and the monthly cost of storing the data is going to grow as your
backup archive gets bigger. The main reason to use S3 is that you get
(hopefully) redundant, (hopefully) geographically diverse storage
without having to pay for a server, rack space, and sysadmin costs. It
will be interesting to see if the price of S3 keeps pace with Moore's
law.

--
Chris Snyder
http://chxo.com/
_______________________________________________
New York PHP Community Talk Mailing List
http://lists.nyphp.org/mailman/listinfo/talk

NYPHPCon 2006 Presentations Online
http://www.nyphpcon.com

Show Your Participation in New York PHP
http://www.nyphp.org/show_participation.php

Re: Anyone using Amazon S3 for backup?

by Adrian Noland-2 :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message



On 1/18/08, Cliff Hirsch <cliff@...> wrote:
Is anyone using Amazon S3 for backup? Instead of getting another independent server, setting up rsync, etc. this seems like a good solution for backing up a server or sql database. Sure, my ISP does daily backups, but having more control by backing up the database off-site would make me sleep better.

Anyone have any experience? Good, bad, suggestions, ideas? Any other alternatives worth looking at?

My big problem is that it costs more to get data out than to put it in.

http://www.amazon.com/S3-AWS-home-page-Money/b?ie=UTF8&node=16427261
Storage
$0.15 per GB-Month of storage used

Data Transfer
$0.10 per GB - all data transfer in

$0.18 per GB - first 10 TB / month data transfer out
$0.16 per GB - next 40 TB / month data transfer out
$0.13 per GB - data transfer out / month over 50 TB

Not a scenario I want to be dealing with during a stressful situation.

Take a look at an actual backup provider such as raidarray.net or rsync.net


_______________________________________________
New York PHP Community Talk Mailing List
http://lists.nyphp.org/mailman/listinfo/talk

NYPHPCon 2006 Presentations Online
http://www.nyphpcon.com

Show Your Participation in New York PHP
http://www.nyphp.org/show_participation.php

Re: Anyone using Amazon S3 for backup?

by Rob Marscher :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

On Jan 18, 2008, at 1:35 PM, Cliff Hirsch wrote:
Is anyone using Amazon S3 for backup?

Here's an oft-referenced article by Jeremy Zawodny:

I personally have only used S3 for serving/storing media and haven't done so on a very large scale yet.


_______________________________________________
New York PHP Community Talk Mailing List
http://lists.nyphp.org/mailman/listinfo/talk

NYPHPCon 2006 Presentations Online
http://www.nyphpcon.com

Show Your Participation in New York PHP
http://www.nyphp.org/show_participation.php

Re: Anyone using Amazon S3 for backup?

by David Mintz-3 :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

I use it to back up my several thousand personal JPEGs and other data, and for under $2/month it is nice to know that my apartment building could burn down and I'd still have my photo collection. I think.

There's a rumor out there that Google is planning a data storage service. Don't be surprised if they beat Amazon on price -- and look at your sh*t to see what ads to serve you.

On Jan 24, 2008 12:23 PM, Rob Marscher <rmarscher@...> wrote:
On Jan 18, 2008, at 1:35 PM, Cliff Hirsch wrote:
Is anyone using Amazon S3 for backup?

Here's an oft-referenced article by Jeremy Zawodny:

I personally have only used S3 for serving/storing media and haven't done so on a very large scale yet.

--
David Mintz
http://davidmintz.org/

The subtle source is clear and bright
The tributary streams flow through the darkness
_______________________________________________
New York PHP Community Talk Mailing List
http://lists.nyphp.org/mailman/listinfo/talk

NYPHPCon 2006 Presentations Online
http://www.nyphpcon.com

Show Your Participation in New York PHP
http://www.nyphp.org/show_participation.php

Re: Anyone using Amazon S3 for backup?

by Chris Snyder-6 :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

On Jan 25, 2008 2:40 PM, David Mintz <david@...> wrote:

> Don't be surprised if they beat Amazon on price -- and look at your sh*t to
> see what ads to serve you.

The biggest difference between Google and most other ISPs:
Google is an advertising company.

They aren't in business to sell services to you, they are in business
to sell your attention to advertisers. I'd rather pay full price at
Amazon and know that I'm the primary customer.

Sent via GMail ;-)

--
Chris Snyder
http://chxo.com/
_______________________________________________
New York PHP Community Talk Mailing List
http://lists.nyphp.org/mailman/listinfo/talk

NYPHPCon 2006 Presentations Online
http://www.nyphpcon.com

Show Your Participation in New York PHP
http://www.nyphp.org/show_participation.php

Re: Anyone using Amazon S3 for backup?

by Leam Hall :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Old thread that comes to mind as we're down-sizing at home. I can do
almost everything on my laptop and check in to a remote SVN but my wife
is a photographer and needs a place to store her images that are not yet
edited for SmugMug. I can easily see 60G by the end of the year, and
given that she's a good and prolific photographer, more as it goes.

One idea is a "distributed amoungst friends" netowrk where I set up some
servers in other people's houses and share space in trade for power and
connectivity. Another is the S3 thing from Amazon, but as noted the lack
of rsync is an issue. My wife's Mac could probably be set up to do
something besides ftp, once we find a location to put stuff.

Any more current ideas?

Leam

On Thu, 2008-01-24 at 12:23 -0500, Rob Marscher wrote:

> On Jan 18, 2008, at 1:35 PM, Cliff Hirsch wrote:
> > Is anyone using Amazon S3 for backup?
>
>
> Here's an oft-referenced article by Jeremy Zawodny:
> http://jeremy.zawodny.com/blog/archives/007624.html
>
>
> I personally have only used S3 for serving/storing media and haven't
> done so on a very large scale yet.
>
>
> _______________________________________________
> New York PHP Community Talk Mailing List
> http://lists.nyphp.org/mailman/listinfo/talk
>
> NYPHPCon 2006 Presentations Online
> http://www.nyphpcon.com
>
> Show Your Participation in New York PHP
> http://www.nyphp.org/show_participation.php

_______________________________________________
New York PHP User Group Community Talk Mailing List
http://lists.nyphp.org/mailman/listinfo/talk

http://www.nyphp.org/show_participation.php

Re: Anyone using Amazon S3 for backup?

by edward potter :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

One software space offered us a backup combo for $1000 per year for a
client biz. We went for A3 and Jungle Disk for .85 cents a month.
What depression?  :-) e



On Sat, Nov 22, 2008 at 7:09 AM, Leam Hall <leam@...> wrote:

> Old thread that comes to mind as we're down-sizing at home. I can do
> almost everything on my laptop and check in to a remote SVN but my wife
> is a photographer and needs a place to store her images that are not yet
> edited for SmugMug. I can easily see 60G by the end of the year, and
> given that she's a good and prolific photographer, more as it goes.
>
> One idea is a "distributed amoungst friends" netowrk where I set up some
> servers in other people's houses and share space in trade for power and
> connectivity. Another is the S3 thing from Amazon, but as noted the lack
> of rsync is an issue. My wife's Mac could probably be set up to do
> something besides ftp, once we find a location to put stuff.
>
> Any more current ideas?
>
> Leam
>
> On Thu, 2008-01-24 at 12:23 -0500, Rob Marscher wrote:
>> On Jan 18, 2008, at 1:35 PM, Cliff Hirsch wrote:
>> > Is anyone using Amazon S3 for backup?
>>
>>
>> Here's an oft-referenced article by Jeremy Zawodny:
>> http://jeremy.zawodny.com/blog/archives/007624.html
>>
>>
>> I personally have only used S3 for serving/storing media and haven't
>> done so on a very large scale yet.
>>
>>
>> _______________________________________________
>> New York PHP Community Talk Mailing List
>> http://lists.nyphp.org/mailman/listinfo/talk
>>
>> NYPHPCon 2006 Presentations Online
>> http://www.nyphpcon.com
>>
>> Show Your Participation in New York PHP
>> http://www.nyphp.org/show_participation.php
>
> _______________________________________________
> New York PHP User Group Community Talk Mailing List
> http://lists.nyphp.org/mailman/listinfo/talk
>
> http://www.nyphp.org/show_participation.php
>



--
IM/iChat: ejpusa
Links: http://del.icio.us/ejpusa
Blog: http://www.preceptress.com/blog
Follow me: http://www.twitter.com/ejpusa
Karma: http://www.coderswithconscience.com
Projects: http://flickr.com/photos/86842405@N00/
Store: http://astore.amazon.com/httpwwwutopic-20
_______________________________________________
New York PHP User Group Community Talk Mailing List
http://lists.nyphp.org/mailman/listinfo/talk

http://www.nyphp.org/show_participation.php

Re: Anyone using Amazon S3 for backup?

by edward potter :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Followup:

The Amazon CTO is know as one of the smartest guy in the biz. If you
see a problem,  suggestion, write him direct. I'm sure he'll reply.

Werner Vogels
CTO - Amazon.com
http://www.allthingsdistributed.com/   [blog]




On Sat, Nov 22, 2008 at 8:42 AM, Edward Potter <edwardpotter@...> wrote:

> One software space offered us a backup combo for $1000 per year for a
> client biz. We went for A3 and Jungle Disk for .85 cents a month.
> What depression?  :-) e
>
>
>
> On Sat, Nov 22, 2008 at 7:09 AM, Leam Hall <leam@...> wrote:
>> Old thread that comes to mind as we're down-sizing at home. I can do
>> almost everything on my laptop and check in to a remote SVN but my wife
>> is a photographer and needs a place to store her images that are not yet
>> edited for SmugMug. I can easily see 60G by the end of the year, and
>> given that she's a good and prolific photographer, more as it goes.
>>
>> One idea is a "distributed amoungst friends" netowrk where I set up some
>> servers in other people's houses and share space in trade for power and
>> connectivity. Another is the S3 thing from Amazon, but as noted the lack
>> of rsync is an issue. My wife's Mac could probably be set up to do
>> something besides ftp, once we find a location to put stuff.
>>
>> Any more current ideas?
>>
>> Leam
>>
>> On Thu, 2008-01-24 at 12:23 -0500, Rob Marscher wrote:
>>> On Jan 18, 2008, at 1:35 PM, Cliff Hirsch wrote:
>>> > Is anyone using Amazon S3 for backup?
>>>
>>>
>>> Here's an oft-referenced article by Jeremy Zawodny:
>>> http://jeremy.zawodny.com/blog/archives/007624.html
>>>
>>>
>>> I personally have only used S3 for serving/storing media and haven't
>>> done so on a very large scale yet.
>>>
>>>
>>> _______________________________________________
>>> New York PHP Community Talk Mailing List
>>> http://lists.nyphp.org/mailman/listinfo/talk
>>>
>>> NYPHPCon 2006 Presentations Online
>>> http://www.nyphpcon.com
>>>
>>> Show Your Participation in New York PHP
>>> http://www.nyphp.org/show_participation.php
>>
>> _______________________________________________
>> New York PHP User Group Community Talk Mailing List
>> http://lists.nyphp.org/mailman/listinfo/talk
>>
>> http://www.nyphp.org/show_participation.php
>>
>
>
>
> --
> IM/iChat: ejpusa
> Links: http://del.icio.us/ejpusa
> Blog: http://www.preceptress.com/blog
> Follow me: http://www.twitter.com/ejpusa
> Karma: http://www.coderswithconscience.com
> Projects: http://flickr.com/photos/86842405@N00/
> Store: http://astore.amazon.com/httpwwwutopic-20
>



--
IM/iChat: ejpusa
Links: http://del.icio.us/ejpusa
Blog: http://www.preceptress.com/blog
Follow me: http://www.twitter.com/ejpusa
Karma: http://www.coderswithconscience.com
Projects: http://flickr.com/photos/86842405@N00/
Store: http://astore.amazon.com/httpwwwutopic-20
_______________________________________________
New York PHP User Group Community Talk Mailing List
http://lists.nyphp.org/mailman/listinfo/talk

http://www.nyphp.org/show_participation.php

Re: Anyone using Amazon S3 for backup?

by Hans Zaunere :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

What's amazing is that someone has an email from a year ago - already well
backed up?  :)

I use these guys for personal and server backups:

http://raidarray.net/

H


> -----Original Message-----
> From: talk-bounces@... [mailto:talk-
> bounces@...] On Behalf Of Leam Hall
> Sent: Saturday, November 22, 2008 7:10 AM
> To: NYPHP Talk
> Subject: Re: [nyphp-talk] Anyone using Amazon S3 for backup?
>
> Old thread that comes to mind as we're down-sizing at home. I can do
> almost everything on my laptop and check in to a remote SVN but my wife
> is a photographer and needs a place to store her images that are not yet
> edited for SmugMug. I can easily see 60G by the end of the year, and
> given that she's a good and prolific photographer, more as it goes.
>
> One idea is a "distributed amoungst friends" netowrk where I set up some
> servers in other people's houses and share space in trade for power and
> connectivity. Another is the S3 thing from Amazon, but as noted the lack
> of rsync is an issue. My wife's Mac could probably be set up to do
> something besides ftp, once we find a location to put stuff.
>
> Any more current ideas?
>
> Leam
>
> On Thu, 2008-01-24 at 12:23 -0500, Rob Marscher wrote:
> > On Jan 18, 2008, at 1:35 PM, Cliff Hirsch wrote:
> > > Is anyone using Amazon S3 for backup?
> >
> >
> > Here's an oft-referenced article by Jeremy Zawodny:
> > http://jeremy.zawodny.com/blog/archives/007624.html
> >
> >
> > I personally have only used S3 for serving/storing media and haven't
> > done so on a very large scale yet.
> >
> >
> > _______________________________________________
> > New York PHP Community Talk Mailing List
> > http://lists.nyphp.org/mailman/listinfo/talk
> >
> > NYPHPCon 2006 Presentations Online
> > http://www.nyphpcon.com
> >
> > Show Your Participation in New York PHP
> > http://www.nyphp.org/show_participation.php
>
> _______________________________________________
> New York PHP User Group Community Talk Mailing List
> http://lists.nyphp.org/mailman/listinfo/talk
>
> http://www.nyphp.org/show_participation.php

_______________________________________________
New York PHP User Group Community Talk Mailing List
http://lists.nyphp.org/mailman/listinfo/talk

http://www.nyphp.org/show_participation.php

Re: Anyone using Amazon S3 for backup?

by Chris Snyder-6 :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

On Sat, Nov 22, 2008 at 7:09 AM, Leam Hall <leam@...> wrote:

> One idea is a "distributed amoungst friends" netowrk where I set up some
> servers in other people's houses and share space in trade for power and
> connectivity.

There's a solid-looking product that does this called CrashPlan:
http://www.crashplan.com/landing/index.html

Haven't used it, but I was impressed enough to consider it at one point.

As another option, the Windows guys at work like Mozy a lot.
http://mozy.com/


Chris Snyder
http://chxor.chxo.com/
_______________________________________________
New York PHP User Group Community Talk Mailing List
http://lists.nyphp.org/mailman/listinfo/talk

http://www.nyphp.org/show_participation.php

Re: Anyone using Amazon S3 for backup?

by Chris Snyder-6 :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

On Sat, Nov 22, 2008 at 8:55 AM, Hans Zaunere <lists@...> wrote:

> I use these guys for personal and server backups:
>
> http://raidarray.net/
>

You didn't choose them because of the cheesy music on their site, did you?

  c
_______________________________________________
New York PHP User Group Community Talk Mailing List
http://lists.nyphp.org/mailman/listinfo/talk

http://www.nyphp.org/show_participation.php

Re: Anyone using Amazon S3 for backup?

by Hans Zaunere :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

> > I use these guys for personal and server backups:
> >
> > http://raidarray.net/
> >
>
> You didn't choose them because of the cheesy music on their site, did
> you?

Yeah that's new - not sure who's bright idea that was.  But their service is
great, so +1 for the technical team, -1 for marketing.

H


_______________________________________________
New York PHP User Group Community Talk Mailing List
http://lists.nyphp.org/mailman/listinfo/talk

http://www.nyphp.org/show_participation.php

Re: Anyone using Amazon S3 for backup?

by sbeam-2 :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

On Saturday 22 November 2008 07:09, Leam Hall wrote:
>  Another is the S3 thing from Amazon, but as noted the lack
> of rsync is an issue

Duplicity now supports S3 natively and uses rsync.
http://www.nongnu.org/duplicity/index.html

I am dying to try it, right now using a homebrew solution based on encfs and
s3fs to backup my home systems but it is slow and clunky.

Sam
_______________________________________________
New York PHP User Group Community Talk Mailing List
http://lists.nyphp.org/mailman/listinfo/talk

http://www.nyphp.org/show_participation.php