|
Discussion Forums
|
Thread: Persistent Storage for Amazon EC2
|
|
|
Replies:
69
-
Pages:
5
[
1
2
3
4
5
| Next
]
-
Last Post:
Jul 8, 2008 12:06 PM
by: mdavidpeterson
|
|
|
Posts:
92
Registered:
9/12/07
|
|
|
|
Persistent Storage for Amazon EC2
Posted:
Apr 13, 2008 9:04 PM PDT
|
|
|
Many of you have been requesting that we let you know ahead of time about features that are currently under development so that you can better plan for how that functionality might integrate with your applications.
To that end, we would like to share some details about a major upcoming feature that many of you have requested - persistent storage for EC2.
This new feature provides reliable, persistent storage volumes, for use with Amazon EC2 instances.
These volumes exist independently from any Amazon EC2 instances, and will behave like raw, unformatted hard drives or block devices, which may then be formatted and configured based on the needs of your application.
The volumes will be significantly more durable than the local disks within an Amazon EC2 instance.
Additionally, our persistent storage feature will enable you to automatically create snapshots of your volumes and back them up to Amazon S3 for even greater reliability.
You will be able to create volumes ranging in size from 1 GB to 1 TB, and will be able to attach multiple volumes to a single instance.
Volumes are designed for high throughput, low latency access from Amazon EC2, and can be attached to any running EC2 instance where they will show up as a device inside of the instance.
This feature will make it even easier to run everything from relational databases to distributed file systems to Hadoop processing clusters using Amazon EC2.
When persistent storage is launched, Amazon EC2 will be adding several new APIs to support the persistent storage feature.
Included will be calls to manage your volume (CreateVolume, DeleteVolume), mount your volume to your instance (AttachVolume, DetachVolume) and save snapshots to Amazon S3 (CreateSnapshot, DeleteSnapshot).
This new functionality is already being used privately by a handful of EC2 customers, and will be publically available later this year.
We will be expanding the private offering as we get closer to launch.
Please
sign-up
here if you are interested in participating.
We hope this information is useful to you as you plan, design and deploy your applications in Amazon EC2.
The Amazon EC2 Team
|
|
Posts:
2,842
Registered:
5/25/06
|
|
|
|
Re: Persistent Storage for Amazon EC2
Posted:
Apr 13, 2008 9:19 PM PDT
in response to: MattG@AWS
|
|
|
Way to go Amazon! This is going to rock the EC2 party!
|
|
Posts:
644
Registered:
3/24/06
|
|
|
Posts:
17
Registered:
4/13/08
|
|
|
|
Re: Persistent Storage for Amazon EC2
Posted:
Apr 13, 2008 10:58 PM PDT
in response to: MattG@AWS
|
|
|
This is really cool, and surely will be very useful. Thanks alot.
Can the volumes be used by more than one instance at a time, e.g. to run gfs or another clustered filesystem?
|
|
Posts:
55
Registered:
11/13/06
|
|
|
|
Re: Persistent Storage for Amazon EC2
Posted:
Apr 13, 2008 11:35 PM PDT
in response to: MattG@AWS
|
|
|
Cool feature add-on to EC2, hoping to test-drive this soon with our online backup software,
Regards,
Jay.
|
|
Posts:
2
Registered:
1/29/08
|
|
|
|
Re: Persistent Storage for Amazon EC2
Posted:
Apr 13, 2008 11:55 PM PDT
in response to: MattG@AWS
|
|
|
Wow, you guys rule, you
really
do.
Just one thought, would it be possible to mount one persistent storage volume on multiple EC2 instances? I can think of a few things where that would be quite useful (if this isn't possible read/write, how about just read).
Thanks for keeping ahead of the game.
--
Project AwsSum - Perl interface and tools for AWS
http://github.com/andychilton/awssum/
Message was edited by: Andrew Chilton
|
|
Posts:
1,180
Registered:
7/7/07
|
|
|
|
Re: Persistent Storage for Amazon EC2
Posted:
Apr 14, 2008 12:00 AM PDT
in response to: MattG@AWS
|
|
|
I'm not sure knowing this is coming and not having it is a better feeling than not knowing if it was coming, but thanks :)
I'm not going to push too far and ask "when?!" but do know that there is genuine excitement and anticipation in the ranks, probably more than for static IPs.
I think it's obvious this is the last missing piece for many of us, even though there were some third party options available.
I'd personally like to thank the developers who provided those third party persistent storage options for filling the gap while Amazon worked on a solution which appears to be more tightly integrated with their infrastructure. And, I will continue using some of the 3rd parties as they make it possible to use S3 as persistent storage even from outside of EC2 instances, which can be quite convenient.
I look forward to trying out EC2 Persistent Storage. Let me know if there's anything I can do to help ;-)
--
Eric Hammond
http://www.anvilon.com
|
|
Posts:
3
Registered:
4/13/08
|
|
|
|
Re: Persistent Storage for Amazon EC2
Posted:
Apr 14, 2008 12:47 AM PDT
in response to: MattG@AWS
|
|
|
Great to hear that. Agree that this is the last missing piece of the whole solution, at least for me. Thank you so much for the effort.
|
|
Posts:
439
Registered:
12/22/07
|
|
|
|
Re: Persistent Storage for Amazon EC2
Posted:
Apr 14, 2008 1:33 AM PDT
in response to: MattG@AWS
|
|
|
Thank you!!!!!!!!!!!!!!!!!!!!!!!!
Here's an idea: you could give access to all these nice guys on the forum :)
|
|
Posts:
77
Registered:
5/5/06
|
|
|
|
Re: Persistent Storage for Amazon EC2
Posted:
Apr 14, 2008 3:12 AM PDT
in response to: MattG@AWS
|
|
|
Is there any info on the fault tolerance of these "permanent" hard drives?
i.e. should we expect them to break - and hence should be using the snapshot functionality to backup to S3 - or is there something very cool behind the scenes where the data on the physical hard drives is fault tolerant as well......?!
Just some guidance / clarification would be great
Cheers
Stuart
|
|
Posts:
439
Registered:
12/22/07
|
|
|
Posts:
22
Registered:
12/11/06
|
|
|
|
wonky signup form
Posted:
Apr 14, 2008 6:10 AM PDT
in response to: MattG@AWS
|
|
|
This looks really intriguing, and I'm dying to try it out. However, I can't seem to -- the form at
https://www.amazon.com/gp/html-forms-controller/ec2-persistent-storage/ seems to be allergic to my Firefox. No matter what text I enter in 'Company/Application Description*', I simply get the form redisplayed, but with this error message:
Please provide a description of what your company or application does.
is anyone else running into this?
anyway, needless to say -- I'm interested in participating. ;)
|
|
Posts:
152
Registered:
3/28/08
|
|
|
|
Re: Persistent Storage for Amazon EC2
Posted:
Apr 14, 2008 6:12 AM PDT
in response to: MattG@AWS
|
|
|
FANTASTIC!! And please let me say Amazon that we really appreciate you sharing this information with us so that we can plan our architecture and designs accordingly in advance of its arrival. This is very key!
I too would like to know about the ability to share a persistent volume across multiple instance. Will this be possible? And if so, can it be done in a read/write fashion across multiple instances?
I really hope so. Think about how great that would be. One thing that jumps to mind is that we could then have our database exist solely on one central place. Multiple instances could then (hopefully) read/write to the db concurrently. And in the event an instance goes down and a new one is brought up, the database would be right there. We would still back up the database to an alternate storage medium such as S3 to keep is safe, but I think this would eliminate the need for a failover database at least in our situation.
Am I thinking about your new persistent storage correctly in this regard? Or is it more a matter of providing a primary storage medium just for a single instance. That in itself will be very helpful, but if it can be shared across instances in read/write fashion that would be incredibly useful and valuable. Thanks!!
|
|
Posts:
11
Registered:
4/14/08
|
|
|
|
Re: Persistent Storage for Amazon EC2
Posted:
Apr 14, 2008 6:39 AM PDT
in response to: MattG@AWS
|
|
|
Is it possible to get some details on any redundancy characteristics of this persistent storage, if any?
|
|
Posts:
16
Registered:
7/8/06
|
|
|
|
Re: Persistent Storage for Amazon EC2
Posted:
Apr 14, 2008 6:41 AM PDT
in response to: MattG@AWS
|
|
|
How does this relate to the existing solutions such as persistentfs?
|
|
|
|
