|
Discussion Forums
|
Thread: Persisting your data after terminating instance
|
|
|
Replies:
6
-
Pages:
1
-
Last Post:
Dec 1, 2007 9:14 AM
by: cdp777
|
|
|
Posts:
4
Registered:
11/28/07
|
|
|
|
Persisting your data after terminating instance
Posted:
Nov 30, 2007 5:14 PM PST
|
|
|
This is a nebis question, but I was wondering do I use S3 to persist my data after I terminate my instance. When an instance essentially its gone and so is your data. What if you've written data to a database and have create code within that instance its gones. Is there any kind of documentation onlining this issue.
|
|
Posts:
35
Registered:
11/16/07
|
|
|
Posts:
5,320
Registered:
3/19/07
|
|
|
|
Re: Persisting your data after terminating instance
Posted:
Nov 30, 2007 6:31 PM PST
in response to: cdp777
|
|
|
There is no way to persist your data AFTER terminating your instance. If you want to save any data, you have to do that BEFORE terminating the instance.
|
|
Posts:
4
Registered:
11/28/07
|
|
|
|
Re: Persisting your data after terminating instance
Posted:
Nov 30, 2007 8:12 PM PST
in response to: Allen
|
|
|
Okay, I sounded a little dyslexic in my post. I meant how do you persist data if an instance is transient and everything is reinitialized on startup? Does a private AMI persist your database? On startup I'd like the database to be at the same point where I left off. I'm going to try a private AMI and see if that solves the issue. I thought I'd ask instead of doing trial by fire.
|
|
Posts:
35
Registered:
11/16/07
|
|
|
|
Re: Persisting your data after terminating instance
Posted:
Nov 30, 2007 11:56 PM PST
in response to: cdp777
|
|
|
Amazon doesn't provide any mechanism to persist data. If you don't want to deploy a cluster solution or even simple replication, what you could do is at least flush the database logs every few minutes and immediately upload the new log files into S3. You could also do the same for full backup dumps every few hours.
Then, if your instance terminates and a new one is started, the first task you can have it do is reconstruct the database by grabbing the latest full backup dump and any recent log files the previous instance saved on S3.
This is a quick and dirty solution and should suffice for non-critical use. To really play it safe you could try to flush those log files even every minute. MySQL and InnoDB should have no problem handling that, especially if you're running on a large or xlarge instance.
|
|
Posts:
35
Registered:
11/16/07
|
|
|
|
Re: Persisting your data after terminating instance
Posted:
Dec 1, 2007 12:12 AM PST
in response to: cdp777
|
|
|
Another idea I came up with a while ago was to use Memcached to store all successful database queries in memory (except for SELECTs) and then dump that data into S3. This will also happen on the application level, so if the database instance goes down, you still have the most recent queries saved up in your application server's Memcache. That and whatever you saved to S3 can be used to start a fresh db instance with pretty much full persistence.
Again, this is a messy "solution"... I personally would not use it in production, especially if you're doing transactions. Flushing InnoDB binary logs on MySQL 5 every 60 seconds should be fine. For extra persistence try out the Memcached solution.
(Memcached:
http://www.danga.com/memcached/)
|
|
Posts:
4
Registered:
11/28/07
|
|
|
|
Re: Persisting your data after terminating instance
Posted:
Dec 1, 2007 9:14 AM PST
in response to: cdp777
|
|
|
I see the process now. The service is really forcing me into a different mindset. I'll look into the Memcached and Innodb ideas and maybe a messaging layer. Thanks for the suggesstions. I think this service has great potential.
|
|
|
|
Sign in to the AWS Management Console
Create an AWS Account
