|
Discussion Forums
|
Thread: Creating 'Lock' Files
 |
This question is not answered.
Helpful answers available: 0.
Correct answers available: 1.
|
|
|
|
Replies:
9
-
Pages:
1
-
Last Post:
Mar 9, 2007 5:26 AM
by: panmanp
|
|
|
Posts:
4
Registered:
2/28/07
|
|
|
|
Creating 'Lock' Files
Posted:
Feb 28, 2007 8:27 PM PST
|
|
|
I am storing a file I'd like multiple users to read and
write
to. I'd rather not repost the entire file in order to set metadata bits on it, so instead we are creating zero byte files we use to acquire locks. The idea is similar to object databases of yore where an object in contention has its updates serialized by issuing locks.
My question is, given how Amazon S3 works with respect to data replication among all the data centers, how feasible is this? Or to put it another way, what is the potential lag where two users may check for the existence of the same file, but because their requests are being handle by different data centers, both users believe they have created the lock file?
Thanks,
John Milan
|
|
Posts:
107
Registered:
9/30/06
|
|
|
|
Re: Creating 'Lock' Files
Posted:
Feb 28, 2007 9:54 PM PST
in response to: John Milan
|
|
|
No, there is no guarantee that when you create a file, it will be immediately visible. So I probably would avoid using it for lock files.
|
|
Posts:
555
Registered:
8/26/06
|
|
|
|
Re: Creating 'Lock' Files
Posted:
Mar 1, 2007 4:07 AM PST
in response to: Robin Sheat
|
 |
Helpful |
|
|
not only is there no guarantee about when it will be visible, there is no guarantee that it will be simultaneously visible at all IP addresses that answer the phone for S3. AFAIK the same applies to metadata, i.e. you would not be able to use ACLs to achieve the same effect for the same reasons.
|
|
Posts:
4
Registered:
2/28/07
|
|
|
|
Re: Creating 'Lock' Files
Posted:
Mar 1, 2007 7:39 AM PST
in response to: lowflyinghawk
|
|
|
I understand there is no guarantee on the immediacy of either files or metadata on files. Is there any guarantee on time for propagation? For instance, if I create a file, is it guaranteed to be available everywhere within 60 seconds? Is metadata propagation faster than file creation-- ie it is guaranteed within 30 seconds.
I've heard all the data centers are interconnected with fiber optics. The speed of light is pretty fast, my hope is data transfer/replication is pretty fast too. Are there data propagation metrics available for the public?
Thanks!
|
|
Posts:
555
Registered:
8/26/06
|
|
|
|
Re: Creating 'Lock' Files
Posted:
Mar 1, 2007 9:57 AM PST
in response to: John Milan
|
|
Helpful |
|
|
AFAIK there is no guarantee, period.
|
|
Posts:
12
Registered:
2/9/07
|
|
|
|
Re: Creating 'Lock' Files
Posted:
Mar 1, 2007 7:26 PM PST
in response to: lowflyinghawk
|
|
|
You can't synchronize on a key, but I could think about some sort of freaky synchronization on a bucket. A bucket can be created only once and if a client succeeded in its creation, so the write to the file with the multiple accesses is OK, if creation failed – the client should wait for some specified period of time. Timestamp synchronization also could be helpful for this case, for example, to eliminate deadlocks. Time is a universal thing, and if some synchronization bucket stays created for more then 5 mins you can consider it as a deadlock victim etc. etc.
Message was edited by: segaa
|
|
Posts:
4
Registered:
2/28/07
|
|
|
|
Re: Creating 'Lock' Files
Posted:
Mar 1, 2007 10:17 PM PST
in response to: segaa
|
|
|
That's an interesting idea. But you can only create a total of 100 buckets per account, which would impact throughput if 1000s of people were contending for items.
I am curious if buckets are somehow treated differently however. For instance, wouldn't buckets have the same data propagation issue?
It would also seem too 'heavyweight' for the task at hand. I only need a zero-length file-- some sort of atom-- to appear more or less instantly across the data centers.
It sounds like this is getting into 'feature request' territory
Thanks all.
|
|
Posts:
183
Registered:
12/20/06
|
|
|
|
Re: Creating 'Lock' Files
Posted:
Mar 1, 2007 10:43 PM PST
in response to: John Milan
|
|
|
How frequent would this lock be used ? If it is not too frequent, SQS may be used as a subsitute(kind of like a semaphore). Though if it is frequent, the cost can be prohibitive.
|
|
Posts:
5
Registered:
3/7/07
|
|
|
|
Re: Creating 'Lock' Files
Posted:
Mar 7, 2007 11:58 PM PST
in response to: segaa
|
|
|
There is also a security consideration here.
Because anyone can create any bucket with any name (as long as it does not exist) bad people could create buckets using your naming scheme and confuse your synchronization algorithms.
|
|
Posts:
47
Registered:
3/17/06
|
|
|
|
Re: Creating 'Lock' Files
Posted:
Mar 9, 2007 5:26 AM PST
in response to: John Milan
|
|
|
Well, if the lockbuckets would take the form of the MD5 of "secretstring+date", then I guess nobody but you could make them, as long as the secret string is really secret.
|
|
|
|
Sign in to the AWS Management Console
Create an AWS Account
