|
Discussion Forums
|
Thread: Amazon S3 POST Support - Beta Release
|
|
|
Replies:
22
-
Pages:
2
[
1
2
| Next
]
-
Last Post:
Jul 12, 2008 7:03 PM
by: matthewjohnr
|
|
|
Posts:
88
Registered:
3/15/07
|
|
|
Posts:
2,716
Registered:
5/25/06
|
|
|
|
Re: Amazon S3 POST Support - Beta Release
Posted:
Dec 17, 2007 5:48 PM PST
in response to: Alyssa@AWS
|
|
|
One question I have is about mime-type. If you want to allow arbitrary file uploads, or even several different types of images, it isn't realistic to define the mime-type in the form. Could the mime-type be part of the x-ignore- feature? That way, I could have some javascript that fills that in based on file extension.
David
|
|
Posts:
169
Registered:
8/11/07
|
|
|
|
Re: Amazon S3 POST Support - Beta Release
Posted:
Dec 17, 2007 9:25 PM PST
in response to: Alyssa@AWS
|
|
|
Could you please tell me a sample php code that would allow my users to upload jpeg images onto my bucket.
Thanks..
Cause i am getting errors that are to weird.
This XML file does not appear to have any style information associated with it. The document tree is shown below.
−
<Error>
<Code>SignatureDoesNotMatch</Code>
−
<Message>
The request signature we calculated does not match the signature you provided. Check your key and signing method.
</Message>
<RequestId>76DED3011AB5AF8F</RequestId>
<SignatureProvided>KhqElAWvyy389wNesvgnHaXk2nk</SignatureProvided>
−
<StringToSignBytes>
65 77 6f 67 49 43 4a 6c 65 48 42 70 63 6d 46 30 61 57 39 75 49 6a 6f 67 49 6a 49 77 4d 44 67 74 4d 44 45 74 4d 44 46 55 4d 54 49 36 4d 44 41 36 4d 44 41 75 4d 44 41 77 57 69 49 73 43 69 41 67 49 6d 4e 76 62 6d 52 70 64 47 6c 76 62 6e 4d 69 4f 69 42 62 43 69 41 67 49 43 42 37 49 6d 4a 31 59 32 74 6c 64 43 49 36 49 43 49 38 59 6e 56 6a 61 32 56 30 62 6d 46 74 5a 54 34 69 49 48 30 73 43 69 41 67 49 43 42 37 49 6d 46 6a 62 43 49 36 49 43 4a 77 64 57 4a 73 61 57 4d 74 63 6d 56 68 5a 43 49 67 66 53 77 4b 49 43 41 67 49 46 73 69 5a 58 45 69 4c 43 41 69 4a 47 74 6c 65 53 49 73 49 43 4a 30 5a 58 4e 30 5a 6d 6c 73 5a 53 35 30 65 48 51 69 58 53 77 4b 49 43 41 67 49 46 73 69 63 33 52 68 63 6e 52 7a 4c 58 64 70 64 47 67 69 4c 43 41 69 4a 45 4e 76 62 6e 52 6c 62 6e 51 74 56 48 6c 77 5a 53 49 73 49 43 4a 30 5a 58 68 30 4c 79 4a 64 4c 41 6f 67 49 46 30 4b 66 51 6f 3d
</StringToSignBytes>
<AWSAccessKeyId>1HW6N7JH212BB25G1PG2</AWSAccessKeyId>
−
<HostId>
CMepuw2VUR4y6GRi/dgDHvC5OcK8/6zrjVPElgFp5uv6FHPbuM5FrGapE5ihXd2Z
</HostId>
−
<StringToSign>
ewogICJleHBpcmF0aW9uIjogIjIwMDgtMDEtMDFUMTI6MDA6MDAuMDAwWiIsCiAgImNvbmRpdGlvbnMiOiBbCiAgICB7ImJ1Y2tldCI6ICI8YnVja2V0bmFtZT4iIH0sCiAgICB7ImFjbCI6ICJwdWJsaWMtcmVhZCIgfSwKICAgIFsiZXEiLCAiJGtleSIsICJ0ZXN0ZmlsZS50eHQiXSwKICAgIFsic3RhcnRzLXdpdGgiLCAiJENvbnRlbnQtVHlwZSIsICJ0ZXh0LyJdLAogIF0KfQo=
</StringToSign>
</Error>
|
|
Posts:
158
Registered:
2/10/06
|
|
|
|
Re: Amazon S3 POST Support - Beta Release
Posted:
Dec 18, 2007 9:35 AM PST
in response to: D. Kavanagh
|
|
|
Hi David--
I am not sure how you are thinking of using the "x-ignore- feature". The "x-ignore-" prefix is used to allow you to have fields that are completely ignored by Amazon S3, and thus guaranteed not to break as we roll in new features in the future.
There currently is not a way to derive the content-type from a File-based upload. On the Javascript front, I do not believe that there is a way to get access to the filename being uploaded, and therefore you cannot set the content-type based on that.
The ability for Amazon S3 to automatically set the content-type or the ability to update this metadata afterwards is something that was discussed in the Proposal POST (
http://developer.amazonwebservices.com/connect/thread.jspa?threadID=18156) and is something that we may do in the future.
For now, one possible workaround may be to ask your users to characterise the data being uploaded and to trust them.
Thanks for your feedback.
Eric
|
|
Posts:
158
Registered:
2/10/06
|
|
|
|
Re: Amazon S3 POST Support - Beta Release
Posted:
Dec 18, 2007 9:48 AM PST
in response to: hareem
|
|
|
Hi hareem--
To start debugging your problem, you may consider using our HTML-based sample code available at
http://s3.amazonaws.com/doc/s3-example-code/post/post_sample.html . This sample will allow you to build a policy document and an HTML form (there's a generate button at the bottom of the page). Given the same Policy document, the Base64 encoding and HMAC should match exactly what your application is generating.
We currently do not have any PHP-specific sample code, but as the only thing that your server really needs to do is to create a Policy document, base 64 encode the policy, and sign the base64 encoding. For help with Base64 encoding and generating the HMAC-based signatures, our REST library available at
http://developer.amazonwebservices.com/connect/entry.jspa?externalID=126&categoryID=47 demonstrates how to do this, but with the key difference is that you are no longer signing headers, but instead signing the Policy document.
If you continue to have problems, please let us know. Including the approximate time that the error occurred with the error would be hugely helpful. Thanks.
Eric
|
|
Posts:
129
Registered:
3/27/06
|
|
|
|
Re: Amazon S3 POST Support - Beta Release
Posted:
Dec 18, 2007 9:53 AM PST
in response to: hareem
|
|
|
I had a similar problem with the downloaded sample html file (along with the js downloads from other sites). I'm running under Firefox. Same credentials worked OK on the on-line version
http://s3.amazonaws.com/doc/s3-example-code/post/post_sample.html
.. so I downloaded it and its associated JS files using Firefox's "Save Page As.." and they worked fine.
Under IE7 if you've not supplied a redirect URL, a blank page is rendered but under Firefox the "page" is treated as a download which fires up the download manager. OK, I guess the ideal would be to have a redirect URL but just setting the returned content/type to html would be nice, and a simple message such as OK (or better still allow for a simple "redirect message" as an alternative to a redirect URL).
Tom
|
|
Posts:
158
Registered:
2/10/06
|
|
|
|
Re: Amazon S3 POST Support - Beta Release
Posted:
Dec 18, 2007 10:05 AM PST
in response to: tomgleeson
|
|
|
Hi Tom--
Thanks for your feedback; we'll fix that in the next release.
Eric
|
|
Posts:
129
Registered:
3/27/06
|
|
|
|
Re: Amazon S3 POST Support - Beta Release
Posted:
Dec 18, 2007 10:19 AM PST
in response to: Eric@AWS
|
|
|
I've figured out why my original downloaded version didn't work. The sha1.js downloaded from
http://pajhome.org.uk/crypt/md5/
needs to have the line
var b64pad = ""; /* base-64 pad character. "=" for strict RFC compliance */
replaced with ..
var b64pad = "="; /* base-64 pad character. "=" for strict RFC compliance */
Tom
|
|
Posts:
129
Registered:
3/27/06
|
|
|
|
Re: Amazon S3 POST Support - Beta Release
Posted:
Dec 18, 2007 10:36 AM PST
in response to: Eric@AWS
|
|
|
Update on Firefox firing up download manager.
This happened when my upload bucket name ended in .com, "normal" buckets (i.e. not ending in anything that windows might regard as a valid file extension) work as per IE7.
|
|
Posts:
35
Registered:
11/16/07
|
|
|
|
Re: Amazon S3 POST Support - Beta Release
Posted:
Dec 18, 2007 8:33 PM PST
in response to: Eric@AWS
|
|
|
Eric,
Nice to see this feature finally going into beta, but as I said before, the current implementation renders the service
entirely useless
.
Here is a re-post of my previous rant/feedback.
I took another look at your specs and realized a major drawback in the current design.
Based on your specs, I'm making the following assumptions (correct me if I'm wrong):
1. Content-Disposition & Content-Type (Store-and-Forward) headers can be specified as a form input element (just like in a PUT).
2. If Content-Disposition is not specified, the object will be stored with NO such header.
3. If Content-Type is not specified in the form document, "the Content-Type will default to the Content-Type that the browser specifies for the file type or, if not specified, it will not be set." (quoted from your specs)
4. "Store-and-Forward" headers cannot be modified once an object is stored (due to current S3 implementation).
5. After a file is successfully uploaded using a POST form, any GET request to that object will be sent along with the provided (if any) Content-Type and Content-Disposition.
In my opinion, the above constraints render the proposed POST implementation useless for any real production use, because:
1. A Content-Type value provided by a user/browser is totally unreliable.
-- Many browsers fail to properly provide the correct MIME-type for a POST file upload. For example, IE on Windows may send a "image/ p jpeg" instead of "image/jpeg" for a JPG, or "image/ x- png" instead of "image/png" for a PNG file. ZIP files can be sent with a general "application/x-compress" MIME-type or specifically "application/x-zip-compressed", "application/x-zip", or "application/zip"
-- Some browsers may also send a generic "application/octet-stream" for any unrecognized MIME binary subtypes. This is not uncommon when sending RAR files (should be "application/x-rar-compressed"), for example.
--- Some browsers just don't send a Content-Type header at all if a MIME-type cannot be found for a file.
Therefore, some S3 GET requests to an object stored in such manner are likely to be unrecognized by the browser making the request. Also, older browsers will default to "text/plain" when provided with no Content-Type, resulting in binary files being displayed raw to the user in the browser.
This also renders the Access Control (ie. ["eq", "$Content-Type", "application/zip"]) quite ineffective without having to consider all possibilities. Regular Expression support would provide a quick and dirty solution.
2. The ability to set a Content-Disposition header in the form document is not that helpful, because when the HTML form is generated we still don't know what type of file the user is about to upload . This means we cannot tell S3 to serve image files with an Inline Disposition Type. With no Content-Disposition header present, most modern browsers will default to the Attachment Disposition Type.
3. This also prevents the developer from including an appropriate Filename parameter in a Content-Disposition header. If no such header is provided to S3, a GET request to the object would force a browser to download the file and use the Object Key as the file name (and most likely such Key would be set in accordance with whatever S3 Key naming scheme is used by the developer).
A possible hack would be for the developer to access the name of the file to be uploaded with JavaScript and the FileUpload object and re-set any header-related form input values prior to form submission. However, this obviously requires that JavaScript be enabled on the client side.
Possible solutions and enhancements to S3:
1. It would be nice to be able to set response headers such as Content-Disposition and Content-Type dynamically upon a GET request, or alternatively provide API to modify any headers previously set for an object without having to delete and re-put.
2. At the very least, enable the developer to use the name of the file uploaded as the Filename parameter of a Content-Disposition.
3. Extend the Policy Document to include post-upload processing. I quickly came up with this just to explain how it could potentially help developers:
(adding these to a Policy Document):
"mimetypes": [
{"image/jpeg": "jpeg jpg" },
{"application/java-archive": "jar war ear" },
{"application/octet-stream": "bin exe dll" },
...
]
"dynamic": [
{"header", "Content-Type" : $filename.GetMimeType(@mimetypes)}
{"set", "$newfilename" : $filename.PregMatch("/^[a-zA-Z0-9\_\.]$/i")}
{"set", "$disptype" : $Content-Type.indexOf("image") > -1 then "inline" else "attachment"}
{"header", "Content-Disposition": "$disptype; filename="$newfilename""}
]
Syntax should obviously be worked on to comply with AWS standards.
I hope you take these concerns to your consideration.
|
|
Posts:
158
Registered:
2/10/06
|
|
|
|
Re: Amazon S3 POST Support - Beta Release
Posted:
Dec 18, 2007 9:05 PM PST
in response to: azdeveloper
|
|
|
Hi azdeveloper--
Thanks for your great feedback. Based on your initial feedback, we did remove our original plan to use the browser-provided Content-Type.
Regarding your suggestions, can you explain to me what you need to accomplish #2 of your suggestions? If you wanted to set the Content-Disposition header to the uploaded file's name, you can do that with the ${filename} variable; this is something that did not exist in our initial proposal that we added based on feedback we received. This variable can be used throughout the form, and I believe you can accomplish what you are after.
Regarding your third suggestion, it is an excellent suggestion and we are considering it. Thanks again.
Eric
|
|
Posts:
35
Registered:
11/16/07
|
|
|
|
Re: Amazon S3 POST Support - Beta Release
Posted:
Dec 18, 2007 10:13 PM PST
in response to: Eric@AWS
|
|
|
Eric,
I understand that the auto-detection feature was removed based on my initial feedback, but doing so without offering an alternative method means that now the POST feature is 100% not ready for any kind of live implementation. The auto-detection was okay as a temporary solution enabling simpler implementations of the POST capabilities (but still prone to user abuse and reliability issues). When I said that feature was not useful I didn't expect you to remove it and at the same time not consider any of the solution I offered in that post.
What I meant in #2 was only relevant for the Content-Disposition header issue. While $filename is indeed available and can be passed as part of the Key, it is still far more useful to enable us to set custom Content-Disposition headers based on that value, thus forcing the client to fetch the object with a filename other than the original $filename (i.e. i@break$your~~mac.jpg to ibreakyourmac.jpg), or set the Content-Disposition type to "inline" if $filename has "jpg" extension, or "attachment" for any other file type. I realize, tho, that this is not as critical as the Content-Type issue, which needs to be resolved ASAP IMHO.
Until you implement an advanced solution to allow us to safely and reliably detect/extract and set the Content-Type and Content-Disposition headers, I think that it is best if you did offer some form of Content-Type auto-detection based on the file extension. To do so, you should use standard MIME-types for as many file types as you can and add some form field where we can select to enable this option.
By the way, in JavaScript it IS possible to extract the filename from the File object (at least in modern browsers), but as I said in my previous post, it should not be trusted or relied upon by the developer. It is also inconceivable to see Amazon officially suggest that we should prompt the user to specify the file-type themselves as part of the upload process
So, to sum up my rant... IMHO:
1. This initial implementation of the POST feature should be treated by developers as nothing more than a proof-of-concept and test/sandbox environment to play with in preparation for a production-ready release.
2. Processing user uploads using S3 & the POST method without having the ability to properly, reliably, and dynamically detect/extract/set the Content-Type and Content-Disposition headers is absurd and not appropriate for real-world use.
Thanks for listening!
|
|
Posts:
376
Registered:
7/17/06
|
|
|
|
Re: Amazon S3 POST Support - Beta Release
Posted:
Dec 18, 2007 11:07 PM PST
in response to: Alyssa@AWS
|
|
|
In my opinion, probably the simplest and safest way to deal with the mime-type issue is to allow the headers on a S3 key to be changed without re-uploading the key data. After the POST completes, a user's server could retrieve just enough of the file to analyze its file type (or retrieve the whole thing if it prefers), then update the headers to reflect the true file type.
|
|
Posts:
35
Registered:
11/16/07
|
|
|
|
Re: Amazon S3 POST Support - Beta Release
Posted:
Dec 18, 2007 11:39 PM PST
in response to: bd_
|
|
|
Absolutely! This is a MUCH-NEEDED feature which will solve this problem as well as open up a whole new host of capabilities! (although there are still easier ways to accomplish this).
Eric - any hope for this?
P.S. Dynamic header modification at the time of a QueryString-Authenticated GET request would be even better! This way a Content-Disposition header could be generated onthefly when signing a URL request.
|
|
Posts:
2,027
Registered:
3/15/06
|
|
|
|
Re: Amazon S3 POST Support - Beta Release
Posted:
Dec 19, 2007 12:42 AM PST
in response to: azdeveloper
|
|
|
In my opinion, what you really want is to be able to store your headers and other metadata related to an object in S3 in SimpleDB. In fact, I can do that now but the problem with my current approach is that I am responsible for merging the headers from SimpleDB with the content from S3. I can do that but then I can't just serve the file directly from S3 anymore. I have to pass it through my software and I lose all of the wonderful scalability of S3.
Mitch
|
|
|
|
Sign in to the AWS Management Console
Create an AWS Account
