Traditionally, the HTML5/Flash Uploader is used to send files to a server which executes a PHP/ASP.NET/Java/etc script to save uploaded files on a hard drive. However nowadays you don't have to organize storage on your servers and many people prefer using Amazon S3 storage instead. You may wonder if it is possible to send files directly to Amazon S3 instead of your server script. Good news is that it is quite easy. Let's dive in!
Primarily, this article explains the usage of JavaScript API. ASP.NET Control and PHP Library users will find code examples in the end of the artice (although it is strongly recommended to read the entire article for better understanding how it works).
Most likely you already have an account in AWS, but even if not, it is not worth describing how to register and login to AWS control panel.
To start uploading files, you need to create a bucket. It is quite similar to Internet domain names and must be unique within the AWS. Bucket keeps objects (i.e. our fields) with unique names (keys) and optional metadata.
Each bucket is associated with an approprite region - i.e. Amazon's data center located in a particular area (for example, United States East Coast, Central Europe, etc). The region name may look like
HTML5/Flash Uploader supports DNS-compliant bucket names only.
myphotos/holiday.jpg in the
mybucket bucket. The URL to access this object is
http://mybucket.s3.amazonaws.com/myphotos/holiday.jpg.
You may have several buckets and they may have different properties. In particular, the important property is ACL - access list, i.e. a list of permissions for each user or a group (like Everyone). To allow uploads, you need to get two keys for the user who have permissions to upload:
As you may learn from the Enabling Cross-Origin Resource Sharing for HTML5 Uploader article, the browsers consider making requests to a server you don't control (i.e. the domain name of the current page is different than a server where you send a request to) as unsafe. Amazon servers are not exception. You should configure CORS policy to make it possible to upload files there.
You can do it in the AWS S3 control panel. Just login, open a bucket, click Properties on the right, expand Permissions and click Add CORS Configuration (or Edit CORS Configuration).
The CORS configuration is just an XML file. You can read more about configuring CORS in Amazon documentation. However if you need some example of CORS policy which makes our demo application working, you can use this one:
<?xml version="1.0" encoding="UTF-8"?>
<CORSConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
<CORSRule>
<AllowedOrigin>http://*</AllowedOrigin>
<AllowedOrigin>https://*</AllowedOrigin>
<AllowedMethod>HEAD</AllowedMethod>
<AllowedMethod>GET</AllowedMethod>
<AllowedMethod>POST</AllowedMethod>
<MaxAgeSeconds>3000</MaxAgeSeconds>
<ExposeHeader>x-amz-storage-class</ExposeHeader>
<ExposeHeader>x-amz-meta-width</ExposeHeader>
<ExposeHeader>x-amz-meta-height</ExposeHeader>
<ExposeHeader>x-amz-meta-category</ExposeHeader>
<AllowedHeader>*</AllowedHeader>
</CORSRule>
</CORSConfiguration>Now we know the AWS terminology and ready to proceed.
Just insert the uploader to a page as usual and configure according to your needs. The only difference with a regular upload - don't specify the ActionUrl param.
The integration with AWS is implemented using the class called amazonS3Extender. Just create it, associate with the uploader, configure and it will do all the magic. Let's see how to do it.
Before you create an instance of amazonS3Extender, you should
you should link the /Scripts/aurigma.imageuploaderflash.amazons3.js
file with your script (if you are using a PHP Library or ASP.NET Control, it is done automatically).
<link rel="stylesheet" type="text/css" href="/aurigma/8.5.33/css/aurigma.htmluploader.control.css" /> <script src="/aurigma/8.5.33/aurigma.htmluploader.control.js" type="text/javascript"></script> <script src="/aurigma/8.5.33/aurigma.imageuploaderflash.min.js" type="text/javascript"></script> <script src="/aurigma/8.5.33/aurigma.imageuploaderflash.amazons3.min.js" type="text/javascript"></script>
Now you are ready to create an instance of the extender and specify its main settings like the destination bucket name, access key and individual settings for each converted file. To associate the extender with the uploader, just pass the uploader object to the constructor.
var u = $au.imageUploaderFlash({
id: 'u1',
// ...
converters: [
{ mode: '*=SourceFile' }
]
});
var ac3 = $au.amazonS3Extender(u);
ac3.accessKeyId( _my_access_key /* you can find in your AWS control panel */);
ac3.bucket("mybucket");
ac3.region("us-east-1");
ac3.converters([
{
acl: 'public-read', /* uploaded files can be viewed by everyone */
key: '${filename}', /* ${filename} preserves the file name, ${guid} creates a unique name */
policy: _policy, /* generated on the server, see further explanations */
signature: _signature /* signed on the server with your secret key */
}
]);
$("#uploader").html(u.getHtml());
A policy file and a signature are the things that make the Amazon S3 upload a bit complicated, but on the other hand, very secure.
A policy file is a JSON file describing the uploaded items. It specifies the constrains the AWS should check and if the POST request does not meet these restrictions, it will be rejected. For example, you may say that you send files with "public-read" permissions to mybucket, the upload is expected to finish not later than at 10:17:35 on Aug 22, 2015, and any filenames are accepted with the following policy file:
{
"expiration": "2015-08-22T10:17:35.000Z",
"conditions": [
{"acl": "public-read"},
{"success_action_status": "200"},
["starts-with", "$key", ""]
]
}
This policy should be passed to the amazonS3Extender as a Base64-encoded string.
If at least one of conditions is broken or the POST request contains some unexpected data, the upload will fail.
However malicious persons don't have any problems forging the policy file. How Amazon makes it secure?
This is where the signature comes into the play. You should use the Secret Key provided by Amazon to sign the policy file using HMAC-SHA1 algorithm (implementation is included to the most of the platforms and languages). The signature you pass to the amazonS3Extender is the Base64-encoded output of the HMAC-SHA1 algorithm.
Read more about AWS policies for POST requests in official documentation
If you are using ASP.NET Control or PHP Library, the policy and signature is created automatically.
Although it is technically possible to create policy and signature on the client or request it with the REST API, you should not do it due to the security reasons. Calculate them on the server and embed to the page instead.
That's it! As soon as you configured an amazonS3Extender object and added a proper policy and signature, files will go directly to a bucket on your AWS account.
What if you need to upload both thumbnails and the original files? That's easy. All you need is to use two converters both in the uploader and amazonS3Extender.
Let's take a look at a code example:
var u = $au.imageUploaderFlash({
id: 'u1',
// ...
converters: [
{ mode: '*=Thumbnail', thumbnailWidth: '100', thumbnailHeight: '100' },
{ mode: '*=SourceFile' }
]
});
var ac3 = $au.amazonS3Extender(u);
ac3.accessKeyId( _my_access_key);
ac3.bucket("mybucket");
ac3.converters([
{
acl: 'public-read',
key: 'thumbnails/${filename}',
policy: _thumb_policy,
signature: _thumb_signature
},
{
acl: 'public-read',
key: 'originals/${filename}',
policy: _orig_policy,
signature: _orig_signature
}
]);
As you can see, we have added two converters to the uploader - one to generate a 100x100 thumbnail and another one - to return the original file. For each converter, we have added an appropriate converter in the amazonS3Extender.
To be able to save thumbnails and originals in the different subfolder, we have modified the key parameter for each of the converter. It means that the policy file for each converter is different - one policy should force the key to start with thumbnails/ and another one - with originals/. And as a consequence, signatures for each converter also should be different.
Amazon services may be expensive. To cut the cost, you may store the files you don't afraid to lose using a so-called Reduced Redundancy Storage. For example, it may be thumbnails (which can be easily re-generated in case of disaster).
You can specify a storage class using the storageClass param. Just set it either to STANDARD or REDUCED_REDUNDANCY.
As you may know, you can send additional data along with files, e.g. descriptions, titles, order ID, categories, etc. But how to have Amazon S3 to recognize them?
To make it possible, the uploader should add your custom fields with x-amz-meta- prefix to the POST request. It can be done using the meta property of the amazonS3Extender's convertor. Also, the policy file should contain the restrictions for these fields. Like this:
Settings:
var u = $au.imageUploaderFlash({
id: 'u1',
// ...
converters: [
{ mode: '*=SourceFile' }
]
});
var ac3 = $au.amazonS3Extender(u);
ac3.accessKeyId( _my_access_key);
ac3.bucket("mybucket");
ac3.converters([
{
acl: 'public-read',
key: '${filename}',
policy: _orig_policy,
signature: _orig_signature,
meta: [
{name: "category", value: "business"},
{name: "case-id", value: "123456"}
]
}
]);
Policy file:
{
"expiration": "2015-08-22T10:17:35.000Z",
"conditions": [
{"acl": "public-read"},
{"success_action_status": "200"},
["starts-with", "$key", ""],
["starts-with", "$x-amz-meta-category", ""],
["starts-with", "$x-amz-meta-case-id", ""]
]
}
However the custom metadata is not the only data the uploader may send. For example, it sends its own information about the files, such as image dimensions, angle, tag, description, etc. To allow this sort of metadata, the meta array supports a special kind of objects which configure predefined fields - here you specify the pair of the name you want to set for the Amazon and the field name:
var u = $au.imageUploaderFlash({
id: 'u1',
// ...
converters: [
{ mode: '*=SourceFile' }
]
});
var ac3 = $au.amazonS3Extender(u);
// ...
ac3.converters([
{
//...
meta: [
{name: "width", field: "SourceWidth_[itemIndex]"},
{name: "height", field: "SourceHeight_[itemIndex]"},
{name: "category", value: "business"},
{name: "case-id", value: "123456"}
]
}
]);
Don't forget about the policy:
{
"expiration": "2015-08-22T10:17:35.000Z",
"conditions": [
{"acl": "public-read"},
{"success_action_status": "200"},
["starts-with", "$key", ""],
["starts-with", "$x-amz-meta-category", ""],
["starts-with", "$x-amz-meta-case-id", ""],
["starts-with", "$x-amz-meta-width", ""],
["starts-with", "$x-amz-meta-height", ""]
]
}
Besides of POST fields, you may want to send HTTP headers to AWS. For example, it may be necessary if you are using CloudFront and want to control the duration how long it keeps files in its cache.
According to the documentation you may need to set the Cache-Control header to max-age=[num of seconds]. So you just need to say uploader to set this header (the uploader can do it without amazonS3Extender) and allow it in the policy file.
var u = $au.imageUploaderFlash({
id: 'u1',
// ...
converters: [
{ mode: '*=SourceFile' }
],
events: {
beforeUpload: function(){
this.metadata().uploadRequestHeaders(
/* Setting max age to one hour */
{"Cache-Control": "max-age=3600"})
}
}
});
var ac3 = $au.amazonS3Extender(u);
// ... amazonS3Extender configuration omitted for brevity ...
u.writeHtml();
The policy should just allow the Cache-Control header:
{
"expiration": "2015-08-22T10:17:35.000Z",
"conditions": [
{"acl": "public-read"},
{"success_action_status": "200"},
["starts-with", "$key", ""],
["eq", "$Cache-Control", "max-age=3600"] /* reject requests with other caching value */
]
}
The ASP.NET Control includes a WebForms control for the amazonS3Extender. It is easier to use comparing to JavaScript API as it creates the policy file and the signature for you. Here how you can insert it into a WebForms application:
<form id="form1" runat="server" class="page">
<aur:ImageUploaderFlash
ID="Uploader1" runat="server" Width="100%" Height="400px">
<Converters>
<aur:Converter Mode="*.*=SourceFile" />
</Converters>
<UploadSettings RedirectUrl="Gallery.aspx" />
</aur:ImageUploaderFlash>
<aur:AmazonS3Extender ID="AmazonS3Extender1" runat="server"
AWSAccessKeyId="<%$ AppSettings:AmazonS3_AWSAccessKeyId %>"
Bucket="<%$ AppSettings:AmazonS3_Bucket %>"
Region="<%$ AppSettings:AmazonS3_Region %>"
SecretAccessKey="<%$ AppSettings:AmazonS3_SecretAccessKey %>"
TargetControlID="Uploader1">
<aur:FileSettings Acl="public-read" Key="files/${filename}" StorageClass="REDUCED_REDUNDANCY">
<Meta>
<aur:PredefinedMetaProperty Name="width" Field="SourceWidth_[ItemIndex]" />
<aur:PredefinedMetaProperty Name="height" Field="SourceHeight_[ItemIndex]" />
<aur:CustomMetaProperty Name="category" Value="business" />
</Meta>
</aur:FileSettings>
</aur:AmazonS3Extender>
</form>
Using PHP Library is also simpler than using the JavaScript API. Here is a code example:
<?php
require_once '../../../ImageUploaderFlashPHP/ImageUploaderFlash.class.php';
require_once '../../../ImageUploaderFlashPHP/AmazonS3Extender.class.php';
// ...
$uploader = new ImageUploaderFlash('Uploader1');
$uploader->setWidth("100%");
$uploader->setHeight("400px");
$uploader->getUploadSettings()->setRedirectUrl("gallery.php");
$converter1 = new Converter();
$converter1->setMode("*.*=SourceFile");
$uploader->setConverters(array($converter1));
$as3 = new AmazonS3Extender($uploader);
$as3->setAWSAccessKeyId($amazon_AWSAccessKeyId);
$as3->setBucket($amazon_Bucket);
$as3->setRegion($amazon_Region);
$as3->setSecretAccessKey($amazon_SecretAccessKey);
$exp_date = time() + 6000;
$fs1 = new FileSettings();
$fs1->setAcl("public-read");
$fs1->setKey('files/${filename}');
$fs1->setStorageClass("REDUCED_REDUNDANCY");
$fs1->setMeta(array(
array("name" => "width", "field" => "SourceWidth_[itemIndex]"),
array("name" => "height", "field" => "SourceHeight_[itemIndex]"),
array("name" => "category", "value" => "")
));
$as3->setFiles(array($fs1));
$uploader->render();
?>