Uploading to Cloud Storages

Uploading to Amazon S3

When you create an account in the Amazon Web Service (AWS) you obtain your Access Key Id and Secret Access Key. These parameters are used for authentication purposes:

• AWS Access Key Id identifies your AWS account. This key is included in every AWS request to identify the user.
• Secret Access Key is never included into requests and used to sign AWS requests. Each request must contain a valid signature calculated using the Secret Access Key, otherwise authentication will fail.

To store objects Amazon S3 uses buckets which are quite similar to Internet domain names and must be unique within the Amazon S3. For example, if the object named myphotos/holiday.jpg is stored in the mybucket bucket, the URL to access this object is http://mybucket.s3.amazonaws.com/myphotos/holiday.jpg. Every bucket has an associated access control list used to verify whether the sender is able to access this bucket. Each object represents a file and metadata stored within a bucket using a unique key. A bucket and key together uniquely identify each object stored in the Amazon S3.

In the simplest case, to create a new Amazon S3 object you need to send the POST request which identifies the sender and defines the object to be created. It means that Image Uploader can be used for this purpose. Thus, to implement this functionality you can take an advantage of one of the approaches described below.

To be sure that all files are uploaded successfully, you can initiate data integrity check. Find short description of this feature along with code snippets in the Checking Data Integrity part of this article.

Using Image Uploader JavaScript

This approach is more complicated but can be used with any Image Uploader supported server platform. It lies in using the amazonS3Extender class defined in the /Scripts/aurigma.uploader.amazons3.js file. However, before you initialize this class instance you should prepare additional data required for uploading to Amazon S3 storage: the security policy and signature.

The policy is a UTF-8 and Base64 encoded JSON document which specifies the date when this policy expires and conditions the request must meet. The expiration date must be in ISO8601 GMT date format. The conditions are used to validate the uploaded object and include definitions for access control list, bucket, key, and metadata fields (both default and custom).

Here is the code sample that creates a policy document which expires in 100 minutes and defines an access control list option, bucket name, uploaded file, two default fields (width and height), and one custom (author).

public string ConstructPolicy()
{
    StringBuilder policy = new StringBuilder();
    DateTime expDate = DateTime.Now.ToUniversalTime().AddSeconds(6000);

    policy.AppendLine("{ \"expiration\": \"" + expDate.ToString("s") + ".000Z\"");
    policy.AppendLine(", \"conditions\": [");
    policy.AppendLine(" {\"acl\": \"public-read\" }");
    policy.AppendLine(" , {\"bucket\": \"" + _bucket + "\" }");
    policy.AppendLine(" , {\"success_action_status\": \"200\"}");
    policy.AppendLine(" , [\"starts-with\", \"$key\", \"\"]");

    policy.AppendLine(" , [\"starts-with\", \"$x-amz-meta-width\", \"\"]");
    policy.AppendLine(" , [\"starts-with\", \"$x-amz-meta-height\", \"\"]");
    policy.AppendLine(" , [\"starts-with\", \"$x-amz-meta-author\", \"\"]");

    policy.AppendLine("]");
    policy.AppendLine("}");

    // Encode the policy using UTF-8.
    byte[] bytes = Encoding.UTF8.GetBytes(policy.ToString());
    
    // Encode those UTF-8 bytes using Base64.
    return Convert.ToBase64String(bytes);
}

To create a signature, sign the policy with your Secret Access Key using HMAC SHA-1 and encode the SHA-1 signature using Base64. See the code sample below, it demonstrates how to create a signature in .NET.

public string CreateSignature()
{
    _policy = ConstructPolicy();

    // Sign the policy with your Secret Access Key using HMAC SHA-1.
    System.Security.Cryptography.HMACSHA1 hmac = new System.Security.Cryptography.HMACSHA1();
    hmac.Key = System.Text.Encoding.UTF8.GetBytes(_secretAccessKey);

    byte[] signb = hmac.ComputeHash(System.Text.Encoding.UTF8.GetBytes(_policy));

    // Encode the SHA-1 signature using Base64.
    _signature = Convert.ToBase64String(signb);
}

Then initialize an amazonS3Extender object and specify your AWSAccessKeyId and bucket using the accessKeyId and bucket properties. However, if you have a host name associated with an Amazon S3 bucket you should additionally specify this host name using the bucketHostName property.

Then for each converted file add a fileSettings instance specifying:

• access control list which should be equal to the acl value defined in the policy (acl)
• key of Amazon S3 object to be created for this converted file (key); here you can use the ${filename} variable to set a name of this file
• policy constructed and encoded before (policy)
• signature (signature)
• set of predefined and custom metadata fields which will be uploaded within this file (meta); these fields should be defined in the policy

<script type="text/javascript" src="Scripts/aurigma.uploader.js">  </script>
<script type="text/javascript" src="Scripts/aurigma.uploader.amazons3.js">  </script>
<script type="text/javascript">
var u = $au.uploader({
    id: 'Uploader1',
    converters: [
        {mode: '*.*=SourceFile'},
        // ... Other converters
    ]
});
    
var as3 = $au.amazonS3Extender(u);
as3.accessKeyId(_AWSAccessKeyId);
as3.bucket(_bucket);
as3.bucketHostName('http://yourdomain.com/');
as3.converters([
    {
        acl: 'public-read',
        key: '${filename}',
        policy: _policy,
        signature: _signature,
        meta: [
            { name: 'Width', field: 'SourceWidth_[itemIndex]'},
            { name: 'Height', field: 'SourceHeight_[itemIndex]'},
            { name: 'Author', value: 'John Smith'}
        ]
    },
    // ... Other converters
]);

u.writeHtml();
</script>

See the Amazon S3 Documentation for the detailed information on how to upload files to Amazon S3 using POST.

Using Image Uploader ASP.NET

This approach is simpler but it is applicable for ASP.NET applications only. It saves you the trouble of preparing the policy and signature yourself and familiarizing with the Amazon S3 API. You just need to perform the following steps to force Image Uploader to upload files directly to Amazon S3 storage:

1. Initialize an AmazonS3Extender object.
2. Specify your AWSAccessKeyId, secretAccessKey, and bucket as values of the AWSAccessKeyId, SecretAccessKey, and Bucket properties.

3. For each converted file add a FileSettings instance and configure it as follows:

   1. Set acl and key for this file using the Acl and Key properties.
   2. Add as many metadata fields as you like by adding PredefinedMetaProperty and CustomMetaProperty objects to the FileSettings.Meta collection.

<%@ Register assembly="Aurigma.ImageUploader" Namespace="Aurigma.ImageUploader" tagprefix="cc1" %>
<%@ Register assembly="Aurigma.ImageUploader" namespace="Aurigma.ImageUploader.AmazonS3" tagprefix="cc1" %>
<!--... Omitted for brevity ...-->
<form id="form1" runat="server">
    <cc1:Uploader ID="Uploader1" runat="server">
        <Converters>
            <cc1:Converter Mode="*.*=SourceFile" />
            <!-- Other converters -->
        </Converters>
    </cc1:Uploader>
    <cc1:AmazonS3Extender runat="server" ID="AmazonS3Extender1" TargetControlID="Uploader1"
        AWSAccessKeyId="<%$ _AWSAccessKeyId %>"
        Bucket="<%$ _Bucket %>" 
        SecretAccessKey="<%$ _SecretAccessKey %>"
        PolicyExpiration="6000">
        <cc1:FileSettings Acl="public-read" Key="${filename}">
            <cc1:PredefinedMetaProperty Field="SourceWidth_[itemIndex]" Name="width" />
            <cc1:PredefinedMetaProperty Field="SourceHeight_[itemIndex]" Name="height" />
            <cc1:CustomMetaProperty Name="Author" Value="John Smith" />
        </cc1:FileSettings>
        <!-- Other files -->
    </cc1:AmazonS3Extender>
</form>

Using Image Uploader PHP

The present approach is to use Image Uploader PHP to upload files to Amazon S3 storage. To implement this functionality Image Uploader PHP exposes the same name classes and the same object model as the ASP.NET control (see the section above). So, to extend Image Uploader with a direct upload to Amazon S3 storage, follow the steps below:

1. Create an AmazonS3Extender object.
2. Specify your AWSAccessKeyId, bucket, and secretAccessKey as values of the AWSAccessKeyId, Bucket, and SecretAccessKey properties respectively.

3. For each converted file add a FileSettings instance and configure it as follows:

   1. Set acl and key for this file using the Acl and Key properties.
   2. Add predefined and custom metadata fields to the FileSettings.Meta array.

require_once "ImageUploaderPHP/Uploader.class.php";
require_once "ImageUploaderPHP/AmazonS3Extender.class.php";

$uploader = new Uploader("Uploader1");
$converters = &$uploader->getConverters();
$converter = new Converter();
$converter->setMode("*.*=SourceFile");
$converters[] = $converter;
//Other converters

$as3 = new AmazonS3Extender($uploader);
$as3->setAWSAccessKeyId($AWSAccessKeyId);
$as3->setBucket($Bucket);
$as3->setSecretAccessKey($SecretAccessKey);
$files = &$as3->getFiles();
$file = new FileSettings();
$file->setAcl("public-read");
$file->setKey('${filename}');
$file->setMeta(array(
  array("name" => "Author", "value" => "John Smith"),
  array("name" => "width", "field" => "SourceWidth_[itemIndex]"),
  array("name" => "height", "field" => "SourceHeight_[itemIndex]")
));
$files[] = $file;
//Other files

$uploader->render();

Checking Upload Integrity

Amazon S3 supports checking integrity of uploaded files using MD5 hash sums. To initiate this check set the AmazonS3Extender.CheckIntegrityCheckIntegrity (ASP.NET)CheckIntegrity (PHP)checkIntegrity (JavaScript) property to true. In this case a Content-MD5 header will be added to your upload, and S3 will verify the MD5 value in the header and actual MD5 value of the uploaded file before storing the file. If the values do not match, S3 will not save the file and will return an error code in the response.

The following code snippet demonstrates how to turn on integrity check.

<%@ Register assembly="Aurigma.ImageUploader" Namespace="Aurigma.ImageUploader" tagprefix="cc1" %>
<%@ Register assembly="Aurigma.ImageUploader" Namespace="Aurigma.ImageUploader.AmazonS3" tagprefix="cc1" %>

<body>
    <form id="form1" runat="server">
        <cc1:Uploader ID="Uploader1" runat="server">
        <%--... Converters and other settings--%>
        </cc1:Uploader>
        <cc1:AmazonS3Extender runat="server" ID="AmazonS3Extender1" TargetControlID="Uploader1"
           CheckIntegrity=True>
        <%--... File settings--%>
        </cc1:AmazonS3Extender>
    </form>
</body>

require_once "ImageUploaderPHP/Uploader.class.php";
require_once "ImageUploaderPHP/AmazonS3Extender.class.php";

$uploader = new Uploader("Uploader1");
//... Converters and other settings
$as3 = new AmazonS3Extender($uploader);
//... Authentication settings
$as3->setCheckIntegrity(true);
//... File settings

$uploader->render();

<script type="text/javascript" src="Scripts/aurigma.uploader.js">  </script>
<script type="text/javascript" src="Scripts/aurigma.uploader.amazons3.js">  </script>
<script type="text/javascript">
var u = $au.uploader({
    id: 'Uploader1',
    //... Converters and other settings
});
    
var as3 = $au.amazonS3Extender(u);
//... Authentication settings
as3.checkIntegrity(true);
//... File settings

u.writeHtml();
</script>

Uploading to Nirvanix

In a common case, to access Nirvanix storage you must provide the username and password you are registered in the service with. Another access parameter is the application key which represents an identifier of the application you are going to access.

By analogy with Amazon S3, to configure Image Uploader to upload files to the Nirvanix service you may use one of the following approaches.

Using Image Uploader JavaScript

This approach is more flexible but requires you to be familiarized with the Nirvanix API. All you need in this case is to send at least two requests to the Nirvanix service, parse returned XML responses, and instantiate a nirvanixExtender object defined in the /Scripts/aurigma.uploader.nirvanix.js file. Each request represents a call to a corresponding Nirvanix method. The URL format for requests is:

http(s)://services.nirvanix.com/ws/{NameSpace}/MethodName.ashx?name=value&name2=value2

When you call some method you get a response in XML format. The default XML response for methods which do not return data is:

<Response>
    <ResponseCode>0</ResponseCode>
</Response>

So, with the nirvanixExtender you can configure Image Uploader to send files directly to the Nirvanix storage from any HTTP-compliant server platform. To implement this functionality go through the following steps:

1. Call the Login method with your appKey, userName, and password:

   /ws/Authentication/Login.ashx?appKey=your_key&userName=your_name&password=your_password

2. Retrieve the sessionToken from the returned response.

3. Call the GetStorageNodeExtended method with the sessionToken obtained on the previous step, size of the file to be uploaded, and the destination folder:

   /ws/IMFS/GetStorageNodeExtended.ashx?sessionToken=your_token&sizeBytes=filesize&destFolderPath=path

4. Retrieve the uploadHost and uploadToken from the returned response.

5. Add a nirvanixExtender object to your Image Uploader configuration. Pass the uploadToken and uploadHost returned by the GetStorageNodeExtended method and the destination folder to this object. Use the uploadToken, uploadHost, and destFolderPath properties for it.

   Note

   The destination folder passed to the GetStorageNodeExtended method must be equal to the one specified with the destFolderPath property.

   JavaScript

   <script type="text/javascript" src="Scripts/aurigma.uploader.js">  </script>
   <script type="text/javascript" src="Scripts/aurigma.uploader.nirvanix.js">  </script>
   <script type="text/javascript">
   var u = $au.uploader({
       id: 'Uploader1',
       converters: [
           {mode: '*.*=SourceFile'},
           // ... Other converters
       ]
   });
   
   var nx = $au.nirvanixExtender(u);
   nx.uploadToken(_uploadToken);
   nx.uploadHost(_uploadHost);
   nx.destFolderPath('/upload/');
   
   u.writeHtml();
   </script>
   

Using Image Uploader ASP.NET

This approach is simpler than the first described. Here you do not need to call Nirvanix methods directly; Image Uploader ASP.NET obtains the required data itself. Thus, you just need to initialize a NirvanixExtender object and specify your appKey, userName, and password as values of the corresponding properties. See the code sample below.

<%@ Register assembly="Aurigma.ImageUploader" Namespace="Aurigma.ImageUploader" tagprefix="cc1" %>
<%@ Register assembly="Aurigma.ImageUploader" namespace="Aurigma.ImageUploader.Nirvanix" tagprefix="cc1" %>
<!--... Omitted for brevity ...-->
<form id="form1" runat="server">
    <cc1:Uploader ID="Uploader1" runat="server">
        <Converters>
            <cc1:Converter Mode="*.*=SourceFile" />
            <!-- Other converters -->
        </Converters>
    </cc1:Uploader>
    <cc1:NirvanixExtender runat="server" ID="NirvanixExtender1" TargetControlID="Uploader1"
        DestFolderPath="/upload/"
        AppKey="<%$ _appKey %>" 
        Password="<%$ _password %>" 
        Username="<%$ _username %>" />
</form>

Using Image Uploader PHP

By analogy with Image Uploader ASP.NET, Image Uploader PHP includes the NirvanixExtender class to support a direct upload to Nirvanix storage. To implement this functionality just set your appKey, userName, and password as values of the AppKey, Username, and Password properties.

require_once "ImageUploaderPHP/Uploader.class.php";
require_once "ImageUploaderPHP/NirvanixExtender.class.php";

$uploader = new Uploader("Uploader1");
$converters = &$uploader->getConverters();
$converter = new Converter();
$converter->setMode("*.*=SourceFile");
$converters[] = $converter;
//Other converters

$nx = new NirvanixExtender($uploader);
$nx->setAppKey($app_key);
$nx->setUsername($username);
$nx->setPassword($password);
$nx->setDestFolderPath("/upload/");

$uploader->render();

See Also

Reference

• AmazonS3Extender ClassAmazonS3Extender (ASP.NET)AmazonS3Extender (PHP)amazonS3Extender (JavaScript)
• FileSettings ClassFileSettings (ASP.NET)FileSettings (PHP)fileSettings (JavaScript)
• NirvanixExtender ClassNirvanixExtender (ASP.NET)NirvanixExtender (PHP)nirvanixExtender (JavaScript)
• POST Field Reference

Manual

• Configuring Files to be Uploaded

Other Resources

• Amazon Simple Storage Service API Reference
• Nirvanix Web Services API Developer's Guide