Enabling Multi-threaded Upload via ActiveX/Java Uploader in Other Platforms

By the nature of HTTP, protocol web servers are designed to handle multiple simultaneous requests. While creating a file upload solution we can employ this feature to increase upload speed. The idea is quite simple: divide data into several pieces and upload them simultaneously, each data piece within a separate HTTP request. This trick can yield up to a 50% increase in upload speed if the channel between a client computer and a web server is fast enough. ActiveX/Java Uploader supports this approach, and this topic explains what changes are required to switch your application to multi-threaded upload.

Multi-threaded upload works only if the packages and/or chunks modes are turned on (to do this, the FilesPerPackage property and/or the ChunkSize property must have a positive value). In this case, the uploading files are processed in multiple pieces (by packages or chunks) rather than all at once. While uploading each file in a separate request (FilesPerPackage = 1) is the simplest way to start with multi-threaded upload, we recommend you to read the Sending All Files at Once or in Several Parts in ActiveX/Java Uploader topic. Also the Upload Modes Best Practices topic can help you to find the most appropriate configuration.

Even though ActiveX/Java Uploader supports up to 10 simultaneous threads, in real world use we have found that the optimal number of connections to use is 3-4. Uploading files in a larger number of simultaneous requests can stress your server too much which will result in the elimination of the desired positive effect.

The Saving Uploaded Files in ActiveX/Java Uploader Other Platforms topic explains how to process uploaded files and outlines three approaches for handling files on the server side in such platforms as ASP, JSP, and ColdFusion. Here we will examine each approach and suggest changes required to work in the multi-threaded mode through the following paragraphs:

ActiveX/Java Uploader Configuration

ActiveX/Java Uploader provides the easiest way to enable multi-threaded upload. To perform this just configure the script for initializing an uploader object by specifying the maxConnectionCount property to a positive value, which defines the number of simultaneous connections. Make sure that you enable packages upload as we discussed above.

As server scripts that parse received HTTP POST requests in the discussed platforms do not restore files from chunks automatically, we do not recommend using the chunk upload mode. Otherwise, you will be required to write code for handling chunks.

The snippet below configures ActiveX/Java Uploader to send files in up to 4 simultaneous threads and to place each file in a separate package.

JavaScript
var u = $au.uploader(
{
    id: 'Uploader1',
    uploadSettings: 
    {
        maxConnectionCount: 4,
        filesPerPackage: 1  
    }
});

This configuration script works for the all server-side upload scripts considered in this topic.

Processing Uploaded Data in ASP

ASP does not have standard file upload processing facilities and has different third-party ActiveX components for this purpose. The code sample below demonstrates how to upload client data using the Dundas Upload component which can process requests incrementally.

If you set up ActiveX/Java Uploader to transfer files in the multi-threaded mode, it can turn out that multiple requests arrive to the server and call the script at the same moment. Thus, you should take steps to avoid simultaneous requests which can possibly cause problems in the case of thread-unsafe code. So, to enable multi-threaded upload for this approach, you should review the code for parsing HTTP POST requests and rewrite thread-unsafe segments.

Here you can see several examples of thread-unsafe code:

  • changing a resource that is not supposed to be changed concurrently (e.g. if you write to a file)
  • using thread-unsafe collections
  • using thread-unsafe libraries

For example, if a script that processes HTTP requests writes information about data to a file, there can be multiple concurrent calls, causing simultaneous file writing. To resolve this issue add synchronization to the file accessing code. The following snippet demonstrates how to do this:

ASP
<%@  language="VBScript" %>
<!--#INCLUDE FILE="Config.asp"-->

<%
    Server.ScriptTimeout = 450
    
    'This variable specifies relative path to the folder, where the gallery with uploaded files is located.
    'Do not forget to put slash at the end of the folder name.
    Dim strGalleryPath
    strGalleryPath = "UploadedFiles/"
    
    Sub ProcessDundasUpload
        'Create Dundas Upload object for upload processing.
        Dim objUpload
        set objUpload = Server.CreateObject("Dundas.Upload.2")
        objUpload.UseVirtualDir = False
        objUpload.MaxUploadSize = 1000000000
    
        'Fetch first source file to populate Form collection.
        Dim objSourceFile
        Set objSourceFile = objUpload.GetNextFile
    
        'Get total number of uploaded files (all files are uploaded in a single package).
        Dim intFileCount, i
        intFileCount = CInt(objUpload.Form("PackageFileCount").Value)
    
        'Lock the Application object
        Application.Lock
        'Open (or create) info file
        Dim txtFile, fileInfo
        Set txtFile = Server.CreateObject("Scripting.FileSystemObject")
        Set fileInfo = txtFile.OpenTextFile(Server.MapPath(strGalleryPath & "info.txt"), 8, True)
    
        'Iterate through uploaded data and save the original file, thumbnail, and description.
        For i = 0 To intFileCount - 1
            If i > 0 Then
                Set objSourceFile = objUpload.GetNextFile
            End If
            'Get source file and save it to disk.
            Dim strFileName
            strFileName = objUpload.GetFileName(objSourceFile.OriginalPath)
            objSourceFile.Save Server.MapPath(strGalleryPath)
            objUpload.FileMove Server.MapPath(strGalleryPath & objSourceFile.FileName), _
            Server.MapPath(strGalleryPath & strFileName)
            
            'Save name of an uploaded file in the info file
            fileInfo.WriteLine(strFileName)
        next
    
        fileInfo.Close
        Application.UnLock        
    End Sub

    If Request.ServerVariables("REQUEST_METHOD")= "POST" Then
        ProcessDundasUpload
    End If
%> 

You can find the ActiveX/Java Uploader configuration for this upload script in the previous paragraph.

Processing Uploaded Data in JSP

To upload data in JSP the following script uses the open-source package called Apache Commons-FileUpload which embeds file upload capability to your web applications. The package is developed by the Apache Software Foundation, and you can find it here: http://commons.apache.org/proper/commons-fileupload/

Like in the previous approach, to enable multi-threaded mode for upload data in JSP you should review the code for parsing HTTP POST requests and rewrite thread-unsafe segments. Otherwise, several simultaneous requests may cause problems. You can see examples of thread-unsafe code in the previous example.

The following thread-safe code saves uploaded files and stores information about them to an XML file. This script expects that ActiveX/Java Uploader configuration is the same as mentioned above.

JSP
<%@page import="sun.org.mozilla.javascript.internal.Synchronizer"%>
<%@page import="java.io.PrintWriter"%>
<%@page import="java.io.FileWriter"%>
<%@page import="java.io.RandomAccessFile"%>
<%@page import="java.util.RandomAccess"%>
<%@page import="com.aurigma.web.imageuploader.*"%>
<%@page import="java.util.HashMap"%>
<%@page import="java.util.List"%>
<%@page import="org.apache.commons.fileupload.servlet.ServletFileUpload"%>
<%@page import="org.apache.commons.fileupload.disk.DiskFileItemFactory"%>
<%@page import="org.apache.commons.fileupload.FileItemFactory"%>
<%@page import="org.apache.commons.fileupload.FileItem"%>
<%@page import="java.io.File"%>
<%@page import="java.util.concurrent.locks.*" %>
<%@page contentType="text/html" pageEncoding="UTF-8"%>
<%
    if (!"POST".equalsIgnoreCase(request.getMethod())) 
        return;
    //Synchronized block.
    synchronized(this)
    {
        BaseGallery gallery = new BaseGallery(pageContext);
        
        // Create a factory for disk-based file items.
        FileItemFactory factory = new DiskFileItemFactory(10240, new File(gallery.getTempFilesAbsolutePath()));
        ServletFileUpload upload = new ServletFileUpload(factory);
        
         // Parse request.
        List /* FileItem */ fileItemsList = upload.parseRequest(request);
        
        // Put them in hash table for fast access.
        HashMap<String, FileItem> fileItems = new HashMap<String, FileItem>();
        for (Object item : fileItemsList) 
        {
            FileItem fileItem = (FileItem)item;
            fileItems.put(fileItem.getFieldName(), fileItem);
        }
        
        // Check that all files are uploaded.
        FileItem requestComplete = fileItems.get("RequestComplete");
        if (requestComplete == null || !"1".equals(requestComplete.getString())) 
            return;
        
        // Clear previously uploaded files.
        if ("0".equals(fileItems.get("PackageIndex").getString())) 
            gallery.empty();
        
        //Get total number of uploaded files.
        int fileCount = Integer.parseInt(fileItems.get("PackageFileCount").getString());
        
        //Iterate through uploaded data and save the original file and description.
        String absSourcePath = gallery.getUploadedFilesAbsolutePath();
        
        for (int i = 0; i < fileCount; i++) 
        {
            //Get source file and save it to disk.
            FileItem sourceFileItem = (FileItem) fileItems.get("File0_" + i);
            String fileName = Utils.getSafeFileName(absSourcePath, sourceFileItem.getName());
            File sourceFile = new File(absSourcePath + File.separator + fileName);
            sourceFileItem.write(sourceFile);
        	
            HashMap<String, String> fileInfo = new HashMap<String, String>(5);
            fileInfo.put("name", fileItems.get("SourceName_" + i).getString());
            fileInfo.put("source", sourceFile.getName());
            gallery.addFile(fileInfo);	
        }
        // Save xml
        gallery.save();
    }
%>

Processing Uploaded Data in ColdFusion

ColdFusion exposes the simple file upload processing facility. The tag named <cffile> provides all the necessary functionality: where you can you specify such attributes as the field name, the folder where you are going to save it, and what to do when file name collision occurs.

ColdFusion allows you to process multiple requests at a time. Therefore the server can attempt to access the same information or resources concurrently in the case of several requests. Thus, the resulting data can be inconsistent or an error may occur. To avoid this you should review the code for parsing HTTP POST requests and rewrite thread-unsafe segments. You can see examples of thread-unsafe code here.

The following snippet demonstrates how to synchronize the application using <cflock> tag which controls simultaneous access to ColdFusion code. This script expects that ActiveX/Java Uploader configuration is the same as mentioned above.

ColdFusion
<cfprocessingdirective pageEncoding="utf-8" />
<!--- Check if it is POST request --->
<cfif NOT CGI.REQUEST_METHOD is "POST">
    <cfabort />
</cfif>

<!--- Check that all files are uploaded. 
	"RequestComplete" field is the last field in the POST request from Image Uploader.
	If it exists, then upload process was not aborted. --->
<cfif NOT IsDefined("FORM.RequestComplete")>
    <cfabort />
</cfif>
	
<cfset galleryPath="../UploadedFiles/" />
<cfset absGalleryPath="#ExpandPath(galleryPath)#" />
<cfset xmlFileName="files.xml" />
<!---Lock the code block.--->
<cflock timeout="5">

    <!---Create XML file which will keep information about files. --->
    <cfset descriptions=XmlParse("<files></files>") />
    <cffile action="write" file="#absGalleryPath##xmlFileName#" 
        output="#toString(descriptions)#" />

    <!---Get total number of uploaded files (all files are uploaded in a single package) and
        iterate through uploaded data and save the original file, thumbnail, and description.--->
    <cfset fileCount=(#Form.PackageFileCount#-1) />
    <cfloop index="i" from="0" to="#fileCount#">
        <cfset name="#Form["SourceName_#i#"]#"/>
        <!--- Get source file and save it to disk. --->
        <cffile action="UPLOAD" filefield="File0_#i#" 
            destination="#absGalleryPath#" 
            nameconflict="MakeUnique"/>
        <cfset sourceFileName="#serverFile#" />
	
        <!---Save file info.--->
        <cfset xmlFile=XmlElemNew(descriptions, "file") />
        <cfset xmlFile.XmlAttributes.source=sourceFileName />
        <cfset xmlFile.XmlAttributes.name=Form["SourceName_#i#"] />
        <cfset ArrayAppend(descriptions.files.XmlChildren, xmlFile) />
    </cfloop>
    <cffile action="write" file="#absGalleryPath##xmlFileName#" 
        output="#toString(descriptions)#" />
    <cfoutput>#fileCount#</cfoutput>
</cflock>

See Also

Reference

Manual