By the nature of HTTP, protocol web servers are designed to handle multiple simultaneous requests. While creating a file upload solution we can employ this feature to increase upload speed. The idea is quite simple: divide data into several pieces and upload them simultaneously, each data piece within a separate HTTP request. This trick can yield up to a 50% increase in upload speed if the channel between a client computer and a web server is fast enough. ActiveX/Java Uploader supports this approach, and this topic explains what changes are required to switch your application to multi-threaded upload.
Multi-threaded upload works only if the packages and/or chunks modes are turned on (to do this, the
FilesPerPackage property and/or the
ChunkSize property must have a positive value).
In this case, the uploading files are processed in multiple pieces (by packages or chunks) rather than all at once.
While uploading each file in a separate request (FilesPerPackage
= 1
) is the simplest way to start with multi-threaded upload, we recommend you to read the
Sending All Files at Once or in Several Parts in ActiveX/Java Uploader topic.
Also the Upload Modes Best Practices topic can help you to find the most appropriate configuration.
Even though ActiveX/Java Uploader supports up to 10 simultaneous threads, in real world use we have found that the optimal number of connections to use is 3-4. Uploading files in a larger number of simultaneous requests can stress your server too much which will result in the elimination of the desired positive effect.
The Saving Uploaded Files in ActiveX/Java Uploader Other Platforms topic explains how to process uploaded files and outlines three approaches for handling files on the server side in such platforms as ASP, JSP, and ColdFusion. Here we will examine each approach and suggest changes required to work in the multi-threaded mode through the following paragraphs:
ActiveX/Java Uploader provides the easiest way to enable multi-threaded upload. To perform this just configure the script for initializing an uploader object by specifying the maxConnectionCount property to a positive value, which defines the number of simultaneous connections. Make sure that you enable packages upload as we discussed above.
As server scripts that parse received HTTP POST requests in the discussed platforms do not restore files from chunks automatically, we do not recommend using the chunk upload mode. Otherwise, you will be required to write code for handling chunks.
The snippet below configures ActiveX/Java Uploader to send files in up to 4 simultaneous threads and to place each file in a separate package.
var u = $au.uploader( { id: 'Uploader1', uploadSettings: { maxConnectionCount: 4, filesPerPackage: 1 } });
This configuration script works for the all server-side upload scripts considered in this topic.
ASP does not have standard file upload processing facilities and has different third-party ActiveX components for this purpose. The code sample below demonstrates how to upload client data using the Dundas Upload component which can process requests incrementally.
If you set up ActiveX/Java Uploader to transfer files in the multi-threaded mode, it can turn out that multiple requests arrive to the server and call the script at the same moment. Thus, you should take steps to avoid simultaneous requests which can possibly cause problems in the case of thread-unsafe code. So, to enable multi-threaded upload for this approach, you should review the code for parsing HTTP POST requests and rewrite thread-unsafe segments.
Here you can see several examples of thread-unsafe code:
For example, if a script that processes HTTP requests writes information about data to a file, there can be multiple concurrent calls, causing simultaneous file writing. To resolve this issue add synchronization to the file accessing code. The following snippet demonstrates how to do this:
<%@ language="VBScript" %> <!--#INCLUDE FILE="Config.asp"--> <% Server.ScriptTimeout = 450 'This variable specifies relative path to the folder, where the gallery with uploaded files is located. 'Do not forget to put slash at the end of the folder name. Dim strGalleryPath strGalleryPath = "UploadedFiles/" Sub ProcessDundasUpload 'Create Dundas Upload object for upload processing. Dim objUpload set objUpload = Server.CreateObject("Dundas.Upload.2") objUpload.UseVirtualDir = False objUpload.MaxUploadSize = 1000000000 'Fetch first source file to populate Form collection. Dim objSourceFile Set objSourceFile = objUpload.GetNextFile 'Get total number of uploaded files (all files are uploaded in a single package). Dim intFileCount, i intFileCount = CInt(objUpload.Form("PackageFileCount").Value) 'Lock the Application object Application.Lock 'Open (or create) info file Dim txtFile, fileInfo Set txtFile = Server.CreateObject("Scripting.FileSystemObject") Set fileInfo = txtFile.OpenTextFile(Server.MapPath(strGalleryPath & "info.txt"), 8, True) 'Iterate through uploaded data and save the original file, thumbnail, and description. For i = 0 To intFileCount - 1 If i > 0 Then Set objSourceFile = objUpload.GetNextFile End If 'Get source file and save it to disk. Dim strFileName strFileName = objUpload.GetFileName(objSourceFile.OriginalPath) objSourceFile.Save Server.MapPath(strGalleryPath) objUpload.FileMove Server.MapPath(strGalleryPath & objSourceFile.FileName), _ Server.MapPath(strGalleryPath & strFileName) 'Save name of an uploaded file in the info file fileInfo.WriteLine(strFileName) next fileInfo.Close Application.UnLock End Sub If Request.ServerVariables("REQUEST_METHOD")= "POST" Then ProcessDundasUpload End If %>
You can find the ActiveX/Java Uploader configuration for this upload script in the previous paragraph.
To upload data in JSP the following script uses the open-source package called Apache Commons-FileUpload which embeds file upload capability to your web applications. The package is developed by the Apache Software Foundation, and you can find it here: http://commons.apache.org/proper/commons-fileupload/
Like in the previous approach, to enable multi-threaded mode for upload data in JSP you should review the code for parsing HTTP POST requests and rewrite thread-unsafe segments. Otherwise, several simultaneous requests may cause problems. You can see examples of thread-unsafe code in the previous example.
The following thread-safe code saves uploaded files and stores information about them to an XML file. This script expects that ActiveX/Java Uploader configuration is the same as mentioned above.
<%@page import="sun.org.mozilla.javascript.internal.Synchronizer"%> <%@page import="java.io.PrintWriter"%> <%@page import="java.io.FileWriter"%> <%@page import="java.io.RandomAccessFile"%> <%@page import="java.util.RandomAccess"%> <%@page import="com.aurigma.web.imageuploader.*"%> <%@page import="java.util.HashMap"%> <%@page import="java.util.List"%> <%@page import="org.apache.commons.fileupload.servlet.ServletFileUpload"%> <%@page import="org.apache.commons.fileupload.disk.DiskFileItemFactory"%> <%@page import="org.apache.commons.fileupload.FileItemFactory"%> <%@page import="org.apache.commons.fileupload.FileItem"%> <%@page import="java.io.File"%> <%@page import="java.util.concurrent.locks.*" %> <%@page contentType="text/html" pageEncoding="UTF-8"%> <% if (!"POST".equalsIgnoreCase(request.getMethod())) return; //Synchronized block. synchronized(this) { BaseGallery gallery = new BaseGallery(pageContext); // Create a factory for disk-based file items. FileItemFactory factory = new DiskFileItemFactory(10240, new File(gallery.getTempFilesAbsolutePath())); ServletFileUpload upload = new ServletFileUpload(factory); // Parse request. List /* FileItem */ fileItemsList = upload.parseRequest(request); // Put them in hash table for fast access. HashMap<String, FileItem> fileItems = new HashMap<String, FileItem>(); for (Object item : fileItemsList) { FileItem fileItem = (FileItem)item; fileItems.put(fileItem.getFieldName(), fileItem); } // Check that all files are uploaded. FileItem requestComplete = fileItems.get("RequestComplete"); if (requestComplete == null || !"1".equals(requestComplete.getString())) return; // Clear previously uploaded files. if ("0".equals(fileItems.get("PackageIndex").getString())) gallery.empty(); //Get total number of uploaded files. int fileCount = Integer.parseInt(fileItems.get("PackageFileCount").getString()); //Iterate through uploaded data and save the original file and description. String absSourcePath = gallery.getUploadedFilesAbsolutePath(); for (int i = 0; i < fileCount; i++) { //Get source file and save it to disk. FileItem sourceFileItem = (FileItem) fileItems.get("File0_" + i); String fileName = Utils.getSafeFileName(absSourcePath, sourceFileItem.getName()); File sourceFile = new File(absSourcePath + File.separator + fileName); sourceFileItem.write(sourceFile); HashMap<String, String> fileInfo = new HashMap<String, String>(5); fileInfo.put("name", fileItems.get("SourceName_" + i).getString()); fileInfo.put("source", sourceFile.getName()); gallery.addFile(fileInfo); } // Save xml gallery.save(); } %>
ColdFusion exposes the simple file upload processing facility. The tag named <cffile> provides all the necessary functionality: where you can you specify such attributes as the field name, the folder where you are going to save it, and what to do when file name collision occurs.
ColdFusion allows you to process multiple requests at a time. Therefore the server can attempt to access the same information or resources concurrently in the case of several requests. Thus, the resulting data can be inconsistent or an error may occur. To avoid this you should review the code for parsing HTTP POST requests and rewrite thread-unsafe segments. You can see examples of thread-unsafe code here.
The following snippet demonstrates how to synchronize the application using <cflock> tag which controls simultaneous access to ColdFusion code. This script expects that ActiveX/Java Uploader configuration is the same as mentioned above.
<cfprocessingdirective pageEncoding="utf-8" /> <!--- Check if it is POST request ---> <cfif NOT CGI.REQUEST_METHOD is "POST"> <cfabort /> </cfif> <!--- Check that all files are uploaded. "RequestComplete" field is the last field in the POST request from Image Uploader. If it exists, then upload process was not aborted. ---> <cfif NOT IsDefined("FORM.RequestComplete")> <cfabort /> </cfif> <cfset galleryPath="../UploadedFiles/" /> <cfset absGalleryPath="#ExpandPath(galleryPath)#" /> <cfset xmlFileName="files.xml" /> <!---Lock the code block.---> <cflock timeout="5"> <!---Create XML file which will keep information about files. ---> <cfset descriptions=XmlParse("<files></files>") /> <cffile action="write" file="#absGalleryPath##xmlFileName#" output="#toString(descriptions)#" /> <!---Get total number of uploaded files (all files are uploaded in a single package) and iterate through uploaded data and save the original file, thumbnail, and description.---> <cfset fileCount=(#Form.PackageFileCount#-1) /> <cfloop index="i" from="0" to="#fileCount#"> <cfset name="#Form["SourceName_#i#"]#"/> <!--- Get source file and save it to disk. ---> <cffile action="UPLOAD" filefield="File0_#i#" destination="#absGalleryPath#" nameconflict="MakeUnique"/> <cfset sourceFileName="#serverFile#" /> <!---Save file info.---> <cfset xmlFile=XmlElemNew(descriptions, "file") /> <cfset xmlFile.XmlAttributes.source=sourceFileName /> <cfset xmlFile.XmlAttributes.name=Form["SourceName_#i#"] /> <cfset ArrayAppend(descriptions.files.XmlChildren, xmlFile) /> </cfloop> <cffile action="write" file="#absGalleryPath##xmlFileName#" output="#toString(descriptions)#" /> <cfoutput>#fileCount#</cfoutput> </cflock>