This documentation is for the old version. Go to the latest Upload Suite docs

Uploading Files Sequentially and Concurrently

Image Uploader allows uploading files in different modes. You have the following options:

Sending All Files at Once or in Several Parts
Sending Requests Concurrently
- Using Several Concurrent Connections
- Benefits and Disadvantages of the Concurrent Upload
Sending Files to Several Servers
- Using Multiple Servers
- Benefits and Disadvantages of Using Multiple Servers

Each mode has its own advantages and disadvantages. Let's look at them more closely.

Sending All Files at Once or in Several Parts

Single Request Mode

The figure below demonstrates how Image Uploader uploads all the files at once. First, it creates one POST request containing all files and fields the user is going to upload (a multipart/form-data block). After that this request is submitted to the server and Image Uploader waits for a response from the server stating that the upload was completely finished.

Figure 1. Uploading all files at once.

Set the FilesPerOnePackageCount property to -1 to make Image Uploader work in this mode.

Multiple Requests Mode

The next figure demonstrates upload in separate parts sequentially. Image Uploader creates a request containing only one or several files (including their thumbnails and other accompanying fields). After that the request is submitted to the server, and Image Uploader waits for a confirmation of a successful upload of this part. When this confirmation is received, the next part is uploaded in the same manner. This procedure is repeated until all the selected files are uploaded.

Figure 2. Uploading files in separate parts.

Set the FilesPerOnePackageCount property to a positive integer (number of files in a request) to make Image Uploader work in this mode.

Comparison of These Upload Modes

When choosing between these upload modes, keep in mind the following:

Memory Usage on Client Side. Generally speaking, uploading all the files at once should consume a lot of memory to be able to upload a large number of files. The multipart/form-data block is an indivisible byte stream, and Image Uploader should commit enough memory to be able to prepare it. So, in case of a single POST request, the control should have a memory block containing all selected files, and in multiple requests mode, it only needs a memory block large enough to contain the part which is currently being uploaded. However, as Image Uploader caches the request to disk using a temporary file, the difference disappears. That means, client-side memory usage is equal for both of these upload modes.
Memory Usage on Server Side. When data is being uploaded, the server accepts the uploaded multipart/form-data block and begins to process it. Different server platforms process uploads in a different way. Some of them accumulate the entire block in memory and, when upload is finished, start splitting it into files, etc. Others work in a more efficient way: they flush uploaded data into temporary files until the upload completes. If it is not the case, and the user uploads data in a huge block (for example, 100 files at the same time), you may get unacceptable memory usage.
See the Writing Server-Side Upload Code topic to get more information on how upload is processed by components of different platforms. If the component you use is not optimized for uploading huge amounts of data, use the multiple requests upload mode.
Note
Reducing resource usage on the server is much more important than on the client side, because, otherwise, it can cause a dramatic performance fall of the entire Web server.
Script Execution Timeout. When a single request mode is used, the script, which processes the data, may work quite for a long time. If the file count is big enough, the script may even abort because of timeout. Increasing timeouts is rather unsafe because it becomes easier to overload the server. That is why upload in parts may help.
Total Time of Upload. Upload consists of three stages: establishing a connection, upload itself, and waiting for a confirmation of success. In the single request upload mode Image Uploader performs connection initialization and finalization only once, whereas in the multiple requests mode it repeats these actions for every file. That is why the latter mode has some time overhead in comparison to the former one. However, usually this difference is not big enough.
Upload Recovery. Sometimes the upload process can be broken because of some external reasons (for example, physical connection problems, timeout, etc). If the single request upload mode is used, the files which were uploaded before the connection was lost will not be saved on the server. It happens because multipart/form-data blocks must be integral and do not support upload resuming. On the opposite, if you use the multiple requests mode, such files are saved, so the user does not need to re-upload them. For more details on upload recovery, refer to the Configuring Automatic Upload Recovery topic.
Use of Multiple Connections. If you set up Image Uploader to operate in the multiple requests mode, you can also make it use several HTTP connections. This way, several parts will be uploaded in parallel and it will take less time to deliver all the files to the server. Also, Image Uploader can establish connections to different servers, and that reduces the processing load on each server. You can read more about upload using several HTTP connections in the Sending Files to Several Servers section later in this topic.
Clarity of Server-Side Business Logic. When the user uploads files using the multiple requests mode, for the server script it looks as if the user has selected and sent each file separately (however the user may have sent all images at the same time). Sometimes it is necessary to know, either these files have been sent as a single block, or the user has sent each file separately. If this is the case, you will have to make the business logic of your server scripts more complicated. To avoid this, use the single request upload mode. However, if you do not need to know, whether the files were sent in a batch, you can use the advantages of the multiple requests mode.

Sending Requests Concurrently

If you choose to upload files in multiple requests, you have an option to send requests concurrently. Of course, you can still send them sequentially, in this case, the upload procedure is described in the Multiple Requests Mode section earlier in this topic.

Using Several Concurrent Connections

Like in the sequential upload, Image Uploader creates a request containing specified number of files and submits it to the server. But in this case Image Uploader does not wait for upload completion of this part before sending another one. The control opens as many connections as you specify and uses them for upload in parallel.

To make Image Uploader work in this mode, set the FilesPerOnePackageCount property to a positive integer (number of files in a request) and the MaxConnectionCount property to the maximum allowed number of parallel upload streams.

Benefits and Disadvantages of the Concurrent Upload

When choosing the concurrent upload mode, keep in mind the following:

Network Usage. Usually when opening several HTTP connections, the upload of files completes faster, as it is performed in parallel. However, if too many connections are opened, it is possible to overload the network, consuming all bandwidth available for the client or for the server. In this case, network collision rate increases, and requests are being sent slower. In the worst situation, concurrent upload may even not give the user essential speed gain. Also, the server can open only a finite number of connections at any given moment. If this limit is exceeded, other clients will not be able to send data.
Server Load. Several HTTP connections established from one client are treated by the server as connections from several clients. That is, several instances of upload processing script will be run at the same time, and each of the instances will require some resources. All in all, it may noticeably slow down the server performance. So again, do not allow too many connections.

In general, concurrent upload completes faster. But based on the above remarks, it is recommended to limit the number of upload streams from each client with 3 or 4 connections, especially if you do not split the load between several servers.

Sending Files to Several Servers

Using Multiple Servers

No matter, whether files are uploaded sequentially or concurrently, the data may be sent to several different servers, if you use the multiple requests mode. The following figure demonstrates how the files are distributed between different servers. When a new upload session is started, the request will be sent to the server which is specified next to the last one used.

Figure 3. Uploading files concurrently to multiple servers.

To make Image Uploader work in this mode, set the FilesPerOnePackageCount property to a positive integer (number of files in a request) and specify URLs to your servers in the Action property, separating them with semicolons. You may optionally set the MaxConnectionCount property to the maximum allowed number of parallel upload streams to enable concurrent upload.

Benefits and Disadvantages of Using Multiple Servers

When choosing to upload files to multiple servers, keep in mind the following:

Server Load. This feature is used to split the processing load between multiple servers. In this case, less resources are required for request processing on each server, ans that speeds up the overall data processing. And if load splitting is used along with the concurrent upload, it will be the most efficient combination.
Server-Side Deployment Costs. Proper load balancing imposes additional requirements to the server-side infrastructure - availability of several servers, a dedicated database server, etc. Also, load balancing should be taken into account at the application design stage. And if it was not, later attempts to introduce new resources may lead to unreasonable business logic complication.