Web Services (ASMX 1.1) Performance Guidelines - Bulk Data Transfer

From Guidance Share

Jump to: navigation, search

- J.D. Meier, Srinath Vasireddy, Ashish Babbar, and Alex Mackman


Contents

Consider using a Byte Array Web Method Parameter

With this approach, you pass a byte array as a method parameter. An additional parameter typically specifies the transfer data size. This is the easiest approach, and it supports cross-platform interoperability. However, it has the following issues:

If a failure occurs midway through the transfer, you need to start again from the beginning. If the client passes an arbitrary amount of data that exceeds your design limitations, you run the risk of running out of memory or exceeding your CPU thresholds on the server. Your Web service is also susceptible to denial of service attacks. Note that you can limit the maximum SOAP message size for a Web service by using the maxRequestLength setting in the <httpRuntime> section of the Web.config file. In the following example, the limit is set to 8 KB.

  <configuration>
    <system.web>
      <httpRuntime maxRequestLength="8096"
          useFullyQualifiedRedirectUrl="true"
          executionTimeout="45"/>
    </system.web>
  </configuration>


Base 64 Encoding For binary data transfer, you can use Base 64 to encode the data. Base 64 encoding is suitable for cross-platform interoperability if your Web service has a heterogeneous client audience.

This approach is more suitable if your data isn't large and the encoding/decoding overhead and size of the payload are not of significant concern. For large-sized data, you can implement a WSE filter and various compression tools to compress the message before sending it over the wire.

References

For more information about Base 64 encoding and decoding, see:


Consider returning a URL from the Web Service

Returning a URL from the Web service is the preferred option for large file downloads. With this approach, you return a URL to the client, and the client then uses HTTP to download the file.

You can consider using the Background Intelligent Transfer Service (BITS), a Windows service, for this purpose. For more information about BITS, see the MSDN article, "Write Auto-Updating Apps with .NET and the Background Intelligent Transfer Service API" by Jason Clark, at http://msdn.microsoft.com/msdnmag/issues/03/02/BITS/.

If you need to use BITS for your .NET application for uploading and downloading of files, you can use the Updater Application Block. For more information, see the MSDN article "Updater Application Block for .NET" at http://msdn.microsoft.com/library/en-us/dnbda/html/updater.asp.

Although returning a URL works for downloads, it is of limited use for uploads. For uploads, you must call the Web service from an HTTP server on the Internet, or the Web service will be unable to resolve the supplied URL.


Consider using Streaming

If you need to transfer large amounts of data (several megabytes, for example) from a Web method, consider using streaming. If you use streaming, you do not need to buffer all of the data in memory on the client or server. In addition, streaming allows you to send progress updates from a long-running Web service operation to a client that is blocked waiting for the operation to return.

In most cases, data is buffered on both the server and client. On the server side, serialization begins after the Web method has returned, which means that all of the data is usually buffered in the return value object. On the client side, the deserialization of the entire response occurs before the returned object is handed back to the client application, again buffering data in memory.

You can stream data from a Web service in two ways:

  • Implementing IList
  • Implementing IXmlSerializable


Implementing IList

The XmlSerializer has special support for types that implement IList whereby it obtains and serializes one list item at a time. To benefit from this streaming behavior, you can implement IList on your return type, and stream out the data one list item at a time without first buffering it.

While this approach provides streaming and the ability to send progress updates, it forces you to work with types that implement IList and to break data down into list items.

Note The serializer is still responsible for serializing and deserializing each individual list item. Another disadvantage is that the progress is reported from the returned type. This means that the instance of your type that implements IList used on the client side must be able to communicate progress to the client application while the serializer calls IList.Add. While this is achievable, it is not ideal.

The following server and client code samples show how to implement this approach. The server code is shown below.


  // Server code
  // If the server wants to return progress information, the IList 
  indexer would 
  // have to know the current progress and return a value indicating that 
  progress 
  // when it is called.
  public class MyList : IList
  {
    int progress=0;
    public object this[int index]
    {
      get
      {
        // Pretend to do something that takes .5 seconds
        System.Threading.Thread.Sleep(500);
        if (progress <= 90)
          return progress+=10;
        else
          return "Some data goes here";
      }
      set
      {
        // TODO:  Add setter implementation
      }
    }
     ... other members omitted
  }
  [WebMethod]
  public MyList DoLongOperation()
  {
    // To prevent ASP.NET from buffering the response, the WebMethod must set 
    // the BufferOutput property to false
    HttpContext.Current.Response.BufferOutput=false;
    return new MyList();
  }

By using the above code, the Web service response is streamed out and consists of 10 progress data points (10 to 100), followed by a string of data.

The corresponding method on the client proxy class must return a type that implements IList. This type must know how to stream items as they are added to the list, and, if required, how to report progress information as it is retrieved from the stream. The relevant MyList member on the client is the Add method:

Note With .NET Framework 1.1 you must manually edit the generated proxy code because Wsdl.exe and the Add Web Reference option in Visual Studio.NET generate a proxy class with a method that returns an object array. You need to modify this to return a type that implements IList. The client code is shown below.


  // Client code
  public class MyList : IList
  {
    public int Add(object value)
    {
      if (progress < 100)
      {
        progress = Convert.ToInt32(value);
        Console.WriteLine("Progress is {0}",progress);
      }
      else
      {
        Console.WriteLine("Received data: {0}",value);
      }
      return 0;
    }
 }

The client's proxy class then contains a method that returns MyList as shown below:

  public ProgressTestClient.MyList DoLongOperation() 
  { 
    ... code omitted 
  }


Implementing IXmlSerializable

Another possible approach is to create a type that implements IXmlSerializable, and return an instance of this type from your Web method and the client proxy's method. This gives you full control of the streaming process. On the server side, the IXmlSerializable type uses the WriteXml method to stream data out:

This solution is slightly cleaner than the previous approach because it removes the arbitrary restriction that the returned type must implement IList. However, the programming model is still a bit awkward because progress must be reported from the returned type. Once again, in.NET Framework 1.1, you must also modify the generated proxy code to set the correct method return type.

The following code sample shows how to implement this approach.


  public class ProgressTest : IXmlSerializable
  {
    public void WriteXml(System.Xml.XmlWriter writer)
    {
      int progress=0;
      while(progress <= 100)
      {
        writer.WriteElementString("Progress",
                               "http://progresstest.com", 
  progress.ToString());
        writer.Flush();
        progress += 10;
        // Pretend to do something that takes 0.5 second
        System.Threading.Thread.Sleep(500);
      }
      writer.WriteElementString("TheData",
                              "http://progresstest.com","Some data 
  goes here");
    }
  }

The Web method must disable response buffering and return an instance of the ProgressTest type, as shown below.


  [WebMethod]
  public ProgressTest DoLongOperation2()
  {
    HttpContext.Current.Response.BufferOutput=false;
    return new ProgressTest();
  }


References For more information about bulk data transfer, see:

Personal tools