How to download and process GZip’d data

Wed, Oct 15

HTML browsers, for the longest time, have had support for downloading and expanding Gzip content. This is an ideal way to compress the size of HTML pages because gzip is at its best when compressing text. As you can imagine, compression is even more critical when we look at consuming XML over the mobile network.

Web Servers like Apache will automatically compress static request for HTML pages using gzip if the request includes the following header:

Accepts-Encoding: gzip

Unfortunately, web services are typically not static content. Webservice are usually served up by application servers that don’t implicitly acknowledge the header. As such, compressing the content is left to the developer to take care of “by hand”. Luckily most modern server-side options come with everything you need to compress your data feeds.

If you are a Java developer, as I am, you may notice that the default JDK comes with support for GZipInputStream and GZipOutputStream classes. Below is a simple example of how you could use the GZip support in Java to compress data written to an HttpServletResponse’s OutputStream:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
protected void service(HttpServletRequest request, HttpServletResponse response)
		throws ServletException, IOException 
{
    String xml = doSomethingToGenerateXML();
 
    // Get the response OutputStream ...
    OutputStream out = response.getOutputStream();
 
    // Create a GZipOutputStream using the response OutputStream
    GZipOutputStream gzip = new GZipOutputStream(out);
 
    // Now write out your XML ...
    gzip.write(xml.getBytes());
 
    // ... and close up
    gzip.flush();
    gzip.close();
    out.close();
 
}

Presto, See how easy that was! Now your job isn’t over yet, this is an iPhone tip after all! The magic to using this compressed content in an iPhone application is to simply tell it that the content is compressed using gzip … and it will take care of the rest! You don’t even need to expand the content before you use it because NSURLConnection will do that for you.

To ensure that NSURLConnection understands that the content was compressed with Gzip, be sure to add the following header in the HttpServletResponse:

Content-Encoding: gzip

So back in the servlet example above, you can add this header as follows:

request.setHeader("Content-Encoding", "gzip");

The gains you will get in reduce data packets will translate into a much more responsive network application, a cheaper phone bill, and a grateful user … so all you WebService API providers out there … COMPRESS YOUR XML!!

6 comments

Although not required, it is recommended that you tell the webservice that you are a client capable of supporting gzip content. By default, even though the iPhone NSURLConnection is capable, it doesn’t add the required header to a request that will announce that fact formally.

To add the header to your request on iPhone you can do something like this

NSMutableURLRequest *request = …
[request addValue:@"gzip" forHTTPHeaderField:@"Accepts-Encoding"];

by Rodney Aiglstorfer on Oct 15, 2008. Reply #

What about the inverse, gzip encoding on the phone before sending an HTTP post?

by Aaron Hill on Jan 10, 2009. Reply #

Yes, are there any experiences with compressing the data I’m going to send?

For the Java part, it’s straightforward how to manually decompress the content, but what do I have to insert here to compress the POST data?

(…)
NSMutableURLRequest* request = [[NSMutableURLRequest alloc] init];
[request setURL: serverURL];
[request setHTTPMethod:@"POST"];
[request setValue:@"text/xml; charset=utf-8" forHTTPHeaderField:@"Content-Type"];
[request addValue:@"gzip" forHTTPHeaderField:@"Accepts-Encoding"];

NSString *xmlRequest = [self getTheDataToPostAsXML];
[request setHTTPBody:[xmlRequest dataUsingEncoding:NSUTF8StringEncoding]];

NSURLConnection* connection = [[NSURLConnection alloc] initWithRequest:request delegate:self];
(…)

Thanks for your help,
Pascal

by Pascal on Jan 28, 2009. Reply #

Check this link: http://www.clintharris.net/2009/how-to-gzip-data-in-memory-using-objective-c/

It contains absolutely clear and straight implementation for in-memory compressing NSData with gzip.

by Mike Orlov on May 14, 2009. Reply #

I think the header should state “Accept-Encoding”, not “Accepts-Encoding”.
See http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html
Also, I think you meant to write:

response.setHeader(“Content-Encoding”, “gzip”);

not

request.setHeader(“Content-Encoding”, “gzip”);

Regards,
Florian

by Florian on Mar 15, 2011. Reply #

When the header “Accept-Encoding” is added, would the NSData received from NSURLConnection get decompressed automatically or do we need look at the header to see whaat the content encoding is and call the NSDataAddtions that someone wrote to decompress NSData in memory. I understand NSURLDownload supports gzip.

Thanks for your help in advance

by VSVasan on Mar 18, 2011. Reply #

Leave a Comment