PUT, DELETE and Digest Auth in Python
Mark Baker notes that the NewsGator REST API doesn't put any operations into the URL. I believe Mark's correct, that was something I conciously tried to avoid. For my purposes GET, POST, and DELETE were really enough, though there were some operations that were convenient to attach to PUT (for example, replacing vs. merging subscription lists; the only difference is the HTTP verb). In fact, there's been a REST version of the API since day 1, deployed alongside the SOAP version, I just never documented it because it needed a redesign. One of the glaring issues was that the URL line contained an op parameter.
When I was doing the design, I did worry about client side support for HTTP verbs other than GET and POST, but it seemed like most languages were pretty good about support. So I included a couple of hacks to get around possible lack of support for a DELETE verb, and I think that you could get by with just GET and PUT. Actually, I had to fall back on that hack when I was writing the sample code.
Python was one of the environments I thought would support PUT and DELETE, and apparently it does, if you use httplib. But the problem with httplib is that authentication support is poor (well, non-existent were it not for the fact that HTTP Basic is nearly trivial to implement). It turns out that there are two problems to solve when connecting to NewsGator: how do one use PUT and DELETE, and how does one authenticate securely?
Basic over SSL is one option, but I regard that as a pain, especially for code under development where one might want to take a network trace of the activity to see what's happening. So the question becomes "how does one do Digest auth?", and given those two questions, you now have two answers: httplib for the first, and urllib2 for the second. I found this surprising. Given the overall architecture of urllib2, which seems designed for a maximum of flexibility, I found it a little amazing that it couldn't support PUT and DELETE.
It turns out that the problem is that urllib2.Request doesn't let you set the method explicitly. Instead, it makes an assumption in the get_method() call: it returns "POST" if self.has_data() returns TRUE, otherwise it returns "GET". Since Python doesn't seal classes, it seemed like the easiest way around this was to write something like this:
class HttpExtendedRequest(urllib2.Request):
def __init__(self, url, data=None, headers={},
origin_req_host=None, unverifiable=False):
urllib2.Request.__init__(self,url,data,headers,origin_req_host,unverifiable)
def get_method(self):
if self.has_data():
return "PUT"
else:
return "DELETE"
if __name__ == '__main__':
pm = urllib2.HTTPPasswordMgrWithDefaultRealm()
# assuming that the credentials are literally username:password
pm.add_password(None,'www.example.com','username', 'password')
ah = urllib2.HTTPDigestAuthHandler(pm)
opener = urllib2.build_opener()
# will issue an HTTP DELETE
request = HttpExtendedRequest('http://www.example.com/somewhere')
response = opener.open(request)
This will submit a DELETE, but the problem with this is that in earlier releases of Python 2.4, digest authentication fails. HTTP Digest requires the HTTP method as part of the digest, and urllib2.HttpDigestAuthHandler (actually, its superclass AbstractDigestAuthHandler) makes the same assumption as Request: in the get_authorization method, it assumes that if the Request object has data, it's a POST, otherwise, it's a GET. That seemed a little silly, given that you can ask a Request for it's method - we just overrode that in the example above, but AbstractDigestAuthHandler has its own ideas about HTTP methods. It turns out that when I went to check, this had been fixed about a year ago, and Python 2.4.3 includes the patch. So in 2.4.3 you can happily use this workaround.