Friday, January 8, 2010

urllib2.HTTPErrorProcessor

With code similar to that I posed in Asynchronous HTTP Request, I was occasionally getting empty responses to my requests. When I added urllib2.HTTPErrorProcessor to the inheritance list for MyHandler, the problem went away. My guess is the server was generating a 503 Service Unavailable responses and my client code wasn't handling it. How one was supposed to know to do this from the documentation, I am unsure. I'm guessing that if the server might provide a redirect for your url, you would also want to inherit from urllib2.HTTPRedirectHandler.

1 comment:

  1. What about calling opener.add_handler using the default urllib2.HTTPxxxHandler() classes?

    I hadn't thought of using multiple inheritance in my HTTPRedirectHandler class but seems like an interesting idea and would certainly allow you to place all this "handler" logic in one class.

    ReplyDelete