>Currently a system I’m working on takes HTTP requests that it needs to send, but doesn’t really care much about the response, and tosses them over to a POE HTTP client that just sends in the background. The problem here is that it does each request concurrently. Normally that’s not an issue at all, but as it turns out if the web server you are sending these too gets overloaded and starts taking a long time to respond to request, all of a sudden you are driving up the load on my app server because of how many backlogged requests I’ve got danging concurrently.

Let’s find a solution for this.  Not a big deal, I don’t really care about the requests much so let’s just queue them up.  So rather than having the generic http ua session tossing things like wild, wrote up a quick WebClient module that maintains a queue of requests and handles them.  Since some requests are more important to others, let’s use a priority queue.  Simple to implement, but POE::Queue comes out of the box since we are already using POE, so we go with that.  For the HTTP client, I’m trying out POE::Declare::HTTP::Client.  I’ve had some issues with the generic component http client, so we’ll see if we fare any better with this one.

So…

Simple API. *check*
Queued requests to prevent overloading. *check*
Priority aware to take care of important things first. *check*
Optional callbacks in case we do care about the response. *check*

Sounds like a winner.  Writing up some test cases now, but looks good and functional.  Thanks CPAN for making this pretty damn easy to throw together.