Messaging
I have been thinking a lot lately about using messaging as a way to implement a crawl-store-index-search kind of system, notably to decouple the processes from each other, making them more asynchronous, and more resilient to spikes and troughs in traffic. I have only gotten as far as testing out some ideas, but I came across this post on Gojko Adzic’s blog about messaging, and why you would want to use messaging, well worth reading I think.
IM: fschiettecatte
A nice essay as far as it goes. As a huge fan of message based architectures it seems to me that he fails to give the reader, presumably a system-architect, any sense of what’s to look be expecting when you go down that path.
For example it is a delight to be able to fire-and-forget. But to enable that delight the messaging platform has to take responsiblity. Responsible messaging platforms need to provide a lot of monitoring, auditing, and exception handling expertise.
I worry, reading something like that, about bait and switch. You buy the delight of fire-and-forget, and later we will charge you an arm and a leg for the platform services you really need.
Any hows, week I’m enjoying playing with RabbitMQ, building bridges, and reading the AMQP spec.
I agree about the delights of fire-and-forget, they are very attractive, but like all systems they need to be watched over and maintained, more systems/boxes means more work, even if automate everything. Personally I like to reduce things to their bare essentials, fewer things to maintain and fewer things to go wrong.
I took a quick look at the RabbitMQ website last night and the erlang requirement put me off a bit, and started to dig into openMQ instead.