Cloud Event Processing – Where’s The Data?
There are several things left to cover in our #TwitYourl proejct. One of the most glaring absences so far is storage – where do we put all of these Tweets? What happens if we’d like to make changes to our RuleBots and test them with historical data? Right now, we can’t.
In keeping with some of the rules of our implementation, an OnRamp is not allowed to update a database directly. This would tie the OnRamp to the database implementation and make our architecture more brittle than it has to be. Also, it would mean that the OnRamp had to have specific information about where it was sending a message – this is not allowed.
Remember, we are architecting for the cloud – the idea of baked in, point to point, connections inside of our cloud should have you recoiling in horror by now. So how can we update databases easily within our Cloud Event Processing framework?
Well, it would be really easy if we had some metadata that described all of the data, or events, that the OnRamps were publishing on the bus. And it would be even neater if all of that metadata was available in a central database or directory. This way, when the infrastructure that #TwitYourl is built on top of is running, all one would need to do is go to some management console, bring up the information about how we’d like to handle a particular event, click the ‘Store in a Database” button and be done with it. Click, click, click and we’re storing events – automagically; somewhere in the cloud!
So, let’s do this for #TwitYourl – I vote for MongoDB. I know that Cassandra seems to be a fairly popular choice this past week, but we’re going to buck that trend and hook up MongoDB inside of our cloud and we’re going to feed Mongo some yummy Tweets!
Sorry, I couldn’t resist.
But before feeding Mongo, we’re going to work on some visualization this week. We’re running a little behind schedule here and we need to get caught up!
