For the past 100 years of ‘modern business’, we have been capturing what happens in our organizations. The main goal has been to support the operational side of the business; the day to day operations of providing products and services. The focus for many organizations has been on keeping an accurate system of record, a representation of the state of our organization.
While the idea of gathering intelligence out of business operations dates back well into the 18th century, the introduction of computers really kicked off the analytical revolution. The sheer processing power at our disposal allowed us to start interpreting the business events that happened and deducing insights out of it.
The tools we use have changed over the years, but the general idea has been that a system of record - the transactional side of the business - feeds the analytical side of the business; both sides being clearly separated from each other. Jobs would run during the night with the sole purpose of shipping data from one side to the other.
But with customers becoming ever more involved in how our businesses operate, there came a need for that information to flow back from the analytical side into the transactional one. Many organizations figured out a trivial solution in shipping insights back from the analytical side to the transactional one just as they did the other way around.
With the introduction of machine learning and AI, most companies again leverage what they know and train their models offline and copy the result over to the transactional side to be used for inference.
While piggy-backing on previous solutions certainly has its merits, it also amplifies the weaknesses of previous iterations. Nightly jobs only work if your company actually has non-office hours. With many companies working at a global scale, this isn’t the default anymore and insights, models and analytics are for a large part required to be continuous. Even for the organizations which benefit from a clear distinction between online and offline hours, the amount of offline time available just isn’t enough to deal with the workload at hand.
Sometimes it is best to take a step back and think through what we actually want to accomplish given the changed requirements we are faced with. That way we can figure out what the end goal should look like and we can start drawing a map to get there.
We can’t draw that map for you since it is highly dependent on how your organization is relying on how your business is functioning today. What we can do however, is provide a glance into what we think is the end goal we should be aiming for.
The series tackles a foundational shift in how we build organizations today, which has more to do with mindset than the tools being used. Obviously we think NATS is the ideal candidate to be the foundation for this new kind of organization and we are pretty sure we can convince you about that. However, this journey is meant to be an informative one for anyone out there who feels there is something off with the current state of affairs. It is meant to be an interactive one; yes we will publish articles but at the same time we would love to hear from you and what you think.
I hope you enjoyed this article as much as I enjoyed writing it. But it doesn't stop there. Hop into the Data Series Slack Channel and share your thoughts. I would love to hear from you.
Until next time!