Real Time Data Processing vs Batch Data Processing

If your business is still on batch data processing you have a hole in your pocket. And its making you lose money, fast. The biggest disadvantage of batch processing is that it creates a time delay. This time delay happens between your transaction receiving and output. Let’s look at how transactions receiving work. All the data your business generates, be it CRM data, POS data, Inventory data, Sales etc. can be viewed as unique transactions that are collected in multiple systems. All this data is stored in different business units within your organization and they all use different interfaces to collect and enter their data. But the important thing to note is that it gets collected somewhere. Now the problem is, you need to enter it in a format that can be viewed by multiple departments. Now, what your team decided to do, was to have all these transactions processed independently at a desired, designated time. This is called Batch data processing. It collects all the inventory data based on the current in and out stock, and uploads it at 12 a.m., when none of your other employees are eating into company bandwidth. Similarly, it collects all the CRM data, POS Data and sales data and uploads it at that ungodly hour. Fast forward to 8 a.m. the next morning. The store manager looks into his i-pad to get the inventory status for the day. He sees data that was uploaded 8 hours prior and cannot see the large shipment that came in at 4 a.m. Your VP of sales is looking to understand how many Doritos was sold yesterday. But your system failed to report that a customer walked in at 2 a.m. and wiped out all the munchies for a party. This is just one example, using a grocery chain as model clay. Do you see the hole in the pocket now? Enter our protagonist: Real time data processing. Real time data processing involves continuous input, process and output of data. It allows data to be processed in a short period of time (we are talking milli-seconds). Going back to our grocery chain example, if a packet of chips was bought at 2:00:00 a.m., your data is updated to reflect that transaction at 2:00:01 a.m. If there was a delivery of avocados at 4:00:00 a.m., your system has a log updated at 4:00:01 a.m. Can you imagine the power of this Real time data? Decisions can be made on the fly, without having to worry if you are looking at the most recent data. So how does this really work anyway? To really understand this, we need to go back to batch processing. Remember I told you that all the data collected throughout the day, gets updated at 12 a.m.? When it comes to Real time processing, each and every transaction that happens can be considered as a 12 a.m. batch upload. In other words, your system goes back and calculates all the transactions that happened throughout the day/ week/ month and adds 1 transaction to that total. All this happens in milliseconds! Each transaction can have multiple records in it and each record can be of multiple sizes. This would mean you now need a powerful web server and tons of money spent in infrastructure management right? Not really. Real time processing is now completely server-less and can be continuously scaled depending on your transaction receiving. If you are interested in reading about a company that benefited from Real time processing, have a look at this Case study.

» »

Author- Santosh Kumar

Santosh Kumar
Santosh Kumar is a senior marketing professional with extensive experience in helping business’ get the very best in BI and Analytics Solutions. If your business is looking to adopt and leverage analytics in any form, he is the person you need to be talking to. Linkedin , Twitter

Related Posts

Menu Engineering: Real-time insights enable menu optimization
How technology can mend the COVID-19 infected supply chain
Remaining true to its mission of “Serving the Underserved” during supply chain disruptions at Prince Retail

Leave a comment

Data Science for Restaurant