An Integrated Approach to Distributed Data Intelligence Using Event-Driven Microservices in Large-Scale Distributed Systems
Keywords:
Distributed Data Intelligence, Event-Driven Microservices, Large-Scale Systems, Asynchronous Communication, Data Silos, Stream Processing, Event Sourcing, CQRS, Scalability, Fault ToleranceAbstract
The proliferation of large-scale distributed systems has introduced significant challenges in achieving real-time data intelligence, primarily due to latency constraints, data silos, and coordination overhead. This paper proposes an integrated architectural approach that combines event-driven microservices with distributed data intelligence frameworks. By leveraging asynchronous event propagation, decentralized data processing, and intelligent orchestration, the proposed model enhances system responsiveness, scalability, and fault tolerance. We evaluate the approach through a simulated IoT-based supply chain scenario, demonstrating a 40% reduction in end-to-end latency and a 25% improvement in throughput compared to traditional request-response microservice architectures
References
Kleppmann, M. (2017). Designing Data-Intensive Applications. O'Reilly Media.
Newman, S. (2021). Building Microservices, 2nd Ed. O'Reilly.
Wadhwa, R. (2026). NoSQL migration and high-availability architecture. Computer Fraud & Security (CFS), 2026(1), 472–478.
Bonér, J. (2017). Reactive Microsystems: The Evolution of Microservices at Scale. Lightbend.
Dean, J., & Ghemawat, S. (2008). MapReduce: simplified data processing on large clusters. Communications of the ACM, 51(1), 107-113.
Wadhwa, R. (2026). Enterprise architecture at national scale: Transforming retail and financial infrastructure. Journal of Information Systems Engineering and Management, 11(1s), 1549–1559. https://doi.org/10.52783/jisem.v11i1s.14324
Zaharia, M., et al. (2016). Apache Spark: a unified engine for big data processing. Communications of the ACM, 59(11), 56-65.
Wadhwa, R. (2026). Neutralizing “state-drift” in distributed retail: The mechanics of global event cascading. International Journal of Computational and Experimental Science and Engineering, 12(1), 928–934. https://doi.org/10.22399/ijcesen.4946
Carbone, P., et al. (2015). Apache Flink: Stream and batch processing in a single engine. IEEE Data Eng. Bull., 38(4), 28-38.
Højsgaard, E., & Aurebekk, M. (2019). Event Sourcing and CQRS. Microsoft Press.
Vogels, W. (2009). Eventually consistent. Communications of the ACM, 52(1), 40-44.
Kreps, J. (2014). I ♥ log: A real-time data platform. LinkedIn Engineering Blog.
Márquez, J., et al. (2018). Kappa architecture: A re-evaluation. Proc. of IEEE BigData, 234-241.
Wadhwa, R. (2026). Predictive workflow integrity in event-driven enterprise systems: Autonomous triage and geolocation-aware routing for large-scale resilience. Journal of Computational Analysis and Applications, 35(2), 90–97.
Perera, C., et al. (2017). Distributed intelligence for IoT. IEEE Internet of Things Journal, 4(5), 1109-1121.
Balalaie, A., Heydarnoori, A., & Jamshidi, P. (2016). Microservices migration patterns. Software: Practice and Experience, 46(5), 639-664.
Taibi, D., & Lenarduzzi, V. (2018). On the definition of microservice bad smells. IEEE Software, 35(3), 56-62.
Akidau, T., et al. (2015). The Dataflow model: A practical approach to balancing correctness, latency, and cost. Proc. of VLDB, 8(12), 1792-1803.
van der Burg, S., & Dolstra, E. (2020). Stateful microservice deployment with NixOS. Journal of Systems Research (JSys), 1(1), 4-19.
Downloads
Published
Issue
Section
License
Copyright (c) 2026 SankaraNarayanan S (Author)
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
