Soft Real Time Processing Pipeline for Healthcare Related Events

Abstract

Data analytics and business intelligence approaches have proven their potential for assisting decision making and in some cases even achieve complete automation. The healthcare industry, public and private, generates and stores large amounts of data. Such data is valuable not only for internal usage but also for commercialization as data centric services, relevant to the surrounding markets and organization, such as the pharmaceutical complex. MedicineOne is a Portuguese technological company that develops and commercializes electronic health record (EHR) and clinic management systems (CMSs). Aware of their privileged position, the company decided to build a soft real time platform to support healthcare event aggregation, analysis and visualization. Building this new system will outfit the company with the necessary technological infrastructure and expertise to start providing new services to their clients, empowering them with insight from their operational data. The extracted intelligence creates conditions to streamline internal processes and improve resource allocation. A direct consequence is better healthcare. Companies operating abroad in homologous positions have proven that this business model is viable. There is a clear demand from the pharmaceutical industry for ways to measure, in a short time-frame, the success of their products and services and how they rank among competitors. Access to healthcare events as they happen would give institutions such as governments and pharmacies an understanding of the state of the healthcare system as well as the health of the general population. It would also make it possible to accurately determine the demand for drugs and diagnostic tests, increasing efficiency. During this project, the intern was responsible for studying the state of the art and become proficient enough to build a scalable soft real time data ingestion platform capable of receiving and aggregating events form CMS / EHR and make them available for processing and querying.