ENGEL, which was founded in 1945, now is the leading manufacturer for injection moulding machines on the global market.
Since then, the amount of data has grown immensely and has also become more and more heterogenous due to newer generations of machine controls.
Taking a closer look at the conglomerations of each and every machine’s log files, one can find 13 different types of timestamps, different archive types and more peculiarities of each control generation. Apparently, this has led to certain problems in automatically processing and analysing the data.
In this talk, you will explore how ENGEL managed to centralise this data in only one place, how ENGEL set up a data pipeline to ingest batch-oriented data in a streaming fashion and how ENGEL migrated their pipeline from an on-premise Hadoop setup to the cloud using Databricks.