A leading manufacturing company is looking for a senior data engineer to join an internal team of experts. In the assignment you'll be responsible for designing and implementing data pipelines (streaming and batch) and data architectures to process IoT telemetry data produced by the customer's fleet of installed equipment base. You will collaborate with analysts, data scientists and business stakeholders to make the IoT data more accessible for reporting, analytics, machine learning and AI applications and services.
To succeed in the assignment you have
- Strong project experience in designing and implementing data pipelines (streaming and batch) on AWS
- Experience and understanding in managing big IoT data
- Experience in applying CI/CD tools and processes on AWS
- Understanding on modern lake house architectures
- Experience with many of the following tools and technologies (and an ability to quickly learn the rest): AWS EMR, AWS Athena, Apache Kafka, AWS Glue, Terraform, CDK, DAB, Gitlab CI, Python, Scala, Databricks (Workflows, live tables, OLTP's), Delta Lake, Spark, DBT, Structured Streaming
- Good English and team working skills
The assignment requires onsite presence at Helsinki capital region.
The customer will fill the position as soon as the good candidate is found, please contact us asap!