Data Engineer_C&D

KKP's Data Exchange team is hiring a Data Engineer to support secure and reliable data movement across the organization, including ingestion into the enterprise data platform and system-to-system integrations. Responsibilities include building scalable ingestion pipelines, implementing data quality checks, designing workflow orchestration, and supporting inbound and outbound data exchange while applying governance and privacy principles.

The role requires hands-on experience delivering end-to-end data solutions, proficiency in Python and SQL, and strong practical experience with Databricks, Apache Spark, and Kafka or similar streaming technologies. Familiarity with workflow orchestration tools such as Airflow or Azure Data Factory, cloud data platforms on Microsoft Azure, and solid engineering practices including testing, version control, and CI/CD is expected.

Develop and maintain scalable and resilient ingestion pipelines, integrating data from APIs, file systems, and databases into Azure.

Implement data quality checks to ensure completeness, accuracy, and consistency across ingestion and delivery processes.

Design and manage workflow orchestration to ensure reliable execution, proactive monitoring, and failure recovery.

Work with system integration stakeholders to define, document, and maintain data exchange requirements for both inbound and outbound flows.

Build reusable and scalable data integration processes that support both ingestion and outbound delivery across the enterprise.

Hands-on experience delivering end-to-end data solutions with a foundation in large-scale data processing, data infrastructure engineering, or data modeling.

Proficiency in Python and SQL with solid engineering practices including testing, version control, CI/CD, and maintainable code.

Strong practical experience with Databricks and Apache Spark for building, optimizing, and operating production-grade data pipelines.

Experience with streaming data or event-driven pipelines using Kafka or similar technologies.

Familiarity with workflow orchestration tools such as Airflow, Azure Data Factory, or similar platforms; experience with Microsoft Azure cloud data platforms.