Senior Data Engineer - SCG

You will be responsible for designing, developing, and optimizing enterprise-scale data platforms on the Microsoft Azure Cloud. This role involves building scalable and secure ETL/ELT data pipelines to support Data Lake initiatives, analytics, and AI/ML projects. You will ensure high-quality, reliable, and cost-efficient data architecture while collaborating with cross-functional teams to deliver robust solutions and mentor junior engineers.

Your day-to-day work includes designing cloud-based data architecture using Azure Data Factory, Databricks, and Microsoft Fabric. You will build ETL/ELT processes to integrate data from SAP, ERP systems, APIs, and Excel into the enterprise Data Lake. Additionally, you will develop and optimize SQL, Python, and PySpark scripts for large-scale data processing and monitor daily pipeline operations to troubleshoot complex technical issues.

• Design and develop scalable data pipelines and cloud-based data architecture using Azure Data Factory, Databricks, Microsoft Fabric, and SSIS.
• Build ETL/ELT processes to integrate data from multiple sources such as SAP, ERP systems, APIs, and Excel into enterprise Data Lake and databases.
• Develop and optimize SQL, Python, and PySpark scripts for large-scale data processing.
• Monitor daily data pipeline operations, troubleshoot complex technical issues, and perform root cause analysis.
• Ensure data quality, accuracy, security, and optimize database performance.
• Prepare technical documentation, operation manuals, and share best practices with the team.
• Support continuous improvement by adopting emerging technologies and mentoring junior data engineers.

• Bachelor's degree in Computer Science, Information Technology, Computer Engineering, or related fields.
• Minimum 5 years of experience in Data Engineering or related roles.
• Strong experience with Microsoft Azure, Azure Data Factory (ADF), Databricks, Microsoft Fabric, and Data Lake.
• Strong proficiency in SQL Server, T-SQL, Python, and PySpark.
• Experience in designing Data Warehouse, ETL/ELT processes, and integrating data from ERP/SAP systems and APIs.
• Knowledge of Cloud Architecture, CI/CD, DevOps practices, and Database Performance Tuning.
• Strong analytical and problem-solving skills with the ability to perform root cause analysis.
• Passion for learning new technologies and mentoring team members.

• Experience with containerization technologies like Docker or Kubernetes.
• Familiarity with infrastructure as code tools such as Terraform or ARM templates.
• Strong communication skills for cross-functional collaboration.
• Experience in Agile or Scrum methodologies.