A
This a Full Remote job, the offer is available from: South Africa
This is a remote position.
Job Overview:
We are seeking a highly skilled Azure Data Engineer with extensive experience in Databricks, PySpark, and Spark SQL to join our team. The ideal candidate will be responsible for designing, developing, and maintaining large-scale data solutions on Azure, with a focus on real-time and batch data processing using Databricks.
Requirements
Key Responsibilities:
• Design, build, and maintain scalable data pipelines on Azure Databricks using PySpark and Spark SQL.
• Develop and optimize ETL processes for handling large data sets.
• Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and implement solutions.
• Ensure data quality, performance, and scalability by implementing best practices for coding and data architecture.
• Create and manage data models, databases, and data lakes on Azure.
• Monitor and troubleshoot data pipe ...