Senior Data Engineer (Remote)

• Job Responsibilities • Data Ingestion and Integration: You will need to ingest data from various sources such as databases, files, APIs, and streaming platforms into Azure using services such as Azure Data Factory for ETL jobs. • Data Transformation and Processing: Large-scale data transformation and processing tasks to prepare the data for analysis and reporting using Azure Databricks and Azure Synapse Analytics and tools like Apache Spark. • Data Storage and Management: Develop, build and manage the appropriate storage solutions for data using Azure SQL Database, Azure Cosmos DB for structured data, Azure Data Lake Storage for big data and unstructured data, and Azure Blob Storage. • Data Warehousing: Implementing data warehousing solutions using services such as Azure Synapse Analytics to create scalable and distributed analytics capabilities for handling large datasets. • Data Modeling and Analysis: Collaborate with machine learning engineers to design and implement data models... for analytical workloads using Azure Analysis Services and Azure Databricks. • Real-time Data Processing: Configure and optimize streaming pipelines to extract insights from live data streams for Azure Stream. • Data Governance and Security: Data governance and security practices that cover data access controls, implement encryption, monitor data quality, and establish data retention policies. • MLOps and Automation: Automating data pipelines, monitoring data processes, and managing deployments using tools like Azure DevOps, Azure Monitor, and Azure Automation to achieve continuous integration and delivery (CI/CD). • Work in a team oriented and agile environment based on designed scrums, sprints and daily stand ups. • Contribution in projects documentation. Must Have Skills • Bachelor’s degree in computer science, computer engineering, or a related field. • 5+ years of experience in data engineering. • Proficiency in programming languages included Python and SQL. • Knowledge and experiences for cloud-based platforms specifically Azure cloud included Azure SQL, Data factory, Synapse, and Data Warehouse. • Participate in modeling Relational and NoSQL databases. • Experience with of MLOps and DevOps principles and best practices. • MS Power BI development and report management experience. • Experience with Agile/Scrum framework. • Excellent communication and collaboration skills