Lead Data Engineer - E-Commerce Technologies

The Offer • Work within a company with a solid track record of success • Flexible working options • Join a well known brand within IT Services Ideal Candidate We seek a Lead Data Engineer to take charge of our data engineering initiatives, focusing on enhancing data collection, storage, and analysis. Key Responsibilities: • Architect and scale a state-of-the-art data infrastructure capable of handling batch and real-time data processing needs with unparalleled performance. • Collaborate closely with the data science team to oversee data systems, ensuring accurate monitoring and insightful analysis of business processes. • Design and implement robust ETL (Extract, Transform, Load) data pipelines, optimizing data flow and accessibility. • Develop comprehensive backend data solutions to bolster microservices architecture, ensuring seamless data integration and management. • Engineer and manage integrations with third-party e-commerce platforms, expanding data ecosystem and capabilities. The Profile Requirements : • A background at a product-centric software company directly contributing to building products (vs providing data/analytics to business stakeholders) • Software development experience with a focus on data engineering. • Extensive experience in building ETL pipelines using tools such as Apache Spark, Databricks, or Hadoop. • Proficiency in Python or Java, with a deep understanding of software engineering best practices. • Expertise in distributed computing and data modeling, capable of designing scalable data systems. • Proficiency with NoSQL databases, including MongoDB, Cassandra, DynamoDB, and CosmosDB. • Proficiency in real-time stream processing systems such as Kafka, AWS Kinesis, or GCP Data Flow. • Skilled in utilizing caching and search technologies like Redis, Elasticsearch, or Solr. • Familiarity with message queuing systems, including RabbitMQ, AWS SQS, or GCP Cloud Tasks. • Experience with Delta Lake, Parquet files, and AWS, GCP, or Azure cloud services. • A strong advocate for Test Driven Development (TDD) and experienced in version control using Git platforms like GitHub or Bitbucket. • Experience at a startup is preferred. • Experience with consumer e-commerce data/technologies would be a bonus. Benefits : • Work Location: Remote • 5 days working The Employer Our client cloud team, comprising highly skilled IT professionals, offers perfect solutions for your needs. Cloud team simply means offshoring IT services. It connects you with a global pool of dedicated and qualified technical teams, irrespective of distance and international borders. You get the flexibility to hire onsite and offshore resources from any location at any time. They have diverse skills to deploy projects without disturbing the ongoing cycle. If you offshore IT services, you can save costs on infrastructure and recruitment.