Web scraping Expert

We are looking for a Senior Web Scraping Engineer to join our team. The ideal candidate deeply understands web technologies and real-world data extraction techniques and has built numerous real-world crawlers/spiders that successfully extract web data at a large scale and maintain it for a long period. This role will involve creating and maintaining scalable and robust complex web scraping systems, ensuring the highest accuracy and performance. Responsibilities • Design, develop, and maintain scalable web scraper systems using various tools and technologies. • Extract and parse data from various websites while adhering to the legal and ethical rules of web scraping. • Write efficient, testable, and reusable code while ensuring high performance and reliability. • Collaborate with team members to identify and understand data needs and translate these into system requirements. • Implement proper error handling and data quality checks to ensure the reliability of the scraped data. • Keep abreast of new trends and best practices in web scraping technology. Qualifications • Proven experience as a Web Scraper or in a similar role. • Proficiency in Python and with experience in frameworks like Scrapy and Beautiful Soup. • Strong understanding of web technologies (JavaScript, XPath, HTTP networking, GraphQL, Fetch API). • Familiarity with web scraping legal and ethical guidelines. • Excellent problem-solving skills and attention to detail. • Strong communication and teamwork skills. • Fluent in English.