Advertisement:



Data Mining Engineer

Hong Kong, Hong Kong
Last update 2025-02-17
Expires 2025-03-17
ID #2579548877
Free
Data Mining Engineer
Hong Kong, Hong Kong,
Modified January 31, 2025

Description

We are seeking a skilled and detail-oriented engineer to join our dynamic team. In this role, you will be responsible for the development, maintenance, and optimization of web crawlers and data collection systems. You will work closely with data engineers, software developers, and data analysts to ensure the smooth operation of our data acquisition infrastructure. The ideal candidate has a strong background in web scraping, data extraction, and system monitoring, along with a passion for solving complex technical challenges. Responsibilities Design, develop, and maintain web crawlers and data extraction pipelines to collect data from various online sources. Monitor and optimize the performance of crawler systems to ensure high availability, scalability, and efficiency. Troubleshoot and resolve issues related to crawler operations and data quality problems. Collaborate with teams to understand data requirements and implement solutions to meet business needs. Develop and maintain documentation for crawler systems, including technical specifications, operational procedures, and troubleshooting guides. Stay up to date with the latest trends and technologies in web scraping, data extraction, and distributed systems. Automate repetitive tasks and improve system reliability through scripting and tool development. Qualifications Bachelor’s degree in computer science, Information Technology, or a related field. Proven experience in web scraping, data extraction, and crawler development using tools such as Scrapy, Beautiful Soup, Selenium, or similar frameworks. Strong programming skills in Python. Understanding of network protocols, IP management, and proxy services. Strong problem-solving skills and the ability to work independently or as part of a team. Excellent communication skills and the ability to document technical processes clearly. Preferred Qualifications Experience with machine learning or natural language processing (NLP) techniques for data extraction. Experience with distributed systems, cloud platforms (e.g., AWS, Azure, GCP), and containerization technologies (e.g., Docker, Kubernetes). Knowledge of database systems (SQL, No SQL) and data storage solutions. Knowledge of data privacy regulations and ethical considerations in web scraping. Familiarity with version control systems (e.g., Git) and CI/CD pipelines. Experience with monitoring tools (e.g., Prometheus, Grafana) and log management systems (e.g., ELK Stack). Location of work: Hong Kong Science Park Annual leave and group medical insurance Good team culture Assistance to apply for an IANG VISAAdditional Information Job Level Entry level Publish Date24/01/2025 Job Ref. No. N/AN/AJob Function Extraction & Mining Research & Development (R&D)Software Development Company Overview Datago Technology Limited is a financial technology company primarily focused on providing Chinese text data analytics. By leveraging big data and artificial intelligence technologies to process large-scale text data from comprehensive sources like news, reports, and social media, it provides quantitative and easy-to-use structured data analytics for the capital market in mainland China and Hong Kong. Our machine-readable text data analytics services allow users to overcome language barriers and extract valuable insights from large amounts of information. The products help users enhance returns, reduce risk, and improve operational efficiencies. Datago users include top asset management firms like hedge funds, security brokers, and research institutions worldwide.
#J-18808-Ljbffr

Job details:

Job type: Full time
Contract type: Permanent
Salary type: Monthly
Occupation: Data mining engineer

⇐ Previous job

Next job ⇒     

 

Contact employer

    Quick search:

    Location

    Type city or region

    Keyword


    Advertisement: