Data Engineer

‍

The DSF is seeking a Data Engineer with a keen focus on blockchain and distributed ledger technology (DLT). This role is pivotal in managing, curating, optimising, and securing datasets specifically related to cryptocurrency discussions across various platforms. The ideal candidate will be adept in web scraping, data quality assurance using AI, data integration, ensuring data security and compliance, and maintaining detailed documentation.

‍

Role and Responsibilities

‍

• Data Collection: Identify relevant chat sources, groups, and forums on platforms discussing particular topics. Maintain and develop web scraping tools or APIs for periodic data extraction.
• Data Quality Assurance: Develop and implement AI-based procedures for quality control of data and data sources to eliminate inaccuracies and anomalies. Create tools for monitoring data sources for changes and updates, adapting data collection and cleaning processes accordingly.
• Data Integration: Collaborate with data scientists and analysts to integrate collected data into various projects and analysis tools. Ensure smooth data flow and integration with other data sources within the organisation.
• Data Security and Compliance: Uphold the security and privacy of collected data in compliance with relevant regulations and company policies.
• Documentation: Maintain clear and comprehensive documentation of data sources, collection methods, and workflows. Produce reports and documentation for both internal and external stakeholders as required.
• Monitoring and Reporting: Develop and maintain systems to monitor the performance and health of data collection processes.

‍

Skill requirements

‍

• Bachelor’s degree in Computer Science, Data Science, or a related field.
• Knowledge of Data Structures and Databases is a must.
• Demonstrable experience in data engineering or a similar role, with a focus on web scraping and data collection.
• Proficient in programming languages such as Python, SQL.
• Knowledge in TypeScript is a must.
• Familiarity with blockchain technology and understanding of DLT principles.
• Knowledge of data privacy laws and compliance requirements.
• Strong analytical and problem-solving skills.
• Excellent communication and collaboration abilities.
• Preferred: Advanced degree in a relevant field.
• Preferred: Experience with big data technologies and cloud services.
• Preferred: Proficiency in AI and machine learning techniques for data quality assurance.

‍

Location: Remote

For any questions or assistance, please feel free to contact careers@dltscience.org.

‍