Semantic Data Engineer
⚲ Remote
16 800 - 22 680 PLN (B2B)
Wymagania
- Data engineering
- SQL
- ETL
- Python
- NoSQL
- Git
- Terraform
- CloudFormation
- SPARQL (nice to have)
- AWS (nice to have)
- AWS Lambda (nice to have)
- Glue (nice to have)
- Jira (nice to have)
Opis stanowiska
O projekcie:
We are looking for a Data Engineer to join an international project supporting pharmaceutical research through the integration and standardization of laboratory data. In this role, you will design and maintain scalable data pipelines, enabling seamless access to high-quality, analysis-ready data for scientists and research teams.
Wymagania:
- Experience in Data Engineering, including SQL, ETL development, and data integration.- Strong Python programming skills.- Hands-on experience with NoSQL databases (document or graph databases).- Familiarity with DevSecOps practices, including CI/CD, Git, Infrastructure as Code (Terraform or CloudFormation), and Docker.- Good communication skills and experience working in cross-functional Agile teams.- Professional proficiency in English.Nice to Have- Experience with Semantic Web technologies (RDF, SPARQL, Knowledge Graphs).- Hands-on experience with AWS, particularly Lambda, Glue, and Step Functions.- Familiarity with Jira and Confluence.- Experience working with scientific, laboratory, or pharmaceutical data environments.
Codzienne zadania:
- Design, develop, and maintain ETL pipelines for integrating laboratory instrument data.
- Build and optimize SQL queries to support efficient data processing and analytics.
- Develop and extend data models and JSON Schemas based on industry standards.
- Collaborate with DevOps teams to implement and maintain cloud-based data solutions.
- Ensure data quality, consistency, and compliance with FAIR data principles.
- Support continuous improvements in data integration processes and platform capabilities.
We are looking for a Data Engineer to join an international project supporting pharmaceutical research through the integration and standardization of laboratory data. In this role, you will design and maintain scalable data pipelines, enabling seamless access to high-quality, analysis-ready data for scientists and research teams.
Wymagania:
- Experience in Data Engineering, including SQL, ETL development, and data integration.- Strong Python programming skills.- Hands-on experience with NoSQL databases (document or graph databases).- Familiarity with DevSecOps practices, including CI/CD, Git, Infrastructure as Code (Terraform or CloudFormation), and Docker.- Good communication skills and experience working in cross-functional Agile teams.- Professional proficiency in English.Nice to Have- Experience with Semantic Web technologies (RDF, SPARQL, Knowledge Graphs).- Hands-on experience with AWS, particularly Lambda, Glue, and Step Functions.- Familiarity with Jira and Confluence.- Experience working with scientific, laboratory, or pharmaceutical data environments.
Codzienne zadania:
- Design, develop, and maintain ETL pipelines for integrating laboratory instrument data.
- Build and optimize SQL queries to support efficient data processing and analytics.
- Develop and extend data models and JSON Schemas based on industry standards.
- Collaborate with DevOps teams to implement and maintain cloud-based data solutions.
- Ensure data quality, consistency, and compliance with FAIR data principles.
- Support continuous improvements in data integration processes and platform capabilities.
🔍 Dekoder Ogłoszenia
🔴
supporting pharmaceutical research through the integration and standardization of laboratory data
Praca może wymagać zrozumienia specyficznej terminologii naukowej i regulacji branżowych, co może być wyzwaniem dla osób bez doświadczenia w tej dziedzinie.
🟡
design and maintain scalable data pipelines
Oznacza to, że będziesz odpowiedzialny za tworzenie i utrzymywanie systemów, które muszą radzić sobie ze znacznym wzrostem ilości danych w przyszłości.
🔴
seamless access to high-quality, analysis-ready data
Może to sugerować, że obecne dane są w złym stanie i wymagają znaczącego wysiłku w celu ich poprawy i przygotowania do analizy.
🟡
Hands-on experience with NoSQL databases (document or graph databases)
Chociaż wymieniono NoSQL, nacisk na SQL w wymaganiach sugeruje, że relacyjne bazy danych nadal będą odgrywać kluczową rolę.
🟡
Familiarity with DevSecOps practices
Oczekuje się, że będziesz aktywnie uczestniczyć w procesach związanych z bezpieczeństwem i wdrażaniem, a nie tylko korzystać z gotowych rozwiązań.