Lead Data Engineer - DataOps (Snowflake & AWS Data Platform)
SQUARE ONE RESOURCES sp. z o.o.
⚲ Warszawa
140–170 zł netto (+ VAT) / godz.
Wymagania
- Snowflake Data Cloud
- AWS
- Python
- Amazon Redshift
- Google BigQuery
- Bash
- Talend
- ETL
- dbt
- Data Build Tool
- data mesh
- Monte Carlo
- Collibra
- Immuta
Opis stanowiska
Nasze wymagania:
Minimum 3–4 years of professional experience as a Data Engineer, DataOps Engineer, or similar role.
Bachelor's degree in Computer Science, Information Technology, Engineering, or a related technical field.
Strong hands-on experience with modern data engineering practices and cloud-based data platforms.
Strong experience with GitLab CI/CD pipelines, automation, and workflow orchestration.
Ability to design and maintain CI/CD processes for data engineering solutions.
Strong proficiency in Python for: Data manipulation Automation scripting Data processing workflows
Advanced SQL skills with experience writing efficient and optimized queries.
Practical experience working with analytical databases.
Strong knowledge of Snowflake is preferred.
Experience with alternative platforms such as Amazon Redshift or Google BigQuery will also be considered.
Experience writing and maintaining Bash scripts for automation and operational tasks.
Ability to design and implement automated tests.
Experience ensuring data quality, consistency, and reliability within data pipelines.
Experience with ETL tools and data integration processes is required.
Experience with Talend or similar ETL platforms is an advantage.
Mile widziane:
Experience with dbt (Data Build Tool) for data transformation and analytics engineering.
Understanding of Data Mesh principles and their practical application.
Familiarity with data governance and data management tools such as: Monte Carlo Collibra Immuta
Experience working in enterprise-scale data environments.
Previous experience acting as a technical lead or driving technical decisions within a team.
O projekcie:
We are looking for an experienced Data Engineer / DataOps Specialist to join a data engineering team responsible for building and automating modern data products within a cloud-based data platform.
The main focus of the role is the development and automation of data products based on Snowflake and AWS S3, with data sourced from SAP HANA views. Data pipelines are orchestrated using custom GitLab CI/CD workflows, ensuring scalable, reliable, and automated data delivery processes.
As a Technical Lead, you will play a key role in defining technical direction, establishing engineering standards, supporting the development team, and ensuring best practices across data engineering and DataOps processes.
This role is ideal for a Data Engineer who combines strong technical expertise with a DataOps mindset and is interested in taking ownership of data platform automation, engineering standards, and technical direction within a modern cloud data environment.
Zakres obowiązków:
Design, develop, and maintain data products using Snowflake and AWS S3.
Build and optimize scalable data pipelines orchestrated through GitLab CI/CD.
Automate data workflows and integrations based on SAP HANA views.
Develop Python-based scripts and automation solutions supporting data processing and operational activities.
Write efficient and optimized SQL queries for analytical data processing.
Implement data quality frameworks, validation processes, and automated testing to ensure data accuracy and integrity.
Collaborate with business stakeholders, data teams, and technical teams to understand requirements and deliver effective data solutions.
Act as a technical leader by defining engineering standards, reviewing solutions, and promoting best practices in Data Engineering and DataOps.
Evaluate and recommend new technologies, tools, and approaches to improve data platform capabilities.
Support continuous improvement of data delivery processes, automation, and operational excellence.
Minimum 3–4 years of professional experience as a Data Engineer, DataOps Engineer, or similar role.
Bachelor's degree in Computer Science, Information Technology, Engineering, or a related technical field.
Strong hands-on experience with modern data engineering practices and cloud-based data platforms.
Strong experience with GitLab CI/CD pipelines, automation, and workflow orchestration.
Ability to design and maintain CI/CD processes for data engineering solutions.
Strong proficiency in Python for: Data manipulation Automation scripting Data processing workflows
Advanced SQL skills with experience writing efficient and optimized queries.
Practical experience working with analytical databases.
Strong knowledge of Snowflake is preferred.
Experience with alternative platforms such as Amazon Redshift or Google BigQuery will also be considered.
Experience writing and maintaining Bash scripts for automation and operational tasks.
Ability to design and implement automated tests.
Experience ensuring data quality, consistency, and reliability within data pipelines.
Experience with ETL tools and data integration processes is required.
Experience with Talend or similar ETL platforms is an advantage.
Mile widziane:
Experience with dbt (Data Build Tool) for data transformation and analytics engineering.
Understanding of Data Mesh principles and their practical application.
Familiarity with data governance and data management tools such as: Monte Carlo Collibra Immuta
Experience working in enterprise-scale data environments.
Previous experience acting as a technical lead or driving technical decisions within a team.
O projekcie:
We are looking for an experienced Data Engineer / DataOps Specialist to join a data engineering team responsible for building and automating modern data products within a cloud-based data platform.
The main focus of the role is the development and automation of data products based on Snowflake and AWS S3, with data sourced from SAP HANA views. Data pipelines are orchestrated using custom GitLab CI/CD workflows, ensuring scalable, reliable, and automated data delivery processes.
As a Technical Lead, you will play a key role in defining technical direction, establishing engineering standards, supporting the development team, and ensuring best practices across data engineering and DataOps processes.
This role is ideal for a Data Engineer who combines strong technical expertise with a DataOps mindset and is interested in taking ownership of data platform automation, engineering standards, and technical direction within a modern cloud data environment.
Zakres obowiązków:
Design, develop, and maintain data products using Snowflake and AWS S3.
Build and optimize scalable data pipelines orchestrated through GitLab CI/CD.
Automate data workflows and integrations based on SAP HANA views.
Develop Python-based scripts and automation solutions supporting data processing and operational activities.
Write efficient and optimized SQL queries for analytical data processing.
Implement data quality frameworks, validation processes, and automated testing to ensure data accuracy and integrity.
Collaborate with business stakeholders, data teams, and technical teams to understand requirements and deliver effective data solutions.
Act as a technical leader by defining engineering standards, reviewing solutions, and promoting best practices in Data Engineering and DataOps.
Evaluate and recommend new technologies, tools, and approaches to improve data platform capabilities.
Support continuous improvement of data delivery processes, automation, and operational excellence.
🔍 Dekoder Ogłoszenia
🔴
Strong hands-on experience with modern data engineering practices and cloud-based data platforms.
Oczekuje się, że kandydat będzie potrafił samodzielnie wdrażać i utrzymywać złożone rozwiązania, a nie tylko je rozumieć.
🔴
Ability to design and maintain CI/CD processes for data engineering solutions.
Może oznaczać konieczność budowania i zarządzania infrastrukturą CI/CD od podstaw, a nie tylko korzystania z istniejących narzędzi.
🔴
Experience ensuring data quality, consistency, and reliability within data pipelines.
Może to być odpowiedzialność za ręczne sprawdzanie jakości danych i naprawianie błędów, a nie tylko za implementację automatycznych mechanizmów.
🟡
Experience with ETL tools and data integration processes is required.
Chociaż wymieniono konkretne narzędzia, może to oznaczać pracę z różnorodnymi, często starszymi lub niestandardowymi systemami integracji danych.
🔴
Understanding of Data Mesh principles and their practical application.
Wymaga się nie tylko teoretycznej wiedzy, ale także umiejętności jej zastosowania w praktyce, co może oznaczać budowanie lub refaktoryzację istniejących architektur.