Senior Data Engineer (Data, Analytics & AI)
AVENGA (Agencja Pracy, nr KRAZ: 8448)
⚲ Remote
25 200 - 27 720 PLN (B2B)
Wymagania
- Snowflake
- SQL
- Python
- Apache Airflow
- dbt
- FastAPI
- REST API
- GitHub
- GitHub Actions
- AWS
- Tableau
- SAP S/4HANA (nice to have)
- ERP (nice to have)
Opis stanowiska
O projekcie:
We are looking for an experienced Senior Data Engineer to join a global Data, Analytics & AI initiative focused on building scalable data solutions that support enterprise reporting, analytics, and operational applications. In this role, you will design and maintain robust data pipelines, transform complex datasets into trusted business-ready products, and expose data through APIs for downstream consumers.
You will work in a modern cloud environment alongside data analysts, software engineers, and business stakeholders to deliver reliable, secure, and scalable data products.
Wymagania:
- 5+ years of experience in Data Engineering.
- Minimum 2 years of hands-on experience with Snowflake in production environments.
- Strong experience designing ETL/ELT pipelines and implementing complex data transformations.
- Advanced SQL skills and solid Python experience for data engineering tasks.
- Experience with orchestration tools such as Apache Airflow, dbt, Prefect, or similar.
- Strong understanding of data warehousing concepts, query optimization, partitioning, clustering, and cost management.
- Experience building and maintaining REST APIs (FastAPI or similar frameworks).
- Experience with Git-based workflows and CI/CD pipelines.
- Familiarity with cloud environments (AWS).
- Ability to work closely with technical and business stakeholders in an Agile environment.
Nice to Have
- Experience working with financial or ERP data.
- Knowledge of SAP (S/4HANA, BW4) or other ERP platforms.
- Experience with data governance or metadata management tools.
- Familiarity with dimensional modelling, Data Vault, or medallion architecture.
- Experience in regulated industries such as finance, life sciences, or healthcare.
- Exposure to BI tools such as Tableau or similar platforms.
Codzienne zadania:
- Design, build, and maintain scalable ETL/ELT pipelines for ingesting, transforming, and delivering data from multiple source systems.
- Blend and consolidate data from heterogeneous sources (ERP systems, APIs, databases, flat files) into trusted datasets.
- Develop and optimize data models and transformation logic to support analytics and reporting use cases.
- Build and maintain orchestration workflows using tools such as Airflow, dbt, or similar solutions.
- Implement CI/CD practices for data workloads and maintain version-controlled, production-ready code.
- Monitor pipeline performance, troubleshoot failures, and ensure data quality, lineage, and reliability.
- Design and develop REST APIs to expose data to internal applications and downstream systems.
- Collaborate with frontend and full-stack engineers to define data contracts and backend integrations.
- Apply cloud data platform best practices related to security, scalability, performance, and cost optimization.
- Partner with business stakeholders to translate complex requirements into technical solutions.
- Contribute to engineering standards, documentation, and best practices while mentoring less experienced team members.
We are looking for an experienced Senior Data Engineer to join a global Data, Analytics & AI initiative focused on building scalable data solutions that support enterprise reporting, analytics, and operational applications. In this role, you will design and maintain robust data pipelines, transform complex datasets into trusted business-ready products, and expose data through APIs for downstream consumers.
You will work in a modern cloud environment alongside data analysts, software engineers, and business stakeholders to deliver reliable, secure, and scalable data products.
Wymagania:
- 5+ years of experience in Data Engineering.
- Minimum 2 years of hands-on experience with Snowflake in production environments.
- Strong experience designing ETL/ELT pipelines and implementing complex data transformations.
- Advanced SQL skills and solid Python experience for data engineering tasks.
- Experience with orchestration tools such as Apache Airflow, dbt, Prefect, or similar.
- Strong understanding of data warehousing concepts, query optimization, partitioning, clustering, and cost management.
- Experience building and maintaining REST APIs (FastAPI or similar frameworks).
- Experience with Git-based workflows and CI/CD pipelines.
- Familiarity with cloud environments (AWS).
- Ability to work closely with technical and business stakeholders in an Agile environment.
Nice to Have
- Experience working with financial or ERP data.
- Knowledge of SAP (S/4HANA, BW4) or other ERP platforms.
- Experience with data governance or metadata management tools.
- Familiarity with dimensional modelling, Data Vault, or medallion architecture.
- Experience in regulated industries such as finance, life sciences, or healthcare.
- Exposure to BI tools such as Tableau or similar platforms.
Codzienne zadania:
- Design, build, and maintain scalable ETL/ELT pipelines for ingesting, transforming, and delivering data from multiple source systems.
- Blend and consolidate data from heterogeneous sources (ERP systems, APIs, databases, flat files) into trusted datasets.
- Develop and optimize data models and transformation logic to support analytics and reporting use cases.
- Build and maintain orchestration workflows using tools such as Airflow, dbt, or similar solutions.
- Implement CI/CD practices for data workloads and maintain version-controlled, production-ready code.
- Monitor pipeline performance, troubleshoot failures, and ensure data quality, lineage, and reliability.
- Design and develop REST APIs to expose data to internal applications and downstream systems.
- Collaborate with frontend and full-stack engineers to define data contracts and backend integrations.
- Apply cloud data platform best practices related to security, scalability, performance, and cost optimization.
- Partner with business stakeholders to translate complex requirements into technical solutions.
- Contribute to engineering standards, documentation, and best practices while mentoring less experienced team members.
🔍 Dekoder Ogłoszenia
🔴
building scalable data solutions that support enterprise reporting, analytics, and operational applications
Projekt może być duży i złożony, wymagający pracy z wieloma systemami i różnymi typami danych.
🔴
transform complex datasets into trusted business-ready products
Oczekuje się, że będziesz musiał radzić sobie z nieuporządkowanymi lub trudnymi do przetworzenia danymi i przekształcać je w użyteczne raporty.
🔴
expose data through APIs for downstream consumers
Może to oznaczać konieczność budowania i utrzymywania interfejsów API, co wymaga dodatkowych umiejętności i odpowiedzialności.
🟡
work closely with technical and business stakeholders in an Agile environment
Będziesz musiał aktywnie komunikować się z różnymi zespołami i rozumieć ich potrzeby biznesowe, co może być wymagające.
🟡
cost management
Oprócz technicznych aspektów, będziesz musiał brać pod uwagę optymalizację kosztów infrastruktury chmurowej.