Senior Data Engineer
⚲ Warsaw, Kraków, Wrocław, Poznań, Katowice
25 200 - 30 240 PLN (B2B)
Wymagania
- Python
- Docker
- SQL
- Databricks
- PySpark
- CI/CD
- GCP (nice to have)
- Airflow (nice to have)
- BigQuery (nice to have)
- API (nice to have)
- FastAPI (nice to have)
- Azure Databricks (nice to have)
- Azure (nice to have)
- AWS (nice to have)
- Kubernetes (nice to have)
Opis stanowiska
O projekcie:
[REMOTE] Senior Data Engineer
We're expanding our AI Data Engineering Team at DS STREAM!
This is a 100% remote role WITHIN POLAND.
Project description:
This initiative centers on a mature, high-availability data hub architected within Azure Databricks, utilizing Unity Catalog, Spark Structured Streaming, and Asset Bundles to ensure robust performance and governance. The solution is built upon the Medallion Architecture framework, providing a scalable foundation that is now being expanded to ingest, transform, and seamlessly integrate new data sources into the existing ecosystem.
You will get
- 100% remote work with the possibility to use an office in Warsaw
- Flexible working hours and engaging company events
- Access to Azure, GCP and Databricks certifications
- Possibility to use an in-house incubator to develop your own ventures
- A supportive community of 50+ senior data consultants with extensive experience in Data Warehouse and Big Data
- and much more!
Wymagania:
We’re expanding our team! 🚀
DS STREAM is growing! We are looking for SENIOR Data Engineers.
You will need at least one of these
- At least 5 years of commercial experience in Data Engineering with focus on large scale data pipelines.
- Proven track record using advanced solutions in Azure cloud and Databricks
- Previous exposure to GCP infrastructure and tools (Airflow, Pub/Sub, BigQuery, VertexAI)
- Knowledge of Spark Structured Streaming
- Proficiency in Python and SQL
- Ability to build pipelines using Github or similar tools
Nice to have
- Ability to build enterprise API integrations with FastAPI or similar
- Experience in leading/mentoring engineers
Codzienne zadania:
- Development of a program for processing large volumes of data
- Ingesting data from APIs
- Developing Databricks pipelines in PySpark / SQL
- Handling Spark Structure Streaming pipelines
- Designing and managing objects in Unity Catalog according to Medallion Architecture
[REMOTE] Senior Data Engineer
We're expanding our AI Data Engineering Team at DS STREAM!
This is a 100% remote role WITHIN POLAND.
Project description:
This initiative centers on a mature, high-availability data hub architected within Azure Databricks, utilizing Unity Catalog, Spark Structured Streaming, and Asset Bundles to ensure robust performance and governance. The solution is built upon the Medallion Architecture framework, providing a scalable foundation that is now being expanded to ingest, transform, and seamlessly integrate new data sources into the existing ecosystem.
You will get
- 100% remote work with the possibility to use an office in Warsaw
- Flexible working hours and engaging company events
- Access to Azure, GCP and Databricks certifications
- Possibility to use an in-house incubator to develop your own ventures
- A supportive community of 50+ senior data consultants with extensive experience in Data Warehouse and Big Data
- and much more!
Wymagania:
We’re expanding our team! 🚀
DS STREAM is growing! We are looking for SENIOR Data Engineers.
You will need at least one of these
- At least 5 years of commercial experience in Data Engineering with focus on large scale data pipelines.
- Proven track record using advanced solutions in Azure cloud and Databricks
- Previous exposure to GCP infrastructure and tools (Airflow, Pub/Sub, BigQuery, VertexAI)
- Knowledge of Spark Structured Streaming
- Proficiency in Python and SQL
- Ability to build pipelines using Github or similar tools
Nice to have
- Ability to build enterprise API integrations with FastAPI or similar
- Experience in leading/mentoring engineers
Codzienne zadania:
- Development of a program for processing large volumes of data
- Ingesting data from APIs
- Developing Databricks pipelines in PySpark / SQL
- Handling Spark Structure Streaming pipelines
- Designing and managing objects in Unity Catalog according to Medallion Architecture
🔍 Dekoder Ogłoszenia
🔴
possibility to use an office in Warsaw
Biuro istnieje, ale nie ma obowiązku z niego korzystać, co może sugerować, że nie jest ono priorytetem ani głównym miejscem pracy.
🔴
Flexible working hours
Może oznaczać prawdziwą elastyczność, ale też konieczność dostosowania się do potrzeb zespołu lub projektów, co może prowadzić do pracy poza standardowymi godzinami.
🔴
engaging company events
Może oznaczać ciekawe wydarzenia, ale też presję na uczestnictwo w nich, nawet jeśli nie jest to obowiązkowe.
🟡
Possibility to use an in-house incubator to develop your own ventures
Jest to ciekawa oferta, ale może wymagać poświęcenia dodatkowego czasu i energii poza głównymi obowiązkami projektowymi.
🟢
A supportive community of 50+ senior data consultants with extensive experience in Data Warehouse and Big Data
Obecność wielu doświadczonych specjalistów to duży plus, ale może też oznaczać, że będziesz częścią dużej struktury, gdzie podejmowanie decyzji może być wolniejsze.