Big Data Developer (heating industry)
⚲ Białystok, Kielce, Lublin, Warszawa
Do uzgodnienia
Wymagania
- Python
- Azure
- Databricks
- SQL
- PySpark
- Distributed pipelines
- Delta Lake optimization
- Databricks workflows
- Databricks Jobs
- GitHub Actions
Opis stanowiska
About the project
For our client, an international leader in the heating industry, we are looking for an experienced developer who will contribute to designing, building, and operationalizing scalable data analytics solutions on Microsoft Azure and Databricks.
Our expectations
• Minimum 6 years in software development within data engineering or analytics environments.
• Expert level knowledge and experience with Python, Azure, Databricks, SQL.
• Experience with scalable data analytics solutions on Microsoft Azure and Databricks.
• Solid experience in production-grade Python engineering (packaging, modular architecture, and concurrency).
• Strong practical knowledge of advanced PySpark, distributed pipelines, and Delta Lake optimization.
• Hands-on experience productionizing Databricks solutions (Workflows, Jobs, and Asset Bundles).
• Proven proficiency in AI-assisted development (GitHub Copilot, specification-driven design) and basic MLflow.
• Active experience in DevOps, including CI/CD automation and build/test/release pipelines via GitHub Actions.
Key tasks
• Designing and building scalable, distributed data pipelines on Microsoft Azure and Databricks, with a focus on Delta Lake performance and optimization.
• Developing production-grade, modular Python applications, ensuring clean architecture, proper packaging, and effective concurrency.
• Productionizing Databricks solutions by setting up and managing end-to-end Workflows, Jobs, and Asset Bundles.
• Implementing specification-driven designs using AI-assisted development tools (like GitHub Copilot) to accelerate delivery, while tracking experiments with MLflow.
• Automating deploy, test, and release processes by building and maintaining robust CI/CD pipelines via GitHub Actions.
For our client, an international leader in the heating industry, we are looking for an experienced developer who will contribute to designing, building, and operationalizing scalable data analytics solutions on Microsoft Azure and Databricks.
Our expectations
• Minimum 6 years in software development within data engineering or analytics environments.
• Expert level knowledge and experience with Python, Azure, Databricks, SQL.
• Experience with scalable data analytics solutions on Microsoft Azure and Databricks.
• Solid experience in production-grade Python engineering (packaging, modular architecture, and concurrency).
• Strong practical knowledge of advanced PySpark, distributed pipelines, and Delta Lake optimization.
• Hands-on experience productionizing Databricks solutions (Workflows, Jobs, and Asset Bundles).
• Proven proficiency in AI-assisted development (GitHub Copilot, specification-driven design) and basic MLflow.
• Active experience in DevOps, including CI/CD automation and build/test/release pipelines via GitHub Actions.
Key tasks
• Designing and building scalable, distributed data pipelines on Microsoft Azure and Databricks, with a focus on Delta Lake performance and optimization.
• Developing production-grade, modular Python applications, ensuring clean architecture, proper packaging, and effective concurrency.
• Productionizing Databricks solutions by setting up and managing end-to-end Workflows, Jobs, and Asset Bundles.
• Implementing specification-driven designs using AI-assisted development tools (like GitHub Copilot) to accelerate delivery, while tracking experiments with MLflow.
• Automating deploy, test, and release processes by building and maintaining robust CI/CD pipelines via GitHub Actions.
🔍 Dekoder Ogłoszenia
🟡
international leader in the heating industry
Może to oznaczać zarówno stabilną, dużą firmę, jak i taką, która dopiero buduje swoją pozycję na rynku, a 'lider' jest określeniem aspiracyjnym.
🟡
experienced developer
Chociaż podano wymagane lata doświadczenia, 'doświadczony' może być subiektywne i niekoniecznie oznacza seniora z szerokim zakresem wiedzy.
🔴
production-grade Python engineering
Wymaga to nie tylko pisania kodu w Pythonie, ale też stosowania dobrych praktyk inżynierskich, co może być trudniejsze niż samo tworzenie skryptów.
🟡
AI-assisted development (GitHub Copilot, specification-driven design)
Chociaż brzmi nowocześnie, może oznaczać, że oczekuje się szybkiego prototypowania i mniejszego nacisku na dogłębne zrozumienie algorytmów.
🟡
basic MLflow
Określenie 'basic' może sugerować, że nie jest to główny obszar odpowiedzialności, ale znajomość podstaw jest wymagana.