Data Engineer with GenAI/LLM
⚲ Warszawa, Gdańsk
25 200 - 28 560 PLN netto (B2B)
Wymagania
- GenAI
- LLM
- Big Data
- Scala
- Spark
- Data Reference Architecture
- Airflow
- Snowflake
- Iceberg
Opis stanowiska
Summary: The primary goal of this role is to enhance the data engineering lifecycle with AI, specifically through the development and maintenance of the Data Packaging Framework. This role is crucial for transforming source data into standardized formats and ensuring effective delivery of Data Products.Industry: banking
Contract Length: 12 months+ prolongation
Type of contract: B2B Contract
Salary grids: 150-170 zł/h
Work Model: hybrid from Gdańsk or Warsaw (3 days office, 2 days remote)
• Main Responsibilities:
• Build and evolve workflows using LLM/agent assistance to improve developer productivity in data pipeline delivery.
• Implement AI-driven schema review and governance as part of automation.
• Drive AI-supported test automation including end-to-end test generation and validation.
• Develop data packaging pipelines converting source data into canonical formats.
• Manage SQL-based transformation logic within data pipelines.
• Support the canonical message schema lifecycle, including validation and compatibility checks.
• Contribute to operational stability, documentation, and Architecture Decision Records (ADRs).
Key Requirements:
• Strong AI / GenAI capability
• Hands-on experience in building LLM-powered automation for engineering workflows
• Experience with AI-assisted review flows integrated into CI/CD pipelines
Nice to Have:
• Familiarity with Data Reference Architecture or data product models
• Knowledge of Airflow, Snowflake, Iceberg, or similar platforms
• Experience with Big Data, Spark and/or Scala.
• Exposure to data governance, metadata, and lineage concepts
• Experience in banking or regulated environments
Contract Length: 12 months+ prolongation
Type of contract: B2B Contract
Salary grids: 150-170 zł/h
Work Model: hybrid from Gdańsk or Warsaw (3 days office, 2 days remote)
• Main Responsibilities:
• Build and evolve workflows using LLM/agent assistance to improve developer productivity in data pipeline delivery.
• Implement AI-driven schema review and governance as part of automation.
• Drive AI-supported test automation including end-to-end test generation and validation.
• Develop data packaging pipelines converting source data into canonical formats.
• Manage SQL-based transformation logic within data pipelines.
• Support the canonical message schema lifecycle, including validation and compatibility checks.
• Contribute to operational stability, documentation, and Architecture Decision Records (ADRs).
Key Requirements:
• Strong AI / GenAI capability
• Hands-on experience in building LLM-powered automation for engineering workflows
• Experience with AI-assisted review flows integrated into CI/CD pipelines
Nice to Have:
• Familiarity with Data Reference Architecture or data product models
• Knowledge of Airflow, Snowflake, Iceberg, or similar platforms
• Experience with Big Data, Spark and/or Scala.
• Exposure to data governance, metadata, and lineage concepts
• Experience in banking or regulated environments
🔍 Dekoder Ogłoszenia
🟡
enhance the data engineering lifecycle with AI
Oznacza to, że AI będzie narzędziem wspomagającym, a nie głównym motorem zmian w procesach data engineeringowych.
🔴
development and maintenance of the Data Packaging Framework
Może oznaczać zarówno tworzenie od podstaw, jak i jedynie drobne modyfikacje istniejącego, być może niedoskonałego, frameworka.
🟡
Build and evolve workflows using LLM/agent assistance to improve developer productivity
Sugestia, że LLM będzie pomagał w tworzeniu i usprawnianiu procesów, ale faktyczny ciężar pracy nadal spocznie na inżynierze.
🟡
Implement AI-driven schema review and governance as part of automation
Może oznaczać wdrożenie istniejących narzędzi AI do przeglądu schematów, a niekoniecznie tworzenie od podstaw zaawansowanych rozwiązań.
🟡
Drive AI-supported test automation including end-to-end test generation and validation
Wskazuje na wykorzystanie AI do wsparcia automatyzacji testów, ale niekoniecznie na pełną autonomię AI w tym procesie.