Senior Data Engineer
⚲ Gdańsk
Do uzgodnienia
Wymagania
- Microsoft Fabric
- Machine Learning
- Data Warehouse
- Lakehouse
- Python
Opis stanowiska
Nasze wymagania:
Hands-on experience in designing, building, and maintaining data platforms, particularly Microsoft Fabric or similar solutions.
Experience in Data Engineering, including data pipeline design, data transformation, and orchestration.
Strong understanding of Lakehouse architecture, Data Warehousing, and data modeling.
Experience implementing AI, Machine Learning, and Generative AI solutions.
Knowledge of Large Language Models (LLMs), vector databases, embeddings, prompt engineering, and Retrieval-Augmented Generation (RAG) architecture.
Experience with Microsoft AI technologies, including Azure AI Services, Azure OpenAI, Microsoft Copilot, and AI capabilities within Microsoft Fabric.
Ability to design and build data solutions that support AI and Machine Learning workloads.
Proficiency in Python, Spark, SQL, and data analytics/AI libraries.
Knowledge of MLOps and AIOps, including model deployment, monitoring, and lifecycle management.
Experience integrating systems through APIs, file-based interfaces, and event-driven architectures.
Hands-on experience with Azure DevOps, including repositories, work item management, and deployment pipelines.
Ability to manage services throughout their entire lifecycle, from design and implementation to maintenance and continuous improvement.
Strong troubleshooting skills related to data platforms, integrations, and AI services.
Ability to produce clear and well-structured technical documentation.
Understanding of security best practices, access control, and data governance.
Knowledge of Responsible AI principles, AI model governance, and ethical AI practices.
Ability to identify AI-related risks and implement appropriate safeguards.
Self-driven with the ability to collaborate effectively with both business and technical teams.
Strong communication skills, with the ability to explain technical concepts to non-technical stakeholders.
Results-oriented mindset with a focus on delivering high-quality, secure, and maintainable solutions.
Experience delivering enterprise data platform, analytics, and AI projects.
Practical experience in Data Engineering, Platform Engineering, and AI Engineering.
Hands-on experience with Microsoft Fabric, Azure Data Services, and AI technologies.
Ability to translate business requirements into scalable data and AI solutions.
Experience supporting and maintaining production AI services.
Experience working in regulated industries, such as the financial sector, is considered a strong advantage.
O projekcie:
You will be responsible for designing, implementing, maintaining, and enhancing enterprise data and AI platforms, reporting directly to the Head of Enterprise Technology. In this role, you will work primarily with Microsoft Fabric, ensuring the platform is secure, scalable, reliable, and well-governed. You will play a key role in delivering data services that support reporting, analytics, business process automation, and AI-driven solutions across the organization.
Zakres obowiązków:
Design, build, maintain, and enhance the Microsoft Fabric data platform, ensuring its availability, performance, scalability, and reliability.
Develop and maintain data pipelines for data ingestion, transformation, and delivery.
Design and manage Lakehouse and Data Warehouse solutions.
Monitor platform health, troubleshoot incidents, and ensure operational continuity.
Design, develop, and maintain data models for reporting, analytics, and AI workloads.
Build optimized datasets and semantic models.
Ensure data quality through validation, error handling, and adherence to data modeling best practices.
Design, implement, and maintain AI solutions using Microsoft AI and Azure AI services.
Develop Generative AI solutions, Retrieval-Augmented Generation (RAG) applications, intelligent search capabilities, and AI agents.
Prepare and manage data for AI models, including prompt engineering, model lifecycle management, and deployment.
Collaborate with business stakeholders to identify AI use cases and continuously monitor the quality and effectiveness of AI solutions.
Integrate source systems with the Microsoft Fabric platform.
Design APIs and data access services for reporting, business applications, and AI solutions.
Integrate AI services with enterprise applications while ensuring compliance with security and architectural standards.
Leverage Azure DevOps for source code management, work item tracking, and deployment automation.
Implement and maintain CI/CD and MLOps practices.
Create and maintain clear, comprehensive technical documentation.
Manage access control, auditing, and data security across the platform.
Ensure compliance with regulatory requirements and Responsible AI principles, including AI model governance, data lineage, monitoring, and risk management.
Translate business requirements into scalable data and AI solutions using Microsoft Fabric, Azure, and AI technologies.
Support the onboarding of new data sources, platform automation, and continuous platform enhancement.
Manage and prioritize the engineering backlog using Azure DevOps.
Act as a technical expert, providing guidance and resolving complex issues related to data platforms and AI solutions.
Hands-on experience in designing, building, and maintaining data platforms, particularly Microsoft Fabric or similar solutions.
Experience in Data Engineering, including data pipeline design, data transformation, and orchestration.
Strong understanding of Lakehouse architecture, Data Warehousing, and data modeling.
Experience implementing AI, Machine Learning, and Generative AI solutions.
Knowledge of Large Language Models (LLMs), vector databases, embeddings, prompt engineering, and Retrieval-Augmented Generation (RAG) architecture.
Experience with Microsoft AI technologies, including Azure AI Services, Azure OpenAI, Microsoft Copilot, and AI capabilities within Microsoft Fabric.
Ability to design and build data solutions that support AI and Machine Learning workloads.
Proficiency in Python, Spark, SQL, and data analytics/AI libraries.
Knowledge of MLOps and AIOps, including model deployment, monitoring, and lifecycle management.
Experience integrating systems through APIs, file-based interfaces, and event-driven architectures.
Hands-on experience with Azure DevOps, including repositories, work item management, and deployment pipelines.
Ability to manage services throughout their entire lifecycle, from design and implementation to maintenance and continuous improvement.
Strong troubleshooting skills related to data platforms, integrations, and AI services.
Ability to produce clear and well-structured technical documentation.
Understanding of security best practices, access control, and data governance.
Knowledge of Responsible AI principles, AI model governance, and ethical AI practices.
Ability to identify AI-related risks and implement appropriate safeguards.
Self-driven with the ability to collaborate effectively with both business and technical teams.
Strong communication skills, with the ability to explain technical concepts to non-technical stakeholders.
Results-oriented mindset with a focus on delivering high-quality, secure, and maintainable solutions.
Experience delivering enterprise data platform, analytics, and AI projects.
Practical experience in Data Engineering, Platform Engineering, and AI Engineering.
Hands-on experience with Microsoft Fabric, Azure Data Services, and AI technologies.
Ability to translate business requirements into scalable data and AI solutions.
Experience supporting and maintaining production AI services.
Experience working in regulated industries, such as the financial sector, is considered a strong advantage.
O projekcie:
You will be responsible for designing, implementing, maintaining, and enhancing enterprise data and AI platforms, reporting directly to the Head of Enterprise Technology. In this role, you will work primarily with Microsoft Fabric, ensuring the platform is secure, scalable, reliable, and well-governed. You will play a key role in delivering data services that support reporting, analytics, business process automation, and AI-driven solutions across the organization.
Zakres obowiązków:
Design, build, maintain, and enhance the Microsoft Fabric data platform, ensuring its availability, performance, scalability, and reliability.
Develop and maintain data pipelines for data ingestion, transformation, and delivery.
Design and manage Lakehouse and Data Warehouse solutions.
Monitor platform health, troubleshoot incidents, and ensure operational continuity.
Design, develop, and maintain data models for reporting, analytics, and AI workloads.
Build optimized datasets and semantic models.
Ensure data quality through validation, error handling, and adherence to data modeling best practices.
Design, implement, and maintain AI solutions using Microsoft AI and Azure AI services.
Develop Generative AI solutions, Retrieval-Augmented Generation (RAG) applications, intelligent search capabilities, and AI agents.
Prepare and manage data for AI models, including prompt engineering, model lifecycle management, and deployment.
Collaborate with business stakeholders to identify AI use cases and continuously monitor the quality and effectiveness of AI solutions.
Integrate source systems with the Microsoft Fabric platform.
Design APIs and data access services for reporting, business applications, and AI solutions.
Integrate AI services with enterprise applications while ensuring compliance with security and architectural standards.
Leverage Azure DevOps for source code management, work item tracking, and deployment automation.
Implement and maintain CI/CD and MLOps practices.
Create and maintain clear, comprehensive technical documentation.
Manage access control, auditing, and data security across the platform.
Ensure compliance with regulatory requirements and Responsible AI principles, including AI model governance, data lineage, monitoring, and risk management.
Translate business requirements into scalable data and AI solutions using Microsoft Fabric, Azure, and AI technologies.
Support the onboarding of new data sources, platform automation, and continuous platform enhancement.
Manage and prioritize the engineering backlog using Azure DevOps.
Act as a technical expert, providing guidance and resolving complex issues related to data platforms and AI solutions.
🔍 Dekoder Ogłoszenia
🟡
Hands-on experience in designing, building, and maintaining data platforms, particularly Microsoft Fabric or similar solutions.
Oczekuje się, że kandydat będzie aktywnie tworzył i zarządzał platformami danych, a nie tylko nadzorował.
🔴
Experience implementing AI, Machine Learning, and Generative AI solutions.
Może oznaczać zarówno tworzenie od podstaw, jak i integrację gotowych rozwiązań AI/ML.
🔴
Ability to manage services throughout their entire lifecycle, from design and implementation to maintenance and continuous improvement.
Kandydat będzie odpowiedzialny za wszystkie etapy życia usługi, co może oznaczać szeroki zakres obowiązków i potencjalne przeciążenie.
🟡
Strong troubleshooting skills related to data platforms, integrations, and AI services.
Oczekuje się, że kandydat będzie rozwiązywał problemy, które mogą być złożone i czasochłonne.
🟡
Ability to produce clear and well-structured technical documentation.
Oprócz pracy technicznej, kandydat będzie musiał poświęcić czas na tworzenie dokumentacji.