Project
Lead Data Engineer (m/f/d) for the BMP Migration Tool
ID: 10352
Our client is a service company that operates within an international corporate group.
Must haves
- Databricks
- Lakeflow
Your Tasks
- Provide hands-on technical leadership for the BMP data migration.
- Lead the profiling and assessment of source data.
- Define and implement data mappings and transformation rules.
- Develop and maintain data pipelines using Databricks, PySpark, Spark SQL and Delta Lake.
- Establish and oversee data quality checks and monitoring.
- Lead source-to-target reconciliation and resolve data discrepancies.
- Guide testing, CI/CD deployment and code quality practices.
- Prepare runbooks and support cutover activities
Requirements
- Strong hands-on experience delivering data migrations and building production-grade data pipelines.
- Extensive practical experience with Databricks, including Unity Catalog, Delta Lake, Lakeflow Connect, Lakeflow Declarative Pipelines and Lakeflow Jobs.
- Strong PySpark and Spark SQL skills.
- Experience with data profiling, source-to-target mapping, reconciliation and data quality checks.
- Familiarity with DQX, Databricks Data Quality Monitoring and Lakebridge.
- Experience with Databricks Asset Bundles, CI/CD and automated testing.
- Ability to prepare runbooks, troubleshoot migration issues and support cutover activities.
- Proven technical leadership, including guiding implementation and coordinating delivery.
- Required knowledge of: Databricks, Unity Catalog, Lakeflow Connect, Lakeflow Declarative Pipelines, Lakeflow Jobs, Delta Lake, PySpark, Spark SQL, DQX, Databricks Data Quality Monitoring, Lakebridge, Databricks Asset Bundles, CI/CD, Lakehouse Federation and AI/BI Dashboards.
- Professional English communication skills; willingness to work in a hybrid setup.
- Availability to start as soon as possible.
The content of this advertisement relates to an order from our customer company in the context of personnel placement. As a recruiter, we support you throughout the entire application process.
Our Xtra Advantages
- 20 years of IT-recruitment experience
- access to a huge talentpool
- individual consulting.