Building a data lakehouse with Databricks and ADLSG2, MEL Data AI & Analytics.

Role: Data Engineer
Meridian Energy Limited

Achievements:

  • Loaded 600 tables into a Databricks proof-of-concept (Delta Live Tables).
  • Rebuilt 50 Power BI reports using dataflows.

Responsibilities:

  • Requirements development (enablement stories, acceptance criteria).
  • Azure SQL Database design and development (tables, views, transformations).
  • Dimensional model design and development (facts, slowly changing dimensions).
  • Power BI development (dataflows, semantic models, apps, pipelines).
  • Data Warehouse monitoring (Azure Data Factory, SQL Agent jobs).

Technology:

  • Databricks
  • Azure Databricks, Azure SQL, Azure Data Factory, Azure Data Lake Generation 2, Power BI,
  • Microsoft SQL Server database engine (MS SQL).

2023

Updated: