Architecture
Daily Azure deployment pipeline.
An end-to-end cloud data engineering project built with Docker, Microsoft Azure, GitHub Actions and Django.
GitHub Repository Deployment BlogThe dataset is automatically refreshed every day using Azure Data Factory and Azure Container Instances.
Daily Azure deployment pipeline.
Originally developed as a take-home assignment, this project evolved into an end-to-end cloud data engineering application. It combines Docker, Microsoft Azure, GitHub Actions, Django and Plotly for deploying and visualising reproducible data pipelines. While it uses public weather data, the architecture is designed to be reusable across many real-world data sources across many industries.