Actium: A Databricks Platform Built to Last

Actium is a Dutch housing corporation based in Assen, in Drenthe, responsible for thousands of homes and the tenants who live in them. Its data platform ran on Databricks and fed reporting on homes, tenancies, rents and maintenance.

The platform worked, but it had grown organically. Notebooks were edited directly in the workspace, jobs were configured by hand, and there was no single place that said what was running in production or why. Each new request was built on top of the last one. The result was a platform that cost more than it should, broke more often than it should, and that nobody liked to touch.

The platform did not need new technology; it needed structure. We redesigned it from the ground up along medallion principles: raw data in bronze, cleaned and checked data in silver, business-ready data in gold, with a clear boundary between each layer.

We restructured the data models to match how Actium actually works, around homes, tenants, contracts and repairs, rather than how the original implementation had happened to grow. And we rebuilt the pipelines with maintainability as a first-class concern: readable code, documented transformations, and nothing that only one person understands.

The biggest change was invisible to the business. We brought the whole platform into Databricks Asset Bundles. A bundle is a project in Git that describes everything Databricks needs: the code, the jobs and pipelines that run it, the schedules, the compute, and the environments it deploys to. One configuration file, databricks.yml, ties it together.

That changes how a platform is run. Nothing is clicked together in the workspace any more; the bundle is the single source of truth. The same bundle deploys to a development, an acceptance and a production workspace, and only the settings per target differ. Click through the bundle below, and switch the target to see what changes.

On top of the bundles we set up CI/CD pipelines in Azure DevOps. Every change starts as a user story on the board. The work happens on a branch, and the pull request is linked to that story, so anyone can see why a line of code changed. Automated checks validate the bundle and run the tests, Actium's own team reviews the change, and only then is it merged and deployed, first to acceptance and then to production.

Compare the two ways of working below. Click a step to see what happens in it.

The pull requests did more than protect the platform. Because every change carries its own story, its own tests and its own explanation, Actium's team could review our work whenever it suited them, and we could pick up their feedback without a meeting in between. The cooperation ran smoothly almost without having to speak to each other.

We did keep in regular contact, of course. But the conversations could be about priorities and direction, not about what had changed where.

We moved the existing workloads onto the new platform step by step, running old and new side by side and comparing the results until they matched. Daily reporting kept running throughout, and the business did not notice the switch.

The result is a Databricks platform that costs less to run, breaks less often, and that Actium's own team can extend with confidence. Every part of it is in code, every change has a reason, and a new pipeline is a pull request, not a project.

• A Databricks platform redesigned from the ground up on medallion architecture principles, with a clear separation between raw, cleaned and business-ready data.
• Data models restructured to match how Actium operates, not how the original implementation had grown.
• Pipelines rebuilt for maintainability: readable code and documented transformations.
• The whole platform in Databricks Asset Bundles, deployed to development, acceptance and production from one source.
• CI/CD pipelines in Azure DevOps where every pull request is linked to a user story, tested automatically and reviewed by Actium's team.
• Existing workloads migrated onto the new platform without interrupting daily operations.