# Complex data architecture and migrations that match to the cent

> Migrations, data models and pipelines where you can prove afterwards that everything is right.

We design and build complex data setups: migrations between databases, data models that keep evolving for years, and pipelines that make data safely available for analytics and AI. What sets us apart is that we build in verification: counts, keys and amounts are automatically compared with the source after every run.

## When to call us

- You want to move from SQL Server or another legacy system to PostgreSQL.
- You want AI or analysts to work on production data without exposing personal data.
- Several systems share a data model and every change threatens to break something.
- Your data is spread across systems and nobody trusts the numbers.

Not for those who only need dashboards; we build the data layer underneath and gladly work with your BI partner.

## What you get

- **Database migrations.** Schema-first migration preserving keys, relations and indexes, repeatable and without touching production.
- **Masking that preserves consistency.** The same real value becomes the same fake value everywhere, so joins and analyses stay correct.
- **Verification as a gate.** A go/no-go check on row counts, exact equality of money amounts, presence of relations and PII leaks.
- **Data models that evolve.** Additive, backwards-compatible changes for systems whose old clients keep existing.
- **Data ready for AI.** Read-only roles, translated and curated views and vector search, so a model can answer questions safely.

## How we approach it

1. **Understand the source.** We inventory tables, relations, sensitive fields and the questions the data must be able to answer.
2. **Repeatable pipeline.** Restore into a throwaway environment, load, mask, verify, publish. Every step can be re-run.
3. **Live only after proof.** The verification step must pass before anyone, human or model, gets access.

## Technology

PostgreSQL, pgvector, SQL Server, pgloader, Presidio, Python, Firestore, Supabase, Redis, Docker

## Proof from our own work

- Masked PostgreSQL copy of six SQL Server databases for a financial services firm, with a repeatable pipeline and automated verification. ([A private AI platform for a financial services firm](https://mg-devs.nl/en/cases/private-ai-platform-fintech/))
- One data contract between a live app and an admin portal, spread over separate databases per organisation, without breaking old app versions. ([One platform, seven apps: the carnival platform](https://mg-devs.nl/en/cases/carnival-app-platform/))

## Frequently asked questions

### How do you migrate from SQL Server to PostgreSQL without risk to production?

By never working on production. We restore a backup into a temporary environment, migrate from there and automatically compare the result with the source. The pipeline is repeatable, so a failed run only costs time.

### How do you let AI work safely with company data?

With a separate, read-only copy in which personal data is masked and keys and amounts stay intact. The model gets a role with read access to curated views only.

### What does 'backwards compatible' mean for data?

That a change does not break existing users. You add fields instead of renaming or removing them, and keep old fields populated for as long as software exists that reads them.

---
Source: https://mg-devs.nl/en/services/data-engineering/
