# A private AI platform for a financial services firm

- **Client:** European financial services firm (name confidential)
- **Sector:** Fintech
- **Period:** 2026 – present
- **Model tiers:** 7, each with an EU fallback
- **Services in the stack:** 9, all on pinned versions
- **Source databases:** 6 SQL Server databases
- **Processing:** European endpoints only

A financial services firm wanted its staff to work with AI, without customer data leaving Europe and without depending on a single provider. We built its own AI platform with a model gateway, EU routing that blocks when it cannot be guaranteed, and a masked copy of production data that AI can answer questions about.

## The challenge

- Staff needed AI support, but public tools are not an option for financial and personal data.
- The organisation had to be able to show where prompts and completions are processed.
- Analysts wanted to ask questions about production data, which is full of personal data and lives in SQL Server.

## What we built

- **Model gateway with tiers per task.** One gateway with seven model tiers: from fast and cheap for everyday questions to heavy models for deep analysis. Every tier has a fallback route to a second European provider.
- **EU routing that fails closed.** All requests are pinned to European endpoints. If a model has no EU endpoint, the EU fallback route takes over; if there is none, the request is refused. Open-source models only run on European infrastructure and are excluded from credit decisions.
- **Masked analytics copy.** A repeatable pipeline turns backups of six SQL Server databases into PostgreSQL: restore into a throwaway environment, load, mask, verify and publish. Personal data is replaced deterministically; keys and amounts stay intact.
- **Verification as a hard gate.** Before publishing, the pipeline checks row counts, exact equality of money amounts between source and target, the presence of relations and possible PII leaks.
- **Search quality measured, not assumed.** An evaluation script measures recall and MRR of search in the knowledge bases. On that basis we chose a deterministic pre-processing step that gives every passage its document title and headings, instead of a costlier variant in which a model rewrites each passage. It needs no model calls and no text leaves the environment.
- **A separate, locked-down analyst environment.** Questions to the database go through a dedicated terminal with a role that may only read the masked copy. We added that separation after seeing that a shared terminal could reveal secrets of other services.
- **Hardened infrastructure.** Users get in through Cloudflare Zero Trust with single sign-on. The nine services publish no ports to the internet themselves, all images are pinned and containers cannot gain extra privileges. An audit of the running server recorded what was still open and the order in which it can be closed safely.

## Outcome

- Staff work with AI in their own environment, without prompts or completions being processed outside Europe.
- Analysts can ask questions in plain language about a read-only copy that contains no real personal data.
- Every architecture choice is recorded with its reason in a decision document compliance can read.
- The organisation is not tied to one AI provider: switching a model is a configuration change.

## Technology

Open WebUI, LiteLLM, OpenRouter EU, Amazon Bedrock EU, PostgreSQL + pgvector, SQL Server, pgloader, Presidio, Docling, Redis, Cloudflare Zero Trust, Docker Compose

## Services involved

- [Private AI platform](https://mg-devs.nl/en/services/private-ai-platform/)
- [AI infrastructure and security](https://mg-devs.nl/en/services/ai-infrastructure-security/)
- [Data engineering](https://mg-devs.nl/en/services/data-engineering/)
- [AI solutions and agents](https://mg-devs.nl/en/services/ai-solutions/)

---
Source: https://mg-devs.nl/en/cases/private-ai-platform-fintech/
