Fintech
Fintech company takes analytics off its production database
A Vertica warehouse and BI dashboards built from scratch, designed to migrate easily later and to keep control of the data in-house
Data warehouse development

Who we worked with
Fintech company
Our client is a fintech company whose reports ran on the same production database as its transactions. It wanted a data warehouse and BI built from scratch.
What we did
From production queries to a warehouse
Reports were running on the production database
Analytics ran as queries straight on the production database. Reports were slow, and running them disrupted transactions. Processes were unregulated and BI expertise in the company was thin, and our client wanted its data to stay fully under its own control.
A proposal with every tool's trade-offs on paper
We started with an architecture proposal. It listed the candidate tools with the advantages and disadvantages of each, and we agreed it with top management. Then we led the build against the agreed requirements. A team of seven ran it over nine months, with two senior data engineers, a data engineer, two BI developers, a DevOps engineer and a project manager who also worked as the business analyst.
The warehouse, its pipelines and the dashboards
The warehouse is Vertica and SQL. The pipelines are Python and Apache Airflow on Kubernetes (Amazon EKS). Our DevOps engineer set up the cluster, CI/CD and infrastructure as code. Three new processing systems were integrated. Dashboards in Tableau and Apache Superset give top management the numbers its decisions rest on. Requests for data and reports now go through Jira Service Desk, and after it came in, the team handling them and the reporting had about 20% less operational workload.

How it was designed
A warehouse built to be moved later
Picking a first platform means guessing at future needs
A first warehouse platform is picked before anyone can see what reporting will need in a few years, so the design should make a later move cheap. The transformation logic lives in Python and Apache Airflow, outside Vertica, and the model is plain SQL. A later move means repointing the pipelines and leaving their code as it is. Control of the data stayed with the client. The warehouse and its pipelines ran in its own AWS account, and our engineers worked through named accounts it issued, each with only the access its role needed.
What runs through the warehouse now
All regular reports now run through the warehouse, and it kept running without issues long after the engagement ended.
The stack
Warehouse, pipelines, BI and the request desk
Vertica
SQL
Python
Apache Airflow
Kubernetes
Amazon EKS
Tableau
Apache Superset
Jira Service Desk
The project in numbers
0
Analytical load left on production
3
New processing systems integrated
9 months
Engagement, with a team of seven
Reports slowing down your production database?
Tell us what your reports run on today and who depends on them. We reply to every inquiry within 24 hours.



