When a Bank Outgrows Colocation
The decision to build a private data center usually comes from one of two places: regulatory pressure or business growth. In our case, it was both. The Central Bank's requirements for data sovereignty and physical security controls made colocation increasingly difficult, and the bank's transaction volumes demanded infrastructure we could fully control.
Building a data center isn't an IT project — it's a construction project with IT at its core. That distinction matters more than you'd think.
Lesson 1: Start with AutoCAD, Not Visio
Network engineers love drawing topology diagrams in Visio or draw.io. But a data center is a physical building. You need architectural plans:
- Floor load capacity — a fully loaded rack weighs 800-1200 kg
- Cable pathways — overhead trays, raised floor routing, fire-rated penetrations
- Power distribution — UPS placement, generator switchgear, PDU routing
- Cooling zones — hot aisle/cold aisle containment, CRAH unit placement
I spent the first month learning AutoCAD well enough to collaborate directly with architects and MEP engineers. That investment saved months of miscommunication later.
Lesson 2: The Construction Site Is Not Your Office
Managing contractors who build physical infrastructure requires a different skill set than managing IT teams. Key differences:
- Daily site visits are mandatory — problems caught on day 1 cost 10x less to fix than problems caught on day 30
- Specifications must be exhaustive — contractors build exactly what's specified, nothing more
- Weather, supply chains, and labor shortages affect timelines in ways software projects don't experience
- Change orders are expensive — think your design through before pouring concrete
Lesson 3: Design for the Next 5-7 Years
A data center is a long-term investment. The decisions I'm most grateful for:
- Oversizing power capacity by 40% — we've already used 60% of "excess" capacity
- Running extra fiber between rooms — dark fiber is cheap during construction, expensive after
- CLOS network architecture from day one — adding leaf switches is trivial compared to re-architecting core/aggregation
- Standardized rack layouts — every rack uses the same template, making operations predictable
The decisions I'd change:
- Starting Nutanix migration sooner — we initially planned to reuse existing VMware licenses, then Broadcom happened
- More generator fuel storage — our initial 8-hour capacity is below what I'd recommend now (24 hours minimum)
Lesson 4: High Availability Is a System, Not a Feature
Banking regulators expect near-zero downtime. That means:
- Redundant everything — dual power feeds, dual cooling, dual network paths
- PostgreSQL HA with automatic failover — HAProxy + Keepalived provides transparent database availability
- Tested disaster recovery — DR plans that aren't tested regularly are fiction
- Network segmentation — production, management, and DR traffic on separate VRFs
The most important HA lesson: test your failover regularly. We discovered issues in our PostgreSQL failover during a planned test that would have caused a 45-minute outage if they'd occurred in production.
Lesson 5: Documentation Is Infrastructure
By the end of the project, we had documented every cable, every power circuit, every network port in NetBox. This wasn't just best practice — it was a regulatory requirement.
But documentation also paid operational dividends:
- New team members could understand the infrastructure in days, not weeks
- Troubleshooting started with accurate data instead of guesswork
- Capacity planning used real utilization data from monitored infrastructure
- Audit compliance was straightforward with up-to-date records
The Result
The data center was commissioned on schedule. All regulatory requirements were met. Banking services achieved the high availability targets. And the team gained experience that no amount of vendor training could provide.
If your organization is considering building a private data center, the most important investment isn't in hardware — it's in experienced planning. The decisions you make in the first month determine the operational reality for the next decade.