N100FN100F
Published on

Data first

Compute moves easily; data does not. Where each piece of data lives, how much it costs to move, how long it takes and what law forces it to stay decides the architecture more than any service.

Service line Cloud Architecture

Data has weight

An application can be deployed in another region in minutes. Moving several terabytes of data takes hours or days, costs money in transfer and may be restricted by law. That is why, in a cloud architecture, data attracts everything else: compute ends up going to where the data is, not the other way round.

Designing by starting with the applications and leaving data until the end is the surest way to end up with expensive transfers, unexpected latency and compliance problems.

Four questions per piece of data

Where data lives and what it costs to moveRegions and environments with data flows marked by illustrative monthly volume and cost.ARCHITECTURE · DATAWhere data lives, and what it costs to movePRIMARY REGION Database Analytics AppSECOND REGION Replica AppON-PREMISES System Local databaseOUTSIDE THE PROVIDER Users BI tool2 TB/month40 €300 GB/month24 €8 TB/month680 €50 GB/month4 €The most expensive flow is the one nobody designed:taking raw data out to the BI tool. Aggregating it beforeit leaves would cut it to a fraction.Illustrative data; they do not correspond to any real caseN100F · n100f.com

  1. Where does it live? Region, provider and service. And if it has copies, where they are.
  2. Who reads it and from where? Every read from another region or from outside the provider has a cost and a latency.
  3. How much moves? The monthly volume transferred between zones, between regions and outbound. Normally what comes in is free and what goes out is paid for.
  4. What constrains it? Data protection regulation, residency required by contract or by sector, retention periods.

Mistakes that are paid for every month

  • Separating the application from its database into different regions: every query crosses the network and it shows in latency and on the bill.
  • Copying data 'just in case' to other regions or providers with nobody reading it.
  • Taking raw data out of the provider to analyse it elsewhere, when it could be aggregated before leaving.
  • Forgetting residency until the first audit.

The right order

  1. Inventory the data: what there is, how much, whose it is and how sensitive.
  2. Decide where each set lives, with its legal and recovery requirement.
  3. Place compute close to the data it uses most.
  4. Design the unavoidable flows between zones and estimate their cost before deploying them.

Once the data is placed properly, most of the other decisions become obvious.

How we apply it: Cloud Architecture.

If your cloud cost keeps growing and nobody knows why, or your architecture depends on whoever built it, let’s talk.

Let’s talk