When you need more than a step change to move to cloud
In the old days (been there, done that), if you were part of team responsible for building and deploying an application, it was more than likely that you had to deal with the infrastructure team so that you could get some compute real-estate to run the darn thing on. Depending on the size of your application, whether you needed a cold/warm/hot standby duplicate, whether you needed some headroom for scaling, and then how many locations/offices, the amount of bare metal compute you needed was probably not insignificant. Luckily for you, the IT Infrastructure team comes to the rescue and ignites the engine that roars to life with the noise we call procurement.
Lucky for us in financial services, it wasn't the numerous directives, regulations, policies and guidance associated with the public/government sector. However, it would entail ensuring that the procurement team run through a process getting the very most for as little money as possible. In my early City days, at Reuters, one bank I worked with loved the market data distribution system so much that fired up the procurement process to get the very best compute they could buy. Numerous cycles of proof of concept, benchmark, and by the time they close to the end of the procurement process, Moore's Law had run interference, whereby the compute power was obsolete and they had to start again with the next generation. Another more recent experience, another bank, went through this process, over time, acquiring 250 servers for their small 'distribution' system, resulting in a lengthy RFP process, negotiation, then the datacentre space, then the people required, etc. Imagine the complex business case that sat around this, essentially saying "if we spend £3m we'll have a distribution system", and, "in another 5-7 years time we'll have to do all over again". All of this was predicated on the fact that you had the foresight and timing to work this into the annual budget approval process. This was and is the process of capital expenditure (Capex), depreciation, long term commitment, technical refresh etc. Rinse/repeat. One of the star qualities with this process is fiscal predictability. You knew how much you're spending, what you've got, and a predicted end of life, all wrapped up in a nice set of accounting handcuffs.
Organisational Layers
These projects and many others all go through this step process, individually taking a business-as-usual state through a step change to a new target operating model. The groups of people involved in this transformational journey includes; the business, engineering, infrastructure, finance, operations, etc. As each group hands off to the next there's friction, in terms of the bilateral handoff between each of the layers of abstraction.
The big question is; are these traditional methods of procurement and IT Infrastructure fit for purpose for an organisation to embrace the cloud. I've already mentioned that from a budgeting standpoint, the traditional layered purchasing route buys you predictability, but counter that with cloud where you are firmly in the usage-based billing, elastic territory. This is most definitely a transition from CapEx to OpEx. Functionally, in a traditional environment you've transitioned to a new snapshot, a new operating model, where the cloud approach encourages constant evolution (DevOps, Agile, etc). Building resilience for business continuity in a traditional approach always meant redundancy (hot/warm/cold standbys with idle compute) whereas in a cloud environment it's self healing, automation, etc.
Another startling contrast on the traditional method of acquiring IT Infrastructure is that of transparency, or lack of. How much extra compute do you purchase in terms of headroom for the future. How much are you using? was it enough headroom. Is it fast enough? How often is it used? With cloud you get full transparency, down to the second, coupled with automation. Need more compute for that end of day counterparty (XVA) risk, auto-scale more compute nodes on the risk cluster, and when it's done, auto-scale back down.
Organisational Transformation
If you look at the agile ethos, it's built on constant change always providing incremental business benefit to customers/clients. There's always a trajectory and a vision to constantly move the business forwards. In some way, should a business that's going to be cloud dependent adopt a similar approach. That may mean forgetting the annual "have you got any good ideas (projects) as we need to budget" syndrome and moving to a sprint like quarterly approach.
Organisationally, the layered approach could benefit from change as well, removing the abstraction and friction and empowering a business pillar from everything from developing, to build/deploy, to provisioning, to operating. Full end to end DevOps on steroids. IT infrastructure could become responsible for shared services such as the telemetry/observability frameworks, collection and also providing core orchestration capability.
At a recent event, one of the speakers spoke about how her organisation had moved to the quarterly agile approach, and, removed some of the elasticity by using reserved instances. Another large institution had also adopted the vertical (vs horizontal/friction) approach but, had morphed IT infrastructure into the guardians of Terraform (IaC) and creating friction, so not quite there.
Organisational challenges lay in store for anyone adopting cloud, in fact, technology becomes the least of your problems because now it's all on tap. Anyone aspiring to move to cloud that hasn't considered the organisational implications, is probably going to find the transition challenging. Consider moving from a friction-based horizontal layered approach to a frictionless agile vertical pillar.
If you have any counter views, suggestions, what worked for you, it would be great to hear your experience.
