⇤ home


Oct. 1st, 2023

Sharing is caring

The minute computers were born, people rushed to find ways to share them. Like any scarce and prohibitively expensive resource, borrowing one made economical sense if you only needed it temporarily, or if demand outstripped supply. In 2023, you rent an H100 in the cloud or an Airbnb facing the Eiffel tower for the same reason.

Glen Beck and Betty Snyder program the ENIAC in building 328 at the Ballistic Research Laboratory

Interestingly, once personal computers became affordable, the need for sharing their resources did not disappear: it just changed timescale. A modern operating system is mostly in the business of sharing resources between processes. Take the task scheduler: its main purpose is to give the illusion of concurrency to thousands of threads contending for a very small number of CPUs. It does so at a timescale that can be as low as a few microseconds.
Memory management is also crucial, in a time when people routinely discuss loading LLMs with billions of parameters into a GPU memory. In fact, "memory is the new CPU": incompressible and hard to oversubscribe, it is the most scarce resource of modern computing.
Underneath the operating system, the hardware itself is busy sharing its resources: whether it's caches or execution units through pipelining, keeping the silicon idle is the ultimate sin. Though at this level the motivation is usually speed and not cost, the two are often interchangeable when you borrow someone else's resources in the cloud: time is money.

CFOs darling, perf engineers foe

While multitenancy is so obiquitous in personal computing that it is often forgotten, discussing multitenancy in the cloud is often a controversial topic. There are two main categories of reasons for that:

Making it happen

The technical challenges are hard, but not insurmountable. The organizational ones are harder, because they involve people. Designing a multi-tenant system carefully trading off performance, cost and security means making a lot of different people in the organization happy. Things that help: Finally, since the changes needed are likely spanning a huge part of the engineering organization, keeping the number of decision makers small helps get things done.

Is it worth it?

It depends of course. We made it work financially at Netflix for a big chunk of containerized compute, but it wasn't without sweat. A long era of high-interest rates might push more companies to explore similar ways to be more efficient with their cloud spend.