WEBVTT

00:00:00.580 --> 00:00:03.431
How many machines did you put
the same tool on this year?

00:00:03.700 --> 00:00:04.760
I stopped counting.

00:00:05.260 --> 00:00:08.160
A Mac, a Windows laptop, a Linux runner:

00:00:08.820 --> 00:00:10.820
three ways to install one tool.

00:00:11.280 --> 00:00:12.460
And nobody blinks.

00:00:12.840 --> 00:00:13.673
Never.

00:00:13.680 --> 00:00:14.780
It's just the tax.

00:00:15.760 --> 00:00:18.900
This is the OCX Podcast for engineers

00:00:18.960 --> 00:00:23.040
who install tools on laptops
and in pipelines. I'm Vera.

00:00:23.320 --> 00:00:24.153
And I'm Max.

00:00:24.480 --> 00:00:26.440
And I'd push back on that word, Vera.

00:00:26.900 --> 00:00:28.080
Nobody's charging you.

00:00:28.540 --> 00:00:29.880
Everyone's paying, though.

00:00:30.660 --> 00:00:33.380
That's episode zero, why it's still true,

00:00:33.600 --> 00:00:36.100
and where the fix has
been sitting all along.

00:00:36.460 --> 00:00:39.020
Monday morning, new hire, new Mac.

00:00:39.560 --> 00:00:42.040
The setup doc was written
years ago for Windows.

00:00:42.480 --> 00:00:44.480
Step one is a Windows installer.

00:00:45.000 --> 00:00:49.600
That step doesn't exist on a Mac, so they
follow the tool's own docs instead.

00:00:50.200 --> 00:00:53.160
A package manager, whatever
version it ships today.

00:00:53.580 --> 00:00:54.840
Fine, same tool.

00:00:55.260 --> 00:00:58.620
Same name, different version,
different place on disk.

00:00:59.100 --> 00:01:02.140
Half the doc's commands
and configs land somewhere else.

00:01:03.140 --> 00:01:03.973
I've done that.

00:01:04.440 --> 00:01:08.240
Rewrote the doc for my machine.
Next hire got a Linux laptop.

00:01:08.600 --> 00:01:09.700
Then CI.

00:01:10.240 --> 00:01:12.891
The runner takes it
from the distribution's packages.

00:01:13.340 --> 00:01:16.660
Two years behind,
the release freezes versions.

00:01:16.940 --> 00:01:17.841
That's not broken.

00:01:18.060 --> 00:01:19.961
That's the distribution doing its job.

00:01:20.000 --> 00:01:20.833
It is.

00:01:21.100 --> 00:01:24.100
One tool, three machines, three versions.

00:01:24.640 --> 00:01:26.541
Everyone did what their platform said.

00:01:26.880 --> 00:01:28.440
Nobody did anything wrong.

00:01:29.020 --> 00:01:31.660
So why does one tool need three ways in?

00:01:31.960 --> 00:01:34.720
The honest answer isn't a bug report.

00:01:35.220 --> 00:01:39.540
Getting one tool onto every machine
that needs it isn't a hard problem.

00:01:39.940 --> 00:01:41.440
It ate somebody's first day.

00:01:41.800 --> 00:01:45.500
And it comes back with every version
bump, every machine,

00:01:45.880 --> 00:01:47.760
every platform again.

00:01:48.200 --> 00:01:52.680
So people stop bumping, and if you're
the platform team, you can't easily move

00:01:52.720 --> 00:01:54.971
the version everyone
else's projects require.

00:01:55.240 --> 00:01:59.400
Every team pays that separately
and pays again for the next tool.

00:01:59.780 --> 00:02:03.960
Where does it actually get paid then?
Installing a tool is one line in a doc.

00:02:04.260 --> 00:02:05.740
One line per platform.

00:02:06.360 --> 00:02:11.140
Every operating system on the team has
its own idea of where a binary lives

00:02:11.300 --> 00:02:13.020
and how it lands on your path.

00:02:13.300 --> 00:02:17.740
On Windows, the terminal you already had
open never hears about the path change.

00:02:18.100 --> 00:02:19.351
Neither does your editor.

00:02:19.760 --> 00:02:23.520
You restart both, and by then,
it looks like the install failed.

00:02:23.900 --> 00:02:26.120
Linux, at least, is one platform.

00:02:26.480 --> 00:02:29.560
Linux is where one build
stops being one build.

00:02:30.120 --> 00:02:33.940
Two C libraries are in common
use, glibc and musl,

00:02:34.300 --> 00:02:37.660
and a binary linked against one
won't start under the other.

00:02:38.040 --> 00:02:40.760
Two builds then for one operating system.

00:02:41.220 --> 00:02:42.800
At least two.

00:02:43.720 --> 00:02:45.700
Call it the libc tax.

00:02:46.000 --> 00:02:49.301
A team that's been at this a while doesn't
live like that, though.

00:02:49.380 --> 00:02:50.213
They pin it.

00:02:50.260 --> 00:02:53.000
In their own packages
with the version baked in,

00:02:53.380 --> 00:02:56.381
installed the way the distribution
installs everything else.

00:02:56.660 --> 00:02:58.511
That sounds like the grown-up answer.

00:02:58.560 --> 00:03:01.240
It works for whoever then owns it.

00:03:02.060 --> 00:03:05.380
Every distribution the runners
use needs its own packaging.

00:03:05.940 --> 00:03:09.860
None of it reaches the laptops,
and installing it needs root.

00:03:10.200 --> 00:03:12.180
And it holds one version per machine.

00:03:12.540 --> 00:03:16.920
So moving between two projects
means reinstalling the other version

00:03:17.380 --> 00:03:20.340
and hoping nothing else leaned
on the one you removed.

00:03:20.680 --> 00:03:24.480
That's where dev containers come in,
a whole environment per project.

00:03:24.760 --> 00:03:26.511
Which helps where teams adopt them.

00:03:26.980 --> 00:03:30.160
Plenty haven't, so the laptop
still needs the tool.

00:03:30.440 --> 00:03:34.320
And a team that pins nothing
just takes whatever the CI runner ships.

00:03:34.720 --> 00:03:36.240
Until the runner changes.

00:03:36.820 --> 00:03:40.860
In early two thousand twenty-five,
a hosted CI provider moved

00:03:40.900 --> 00:03:44.300
its default Linux runner to the next
release of the distribution.

00:03:44.880 --> 00:03:47.660
The new image shipped
fewer pre-installed tools.

00:03:48.080 --> 00:03:52.420
So a tool you never installed because it
was always just there is gone.

00:03:52.760 --> 00:03:54.380
Nobody edited a workflow.

00:03:54.880 --> 00:03:56.880
The meaning of latest changed.

00:03:57.200 --> 00:04:00.601
For some projects, that runner
is also where the release gets built.

00:04:00.920 --> 00:04:02.720
Then the libc tax follows.

00:04:03.140 --> 00:04:05.620
The release is built
against the runner's GLC,

00:04:06.040 --> 00:04:09.120
so the runner decides
the oldest Linux it runs on.

00:04:09.400 --> 00:04:12.380
And less than two years later,
that default moved again.

00:04:12.640 --> 00:04:16.279
A laptop does the same thing,
just without a red pipeline to tell you.

00:04:16.600 --> 00:04:19.040
Its package manager keeps its own copy

00:04:19.079 --> 00:04:23.320
of a language interpreter, and the tools
you install through it share that copy.

00:04:23.660 --> 00:04:25.061
Until something upgrades it.

00:04:25.220 --> 00:04:27.420
Because something else wanted a newer one.

00:04:28.100 --> 00:04:30.400
Tools you haven't touched in months break.

00:04:31.120 --> 00:04:32.221
Here's the other half.

00:04:32.560 --> 00:04:36.340
People love that package manager
because its versions are current.

00:04:36.760 --> 00:04:38.440
Oh, that's the same property.

00:04:38.760 --> 00:04:40.840
Current and unpinned are one thing.

00:04:41.340 --> 00:04:43.941
You find out which one you
wanted after the upgrade.

00:04:44.280 --> 00:04:47.081
Then pinning through the system's
packages is too stiff,

00:04:47.340 --> 00:04:48.941
and not pinning moves under you.

00:04:49.520 --> 00:04:51.580
The project has to pin its own tools.

00:04:51.880 --> 00:04:55.720
For a small team, at one version,
that means four places,

00:04:56.180 --> 00:05:01.120
the CI runners, a couple of laptops,
and the image the service gets built into.

00:05:01.440 --> 00:05:06.580
Each with its own mechanism, a package
list, a formula, an archive per platform.

00:05:06.840 --> 00:05:09.620
And an image pinned
to something slightly different.

00:05:10.200 --> 00:05:12.580
None of the four knows
the other three exist.

00:05:13.180 --> 00:05:17.160
Most teams end up with the honest
incumbent, one install script,

00:05:17.420 --> 00:05:19.120
and all four pointed at it.

00:05:19.400 --> 00:05:20.651
Which is a table, really.

00:05:21.060 --> 00:05:24.700
Platforms and download links edited
by hand at every version bump.

00:05:25.020 --> 00:05:28.360
A column for the operating system,
a column for the architecture.

00:05:28.660 --> 00:05:30.340
And nothing for the C library.

00:05:30.660 --> 00:05:32.820
Not which one and not how old.

00:05:33.400 --> 00:05:37.940
On an older Linux host, the script says
everything's installed, exits clean,

00:05:38.340 --> 00:05:41.020
and leaves a tool
that fails on its first run.

00:05:41.340 --> 00:05:45.860
Why is anyone still handwriting that
table? There are whole categories of tools

00:05:45.880 --> 00:05:46.981
for installing things.

00:05:47.220 --> 00:05:51.280
Each one's good at its own job,
and none of those jobs is this one.

00:05:51.920 --> 00:05:55.060
The distributions packages
manage an operating system.

00:05:55.580 --> 00:05:58.500
A language package manager
serves one ecosystem.

00:05:58.820 --> 00:06:01.171
And the releases page
doesn't install anything.

00:06:01.420 --> 00:06:02.340
It's file hosting.

00:06:02.680 --> 00:06:04.780
Build systems have the same problem.

00:06:05.432 --> 00:06:09.532
One that needs a tool at a specific
version carries its own downloader

00:06:09.832 --> 00:06:11.672
for every platform it runs on.

00:06:11.750 --> 00:06:13.312
What about the version managers

00:06:13.352 --> 00:06:15.152
that fetch the release file for you?

00:06:15.152 --> 00:06:19.332
The ones that pick it by filename break
when a project renames its files.

00:06:19.932 --> 00:06:23.692
And a publisher that ships through many
channels finds each one works

00:06:23.732 --> 00:06:26.132
for some users and none for all.

00:06:26.472 --> 00:06:28.752
So everyone's maintaining the same table.

00:06:29.092 --> 00:06:30.093
Back to Monday then.

00:06:30.552 --> 00:06:32.712
Why does one tool need three ways in?

00:06:33.032 --> 00:06:33.865
It doesn't.

00:06:34.352 --> 00:06:36.053
Add up what we've been asking for,

00:06:36.412 --> 00:06:39.972
every platform the team uses,
the right build for each host,

00:06:40.372 --> 00:06:41.652
versions that stay put.

00:06:41.972 --> 00:06:43.432
And no new service to run.

00:06:43.812 --> 00:06:45.363
That's the one that narrows it.

00:06:45.792 --> 00:06:50.652
Your team runs a system that stores
multi-platform artifacts, authenticated,

00:06:50.932 --> 00:06:55.692
access controlled over TLS.
It's the container registry.

00:06:56.012 --> 00:06:57.572
Which sorts by platform too.

00:06:58.252 --> 00:07:02.072
One name, a different image for each
operating system and architecture.

00:07:02.332 --> 00:07:04.992
The industry standardized
the box and the port,

00:07:05.272 --> 00:07:07.952
and the tools inside
still travel the old way.

00:07:08.292 --> 00:07:10.591
That's not us running
our tools in containers.

00:07:10.591 --> 00:07:11.424
Different job.

00:07:11.652 --> 00:07:12.592
It's transport.

00:07:13.032 --> 00:07:14.833
You already have the infrastructure.

00:07:15.072 --> 00:07:18.972
The registry your team already
runs is what OCX is built on.

00:07:19.472 --> 00:07:22.752
It's a package manager that moves
tools through that registry.

00:07:23.272 --> 00:07:24.473
Where does the pin live?

00:07:24.632 --> 00:07:25.652
In your repository.

00:07:26.152 --> 00:07:30.072
A file committed next to the code says
which tools this project needs.

00:07:30.492 --> 00:07:34.452
You clone it, pull, and the tools
are on your path in that project.

00:07:34.892 --> 00:07:38.412
The new hire from Monday opens
the repo instead of the setup doc.

00:07:38.752 --> 00:07:42.192
And gets the version the project
chose, built for their machine.

00:07:42.812 --> 00:07:45.612
CI reads the same file and the same lock,

00:07:45.992 --> 00:07:48.752
the same toolchain
on the laptop and in CI.

00:07:49.312 --> 00:07:52.532
A version bump is one change,
reviewed like any other.

00:07:52.992 --> 00:07:53.992
And the libc tax?

00:07:54.372 --> 00:07:55.205
Not all of it.

00:07:55.592 --> 00:07:57.912
Which family, glibc or musl,

00:07:58.212 --> 00:08:01.752
each host now picks for itself
from one index per tool.

00:08:02.292 --> 00:08:05.932
How old a glibc the build needs
is still set where it's built.

00:08:06.272 --> 00:08:07.352
We didn't answer how,

00:08:07.452 --> 00:08:08.285
though.

00:08:08.312 --> 00:08:11.352
A registry was built to hand
images to container runtimes,

00:08:11.792 --> 00:08:15.432
not a command line tool to a Mac,
a Windows laptop, and a Linux runner.

00:08:15.952 --> 00:08:17.132
That's episode one.

00:08:17.672 --> 00:08:20.352
Your registry is already
a package backend.

00:08:20.872 --> 00:08:22.952
Everything's at ocx.sh.

00:08:23.772 --> 00:08:26.072
Same toolchain on every laptop.

00:08:26.392 --> 00:08:27.492
And in CI.

00:08:28.112 --> 00:08:29.163
See you next episode.

