Podcast
Join Viktor, a proud nerd and seasoned entrepreneur, whose academic journey at Santa Clara University in Silicon Valley sparked a career marked by innovation and foresight. From his college days, Viktor embarked on an entrepreneurial path, beginning with YippieMove, a groundbreaking email migration service, and continuing with a series of bootstrapped ventures.
Links
Follow Me
Follow Me
Join Viktor, a proud nerd and seasoned entrepreneur, whose academic journey at Santa Clara University in Silicon Valley sparked a career marked by innovation and foresight. From his college days, Viktor embarked on an entrepreneurial path, beginning with YippieMove, a groundbreaking email migration service, and continuing with a series of bootstrapped ventures.
Unpacking Docker's Journey: Justin Cormack, on DevOps, Containerization, and the Future of Wasm
Play On
08 SEP • 2024
1 hour 27 mins
Share:
In this captivating conversation, Viktor sits down with Justin to explore the evolution of Docker, the challenges and triumphs of containerization, and the exciting potential of Wasm. As a key figure in Docker’s history, Justin shares his unique perspective on the company’s journey, its impact on the DevOps community, and how Docker continues to innovate in an ever-changing tech landscape.
The conversation begins with Justin’s first encounter with Docker, which sets the stage for his long and ongoing relationship with the company. Viktor and Justin then delve into the often-debated topic of containers vs. virtual machines (VMs), where Justin explains why this framing is misleading and how Docker’s real success lies in simplifying application packaging, deployment, and management.
As they navigate through the early days of Docker, Justin recounts the challenges the company faced in a small market with significant competition. Despite these hurdles, Docker emerged as a disruptive force, particularly among early cloud adopters who recognized the potential benefits of containers. Viktor and Justin also discuss Docker’s decision to start charging for Docker Desktop, which sparked controversy within the tech community.
The conversation then shifts to Docker Hub, its infrastructure, and the challenges of managing a global service at scale. Justin offers an inside look at how Docker has adapted to meet growing demands, from the early days when outages made headlines to the present where Docker Hub has become an indispensable resource for millions of developers.
Justin also takes Viktor on a journey through the evolution of Docker image security, explaining how early images had cryptographic signatures but no real content hashes, making them vulnerable. He discusses how Docker transitioned to the V2 format, introducing proper content hashing and significantly improving image security.
One of Docker’s latest innovations, Docker Scout, is another topic of discussion. Justin explains how this tool generates Software Bill of Materials (SBOMs) and tracks vulnerabilities in real-time, allowing developers to stay on top of security issues without needing constant re-scanning. Viktor and Justin explore the potential of WebAssembly (Wasm), discussing its possibilities in both server-side and browser-based applications.
As they conclude their conversation, Justin shares his thoughts on the future of Docker and cloud-native development. He emphasizes Docker’s focus on empowering developers and providing seamless tooling for containerized environments, which remains at the core of its mission.
Why listen to this episode? If you’re a developer, DevOps enthusiast, or tech leader interested in the evolution of containerization and Wasm, this conversation is packed with valuable insights. From Docker’s early days of rapid growth and market challenges to its present role as a leader in cloud-native development, Justin Cormack’s unique perspective offers listeners a front-row seat to the pivotal moments that shaped Docker and its impact on the tech world.
What will you learn? By tuning into this episode, you’ll gain a deeper understanding of Docker’s journey, the lessons learned along the way, and where it’s headed next in the world of cloud-native and containerized development. You’ll also discover how Docker continues to address security challenges in modern software development and explore the exciting potential of Wasm.
Don’t miss out on this enlightening conversation! Join Viktor Petersson and Justin Cormack on Nerding Out with Viktor as they delve into the fascinating world of containerization and WebAssembly (Wasm).
Transcript
Transcript
Show/Hide Transcript
Welcome back to another episode of nerding out with Victor.
Today I'm joined by Justin Cormack.
Hey, Justin.
Hi.
Nice to nerd out with you.
Amazing.
So I know Justin from, I guess, the old school London DevOps scene.
Our paths ran across from many years ago, I guess.
And we've been bumping into each other at various conferences.
I think last conference was state of open con, I believe.
And that's when I realized that I should have you on the show.
So thank you for hopping on.
Yes.
The London scenes.
Yeah, there's a lot of people who I kind of met through those things, even from way back when, like a lot of days before, even before I joined Docker.
There's people I met back then in the years.
It's been a great place to meet people.
I mean, the London Kubernetes scene in particular.
And obviously that spilled into the.
Well that started with the docker scene.
The Kubernetes scene was way ahead, London pushed well, by its weight, or punched way above its weight, I would say, from that perspective.
So there was a lot of strong DevOps people in London that really kicked things off.
So it was a really good scene to be in those days, I think.
Yeah.
And.
All right, so let's talk a bit more about Docker, because you are the CTO of Docker.
So let's talk about Docker.
I have some other topics, but we're largely themed around Docker.
So with that, let's start with a very simple question.
What was your first docker experience?
So my first docker experience was.
I mean, it was.
It was.
I can't remember exactly what the first experience was.
I remember roughly when it was.
It was.
It was.
It must have been about ten years ago.
I've been a doctor for nine years.
So it was.
I think it was around a year before that, maybe a little bit somewhere in that area, you know, as you say, there was a kind of.
There was that strong London scene where, you know, things were.
Things were happening.
My.
I was.
I had a.
My first kind of experience in the kind of DevOps space really was puppethe I was doing.
I did a bunch of what we called sysadmin work back then, before it was DevOps.
And I came across this automation stuff, and I thought, hold on, this puppet thing that's actually different from the other stuff and this terrible manual configuration work that were doing before that.
So that kind of got me interested overall in what was changing.
And I was always interested in kind of this sort of system manageability area.
I actually ran the London Coreos meetup for a while.
Oh yeah, thats a blast of the past.
I think the next thing after puppet I got cited by was actually the kind of the really early coreos when it was ETCD.
And that was a thing to manage your distributed etsy which was like before it was, you know before it was actually really about containerization.
And somewhere in that period I ran into Docker.
I mean I think the first kind of formal thing I don't.
We worked on a joint ebook on using Docker with a bunch of us back in somewhere in that period I wrote a Docker security chapter for that book.
So.
But yeah, somewhere around.
So container security was something that, because I was always something I was interested in really early on and yeah so it was around that period.
But I don't actually remember the, I actually remember the first Docker run experience.
It must have, which is kind of disappointing.
I should be happy to be able to say oh yeah, the first time I did Docker run I thought this is the future and I'm going to end up working at Docker.
But it didn't actually work out like that at all.
Yeah, I mean I remember my first Docker experience and it was such a mind boggling experience from a, I guess principal perspective in the sense of like you came from vms and now you came with this concept.
I mean I've been using like freebiece to JRS and whatnot in the past, which is kind of similar, but it was just this mind shift of like is a single process and that took a bit of time to get your head around, I guess.
And you still see a lot of people trying to get their head around that, even to this day.
Right.
So let's talk a bit about, well obviously you have a lot of exposure to the community, I presume, with Docker.
So what do you see as the most common like misconceptions about Docker?
Well, I don't know.
I think that there's a number of things.
I mean it's not so much misconceptions.
I think it's a matter of people having different ideas about what the, you know, what the important changes that happened in the world are because of Docker or with Docker by using Docker and what the way to frame it is for so many years at the beginning, you know, beginning and it's a little bit less so now, but it still comes up, you know there's the VMS versus containers.
Like, that's a thing.
And that, like.
And the idea that this is an important distinction, I think, is really misleading.
I think, you know, obviously, like, people frame things in terms of what came, what they were doing before, but it wasn't ever really a VMS versus containers in the sense it.
And sometimes you got the impression that it might go down that route.
But I think it became a sort of, this is the way we talk about Docker thing, not a really useful thing, because I think that a lot of the VMS versus containers discussion was actually about efficiency of isolation.
Like, were containers sufficiently isolated?
And as a security boundary or not?
But there were two things about that.
One is the answer for, you know, for multi tenancy purposes was no.
Yeah.
And that was fairly clear for quite a long time that the answer for multi tenancy was no.
For single tenancy.
Well, yes.
Like, actually, single tendency.
Isolation wasn't that important.
It's better than nothing.
Yeah, it's better than nothing.
Absolutely better than nothing.
Better than running processes on the same VM without any isolation.
But the security boundary wasn't what made Docker successful.
And the framing of containers versus VMS really was trying to frame it in terms of that.
And it was a very confused conversation over the.
The eight years or so.
It was that often the main conversation about containers was containers versus vms.
Like, it wasn't very helpful to anyone trying to, like trying to adopt, because it didn't tell you what the benefits of using containers were, which wouldn't, and what the reasons that containers around were.
They weren't there to repair.
Almost everyone who uses containers in a vm.
Like.
Right.
Like many things in the.
In, you know, in our ecosystem, they were largely additive, not replacing.
And so there wasn't really, for most people, averse at all.
They were going to use both.
And it.
But the implicit story was about security isolation.
And was the security isolation good enough?
But that, like, literally for most people, that really wasn't the issue.
It wasn't why they were using containers.
It wasn't important.
Like, I think the.
I mean, there were, you know, there were conversations about security and security.
I'm not saying security is not important, but it wasn't the principal part of the threat model.
And it wasn't, you know, the adoption of containers was entirely tied to, you know, organizational changes, the rise of microservices, the explosion in the number of developers, repeatability, you know, understandability of your environment, you know, configuration as code, and all those things that were happening at the same time.
And that was the important thing.
And none of that was captured in the containers versus vms.
I think there's a sort of.
I think there was a sort of alternate universe in which, you know, vms could have tried to be more competitive with containers and tried to adopt some of those processes.
And you saw, I mean, and I kind of.
I was.
I was actually kind of interested in that.
You know, over the years, we saw the little bits of that.
You know, we had.
God, well, I've even forgotten where it's got the hashicorp tool for Packer.
It was probably the earliest thing for creating VM environments.
Repeatably.
There were other tools since then, but that was very early on, like, okay, we want to do repeatability with vms.
How do we do it?
I worked on Linux kit, which was shipped into Docker desktop, but that was also a way of repeatedly creating vms for running containers.
But.
And then there was, you know, far cracker became like suddenly, oh, okay, lightweight.
You can run really lightweight vms.
I mean, again, there'd be more work.
Before that, there was the whole CAta containers piece, like vms and kubernetes and things like that.
So there was actually this whole kind of timeline where a whole bunch of people were trying to make vms more like containers.
And arguably, you know, when you look at the services and the cloud providers run fargate and things, they are vms as containers, but they have a container API, not a VM API.
So they're actually, they just went around the route of, well, containers are what people want, but they want the VM isolation.
We'll make this hybrid, but it looks like from anyone's point of view, it is a container.
And then, like, it has all the important properties of container.
It's compatible with the container ecosystem, not the VM ecosystem.
Yeah, I mean, to me, I think the reason why Docker won in a way over that was around more packaging for me was more like, you can pull a pre configured image.
It became more like an application packaging more than.
Yeah, no, absolutely.
And I think that's the thing.
That's what I mean, like that there is maybe a.
There's maybe a path where vms went down that packaging route.
Right.
Nami kind of tried to do that.
Right.
And they had, I think if, you know, I think, as I said, like, there's a few of these routes towards that, but they didn't really.
It never took off.
And containers did, you know, very much to do a kind of reduction in complexity that made those things easier to understand because Vmsheen, you know, were just.
Big, so much more surface area.
Right.
Yeah.
And had, yeah.
If you could take out a lot of the stuff just, and just simplify it down to the application piece, not the whole system piece.
Now, I mean, again, there's kind of, we've had discussions about minimal containers and.
Yeah.
But like, fundamentally, yeah, the idea was just, yeah, take the applications because that's simpler and that's also separation of concerns, whereas the VM, if you ship a whole VM image, it's not got separation of concern as well.
Absolutely.
Absolutely.
Interesting.
So let's take a step back.
So back in 2019, some big changes had happened at Docker.
Right.
So the company was spun out into two.
One pot was sold and that was capped.
Do you want to shed some light?
Because I think that's a bit of confusing story for people at large, what actually happened.
Do you want to shed some light on what actually around that?
Yeah.
So, I mean, it's a complicated story in a way.
Like, it goes back to, you know, goes back to a number of things, I think, first of all, you know, Docker Washington, explosively successful very quickly, and that in a way that very few open source projects or products ever are.
And that was difficult in many ways.
It was such a success so fast, and it created this giant ecosystem around the cloud native ecosystem around it.
And there was a huge amount of interest and money and investment in that area.
And I think that it was so unprecedented that having that huge ecosystem that it was very unclear for Docker, the company what the business that Docker should be in should be.
And because everything was still really early and there was such a lot going on.
The whole cloud native space was created from scratch pretty much over those years.
And so the business that Docker chose was, you know, I think I can see the reasons at the time, but it was like, it was kind of the, it seemed like the right thing at the time.
So it was basically our product was Docker Enterprise, which was, you know, a production side largely on prem, you know, run, you know, runtime production side product for Ops people.
Docker Swarm, right.
It was originally based on Docker Swarm.
We migrated it to support Kubernetes and swarm, you know, so it was actually, it supported both.
I think there was a, that was another something we can talk about, like, internally later.
I think that.
But, yeah, because it was a production orchestrator solution, but that market at the time was still quite small back in 2018 2019, because a lot of people were still trialing these things and not really running applications at scale back then.
So a lot of customers, we talked to customers and they'd have a few applications that they would try on, even, as you say, or swarm, whatever, they own containers.
They agreed it was a future, but they had a lot of work to do to get there.
So the market size, relatively small, it was extremely competitive.
Oh, yeah.
Back in those days, you had a lot of people playing that game.
I mean, you know, red hat.
Red hat was obviously a big player on the.
On Prem, but they.
I mean, they bought Coreos just really.
Just to shut them down and stop them competing.
Yeah, that's really disappointing, right.
Because I was.
I was really bullish at Core Os.
I really liked Coreos back in those days, but then they just went to die at red hat.
But, yeah, that's a different story.
But there was a.
There was.
I mean, but that's the thing.
Like, the market was too small to support the number of people there.
But I think there was another.
There was another thing that I think, you know, and I didn't really, you know, so that was the kind of obvious competitive thing at the time.
There was too many people in a too small a market.
But the other thing I didn't really appreciate until a little later, I think, was that the early adopters who had made Docker a success and JSON Docker were not the people were selling to.
They were not running on Prem.
They were the people who'd gone into the cloud early as well, because containers in the cloud were part of the same thing, the whole cloud native thing.
All our early adopters were early cloud adopters.
And so the people were selling to who wanted on prem often actually weren't even using Docker at the time.
We were trying to sell it.
And I remember having a meeting with one bank, and there was a lot of these meetings back then, and they'd taken a year to sign an evaluation agreement.
They were going to try it out.
This was a very slow sales cycle.
A lot of the time they were also coming and saying, we've heard about this Docker thing, it's amazing.
And this was great.
We were a small startup that could sell to have these sales conversations with big companies, but they didn't use Docker, but they believed it was a magic thing that they could sprinkle on their terrible systems to make things better, but they weren't using it, whereas there were all these people who were using Docker.
And were being very successful with Docker, but weren't.
We weren't actually.
We didn't have a product to sell to them.
Right.
But, yeah, the thing with, the thing does this was like, Docker was used by everybody, but nobody paying Docker.
Right.
That was like, very successful in that sense, but not.
Yeah, like you're saying, like, not the people actually pay for it.
Right.
But I think so, you know, when.
So we sold the.
So we sold that Docker enterprise business to Marantis because, and I think, you know, Marantis was originally big on OpenStack.
They had a big Openstack customer base, but they could see the writing on the wall with kubernetes.
They'd been trying to build their own kubernetes solution, I think, gradually, but acquiring ours accelerated them.
I think they, you know, they did quite well out of their business in the air.
It took them a while, but that's still.
They grew the business.
It took time and patience, but they had the customer base, particularly telcos was their core market, where they knew the customers well.
They already were in with them on OpenStack, and they could introduce kubernetes on top of OpenStack to them.
And so it kind of made sense from that point of view.
And we, you know, we kind of, were kind of left with this, with the other bits of the business in a way that, but with a strong desire to focus on those original customers who had made Docker successful.
So Docker Hub stayed inside of this new entity, right?
Yeah, so we had basically had Docker hub, Docker desktop, the open source Docker engine composer.
And that was kind of, you know, that's basically what we kind of started with.
And so, but we, yeah, so we had a clear desire to go back to the original customers who'd made Docker successful and who were using Docker in the cloud and also to go back, you know, to work with developers rather than just the ops side of things.
Because, again, the ops side of things was by, you know, by then being dominated by the cloud providers.
We didn't want to do the on prem thing because we recognized that the cloud was, you know, very tied to the success of containerization and those things.
So we, so that's why we refocused on the developer market and on the, on people using, primarily using the cloud.
Not all the customers using the cloud, but most of them.
When you talk to them, a lot of people have a very strong tie between containerization and the cloud.
These journeys go together for them sometimes they're incentivizing people.
You have to move to the cloud, but you also have to use Docker containerized to do that.
And because we don't want to just lift and shift to the cloud, because that's not going to give us the value.
We want to do the whole cloud native route.
And so we very much didn't want to be in this kind of world where we're tied to this kind of on prem only and just separated from the cloud.
And I think that, you know, when I look like, again, when it's coming, kind of coming back to the conversation about containers versus vms, again, like, that wasn't the question.
The things that all grew up together were the cloud, the organizational changes, the kind of, you know, the stuff that the team topologies work and the kind of like, how should we work together?
How should we, you know, how should we build applications now we have hundreds of developers, nothing.
Three.
How do we have understandable things that people can work with that are repeatable and they can understand, you know, they can under, you know, we reduce the kind of complexity burden that people have to understand the whole kind of piece around.
I mean, I kind of, looking back, I kind of think that, you know, when you, when we, when you introduce something radical, like darker, you've got to, you have a sort of change budget where you can change some things about how people work, but not too much containers, unlike, say, serverless, you didn't have to rewrite your applications.
You could just take your applications exactly as they were.
But we insisted on two things roughly, neither of which were really forced by the technology they would, but they kind of, we managed to bring people along, I think one of which, as you mentioned before, is one thing perk and one process per container.
Yeah.
Again, people, some people know that you can run a process supervisor and run multiple things in a container, but when.
You introduce Ssh into a container, that's where, you know, you can't miss the boat.
Yeah, yeah, totally.
I mean, but, you know, it's possible, but I.
That separation really helps modulize things, and it also fits the microservices thing.
And the other one was, you have to redeploy a container, you can't update it in place.
That was really the radical one because that suddenly allowed you to understand that the thing that you built was exactly the same thing in production.
Being able to track that content hash all the way through the life cycle and be able to know, like, you know, Thursday last week, it was this version that was in production, and it had this issue, and we rolled out this version to fix that issue, like, and being able to see that rather than the kind of the old, you know, before Dogra, I still, you know, I remember the FTP into production.
Right.
Change your, change your PHP code.
Yeah.
But this goes back to the whole like, pet versus cattle debate, right, with the febrile vms that kind of Amazon kind of pioneered, right, with easy to.
Netflix pioneered.
All right, fair enough.
Okay.
Well, actually, although the pets versus cattle was actually a CERN presentation.
Was it?
Yes, it's actually.
Yeah, it was actually CERN.
So CERN's a.
CERN has actually been on the housing edge of a lot of this stuff too, because they're a really interesting organization.
We went to CERN for a CNCF meeting earlier this year, and it was some time and it was there.
They're really rolling out kubernetes at scale now.
And it's a technologically, it's a fascinating organization and they've really met a lot of issues that people have in advance.
And I have been quite pioneering in all of this work.
But they're also so weirdly different from other organizations in the kind of things they do that they also, the kind of things they do also seem very different.
So it's a really fascinating place to visit and understand what I can imagine.
But, yeah, like that.
The whole concept of ephemeral vms, I remember back in mid two thousands, early 2010, whatever, when this console, ephemeral vms became normalized, I guess it was still, if you came from like the sysadmin world, it was still a mind boggling concept that these are throwaway vms and the whole IIC, it really took a while to get behind it mentally, I think.
Yeah, because I worked for cloud vendor at the time and it was just hard to just get your head around.
It was, it was fascinating because Amazon, especially in the early days, really pushed the idea, like the cloud was cloud vms were different because they are potentially ephemeral.
They might go away outside your control.
But it was interesting when Google Cloud launched, they spent a lot of time actually making that never happen.
And they had vm migration behind the scenes so that your vmsheen weren't like that.
And then Azure did a lot more lift and shift than the other clouds as in the early days.
And I mean, I remember they would come to our customers and say, would you like to sell your data center that you have all your machines in to us to turn into an azure data center.
We'll produce exactly the same thing for you.
Running an azure, you'll never notice the difference, but it'll become an Azure data center.
That was how they, that was how they grew at the early point, which is really fascinating to the customers like got these offers and it's like, it makes sense, right.
Because Windows never ephemeral in that sense, right?
Yeah, but I think that, but Amazon kind of just stopped pushing that message later because the more traditional clients came along and it didn't resonate so well with them that things could be thrown away.
But I think those of us like me and you who grew up in the era of pets versus cattle and female vms still treat them like that and think that's a good thing and also scale them and kubernetes also forces you to treat them like that and like, you know, I think that we internalized that stuff was kind of, yeah, it was a big shift but we did internalize it.
Yeah.
In that period, containers obviously were even more so.
I was about to say that's, and it kind of created natural segue into containers which is just that in the next instance, I guess, or lower down.
Right.
So yeah.
Like I, I can't see a way going back to the whole nurturing and patching servers.
That sounds like a dreadful life.
Well I'm not getting it, but a lot of it was just, it was forced by the scale of the things were trying to do, you know.
Yeah.
We just got more applications and that's never going away and you just have to automate when you get bigger.
Oh yeah.
Imagine running a service where you can't just say, oh, give me five replica sets of this and then where does it run?
You don't care because it is by definition.
But it's also philosophical difference between, I guess, how the Linux worldviews an application which the way how the Windows world I guess views an application.
Right.
So I think that is also why our cohort easily, I don't know if it's true.
I don't, I wouldn't, well I think, I don't know.
I think that Linux wasn't always like that.
I think we took a bunch, I think we took a bunch of ideas and experience.
I think that, you know, I think that, you know, there was a, there was that period around just trying to think exactly when it, there was a period, I think it started in the late nineties where the cutting edge of hard applications and things at scale, moved to Linux.
I remember I worked with for a while a long time ago with people at MySpace.
MySpace was a windows shop originally.
Were they?
I didn't know that.
Yeah, they were kind of weird.
They were based in LA, they were windows shop.
But I met the Linux team at MySpace who were subversively taking over a bunch of the staff and kind of rolling out more of the Linux way of thinking and that kind of disposability and like, and the performance engineering stuff was hitting Linux more and the distributed system stuff and the, you know, the kind of things like, I mean there were a bunch of language communities.
Erlang was very early in their kind of disposability and process ephemerality, but in the language, not in the runtime, because.
They had their VM, right, they're allowing VM to run everything so you can't just.
But it had the whole kind of, you know, it was built for 100% uptime telco platforms where you had to.
And it was crash.
All the things about crash, you know, expect your processes to crash and another process is there to restart it.
All that stuff was, you know, it's been there for a long time.
And I think it actually like, I think Erlang did.
I mean I knew a bunch of people who were, became container people who had been Erlang people and programmed Erlang.
And I think some of those, you know, some of those ideas definitely filtered in from there even to people who didn't, who weren't really far fetched.
It's not that far fetched if you view the container as that boundary instead of the relying VM.
I guess there is some overlap in terms of philosophy, I guess, you know, strange.
Yeah, I think, yeah, I think that, yeah, it's different, but yeah, definitely some philosophical overlap about, you know, what you, what's, what has to be reliable, what has, what's ephemeral, what's disposable or, you know, how you manage.
I mean I think that, you know, being careful about how you manage data to make sure that your data has persisted when the processes crash and all those types of things and being able to just restart things and expecting to get better uptown out of unreliable system, fundamentally unreliable system.
You don't expect the system to be reliable bit versus the, you know, probably the, whatever it was the seventies, eighties kind of tandem hp, tandem build, reliable hardware that never fails kind of solutions that were.
Yeah, yeah.
These boxes are still running though.
The old mainframes still running reliably to this day.
Right?
Yeah, totally.
Totally.
Yeah.
Cool.
Before we switch into more nerdery around Docker, the one thing I feel like I need to cover is, I guess the debacle around the price changes around Docker desktop because that was obviously something that blew up massively in the tech community.
Do you want to give the vantage point from Docker?
Because that was obviously a big debate around that.
I mean, I think that, you know, I think the, I mean, I think that overall, I think there was a smaller number of people making noise as often the case, rather than a good reflection of the world.
I mean, I think that, you know, you know, somehow at the end, you know, companies have to charge for something to survive 100%.
I mean, absolutely.
Of course.
Docker desktop, you know, I think like people are, people don't like, you know, it's definitely the case that going from something that you give away for free to something that's charged for.
People do get a bit, do get upset.
Although we did, you know, have a lot of exclusions as to who didn't have to pay for that.
Yeah.
Maybe share some light on like what the parameters were for the price increases.
So for those not who do not remember the debate or the.
Yeah, yeah.
I mean, you know, we, you know, we excluded a lot of people, anyone working in a small company basically with less than 250 employees or 10 million revenue, which was to basically exclude startups.
And a lot of the people who were part of our community, although to be honest, a lot of them do pay anyway because they get value out of the bits that you pay.
But I think that our primary aim was to charge larger, which was really charge the larger companies that were getting a lot of value out of.
Docker desktop had fleets often thousands of people using it.
It was a core part of their development environment.
And you know, to be honest, like we didn't charge, you know, the price was quite cheap.
Right.
And so the kind of noise was mostly amongst people who to a large extent weren't trying to judge anywhere.
And I think that, I think we, I think in general a lot of people understood the kind of, the kind of charging people who had the ability and willingness to pay made sense.
And I think that.
But yeah, overall, you know, as a strategy I would, you know, if people are asking about it, I generally say do try and avoid charging for things that were once free.
I definitely avoid charging things and things that were once open source.
I'm.
Docker desktop was never open source.
It was always a proprietary product.
And I think some people thought it was open source, and we've done that because some people then never really understood that.
But it was never open source.
We never closed sourced anything.
We've never.
I don't think those routes.
I think those routes upset people a lot more than, you know.
I think it was probably more philosophical debate than an actual, like, I'm impacted by this.
Right?
Yeah.
Yeah.
And I think that.
But overall, you know, it was.
I think overall, you know, it's.
It was the right thing to do.
It had a good outcome.
Like, the people, it's allowed us to invest, continue to invest in Docker desktop, and people are happy with it, and we're providing the things that those customers want from it.
All right, so obviously you need to pay.
I understand that.
And that brings me to a natural segue to Docker hub, which, if my understand is correct, which is probably by far biggest expense Docker has.
Maybe, except for payroll.
Hosting, this is a pretty significant cost that needs to be paid for somewhere.
Can you shed some light on the sheer volume and infrastructure cost for your hosting Docker hub, which most people use for free?
Right.
I mean, it's not as high a cost as you kind of make it.
Like, it's not.
It's well down below payroll.
Oh, okay.
Fair enough.
Yeah.
It's not, it's not.
I mean, certainly when we restructured Dhaka, it was a huge, right thing, but, you know, but now we've grown, it's not so significant.
So, but I think that, I mean, it's interesting because it, you know, a lot of the, it's interesting how the cost of actually running a service like that has changed over the years.
So we, it's now, but mostly since Amazon introduced teard storage pricing, the storage costs actually are now kind of quite manageable because a lot of it ends up in cold storage because it doesn't actually get accessed very much.
But we are kind of, the cloud providers still have these egress charges that are designed to keep traffic inside their own cloud.
And while the European Union has been kind of pushing them a little bit about, is this uncompetitive and so on?
Because we run a global service that's accessible to everywhere.
You know, egress is.
Egress is difficult when you're doing something like that at scale.
Right.
We, it's a complicated thing, and, but, you know, egress charges are, you know, they're high enough that they kind of affect how people are designing services and things in a significant way.
And they're clearly, I mean, there's arguments about the why of inbound versus outbound.
Like there are some economic reasons, like in terms of traffic is not symmetric, inbound and outbound.
And like they're, you know, the way it's charged it, there's some justification, but it's, you know, there's also a reasonable argument that it is purely anti competitive, competitive thing to stop people running cross data centers.
They are roughly the same.
You have symmetric lines usually in and.
Out of symmetric lines, but doesn't mean that the binding point on your 95th percentile, it could be one direction, not the other.
But I mean, yeah, but yes, your lines are definitely symmetric and the charging model is very different.
Yeah, it's very interesting.
I'm curious to see what the competition authorities in various places decide about this because they clearly, it clearly makes building kind of multi cloud infrastructure extremely expensive for a lot of businesses.
I mean it's, oh, it's a massive lock in.
Absolutely.
Yeah.
So, and just for those not familiar with the kind of orchestration of Docker hub, it's essentially Docker registry, which is essentially a thin layer on top of s three.
But it's s three compatible, I guess.
So the vast majority of all the hub data sits at s three.
Eventually it's tiered using istorage for less frequent.
Is that, is that a fair representation of the architecture?
I mean that's a very simplified version.
Yeah, I mean I think that, yeah, I think that, you know, I think that, yeah, I mean it runs on kubernetes.
It's, there's.
We do.
Yeah, it is, yeah.
It's based on the open source distribution code that's in CF now.
It's, you know, we've forked it at times and gone back and merged our changes at times.
That's fair.
You know, the core of it's there but we still, we have a lot of, there's a lot of other stuff around, around it.
There's the whole UIP.
So there's a whole layer with scout where we're indexing data inside the images and s bombs and things like that as well now and things like that.
So there's, on top of that.
So yeah, it is a big Kubernetes cluster.
It used to, I mean we've always, yeah, I think it, we've migrated into kubernetes probably at least five years ago now.
We used to host a lot of stuff on swarm ourselves, but we moved over to Kubernetes but yes, it's an interesting thing because it's just because of the whole scale of it and that adds a lot of complexity.
I remember years ago being in a meeting, this was back in 2017 or something.
I remember sitting in a meeting because Hub had gone down and noticing that the meeting I was in was number one on hacker news.
Soccer hub is down and it's like, look, our meetings on top of hacker news.
We better fix this.
But reliability is, you know, reliability is for.
Yeah.
And can you share some like sheer volume of pools per day on hub right now?
Do you have any daytime?
I don't have it at hand.
It's, I mean it's, you know, it's, you know, traffic is every, usually at Docker Connor's, I think we release them figures on how many pulls there's been.
You know, and it's, it continues to, you know, continues to grow.
You know, I think that.
I think that, you know, I mean, it's another part of the, you know, it's an important part of why darker was successful Washington, you know, and, you know, as we're, you know, talking about the things that were different about doggo from what came before and since, it's like having that catalog of publicly maintained high content was a key part of what, you know, the fact that you can have that magical docker experience of like, I can just docker run Ubuntu and I get Ubuntu.
Yeah, and this is kind of hundred kind of, you know, magical in.
And it's part of that experience, but it's also part of the, you know, the whole productivity of all the components I need to build applications there and available and I can trust and rely on them.
And Docker Hub, so really keep, it's a key part of that.
And it's very different from, you know, a lot of the other registries, which are largely private registries.
Tucker hub is still largely a private public registry.
There are lots of customers using as private registry, but the public part is still way bigger than that.
I mean, the cloud, you look at ECR there is ECR public, but it's very small private.
Yeah, I mean, most cloud vendors do offer some level of caching off the public hub images as well.
Right?
So, yeah, I mean, that's been part of, you know, working with them to, you know, just make sure that, you know, that they are caching things effectively just because it's like, just, you know, that's, they're the source of a lot of traffic.
Right.
Yeah, that makes it work out how.
To make it more efficient on both sides.
Yeah, that makes a lot of sense.
So you mentioned Docker Scout.
I want to zoom into that because I don't think a lot of people are familiar with Docker Scout, but it does a lot of things.
You mentioned s bombs.
I had a lot of content on the podcast about s bombs already.
And Docker Scout can generate sBoms, both Cyclone DX and SPDX, from docker image using.
I think it is a wraparound anchor.
Sif.
I think.
No, it's.
Well, it uses a bunch of stuff and a bunch of its own code.
So it's kind of,
We do use a.
We use a mix of tools internally.
It's not just based on what thing found.
We found things that.
Okay, yeah, it's not.
It's,
But the.
But in a way that.
I mean, the.
The creating s bombs bit is not like the core.
The.
The way we see the core value of it.
The way.
Right.
Like we.
The.
You know, I think that one of the things that.
One of the kind of design things about how people were managing, you know, understanding vulnerabilities was, you know, and we.
You know, we used to do this with the services we had on Docker, things like that.
It was like you would.
You would take an image, scan it, look at vulnerabilities, and give the user reports on that.
But we really wanted to separate out the two stages of creating an SBA on what's in the image versus the bit about, is there a vulnerability?
Because if you separate those, that lets you actually scout is basically a real time database with live feeds from different sources.
As soon as a vulnerability happens, you can get an alert about it without having to run the image through a pipeline again, to find that.
Because we link the vulnerability to the SBom item that we've indexed in the database, and we say, okay, these images now, as of a minute ago, now have a vulnerability, and then we can look and see if they in production.
Are they things that you're actively using?
And then we can help you remediate that.
So it's like separating out what's in it and is there a problem about what's in it?
Pieces.
And running them separately allows this real kind of much more responsive tooling.
So, to talk to me, you do some interesting stuff with, I guess, at the station and embedding s bombs in kind of metadata.
And I believe that's stored in the OCI as part of the image.
Right.
Is that correct assumption or.
I haven't actually done a deep dive on this.
Yeah, I mean, I think that, I mean we actually, scale supports a lot of different formats as well, but yeah, we've been working on doing that because, you know, because we actually, you know, we maintain build kit, which is what the majority of people are using for doing builds.
That means that we can, if we can add these s bombs at build time and then carry them, you know, all the way through, then we can, you know, then we can use that to index.
So rather than, you know, if you add the s bomb at build time with buildkit, then we don't need to scan the image at all.
We can just read the s bomb.
And we feel that having it attached to the image is helpful because it, I mean, I think that there's been discussion, you know, there's different models because, you know, six door effectively lets you attach it externally in the transparency log.
But that's kind of, there's a lot of use cases where it's actually more effective, easier for people to deal with if it's on the index.
Like if you're in a air gapped environment where you don't want to reach out to an external thing.
And to be honest, and like the transparency log is not a high performance database either.
And so, but we don't actually, like, we actually index that in the scout database for fast access again.
So that, because again we actually want to be able to alert you when there's a vulnerability change on something.
So we still want to maintain a copy of that in the database, but it's a copy and the source of truth is what you add to the image, what you put in the image.
So you can manage the s bom if you want.
You can use whatever tooling you want to construct the s bom that you attach to the image.
So you can be sure that, because I think part of the reason for doing s bombs at build time rather than by scanning is actually quite hard to retroactively work out what was used to build something after the tooling there has gone better in many cases, but it's still like, you know that at build time you've got the complete information about what went into it because it's all there to build it with.
And you can often you have even more information like what were the build tools, what were the internal build stages and the tools that even if they don't appear in the output they could potentially affect the output.
Like if you have a compiler bug, you need to know which version the compiler is even though the compiler is not necessarily linked into the final image.
Although often like tools I go will put metadata about what the compiler version actually was in there.
Not every language tooling will do that.
And traditional C tooling doesn't put any metadata about the compiler version in there.
Sboms and clay is a whole different kind of word in general.
Yeah.
I mean, but I think that like, you know, so having, creating sboms at build time gives you that complete control where you can just take the image, you can take the data from the actual build tooling and the, and feed it straight into the SBom directly rather than trying to reverse engineer out of what came out of the container image.
So and then if you attach it then, you know, the information is there.
So then you can feed it through any tooling.
And there's a broader kind of, you know, there's a broader, you know, this is broader attestation projects around, particularly around in toto which is a CNCF project which is really around, you know, it's a project around attaching build process attestations in the broadest sense, not just s bombs.
How was this built?
Who built it, where did they build it?
And so there's a lot of you can add that metadata like this was built in Docker's AWS account.
So, you know, we can take the assession from that, from the metadata that Amazon provides.
You can get it, you can get a sign, you can get assigned signature from Amazon saying that this payload was built, this boot time payload was in AWS at this point and things like that.
So there's a lot more data that you can really get.
A reproducible thing that gives you more information that gives you certainty about, you know, how the build process happened, where it happened, what happened and so on and that type of information, I think.
And again, the GitHub assertion from GitHub actions.
Yeah, I was going to say, yeah.
So all those kinds of things together give you this whole question of things that give you more certainty.
And I think that our view is generally that bundling all that together in the OCI format is useful so that it's all self contained technically from the kind of the statements are about hashes and don't have to be bundled, but the bundling convenient to keep things together.
And that's what the sort of extensibility of the ACI format was kind of designed for.
And if you look at any of the official builds in Hub today, do they all have this included as metadata to all those images or is this just you can use as your toolchain?
We're rolling it out.
It's not, I think, on all of them yet, but it's on a lot of them now.
So it's kind of in, you know, we're kind of going through this process and being our first customer, zero for ourselves.
I mean, I was very insistent when we're doing this that like, if it doesn't work for us, then yeah, it's not good enough for other people.
I mean, our use cases is kind of complicated, but I mean, I think that, you know, if there are, if you can't validate the attestations on Docker official images, and use them, then that's not working for people, I think because I did a talk about this many years ago in Kubecon about how, you know, with the earlier work we did like no one in practice was validating the success signatures we put on Docker official images.
So because it was very hard to, and it's difficult, and you kind of see this with the adoption of HTTPs, it's hard to incrementally go from an ecosystem that doesn't have signing and security checks to one that does in a way that doesn't leave holes where, because lots of stuff is unvalidatable.
And so you kind of tend to be in this position where you tend to fall back to, oh, if it doesn't, you know, you can fall back to oh, we don't have to validate it because it must be part of the unvalidatable set.
And trying to get to a point where you carve out things, sets of things that are known to be validatable and therefore you should validate them.
You know, it's a kind of slow process movingly because it's a to that.
And so I think we're still kind of on that journey to where everything is validatable in the cloud native ecosystem.
But it's just, it is just a hard path going from there.
It's kind of weird when you look at it like there was a whole like Docker images in the earliest release all had signatures on them, but the signatures were kind of useless unless they're still there in the v one format.
But they were kind of, they kind of, they were used to signatures or just hashes.
They were, they were cryptographic signatures on the earliest.
Weirdly, the earliest versions didn't have real content hashes, but they just have signatures.
But the signatures were for keys that were just randomly generated for the user.
You couldn't validate keys.
So for most users.
So it was kind of, they weren't actually much use right then, but then they weren't actually content.
Cryptographic hashes of content for a while.
That was the first big, that was the migration to the v two format.
That was real content hashes because there was a big potential security vulnerability without the content hashes of.
Right back in, when I joined Docker, this was just being, was in the process of being rolled out as big change in 2015.
And each layer is SHA 256 sign or something like that.
Right?
Yeah.
Hash 39s.
Yeah.
But they were originally, they were actually random things that looked like hashes, but they weren't in the earliest versions.
You could basically poison the cache.
Bye.
Gracing something with the same hash as some bit of well known content.
If you could get someone to pull it before the well known content, it would then replace your layer, which was like.
It was, it was, yeah, it was a, you know, anyway, so that was where, that was what v two was about, I think, you know, we noticed that internally and that was the whole v one, v two.
Right.
Docker format, which is.
Yeah, as I said, 2015 lost in the midst of time now.
But, you know, it's kind of.
But again, I think that the content hash, again, is like one of these key things enables this ecosystem, like, enables these pieces around s bombs and signing and things because you've suddenly got that.
You've got this thing that has gotten an immutable content hash.
You don't update it in production.
You know, what's.
And you have a root of trust.
And you have a route.
Yeah, exactly.
And so, you know, that it's those kind of pieces that are really important to building out.
Building out this ecosystem.
And will there be tooling for like, exporting to like a Cyclone Dx s bomb?
And I guess, is it in one of these formats or is it agnostic to the two formats?
Because, I mean, that battleground is still being played out.
Right.
Between sports.
Yeah, I mean, I think that stuff is all kind of.
Yeah, so, you know, I think that everyone's making their stuff interoperable at the moment, and I think that.
And converting these things, I mean, I think that, you know, that's actually not the difficult bit.
Like, the difficult bits are actually, like, around standardization of all sorts of other smaller pieces about like, what you.
How do we name packages and things like that, like, turn out to be.
Yeah, I'm part of a working group with CISA for, for a bunch of this stuff.
And that's obviously something that comes up a lot.
It's just.
How do you actually name things?
Like is it.
Is it Microsoft or is it Microsoft Inc.
Or whatever?
How you capitalize things and like.
And those things do have real consequences, right?
At scale.
Yeah, yeah, totally.
And I think those are the things that if we can standardize like write down all that detail, then a lot of the other stuff about the format stuff will kind of become much easier to.
Because you just be a conversion process.
Right.
I think.
Yeah, go ahead.
Sorry.
I think there's like, I mean, I think there's a lot of.
A lot of.
It's just about trying to reflect the right amount of domain complexity.
Like what, you know, what is an S bomb?
What do we include?
What does it mean?
Like what do we.
Like, you know, the example I was talking about things that are used in the build but do not appear in the artifact.
Are those part of an S bomb or not?
What, you know, another thing we have with containers is like layers that there's stuff that's overwritten in layers.
And we've had a lot of tooling issues around like some tools just index all the layers and look at what's in all the layers.
And so they'll show you something that is in a hidden piece because it's been overridden by another layer.
And like is that, you know, technically that's not a vulnerability, but I.
It's potentially a vulnerability in a different image.
And a lot of the stuff we did with scout was about, we do have a full layer model because we want to be able to tell you how to remediate issues.
And to remediate an issue, you need to know which layer the things in.
So if you don't have a layered model, you can't realize which bit of it you need to remediate and how the things been built up.
So we have a full layering model and so we can give you breakdown by layout, but we use that to say you should update the base image.
You should update this image in order to fix it.
Because sometimes it's a different team that's responsible for updating the different images because like developers might build the application, but there's a team that builds base images and you need to know which of them to talk to.
And so again, like some of the thing is about reflecting the domain complexity in the model of the data that you're extracting from.
These things and.
Yeah, and things like container.
I mean, layers are an interesting thing with container images because they're kind of an implementation detail, but they kind of leak into a lot of things.
Yeah, absolutely.
Yeah.
Particularly from the voldemorty standpoint, that's obviously a very difficult one because they may or may not be relevant.
Yeah, exactly.
And they're relevant to some questions and not others.
Yeah.
And so they're kind of.
And.
Yeah, and they.
But they, you know, the classic s bomb doesn't understand them.
Yeah, exactly.
I mean, the classic s bomb is still to be, I would say still to be defined because we don't really.
It's still really much a moving target.
I mean, I think that.
Yeah, I think that there's been a lot of.
Yeah, there's a lot of improvement and alignment in the ecosystem around.
Yeah.
The problem space and what the solutions look like.
But it's.
Yeah, it's, you know, a lot of it is about like the domain complexity versus.
Yeah, and the tooling, from my experience is very theoretical and like, when you actually started using it for real use cases, that's when it start to break down and that's.
Yeah, but it's.
Yeah, there are a lot of great theoretical documents about how to use this, but then when you actually try to use it, you realize that, oh, the tooling actually falls short rather than.
Yeah, no, totally.
I mean, again, that's something that we've, you know, we spend a lot of time on with scout.
Our aim is to make it work for developers.
That's our, you know, because developers are the people who are usually being called on to actually fix the issues.
Yeah, and.
But it's, you know, I think a lot of the time.
Yeah.
As you, as you say, the kind of theory doesn't work particularly with the scale.
Organizations are big and they have a lot of stuff going on and a lot of the tools don't really scale to a large organization, a large, complex organization.
They're great for greenfields, but when you actually start doing a legacy, that's when it gets really complicated.
Yeah, I mean, yeah, it's like, you know, I think this is the case with a lot of that sort of tool that the steady state.
In a steady state situation in theory, when you've got to a good state, is very different from the.
I've turned this tool on.
It tells me I've got a million problems.
What should I do?
Right, right, absolutely.
Well, there's a billion things to more talk about that, but interest of time.
The last thing I wanted to talk a bit about, which is something I think you guys are doing some interesting moves on, is wasmouse.
And I know you've been toying with that for a bit.
Tell me about dockers wasm story.
And what does that mean in terms of your market?
Go to market?
Yeah, I think wasm is already an interesting area.
You know, I think that for, you know, for a while there was a whole kind of, like, were talking about VMS versus container.
Like there was a whole kind of wasm versus container, wasm versus docker kind of thing.
Because again, like, superficially it looks like a container.
It's, you know, it's a thing and it's got an isolation layer and it's a sort of package of stuff.
Right.
But again, a lot of that's kind of a bit superficial.
And like, you know, the interesting, I think there's a lot of.
There's a lot of interesting things about wasm and what it could can be used for and what it will be used for and how and why.
I think that there's, you know, I think that, and I think there's actually one of the things that's really nice about the wasm community is that there is such a diversity of use cases and very different things you can do.
And they're, you know, so we've seen a lot of success for things like Wasm as a plugin, you know, plug in extension language for things like envoy, for example.
Lots of, lots of other projects have it.
And I used to do a bunch of work on Lua back in the day.
And as an extension of language, I kind of interested in extension languages for things.
They're really, they're really powerful way of building.
Building stuff that you can extend.
And, you know, I think that's.
There's a whole piece with people like fastly using it for that open rusty.
And all that stuff.
Yeah, well, yeah, open rusty.
Well, yeah, open rusty was.
Yeah, and the day with Lua, but anyway, but it's like that kind of area is one area, I think, that WaSM for the web is an amazing enabling technology that helps people build web applications that are just much more powerful.
Again, there's been a very long story from Gmail was one of the first of these complex web applications that was largely built from, largely built in one of the JVM languages, and they can cross compile it to JavaScript to run in the browser.
And there's been this whole long technology tread of that.
And WASM is a much more effective and sane way of doing that.
And then there's this server side wasm, which is the bit that's most often compared to containers.
And that's the bit where to a large extent, we've been working on shipping that shipping tooling to support that.
I mean, it's something that we've been looking at for a really long time.
I think back in, I think it was 2016 or 2017 time were experimenting with it.
And that's when Solomon tweeted around with him.
And it was in his phase when he used to post a lot of stuff on Twitter and hacker news and really kind of get a lot of attention.
And he tweets the thing about like, oh, if Wasm had existed, we never would have had to build Docker or something like that, which has been then since quoted in every single wasm presentation ever since.
And it's kind of like sort of put docker on the map as like, in that kind of space, but it kind of isn't.
I don't think it's a very, it doesn't, it's not a very useful way of looking at it.
Again, like the containers versus vms, like, so what's the, you know, what is the outcome you're trying to achieve for, you know, for the developer by shipping things and wear them?
And generally, you know, I think the areas that are interesting for wasm are, one, is a lot of people are building quite lightweight, serverlessly type stuff with wasm.
I think that, I think some of the sort of wasm is smaller kind of commentary is misleading because people look at people's containers and ask and discover that people have very large containers, and then they build kind of toy applications in rust and wasm and discover they're very small and kind of say, this is much more.
I think a lot of the time, people don't understand why people's applications are so large in containers and people don't talk about it a lot.
But, you know, there are often good reasons why people have multi gigabyte containers, despite the fact it seems a weird thing to do.
Often very good.
Bad reasons, too.
But yes, there are bad reasons too, but those reasons are not necessarily, those reasons are not actually necessarily.
They're not really about the runtime, they're about the application.
And it comes back to, like, the whole conversation we have about, like, how do you build your applications?
Can you use the same application?
Do you have to build applications again?
Do you have to build them in a particular way?
But I think that like WAsm does have, you know, that I think there are some interesting things about the Wasm runtimes that are quite lightweight and look more like a, more like what people, what we thought the JVM might be going to look like back in the day, but it never quite ended up like that for reasons.
The security model is much better understood now.
The interface model, which is where the, you know, the security issues and the JVM were back in the day.
Yeah.
And this, I mean I think that there's a lot of interesting things.
I mean I think the whole, I mean it's hard, but the whole we can write code in multiple languages and interface them together and have libraries that are security isolated from each other is a really interesting area.
It's kind of, it's at the moment it's very niche.
I mean there was an attempt to do security isolated libraries in JavaScript, for example, and like, because, and there was a bunch of really good work on that, which actually shit.
But like, because like libraries have high privilege levels right now.
Like a library that's supposed to do some calculations can also access the network and exfiltrate your data.
And you can't really stop this because we don't, because everything ends up in a global namespace and doesn't have least privilege.
And WASM is a potential way of actually addressing that kind of issue potentially.
So there's a lot of interesting stuff we've really been focusing on at the moment.
Enablement of kind of build and run of WASM applications in the kind of roughly similar sort of space to where we are because that's where we've seen most interest.
But it's such a diverse ecosystem that I think there's a lot of interesting paths about where adoption will happen.
And it's not just going to be one thing, you know, it's not going to be one thing.
And you know, I think there's like, you know, these different use cases of whereas, and we talked about it at the moment, you know, kind of, you know, extension stuff is very different from in browser stuff.
And it was actually interesting, like when I worked on, when I did stuff with lure, I run the London lure meetup back in the day.
And it was really interesting because again, like there were you, it was hard to have a uniform conversation around a language that was used for so many different things.
And I think that wasm has the same thing, that there's a lot of different use cases and they're, and they're kind of pushing in slightly different directions, but they are, they're using a set of common tooling and they, you know, they're building this stuff out together for different use cases.
Are you more bullish on the server side of wasm to kind of eat up node js style workloads or are you mobile on the browser side of things?
I actually don't like.
I think it's in a way, I think the, I don't know, I think that there's a kind of thing you could get.
I mean node, Washington node was kind of exciting because it was like one language for the front end and back end, and wasm potentially I think is interesting, we could do that from the other direction because it was like node was, well the one language has to be JavaScript because that's what the browser has.
So the backend language now has to be JavaScript.
And that was great for the JavaScript people and I got people who then adopted for everybody else, kind of bad for everyone else.
And like, I mean I try, I tried to become a JavaScript person as a back end person at that time and I like, I kind of, I tried quite hard for quite a while and I decided it wasn't for me.
But I think that, I think there's an interesting thing of like how much can we go the other way of like take someone who's a, a go programmer and help them ship go via Wasm into the browser so they can build, you know, more of the application.
I think you can't exclusively build applications in WASM alert in the browser, although there have been some people trying to do that like I think.
So I ported like Libreoffice to Wasm and had it up and running like fully.
So I mean, yeah, but there's still a bit of JavaScript.
There was actually a, like a really radical proposal have an index wasm, kind of like why can't the entry point into a website p wasm rather than JavaScript?
But I think there's still issues there.
But you still need some, like you.
Need to invoke it somehow.
Yeah, but also invoking the UI from WASM story is a bit messy in terms of it's a bunch of, but in principle I think, yeah, you could, and I think the rust community are kind of pushing that direction most because they're quite, most of the WASM communities here are rust people and they don't like JavaScript.
Rust people.
I don't think you could find bigger, I mean bigger difference between programmers than rust programmers and JavaScript programmers that's probably as far apart they can get.
Yeah.
So I think that a lot of them are kind of pushing in some place direction.
And I think I know, I mean, rice is a good language for.
I mean, I think it's changed now that garbage collection in WaSm is a real thing now.
But it was the case that it was a good language from the, because WaSm didn't have a GC, but now with WASM GC, I think there's like, there's a.
I think the, because I think I joked a few years ago that you can write wasm in any language you like as long as it's rast.
But it kind of felt like that for a while that there were actually really strong reasons why actually it was the best language to write wasm in for almost all use cases.
I think that's, you know, and it was kind of accidental because it wasn't what Russ was targeting either, which is kind of interesting, but it kind of was like that.
But I think that's, yeah, with Wasm GC and the maturity of the other language ecosystem tool chains to wasm and things, that's actually changed a lot in the last couple of years.
Yeah, it's now much broader, but there are still, I still think that there's, I mean, I think that, you know, if you look at go, for example, like tiny go has been the go to way of getting into wasm for a long time.
Just because the go runtime compiler wasm is big.
They don't, they like their GC implementation, they don't want to use the upstream one.
And it's become a bit, you know, I remember when, with, when cloudflare workers launched their wasm thing, like you couldn't, like Cloudflare had a very small memory allocation for workers at the, when they launched it.
I think it's increased now, but like, it was like you couldn't fit the go run time into a worker because it was right.
Who compiled it to wasm is too big.
And they were like, well, we're gonna have a shared library kind of layer for the go runtime or whatever.
We're gonna work around this kind of thing.
And it was like a lot of work.
So, you know, it really worked.
There were these kind of barriers of like, hard to make a small wasm application in go.
Tiny go filled that niche actually really well.
And tiny go is a kind of fun language anyway, and quite a lot of people are using it, but it was, you know, it's kind of, you know, again, that's kind of the tool chain and kind of language ecosystem is just taking, you know, taking a long time to mature.
But it's really, I mean, as I said, it's really changed over the last year or year, two years, and that's really much more mature now and it's becoming more realistic for people to experiment with.
So what's that?
Is there a docker angle here?
Like, is there going to be like a docker hub as a CDN for browser based wasm?
I think, I don't know about browser based wasm.
You know, for, I mean, we, because I think that the, I think that, I think we're starting with the server side because that's where the audience for Docker Hub are.
The browser based wasn't people kind of come through other tooling routes at the moment but not saying never.
I think that, I mean, I kind of think that one of the interesting things about wasm, although, is just that universality.
It runs in the browser and on the server and that's like, I think that's a really interesting thing that's different from what came before.
I mean, we kind of had that very briefly with Java applets like two decades ago.
Is over.
Yeah, but I mean I think that like that, but there's, but it's actually very, like, it's actually the ecosystems are quite diverged and there are, I mean, although there are, you know, there is a community of people trying to do things that support, or rather support, you know, there's people who are supporting the web APIs on server side or on which, like for example, actually cloudflare workers is a great example of that where their core APIs are the browser APIs.
And so you can mostly write browser code that works there.
But again, even like there's still things that they have that are, you know, because the environment is different from the browser.
The biggest one has always been like inbound socket.
You can't listen to a socket in the browser and that's a core activity you do on a server.
And arguably that one thing has been the thing that's kind of diverged those ecosystems in terms of APIs the most.
And so I think that, and I think that the value of WASM on the web has mostly been being able to use libraries and code that was written for other, you don't want to go and rewrite some complicated rendering library or something that you've already got in c, again in JavaScript or on the web.
If you can just compile it to wasm and just run it, that's great, performs well.
So it's still not a kind of uniform, I just write things that run everywhere kind of world.
Yeah.
But we're starting to see that's the kind of, there's these bits of promise and ambition that WaSm has that are quite exciting.
Yeah, I'm super excited about WASM.
I think that could be a nice solution for a better web really.
Right.
Where we can write when a website currently is absolutely okay.
Apparently to have a website that's like 300 megs to download and then like if that's the ballpark of data we're talking about, then writing it that in WaSM, then obviously 300 megs for a rust binary compiled is a lot more than 300 megs of JavaScript, right.
So.
Well, hopefully it's more optimized.
So I'm super excited about that.
Prospects from particular on that I'm less excited about the server side, more excited about the browser side, us, honest.
But yeah, Justin has been super interesting.
We found a lot of ground.
There are plenty of more things I would like to cover, but I do realize that we are kind of running up on time here, so maybe we'll do a follow up in a future episode.
There have been more content because there are plenty more on my list here I would like to cover.
Cool.
Christ, great to talk.
Thanks so much.
Thank you.
Have a good one.
Cheers, bye.
Found an error or typo? File PR against this file or the transcript.