Episode 5: Luke Hoban, CTO of Pulumi, co-founder of TypeScript
Luke Hoban, Beyang Liu
If you write code on the modern web, it's almost certain that your life has been shaped significantly by Luke Hoban's work. Luke has worked on developer tools his entire career. He started out on Visual Studio, C#, and .NET in the early 2000s, later joined the ECMAScript standards body as a representative of Microsoft, and then became one of the co-founders of the TypeScript programming language. Today, he is the CTO of Pulumi, an infrastructure-as-code company that lets you write your deployment config as code in your favorite language.
Luke shares stories from the early days of TypeScript and talks about how it evolved from a two-man team to one of the most successful programming languages and open-source projects. We discuss important inflection points and design decisions that played a key role in TypeScript's runaway success. We also dive into the symbiotic relationship that TypeScript had with another early project just getting off the ground at the time: VS Code. Luke also shares his learnings from his stint at AWS, how his role at Pulumi combines his two passions for programming languages and cloud infrastructure, and how Pulumi brings the niceties of the IDE experience to an area that sorely needs it—infrastructure configuration management.
Luke Hoban: https://twitter.com/lukehoban
Pulumi: https://www.pulumi.com/, https://twitter.com/PulumiCorp
Joe Duffy, founder and CEO of Pulumi: https://twitter.com/funcOfJoe
Q Basic: https://en.wikipedia.org/wiki/QBasic
Gorilla.BAS and Nibbles.BAS: https://en.wikipedia.org/wiki/Gorillas_(video_game), https://en.wikipedia.org/wiki/Nibbles_(video_game)
Visual Basic: https://en.wikipedia.org/wiki/Visual_Basic
Steve Lucco, Technical Fellow at Microsoft, co-founder of TypeScript: https://www.linkedin.com/in/steve-lucco-b5084958/
Visual Studio: https://en.wikipedia.org/wiki/Microsoft_Visual_Studio
ECMAScript Standards Body: https://www.ecma-international.org/memento/tc39.htm
Erich Gamma, creator of Eclipse and VS Code, member of Gang of Four: https://en.wikipedia.org/wiki/Erich_Gamma
Gang of Four: https://en.wikipedia.org/wiki/Design_Patterns
VS Code: https://code.visualstudio.com/
Anders Hejlsberg: https://en.wikipedia.org/wiki/Anders_Hejlsberg
ECMAScript 4: https://en.wikipedia.org/wiki/ECMAScript#4th_Edition_(abandoned)
Microsoft CodePlex: https://en.wikipedia.org/wiki/CodePlex
Angular 2: https://angular.io/
Shanku Niyogi, colleague of Luke who suggested he write the Go VS Code extension: https://twitter.com/shankuniyogi
Handover of VS Code Go from Microsoft to Go: https://code.visualstudio.com/blogs/2020/06/09/go-extension
VS Code Go extension: https://marketplace.visualstudio.com/items?itemName=golang.Go
HCL (HashiCorp Configuration Language): https://github.com/hashicorp/hcl
If you notice any errors in this transcript, you can propose changes to the source.
Beyang Liu: All right. I'm here with Luke Hoban, cofounder of the TypeScript programming language project and now CTO of Pulumi, an infrastructure-as-code tool that lets you describe your deployment config as, well, code in your favorite programming language. Luke, welcome to the show.
Luke Hoban: Thanks. Thanks for having me.
Beyang: So before we dive into all the cool stuff we're going to talk about, we always like to ask people to tell kind of their backstory as a programmer, starting with how you got into programming in the first place.
Luke Hoban: Yeah. So I think my story probably is similar to a number of folks who started off with programming in the early 90s as a kid. I think for me my parents via their work had sort of an early IBM PC clone in the house, and it had DOS on it. And was trying to find things to do with the computer just as a kid being stuck at home. And I think one of the things I found was Q Basic. And Q Basic had with it a few sample sort of projects that you could start. And some folks who were around in that time will remember Gorilla.BAS and Nibbles.BAS, were these two sample projects, which were actually games you could play within Q Basic.
Luke: And so I think my initial entry was just trying to find a way to play games on this computer that we had in the house. But to get to the game, you had to actually open up the source code file, and so you got introduced this idea that games and things on the computer were built using software, and that you could modify it. It was just a file of text that you could change. And so for me it started as, how can I play games on this computer? And I think it became, oh, I can make changes to this. I can change the color of this thing. I can change the speed of this ball. And so that gave me the idea that software was a thing you could actually change and modify and kind of started my interest in doing various other things with software.
Beyang: And was it kind of a straight path from there to Microsoft, or what were kind of the steps in between?
Luke: Yeah, so well, I think that was back by probably 10 or 11 or something, so certainly took a few steps to get from there into sort of a professional career. I think I was interested in computers and software kind of all throughout my teenage years and things. And I had a chance to do a lot more of that through high school and things. But yeah, no. Then when studied computer science and math and got really into sort of the programming languages space in particular in college, so studying programming languages from an academic kind of perspective.
Luke: I then got the chance to kind of intern at Microsoft, and just sort of by happenstance... I don't think I'd even expressed particular interest in it, but I ended up getting a chance to intern in the C# team. I got a perspective on what it's like to kind of work on programming languages in the real world, not just in the academic world. And really enjoyed it. I got a chance to work with a lot of people whose names I had known over the years as sort of legends. And they were real people, and got a chance to work with them on building up C# and .NET in kind of the early days. And I sort of continued to spend a lot of time in the programming language and developer tool space over the last 15 years or so.
Beyang: That's awesome. So you kind of started your career off in developer tools, and I guess that means you've been working in developer tools for a while now. What are some of the things, as you look at how things are today, what was it like back then when you first were starting out? I guess it was early 2000s?
Luke: Yeah. I think I'd joined Microsoft in 2004, so yeah, around then.
Luke: And I think it was interesting is I think different things were happening in different parts of the industry for sure. But Microsoft, that was the early days of .NET. That was a broad sort of application platform. I think it was really .NET and building on the back of sort of success that Microsoft had had in two areas. One was sort of Visual Basic, sort of democratizing access to build a line of business applications, which I think had really started to unlock kind of a ton of commercial opportunity across the entire... not just the sort of raw software industry. But all kinds of companies were now using software to kind of build rich applications. And Visual Basic had sort of made that accessible to a much wider set of people.
Luke: And then I think the other piece was sort of calm in the sort of Microsoft days had really enabled what became Office and all the rich sort of desktop software that had been built up kind of around the model of all the Office applications. And so .NET was sort of sitting on top of that and trying to sort of get to the next level on that. And I think it had a lot success, and Java was kind of very big around the same time, and it was taking over kind of the enterprise sort of IT market.
Luke: So a lot has changed since then, the sort of rise of the client-side web obviously happened five, seven years after that and completely changed how people thought about delivering software. And then of course the cloud has happened over the last five to 10 years and has, again, really completely transformed how people think about building and delivering software.
Beyang: I really want to dive into kind of your experiences at Microsoft over the years, but first I want to kind of get up to kind of present day, so we kind of get the whole scope of your career up front. So I guess you were at Microsoft for a while. You eventually landed on the TypeScript project. Can you tell us a little bit about that?
Luke: And he was saying, "They're kind of falling off a cliff in terms of the toolchain and the experience they get for building these things. They're building as complicated of software, but without the IDE tooling, without the type checking, without any of the things to fall back on." So they were sort of demanding that there was... "What are you going to do about this? You guys own the experience around this language, this tool system, this toolchain. What can we do to make this something that thousands of developers can productively work in in Office?"
Luke: I think we saw the same thing was happening in other parts of the industry that were kind of at the leading edge of this. Google had a number of projects that they were using internally, and some of them had started to make it outside of Google that were managing the same kind of complexity that they were seeing. [crosstalk 00:08:57]-
Beyang: Yeah. Dart, right?
Luke: Well, Dart was a little bit later actually. But at the time it was actually [crosstalk 00:09:02]-
Beyang: Oh, okay. Got it.
Luke: Something called Closure Compiler, which is still used by Google.
Beyang: Oh, right. Yeah.
Luke: And so that was sort of the genesis of working on TypeScript, and, yeah, we started off as just a small project that Steve and I were working on, and then called a few more folks in, and it ended up growing into a pretty big thing now. It's quite heavily used obviously across the industry.
Beyang: What was kind of your... Did you have... You said you were working on a couple of web-based projects. Was there any project in particular that you were initially targeting as "Hey, we really want these people to be building their applications in TypeScript?"
Luke: Yeah, I think we were initially pretty focused on internal teams. And I think when we started working on the project, we weren't sure it was ever going to be something which would exist outside of Microsoft. The possibility that it would be just a tool for some internal use, and I think Office was obviously the one that was the big motivation. And they were the people that we kind of talked to the most, but as with most of these things, big, huge teams with Office that are already trying to solve their problems in practice today aren't normally the best candidates to be sort of your first user for something new.
Beyang: Yeah, yeah.
Luke: And so we talked to folks in various parts of the Office team quite a bit. But I think they weren't ready to go and dive off of a cliff and do some totally new thing in the early days. There was actually a team though that was closer to us. It was actually in the same organization as us. That was someone named Erich Gamma had joined Microsoft, one of the authors of the Gang of Four.
Beyang: Yeah, yeah. Eclipse and VS Code. Right?
Luke: Yep. And so Erich had joined Microsoft right around the same time and was very interested in the same space. And he was very interested in building a kind of a web-based IDE. When he started doing that, he actually knew us. He'd talked to us a lot, and he was like, "I should use this early TypeScript thing." And so from the very, very early days, he and his team were customer effectively of this very early-stage version of TypeScript. And I'd say right he started adopting it, it barely worked honestly. So a lot of credit to him for sticking with it and not abandoning it in the early days.
Luke: But it did mean... I think that was really critical to sort of get that feedback loop from a real project that was kind of betting on TypeScript from very, very early on. And, of course, Erich and his team have gone on to build... the software they were building at that time was really the core pieces of what ultimately became VS Code, which didn't ship publicly for another three to four years. But it was used in a bunch of things inside Microsoft in between then.
Beyang: I mean, that almost sounds like a match made in heaven, because the language support for TypeScript and VS Code is fantastic, and now it's written in TypeScript. And that sort of feedback loop, it sounds like... The conduction of those two tools, I've certainly got a lot of value from both of them. And it's great that you were able to kind of feed off each other from the very beginning.
Luke: Yeah. And I think it's particularly kind of fun to have an IDE being built in the language it's being written in, in a sense. And I think the folks, Erich and the folks on his team, I think really enjoyed that they were able to... It's sort of that dogfooding. Right? They were able to build the tools that helped them be productive, and so they very early on had this experience of kind of that super quick feedback loop. And it's something that, for folks who do build developer tools... I think it's one of the things a lot of people love about building developer tools is the idea you can be your own customer in a sense, and you can really get that super quick feedback loop. You don't have to go out and interview your customers. You can feel it everyday.
Luke: And so I think they initially actually focused pretty heavily on that TypeScript experience, the Markdown experience, because that was the other thing they were doing is writing a lot of docs and things like that in Markdown. But those experiences inside the IDE that they could use heavily and sort of get that day-to-day feel of what it was like. And TypeScript was a big part of that, and so they invested heavily in kind of the TypeScript tooling, which meant that we were able to show off refactoring support and really rich features much earlier than we might have otherwise been able to kind of show those to people.
Beyang: Who was the founding team of TypeScript, and when did various kind of key people come onboard the project?
Luke: Yeah, I think when we started, Steve Lucco and I kind of were... Start of the project, we started working on some things there. I think we had a handful of other folks who were involved early on doing some engineering work related to the project. And some of it was just people who were part of some of this neighboring projects, like the Chakra engine and things like that who kind of were involved in some of the design discussions and things like that.
Luke: So I think in Anders'... I mean, Anders is one of the favorite people I've ever had the chance to work with in my life, and I've been lucky to have the chance to work with him on several projects. And he just has an intuition about program language design and developer tool experience that is really just incredible. And he's one of the people who, whenever he and I disagree, I know I'm wrong. And I know I just have to figure out why it is I'm wrong, and I have to wait until my mind can catch up. But he just has a really great point of view and design sense and aesthetic around developer experiences, so... Obviously had a huge part in shaping in kind of what TypeScript ended up being was having Anders be heavily involved from pretty early on.
Beyang: What was the language like back in those early days? If a TypeScript programmer were to go back in time and look at the language as it was then, would they kind of recognize it, or has it changed substantially and drastically?
Luke: Yeah, I think it is... I think they would recognize it. I mean there's... One of the key things was a principle of kind of TypeScript, what this... The code name for the project was Strata back at the time, and so I might find myself saying that. But the thing that was...
Beyang: What does that name come from?
Luke: I think we had a set of code names that were all towns in Italy and France.
Luke: Venice was one of the other projects. And so these were all towns in that area of the world. I think there was several other projects that... I think the Chakra engine was called [Ez 00:16:55] back in those days.
Luke: I think there's some places where there was some syntax that was different. The way that types are currently in TypeScript are variable name, colon, type. And I think back in the earliest days we had type name, space, variable name. Instead of var X equals something, it would be int x equals, or a number x equals.
Beyang: Oh, wow.
Luke: And actually one of the motivations for that was that C# had var as the kind of any type. It had recently introduced the ability to basically have an any type in the type system that [crosstalk 00:18:55].
Beyang: Yep. Yep.
Beyang: Got it. I mean, the way you explained the decision process behind that decision is really fascinating, but I imagine there must have been a lot of other kind of things like that about the syntax and ergonomics of the language. How do you settle debates about things like that? Programmers, we can't even agree on tabs versus spaces. When you're creating a language, how do you make those choices?
Luke: Yeah, I think it is one of those things where it's definitely an art, not a science. And so you can bring data into the discussion about, hey, what do other languages do? How complicated is it from a parsing perspective? That much of it you can bring into those discussions. But at the end of the day, it is just an aesthetic choice, and it is part of just the really thinking about kind of how it's going to feel, how it's going to interact with IDE tooling, how it's going to... even what it's going to remind people of in terms of other languages they might have worked with, and how that's going to set up their mental pathways to sort of use things the correct way versus the incorrect way by default.
Luke: And so that is something that I think I kind of mentioned, Anders is really... has just a great aesthetic around this kind of thing. And I think it really helped kind of having him there to kind of break the ties and make the decisions and kind of be the person who could just have a great intuition about what was the right thing to do from a syntax and experience perspective around the language.
Beyang: So TypeScript, I believe the project was begun in 2012. Right?
Luke: Yeah, that sounds right.
Luke: [crosstalk 00:21:59] 2011. But, yeah. That sounds about right.
Beyang: Okay. Got it. Or maybe it was first publicly announced in 2012, because I remember it was very shortly after we started Sourcegraph, I think, that I first saw the first public announcement that I saw about TypeScript. And I kind of want to talk about how Microsoft has changed since then, because certainly from the outside looking in, it seems like a very different company now. It seems like Microsoft as an organization has embraced open source, and they have projects like TypeScript and VS Code that have been wildly successful. But if we go back in time to 2012, 2013, I feel like the brand perception among developers of Microsoft was very different. I think that might have also been reflected in the culture a little bit. I think TypeScript might have been the first major open source project from Microsoft in a while. Can you talk about kind of getting an open source project like this off the ground inside the Microsoft of 2011, 2012?
Luke: Yeah. So I think I can certainly speak to kind of what it was like back then, both kind of before that and in the few years after that. Obviously I left Microsoft in, I think, 2017, so in terms of I think a lot of the changes there have continued and maybe even accelerated over the time since I left Microsoft. But certainly I think open source wasn't a huge part of Microsoft's DNA obviously back in the early 2010s. Right? And there was some famous sort of anti-open source kind of rhetoric that had come out of Microsoft not many years before that.
Luke: One thing I think that is interesting though is Microsoft has... had back at that time even when I first joined and still has a quite large developer division, which is a whole division of the company is focused on building developer tools. And that division had existed for some time, and I think gives... Unlike a lot of other companies, I think Microsoft has a strong identity around building developer tools and in fact has a whole business around this. Right? I mean, it was sort of famously one of Microsoft's many billion-dollar businesses was actually just its developer tools business. So one of the biggest developer tools businesses, even sort of strictly independent of anything else that Microsoft did, it had a big business building and delivering developer tools.
Luke: And so I think that gave it a center of strength around building things for developers that was maybe even slightly independent of what mattered for Windows or Windows Server or Office or anything like that. And so I think some of that focus on open source I think really did come from that developer division, and the developer... the folks building tools for developers seeing what was happening across the rest of the industry and just how much the developer tools landscape was being reshaped by open source projects, whether it was open source tools, whether it was open source frameworks and libraries. Lots of different things were reshaping how developers thought about what they built, and that that was not just something that was in some very specific Linux-based world. It was something that was happening across the industry even for folks building software on Windows or building software from the web.
Luke: And so I think there was that strong sense that open source was this important force for developers that was present through that time period. Yeah, no. I think in terms of... in actual open source projects, there had been some relatively small open source projects before TypeScript obviously. But I say TypeScript was one of the early open source projects that Microsoft delivered.
Luke: One interesting thing, this may be of historical interest, when we launched TypeScript, we did have a lot of debate internally about making it open source. And sometimes I think it was possibly just because people at the time didn't necessarily think TypeScript was going to be particularly big thing. Right? And I think it was just sort of an interesting project that had seen some usage internally, and we just thought it was sort of a nice thing to get out there. But there was a lot of debate about making it open source, and I think ultimately got approval to say, "Hey, we're going to go and release this as open source."
Luke: But the one caveat on that was, but we need to put it on CodePlex, not on GitHub. And CodePlex was this site that Microsoft had that was an open source code-sharing site, very similar to GitHub. Honestly back in those days I think it wasn't even 100% clear. I think it was 80% clear, but not 100% clear that GitHub was going to dominate this space. And so this was part of, hey, if we're going to do these first-party kind of open source projects, let's put them on CodePlex. Let's create the home. Let's dogfood our own open source code-hosting tools as well.
Luke: And we put it there I think about a year, maybe a year and a half later, we did end up moving it over to GitHub. But I think it became more and more clear that GitHub was where the community and ecosystems were. And so we did move it to GitHub, but obviously it's lived there for many, many years now. Since then, it has built up a huge community and ecosystem around that GitHub project.
Luke: But, yeah, it was interesting that even in the early days of embracing open source, it was almost like, hey, if we're going to be embracing open source, let's go and do it in our own open source community and build off our own center of excellence and strength around open source around CodePlex. And I think that might have been an alternate history where CodePlex became the dominant code-hosting platform. [crosstalk 00:27:39].
Beyang: Yeah. You mentioned that in the beginning of the project, it wasn't clear whether TypeScript was going to take off. These days, it's obviously... I think it's almost ubiquitous. It's everywhere you look. Was there a single inflection point, or maybe a collection of milestones that you remember hitting that were kind of step functions upward toward increasing adoption?
Luke: Yeah. There's two things that I kind of always think back to that I think were really... really changed the way that I thought about what TypeScript was going to be, and I think had a big impact on its growth in the industry. I think the first happened I think the day after we actually launched TypeScript publicly. And I think when we launched it, I obviously thought it was great. I'd spent a year and a half working on it before we launched it and was very passionate about it and had been singing its praises everywhere I could go.
Luke: And I think that ended up being this really critical thing. A, it sort of showed me just that the power that these open source communities can have to just aggregate value together, organically evolve these assets. And this didn't have to be perfect, and so people could just incrementally improve them, incrementally add them, incrementally share them, and that became value that everyone then got. And so it became the case that most folks coming to TypeScript actually could get started very quickly, because the library they were working with did have a type description. And if they hit a limit with it, they could go and submit a PR back into that DefinitelyTyped project.
Luke: And so I think that open source ecosystems and the power they can have to unblock some of these kind of difficult scale challenges was one my early kind of lessons. And I think had that project not popped up, it could have... TypeScript may not have ever fully taken off, and it could have taken several more years for it to have any big impact. But I think now in some sense the fact that there is that giant library of type descriptions for TypeScript is this, even independent of TypeScript itself, that is this amazing asset the industry has, that sort of API surface area of all these libraries is now formally defined. And so I think that's a great thing.
Luke: The second inflection point I think about, it was actually after I stopped working on the project. And I still talked a lot with some of the folks who were kind of leading the project at the time, but it was actually when the Angular 2 decided to adopt TypeScript as this sort of default language for Angular, for their second kind of version of Angular. And I think this was a case where TypeScript had been used by a lot of big companies, and things like that. But it's just the first example where a big kind of ecosystem adopted it en masse and kind of made it the default and created a distribution channel for it where everyone who wanted to go and learn and use Angular 2 was getting introduced to TypeScript on that journey. And [crosstalk 00:32:24]-
Luke: And I think then kind of went on to get great experiences in React and other frameworks. And so now these days I think most frameworks have tried to make sure they have a good experience for TypeScript out of the gate. And obviously in things like Deno and things like that, which are built around TypeScript from the get-go. And so, but I think that first example with Angular I think was really key to adoption.
Luke: And I think that was also... wasn't just an accident in a sense. I think that was something that the folks on the TypeScript team worked closely with some of the folks in the Angular project to make sure that it could work well with that and to make sure they did have support. And I think they did actually. Microsoft and Google did a joint announcement of that, which was at that point in time Microsoft and Google were as much enemies as anyone could be. I think that was the popular perception of things. But the fact that they were collaborating on open source developer tools, I think was great.
Beyang: Yeah. That's awesome. I think over the course of any project there's kind of trade-offs you have to make. You have to balance kind of decisions of vision and personal preference from the creators with all the different feature requests that you get from the community and your users. As you think back over the course of the evolution of TypeScript, were there any features that you ended up rejecting that you thought maybe that should have gone in, or any kind of contentious debates about things that did or didn't make it into the language?
Luke: But there was a couple of things I remember early on. The one actually... Just one of the interesting things when you add types to a language is that it turns out you really end up wanting to also add something like classes at the same time. And that's because when you add types, but if you don't have classes, you kind of have to say everything twice. You have to describe the interface for the class-like thing in one place, and then you have to describe the constructor for the class-like thing in another place, and the methods and everything. And so it's sort of like you have to describe the interface and the implementation independently.
Luke: And so one of the things people love with classes is they give you kind of an easy way to describe both the interface and the implementation together in a very succinct way. And so you can use types of course without classes, but classes become... they feel even more natural when you're both describing types and describing implementation.
Beyang: Now what is it like to be part of a standards body like the ECMAScript Standards Body? I imagine... standards are extremely important. It's important all voices are heard. But at the same time I imagine it's quite difficult to find solutions that make everyone happy.
Luke: Yeah, no, I think it's particularly interesting. The thing about standards is they just have a lot of different parties who are kind of involved. And for the ECMAScript standard, I mean it was really, as you might expect, it was sort of all the big players in the web and browser space. Mozilla was very heavily involved there. Google was very heavily involved there. Apple was involved there. Microsoft was involved there. But then a lot of other folks who just had various interests in the evolution of that language.
Luke: And so it did mean that it was more of just a... honestly kind of a fun collaborative environment for evolving this language which is very critical for so many developers and had become such a key part of the web. But a venue where smart folks from all the big software companies at the time were participating in that and helping to drive that forward and bring in the perspectives they got from their various user bases to bear on that.
Luke: But I think it was a great... and it was honestly as much of a sort of deep technical language design community as the C# or TypeScript teams that I'd worked on were. I guess with a bit more of a diverse kind of perspective that was brought from all these different organizations. And yeah, there's definitely some things move slowly in the standards world, and a bunch of these standards have been worked on for many, many years.
Beyang: Yeah, definitely. So in addition to .NET and TypeScript, you made a substantial contribution to another programming language community. And I want to ask you about that. So you were the author, I believe, of the VS Code Go extension. What was the backstory behind that?
Luke: Yeah. I... Actually just it was... The Go extension just last week I think was transitioned from being managed by Microsoft-
Beyang: Yeah, handed off.
Luke: ... to the Go team at Google. So it's actually now mainly... was a project that was my project and then kind of handed over to be managed by some folks at Microsoft when I left Microsoft. And now it's kind of moving over into the Go team. So I wrote a little kind of a little bit of backstory on that as part of that handover.
Luke: But no, I think it was a... I had been involved... I mentioned earlier kind of we had been working on the TypeScript side with Erich Gamma and his team from the very early days. And so I, even after I left the TypeScript project, I did a lot of work with Erich and his team even before they kind of decided to build VS Code. And then they launched VS Code, and then I think it was about nine months or a year later they were planning on doing a sort of big announcement of open sourcing VS Code. When VS Code was originally launched actually it wasn't open source. But about nine months later or a year later or something they did open source it.
Luke: And as part of that launch announcing it was open source, they also wanted to highlight the extensibility model and sort of open up the API for folks to build extensions for VS Code. And I was providing kind of feedback to their team on the API and some of the things related to the extensibility model, and I remember in a conversation we were having just about how we would position the extensibility model when we launched this.
Luke: And one of the things we really wanted to do was make sure that the VS Code was perceived as not just being a Microsoft ecosystem .NET IDE. That we really wanted to make it clear that it was meant to be a developer tool for any language and any kind of tool that any developer on any platform. That was another key thing about VS Code obviously was that it wasn't Windows only, which was I guess surprising at that time for Microsoft.
Luke: And I think one of my colleagues at the time, Shanku Niyogi, sort of made what I heard as a joke. He's since told me it wasn't a joke, but he's like, "Well, we could add support for Go." Because Go at the time was a fairly new language. It had a ton of buzz. It was growing really quickly. It was kind of perceived as being sort of an anti-C# and Java in a sense. Right? It was the new wave of what server languages would look like. And I think he kind of threw that out there as like a, hey, this would be a way of sort of showing the other extreme. Right? A language that's built by Google, a language that's sort of perceived as being competitive in a sense with the C# and Java kind of world. Having support for that would really show that this is not your traditional Microsoft developer tool.
Luke: And so, yeah, that kind of stuck in the back of my head, and then when I was playing around getting feedback to the VS Code team, I thought, hey, well I don't just try to build a tool in for Go as the example to use to give them feedback. And so started just playing around with that kind of over the next few nights and things and had a simple demo of something and showed it to Erich. And I think he was kind of incredulous that it could even work, because the way these things worked at that time... And I think you obviously know. You built kind of IDEs and that sort of thing as well.
Luke: But it worked by shelling out to a binary every time you hovered over something in the IDE. We would shell out to a binary on that, get back the result, and then present that in the tooltip. And so you're potentially doing this tens of times a second. Right? As you're hovering over things. And he, Erich, was like, "Aw, there's no way that could be performed." Right? Always do this stuff in memory. We have all these techniques that are fancy around how we kind of manage this kind of IDE tooling.
Luke: But it turned out that Go binaries are really fast. It's one of those things about building flat, native binaries is they load fast, and they run... They don't have a lot of up-front spin-up time. And so, yeah, so we had this tool that actually worked surprisingly well. I think it used a lot of techniques that some of the other Go tools in the ecosystem at the time had been pioneering like the Vim Go extension. That was probably the most popular tool. Still is one of the most popular tools for working with Go. I'd shown this pattern of using all of these little, little tools that were available in the Go ecosystem and kind of plugging them all together. And yeah, so we did that.
Luke: We initially, at that launch of the open sourcing of VS Code and the release of the extensibility model, we showed off this, just as an example of what you could do with the extensibility model, we showed off this Go support. Erich onstage at the conference we were announcing at kind of demoed that for a couple of minutes. And yeah, I think even then we didn't really think it was necessarily going to be something that a lot of people used. It was more of a marketing exercise to go show off what was possible.
Luke: But it turned out there was a lot of desire for richer tooling in the Go space as Go was taking off. And a lot of people started picking it up, and I started spending more time kind of supporting it in the open source project. And yeah, it ended up kind of taking off. And I think these days it's now one of the most used tools out there for Go development, which is kind of cool.
Beyang: How much Go had you written before writing this plugin?
Luke: This [inaudible 00:47:06]. No actual Go, like no production Go. Because I am just sort of a student of programming languages in a sense, I... whenever any interesting new program language comes out, I play around with it I'd say from an academic kind of perspective. So I understood the language sort of from a what-it-is perspective. But I had never been a user of Go at all. And so that did... I think that did mean I didn't understand what was important for the tool viscerally in the same way that I think I came to understand what was important for the tool over the coming years when I started working on it.
Luke: But it is always one of the interesting things, as you build tools I mentioned that sort of quick feedback loop. If you are somebody who... building a tool for yourself, I think you can get that feedback loop. I'd say I didn't have that experience at the time, and in fact it was many years before I... Now these days I write a lot of Go code, and I use that Go extension sort of everyday. And so I am a user of it and get to sort of reap the rewards of that. But at the time, no. I was definitely building it for someone else in a sense.
Beyang: Yeah. That's interesting. You're now the... Well, you're the second person I know of who wrote an editor plugin for a language that they didn't have that much experience with. The other person is my teammate, Felix Becker, who wrote the... I think it was one of the first PHP language servers. Not written much PHP before. And I wonder if there's something to that experience of... Because if you're already proficient in a language, you've kind of gotten used to the tooling ecosystem in that language, and you don't... Maybe that blinds you to a certain extent, to seeing how it might be improved. Whereas if you're new to it, you're like, "Oh, I'm used to using these tools. How can I adapt that experience to this new ecosystem?"
Luke: Yeah. I think there is an aspect of that. And I think it can in a sense go both ways, but I think there was a sense back in those days for example that... There was a strong sense that Go developers didn't want IDE tools.
Beyang: Yeah. Yeah, right.
Luke: The Go developers wanted to be in Vim or Sublime and wanted to have this very simple experience. Go, it's sort of part of Go's aesthetic in a sense is to be this fairly simple language. And I think you go talk to the folks who have... creators and leads of the Go language, and I think one piece that's kind of related is actually is they... I think it's Rob Pike. I forget. Maybe it's...
Beyang: Mm-hmm (affirmative).
Luke: I think it was Rob Pike who had a comment that it's basically somebody asked about debugging support in Go. And he's like, "The only debugger you ever need is printf debugging." And I think there was that sense from the Go project itself that... and from a lot of the early users of Go, that that was part of the ethos of the language was to not have this kind of heavyweight Java and C#-style IDE experiences.
Luke: And so I think it did take a little bit of that, an outsider coming from those perspectives, Erich Gamma coming from the Eclipse kind of world and the VS Code world. Me coming from the C# and .NET TypeScript world. I think having that perspective that, hey, maybe we don't want to go all the way there. We don't want to have this feel like Java and Eclipse, but there might be a step in between just raw Vim and full Visual Studio.
Luke: And I think VS Code itself is sort of designed around that idea that there might be this point in between. There might be this thing that's kind of halfway between a text editor and a IDE that kind feel a lot more lightweight than an IDE but can feel a lot richer than a text editor. And I think the Go extension and sort of bringing some of the key tooling capabilities that VS Code was able to light up in a natural way around hovertips and error feedback and some of those sorts of things, I think it did come in some sense from a bit of that mindset, that, hey, I need tooling. Some of this stuff can actually be valuable even for languages like Go.
Beyang: Yeah, it makes sense. So I feel like I could spend an entire day talking with you about TypeScript and your work in the Go ecosystem and your time at Microsoft. But I want to get to what you're doing now, which is as interesting, if not more. You're a CTO of a company called Pulumi that creates this infrastructure-as-code tool that we use at Sourcegraph and I can personally vouch for. It's fantastic. Kind of the idea is that you can write your deployment config as kind of just code in your favorite programming language. So tell us about the journey from Microsoft to that position.
Luke: Yeah, so I think I left Microsoft back, I think, 2017, and ended up going to AWS actually and working on EC2 and doing a bunch of just raw cloud compute stuff there. Had a great there. I mean, AWS is just an absolutely kind of business, and EC2 in particular is just the side of that business, the rate of growth is incredible and really, really fun, and some great folks there.
Luke: I think I obviously had sort of had an understanding of what was going on with the cloud just from being at Microsoft and seeing Azure and then things like that. But certainly being at AWS gives you just a whole other level of perspective on kind of what is happening around the cloud and the breadth of adoption it was getting and how that was changing how sort of everyone was building and delivering software.
Luke: And I'd say one of the things that I had been thinking about in the back of my head for a long time was just, hey, I kind of come from this programming languages perspective, this dev tools perspective. That's my hammer. That's my thing I bring to the table is, this is a thing I can use to kind of help solve these problems. And been thinking for a while about in this cloud space, there's these incredible building blocks that these cloud providers have made available. AWS has hundreds and hundreds of these services which take away operational complexity and really simplify what you can build and how quickly you can bring that to market.
Luke: And the problem was really just those building blocks are still really hard to kind of put together. To actually go and build an application out of those building blocks requires an incredible level of expertise and proficiency in infrastructure development and sort of infrastructure operations. And so I sort of felt like there should be a kind of programming languages and developer tools and software engineering way of thinking about that. That in those disciplines we have built up great tools for taking great raw building blocks, whether they're operating system primitives or standards-based technologies in the web browser and democratizing access to them by adding layers of abstraction and layers of software engineering capabilities on top of them and layers of programming languages and IDEs and things.
Luke: And so I think the genesis of kind of Pulumi was me and Joe Duffy, who is our founder and CEO, really believing that there is an opportunity to significantly change how we think about applying software engineering to cloud infrastructure and to really harnessing all the value that's out there from the cloud providers. And I think we've been on that journey now for about three years, kind of watched the Pulumi open source project about two years ago.
Luke: I've really seen that take off really significantly over the last two years, and I think there has been a lot of desire out there in the industry to really bring to bear what software engineering and all of these things we've talked about over the last little while in this conversation, bring a lot of that same thinking into the cloud infrastructure space and make it accessible to a broader collection of users.
Beyang: So how would you describe Pulumi to someone who has never used it before? There are a number of other tools that are kind of in the infrastructure-as-code space. How's Pulumi different from those, and why should people use it?
Luke: Yeah, so I think Pulumi is really... One way I like to talk about it is sort of, think it was, not infrastructure-as-code but infrastructure-as-software. So really trying to bring all those software engineering capabilities that we know about, whether it's the ability to create reusable abstractions or to publish reusable packages or use IDE tooling or debug or test or all these things that we know that sort of support rich, robust scaling up of the kind of applications you can build, bringing all that into infrastructure-as-code space.
Luke: And so the core thing that sort of is the most obvious piece of that is that Pulumi lets you use existing programming languages, so TypeScript or Python or Go or .NET, to build your infrastructure. So instead of you having to write JSON or YAML or HCL or some of these very constrained domain-specific languages, you can use the language that folks... that they're some of the most popular programming languages out there that folks have familiarity with to describe infrastructure.
Luke: And that's both useful, because it's familiar. I know these languages aren't a foreign thing to large swaths of developers and sort of DevOps folks in the industry. But I'd say that the biggest benefit is that it brings to bear the ecosystems that I just mentioned around tooling and package management and IDEs and that sort of thing. So all those benefits that you get from those language ecosystems are now the benefits you also get inside your infrastructure.
Luke: And then most importantly, programming languages... sort of a thing that programming languages are fundamentally really great at is creating the abstractions. Right? I think the key... If there's one thing that programming languages are about, it's giving names to pieces of functionality and abstracting the way and putting them in a library and distributing that library. And this is... that idea of building reusable abstractions is sort of one of the key things that we want to see unlocked in the cloud infrastructure space.
Luke: And then we believe that programming languages really help to accelerate that trend. And so we've seen that in practice, that people build really... we build some really rich libraries, and other folks out there in the community and internally inside companies that are using Pulumi are building really rich reusable libraries, sharing them within their organization and using that to scale up what they can build. Because they don't have to copy, paste thousands of lines of YAML all other their codebase.
Beyang: Yeah. I mean that's, from my experience, using it to... And I certainly... I was a little bit skeptical at first. It was actually a teammate of mine, Geoffrey Gilmore, who kind of brought it into our team. But I think once I had that experience of writing our Kubernetes cluster configuration as just a bunch of TypeScript. And you had autocomplete and all the kind of fancy IDE stuff. It just felt a lot more natural and intuitive, and it's... I feel like it's one of those things that you kind of have to experience, and then you kind of see that the difference is night and day.
Luke: Yeah, I agree. And I think it's actually kind of related to some of the conversation about even what we were talking about with the Go tooling, and just having that perspective of, hey, even if the sort of existing ethos is right around this very simple, lowest-common-denominator, text-based idea.
Luke: There can be a lot of value that comes from bringing these richer tooling, whether it's the idea that when I'm typing... Even just the AWS API is enormous, for example. Right? And I think people think of big APIs out there like the iOS APIs or the web standards APIs. I'd say AWS is probably bigger than either of those, in terms of the total number of resources and properties that are available to manage things within AWS. It's just absolutely enormous API surface area.
Luke: And so just having that IDE tooling that exposes help text just in the IDE as you're typing about [crosstalk 00:58:55]-
Luke: ... that exposes squiggles on things when you are missing some required property. All these little pieces of feedback that help you to explore and understand and quickly iterate on those things are really valuable in the cloud space. And I think there's something that the existing community sort of had not really seen as important. But I think now we're seeing, as more folks with a software engineering and developer mindset are coming into the cloud space, I think there's an increasing number of folks who are kind of looking for, "How do I get that same level of productivity, reliability, ability to scale up complexity. How do I bring that in?"
Luke: And I think some of it is, again, coming from that mindset of tools and IDEs in programming languages and bringing that into the space.
Beyang: In the space of kind of infrastructure configuration management tools or infrastructure-as-code tools, there's kind of this distinction people draw between declarative and imperative approaches, where imperative is kind of like the bash script mentality, where you're doing one thing after another, and there's kind of state that you're modifying. And then there's declarative, which the tool that always comes to mind when people say that word is Terraform. You just declare kind of the infrastructure that you want to appear, and then it figures out how to kind of realize that for you. Which... Does Pulumi fall into either of those paradigms?
Luke: Yeah. I think it's a great question. One of the sort of key questions a lot of folks ask when they first come to Pulumi, and... In my view, the thing that is sort of really unique in some sense about Pulumi is that it really tries to get the best of both of those worlds. And I think it tries to show that the decision to either/or, there is the ability to sort of get what people love about declarative...
Luke: And when I say "declarative," I try to clarify that there's notion of sort of desired state configuration. Really the key thing about Terraform and CloudFormation and other desired-state models is that the thing that you write down is the desired state you want the system to be in. And then the tool is going to try to get you from the state you are in to that desired state. It's not going to do this in the actions you specify. It's going to say, "I know that the current state is this, and the desired state is this, so I'm going to figure out what the minimum way to get there is safely. I'm going to preview that and understand whether you want to take those steps, and then I'm going to go and do those steps for you."
Luke: And so this idea of desired-state configuration versus kind of imperative modification of this state is, I think, the really key thing that Terraform and CloudFormation, other tools like that, have shown can be really effective in the cloud space.
Luke: Then there's the question of how do you author that desired state. So do you author that desired state in something like YAML, or do you author that desired state using code that imperatively can build up that graph of what the desired state is? And I think there... My feeling is, when you have 10 resources, it's reasonable to think that it's simpler to just write it in YAML or write it in HCL or something like that. There's a small number of these things. You can write them fairly directly. That's at that scale that really fits the job well.
Luke: The thing that we see is that, especially as folks are moving into modern cloud technologies, things like serverless and containers and especially Kubernetes, the number of resources they're managing, the speed at which those resources are changing, it's going incredibly fast. So it's going from I've got tens of resources to now I've got, in some cases, tens of thousands of resources. And at that scale you can't just have hundreds of thousands or millions of lines of YAML and copy, paste stuff around. Right? You have to have some organizing principles at a higher level. And that means you have to have sort of abstractions to tame that.
Luke: And abstractions just... People have tried to do that on top of these YAML-like things, but inevitably as soon as you try and introduce abstraction, you end up trying to recreate a programming language effectively. And so I think what our view is, instead of trying to turn YAML and things like that into a programming language, let's take programming languages that we know work well. Let's embrace the idea that we can imperatively construct the desired state, but let's still maintain all those benefits of being a desired-state configuration model.
Luke: So we think that that really can scale up to the complexity of the kind of things people are building now with these modern cloud technologies and at the same time can provide all this rich tooling around that development experience.
Beyang: Yeah. That thing you just said about having some sort of imperative programming language to generate a configuration that then you can apply declaratively, that kind of... We actually do... There are two places in our codebase where that happens now actually. One is, we use Buildkite as a CI service, and you describe your CI pipeline as a bunch of YAML. But we actually have a small Go program that generates that YAML for us, and then that gets applied, becomes a real pipeline.
Beyang: And then for the Kubernetes cluster, the distribution... This is not so much the case now, but once upon a time, we had a Go program that would also take in some configuration points and then generate, spit out a bunch of YAML that then you could apply to run in cluster.
Beyang: So I think you're onto something there. There's something about the... At some point there's customization or complexity that you have to capture in describing what you actually want to be applied that is hard to do in kind of a declarative DSL. But at the same time, you want that kind of guarantee of... I don't want to worry about implicit state that I kind of created or affected along the way. I just want this thing that it spits out, this YAML configuration or whatever, is the source of truth, and this is going to be applied to the cluster. I don't know if I'm making sense there, but...
Luke: No, that's very true. That pattern you kind of mentioned, I think, when we talk to development teams, and just about everyone out there has had this experience of building a tool which generates config and building some software that does use imperative tooling to build, whether it's YAML or HCL or JSON or whatever it is. Because those formats just don't scale up. As complexity increases, they are not human-editable formats at certain scales.
Luke: And software is sort of the proven thing that can scale up to tremendous levels of complexity. You look at... one of the analogies I often make is sort of that these YAML and HCL and kind of things are sort of like the assembly code of application development. Right? And assembly code is great when you're writing hundreds of lines of assembly.
Luke: Had we not invented C and all these programming languages that sort of built on top of that and continue to raise the abstraction level and built the library ecosystems that the C standard library and all the library ecosystems that then came about with Java and these kind of things, we would have been stuck reinventing the wheel. We would have been scaling up complexity. It would have been very hard with assembly, because you're just copy-pasting stuff. It's reusability is much harder. You get colon conventions and stuff like that are tough.
Luke: And so I kind of think of a similar thing where with these configuration formats, they can work really well at a small scale. And I think the cloud for a lot of people has been small-scale for the last few years. And I think a lot of folks are now hitting the point where, as they really lean into some of these cloud-native technologies, cloud is not small-scale anymore. It's big. It's an intertwined part of how they build and deliver and deploy and manage their software, and the lines between the code that they own, and the managed services they use inside their cloud provider is becoming increasingly intertwined.
Luke: And so that is leading them to... It's not okay now to just manage that using assembly code. Right? You really do need to use these higher-level tools to manage them.
Beyang: Yeah. I kind of get the sense that managing your infrastructure and the configuration, it's such a painful problem. And there's been a lot of tools that have sprung up. A lot of them in the past that five, 10 years to kind of deal with that problem. Do you think that the field is going to converge moving forward, like people kind of standardize on one or maybe a few solutions? Or do you think that different kind of deployment tools will naturally evolve towards different computing platforms or different language verticals or ecosystems?
Luke: Yeah, I don't know. I think the cloud infrastructure space is still honestly... and even though it's been near 10 years since EC2 was launched, I think it's still very early days. And I think there's still a lot changing in the cloud space. And so I think there's going to continue to be quite a bit of evolution. I don't think we're at a point where we're just going to start solidifying into one particular tool.
Luke: But I don't know exactly where that'll go. I think there's a few interesting kind of trends that I'd say I see. I think one is this idea that complexity and... both total complexity of cloud infrastructure and the rate of change of cloud infrastructure are increasing quite rapidly. And especially as folks are moving into these kind of relatively newer technologies which are seeing a ton of adoption now, I think that is going to lead to a different class of tools being popular as those become the norm than what's popular kind of over the last five years when running and EC2 VM or two was what it meant to use the cloud.
Luke: And so I think that is going to drive some change there. And I think some of it's going to be tools that manage that complexity. Some of it's going to be other things. But I think the other piece that is interesting is an increasing sort of shift towards taking automation further and further here. And I think you're starting to see this in the Kubernetes world, for example, with the operator pattern. You're seeing folks move from human in the loop on a lot of these deployment tasks to automated tasks maybe inside a Kubernetes cluster, where there's some deployments are being automated either via CI/CD pipeline or by an operator or something like that.
Luke: And so I think there's interesting things that are going to happen there. I always hate to say kind of "no ops." It's sort of this term that gets [inaudible 01:09:34] little bit. I think there's no chance that we're going to end up in a world that's really, really "no ops," but I think there is sort of increased levels of automation that are going to happen, just because as you get to this complexity and speed of change, you have to find ways to remove humans from the loop on more of these things and automate more of these things.
Luke: And so I think for the kind of infrastructure deployment and management space, I do think this is going to be an interesting trend in that direction.
Beyang: Yeah. That's interesting. One of the other people we talked to for the show is David Cramer. He's the creator of Sentry, the application error monitoring tool, and he made the prediction that in five years people weren't going to use Kubernetes anymore. Or if they did, it would be hidden in such a way that you wouldn't realize it's Kubernetes. Do you feel the same way, or do you think in five years' time a lot of your development team will have to kind of understand kubectl and a lot of the kind of concepts in Kubernetes?
Luke: Yeah, I don't know. Five years is a long time in cloud space.
Luke: And I think a lot is going to change over the next five years. I think Kubernetes obviously is become a huge staple. I think when we talk to folks using Pulumi, just the number of organizations that I talk to who are moving towards Kubernetes as their platform of choice for kind of their modern cloud initiatives is really striking. It's just showing up kind of in almost all the conversations we have with teams about their future cloud direction.
Luke: So I think there's definitely going to be a large move in that direction. I'd say in my mind I would be disappointed in some sense if Kubernetes was the end state of this. I sort of fundamentally believe that we want to get to a world which is significantly more developer-friendly and democratizes access to these capabilities to another order of magnitude beyond kind of where Kubernetes is. I think Kubernetes makes some things quite a bit simpler, but it really is still not nearly where I would hope we kind of get to in terms of these platforms. And that may happen within the Kubernetes ecosystem. It may happen by something else coming around that jumps-
Luke: It might happen with serverless. And I think either with serverless itself or with some of the things that are doing serverless patterns involving within the Kubernetes ecosystem itself. I sort of... My guess is it won't... In five years it won't look exactly like Kubernetes, and it won't look exactly like serverless, but it will have lots of things, lots of elements of both of those. And I think there'll be a bit more of a continuum there. And it will be significantly more focused on the developer experience, not so much the kind of operator experience. I think more of that will fall into the background, as you kind of suggested. And we'll see things that empower developers to move quickly and reliably becoming the platforms that we talk about.
Beyang: Makes sense. Before I let you go, I want to get your take on kind of a random question. But given your background with programming languages and IDEs, I wanted to ask you. Cloud IDEs, passing fad or here to stay?
Luke: Yeah. This is a funny one. I've been sort of involved in various cloud IDE-kind of things for many years. And I'd say I think I started off being really bullish on cloud IDEs, probably on the order of 10 years ago. And really thinking that this was the path to go. And I think the more time I've spent with them, we've... As an industry, we've built really exciting kind of incarnations of this idea over the years. And every time so far we've kind of seen that they aren't the thing that developers end up picking. Right?
Luke: Developers still end up today wanting to default back into something on their local machine where they can plug into their whole local toolchain, and they can understand the environment they're working in and have complete control over it. I sort of continue to be... sort of to your five-year point, it's one of those things where in any near-term timeframe, I don't see that changing.
Luke: And yet I continue to think that over the longterm it sort of has to change in some sense. I think that that change is going to come. And I don't know what this will exactly look like, but I kind of feel like it'll have to come from something about the developer desktop experience changing fundamentally that we no longer think about our machines as... It's almost like we no longer think about our machines as pets. We think about them more as cattle or something. Right? Sort of something similar to that transition where we, as developers, shift the way we think about the environments in which we do development in a way which makes it possible for cloud IDEs.
Luke: And it will probably require cloud IDEs being able to offer something fundamentally better that helps people pull across that boundary. And I don't know what that'll be yet, but I'm excited to see.
Beyang: All right. One final question. If someone listening wants to try Pulumi now, what should they do?
Luke: Yeah, so we've got the website at pulumi.com. You can go there, click "get started," walk through a little tutorial that'll show you how to use Pulumi with your favorite cloud, whether that's AWS or Azure or GCP or just overall Kubernetes. And then the Pulumi open source project is github.com/pulumi/pulumi. So all the core pieces both Pulumi are open source. You can go take a look at the project and engage with the community and the community Slack and let us know what you think.
Beyang: My guest today has been Luke Hoban. Luke, thanks for being on the show.
Luke: Thanks for having me.