Jean Yang, Founder and CEO of Akita Software
Jean Yang, Beyang Liu
Jean Young is the founder and CEO of Akita software, the eBPF-powered API observability service. Prior to starting Akita, Jean was a professor at CMU where she worked on programming language design.
Show Notes
Highlights:
- Making like little VB interfaces
- Computer science is really different to programming
- Trialing co-founders
Transcript
Beyang: Welcome back to another edition of the Sourcegraph podcast. Today I'm here with Jean Young, the founder and CEO of Akita software, the eBPF-powered API observability system. Prior to starting Akita, Jean was a professor at CMU where she worked on programming language design. Jean, welcome to the show.
Jean: Thank you. Glad to be here.
Beyang: Awesome. So to kick things off, how did you get into programming?
Jean: Both my parents were computer scientists, so they had started out as electrical engineers way back in the day before computer science was a thing. They both got into computers. So we had a terminal in our home pretty early on and, I guess like many other parents, they liked to teach their kids what they do.
So I learned BASIC pretty early—I think they taught it to me when I was, like, six. We had a computer in our home pretty early. I taught myself web programming and pretty much as soon as I got an internet connection, I just got really into the internet. And in fact, I had my internet time limited. And so it felt like a thing you're not really supposed to do, you're like rebelling... So I think that programming was a thing that everybody in my family does, but web programming, like, that's the thing that you only get to do for a limited time a day.
Beyang: That's hilarious. Do you have an earliest programming memory? Like, something to do with BASIC or web programming that stands out?
Jean: I was pretty into Visual Basic actually, because I would make these little visual interfaces to click through, but you couldn't really share it. So that's actually why I got really into web stuff. Because before the web, I was like, "All right, I'm making like little VB interfaces, but there's nobody else—like literally nobody else—that can run them." So I didn't really understand. I was just like, "How do people ship this stuff? Like, how's anything work?"
But I think that when I started doing web stuff and I discovered JavaScript—which is funny, because I don't do any frontend stuff anymore—but I was like, "Whoa! Stuff that runs on the internet is amazing. You can just share it with people."
Beyang: That's awesome. What were some of the things that you built on the web in those early days?
Jean: A lot of websites. I had a website about everything that I cared about. I had a Tamagotchi website—I was really devoted to it. I built my friends websites. I had chat rooms on all my websites. It was a very very social for me. It was very much about interconnectedness.
Beyang: That's awesome. That's awesome. Did you make any friends on the internet?
Jean: No, not really. I think most of my internet friends were, like, pretty random, creepy people. I mostly just talked to my existing friends and I think that my parents warned me off internet friends pretty early on. So I think that was not something I spent a lot of time doing.
Beyang: Makes sense. Cool. And from there, was it kinda like pretty much a straight shot to studying computer science in university? Or what led you down that path?
Jean: Yeah, that's a great question. So, my mom, my dad they were both doing computer science and programming—my mom still works as a professional programmer today. My uncle also. I'm the oldest cousin, but most of my cousins, like everybody, except for one person—all my cousins in America, I guess But I think that, when I was figuring out what I wanted to do, I think my feeling was like, we immigrated to America so that we could have other options. You can do anything. You can be a doctor, you can be a lawyer, you can go into politics. You can't be the president, but you can be many other things. And they really wanted me to explore my options. Because they were all like, "We ended up in this because the government assigned it to us. Maybe you should do something else."
Yeah. And then I think there was also a lot of gendered stuff, you know. At the time, computer science wasn't as mainstream. I think a lot of people saw it, some of my teachers saw it as not really, like, a "girl thing"—which I think they would really take back today. But at the time, no one knew very much about it.
So I declared economics my first year of college. I realized I didn't care at all about the economics classes. Like, when it came time to fill out my course syllabus, I just copied it off my friend. I was like, "Hey, what courses are you taking? I'll just take the same ones."
Beyang: Okay, so your natural proclivities and inclinations led you back to computer science. At what point did you make it official? Was there a point where you' were like, "Ah, this economics—
Jean: Yeah, sophomore year. Because I took computer science classes the whole time. I was like, "Man, this economics thing isn't working out." So I was actually pre-med for one semester and I took two semesters of organic chemistry. And then I was like, "This pre-med stuff is really intense. It's the opposite of what I wanna be doing." Also, like, taking organic chemistry without taking other chemistries was pretty rough, but I figured it out by the end.
Beyang: I feel like everyone's, "Oh, that's, like, a huge leap..."
Jean: Yeah, I was like, "You know, I was pretty good at AP Chem, so I think I can just do it." I remember for the whole first semester, I just didn't really know... I didn't know what anything was called, because it's a lot of nomenclature. So I completely studied the wrong section of the textbook for all of the tests. So then when it came to the final, I went to the final study session, and I was like, "Hey, I'm not doing so well in this class." And then I was like, "Oh! I've been studying the wrong section of the book." And then I did really well on the final, because I finally studied the right section of the book. And then the TA was like, "Oh my God, I've never seen someone improve so much. You can do anything." And I was just like, "No, I was really just studying the wrong section of the book. But yes, it was you, you helped me." Yeah, I think every teacher likes to believe they saved someone, so I feel like she really believed she saved me. But I was really confused, because I didn't even know what anything was called. I was like, "Let me just study the metal section of the book" and "Oh shoot. Like, the whole test is on the other book section."
Beyang: That's crazy. Okay. So what I'm getting so far is, you come from a family of geniuses and you are yourself a genius.
Jean: Well, I think computer science is just a skill. It's like writing or math, anyone can do it. Anyone can get better at it. Yeah. I think when that's your family, it's just like, you start learning it, you learn more of it, and anyone can get better at it. And I think that's a big part of why I believe in the accessibility of this stuff, actually. I think there's no magic about it. There should be less and less magic about this stuff as it becomes more and more a part of our lives.
Beyang: Hundred percent. When you were going through undergrad CS, having had the background that you did, you know, exposure to computers at an early age, do you feel like your experience was unique in some ways, or, how did you view yourself as going through the program?
Jean: I think computer science is really different than programming. And I think that if you have been exposed to computers more, it doesn't actually help you in a lot of stuff. Like, it helps you some because you don't have to make the initial leap to procedural thinking and thinking algorithmically a little bit. So I taught at Carnegie Melin for a couple years, and when I was teaching, something that people told me there and that I think is true just across everything I've seen, is people who show up having programmed a lot still haven't really learned the concepts most of the time. Like, you might have programmed but how well do you understand algorithms or how well do you understand recursion?
And I think that I had a bit of a chip on my shoulder initially. Like, I should know this stuff better, I've seen it for a long time. And I think that got in my way. I think there's just stuff everyone has to learn initially.
Beyang: Yeah, a hundred percent.
Jean: And I think that this is really at odds with... I think a lot of people are like, "Oh, computer science education... Why do that if you've been programming?" But I think for me, learning the concepts was quite a different thing than learning web programming, for instance.
Beyang: Yeah, there's something to be said for learning the principle foundations and the theory behind something, thinking through it in a rigorous fashion.
Jean: Yeah. Or thinking algorithmically or proving things about what you're programming and proving bounds about your algorithms. I think that most people never have to do it again, but it was definitely useful.
Beyang: Do you think that appeal—like, it was this new part of computer science that kind of transitioned from hacking on stuff and just being very hands-on on the application side to like diving into the theory—did that lead you into programming languages? Was that kind of the motivation or interest there?
Jean: Yeah. Yeah. Because I think for me, one of the main reasons I didn't want to do computer science or I listened to other people when they were like, "Oh, you probably don't wanna do this" was I thought the tools were just crap. I was like, "Man, nothing makes any sense. It works, like, 80% of the time." I didn't realize that there are fundamental concepts that this was based on. Because I was like, "All this stuff is kind of a mess."
And I feel like, if I had learned it theory first, I'd just be like, "Okay, yeah." But when I learned the theory after experiencing the tools for a long time, I was like, "What, you mean this can be clean? And it was supposed to be like this?" And so I think it was partly that—where I was like, "Okay, I think it'd be very interesting to spend my career cleaning this up."—and I think another part of it was very selfishly motivated where I loved the theory, I loved my theory of computation class, I thought algorithms was really fun. And so I asked myself, "How can I do more of it? Because this does not seem to apply to the kind of hacking I was doing before. So what kind of stuff could I be doing to get closer to doing this kind of work in my everyday life?"
And so I will say now, as a company founder, I don't really get to do that stuff, but I feel like it's based on those ideas.
Beyang: When you graduated from undergrad, I feel like maybe a fork in the road for a lot of people is, do you go and get a graduate degree and do research or do you go into industry? At that point in your life, was there like a question in your mind or was it pretty straightforward?
Jean: I had interned at Google the summer before my senior year—that was summer 2007. Google was very hot then—well, it's still very hard to get a job there. But it was like, not clear if you're graduating from a top school, if you could get a job there. I had a return offer from there, so like, my friends were like, "Oh my gosh, that's where you're gonna go." And at the same time I was like, "Okay, I've done a few internships in industry at this point, I don't know that that's what I wanna do, because I would be using state-of-the-art tools, but I feel like the tools could be better. So where could I work on that?"
And at the time, there were compilers you could work on. There was a company called Green Acres in Santa Barbara—they made compilers, I think like for defense, there were a few other places working in compilers at the time, but it was very far from the mainstream. The kind of dev tools they were working on was... One example of a project they finished the summer I was there was, they made distributed linking, so distcc. Because before that, linking was not distributed and it took, like, ten minutes to link any code! But, you know, I was like, "Okay, I think you can move the needle on compilation performance or, like, performance things, but anything semantically—like, having to do with programming tools of the kind that I wanted to do—there just really weren't opportunities for doing that in the industry at the time.
And so to me, the question was, "Where can you go to get better at doing the kind of work that I wanted to do?" Which is new kinds of tools, better semantic tools for understanding software systems. I applied to grad school. At the time, I had deferred my Google offer to April or whenever the grad school decisions were.
So if anyone tuning in is, um, an undergrad, a lot of undergrads don't know this, but people make you offers and tell you, you have to tell them by October. You can defer them until whenever your actual decisions are. And at the time, I was like, "Google is a very valid option. It depends on what grad schools I get into."
My undergrad advisor even said, "Look, there is a chance you may not get into the schools you want to, like, it's never a guarantee. You need to be prepared." Because I was like, "I'll go to grad school if I can get into the schools I want." And it wasn't a very big set, so I was prepared for all outcomes.
But then I ended up getting into some schools I wanted to, I visited them. I really liked MIT. And so that was the moment when I made the decision.
Beyang: Yeah. I feel like, up until a couple years ago, there wasn't really a large developer-tool ecosystem either. Like, the main options, like you said, were those compiler companies.
Jean: Yeah. There's really been a boom in funding in dev tools in the last few years and I've seen you've written about that. And I think the diversity of dev tools you can work on has also really expanded. But before, even in the bigger companies... So I will say, like, a couple years after I made the grad school versus not grad school decision, like Google, Microsoft—and Microsoft has always had pretty good dev tools actually—but anything that was not Microsoft, Google, Facebook started investing a lot more in dev tools, and then you saw startups joining in that game after that.
Beyang: Yeah. Yeah. What did you get up to at MIT?
Jean: So during grad school, I created a new programming model and programming language. The language was called Jeeves. The programming model was called policy-agnostic programming. The main idea was that it factored out information flow and security access checks out of your code. So the general thing I was interested in was like, "I'm interested in removing boiler plate from code." At the time, I was still very interested in programming languages or the vehicle... Because I was like, "Okay, code is a mess." And so there's many ways you can go around cleaning up code. And at the time, my undergrad professors very much believed in the philosophy that better programming models and better programming languages are what it's going to take to clean up the mess of software that was out there. And so the premise is, if only we all use better languages, our lives would be better.
And in some sense, that's true. Like, you know, the mainstream has really shifted towards that in the last ten, fifteen years. So at the time I was like, "Okay, what's the next thing after memory management that would really clean things up?" And so I was, like, factoring access checks out of code—if you write any web program, like, half of your code is access checks and everyone gets them wrong.
Something that's been cool to see since then is that there are a bunch of startups now, and there are a bunch of things that are like, let's factor this out as a module, as a library, as like all kinds of stuff. But at the time, I just really worked on, "Okay, semantically, what does it look like to factor this out?" (If you wanted to do information flow and not just access control.) So access control is like, "Here's a value. Let me protect it." Information flow is like, "Here's a value. Let me do computations based on the value."
So let's say my location is secret. An information flow violation would be like, "Show me all users who are in the Bay Area right now." And if my location is secret, like, my location should not contribute to this count. So it was really like, "Let's take this to the next level. How do we track that?"
I came to some interesting conclusions while working on this, because I was like, "Okay, cool. This is great as a theoretical model, this is how you would do it." Other people have told me, "I've read your thesis. It influenced how I implemented this library." And I've been like, "Great."
But for me, I was like, "Okay, I want to clean up actual code in real life." And so I started implementing a domain-specific language based on my language. And I started building web apps based on it, because that's been my goal of who I wanna help, what kind of apps I wanna help the whole time.
And I discovered something which is... in web apps, most of the code is in application code: it's application database code, it's your ORM code, you call it to the database. And I said to myself, "Oh shoot. That was outside of what I was modeling in my application language." And so this is where the illusion broke for me: That any kind of language-level guarantees really solve all of your problems, because that was when I started realizing just the, the sheer scale of software heterogeneity. Because even in a simple model-view controller web app, you have your model, you have your view, and you have your controller. That's three heterogeneous runtimes. It's three heterogeneous languages. There's no way you could... well, you could have a single one, but that's just so unlikely.
And so that was where I started thinking about, "Oh shoot, anytime you do anything in one of these web systems, you're calling outside of your language." And so if you're coming from the "We're doing this for the guarantees"... If you have a completely closed system, like a spaceship or like an operating system or something like that, you can verify something down to the bare metal, and I think Amazon does a lot of this, like, hypervisor kernel, like, verification stuff. But anything web, you're just kinda screwed. Like, you have so many parts, they're all running in different runtimes. It's worse than running in different languages, because if you have any kind of managed runtime—and, like, any database has a pretty substantial runtime—like, you just have so many components and it's really about, like, your interacting components... is what I realize at that point.
And so to answer the question, What did I get up to in grad school? I made a new programming language. And what did I learn from that? No programming language is gonna save you! There's too many parts.
Beyang: Interesting. Interesting. Was that, like, a gradual realization or was it a sudden epiphany? Do you remember?
Jean: There was a sudden realization, but I was also very deep in it. Because it was... my first paper was, "I made this new programming language." And then I went to my advisor three quarters of the way through and I was like, "Hey, I figured out the problem: Every time I call it to the database, it actually subverts my guarantees." And he is "Oh, that's great. You have your next paper." And so then, I wrote that next paper, and I think that it was during the process of publishing that next paper that I realized that this was a bigger problem, because I spent something like three, four years convincing people that this should be a paper.
There were a few snags. So one was, most ways of modeling programming languages only model one. So how do you actually model, like, a programming model? So you have lambda calculus, right? But how do you actually model lambda calculus with a SQL database inside? There were actually like, no way... not really many clean ways that I had seen of doing that before. So like, how do you model that in the formal programming languages way? That was like a huge challenge. And then once even we did that, people were like, "Why would you ever do that? You already have your paper about your language. Like, you're done."
And I guess I realized a few things from that. Like, most of the language-modeling community was really not focused on this cross-language thing. Cross-language is super-messy. It took us, like, two years to really figure out how to get the modeling of the language plus the database, right? And I was like, "Man, there's not that many people thinking about this in a principled way and everyone is just doing it."
And so it was... there was the initial realization, and then "Where to best do that work?" was a series of gradual—not even realizations, but trying to do things and getting thwarted and in various ways.
Beyang: So, when you graduated and became a professor, was that kind of thinking top of mind for you? Did that inform the research direction you picked?
Jean: Yeah, absolutely. At the time, I knew that probably at some point, I would want to build the kind of tools I wanted to build in the industry, because I was hitting these edges of what you could do theoretically in academia. But again, at the time—so I was finishing grad school in 2015... at the time, I would say this was before dev-tool startups really took off—the major players were still Microsoft, Facebook, Google, a few other places, and the name of the game at all those places was scale. And so if you had a small PHP or Python extension that could check all of the code at one of these large companies, you could build it. And everything else had a lot of trouble.
Beyang: Got it. Got it. So, seeing that you went into becoming a professor to see if you could drive research...
Jean: Yeah, so I felt like, "The kind of problems I wanna work on, academia is still the best place. I'll take some time here, work out my ideas." I really didn't think it would happen so quickly that the landscape shifted in startups. Yeah, I think it was really like, I feel like 2013, 2014 you started seeing it, but, like, most of the shift happened post-2015.
Beyang: Yeah, I can definitely confirm that. Like, Sourcegraph got started in 2013. And in those days, if you said the words "developer tool" to investors...
Jean: Oh my gosh, yeah!
Beyang: ...they pick up their bags and run the opposite direction!
Jean: It was not hot, yeah.
Beyang: But yeah, fast forward to now, it's one of the hottest areas and for good reason. I feel like a couple big inflection points or milestones, I guess, you know, the GitHub of acquisition by Microsoft, and then you had companies like GitLab and Datadog come along, and just a lot more companies like that, that proved this model. Like, "Hey, more organizations are building software, and guess what? They all care about developer productivity, they all care about tooling." And a lot of these problems are actually, like, common across organizations.
Jean: Yeah. Yeah, absolutely. And I think that something that's been interesting to see is... I keep on calling out Microsoft, because Microsoft has been doing this all along! And I feel like, also something shifted around the time Microsoft actually started doing less of it. So like, a lot of the Microsoft people went to Google, went to Facebook, and, like, it started getting spread around.
Beyang: You know, it's interesting. I think of Microsoft as really the original dev-tooling company.
Jean: Oh, absolutely. Yeah.
Beyang: Like, nineties-era Microsoft, people always think of, like, the antitrust, like competing hard with Apple. But I really think that, the reason that Windows won, a big factor of it in the nineties, was because they built... they prioritized the developer experience. They had this wonderful vertical suite of tools.
Jean: Yep, yep. I completely agree. And they hired a lot of the OG developer-tools people. I think, DevDev at Microsoft still continues to do some of the best developer-experience work.
Beyang: It's kinda interesting too, because I feel like that was maybe the playbook for maybe, like, "Tech version 1". Because in those days, you didn't really have a... like, there was no internet, there was no open ecosystem, and so the way to win was to make your platform the most appealing for third-party developers to build on. That's how you got the kind of coverage of all the applications that your customers wanted to do on top of your operating system. And in some ways, like, it's still true to a certain extent. You do still have vertically-integrated, walled-garden ecosystems in tech today. And that is a priority, to prioritize developer experience for the companies that own those ecosystems. But today we also have the web and the entire open-source ecosystem. And in some sense, like, the developer-tool-startup ecosystem that's popped up in recent years is kinda complimentary to that situation.
Jean: Yeah. Yeah. It's been super interesting to watch. And I think that no dust has settled yet, which makes it very fun. Like, anyone who tries to say the dust is settling... it's just not true.
Beyang: Yeah. We're still early days. It's the wild west. It's really fun outside.
Jean: Yep. Yep. Yep. Yeah. And something that's really cool to me is, I went into this—and I presume you went into this too—because you saw that the developer experience was just so bad.
Beyang: As you were going into starting as a professor at CMU, was this all top of mind or did it inform your research area?
Jean: The ecosystem changes themselves weren't top of mind. I wasn't like, "I wanna capitalize on the ecosystem." To me, I was still trying to get at "Web programming needs to be easier. It needs to be easier for developers to wrangle the mess of code they get. Everything feels tangled up. How can I make it better?" And so to me the question was always, "How can I improve the state of web programming today?" And not so much, "What's the right time to do this kind of thing?" And to me, I thought we were many more years out from me being able to do the kind of work that we're doing at Akita now.
It helped that I realized that what we should really be doing is working on better better tools for telling developers what's going on, instead of better tools for helping developers do something new. I think a lot of the work in academic programming languages, and a lot of the work from people who make programming languages for fun, or work coming out of labs, or these these design collectives... a lot of them about, you know, "Here's a fun way to write more code." But to me throughout the whole time really, I was working on programming tools. The bigger problem is, What do people do with the code they've already written?
And that was a big part of why I was really interested in, "Here's how we add a new dimension to code." Like, my whole PhD thesis was motivated by, "If you've written some code and you wanna add some policy to some data, it is hell right now, because you have to go in and thread all that through. What if you could decouple it?"
The overarching theme for that is, if you have some code and you wanna do anything different to it—whether it's add a new feature, whether it's make a change that threads throughout, whether it's add instrumentation for some kind of monitoring and observability that you need to thread throughout—it is a really big pain right now to do anything with code you have existing. And I feel like that's a big motivation for what you guys at Sourcegraph do, it's a really big motivation for what we at Akita do.
The legacy code problem was a very big one, the heterogeneity of code was a very big one. And so I was just going after, chipping at these problems in ways that I thought were potentially the most medium-term impactful. And what I mean by medium-term impactful is, it was not necessarily, "This is something we can do today." That's luxury of the fact that I was in academia.
To me also, if you're working on developer experience, ten, fifteen years out makes no sense. Like, back before... I feel like, in the seventies, you could be like, theoretically developers will exist one day and I can make all these programming models. And it's great, because now we've picked up a lot of the ideas from the seventies. But everything that's happening now, it's like, what is going to win in the next five years determines what are the big problems to work on in the next ten years. To me, a lot of long-term work in this space didn't make a lot of sense for me for that reason.
Beyang: Makes sense. At what point... was there, like, a catalyst that made you take the leap from academia into starting a startup or was that just a natural progression of your work?
Jean: I looked around, I looked at what was happening in the space, I looked at the needs of people. I feel as an academic program languages person, I was on the very practical end of trying to stay in touch with developers. I would do my tour of Silicon valley companies about once a year and just check things out, figure out what people are up to. There are many more-existential questions you have as an academic working on something that's trying to be useful to the like, to industry because you really... I pretty regularly ask myself is still, "Is this the right thing to be doing? Is this the right place to be doing this thing?" And when I asked myself these questions, I realized the thing I wanna be doing is slightly different and the place to do it is by starting a company.
And I wasn't sure at the time actually. I talked to people at Facebook, I talked to people at Google, and I think that, if there's certain kind of problems you wanna solve and the places that might let you solve them are dynamically changing every year, it's always good to reevaluate.
Beyang: What was the inception of a Akita? Did you have, like, a working group or a brain trust or a co-founder that you started out with?
Jean: It was really just me. I trialed a couple of co-founders and I ultimately decided to take the leap by myself because the couple of co-founders I tried, they were interested in... So initially Akita was much more focused on privacy because a lot of my work was, you know, programming languages applied to privacy. So I was like, "Okay, we can start with observability but for privacy, because that's what my work has been." And so I always had... My intuitive sense was that I wasn't tied to privacy. It was really about certain kinds of programming paradigm shifts. And so the couple of co-founders I initially trialed, they were very tied to privacy and data and I felt, like, "Okay, if we pivot out of that, it's not gonna go very well." And we ultimately did decide to pivot out of that. And a decision I made upfront was like, "They're not committed to the same thing as me." Like, I was very committed to like, "There's a paradigm I wanna push through. There's a way of thinking about software I wanna push through." And I think that, for someone... if they're like, "This is the way to build a business for this kind of data or this kind of privacy," you would just make a different set of choices about what you are and aren't willing to change.
Beyang: Got it. So you didn't wanna restrict yourself to that specific genre of tooling, I guess.
Jean: Right. To me it was about like, "I want people to understand what their software is doing better. Like, whatever insertion we can get a hold of, we take it." And so I think initially, if there had been someone who was like, "I agree with you on, you know, programming can be different in this way,"—and I feel like my founding team eventually was that—but I think initially, it was what I learned from trailing co-founders was like, you have to care about the same thing in the same way.
Beyang: Can you describe for people who aren't familiar with it, what is Akita today? What's the simple introduction?
Jean: So Akita is going after drop-in API observability. So our high-level goal is to help developers build more robust and trustworthy software. We believe the way to do it is by making it as easy as possible for developers to understand their running systems. And what this means to us right now, is their production systems.
I think eventually, there's a whole space. There's what happens in production, what happens in staging, and then, you know, left you shift. But what Akita is today is, we are drop-in, we're eBPF-based, and we're completely based on passive traffic observation. And so all you have to do is drop us in, let us passively watch traffic—we don't stand in the flow of traffic at all. Our footprint is basically the additional memory footprint of our passive watching agent. And we tell developers, "This is what endpoints you have. This is the usage of the endpoints. This is what they're doing. This is how they're talking to each other." And now you might think, "Okay, there's Datadog, there's Grafana, there's all this other stuff, what does Akita do?"
We got into this space because people using these other tools even were like, "It's so hard. Like, we're not really fully using them. It's actually really hard to get this set up." For us to set up APM on our systems, one of our senior—very senior—engineers had to spend a week upgrading our libraries. You had to kinda like figure it out, wasn't even clear, like, "Okay, you have to update this." It was just broken dependencies for a week. And we had three services at the time. This was like, before we even wrote a bunch more code.
So then we were like, "Okay. Yeah." Because I think initially, I had many questions, like, "Where is the highest impact place we could tell developers what's going on with their code?" So we started out thinking, like, maybe people want to know where their data goes. We realized, no, like, people actually don't. The lawyers do, but a spreadsheet is good enough.
And then where we landed is much more basic than what we expected. Developers don't know what endpoints they have. They don't know how those endpoints are being used. If you have something like CloudWatch, even if you have something like Datadog or Grafana, you have to have a pretty good handle on your system to have instrumented it, to set it up in a per-endpoint way.
And so one complaint we see a lot is, if someone's using something like CloudWatch, it's like, "Your whole system is down somewhere, somehow!" What people really wanna know is, "Which endpoint and, like, how bad is it?" And so the abstraction of an API to us is both, like, "We can reconstruct that by watching traffic alone, we don't have to get in your code" and also, it's a very useful abstraction, because we have users come to us and they're like, "Look, I only care about three of my endpoints, but in order to, like, properly instrument those three, like, CloudWatch doesn't make it easier or possible to configure that."
And even if you look at Grafana or Datadog, endpoint isn't really an abstraction necessarily, like, you have to map it all yourself. And so for us, like, we figured out where our insertion is, it's basically people don't want to do very much work and they want you to get in there and just start telling them things. And so for us it's really just been a process of, like, us getting in there and being like, "Hey, like we can tell you these things, which ones do you care about?" And it turned out, you know, the fancy stuff actually was not what got people. It's, like, the much more basic stuff.
Beyang: So, just to restate that real quick, is the picture that you're after really this kinda like boxes-and-arrows picture of, each box is like a process or a service that's deployed and then you show me the endpoints that that one listens on and talks through? And kinda like what other services that each box is communicating with? Is that kinda like the runtime picture that you're after?
Jean: Yeah, that's something we started out being able to show. I will say that what we've gotten requested on is even more zoomed in, because people are like, "I've got one service and no clue what it does." So we were like, "You really don't know that?" And they're like, "Yeah." So we're like, "Here's the whole graph. This is talking to this, here..." And people are like, "No, no, no. I have three, I really only care about one, and really of that, like, four endpoints. But even just, like, getting my head around that is hard."
Beyang: It's like, "Here's this creature. I inherited it from someone else who inherited it from someone else. What does it do?"
Jean: Or, "I inherited from myself. It's evolved over time. People are using it in ways that I didn't expect..." I said on another podcast I was recently on, once you ship code, it's legacy, because it's out of your control and it's not paged in anymore. And, like, that's essentially what we found. Like, as soon as it hits production, it's talking to stuff you didn't know, it's behaving in ways that you're like, "Uh huh."
And so for us, we're just like, "Okay, if we said our mission was help people understand what the heck their production systems are doing and they want this basic stuff, this is how we fulfill our mission." And to me, the vision is eventually, there's so much stuff you can build on top of this. We've talked about this before, the same way Sourcegraph is, like, for any question people have about their code, you wanna be... I think for us, it's like any question people have about their behavior, we wanna be able to give that—with as little work as possible—to the developer.
Beyang: Yeah, that's awesome. I feel like one of the key friction points in development is this, like, translation that has to happen between a static view of the code—which is, like, what you're thinking about when you're editing source—and the dynamic runtime view of the code, like, what's actually instantiated, what's actually running.
Jean: Yeah, and that's been super elusive. And it's like a mountain we had to climb, to figure out our positioning was, most of the observability tools out there and the monitoring tools are very ops-focused, and so for us, we're like, we wanna build a very dev-focused tool here. Because developers need to understand what's going on. So I think a lot of the ops-focused tools, it's like, "We're gonna let you instrument everything you need to instrument, and we're gonna let you do everything." And if you're really building a developer-focused tool for this space, it's really about helping developers get their heads around semantics, which like, none of the monitoring tools do. And for the observability tools, they're kind of dev-focused in a different way. It's like, if you're the dev and you have control of your system, you'll take an observability-minded mindset from the beginning. Then you can get, like, all of the benefits of a Honeycomb or a Datadog or a Grafana.
And for us, we're like, "Let's say you're a different kind of dev. You're what I call a 99% developer. You inherit a system, you work nine to five, you don't live and breathe tools. Like, you don't get your your social cred from knowing about every tool that's on the street. You're not necessarily motivated or able to instrument your code correctly from day one because it's not day one anymore. And you're like, 'I get no joy from learning this tool on the weekend.'" And so, like, what happens to those developers? Because like, right now, for other observability tools, like, those developers are like the black holes of the landscape. You have a few, like, really nicely instrumented services and everything else, just like, "Okay, that was either inherited or that team doesn't care about this." And something we've seen is, like, you'll have a company, you'll have observability evangelists, and people are like, "Yeah, if you adopt all these nice things from day one, then you can have all this! Don't you wanna be in?" And we're just... like, I've seen this play out with programming languages, and if there's no active reason to put in all this work, people aren't going to. And as we're waiting for this to happen, more and more legacy code is piling up that wasn't done this way and inevitably, in one to two years, we're gonna find better ways of doing it. So you're always... like, the legacy problem is even bigger, for if you need to instrument your code or understand it. And so for us, like, our whole thing is just like, "How can we really get in there and just tell people as much as possible?" It doesn't have to be perfect. We're, like, the phrase we say a lot on our team is, "B plus on day-zero solution." Like, it's never going to be your power tool, you're always going to need something better if you want full control or anything like that. Just like if you wanted to program in something beautiful, probably Python is not gonna be it. But does it get the job done? Is it, like, the most popular language today? Yeah.
Beyang: You gotta go with the system that already exists, right? There's no world in which you get to rebuild this huge mountain of code from scratch.
Jean: Yeah. Yeah. So like for us... it's really interesting, because a couple years ago I wrote a blog post that was called "From Balenciaga to Basics", like, how we came from "Do you wanna know precisely where all your data flows?" to "Do you wanna know where your APIs are?" And I feel like every month, our process has been more and more basics. We're like, "Don't you wanna know this?" It's like, "Oh actually, you really wanna know this." And that was much more basic. And I think that's been really good for us. It's been really eye opening as well.
Beyang: You know, it's interesting, it's kinda crazy that we don't have something like this already. If you were to describe, to maybe like in another engineering profession, like, "Hey, the state of the world in code is like, all our running systems, we basically have very little idea of."
Jean: Yeah. Yeah. And I think that other engineering analogy is something I've really been obsessed with for the last ten, fifteen years. There was a professor, Daniel Jackson, I don't know if you encountered him at MIT? But he was doing a lot of lightweight software verification. So like, "How do we prove software systems are correct and not completely soundless?" It's, like, similar to Sourcegraph, but it's if you made lightweight models of your system, and then you tried to ask questions about them. Like, "Is it true that the system will never fall down in this way and that way?" And he gave these incredible talks that talked about engineering analogies. Like, if you're building a bridge, this is how you would measure it, and how you would prevent it from falling down. Or, if you're building a building, here are building codes. And something that always blew my mind was like, there's not the equivalent for software.
And I think, what you all are doing at Sourcegraph is one dimension of it. It's like, "We're gonna get at software by seeing if you can search for certain patterns." I think that a lot of people in my field use that engineering analogy to be like, "Well, this is why we have to model every stick and stone, like, every instruction." Like, the first paper I ever published was called "Safe to the Last Instruction" and we modeled... we built this mini OS kernel that was completely verified. And you know, that was the approach I believed in for a lot of years. But if you think about how engineers do it, they're not, like, measuring every atom. They're not, like, modeling every brick.
Beyang: It's a lot of duct tape.
Jean: Yeah. They, like, put load on it and then they're like, "Does it break? Yes, no." It's very test driven. It's very like, "What did we end up with?" And like, "How do we measure a bridge or how do we measure a building?" is very much about like, "How do we take the artifact that we've built and do stuff to it?" And so this is why I think, what you guys are doing at Sourcegraph, I think things that help you just stress-test production systems—so I think, chaos engineering, all that stuff—that's the first wave of it. I think there's gonna be a lot more of this stuff. I think it's gonna be much more like stress testing and load testing and, like, telling people, "This is what's there." Then, like, "I'm gonna test every property." Not to call out certain kinds of testing that I think are are cool...
Beyang: It's okay. Feel free to be opinionated on this. Opinions are great.
Jean: So I think that a lot of people who are like, "We're gonna measure software, blah, blah, blah" are like, "Property testing! That's the way." But I'm like, "You gotta fricking specify every property." So we have people showing up for our beta and they're like, "We wanna find the properties that property testing miss." And I'm like, "That's gonna be every property, man." Like, no one wanted to write types, you think they're gonna write properties?
So I think that just, like, as much inference as possible for for basically load testing and measuring and getting people's heads around production is where the future is.
Beyang: Awesome.
Jean: Not to discount, like, pre-production! I think, like, pre-production... you guys have it under control. You're locking it down. I think, like, the next frontier is like, what's gonna happen in production?
Beyang: No, I think that there's a really nice complementarity.
Jean: Yeah. Oh, I completely agree. Yeah. Yeah. And I saw you had John Kodumal on your podcast. I think that, like, playing with code search plus feature flagging plus telling people what actually happens... I think the future is, like, that kind of ecosystem. Like, piping things back from production all the way left and then having control over it. That's really, I think, where the future is. And I think that each piece, like, any piece alone, isn't everything. Production without pre-production stuff doesn't exist very well because you can only get so far being, like, that box on the outside. Yeah. So I think that the future is, like, something that has a whole loop.
Beyang: Yeah. I feel like a lot of what you do and what we do and also what LaunchDarkly does, is really just de-obfuscating what's going on in code and making things accessible.
Jean: Yeah. And I think that previously, it was like, "We're just gonna de-obfuscate the next code you're gonna write tomorrow" and what I like about all of our stuff is like, "No, there's a bunch of stuff that already exists. You need to get your head around it. Here's how."
Beyang: Yeah. The system already exists. We gotta run with the things that we have in production today. They are that way for a reason. Right? Like, they've accrued this amount of code because you get edge cases when your thing is in production that you have to handle. And in some sense, it's almost like so much of the software that we build is actually quite simple to explain in layman's terms, because, like, the algorithms and data structures are pretty straightforward. But it's, like, the translation from that high-level, English-level explanation to, "Okay, here's all the source code that wraps around that," or like, "Here's all the..."
Jean: Yeah... "Here's the source code. Here's what it interacts with. Here's how it's layered over time..." And I think the piece, like, the thing I really took away from my PhD was like, add any dimension to that code—like, imagine adding new permissions to old code. Like, you have to touch everything, displace all the semantics...
And then I realize, like, imagine adding anything to old code that's a cross-cutting concern. It's the same game. And you're gonna have all these questions, you can't really get your head around it. But yeah, I think that right now, like, code is seen as, like what you said, it's a very static thing. But we need ways to engage with code that has been written.
So an analogy I really liked from when I was working more on privacy was, people would say, "Data is like a corrosive acid." It's just sitting there waiting to explode or, like, it's degrading. And I feel like code that's been written in the past that you haven't opened in a while, that's, like, a bigger corrosive acid, because it's not going away. It's like, you can't rip it out. Like data, you can rip it out and maybe you lose some users and they're unhappy, but you rip out old code and your cycle's down. And I think that it took people a lot of years to realize data is a corrosive acid, and I think people are finally starting to realize code is a bigger corrosive acid. And we just haven't really been thinking about it that way.
Beyang: Yeah. Bit rot is real.
Jean: Yeah. And you can't just excise the rot. Like, you have to live with it. So what are you gonna do?
Beyang: It's a core dependency.
Jean: Yeah. Yeah. Because I think there's a lot of optimism. Like, I think that software engineering has really shifted to be very far from programming, where people were like, "Oh yeah, code is just code. We can always rewrite it. Like blah, blah, blah, blah." Like, software artifacts are completely different things and I think that only in the last five years have I really seen the industry really think about it as a whole, that way. And I feel like we're still very early days in that.
Beyang: As far as the upcoming roadmap for Akita looks, what are some of the exciting features or things on the horizon that you're really looking forward to?
Jean: Yeah, for us, it's doubling down on building for small to medium 99% developers. And so what we found is that these are the people that we're resonating very well with right now. I think for a while, like startups play the game, "What size do we really focus on?" And I think that, like, for a while, we were like, "Okay, we're gonna, like, work on a lot." But then we're like, "If we do bigger companies, actually there's, like, major other scaling issues because we're processing API traffic in real time."
And so we're doubling down on serving the needs of small to medium 99% developers. And so that means building a lot of the features they've been requesting, so alerts is some... so, like, right now, you can drop us in, you can see metrics and errors, you can see your API endpoints, you can see usage, you can sort by latency P99.
So then people are like, "We want alerts on top of that. We want other things." And so we're really excited to build that. And the guiding philosophy of our staff is like, as little programmer intervention as possible—that's, like, that's our audience. And we're... like previously, our beta was also not accessible to a lot of people because we were, like, just getting real-time, like, we got real-time in Q2, which took a while. And then we're getting real-time for more than a handful of users was also, like, a thing that we did in Q2. And so now we're finally opening up our beta a little more, getting our onboarding more smooth, like, that's what we're shipping this next month. And so we're really excited just to get more users onto our platform, like banging on the rough edges. And so we've been starting to ramp that up more.
Beyang: Cool. Awesome. If folks are listening to this and want to give it a spin or try it out, where should they go?
Jean: So Akita software.com. We have a "join our beta", yeah, we have a "join our beta" button. Just sign up. I think, especially if you join with your work email, you're more likely to get in. Because for us, like, the use case we're really going hard after is, like, IC developer in a small-to-medium sized company or team, so it's, like, a company without a ton of DevOps resources, it's a developer who wants observability on their own systems. And so if you sign up with your work email or your real name, even—I usually go and look up the people, because our beta still has small enough capacity, I guess, I have enough time on our hands that I'm like, "Okay, we really wanna be making sure this is a good match." So yeah, if you sign up, I'll look you up and if it looks like your use case might be a good fit, write to me if I miss you. So there are people who have written to me and they're like, "Hey, here's my use case," or like, "I'm working on this for this project. Here's what it is." Anyone with a use case that sounds like we can help with and would be a good test for our system, I try to let in.
Beyang: Awesome. And you're pretty active on Twitter, right?
Jean: Yeah. I'm pretty active on Twitter. I don't see everything that comes my way, necessarily.
Beyang: Okay. Fair enough. So that's not the best channel. Just email you then, or...
Jean: Yeah, if you tweet @AkitaSoftware—and the Akita Software DMs are actually open. If you tweet at me I am so active that I don't see everything—but if you tweet at Akita Software, that's the sweet spot, I think.
Beyang: Okay,cool. Pro tip right there. All right, Jean, thanks so much for taking the time. This is an awesome conversation. Thanks for sharing your journey as a programmer and also telling us more about Akita. It's a really cool...
Jean: Yeah, thanks so much for having me on. This was super-fun. And I think that, you know, like, the Sourcegraph part and the Akita part and a few other parts, that's like the next, the next era of dev tools is really facing reality, getting people's heads around code that exists. So I'm really excited about that.
Beyang: Awesome. As am I.