By Sam Boyer on July 14, 2017
Liveblog by Alan Shreve (@inconshreveable)
Update: Slides for this talk have been posted here.
Presentation by Sam Boyer @sdboyer.
Sam fell into software engineering by accident, but decided to stick around. He loves complex, ecosystem-class problems, cares deeply about building healhty communities, and is always looking for ways to bring technology's enormous potential to bear on the world's critical problems. Nowadays, he's a Site Reliability Engineer at VividCortex.
The new era of package management - we've all been waiting for it. We're on the road towards official tooling for package management. It's going to be a real sea change for Go development.
Let's all be on the same page about what package management is:
Step 1. Write Code
Step 2. Pull in other people's code
Step 3. PAIN
Package management is primarily about:
Package and dependency management is technically different, but we'll talk about them interchangeably here.
Package management: we don't want to deal with it, not fun.
Package management is more than just tech. We're talking about building tools that interact with other developers. Not just a technical problem. Tools -> Social Systems.
Just like how telephones have changed our social systems, dependency management systems change our social systems. Dependency management makes boring social systems but they are still important because they are the mediators of coordination between people writing software.
Analogy with pidgin languages: A pidgin is a new language that emerges when two unintelligible languages meet. They usually emerged around trade routes. No one speaks them natively (pidgin language by definition). Pidgins are not a great tool for culture. Dependency management in Go was like a pidgin language. There was no official tooling, similar to the lack of official language when pidgins emerged. Two parties trying to coordinate dependencies used or built whatever tools best suited them, but there was no overall community buy-in.
Community buy-in and a standard tool is exciting because the quality of software will be better with an official coordination tool or 'language.' We're looking forward to more than a new era of package management, we're looking forward to a new era of Go.
Let's talk about the fundamental problems in Go package management.
The Go community has worked around it with discipline:
Vendor is not without its own problems:
These problems became very prominent when vendor was enabled by default.
Too often, when trying to create a unifying standard, we just create another competing standard. It's a testament to the awesome Go community that we are not creating a 15th standard, but one that we will be adopting together.
So we started an initiative to put together a group of Go community leaders to work on package management. The task was to take community feedback, write design docs, and understand competing concerns with respect to package management.
This is the result. We call it 'The Official Experiment'. Those words are carefully chosen.
There was a huge amount of community participation in the mailing list while it was developed.
ensure command keeps state in sync. ensure is about making sure the states are in sync according to the defined rules. The whole dep tool is designed around state resolution instead of requiring the user to issue many imperative commands. This means it is a very opinionated tool. So much so that we have our own mascot.
The ensure model is leaking into other communities' dep tools (like npm).
dep is at the center of where we are right now, but it's not the end goal. the end goal is to move this functionality into the official toolchain. Everything is being designed with the intent that it we would like it to eventually become part of the Go toolchain.
In this way dep is a stepping stone. This is the first time we're looking at eventually bringing a huge amount of third-party community code into the official toolchain. This is uncharted territory and still a very big experiment.
One key insight from dep: what we're discussing is a third space beyond vendor & GOPATH. A third space would have a versioned set of packages on disk.
The MIT license: "FITNESS FOR A PARTICULAR PURPOSE" is not guarantee. we're here because that's true. That's true of all code and its an inherent complexity in working in software. dep management can't remove that problem, but it matters because it can make that complexity assailable. The dep complexity burden is something that good tools can help you tame.
dep lives up to theatgoal of managing the complexity.
Huge thank you's to all the maintainers and contributors to all Go dependency management tools. And thank you to the Go team for their trust and especially to the community for helping make this project a reality.