3.2 retrospective

“Regardless of what we discover, we understand and truly believe that everyone did the best job they could, given what they knew at the time, their skills and abilities, the resources available, and the situation at hand.” –Norm Kerth, Project Retrospectives: A Handbook for Team Review

Review previous retrospective https://github.com/sourcegraph/sourcegraph/blob/master/doc/dev/retrospectives/3_0.md

Action items

  • Nick make improvements to documented retrospective process

    • Remove survey and directly submit feedback to a Google Doc.
    • Ensure release checklist gives enough time to submit feedback.
    • Remove topic grouping.
    • Add guidance to favor discussing more topics.
    • Be more strict about time limits. Presenter has a visual indicator that everyone can see for remaining time.
  • Testing

    • Chris is working to make e2e tests more reliable. The team is thankful for his work enabling e2e tests again.
    • Chris will add docs and do a breakfast and learn based on docs.
    • Issac to determine if Search needs a form of testing that is not satisfied by e2e tests or unit tests.
  • User flows

    • Quinn to figure out how to document user flows and owners of those user flows better.
    • Stephen update PR template to remind devs to update docs.
  • Planning

    • Ryan starts compiling the blog post at the beginning of the release cycle to understand what the problem statements and user stories are.
    • Nick document agency, ownership, expectations of engineering owners
    • Nick create checklist that product delivers to eng. Eng responsibility is to verify all requirements are met.
      • Core requirements for project to be “completed”
      • Wireframes

Feedback

Testing

  • (7 votes) (15 minutes) It feels like we don’t yet have the reliable e2e testing infrastructure that we need. (Loic +1, Tomás +1, Geoffrey +2, Nick +1, Farhan +1, Chris +1)
  • (5 votes) It would be helpful to do a lunch and learn (anchored by documentation) about how to write e2e tests / common pitfalls.(ryan +1, Beyang +2, Farhan +1, Francis +1)
  • (1 vote) I (Chris) spent half of my iteration on tests+CI and broke a lot of my teammates’ builds, but neither e2e tests nor Percy have caught any real bugs yet (Chris +1)
  • The 3.1 and 3.2 releases have been smooth. The process is definitely improved, but we have also shipped fewer features. Will releases with more features go as smoothly?

Planning

  • (9 votes) (25 minutes) I continue to think and see examples of where we as a team need to holistically look at our product’s end-to-end experience and make sure things work from that perspective. Fixing our individual issues is not enough. (Tomás +1, Stephen+2, Loic +1, Farhan +1, Nick +1, Geoffrey + 1, Ryan +1, Francis +1)
  • (7 votes) (25 minutes) I would prefer planning product specification and design details to occur upfront. We can continue to remove friction during the development phase by deciding on these details as much as possible early on. The quicker we can get something out, the more potential we have to do UX testing, gather feedback, and iterate.” (Farhan +1, Stephen+1, Ryan +1, Loic +1, Tomás +1, Nick +1, Geoffrey +1)
  • (4 votes) There was low priority work done in this release that didn’t align with the goals of fixing things that can be improved. e.g theme switcher system property that doesn’t work in Chrome or Firefox. (Chris +2, Ryan+1, Beyang +1)
  • (1 vote) Although I did much better with this release, I still did not finish all of my issues for 3.2 and over-committed with what I could get done. I want to get better at only committing to things I am positive I will have time for. (Farhan +1)
  • I’m disappointed that the docs UI changes didn’t make it in. We need to discuss a process around designs coming first, feedback and revision, then implementation. I think that would’ve saved a lot of time.
  • (1 vote) LOVING THE FIXED RELEASE PROCESS. It really helps with planning, knowing that there is hard deadline and that work must be structured in a way that either has to be flagged off/hidden, or shippable minus some features that are not critical. (Beyang +1)
  • Getting better at getting the necessary input from design and code reviewers to ship the main features.
  • Didn’t finish any of the auth-related issues, need to do better about triaging those properly.
  • I liked that I was able to work on tech debt, refactoring, and testing. I fixed a bunch of long-standing bugs, and it felt great to get them out of the way without feeling like I was working on the “wrong” thing.
  • I wish I had planned my refactoring and testing work better. I didn’t set solid targets for things to accomplish, so I didn’t hold myself accountable for shipping the refactors and tests.
  • I want to get better at sharing progress, not waiting until I think it’s perfect.

Release process

  • (6 votes) I think we need to be more mindful of our users when considering merging a change. If a change is agreed to already be a good improvement over the status quo and solves issues that users are facing, we should be certain that change lands in the next release and follow up on any further suggestions in subsequent PRs and releases.” (Nick +1, Loic +1, Chris +1, Stephen +2, Tomás +1)
  • (3 votes) We should have a deadline for opening up PRs that will make it into the release branch(Francis +1, Loic +1, Ryan +1)
  • (1 vote) Any dev rel activity, e.g. docs issues tied to a release must also adhere to being code complete by the 15th and only vital commits cherry picked (this should be the rare exception) (Nick +1)
  • (1 vote) The hardest part about being release captain is advising developers that their feature is not going to ship because it isn’t ready. (Francis +1)
  • It was calm for me. No last minute fires to put off. I’m happy. I learned that we’re serious about stability and sustainability.
  • I need to prepare earlier for developer relations tasks and create a developer relations specific release checklist.

Customer request prioritization

  • (4 votes) There was a customer request that came in towards the end of the release cycle. It was important / high-impact enough that it preempted my other tasks, but I didn’t clarify how “complete” the solution needed to be. As a result, I spend 3x more time that I needed to. If I had clarified the scope earlier, I could have avoided doing that extra research until we had more signal that we should continue working on this. (Tomás +1, Geoffrey +1, Francis +1, Beyang +1)
  • Did a better job of prioritizing customer requests appropriately

Other

  • I’m happy with the quality of the first Terraform plan for AWS on EC2. There are several design patterns that can be re-used across all cloud vendors
  • Loved hearing about the success with Netflix Chaos Monkey and the encouraging numbers regarding customer instances and DAUs stemming from that