Thunder Talks

The thunder talks are the second round of lightning talks from many speakers.

Note: This post was live-blogged at dotGo 2017. Let us know on Twitter (@sourcegraph) if we missed anything. All content is from the talk; any mistakes or misrepresentations are our fault, not the speaker's.

gocv.io

Ron Evans

• Creator of gobot.io
• New project: gocv.io OpenCV in Go!
• CV == Computer Vision
• Go -> CGO -> C -> C++
• Linux, Darwin and Windows.
• Live demo of face tracking.
• Live demo of rotating through several video filters with the webcam facing the audience.

Using Go on Android and Python

Laurent Leveque

• From Go to Android - gomobile bind
  • Major downside is it only supports one type byte[]
• From Go to Python - go build buildmode=c-shared
• Will have to write C code to manage memory. This is tricky and repetitive, so you can just write it once and use go to generate the glue code.
  • Annotate your API methods
  • Write your own code generator: go/parser and go/ast to naviage your Go code programmatically
  • Construct your output with text/template
  • Use //go:generate to trigger your code generator
• You can easily combine this by serializing your inputs and outputs with Protobuf
  • No more limitation with Gomobile (only byte[])
  • Less C code (only one type to handle)
  • Protobuf is cross language.
  • You can also generate API stubs.
• Protobuf:
  • protoc --go_out=plugins=myStubGenerator:$(pwd) *.proto
  • protoc-gen-go is written in Go and has a plugin system.
• Conclusion:
  • Protobuf tools simplify cross-platform lib dev.
  • Go tools help to generate all the wrapping code.
  • This approach works with other languages (with C bindings)
• Example by speaker goprotopy

Go in Data Science

Diana Ortega

• Backend engineer working with data scientists
• Deep learning project - A lot of data from different sources and different types -> magic -> graphs
• Magic is dev and data scientists which massage and aggregate the data.
• Normalize data - normally use a pipeline, for example pachyderm. pachyderm is written in go but is language agnostic since the pipelines run on k8s and docker.
• notebook: gophernotes useful for testing hypothesis.
• gonum: libraries for matrices and linear algebra, statistics, prob distributions, etc
• For more information about go + datascience visit gopherdata.io.

Go for real time streaming architectures

Mickael Remond

• What are straming archs? Use events rather than shared states.
• Streaming arch is often represented as pipelines.
• Also called reactive architectures
• Microservices are often using streaming, since it can be used as a glue between independent services.
• Central Realtime Engine, often Kafka
• Processing Frameworks: Streaming engines spark streaming, flink, kafka streams, akka streams.
• App --events-> Kafka --> Processing Framework
• Cloud-native way:
  • Use go services using sarama, Sarama cluster, and protobuf. Also can use Nats instead of Kafka (written in Go).
  • Packaging you can use Docker/k8s. Then you can scale horizontally by just adjusting the replica count of a deployment.
• Benefits
  • Simple/manageable code
  • Performance
  • Scalability
  • Workers themselves can also use Go concurrency for further performance

Rethinking errors in Go

Marcel van Lohuizen

func writeToGS(ctx context.Context, bucket, dst, src string) (err error) {
    client, err := storage.NewClient(ctx)
    if err != nil {
        return err
    }
    defer client.Close()
 
    w := client.Bucket(bucket).Object(dst).NewWriter(ctx)
    defer w.CloseWithError(err)
 
    _, err = io.Copy(w, r)
    return err
}

The above code handles some errors, but is not fully correct:

• It needs to handle panics
• It needs to return the error from Close

Putting it all together the code becomes very complicated:

func writeToGS(ctx context.Context, bucket, dst string, r io.Reader) (err error) {
    client, err := storage.NewClient(ctx)
    if err != nil {
        return err
    }
    defer client.Close()
 
    w := client.Bucket(bucket).Object(dst).NewWriter(ctx)
    err = errPanicking
    defer func() {
        if err != nil {
            _ = w.CloseWithError(err)
        } else {
            err = w.Close()
        }
    }
    _, err = io.Copy(w, r) {
    return err
}

How to simplify? Semantics first. An error is recoverable, a panic is not (sort of). Marcel created a burner package github.com/mpvl/errc which unifies error panic and defer. It funnel all errors, including panics, to a single error variable:

func writeToGS(ctx context.Context, bucket, dst, src string) (err error) {
    e := errc.Catch(&err)
    defer e.Handle()
 
    client, err := storage.NewClient(ctx)
    e.Must(err)
    e.Defer(client.Close, errc.Discard)
 
    w := client.Bucket(bucket).Object(dst).NewWriter(ctx)
    e.Defer(w.CloseWithError)
 
    _, err = io.Copy(w, r)
    e.Must(err)
}