Protocol buffers are Google's language-neutral, platform-neutral, extensible mechanism for serializing structured data – think XML, but smaller, faster, and simpler. You define how you want your data to be structured once, then you can use special generated source code to easily write and read your structured data to and from a variety of data streams and using a variety of languages.
IMPORTANT NOTE: As of Aug 2015, Protocol Buffers v3 is in ALPHA release, so we will avoid it. Go is supported in proto3, but the PB for Go package definitely is for proto2, so we will be using that version.
proto2 vs proto3
- A practical guide to protocol buffers in Go - good overview of PB's for Go.
- Getting Started With Go and Protocol Buffers
- golang / protobuf Repository
- Protocol Buffer Basics: Python
- Python Generated Code
- creating a python dictionary like object from protocol buffers for use in pandas
Apparently it is generally not necessary to version the packages. According to this StackOverflow post, Protocol Buffers are quite forgiving about versioning:
Google designed protobuf to be pretty forgiving with versioning:
unexpected data is either stored as "extensions" (making it round-trip safe), or silently dropped, depending on the implementation new fields are generally added as "optional", meaning that old data can be loaded successfully however:
- do not renumber fields - that would break existing data
- you should not normally change the way any given field is stored (i.e. from a fixed-with 32-bit int to a "varint")
Generally speaking, though - it will just work, and you don't need to worry much about versioning.