Important
For whole buffered data (i.e. it fits into memory) use 4kills/go-libdeflate!
4kills/go-libdeflate is much faster (at least 3 times) and completely compatible with zlib!
With that said, if you need to stream large data from disk, you may continue with this (go-zlib) library.
This ultra fast Go zlib library wraps the original zlib library written in C by Jean-loup Gailly and Mark Adler using cgo.
It offers considerable performance benefits compared to the standard Go zlib library, as the benchmarks show.
This library is designed to be completely and easily interchangeable with the Go standard zlib library. You won't have to rewrite or modify a single line of code! Checking if this library works for you is as easy as changing imports!
This library also offers fast convenience methods that can be used as a clean, alternative interface to that provided by the Go standard library. (See usage).
- zlib compression / decompression
- A variety of different
compression strategies
andcompression levels
to choose from - Seamless interchangeability with the Go standard zlib library
- Alternative, super fast convenience methods for compression / decompression
- Benchmarks with comparisons to the Go standard zlib library
- Custom, user-defined dictionaries
- More customizable memory management
- Support streaming of data to compress/decompress data.
- Out-of-the-box support for amd64 Linux, Windows, MacOS
- Support for most common architecture/os combinations (see Installation for a particular OS and Architecture)
For the library to work, you need cgo, zlib (which is used by this library under the hood), and pkg-config (linker):
Install cgo
TL;DR: Get cgo working.
In order to use this library with your Go source code, you must be able to use the Go tool cgo, which, in turn, requires a GCC compiler.
If you are on Linux, there is a good chance you already have GCC installed, otherwise just get it with your favorite package manager.
If you are on MacOS, Xcode - for instance - supplies the required tools.
If you are on Windows, you will need to install GCC. I can recommend tdm-gcc which is based off of MinGW. Please note that cgo requires the 64-bit version (as stated here).
For any other the procedure should be about the same. Just google.
Install pkg-config and zlib
This SDK uses zlib under the hood. For the SDK to work, you need to install zlib
on your system which is super easy!
Additionally we require pkg-config which facilitates linking zlib
with this (cgo) SDK.
How exactly you install these two packages depends on your operating system.
brew install zlib
brew install pkg-config
Use the package manager available on your distro to install the required packages.
Here, you can either use WSL2 or MinGW and from there install the required packages.
To get the most recent stable version of this library just type:
$ go get github.com/4kills/go-zlib
You may also use Go modules (available since Go 1.11) to get the version of a specific branch or tag if you want to try out or use experimental features. However, beware that these versions are not necessarily guaranteed to be stable or thoroughly tested.
This library is designed in a way to make it easy to swap it out for the Go standard zlib library. Therefore, you should only need to change imports and not a single line of your written code.
Just remove:
import compress/zlib
and use instead:
import "github.com/4kills/go-zlib"
If there are any problems with your existing code after this step, please let me know.
This library can be used exactly like the go standard zlib library but it also adds additional methods to make your life easier.
var b bytes.Buffer // use any writer
w := zlib.NewWriter(&b) // create a new zlib.Writer, compressing to b
w.Write([]byte("uncompressed")) // put in any data as []byte
w.Close() // don't forget to close this
w := zlib.NewWriter(nil) // requires no writer if WriteBuffer is used
defer w.Close() // always close when you are done with it
c, _ := w.WriteBuffer([]byte("uncompressed"), nil) // compresses input & returns compressed []byte
b := bytes.NewBuffer(compressed) // reader with compressed data
r, err := zlib.NewReader(&b) // create a new zlib.Reader, decompressing from b
defer r.Close() // don't forget to close this either
io.Copy(os.Stdout, r) // read all the decompressed data and write it somewhere
// or:
// r.Read(someBuffer) // or use read yourself
r := zlib.NewReader(nil) // requires no reader if ReadBuffer is used
defer r.Close() // always close or bad things will happen
_, dc, _ := r.ReadBuffer(compressed, nil) // decompresses input & returns decompressed []byte
-
Do NOT use the