How LZW Patents Led to gzip, and the Web to Brotli

gzip and Brotli are often presented as an old algorithm and its replacement, reducing the comparison to a single savings percentage. That misses the reason each format exists. gzip emerged as a practical answer to the patent restrictions surrounding the Unix compress utility, whereas Brotli was designed to represent Internet data more densely. Both use ideas from the Lempel-Ziv family, so Brotli extends an existing line of compression research rather than starting over.

Why GNU needed a replacement for compress

The Unix compress utility was widely used in the 1980s and relied on LZW. Unisys and IBM filed related patent applications in 1983, Terry Welch published his description of the method in 1984, and Welch's US patent was issued in December 1985. For a free software project, this was not merely a theoretical legal concern. A compatible compressor could not be distributed confidently without dealing with licensing.

The GNU project explicitly identifies the Unisys and IBM patents as the reason it developed a replacement. Jean-loup Gailly wrote the original gzip program, and Mark Adler supplied its decompression code. By 1992, gzip had joined the GNU toolset and produced a new format rather than an LZW-compatible stream, while retaining the ability to decode older compress files because the relevant patent claims treated encoding and decoding differently.

The word zip in the name does not make gzip a kind of ZIP archive. PKWARE introduced ZIP in 1989 as a container that can collect multiple files with a directory and metadata. gzip normally wraps one data stream, which is why Unix users commonly combine a directory with tar before compressing the resulting stream.

The connection between the formats is DEFLATE. According to the gzip historical notes, the first version of the method appeared in Phil Katz's publicly released ZIP 0.9 in July 1991 and later became part of the ZIP 2.0 line. gzip used the same compressed stream format inside a different wrapper. In May 1996, RFC 1951 documented DEFLATE and RFC 1952 documented gzip. The RFCs standardized formats that were already in use: RFC 1951 says version 1.3 converted an existing specification to RFC style and credits Phil Katz with designing the DEFLATE format.

How DEFLATE turns repetition into references

DEFLATE reads its input as bytes and looks for sequences that already occurred in the preceding 32 KiB. Instead of writing the sequence again, it can store a pair containing the match length and a backward distance. If background-color occurs repeatedly in a CSS file, a later instance may become a short reference to earlier bytes.

Literal bytes, lengths, and distances are then represented with Huffman trees. Frequent values receive shorter bit codes, while rarer values receive longer ones. A compressor can spend more time searching for matches to obtain a smaller result or stop earlier in favor of speed. The stream format stays the same because the RFC defines what a decoder must accept, not one mandatory strategy for producing it.

gzip places a header and trailer around DEFLATE, including a CRC32 and the original stream size modulo 2³². Raw DEFLATE, zlib-wrapped DEFLATE, and gzip are therefore not interchangeable byte strings even when they carry data compressed by the same method. Each has a different wrapper, and a decoder needs to know which representation it received.

Brotli develops a familiar idea

Google introduced Brotli in late 2013, and the format was published as RFC 7932 in July 2016. Its authors built a general-purpose compressor with particular attention to the density of Internet data and the resources needed for decoding, as the team's Brotli research paper explains.

Brotli also replaces repeated sequences with backward references and uses Huffman codes, but it adds a more flexible window, context modeling, and a built-in dictionary of 122,784 bytes. The dictionary contains words and fragments from natural languages and program text. A set of 121 transformations can add prefixes and suffixes or alter capitalization. A compressed stream can refer to one of these variants near the beginning of a small document, before the input has built much repetition of its own.

This is why Brotli is often especially effective on HTML, CSS, JavaScript, JSON, and other text, but the dictionary does not guarantee a win for every file. A long document with many internal repetitions can compress well in either format, a short random byte sequence may grow because of format overhead, and JPEG, WebP, MP4, or archive files normally already use specialized compression. A universal claim of 20 or 30 percent improvement has no meaning without a corpus, implementation, and quality setting.

Transfer size is only one web cost

A browser advertises acceptable representations through Accept-Encoding, and the server identifies its selection with Content-Encoding. Both gzip and br appear in the current IANA content-coding registry, allowing a server to keep several versions of the same resource and choose one for a particular client.

For static files, expensive encoding is often acceptable because CSS or JavaScript can be compressed once during a build and served many times. Dynamic HTML or JSON may be encoded during each request, so transfer savings compete with CPU time and response latency. A higher quality level asks an encoder to search longer for a dense representation. The official Brotli implementation exposes quality values from 0 through 11, and higher quality generally means slower compression.

A practical configuration is therefore more nuanced than always choosing Brotli. High-quality precompression is worth evaluating for static text assets, dynamic responses need a level selected from measurements, and gzip remains a compatible alternative. The useful comparison is the size and encoding time of a site's actual files, not the names of the formats.

What GZip & Brotli Tool shows

GZip & Brotli Tool accepts UTF-8 text and compresses it locally as gzip, zlib-wrapped DEFLATE, raw DEFLATE, or Brotli. The first three use the browser's CompressionStream; Brotli uses a locally vendored WebAssembly build. The current tool runs Brotli at quality 11 without a level selector, while the browser controls the native DEFLATE parameters.

The result can be displayed as Base64 or Hex, but these are text encodings of the finished bytes rather than another form of compression. The statistics compare the UTF-8 input with the raw compressed stream before Base64 or Hex conversion, so the copied string is longer than the displayed compressed byte count. A small input can also produce a compressed stream larger than the original, which the tool reports as a positive change.

Decompression can use an explicit container and encoding or attempt automatic detection. gzip has a recognizable signature and zlib has a header, while raw DEFLATE and Brotli have no universal magic number. Auto-detection must therefore try decoders and remains heuristic; selecting the format and Base64 or Hex explicitly is safer for ambiguous input.

The tool is useful for learning the formats, inspecting small text payloads, and comparing one piece of input, but it is not a server benchmark. All processing stays in browser memory and no data is uploaded. Production settings still need measurements on representative assets at comparable quality levels.

Related tools

GZip & Brotli ToolCompress text into GZip or Deflate Base64 strings, or decompress encoded payloads back to text in your browser.Encryption ToolboxIdentify an unknown cipher from its charset, index of coincidence, frequencies and Kasiski repeats; encode one message with every reversible cipher at once with round-trip checks; encrypt with AES-256-GCM; and solve Caesar, Vigenère, Playfair, ADFGX, Bifid or rail fence in the workbench — all in the browser.Hash GeneratorCompute MD5, SHA-1, SHA-256/384/512, SHA-3 and CRC32 digests from text or files, live as you type, with HMAC keyed hashing and hash comparison.JSON ToolboxValidate syntax, format or minify payloads, and browse nested API responses and configuration files as a collapsible tree.