September 15th, 2024

Compression Dictionary Transport

The draft-ietf-httpbis-compression-dictionary-19 proposes a dictionary-based compression mechanism for HTTP, enhancing performance and reducing bandwidth, with guidelines for usage and negotiation, currently under evaluation for standardization.

Read original articleLink Icon
Compression Dictionary Transport

The document draft-ietf-httpbis-compression-dictionary-19 outlines a mechanism for dictionary-based compression in the Hypertext Transfer Protocol (HTTP). This approach allows clients and servers to reduce the size of transmitted data, enhancing performance and minimizing bandwidth usage. The draft extends existing HTTP compression methods and provides guidelines for the delivery and utilization of compression dictionaries. It specifies how designated HTTP responses can serve as external dictionaries for future responses, particularly for compression schemes like Brotli and Zstandard. The document also details the HTTP headers necessary for negotiating dictionary usage and registers content encoding values for these compression methods. Two primary use cases are highlighted: using a previous version of a resource as a dictionary for a newer version, and referencing an external dictionary for common content patterns across multiple resources. The draft is currently in the IESG evaluation stage and is intended for publication as a Proposed Standard.

- The draft specifies a mechanism for dictionary-based compression in HTTP.

- It aims to improve performance and reduce bandwidth consumption.

- It extends existing HTTP compression methods and provides negotiation guidelines.

- Two main use cases include version upgrades and common content compression.

- The document is under evaluation for publication as a Proposed Standard.

Link Icon 8 comments
By @freeqaz - 2 months
This is the link you want if you want to read the actual meaty contents of this page. Took me a sec to find. https://datatracker.ietf.org/doc/html/draft-ietf-httpbis-com...
By @bcoates - 2 months
Any evidence this actually causes material performance improvement?

Pre-shared compression dictionaries are rarely seen in the wild because they rarely provide meaningful benefit, particularly in the cases you care about most.

By @dexterdog - 2 months
Why can't we implement caching on the integrity sha of the library so they can be shared across sites? Sure there is technically an attack vector there but that is pretty easily scannable for tampering by verifying with trusted CDNs.

With something like that you could preload all of the most common libraries in the background.

By @kevincox - 2 months
It is interesting that a proxy won't be able to see the complete response anymore. It will see the dictionary ID and hash but without a copy the server's response won't be fully intelligible to it.
By @hedora - 2 months
I’m guessing the primary use case for this will be setting cookies. (Google fonts or whatever sends different dictionaries to different clients, the decompression result is unique per client).

I wonder what percentage of http traffic is redundant compression dictionaries. How much could this actually help in theory?

By @wmf - 2 months
Interesting that they removed SDCH in 2017 and now they're adding it back. Let's hope this version sticks.
By @londons_explore - 2 months
This is going to go the same way as HTTP/2 Server Push (removed from Chrome in 2022).

Great tech promising amazing efficiencies and speedups, but programmers are too lazy to use them correctly, so they see barely any use, and the few places it is used correctly are overshadowed by the few places it is used wrongly hurting performance.

Lesson: Tech that leads to the same page loading slightly faster generally won't be used unless they are fully automatic and enabled by default.

Http push required extra config on the server to decide what to push. This requires extra headers to determine what content to compress against (and the server to store that old/common content).

Neither will succeed because the vast majority of developers don't care about that last little bit of loading speed.