From 9d3efb6bd1465735e9b23ef2ee5752a26566262e Mon Sep 17 00:00:00 2001 From: Claude Date: Mon, 22 Jun 2026 19:50:37 +0000 Subject: [PATCH 01/10] Make payload compression a per-hop, per-frame decision Rework compression in both moq-lite and the MoQ Payload Compression Extension so the publisher only signals *whether* a track is worth compressing, while the algorithm is negotiated per hop and recorded per frame/object. A single frame can opt out (e.g. a small JSON merge-patch delta that DEFLATE would enlarge), and different hops can use different algorithms. moq-lite: - Publisher Compression in TRACK_INFO becomes a boolean hint (reserved values >1 are treated as 1). - New SETUP Compression parameter: each endpoint advertises the algorithms it can decompress, negotiated per hop. - New per-frame Compression field in FRAME and datagram bodies, present only when the hint is set, naming the algorithm used (none/deflate). - New Compression section defining the algorithm IDs (shared with the extension) and relay behavior, plus compression security notes. moq-compression: - COMPRESSION track property becomes a boolean hint. - Algorithms are advertised in preference order; the hop default is the receiver's most-preferred mutually-supported algorithm. - New COMPRESSION_ALGORITHM object property overrides the hop default per object (typically none, to opt out), keeping the common case free of per-object signaling. - Relay, security, and IANA updated (new Object-scope property). Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_01W8bLV6vHzucLNvDhPk3bMP --- draft-lcurley-moq-compression.md | 84 ++++++++++++++++++---------- draft-lcurley-moq-lite.md | 96 +++++++++++++++++++++++--------- 2 files changed, 124 insertions(+), 56 deletions(-) diff --git a/draft-lcurley-moq-compression.md b/draft-lcurley-moq-compression.md index 13f7302..ce59642 100644 --- a/draft-lcurley-moq-compression.md +++ b/draft-lcurley-moq-compression.md @@ -25,9 +25,9 @@ informative: --- abstract This document defines a payload compression extension for MoQ Transport {{moqt}}. -A track-level Compression property lets the original publisher signal that a track's object payloads are worth compressing, and with which algorithm. -Compression is then applied independently on each hop: a payload is compressed only on a hop that has negotiated the extension and whose receiver supports the algorithm, and is sent verbatim otherwise. -Each object is compressed independently so objects remain individually decodable, and the decompressed bytes — the actual object — are unchanged end to end. +A track-level Compression property is a boolean hint by which the original publisher signals that a track's object payloads are worth compressing. +The algorithm is negotiated independently on each hop, and compression is applied per hop: an object is compressed only on a hop that has negotiated the extension and a shared algorithm, and is sent verbatim otherwise. +Each object is compressed independently so objects remain individually decodable, an object can opt out when compression would not help, and the decompressed bytes — the actual object — are unchanged end to end. --- middle @@ -43,10 +43,10 @@ But MoQ also carries non-media tracks — JSON, text, telemetry, captions, uncom For these tracks there is no standard, transport-visible way to compress payloads, so each application reinvents it, and relays cannot help. Like HTTP Transfer-Encoding, the on-wire compression is a hop-by-hop optimization: it does not conceptually change the object payload — the decompressed bytes *are* the object — it only changes how those bytes are carried over a single hop. -What this extension adds on top is an end-to-end *signal*: a track property by which the original publisher marks the content as worth compressing and names the algorithm. The signal travels end-to-end; the compression happens per hop. +What this extension adds on top is an end-to-end *signal*: a boolean track property by which the original publisher marks the content as worth compressing. The signal travels end to end; the choice of algorithm and the compression itself happen per hop. -- **Publisher signals, hops apply**: the COMPRESSION track property is set by the original publisher and carried end-to-end, but a payload is only compressed on a hop that negotiated the extension and whose receiver supports the algorithm. Where the extension is not negotiated, the same payload travels verbatim. -- **Per object, independently**: each object payload is an independent compressed stream with no shared dictionary or state between objects. This keeps every object individually decodable and avoids head-of-line decoding within a group. +- **Publisher signals, hops apply**: the COMPRESSION track property is set by the original publisher and carried end to end, but a payload is only compressed on a hop that negotiated the extension and a shared algorithm. Where the extension is not negotiated, the same payload travels verbatim. +- **Per object, independently**: each object payload is an independent compressed stream with no shared dictionary or state between objects. This keeps every object individually decodable, avoids head-of-line decoding within a group, and lets an individual object opt out of compression when it would not benefit. # Setup Negotiation @@ -65,44 +65,69 @@ COMPRESSION Setup Option { **Algorithm**: One or more Algorithm identifiers (see [Compression Algorithms](#compression-algorithms)) that the sender can decompress, each a varint, filling the Option Value. +They are listed in the sender's order of preference, most-preferred first. The identifier `none` (0) MUST NOT be listed (it requires no negotiation). A sender MUST NOT compress with an algorithm the receiver did not advertise in its SETUP. -This makes the on-wire state unambiguous on every hop without any per-object signaling: a receiver decompresses a track's payloads **if and only if** the COMPRESSION track property is present and the receiver advertised that algorithm in its own SETUP. In every other case — the property absent, the extension not negotiated, or the algorithm not advertised by the receiver — the sender was not permitted to compress, so the receiver treats the payloads as verbatim. +The negotiated algorithm for a hop — the **hop default** — is the first algorithm in the receiver's advertised list that the sender can also produce; if the lists do not intersect, the hop has no default and every payload travels verbatim. +This keeps the common case free of per-object signaling: where the COMPRESSION track property is present and the hop has a default, a receiver decompresses each object with the hop default unless that object carries a [per-object override](#per-object-override) naming a different algorithm (in particular `none`, to send it verbatim). Where the property is absent, the extension is not negotiated, or the algorithm lists do not intersect, the sender was not permitted to compress, so every payload is verbatim. # COMPRESSION Track Property -The COMPRESSION property is the original publisher's signal that a track's object payloads are worth compressing, and which algorithm to use. +The COMPRESSION property is the original publisher's signal that a track's object payloads are worth compressing. It is a track-level Key-Value-Pair carried with the track's properties (see {{moqt}} Section 2.5 and Section 12), set by the original publisher and forwarded unchanged by relays. Because the value is a single integer, COMPRESSION uses an even Type so the value is a bare varint: ~~~ COMPRESSION Track Property { Type (vi64) = 0xC03D0 - Value (vi64) ; Algorithm identifier + Value (vi64) ; boolean hint } ~~~ **Value**: -The Algorithm identifier the publisher recommends for this track's payloads. -The absence of the property, or a value of `none` (0), means the track is not marked for compression and its payloads are always transmitted verbatim. +A boolean hint: `1` means the track's payloads are worth compressing; `0`, or absence of the property, means they are not and are always transmitted verbatim. +Values greater than `1` are reserved for future use and MUST be treated as `1` by a receiver that does not understand them, so the hint stays additive. +The property names no algorithm: which algorithm is used, if any, is the per-hop negotiated [hop default](#setup-negotiation), overridable [per object](#per-object-override). The property is fixed for the lifetime of the track and MUST NOT change. A relay MUST forward it unchanged on every hop, including a hop that has not negotiated the extension: there it is simply an ignored unknown Key-Value-Pair, but forwarding it lets a further-downstream hop that does negotiate the extension still act on the publisher's signal. -Compression is enabled only by the combination of this track property and the extension being negotiated on a hop. -A publisher MUST NOT compress object payloads on a track that does not carry the COMPRESSION property, and there is no way to enable compression on a per-object basis: the property governs the whole track, and on a compressing hop every non-empty payload is compressed. +Compression is enabled only by the combination of a non-zero hint and the extension being negotiated with a shared algorithm on a hop. +A publisher MUST NOT compress object payloads on a track that does not carry a non-zero COMPRESSION hint. -Whether payloads are actually compressed is decided per hop: +Whether a given object is actually compressed is decided per hop and per object: -- On a hop where the extension is negotiated and the receiver advertised the property's algorithm, every non-empty object payload MUST be compressed with that algorithm, and the receiver decompresses it. -- On any other hop — the extension not negotiated, or the receiver did not advertise that algorithm — payloads are sent verbatim. The receiver either never sees the property (an ignored unknown Key-Value-Pair) or sees it but knows the sender was not permitted to compress for it, so it treats the payloads as verbatim either way. +- On a hop where the extension is negotiated and a [hop default](#setup-negotiation) exists, each non-empty object payload is compressed with the hop default and the receiver decompresses it — unless the object carries a [per-object override](#per-object-override), which names the algorithm actually used (including `none`, to send that object verbatim). +- On any other hop — the extension not negotiated, or the algorithm lists do not intersect — payloads are sent verbatim. The receiver either never sees the property (an ignored unknown Key-Value-Pair) or sees it but knows the sender was not permitted to compress for it, so it treats the payloads as verbatim either way. +A sender SHOULD send an object verbatim (via a `none` override) whenever the hop default would not make that object smaller — for example a small JSON merge-patch delta that DEFLATE would enlarge. Compression applies to the object payload only; object properties and message framing are never compressed. -An empty payload (size 0) MUST NOT be compressed and remains empty on the wire. +An empty payload (size 0) MUST NOT be compressed and remains empty on the wire; it needs no override. A publisher SHOULD set COMPRESSION only for payload types that benefit from it. -Already-compressed media SHOULD omit it (or use `none`). +Already-compressed media SHOULD omit it (or use `0`). + + +# Per-Object Override {#per-object-override} +The COMPRESSION_ALGORITHM property is an optional object-level Key-Value-Pair that overrides, for a single object, the algorithm a hop would otherwise apply. +Because the value is a single integer, it uses an even Type so the value is a bare varint: + +~~~ +COMPRESSION_ALGORITHM Object Property { + Type (vi64) = 0xC03D2 + Value (vi64) ; Algorithm identifier +} +~~~ + +**Value**: +The [algorithm](#compression-algorithms) actually used for this object's payload on this hop. +`none` (0) means the object is carried verbatim; any other identifier names the algorithm whose output the payload is. +A sender MUST NOT name an algorithm the receiver did not advertise in its SETUP. + +The property is meaningful only on a hop that has a [hop default](#setup-negotiation); where present it replaces the hop default for that object alone, and elsewhere objects are always verbatim and any COMPRESSION_ALGORITHM property MUST be ignored. +Unlike the boolean COMPRESSION hint, it is not the publisher's end-to-end signal: because it records what a hop actually did, a relay rewrites or removes it to reflect what it did on each downstream hop, exactly as it does for the payload bytes. +Its typical use is a `none` override that keeps an incompressible or tiny object verbatim while the rest of the track is compressed. # Compression Algorithms {#compression-algorithms} @@ -119,12 +144,12 @@ There is no shared dictionary or state between objects, so each object decompres # Relay Behavior -A relay forwards the COMPRESSION track property unchanged — it is the publisher's end-to-end signal — and applies compression independently on each hop. +A relay forwards the boolean COMPRESSION track property unchanged — it is the publisher's end-to-end signal — and applies compression independently on each hop. -On its upstream subscription, the relay receives payloads compressed if and only if that hop compressed them (the extension negotiated and the relay advertised the algorithm); it decompresses them as needed. -On each downstream subscription the relay serves, it compresses payloads with the track's algorithm when that downstream negotiated the extension and advertised the algorithm, and sends them verbatim otherwise. +On its upstream subscription, the relay receives each object compressed with that hop's default unless the object carried a per-object override; it reads the [COMPRESSION_ALGORITHM](#per-object-override) property, or the hop default in its absence, to decompress as needed. +On each downstream subscription the relay serves, it compresses each object with that downstream's hop default when one exists and sends objects verbatim otherwise, rewriting or removing the per-object COMPRESSION_ALGORITHM property to reflect what it actually did on that hop. -Compression is thus driven by the publisher's track property, not by the relay: a relay does not compress a track the publisher did not mark. +Compression is thus driven by the publisher's hint and each hop's negotiation, not by the relay's own initiative: a relay does not compress a track the publisher did not mark. In every case the decompressed bytes delivered to the application MUST be identical to what the origin published. A relay or generic library MUST NOT inspect or modify the decompressed contents unless otherwise negotiated; only recompression that preserves the decompressed bytes exactly is permitted. @@ -132,20 +157,20 @@ A relay or generic library MUST NOT inspect or modify the decompressed contents # Security Considerations Compressing data that mixes attacker-controlled and secret content in the same object can leak the secret through compressed size, as in the CRIME and BREACH attacks. -A publisher MUST NOT set COMPRESSION on a track whose object payloads combine secret material with attacker-influenced material. +A publisher MUST NOT set a non-zero COMPRESSION hint on a track whose object payloads combine secret material with attacker-influenced material. Because compression here is per-object with no cross-object dictionary, the exposure is bounded to within a single object, but it is not eliminated. A malicious sender could emit a small compressed payload that decompresses to a very large buffer (a "decompression bomb"). A receiver MUST bound the size of a decompressed object payload. If the bound is exceeded it MUST reset the affected Subscribe/Fetch stream (rather than allocate unbounded memory) and MAY close the session with a PROTOCOL_VIOLATION if it considers the peer abusive; the reset is stream-scoped so a single bad object does not tear down unrelated subscriptions. -Compression is orthogonal to {{moqt}} end-to-end encryption: an encrypted payload is effectively incompressible, so a publisher using end-to-end encryption SHOULD omit COMPRESSION (or use `none`). +Compression is orthogonal to {{moqt}} end-to-end encryption: an encrypted payload is effectively incompressible, so a publisher using end-to-end encryption SHOULD omit COMPRESSION (or use `0`). # IANA Considerations This document requests the following registrations. High, distinctive values are requested to avoid the low ranges reserved by {{moqt}} and to minimize collisions with provisional registrations by other extensions; they also avoid the greasing pattern (`0x7f * N + 0x9D`). -The parameter Type is even so that its value is a bare varint with no length prefix (see {{moqt}} Section 2.5). +Each Type is even so that its value is a bare varint with no length prefix (see {{moqt}} Section 2.5). ## MOQT Setup Options @@ -157,11 +182,12 @@ This document requests a registration in the "MOQT Setup Options" registry ({{mo ## MOQT Properties -This document requests a registration in the "MOQT Properties" registry ({{moqt}} Section 15.8), used for object and track properties. +This document requests registrations in the "MOQT Properties" registry ({{moqt}} Section 15.8), used for object and track properties. -| Value | Name | Scope | Reference | -|:--------|:------------|:------|:--------------| -| 0xC03D0 | COMPRESSION | Track | This Document | +| Value | Name | Scope | Reference | +|:--------|:----------------------|:-------|:--------------| +| 0xC03D0 | COMPRESSION | Track | This Document | +| 0xC03D2 | COMPRESSION_ALGORITHM | Object | This Document | ## MOQT Compression Algorithms diff --git a/draft-lcurley-moq-lite.md b/draft-lcurley-moq-lite.md index a6b71bf..4848768 100644 --- a/draft-lcurley-moq-lite.md +++ b/draft-lcurley-moq-lite.md @@ -498,12 +498,14 @@ DATAGRAM Body { Subscribe ID (i) Group Sequence (i) [Timestamp (i)] + [Compression (i)] Payload (b) } ~~~ `Timestamp` is present only when the Track's `Publisher Timescale` (see [TRACK_INFO](#track-info)) is non-zero. -When `Publisher Timescale` is 0, the field is omitted from the wire and the datagram body consists of just `Subscribe ID`, `Group Sequence`, and `Payload`. +`Compression` is present only when the Track's `Publisher Compression` hint is non-zero (see [TRACK_INFO](#track-info)). +Each field is independently omitted when its condition does not hold; with both absent the datagram body is just `Subscribe ID`, `Group Sequence`, and `Payload`. **Subscribe ID**: The Subscribe ID of an active subscription on the same session. @@ -519,7 +521,7 @@ Any varint value (including 0) is a valid absolute timestamp. **Payload**: The frame payload, extending to the end of the datagram. -If the Track's `Publisher Compression` is non-zero, the payload is compressed using the negotiated algorithm (see [TRACK_INFO](#track-info)). +When the `Compression` field is present and non-`none`, the payload is compressed with that [algorithm](#compression-algorithms); the per-hop negotiation and per-frame choice are the same as for [FRAME](#frame). The total datagram body (including all header fields above and the compressed payload if applicable) MUST NOT exceed 1200 bytes. This limit ensures the datagram fits within the minimum QUIC path MTU without IP-layer fragmentation. Payloads that would not fit MUST be sent as a Group Stream instead. @@ -586,13 +588,15 @@ The parameter-specific value, interpreted according to Parameter ID. A capability is available for the session only if the relevant endpoint advertises it; an absent parameter means the sender does not support that capability. The following Setup Parameters are defined: -|------|----------|-------------| -| ID | Name | Value | -|-----:|:---------|:------------| -| 0x1 | Probe | Level (i) | -|------|----------|-------------| -| 0x2 | Path | Path (s) | -|------|----------|-------------| +|------|-------------|-------------------| +| ID | Name | Value | +|-----:|:------------|:------------------| +| 0x1 | Probe | Level (i) | +|------|-------------|-------------------| +| 0x2 | Path | Path (s) | +|------|-------------|-------------------| +| 0x3 | Compression | Algorithm (i) ... | +|------|-------------|-------------------| ### Probe Parameter {#probe-parameter} The Probe Parameter advertises the sender's capability level when acting as a publisher on a [Probe Stream](#probe). @@ -627,6 +631,14 @@ The remaining bindings convey the path in their own handshake. A relay MUST NOT forward the Path Parameter; like other per-hop setup metadata it applies only to this hop (see [Session](#session)). +### Compression Parameter {#compression-parameter} +The Compression Parameter advertises the payload compression [algorithms](#compression-algorithms) the sender can *decompress* on this hop. +The Parameter Value is a sequence of algorithm identifiers, each a variable-length integer, packed back-to-back to fill the Parameter Length; the identifier `none` (0) MUST NOT be listed, since verbatim transfer needs no negotiation. + +Negotiation is per direction and per hop: an endpoint's advertisement governs only what may be compressed when sending *to* it, and a relay MUST NOT forward the parameter (see [Session](#session)). +A sender MUST NOT compress a frame with an algorithm the receiver did not advertise here, so an endpoint that omits the parameter (or advertises an empty list) receives every payload verbatim regardless of the Track's `Publisher Compression` hint. +Which advertised algorithm is used for a given frame is the sender's choice, named in that frame (see [Compression](#compression-algorithms)). + ## ANNOUNCE_REQUEST A subscriber sends an ANNOUNCE_REQUEST message to indicate it wants to receive an ANNOUNCE_BROADCAST message for any broadcasts with a path that starts with the requested prefix. @@ -852,22 +864,36 @@ When `Publisher Timescale` is 0, the per-frame `Timestamp Delta` field is omitte Common values include `1000` (milliseconds), `1000000` (microseconds), `48000` (audio sample rate), and `90000` (RTP video clock). **Publisher Compression**: -The compression algorithm applied to every Frame `Payload` on this Track. +A hint from the original publisher that this Track's payloads are worth compressing. + +- `0`: not worth compressing; payloads are always sent verbatim (the default). +- `1`: worth compressing; payloads MAY be compressed on any hop that has negotiated an algorithm. + +Values greater than `1` are reserved (e.g. for a future publisher-recommended algorithm); a subscriber that does not understand a reserved value MUST treat it as `1`, so the hint stays additive and never blocks the Track. + +The hint is carried end to end but names no algorithm and by itself causes no compression. Which algorithm is used, if any, is negotiated independently on each hop (see the [Compression Parameter](#compression-parameter)) and named in each frame (see [Compression](#compression-algorithms)). When the hint is non-zero, every FRAME and datagram on the Track carries a `Compression` field; when it is `0`, that field is absent and all payloads are verbatim. + +The publisher SHOULD set the hint only for payload types that benefit from compression (e.g. JSON, text, uncompressed binary structures); already-compressed media (e.g. H.264, Opus, AV1) gains nothing and SHOULD leave it `0`. See [Security Considerations](#security-considerations) for content that MUST NOT be marked compressible. + +## Compression {#compression-algorithms} +moq-lite can compress Frame payloads hop by hop, like HTTP Transfer-Encoding: the decompressed bytes *are* the payload and are identical end to end, but how they are carried MAY differ on each hop, and a payload compressed on one hop MAY be verbatim on the next. Compression is governed by three independent pieces: -- `none` (0): payloads are transmitted verbatim (default). -- `deflate` (1): payloads are compressed with raw DEFLATE as defined in {{!RFC1951}}, with no zlib or gzip framing. +- the Track's `Publisher Compression` hint (see [TRACK_INFO](#track-info)) — the publisher's end-to-end signal that payloads are worth compressing; +- the per-hop [Compression Parameter](#compression-parameter) — each endpoint advertises the algorithms it can decompress; and +- the per-frame `Compression` field (see [FRAME](#frame) and [Datagrams](#datagrams)) — names the algorithm actually used for each payload. -Compression is applied per-frame: each Frame `Payload` is an independent compressed stream with no shared dictionary or state between frames. -This keeps frames independently decodable and avoids head-of-line decoding within a group. -The Frame `Message Length` describes the compressed (on-wire) size. -An empty payload (size 0) MUST NOT be compressed and remains empty on the wire. +The following algorithms are defined. The identifiers are shared with the MoQ Payload Compression Extension for MoQ Transport so that a relay bridging the two needs no translation. -The publisher SHOULD only enable compression for payload types that benefit from it (e.g. JSON, text, uncompressed binary structures). -Already-compressed media (e.g. H.264, Opus, AV1) gains nothing and SHOULD use `none`. -A subscriber that does not recognize the value MUST NOT open a Subscribe or Fetch stream for the track; if it already opened one before receiving TRACK_INFO, it MUST reset that stream with a protocol violation. The Track Stream itself needs no reset — the publisher FINs it after TRACK_INFO. +| ID | Name | Description | +|---:|:--------|:--------------------------------------------------------| +| 0 | none | The payload is carried verbatim. The default. | +| 1 | deflate | Raw DEFLATE {{!RFC1951}}, with no zlib or gzip framing. | -A relay MAY transcode payloads between compression algorithms (including bridging different protocol versions, e.g. a moq-lite-05 publisher to a moq-lite-04 subscriber) provided the decompressed bytes are identical to what the publisher produced. -A relay SHOULD NOT compress an originally-uncompressed payload unless there is a strong content signal that compression is beneficial (e.g. the track name ends in `.json`), because the relay cannot otherwise predict whether compression will help or hurt. +Each payload is compressed independently: it is a self-contained stream with no shared dictionary or state between frames, so every frame is independently decodable and there is no head-of-line decoding within a group. The Frame `Message Length` describes the compressed (on-wire) size. An empty payload (size 0) MUST NOT be compressed and MUST use `none`. + +A sender MUST NOT use an algorithm the receiver did not advertise in its [Compression Parameter](#compression-parameter); where the two have negotiated no algorithm, every payload is sent with `Compression` = `none`. A sender SHOULD use `none` for any individual payload that compression would not make smaller — for example a small JSON merge-patch delta that DEFLATE would enlarge — so the per-hop, per-frame decision stays decoupled from the publisher's whole-Track hint. + +A relay applies compression independently on each hop, driven by each downstream's negotiated algorithms — not by the relay's own initiative, and never on a Track the publisher did not mark. On its upstream subscription it reads the algorithm named in each frame and decompresses as needed; on each downstream subscription it (re)compresses each frame with an algorithm that downstream advertised, or sends `none`. A relay MAY transcode between algorithms, including bridging protocol versions (e.g. a moq-lite-05 publisher to a moq-lite-04 subscriber), provided the decompressed bytes are identical to what the publisher produced. A relay or generic library MUST NOT inspect or modify the decompressed contents unless otherwise negotiated. ## SUBSCRIBE_OK {#subscribe-ok} A SUBSCRIBE_OK message confirms a subscription and resolves its absolute start group. @@ -1035,13 +1061,14 @@ The FRAME message is a payload within a group. ~~~ FRAME Message { [Timestamp Delta (i)] + [Compression (i)] Message Length (i) Payload (b) } ~~~ -`Timestamp Delta` is present only when the Track's `Publisher Timescale` (see [TRACK_INFO](#track-info)) is non-zero. -When `Publisher Timescale` is 0, the field is omitted from the wire and the FRAME consists of just `Message Length` and `Payload`. +`Timestamp Delta` is present only when the Track's `Publisher Timescale` is non-zero, and `Compression` only when the Track's `Publisher Compression` hint is non-zero (see [TRACK_INFO](#track-info)). +Each is independently omitted when its condition does not hold; with both absent the FRAME is just `Message Length` and `Payload`. **Timestamp Delta**: A signed delta from the previous frame's timestamp, in the Track's negotiated `Timescale`. @@ -1053,10 +1080,15 @@ Encoded as a zigzag-mapped variable-length integer: Zigzag interleaves non-negative and negative values (`0 → 0, -1 → 1, 1 → 2, -2 → 3, 2 → 4, ...`) so small magnitudes of either sign fit in a 1-byte varint and there is exactly one wire encoding for zero. The first frame of a group is delta-encoded from `0`, so its `Timestamp Delta` is the zigzag encoding of the absolute timestamp. +**Compression**: +The [algorithm](#compression-algorithms) used for this frame's `Payload`, present only when the Track's `Publisher Compression` hint is non-zero (see [TRACK_INFO](#track-info)); when the hint is `0` the field is absent and the payload is always verbatim. +`none` (0) carries the payload verbatim; any other identifier names the algorithm whose output the payload is. +The sender chooses per frame and MUST NOT use an algorithm the receiver did not advertise in its [Compression Parameter](#compression-parameter); it SHOULD use `none` whenever compression would not shrink the payload. An empty payload MUST use `none`. + **Payload**: An application-specific payload. -If the Track's `Publisher Compression` is non-zero, the payload is compressed using the negotiated algorithm (see [TRACK_INFO](#track-info)) and the `Message Length` describes the compressed size. -A generic library or relay MUST NOT inspect or modify the decompressed contents unless otherwise negotiated; recompression that preserves the decompressed bytes exactly is allowed (see [TRACK_INFO](#track-info)). +When the `Compression` field is present and non-`none`, the payload is the compressed form and `Message Length` describes its compressed (on-wire) size; the receiver decompresses it with the named [algorithm](#compression-algorithms) before delivery. +A generic library or relay MUST NOT inspect or modify the decompressed contents unless otherwise negotiated; recompression that preserves the decompressed bytes exactly is allowed (see [Compression](#compression-algorithms)). # Appendix A: Changelog @@ -1079,7 +1111,7 @@ A generic library or relay MUST NOT inspect or modify the decompressed contents - Removed `Publisher Max Latency`. The publisher's retention guarantee is no longer part of the wire format; retention for FETCH and future subscriptions is best-effort and left to the publisher. - Timestamp-based expiration replaces wall-clock arrival time when a Track timescale is negotiated. - Added QUIC datagram delivery for groups, sharing Subscribe IDs with existing subscriptions (no separate control stream). -- Added `Publisher Compression` to TRACK_INFO for per-frame payload compression (`none` or `deflate`). +- Added payload compression. `Publisher Compression` in TRACK_INFO is now a boolean end-to-end hint that a Track's payloads are worth compressing; the algorithm is negotiated per hop via a new SETUP `Compression` parameter (each endpoint advertises the algorithms it can decompress) and named per frame by a new `Compression` field in FRAME and datagram bodies (present only when the hint is set). `none` (0) and `deflate` (1) are defined, sharing identifiers with the MoQ Payload Compression Extension. Because each payload is an independent stream, a frame can opt out (`none`) when compression would not shrink it — e.g. a small JSON merge-patch delta — while larger payloads on the same Track are compressed, and different hops can use different algorithms. - Added Qmux [qmux] transport bindings for TCP/TLS and WebSocket, for environments where UDP is unavailable. The WebSocket binding uses the WebSocket message framing in place of the Qmux Record `Size` field. ## moq-lite-04 @@ -1176,7 +1208,17 @@ Some of these fields occur in multiple messages. # Security Considerations -TODO Security +TODO: general security considerations. + +## Payload Compression +Compressing data that mixes attacker-controlled and secret material in the same payload can leak the secret through the compressed size, as in the CRIME and BREACH attacks. +A publisher MUST NOT set a non-zero `Publisher Compression` hint on a Track whose payloads combine secret material with attacker-influenced material. +Because compression is per-frame with no cross-frame dictionary, the exposure is bounded to within a single payload, but it is not eliminated. + +A malicious sender could emit a small compressed payload that decompresses to a very large buffer (a "decompression bomb"). +A receiver MUST bound the size of a decompressed payload; if the bound is exceeded it MUST reset the affected stream rather than allocate unbounded memory, and MAY close the session with a PROTOCOL_VIOLATION if it considers the peer abusive. + +Compression is orthogonal to end-to-end encryption: an encrypted payload is effectively incompressible, so a publisher using end-to-end encryption SHOULD leave `Publisher Compression` at `0`. # IANA Considerations From 607ed837b152a805462c0f16d6fb1d800227983d Mon Sep 17 00:00:00 2001 From: Claude Date: Mon, 22 Jun 2026 22:08:49 +0000 Subject: [PATCH 02/10] moq-lite: make compression group-scoped (one DEFLATE stream per group) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Rework payload compression so a whole group is compressed as a single DEFLATE stream rather than each frame independently. Within a group the frames already share one ordered, reliable QUIC stream, so the group — not the frame — is the random-access unit; compressing across it gains cross-frame redundancy and compresses the framing too, while the first frame of a group stays independently decodable and groups remain mutually independent. - Publisher Compression in TRACK_INFO stays a boolean end-to-end hint. - SETUP Compression parameter is now a presence flag (capability), not an algorithm list. - A per-group Compression field (0 verbatim, 1 DEFLATE) is carried in GROUP, in datagram bodies, and in a reintroduced minimal FETCH_OK (present only when the hint is set). The per-frame Compression field is gone. - DEFLATE is the only scheme; others are left to future extensions, one in effect per hop. - Security note updated: the CRIME/BREACH window is now per group, not per frame. This supersedes the earlier per-frame, enum-based compression on this branch. Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_01W8bLV6vHzucLNvDhPk3bMP --- draft-lcurley-moq-lite.md | 90 ++++++++++++++++++++++----------------- 1 file changed, 50 insertions(+), 40 deletions(-) diff --git a/draft-lcurley-moq-lite.md b/draft-lcurley-moq-lite.md index 4848768..d47bcd7 100644 --- a/draft-lcurley-moq-lite.md +++ b/draft-lcurley-moq-lite.md @@ -309,7 +309,7 @@ When the accepted track has already ended with no matching groups there is no st A rejection is a stream reset: if the publisher cannot serve the subscription — the track does not exist, or it otherwise refuses — it MUST reset the stream rather than leave it pending, and SHOULD do so promptly (within roughly a round trip) so the subscriber is not left waiting. A subscription the publisher accepts but has no groups for yet is not a rejection: for a live track the publisher MAY withhold SUBSCRIBE_OK until the first matching group resolves the start. A subscriber therefore distinguishes "pending" from "refused" by the stream reset, not by a timeout. The Subscribe Stream does not carry the track's publisher properties — those are immutable and fetched once via a [Track Stream](#track-stream) (see [TRACK_INFO](#track-info)). -The subscriber MUST have the track's TRACK_INFO before it can parse the FRAME messages that arrive on Group Streams, since the timescale and compression determine the frame wire format; it MAY open the Track and Subscribe streams concurrently and buffer frames until TRACK_INFO arrives. +The subscriber MUST have the track's TRACK_INFO before it can parse the FRAME messages that arrive on Group Streams, since the timescale determines the frame wire format and the compression hint determines whether the stream is DEFLATE-compressed; it MAY open the Track and Subscribe streams concurrently and buffer frames until TRACK_INFO arrives. The publisher sends SUBSCRIBE_OK once the absolute start group is resolved, and SUBSCRIBE_END once no further groups will be produced (see [SUBSCRIBE_OK](#subscribe-ok) and [SUBSCRIBE_END](#subscribe-end)). The publisher closes the stream (FIN) only once every group from start to end has been accounted for, either via a GROUP stream (completed or reset) or a SUBSCRIBE_DROP message. @@ -321,7 +321,7 @@ Either endpoint MAY reset/cancel the stream at any time. A subscriber opens a Fetch Stream (0x3) to request a single Group from a Track. The subscriber sends a FETCH message containing the broadcast path, track name, priority, and group sequence. -The publisher responds with FRAME messages directly on the same bidirectional stream — there is no response header. +The publisher responds on the same bidirectional stream. When the Track's `Publisher Compression` hint is non-zero (see [TRACK_INFO](#track-info)) the response begins with a [FETCH_OK](#fetch-ok) message carrying the group's `Compression` field, followed by the FRAME messages; otherwise it is bare FRAME messages with no response header. The Subscribe ID and Group Sequence for the returned FRAME messages are implicit, taken from the original FETCH request. As with a subscription, the subscriber MUST already have the track's [TRACK_INFO](#track-info) to parse the returned frames; because the properties are immutable, a single Track Stream lookup is reused across every FETCH of that track (group-by-group fetches do not re-fetch it). The publisher FINs the stream after the last frame, or resets the stream on error. @@ -473,6 +473,7 @@ See the [Session](#session) section for how an endpoint avoids waiting on the pe A publisher creates Group Streams in response to a Subscribe Stream. A Group Stream MUST start with a GROUP message and MAY be followed by any number of FRAME messages. +When the Track is compressed on this hop (see [Compression](#compression)), the FRAME messages following the GROUP message form a single DEFLATE stream. A Group MAY contain zero FRAME messages, potentially indicating a gap in the track. A frame MAY contain an empty payload, potentially indicating a gap in the group. @@ -504,7 +505,7 @@ DATAGRAM Body { ~~~ `Timestamp` is present only when the Track's `Publisher Timescale` (see [TRACK_INFO](#track-info)) is non-zero. -`Compression` is present only when the Track's `Publisher Compression` hint is non-zero (see [TRACK_INFO](#track-info)). +`Compression` is present only when the Track's `Publisher Compression` hint is non-zero (see [TRACK_INFO](#track-info)), with the same meaning as in the [GROUP](#group) message: `0` verbatim, `1` raw DEFLATE. Each field is independently omitted when its condition does not hold; with both absent the datagram body is just `Subscribe ID`, `Group Sequence`, and `Payload`. **Subscribe ID**: @@ -521,7 +522,7 @@ Any varint value (including 0) is a valid absolute timestamp. **Payload**: The frame payload, extending to the end of the datagram. -When the `Compression` field is present and non-`none`, the payload is compressed with that [algorithm](#compression-algorithms); the per-hop negotiation and per-frame choice are the same as for [FRAME](#frame). +A datagram is a single-frame group: when `Compression` is `1` the payload is a single raw DEFLATE stream, and when `0` (or absent) it is verbatim (see [Compression](#compression)). The total datagram body (including all header fields above and the compressed payload if applicable) MUST NOT exceed 1200 bytes. This limit ensures the datagram fits within the minimum QUIC path MTU without IP-layer fragmentation. Payloads that would not fit MUST be sent as a Group Stream instead. @@ -595,7 +596,7 @@ The following Setup Parameters are defined: |------|-------------|-------------------| | 0x2 | Path | Path (s) | |------|-------------|-------------------| -| 0x3 | Compression | Algorithm (i) ... | +| 0x3 | Compression | (empty) | |------|-------------|-------------------| ### Probe Parameter {#probe-parameter} @@ -632,12 +633,11 @@ The remaining bindings convey the path in their own handshake. A relay MUST NOT forward the Path Parameter; like other per-hop setup metadata it applies only to this hop (see [Session](#session)). ### Compression Parameter {#compression-parameter} -The Compression Parameter advertises the payload compression [algorithms](#compression-algorithms) the sender can *decompress* on this hop. -The Parameter Value is a sequence of algorithm identifiers, each a variable-length integer, packed back-to-back to fill the Parameter Length; the identifier `none` (0) MUST NOT be listed, since verbatim transfer needs no negotiation. +The Compression Parameter advertises that the sender can *decompress* payloads on this hop (see [Compression](#compression)). +Its Parameter Value is empty; presence alone is the capability. Negotiation is per direction and per hop: an endpoint's advertisement governs only what may be compressed when sending *to* it, and a relay MUST NOT forward the parameter (see [Session](#session)). -A sender MUST NOT compress a frame with an algorithm the receiver did not advertise here, so an endpoint that omits the parameter (or advertises an empty list) receives every payload verbatim regardless of the Track's `Publisher Compression` hint. -Which advertised algorithm is used for a given frame is the sender's choice, named in that frame (see [Compression](#compression-algorithms)). +A sender MUST NOT compress when sending to a receiver that did not advertise this parameter, so an endpoint that omits it receives every Group Stream, Fetch Stream, and datagram verbatim regardless of the Track's `Publisher Compression` hint. ## ANNOUNCE_REQUEST @@ -867,33 +867,26 @@ Common values include `1000` (milliseconds), `1000000` (microseconds), `48000` ( A hint from the original publisher that this Track's payloads are worth compressing. - `0`: not worth compressing; payloads are always sent verbatim (the default). -- `1`: worth compressing; payloads MAY be compressed on any hop that has negotiated an algorithm. +- `1`: worth compressing; payloads MAY be compressed on any hop that has negotiated compression. -Values greater than `1` are reserved (e.g. for a future publisher-recommended algorithm); a subscriber that does not understand a reserved value MUST treat it as `1`, so the hint stays additive and never blocks the Track. +Values greater than `1` are reserved for future use; a subscriber that does not understand a reserved value MUST treat it as `1`, so the hint stays additive and never blocks the Track. -The hint is carried end to end but names no algorithm and by itself causes no compression. Which algorithm is used, if any, is negotiated independently on each hop (see the [Compression Parameter](#compression-parameter)) and named in each frame (see [Compression](#compression-algorithms)). When the hint is non-zero, every FRAME and datagram on the Track carries a `Compression` field; when it is `0`, that field is absent and all payloads are verbatim. +The hint is carried end to end but by itself causes no compression: compression also requires the receiving hop to advertise the [Compression Parameter](#compression-parameter), and is applied per group and signaled by a `Compression` field (see [Compression](#compression)). When the hint is non-zero, that field is present in every GROUP message, FETCH_OK message, and datagram body; when it is `0`, the field is absent everywhere and all payloads are verbatim. The publisher SHOULD set the hint only for payload types that benefit from compression (e.g. JSON, text, uncompressed binary structures); already-compressed media (e.g. H.264, Opus, AV1) gains nothing and SHOULD leave it `0`. See [Security Considerations](#security-considerations) for content that MUST NOT be marked compressible. -## Compression {#compression-algorithms} -moq-lite can compress Frame payloads hop by hop, like HTTP Transfer-Encoding: the decompressed bytes *are* the payload and are identical end to end, but how they are carried MAY differ on each hop, and a payload compressed on one hop MAY be verbatim on the next. Compression is governed by three independent pieces: +## Compression {#compression} +moq-lite can compress payloads hop by hop, like HTTP Transfer-Encoding: the decompressed bytes *are* the payload and are identical end to end, but how they are carried MAY differ on each hop — a group compressed on one hop is decompressed and MAY be re-sent verbatim on the next. Compression is governed by: - the Track's `Publisher Compression` hint (see [TRACK_INFO](#track-info)) — the publisher's end-to-end signal that payloads are worth compressing; -- the per-hop [Compression Parameter](#compression-parameter) — each endpoint advertises the algorithms it can decompress; and -- the per-frame `Compression` field (see [FRAME](#frame) and [Datagrams](#datagrams)) — names the algorithm actually used for each payload. +- the per-hop [Compression Parameter](#compression-parameter) — each endpoint advertises that it can decompress; and +- a per-group `Compression` field — carried in each [GROUP](#group) message, [FETCH_OK](#fetch-ok) message, and [datagram](#datagrams) body, present only when the Track's hint is non-zero — that states whether that group is compressed and with which scheme. -The following algorithms are defined. The identifiers are shared with the MoQ Payload Compression Extension for MoQ Transport so that a relay bridging the two needs no translation. +`Compression` is `0` (verbatim) or `1` (raw DEFLATE {{!RFC1951}}); other schemes are left to future extensions, with at most one in effect per hop. A sender MUST NOT use a non-zero value unless the receiver advertised the [Compression Parameter](#compression-parameter); it SHOULD leave a group at `0` when compression would not make it smaller — for example a one-frame group carrying a small JSON merge-patch delta that DEFLATE would enlarge. -| ID | Name | Description | -|---:|:--------|:--------------------------------------------------------| -| 0 | none | The payload is carried verbatim. The default. | -| 1 | deflate | Raw DEFLATE {{!RFC1951}}, with no zlib or gzip framing. | +Compression is **group-scoped**. When a group is compressed, its entire sequence of FRAME messages is a single DEFLATE stream, reset at each group boundary: on a Group Stream this is the FRAME sequence following the GROUP message; on a Fetch Stream it is the FRAME sequence following the FETCH_OK; in a datagram it is the single frame's payload. The first frame of a group is therefore independently decodable — a subscriber joining at a group boundary needs nothing earlier — while later frames in the same group share the compression context, so both framing bytes and cross-frame redundancy are compressed. There is no shared state between groups. A group with no frames carries no DEFLATE stream. -Each payload is compressed independently: it is a self-contained stream with no shared dictionary or state between frames, so every frame is independently decodable and there is no head-of-line decoding within a group. The Frame `Message Length` describes the compressed (on-wire) size. An empty payload (size 0) MUST NOT be compressed and MUST use `none`. - -A sender MUST NOT use an algorithm the receiver did not advertise in its [Compression Parameter](#compression-parameter); where the two have negotiated no algorithm, every payload is sent with `Compression` = `none`. A sender SHOULD use `none` for any individual payload that compression would not make smaller — for example a small JSON merge-patch delta that DEFLATE would enlarge — so the per-hop, per-frame decision stays decoupled from the publisher's whole-Track hint. - -A relay applies compression independently on each hop, driven by each downstream's negotiated algorithms — not by the relay's own initiative, and never on a Track the publisher did not mark. On its upstream subscription it reads the algorithm named in each frame and decompresses as needed; on each downstream subscription it (re)compresses each frame with an algorithm that downstream advertised, or sends `none`. A relay MAY transcode between algorithms, including bridging protocol versions (e.g. a moq-lite-05 publisher to a moq-lite-04 subscriber), provided the decompressed bytes are identical to what the publisher produced. A relay or generic library MUST NOT inspect or modify the decompressed contents unless otherwise negotiated. +A relay applies compression independently on each hop, driven by each downstream's negotiation — not by the relay's own initiative, and never on a Track the publisher did not mark. On its upstream subscription it decompresses each compressed group as needed; on each downstream subscription it sets `Compression` per group, compressing when that downstream advertised the capability and sending `0` otherwise. A relay MAY bridge protocol versions (e.g. a moq-lite-05 publisher to a moq-lite-04 subscriber) provided the decompressed bytes are identical to what the publisher produced. A relay or generic library MUST NOT inspect or modify the decompressed contents unless otherwise negotiated. ## SUBSCRIBE_OK {#subscribe-ok} A SUBSCRIBE_OK message confirms a subscription and resolves its absolute start group. @@ -990,11 +983,27 @@ See the [Prioritization](#prioritization) section for more information. **Group Sequence**: The sequence number of the group to fetch. -The publisher responds with FRAME messages directly on the same stream — there is no response header. +The publisher responds on the same stream: when the Track's `Publisher Compression` hint is non-zero the response begins with a [FETCH_OK](#fetch-ok) message, followed by the FRAME messages; otherwise it is bare FRAME messages with no response header. The subscriber parses them using the track's [TRACK_INFO](#track-info), which it MUST already have (see the [Track Stream](#track-stream)); the group sequence is implicit from the FETCH request. The publisher FINs the stream after the last frame, or resets on error. There is no FETCH_ERROR message — the publisher signals failure by resetting the stream. +## FETCH_OK {#fetch-ok} +FETCH_OK is the first message of a fetch response, sent by the publisher only when the Track's `Publisher Compression` hint is non-zero (see [TRACK_INFO](#track-info)). +It carries the `Compression` field for the single group being fetched; the FRAME messages follow on the same stream. +When the hint is `0` it is omitted and the response is bare FRAME messages. + +~~~ +FETCH_OK Message { + Message Length (i) + Compression (i) +} +~~~ + +**Compression**: +The compression scheme for the fetched group, with the same meaning as in the [GROUP](#group) message: `0` verbatim, `1` raw DEFLATE. +When `1`, the FRAME messages following this FETCH_OK form a single DEFLATE stream (see [Compression](#compression)). + ## PROBE PROBE is used to measure the available bitrate of the connection. @@ -1042,6 +1051,7 @@ GROUP Message { Message Length (i) Subscribe ID (i) Group Sequence (i) + [Compression (i)] } ~~~ @@ -1054,6 +1064,11 @@ The sequence number of the group. This SHOULD increase by 1 for each new group. A subscriber MUST handle gaps, potentially caused by congestion. +**Compression**: +The compression scheme applied to this group's frame sequence, present only when the Track's `Publisher Compression` hint is non-zero (see [TRACK_INFO](#track-info)). +`0` means the FRAME messages following this GROUP message are verbatim; `1` means they are a single raw DEFLATE stream (see [Compression](#compression)). +When the hint is `0`, the field is absent and the frames are always verbatim. + ## FRAME The FRAME message is a payload within a group. @@ -1061,14 +1076,14 @@ The FRAME message is a payload within a group. ~~~ FRAME Message { [Timestamp Delta (i)] - [Compression (i)] Message Length (i) Payload (b) } ~~~ -`Timestamp Delta` is present only when the Track's `Publisher Timescale` is non-zero, and `Compression` only when the Track's `Publisher Compression` hint is non-zero (see [TRACK_INFO](#track-info)). -Each is independently omitted when its condition does not hold; with both absent the FRAME is just `Message Length` and `Payload`. +`Timestamp Delta` is present only when the Track's `Publisher Timescale` (see [TRACK_INFO](#track-info)) is non-zero. +When `Publisher Timescale` is 0, the field is omitted from the wire and the FRAME consists of just `Message Length` and `Payload`. +The FRAME fields are always the uncompressed contents; when the group is compressed (see [Compression](#compression)) the whole frame sequence is carried inside the group's DEFLATE stream, so `Message Length` remains the uncompressed payload size. **Timestamp Delta**: A signed delta from the previous frame's timestamp, in the Track's negotiated `Timescale`. @@ -1080,15 +1095,10 @@ Encoded as a zigzag-mapped variable-length integer: Zigzag interleaves non-negative and negative values (`0 → 0, -1 → 1, 1 → 2, -2 → 3, 2 → 4, ...`) so small magnitudes of either sign fit in a 1-byte varint and there is exactly one wire encoding for zero. The first frame of a group is delta-encoded from `0`, so its `Timestamp Delta` is the zigzag encoding of the absolute timestamp. -**Compression**: -The [algorithm](#compression-algorithms) used for this frame's `Payload`, present only when the Track's `Publisher Compression` hint is non-zero (see [TRACK_INFO](#track-info)); when the hint is `0` the field is absent and the payload is always verbatim. -`none` (0) carries the payload verbatim; any other identifier names the algorithm whose output the payload is. -The sender chooses per frame and MUST NOT use an algorithm the receiver did not advertise in its [Compression Parameter](#compression-parameter); it SHOULD use `none` whenever compression would not shrink the payload. An empty payload MUST use `none`. - **Payload**: An application-specific payload. -When the `Compression` field is present and non-`none`, the payload is the compressed form and `Message Length` describes its compressed (on-wire) size; the receiver decompresses it with the named [algorithm](#compression-algorithms) before delivery. -A generic library or relay MUST NOT inspect or modify the decompressed contents unless otherwise negotiated; recompression that preserves the decompressed bytes exactly is allowed (see [Compression](#compression-algorithms)). +`Message Length` is the uncompressed payload size; any compression is group-scoped (see [Compression](#compression)) and applied to the whole frame sequence on the stream, not to this field individually. +A generic library or relay MUST NOT inspect or modify the decompressed contents unless otherwise negotiated; recompression that preserves the decompressed bytes exactly is allowed (see [Compression](#compression)). # Appendix A: Changelog @@ -1099,7 +1109,7 @@ A generic library or relay MUST NOT inspect or modify the decompressed contents - Added a SETUP message, sent once on a unidirectional Setup Stream (0x1) at the start of the session and FIN'd immediately. It carries a list of Setup Parameters for negotiating optional capabilities and extensions per-hop, replacing the prior stream-probing approach (version is still negotiated via ALPN, not SETUP). Endpoints keep exchanging non-Setup streams without waiting for SETUP, buffering only a stream whose encoding a negotiated extension would change; unknown stream types are still reset as a fallback. - Added a SETUP `Probe` parameter advertising the publisher's capability level: `None`, `Report` (measure and report the estimated bitrate), or `Increase` (additionally pad to probe for bandwidth above the current sending rate). The levels are nested since probing without measuring is meaningless. A subscriber must not rely on a level the publisher did not advertise. - Added a Track Stream (0x6): a TRACK request that the publisher answers with a single TRACK_INFO message and then FINs. TRACK_INFO carries the Track's immutable publisher properties (`Publisher Priority`, `Publisher Ordered`, `Publisher Timescale`, `Publisher Compression`). It is fetched once and cached, so the properties are no longer echoed on every response — notably, group-by-group FETCHes reuse one lookup. -- Removed FETCH_OK and trimmed SUBSCRIBE_OK down to a single resolved start group. Publisher properties moved to TRACK_INFO; a FETCH returns bare FRAME messages. All publisher properties are immutable for the lifetime of the Track — a publisher-side change would otherwise have to fan *out* to every downstream of a relay, whereas subscriber properties fan *in* and may still change via SUBSCRIBE_UPDATE. +- Removed FETCH_OK and trimmed SUBSCRIBE_OK down to a single resolved start group. Publisher properties moved to TRACK_INFO; a FETCH returns bare FRAME messages (a minimal FETCH_OK is reintroduced only for compressible tracks, to carry the per-group compression flag; see the compression entry). All publisher properties are immutable for the lifetime of the Track — a publisher-side change would otherwise have to fan *out* to every downstream of a relay, whereas subscriber properties fan *in* and may still change via SUBSCRIBE_UPDATE. - Split the resolved group range across SUBSCRIBE_OK and a new SUBSCRIBE_END. SUBSCRIBE_OK resolves the absolute start (`>=` the requested start; a larger value implicitly drops the leading range), and SUBSCRIBE_END signals that no group will follow a given sequence (stragglers within the range may still be dropped before FIN). SUBSCRIBE_OK keeps the MoqTransport name and its role as the publisher's positive response. - Renamed `Start Group`/`End Group` to `Group Start`/`Group End` in SUBSCRIBE, SUBSCRIBE_UPDATE, and SUBSCRIBE_DROP for consistency with the entity-first naming used elsewhere (e.g. `Group Sequence`). Wire format unchanged. - Allowed a duplicate `active` ANNOUNCE_BROADCAST to atomically replace the prior advertisement (equivalent to UNANNOUNCE+ANNOUNCE_BROADCAST). Used when only the origin or hop path changes (e.g. relay failover) without interrupting the broadcast. No new wire enum value — the existing `active` status carries the new metadata. @@ -1111,7 +1121,7 @@ A generic library or relay MUST NOT inspect or modify the decompressed contents - Removed `Publisher Max Latency`. The publisher's retention guarantee is no longer part of the wire format; retention for FETCH and future subscriptions is best-effort and left to the publisher. - Timestamp-based expiration replaces wall-clock arrival time when a Track timescale is negotiated. - Added QUIC datagram delivery for groups, sharing Subscribe IDs with existing subscriptions (no separate control stream). -- Added payload compression. `Publisher Compression` in TRACK_INFO is now a boolean end-to-end hint that a Track's payloads are worth compressing; the algorithm is negotiated per hop via a new SETUP `Compression` parameter (each endpoint advertises the algorithms it can decompress) and named per frame by a new `Compression` field in FRAME and datagram bodies (present only when the hint is set). `none` (0) and `deflate` (1) are defined, sharing identifiers with the MoQ Payload Compression Extension. Because each payload is an independent stream, a frame can opt out (`none`) when compression would not shrink it — e.g. a small JSON merge-patch delta — while larger payloads on the same Track are compressed, and different hops can use different algorithms. +- Added payload compression, scoped per group. `Publisher Compression` in TRACK_INFO is a boolean end-to-end hint that a Track's payloads are worth compressing; a new SETUP `Compression` parameter negotiates the capability per hop; and a per-group `Compression` field — in each GROUP message, datagram body, and a reintroduced FETCH_OK message (present only when the hint is set) — states whether that group is compressed. When set, the group's entire FRAME sequence is a single raw DEFLATE stream, reset at each group boundary, so the first frame stays independently decodable while later frames share the context. `0` is verbatim and `1` is DEFLATE; other schemes are left to future extensions (one in effect per hop). There is no per-frame compression. - Added Qmux [qmux] transport bindings for TCP/TLS and WebSocket, for environments where UDP is unavailable. The WebSocket binding uses the WebSocket message framing in place of the Qmux Record `Size` field. ## moq-lite-04 @@ -1213,7 +1223,7 @@ TODO: general security considerations. ## Payload Compression Compressing data that mixes attacker-controlled and secret material in the same payload can leak the secret through the compressed size, as in the CRIME and BREACH attacks. A publisher MUST NOT set a non-zero `Publisher Compression` hint on a Track whose payloads combine secret material with attacker-influenced material. -Because compression is per-frame with no cross-frame dictionary, the exposure is bounded to within a single payload, but it is not eliminated. +Because compression is group-scoped, the exposure is bounded to within a single group — which may combine several frames, a wider window than a single frame — but it is not eliminated. A malicious sender could emit a small compressed payload that decompresses to a very large buffer (a "decompression bomb"). A receiver MUST bound the size of a decompressed payload; if the bound is exceeded it MUST reset the affected stream rather than allocate unbounded memory, and MAY close the session with a PROTOCOL_VIOLATION if it considers the peer abusive. From 128af6829458a20313fae45013ae450a6f0d0e79 Mon Sep 17 00:00:00 2001 From: Claude Date: Mon, 22 Jun 2026 22:31:57 +0000 Subject: [PATCH 03/10] moq-lite: compress only frame payloads, not the framing Switch group compression from "compress the whole frame sequence" to "compress only the payloads." The payloads of a group still form one DEFLATE stream (reset per group, sliced per frame into each frame's opaque Payload), so cross-frame redundancy is retained, but the FRAME framing stays in the clear. This is for version agility and caching: with the framing uncompressed, a relay or cache can keep payloads compressed in memory, read frame metadata without inflating, and re-frame a group across transport versions (new GROUP/FRAME headers) without decompress/recompress. None of that is possible if the framing is inside the DEFLATE blob. Message Length is again the on-wire (compressed) Payload size. Also revert the moq-compression extension to its pre-PR state: its earlier enum/per-object-override design is superseded by this model and will be redone to mirror the final moq-lite design. Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_01W8bLV6vHzucLNvDhPk3bMP --- draft-lcurley-moq-compression.md | 84 +++++++++++--------------------- draft-lcurley-moq-lite.md | 23 ++++----- 2 files changed, 41 insertions(+), 66 deletions(-) diff --git a/draft-lcurley-moq-compression.md b/draft-lcurley-moq-compression.md index ce59642..13f7302 100644 --- a/draft-lcurley-moq-compression.md +++ b/draft-lcurley-moq-compression.md @@ -25,9 +25,9 @@ informative: --- abstract This document defines a payload compression extension for MoQ Transport {{moqt}}. -A track-level Compression property is a boolean hint by which the original publisher signals that a track's object payloads are worth compressing. -The algorithm is negotiated independently on each hop, and compression is applied per hop: an object is compressed only on a hop that has negotiated the extension and a shared algorithm, and is sent verbatim otherwise. -Each object is compressed independently so objects remain individually decodable, an object can opt out when compression would not help, and the decompressed bytes — the actual object — are unchanged end to end. +A track-level Compression property lets the original publisher signal that a track's object payloads are worth compressing, and with which algorithm. +Compression is then applied independently on each hop: a payload is compressed only on a hop that has negotiated the extension and whose receiver supports the algorithm, and is sent verbatim otherwise. +Each object is compressed independently so objects remain individually decodable, and the decompressed bytes — the actual object — are unchanged end to end. --- middle @@ -43,10 +43,10 @@ But MoQ also carries non-media tracks — JSON, text, telemetry, captions, uncom For these tracks there is no standard, transport-visible way to compress payloads, so each application reinvents it, and relays cannot help. Like HTTP Transfer-Encoding, the on-wire compression is a hop-by-hop optimization: it does not conceptually change the object payload — the decompressed bytes *are* the object — it only changes how those bytes are carried over a single hop. -What this extension adds on top is an end-to-end *signal*: a boolean track property by which the original publisher marks the content as worth compressing. The signal travels end to end; the choice of algorithm and the compression itself happen per hop. +What this extension adds on top is an end-to-end *signal*: a track property by which the original publisher marks the content as worth compressing and names the algorithm. The signal travels end-to-end; the compression happens per hop. -- **Publisher signals, hops apply**: the COMPRESSION track property is set by the original publisher and carried end to end, but a payload is only compressed on a hop that negotiated the extension and a shared algorithm. Where the extension is not negotiated, the same payload travels verbatim. -- **Per object, independently**: each object payload is an independent compressed stream with no shared dictionary or state between objects. This keeps every object individually decodable, avoids head-of-line decoding within a group, and lets an individual object opt out of compression when it would not benefit. +- **Publisher signals, hops apply**: the COMPRESSION track property is set by the original publisher and carried end-to-end, but a payload is only compressed on a hop that negotiated the extension and whose receiver supports the algorithm. Where the extension is not negotiated, the same payload travels verbatim. +- **Per object, independently**: each object payload is an independent compressed stream with no shared dictionary or state between objects. This keeps every object individually decodable and avoids head-of-line decoding within a group. # Setup Negotiation @@ -65,69 +65,44 @@ COMPRESSION Setup Option { **Algorithm**: One or more Algorithm identifiers (see [Compression Algorithms](#compression-algorithms)) that the sender can decompress, each a varint, filling the Option Value. -They are listed in the sender's order of preference, most-preferred first. The identifier `none` (0) MUST NOT be listed (it requires no negotiation). A sender MUST NOT compress with an algorithm the receiver did not advertise in its SETUP. -The negotiated algorithm for a hop — the **hop default** — is the first algorithm in the receiver's advertised list that the sender can also produce; if the lists do not intersect, the hop has no default and every payload travels verbatim. -This keeps the common case free of per-object signaling: where the COMPRESSION track property is present and the hop has a default, a receiver decompresses each object with the hop default unless that object carries a [per-object override](#per-object-override) naming a different algorithm (in particular `none`, to send it verbatim). Where the property is absent, the extension is not negotiated, or the algorithm lists do not intersect, the sender was not permitted to compress, so every payload is verbatim. +This makes the on-wire state unambiguous on every hop without any per-object signaling: a receiver decompresses a track's payloads **if and only if** the COMPRESSION track property is present and the receiver advertised that algorithm in its own SETUP. In every other case — the property absent, the extension not negotiated, or the algorithm not advertised by the receiver — the sender was not permitted to compress, so the receiver treats the payloads as verbatim. # COMPRESSION Track Property -The COMPRESSION property is the original publisher's signal that a track's object payloads are worth compressing. +The COMPRESSION property is the original publisher's signal that a track's object payloads are worth compressing, and which algorithm to use. It is a track-level Key-Value-Pair carried with the track's properties (see {{moqt}} Section 2.5 and Section 12), set by the original publisher and forwarded unchanged by relays. Because the value is a single integer, COMPRESSION uses an even Type so the value is a bare varint: ~~~ COMPRESSION Track Property { Type (vi64) = 0xC03D0 - Value (vi64) ; boolean hint + Value (vi64) ; Algorithm identifier } ~~~ **Value**: -A boolean hint: `1` means the track's payloads are worth compressing; `0`, or absence of the property, means they are not and are always transmitted verbatim. -Values greater than `1` are reserved for future use and MUST be treated as `1` by a receiver that does not understand them, so the hint stays additive. -The property names no algorithm: which algorithm is used, if any, is the per-hop negotiated [hop default](#setup-negotiation), overridable [per object](#per-object-override). +The Algorithm identifier the publisher recommends for this track's payloads. +The absence of the property, or a value of `none` (0), means the track is not marked for compression and its payloads are always transmitted verbatim. The property is fixed for the lifetime of the track and MUST NOT change. A relay MUST forward it unchanged on every hop, including a hop that has not negotiated the extension: there it is simply an ignored unknown Key-Value-Pair, but forwarding it lets a further-downstream hop that does negotiate the extension still act on the publisher's signal. -Compression is enabled only by the combination of a non-zero hint and the extension being negotiated with a shared algorithm on a hop. -A publisher MUST NOT compress object payloads on a track that does not carry a non-zero COMPRESSION hint. +Compression is enabled only by the combination of this track property and the extension being negotiated on a hop. +A publisher MUST NOT compress object payloads on a track that does not carry the COMPRESSION property, and there is no way to enable compression on a per-object basis: the property governs the whole track, and on a compressing hop every non-empty payload is compressed. -Whether a given object is actually compressed is decided per hop and per object: +Whether payloads are actually compressed is decided per hop: -- On a hop where the extension is negotiated and a [hop default](#setup-negotiation) exists, each non-empty object payload is compressed with the hop default and the receiver decompresses it — unless the object carries a [per-object override](#per-object-override), which names the algorithm actually used (including `none`, to send that object verbatim). -- On any other hop — the extension not negotiated, or the algorithm lists do not intersect — payloads are sent verbatim. The receiver either never sees the property (an ignored unknown Key-Value-Pair) or sees it but knows the sender was not permitted to compress for it, so it treats the payloads as verbatim either way. +- On a hop where the extension is negotiated and the receiver advertised the property's algorithm, every non-empty object payload MUST be compressed with that algorithm, and the receiver decompresses it. +- On any other hop — the extension not negotiated, or the receiver did not advertise that algorithm — payloads are sent verbatim. The receiver either never sees the property (an ignored unknown Key-Value-Pair) or sees it but knows the sender was not permitted to compress for it, so it treats the payloads as verbatim either way. -A sender SHOULD send an object verbatim (via a `none` override) whenever the hop default would not make that object smaller — for example a small JSON merge-patch delta that DEFLATE would enlarge. Compression applies to the object payload only; object properties and message framing are never compressed. -An empty payload (size 0) MUST NOT be compressed and remains empty on the wire; it needs no override. +An empty payload (size 0) MUST NOT be compressed and remains empty on the wire. A publisher SHOULD set COMPRESSION only for payload types that benefit from it. -Already-compressed media SHOULD omit it (or use `0`). - - -# Per-Object Override {#per-object-override} -The COMPRESSION_ALGORITHM property is an optional object-level Key-Value-Pair that overrides, for a single object, the algorithm a hop would otherwise apply. -Because the value is a single integer, it uses an even Type so the value is a bare varint: - -~~~ -COMPRESSION_ALGORITHM Object Property { - Type (vi64) = 0xC03D2 - Value (vi64) ; Algorithm identifier -} -~~~ - -**Value**: -The [algorithm](#compression-algorithms) actually used for this object's payload on this hop. -`none` (0) means the object is carried verbatim; any other identifier names the algorithm whose output the payload is. -A sender MUST NOT name an algorithm the receiver did not advertise in its SETUP. - -The property is meaningful only on a hop that has a [hop default](#setup-negotiation); where present it replaces the hop default for that object alone, and elsewhere objects are always verbatim and any COMPRESSION_ALGORITHM property MUST be ignored. -Unlike the boolean COMPRESSION hint, it is not the publisher's end-to-end signal: because it records what a hop actually did, a relay rewrites or removes it to reflect what it did on each downstream hop, exactly as it does for the payload bytes. -Its typical use is a `none` override that keeps an incompressible or tiny object verbatim while the rest of the track is compressed. +Already-compressed media SHOULD omit it (or use `none`). # Compression Algorithms {#compression-algorithms} @@ -144,12 +119,12 @@ There is no shared dictionary or state between objects, so each object decompres # Relay Behavior -A relay forwards the boolean COMPRESSION track property unchanged — it is the publisher's end-to-end signal — and applies compression independently on each hop. +A relay forwards the COMPRESSION track property unchanged — it is the publisher's end-to-end signal — and applies compression independently on each hop. -On its upstream subscription, the relay receives each object compressed with that hop's default unless the object carried a per-object override; it reads the [COMPRESSION_ALGORITHM](#per-object-override) property, or the hop default in its absence, to decompress as needed. -On each downstream subscription the relay serves, it compresses each object with that downstream's hop default when one exists and sends objects verbatim otherwise, rewriting or removing the per-object COMPRESSION_ALGORITHM property to reflect what it actually did on that hop. +On its upstream subscription, the relay receives payloads compressed if and only if that hop compressed them (the extension negotiated and the relay advertised the algorithm); it decompresses them as needed. +On each downstream subscription the relay serves, it compresses payloads with the track's algorithm when that downstream negotiated the extension and advertised the algorithm, and sends them verbatim otherwise. -Compression is thus driven by the publisher's hint and each hop's negotiation, not by the relay's own initiative: a relay does not compress a track the publisher did not mark. +Compression is thus driven by the publisher's track property, not by the relay: a relay does not compress a track the publisher did not mark. In every case the decompressed bytes delivered to the application MUST be identical to what the origin published. A relay or generic library MUST NOT inspect or modify the decompressed contents unless otherwise negotiated; only recompression that preserves the decompressed bytes exactly is permitted. @@ -157,20 +132,20 @@ A relay or generic library MUST NOT inspect or modify the decompressed contents # Security Considerations Compressing data that mixes attacker-controlled and secret content in the same object can leak the secret through compressed size, as in the CRIME and BREACH attacks. -A publisher MUST NOT set a non-zero COMPRESSION hint on a track whose object payloads combine secret material with attacker-influenced material. +A publisher MUST NOT set COMPRESSION on a track whose object payloads combine secret material with attacker-influenced material. Because compression here is per-object with no cross-object dictionary, the exposure is bounded to within a single object, but it is not eliminated. A malicious sender could emit a small compressed payload that decompresses to a very large buffer (a "decompression bomb"). A receiver MUST bound the size of a decompressed object payload. If the bound is exceeded it MUST reset the affected Subscribe/Fetch stream (rather than allocate unbounded memory) and MAY close the session with a PROTOCOL_VIOLATION if it considers the peer abusive; the reset is stream-scoped so a single bad object does not tear down unrelated subscriptions. -Compression is orthogonal to {{moqt}} end-to-end encryption: an encrypted payload is effectively incompressible, so a publisher using end-to-end encryption SHOULD omit COMPRESSION (or use `0`). +Compression is orthogonal to {{moqt}} end-to-end encryption: an encrypted payload is effectively incompressible, so a publisher using end-to-end encryption SHOULD omit COMPRESSION (or use `none`). # IANA Considerations This document requests the following registrations. High, distinctive values are requested to avoid the low ranges reserved by {{moqt}} and to minimize collisions with provisional registrations by other extensions; they also avoid the greasing pattern (`0x7f * N + 0x9D`). -Each Type is even so that its value is a bare varint with no length prefix (see {{moqt}} Section 2.5). +The parameter Type is even so that its value is a bare varint with no length prefix (see {{moqt}} Section 2.5). ## MOQT Setup Options @@ -182,12 +157,11 @@ This document requests a registration in the "MOQT Setup Options" registry ({{mo ## MOQT Properties -This document requests registrations in the "MOQT Properties" registry ({{moqt}} Section 15.8), used for object and track properties. +This document requests a registration in the "MOQT Properties" registry ({{moqt}} Section 15.8), used for object and track properties. -| Value | Name | Scope | Reference | -|:--------|:----------------------|:-------|:--------------| -| 0xC03D0 | COMPRESSION | Track | This Document | -| 0xC03D2 | COMPRESSION_ALGORITHM | Object | This Document | +| Value | Name | Scope | Reference | +|:--------|:------------|:------|:--------------| +| 0xC03D0 | COMPRESSION | Track | This Document | ## MOQT Compression Algorithms diff --git a/draft-lcurley-moq-lite.md b/draft-lcurley-moq-lite.md index d47bcd7..8638a61 100644 --- a/draft-lcurley-moq-lite.md +++ b/draft-lcurley-moq-lite.md @@ -309,7 +309,7 @@ When the accepted track has already ended with no matching groups there is no st A rejection is a stream reset: if the publisher cannot serve the subscription — the track does not exist, or it otherwise refuses — it MUST reset the stream rather than leave it pending, and SHOULD do so promptly (within roughly a round trip) so the subscriber is not left waiting. A subscription the publisher accepts but has no groups for yet is not a rejection: for a live track the publisher MAY withhold SUBSCRIBE_OK until the first matching group resolves the start. A subscriber therefore distinguishes "pending" from "refused" by the stream reset, not by a timeout. The Subscribe Stream does not carry the track's publisher properties — those are immutable and fetched once via a [Track Stream](#track-stream) (see [TRACK_INFO](#track-info)). -The subscriber MUST have the track's TRACK_INFO before it can parse the FRAME messages that arrive on Group Streams, since the timescale determines the frame wire format and the compression hint determines whether the stream is DEFLATE-compressed; it MAY open the Track and Subscribe streams concurrently and buffer frames until TRACK_INFO arrives. +The subscriber MUST have the track's TRACK_INFO before it can parse the FRAME messages that arrive on Group Streams, since the timescale determines the frame wire format and the compression hint determines whether frame payloads are DEFLATE-compressed; it MAY open the Track and Subscribe streams concurrently and buffer frames until TRACK_INFO arrives. The publisher sends SUBSCRIBE_OK once the absolute start group is resolved, and SUBSCRIBE_END once no further groups will be produced (see [SUBSCRIBE_OK](#subscribe-ok) and [SUBSCRIBE_END](#subscribe-end)). The publisher closes the stream (FIN) only once every group from start to end has been accounted for, either via a GROUP stream (completed or reset) or a SUBSCRIBE_DROP message. @@ -473,7 +473,7 @@ See the [Session](#session) section for how an endpoint avoids waiting on the pe A publisher creates Group Streams in response to a Subscribe Stream. A Group Stream MUST start with a GROUP message and MAY be followed by any number of FRAME messages. -When the Track is compressed on this hop (see [Compression](#compression)), the FRAME messages following the GROUP message form a single DEFLATE stream. +When the Track is compressed on this hop (see [Compression](#compression)), each frame's payload is a slice of the group's single DEFLATE stream, while the FRAME framing stays uncompressed. A Group MAY contain zero FRAME messages, potentially indicating a gap in the track. A frame MAY contain an empty payload, potentially indicating a gap in the group. @@ -884,7 +884,9 @@ moq-lite can compress payloads hop by hop, like HTTP Transfer-Encoding: the deco `Compression` is `0` (verbatim) or `1` (raw DEFLATE {{!RFC1951}}); other schemes are left to future extensions, with at most one in effect per hop. A sender MUST NOT use a non-zero value unless the receiver advertised the [Compression Parameter](#compression-parameter); it SHOULD leave a group at `0` when compression would not make it smaller — for example a one-frame group carrying a small JSON merge-patch delta that DEFLATE would enlarge. -Compression is **group-scoped**. When a group is compressed, its entire sequence of FRAME messages is a single DEFLATE stream, reset at each group boundary: on a Group Stream this is the FRAME sequence following the GROUP message; on a Fetch Stream it is the FRAME sequence following the FETCH_OK; in a datagram it is the single frame's payload. The first frame of a group is therefore independently decodable — a subscriber joining at a group boundary needs nothing earlier — while later frames in the same group share the compression context, so both framing bytes and cross-frame redundancy are compressed. There is no shared state between groups. A group with no frames carries no DEFLATE stream. +Compression is **group-scoped** but applied only to frame payloads, never to the FRAME framing. Within a group the payloads are compressed as a single DEFLATE stream — reset at each group boundary — whose output is partitioned at frame boundaries: the compressor MUST flush at the end of each frame so that frame's portion of the stream is exactly the bytes stored in its `Payload` (delimited by `Message Length`). The first frame of a group starts the stream fresh, so a subscriber joining at a group boundary needs nothing earlier, while later frames share the compression context and so retain cross-frame redundancy across the group. There is no shared state between groups; a frame with an empty payload contributes nothing to the stream. + +Compressing only the payloads, and leaving the framing in the clear, is deliberate. A relay or cache can hold the payloads compressed in memory and forward them without inflating, and can re-frame a group — for example to bridge a future transport version that changes the GROUP or FRAME headers — without touching the compressed payloads. Neither is possible if the framing is buried inside the compressed stream. A relay applies compression independently on each hop, driven by each downstream's negotiation — not by the relay's own initiative, and never on a Track the publisher did not mark. On its upstream subscription it decompresses each compressed group as needed; on each downstream subscription it sets `Compression` per group, compressing when that downstream advertised the capability and sending `0` otherwise. A relay MAY bridge protocol versions (e.g. a moq-lite-05 publisher to a moq-lite-04 subscriber) provided the decompressed bytes are identical to what the publisher produced. A relay or generic library MUST NOT inspect or modify the decompressed contents unless otherwise negotiated. @@ -1001,8 +1003,7 @@ FETCH_OK Message { ~~~ **Compression**: -The compression scheme for the fetched group, with the same meaning as in the [GROUP](#group) message: `0` verbatim, `1` raw DEFLATE. -When `1`, the FRAME messages following this FETCH_OK form a single DEFLATE stream (see [Compression](#compression)). +The compression scheme for the fetched group, with the same meaning as in the [GROUP](#group) message: `0` verbatim, `1` the frame payloads form a single raw DEFLATE stream sliced per frame (see [Compression](#compression)). ## PROBE PROBE is used to measure the available bitrate of the connection. @@ -1065,9 +1066,9 @@ This SHOULD increase by 1 for each new group. A subscriber MUST handle gaps, potentially caused by congestion. **Compression**: -The compression scheme applied to this group's frame sequence, present only when the Track's `Publisher Compression` hint is non-zero (see [TRACK_INFO](#track-info)). -`0` means the FRAME messages following this GROUP message are verbatim; `1` means they are a single raw DEFLATE stream (see [Compression](#compression)). -When the hint is `0`, the field is absent and the frames are always verbatim. +The compression scheme applied to this group's frame payloads, present only when the Track's `Publisher Compression` hint is non-zero (see [TRACK_INFO](#track-info)). +`0` means each frame `Payload` is verbatim; `1` means the payloads form a single raw DEFLATE stream, sliced per frame (see [Compression](#compression)). The FRAME framing is never compressed either way. +When the hint is `0`, the field is absent and the payloads are always verbatim. ## FRAME @@ -1083,7 +1084,7 @@ FRAME Message { `Timestamp Delta` is present only when the Track's `Publisher Timescale` (see [TRACK_INFO](#track-info)) is non-zero. When `Publisher Timescale` is 0, the field is omitted from the wire and the FRAME consists of just `Message Length` and `Payload`. -The FRAME fields are always the uncompressed contents; when the group is compressed (see [Compression](#compression)) the whole frame sequence is carried inside the group's DEFLATE stream, so `Message Length` remains the uncompressed payload size. +The framing is never compressed; only the `Payload` is (see [Compression](#compression)), so `Message Length` is always the on-wire `Payload` size. **Timestamp Delta**: A signed delta from the previous frame's timestamp, in the Track's negotiated `Timescale`. @@ -1097,7 +1098,7 @@ The first frame of a group is delta-encoded from `0`, so its `Timestamp Delta` i **Payload**: An application-specific payload. -`Message Length` is the uncompressed payload size; any compression is group-scoped (see [Compression](#compression)) and applied to the whole frame sequence on the stream, not to this field individually. +When the group is compressed (see [Compression](#compression)) this holds this frame's slice of the group's DEFLATE stream and `Message Length` is its compressed size; otherwise it is verbatim. The framing around it is never compressed. A generic library or relay MUST NOT inspect or modify the decompressed contents unless otherwise negotiated; recompression that preserves the decompressed bytes exactly is allowed (see [Compression](#compression)). @@ -1121,7 +1122,7 @@ A generic library or relay MUST NOT inspect or modify the decompressed contents - Removed `Publisher Max Latency`. The publisher's retention guarantee is no longer part of the wire format; retention for FETCH and future subscriptions is best-effort and left to the publisher. - Timestamp-based expiration replaces wall-clock arrival time when a Track timescale is negotiated. - Added QUIC datagram delivery for groups, sharing Subscribe IDs with existing subscriptions (no separate control stream). -- Added payload compression, scoped per group. `Publisher Compression` in TRACK_INFO is a boolean end-to-end hint that a Track's payloads are worth compressing; a new SETUP `Compression` parameter negotiates the capability per hop; and a per-group `Compression` field — in each GROUP message, datagram body, and a reintroduced FETCH_OK message (present only when the hint is set) — states whether that group is compressed. When set, the group's entire FRAME sequence is a single raw DEFLATE stream, reset at each group boundary, so the first frame stays independently decodable while later frames share the context. `0` is verbatim and `1` is DEFLATE; other schemes are left to future extensions (one in effect per hop). There is no per-frame compression. +- Added payload compression, scoped per group. `Publisher Compression` in TRACK_INFO is a boolean end-to-end hint that a Track's payloads are worth compressing; a new SETUP `Compression` parameter negotiates the capability per hop; and a per-group `Compression` field — in each GROUP message, datagram body, and a reintroduced FETCH_OK message (present only when the hint is set) — states whether that group is compressed. When set, the group's frame payloads (only the payloads, never the framing) form a single raw DEFLATE stream, reset at each group boundary and sliced per frame into each frame's opaque `Payload`; the first frame starts the stream fresh while later frames share the context. Keeping the framing uncompressed lets a relay store payloads compressed and re-frame across transport versions without recompressing. `0` is verbatim and `1` is DEFLATE; other schemes are left to future extensions (one in effect per hop). - Added Qmux [qmux] transport bindings for TCP/TLS and WebSocket, for environments where UDP is unavailable. The WebSocket binding uses the WebSocket message framing in place of the Qmux Record `Size` field. ## moq-lite-04 From 105273023d439bf32ff5d11a11e13d646ead16de Mon Sep 17 00:00:00 2001 From: Claude Date: Mon, 22 Jun 2026 22:53:22 +0000 Subject: [PATCH 04/10] moq-lite: drop the per-group compression toggle (flagless) With the shared per-group DEFLATE stream and the RFC 7692 trailing-bytes trim, per-frame overhead is ~1 byte, so a per-group on/off toggle isn't worth a wire field. Compression is now decided purely by the end-to-end Publisher Compression signal plus per-hop negotiation. - Publisher Compression stays an end-to-end signal ("good candidate for compression") that fans out unchanged, so a hop that doesn't compress still forwards it for a downstream hop to act on. - Remove the per-group Compression field from GROUP and datagram bodies, and remove FETCH_OK (it only carried that field). A receiver infers compression from the signal plus its own SETUP advertisement. - Spell out the RFC 7692 trim (strip the redundant 00 00 FF FF, since we frame the slices ourselves) and that the decoder keeps one context per group. Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_01W8bLV6vHzucLNvDhPk3bMP --- draft-lcurley-moq-lite.md | 62 +++++++++++++-------------------------- 1 file changed, 20 insertions(+), 42 deletions(-) diff --git a/draft-lcurley-moq-lite.md b/draft-lcurley-moq-lite.md index 8638a61..32d3cf1 100644 --- a/draft-lcurley-moq-lite.md +++ b/draft-lcurley-moq-lite.md @@ -321,7 +321,7 @@ Either endpoint MAY reset/cancel the stream at any time. A subscriber opens a Fetch Stream (0x3) to request a single Group from a Track. The subscriber sends a FETCH message containing the broadcast path, track name, priority, and group sequence. -The publisher responds on the same bidirectional stream. When the Track's `Publisher Compression` hint is non-zero (see [TRACK_INFO](#track-info)) the response begins with a [FETCH_OK](#fetch-ok) message carrying the group's `Compression` field, followed by the FRAME messages; otherwise it is bare FRAME messages with no response header. +The publisher responds with FRAME messages directly on the same bidirectional stream — there is no response header. When the Track is compressed on this hop (see [Compression](#compression)), each frame's payload is a slice of the group's DEFLATE stream, as on a Group Stream. The Subscribe ID and Group Sequence for the returned FRAME messages are implicit, taken from the original FETCH request. As with a subscription, the subscriber MUST already have the track's [TRACK_INFO](#track-info) to parse the returned frames; because the properties are immutable, a single Track Stream lookup is reused across every FETCH of that track (group-by-group fetches do not re-fetch it). The publisher FINs the stream after the last frame, or resets the stream on error. @@ -499,14 +499,12 @@ DATAGRAM Body { Subscribe ID (i) Group Sequence (i) [Timestamp (i)] - [Compression (i)] Payload (b) } ~~~ `Timestamp` is present only when the Track's `Publisher Timescale` (see [TRACK_INFO](#track-info)) is non-zero. -`Compression` is present only when the Track's `Publisher Compression` hint is non-zero (see [TRACK_INFO](#track-info)), with the same meaning as in the [GROUP](#group) message: `0` verbatim, `1` raw DEFLATE. -Each field is independently omitted when its condition does not hold; with both absent the datagram body is just `Subscribe ID`, `Group Sequence`, and `Payload`. +When `Publisher Timescale` is 0, the field is omitted from the wire and the datagram body is just `Subscribe ID`, `Group Sequence`, and `Payload`. **Subscribe ID**: The Subscribe ID of an active subscription on the same session. @@ -522,7 +520,7 @@ Any varint value (including 0) is a valid absolute timestamp. **Payload**: The frame payload, extending to the end of the datagram. -A datagram is a single-frame group: when `Compression` is `1` the payload is a single raw DEFLATE stream, and when `0` (or absent) it is verbatim (see [Compression](#compression)). +A datagram is a single-frame group: when the Track is compressed on this hop (see [Compression](#compression)) the payload is a single raw DEFLATE stream, otherwise it is verbatim. The total datagram body (including all header fields above and the compressed payload if applicable) MUST NOT exceed 1200 bytes. This limit ensures the datagram fits within the minimum QUIC path MTU without IP-layer fragmentation. Payloads that would not fit MUST be sent as a Group Stream instead. @@ -864,31 +862,32 @@ When `Publisher Timescale` is 0, the per-frame `Timestamp Delta` field is omitte Common values include `1000` (milliseconds), `1000000` (microseconds), `48000` (audio sample rate), and `90000` (RTP video clock). **Publisher Compression**: -A hint from the original publisher that this Track's payloads are worth compressing. +The original publisher's end-to-end signal that this Track's payloads are good candidates for compression. -- `0`: not worth compressing; payloads are always sent verbatim (the default). -- `1`: worth compressing; payloads MAY be compressed on any hop that has negotiated compression. +- `0`: not a compression candidate; payloads are always sent verbatim (the default). +- `1`: a compression candidate; payloads are compressed on any hop that has negotiated compression, and verbatim elsewhere. -Values greater than `1` are reserved for future use; a subscriber that does not understand a reserved value MUST treat it as `1`, so the hint stays additive and never blocks the Track. +Values greater than `1` are reserved for future use; a subscriber that does not understand a reserved value MUST treat it as `1`, so the signal stays additive and never blocks the Track. -The hint is carried end to end but by itself causes no compression: compression also requires the receiving hop to advertise the [Compression Parameter](#compression-parameter), and is applied per group and signaled by a `Compression` field (see [Compression](#compression)). When the hint is non-zero, that field is present in every GROUP message, FETCH_OK message, and datagram body; when it is `0`, the field is absent everywhere and all payloads are verbatim. +The signal is carried end to end and forwarded unchanged by relays, so even a hop that does not compress passes it along for a further-downstream hop to act on. It does not by itself cause compression: that also requires the receiving hop to advertise the [Compression Parameter](#compression-parameter), and there is no per-group flag on the wire — a receiver decompresses if and only if it holds a non-zero signal for the Track and advertised the capability itself (see [Compression](#compression)). -The publisher SHOULD set the hint only for payload types that benefit from compression (e.g. JSON, text, uncompressed binary structures); already-compressed media (e.g. H.264, Opus, AV1) gains nothing and SHOULD leave it `0`. See [Security Considerations](#security-considerations) for content that MUST NOT be marked compressible. +The publisher SHOULD set it only for payload types that benefit from compression (e.g. JSON, text, uncompressed binary structures); already-compressed media (e.g. H.264, Opus, AV1) gains nothing and SHOULD leave it `0`. See [Security Considerations](#security-considerations) for content that MUST NOT be marked compressible. ## Compression {#compression} -moq-lite can compress payloads hop by hop, like HTTP Transfer-Encoding: the decompressed bytes *are* the payload and are identical end to end, but how they are carried MAY differ on each hop — a group compressed on one hop is decompressed and MAY be re-sent verbatim on the next. Compression is governed by: +moq-lite can compress payloads hop by hop, like HTTP Transfer-Encoding: the decompressed bytes *are* the payload and are identical end to end, but how they are carried MAY differ on each hop — a group compressed on one hop is decompressed and MAY be re-sent verbatim on the next. Compression is governed by two pieces, with no per-group flag on the wire: -- the Track's `Publisher Compression` hint (see [TRACK_INFO](#track-info)) — the publisher's end-to-end signal that payloads are worth compressing; -- the per-hop [Compression Parameter](#compression-parameter) — each endpoint advertises that it can decompress; and -- a per-group `Compression` field — carried in each [GROUP](#group) message, [FETCH_OK](#fetch-ok) message, and [datagram](#datagrams) body, present only when the Track's hint is non-zero — that states whether that group is compressed and with which scheme. +- the Track's `Publisher Compression` signal (see [TRACK_INFO](#track-info)) — the original publisher's end-to-end mark that the payloads are good candidates for compression; and +- the per-hop [Compression Parameter](#compression-parameter) — each endpoint advertises that it can decompress. -`Compression` is `0` (verbatim) or `1` (raw DEFLATE {{!RFC1951}}); other schemes are left to future extensions, with at most one in effect per hop. A sender MUST NOT use a non-zero value unless the receiver advertised the [Compression Parameter](#compression-parameter); it SHOULD leave a group at `0` when compression would not make it smaller — for example a one-frame group carrying a small JSON merge-patch delta that DEFLATE would enlarge. +A hop compresses a Track's payloads when, and only when, the Track's signal is non-zero **and** the receiver advertised the Compression Parameter; otherwise they are verbatim. A receiver therefore needs no per-group flag: it decompresses if and only if it holds a non-zero signal for the Track (from cached [TRACK_INFO](#track-info)) and advertised the capability itself. Because the signal is carried end to end and forwarded unchanged, a hop that has not negotiated compression still passes it along, so a further-downstream hop that has can act on it. -Compression is **group-scoped** but applied only to frame payloads, never to the FRAME framing. Within a group the payloads are compressed as a single DEFLATE stream — reset at each group boundary — whose output is partitioned at frame boundaries: the compressor MUST flush at the end of each frame so that frame's portion of the stream is exactly the bytes stored in its `Payload` (delimited by `Message Length`). The first frame of a group starts the stream fresh, so a subscriber joining at a group boundary needs nothing earlier, while later frames share the compression context and so retain cross-frame redundancy across the group. There is no shared state between groups; a frame with an empty payload contributes nothing to the stream. +Compression is **group-scoped** and applied only to frame payloads, never to the FRAME framing. Within a group the payloads form a single raw DEFLATE {{!RFC1951}} stream, reset at each group boundary, whose output is partitioned at frame boundaries: the compressor flushes at the end of each frame so that frame's portion of the stream is exactly the bytes stored in its `Payload` (delimited by `Message Length`). Because moq-lite delimits the slices itself, the trailing four `00 00 FF FF` bytes a flush emits are redundant and MUST be removed from each `Payload`; the decoder re-inserts them before inflating, as in {{?RFC7692}}. A receiver maintains a single DEFLATE decoder per group, reset at each group boundary, and feeds each frame's `Payload` through it in order: the first frame starts the decoder fresh, so a subscriber joining at a group boundary needs nothing earlier, while later frames reuse the decoder's window and so retain cross-frame redundancy across the group. There is no shared state between groups; a frame with an empty payload contributes nothing to the stream. Compressing only the payloads, and leaving the framing in the clear, is deliberate. A relay or cache can hold the payloads compressed in memory and forward them without inflating, and can re-frame a group — for example to bridge a future transport version that changes the GROUP or FRAME headers — without touching the compressed payloads. Neither is possible if the framing is buried inside the compressed stream. -A relay applies compression independently on each hop, driven by each downstream's negotiation — not by the relay's own initiative, and never on a Track the publisher did not mark. On its upstream subscription it decompresses each compressed group as needed; on each downstream subscription it sets `Compression` per group, compressing when that downstream advertised the capability and sending `0` otherwise. A relay MAY bridge protocol versions (e.g. a moq-lite-05 publisher to a moq-lite-04 subscriber) provided the decompressed bytes are identical to what the publisher produced. A relay or generic library MUST NOT inspect or modify the decompressed contents unless otherwise negotiated. +DEFLATE is the only scheme this version defines; a future extension MAY negotiate another, in which case at most one is in effect per hop. + +A relay applies compression independently on each hop, driven by each downstream's negotiation — not by the relay's own initiative, and never on a Track the publisher did not mark. On its upstream subscription it decompresses each compressed group as needed; on each downstream subscription it compresses when that downstream advertised the capability, and sends verbatim otherwise. A relay MAY bridge protocol versions (e.g. a moq-lite-05 publisher to a moq-lite-04 subscriber) provided the decompressed bytes are identical to what the publisher produced. A relay or generic library MUST NOT inspect or modify the decompressed contents unless otherwise negotiated. ## SUBSCRIBE_OK {#subscribe-ok} A SUBSCRIBE_OK message confirms a subscription and resolves its absolute start group. @@ -985,26 +984,11 @@ See the [Prioritization](#prioritization) section for more information. **Group Sequence**: The sequence number of the group to fetch. -The publisher responds on the same stream: when the Track's `Publisher Compression` hint is non-zero the response begins with a [FETCH_OK](#fetch-ok) message, followed by the FRAME messages; otherwise it is bare FRAME messages with no response header. +The publisher responds with FRAME messages directly on the same stream — there is no response header; when the Track is compressed on this hop each frame's payload is a slice of the group's DEFLATE stream (see [Compression](#compression)). The subscriber parses them using the track's [TRACK_INFO](#track-info), which it MUST already have (see the [Track Stream](#track-stream)); the group sequence is implicit from the FETCH request. The publisher FINs the stream after the last frame, or resets on error. There is no FETCH_ERROR message — the publisher signals failure by resetting the stream. -## FETCH_OK {#fetch-ok} -FETCH_OK is the first message of a fetch response, sent by the publisher only when the Track's `Publisher Compression` hint is non-zero (see [TRACK_INFO](#track-info)). -It carries the `Compression` field for the single group being fetched; the FRAME messages follow on the same stream. -When the hint is `0` it is omitted and the response is bare FRAME messages. - -~~~ -FETCH_OK Message { - Message Length (i) - Compression (i) -} -~~~ - -**Compression**: -The compression scheme for the fetched group, with the same meaning as in the [GROUP](#group) message: `0` verbatim, `1` the frame payloads form a single raw DEFLATE stream sliced per frame (see [Compression](#compression)). - ## PROBE PROBE is used to measure the available bitrate of the connection. @@ -1052,7 +1036,6 @@ GROUP Message { Message Length (i) Subscribe ID (i) Group Sequence (i) - [Compression (i)] } ~~~ @@ -1065,11 +1048,6 @@ The sequence number of the group. This SHOULD increase by 1 for each new group. A subscriber MUST handle gaps, potentially caused by congestion. -**Compression**: -The compression scheme applied to this group's frame payloads, present only when the Track's `Publisher Compression` hint is non-zero (see [TRACK_INFO](#track-info)). -`0` means each frame `Payload` is verbatim; `1` means the payloads form a single raw DEFLATE stream, sliced per frame (see [Compression](#compression)). The FRAME framing is never compressed either way. -When the hint is `0`, the field is absent and the payloads are always verbatim. - ## FRAME The FRAME message is a payload within a group. @@ -1110,7 +1088,7 @@ A generic library or relay MUST NOT inspect or modify the decompressed contents - Added a SETUP message, sent once on a unidirectional Setup Stream (0x1) at the start of the session and FIN'd immediately. It carries a list of Setup Parameters for negotiating optional capabilities and extensions per-hop, replacing the prior stream-probing approach (version is still negotiated via ALPN, not SETUP). Endpoints keep exchanging non-Setup streams without waiting for SETUP, buffering only a stream whose encoding a negotiated extension would change; unknown stream types are still reset as a fallback. - Added a SETUP `Probe` parameter advertising the publisher's capability level: `None`, `Report` (measure and report the estimated bitrate), or `Increase` (additionally pad to probe for bandwidth above the current sending rate). The levels are nested since probing without measuring is meaningless. A subscriber must not rely on a level the publisher did not advertise. - Added a Track Stream (0x6): a TRACK request that the publisher answers with a single TRACK_INFO message and then FINs. TRACK_INFO carries the Track's immutable publisher properties (`Publisher Priority`, `Publisher Ordered`, `Publisher Timescale`, `Publisher Compression`). It is fetched once and cached, so the properties are no longer echoed on every response — notably, group-by-group FETCHes reuse one lookup. -- Removed FETCH_OK and trimmed SUBSCRIBE_OK down to a single resolved start group. Publisher properties moved to TRACK_INFO; a FETCH returns bare FRAME messages (a minimal FETCH_OK is reintroduced only for compressible tracks, to carry the per-group compression flag; see the compression entry). All publisher properties are immutable for the lifetime of the Track — a publisher-side change would otherwise have to fan *out* to every downstream of a relay, whereas subscriber properties fan *in* and may still change via SUBSCRIBE_UPDATE. +- Removed FETCH_OK and trimmed SUBSCRIBE_OK down to a single resolved start group. Publisher properties moved to TRACK_INFO; a FETCH returns bare FRAME messages. All publisher properties are immutable for the lifetime of the Track — a publisher-side change would otherwise have to fan *out* to every downstream of a relay, whereas subscriber properties fan *in* and may still change via SUBSCRIBE_UPDATE. - Split the resolved group range across SUBSCRIBE_OK and a new SUBSCRIBE_END. SUBSCRIBE_OK resolves the absolute start (`>=` the requested start; a larger value implicitly drops the leading range), and SUBSCRIBE_END signals that no group will follow a given sequence (stragglers within the range may still be dropped before FIN). SUBSCRIBE_OK keeps the MoqTransport name and its role as the publisher's positive response. - Renamed `Start Group`/`End Group` to `Group Start`/`Group End` in SUBSCRIBE, SUBSCRIBE_UPDATE, and SUBSCRIBE_DROP for consistency with the entity-first naming used elsewhere (e.g. `Group Sequence`). Wire format unchanged. - Allowed a duplicate `active` ANNOUNCE_BROADCAST to atomically replace the prior advertisement (equivalent to UNANNOUNCE+ANNOUNCE_BROADCAST). Used when only the origin or hop path changes (e.g. relay failover) without interrupting the broadcast. No new wire enum value — the existing `active` status carries the new metadata. @@ -1122,7 +1100,7 @@ A generic library or relay MUST NOT inspect or modify the decompressed contents - Removed `Publisher Max Latency`. The publisher's retention guarantee is no longer part of the wire format; retention for FETCH and future subscriptions is best-effort and left to the publisher. - Timestamp-based expiration replaces wall-clock arrival time when a Track timescale is negotiated. - Added QUIC datagram delivery for groups, sharing Subscribe IDs with existing subscriptions (no separate control stream). -- Added payload compression, scoped per group. `Publisher Compression` in TRACK_INFO is a boolean end-to-end hint that a Track's payloads are worth compressing; a new SETUP `Compression` parameter negotiates the capability per hop; and a per-group `Compression` field — in each GROUP message, datagram body, and a reintroduced FETCH_OK message (present only when the hint is set) — states whether that group is compressed. When set, the group's frame payloads (only the payloads, never the framing) form a single raw DEFLATE stream, reset at each group boundary and sliced per frame into each frame's opaque `Payload`; the first frame starts the stream fresh while later frames share the context. Keeping the framing uncompressed lets a relay store payloads compressed and re-frame across transport versions without recompressing. `0` is verbatim and `1` is DEFLATE; other schemes are left to future extensions (one in effect per hop). +- Added payload compression, scoped per group. `Publisher Compression` in TRACK_INFO is the original publisher's end-to-end signal that a Track's payloads are good candidates for compression; a new SETUP `Compression` parameter negotiates the capability per hop. A hop compresses when, and only when, the signal is set and the receiver advertised the parameter — there is no per-group or per-frame flag on the wire. When compressed, a group's frame payloads (only the payloads, never the framing) form a single raw DEFLATE stream, reset at each group boundary and sliced per frame into each frame's opaque `Payload`; the redundant trailing `00 00 FF FF` of each flush is stripped (as in RFC 7692) since moq-lite frames the slices itself, and the decoder keeps one context per group. Keeping the framing uncompressed lets a relay store payloads compressed and re-frame across transport versions without recompressing. DEFLATE is the only scheme; others are left to future extensions (one in effect per hop). - Added Qmux [qmux] transport bindings for TCP/TLS and WebSocket, for environments where UDP is unavailable. The WebSocket binding uses the WebSocket message framing in place of the Qmux Record `Size` field. ## moq-lite-04 From 0d7a1f26a5df9d7a1139a16eac92016b1ad0c65c Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 23 Jun 2026 00:34:52 +0000 Subject: [PATCH 05/10] compression: negotiate deflate/zstd per hop; rewrite moq-compression to match MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Add a second algorithm (zstd) and per-hop algorithm negotiation, now that browsers do zstd natively and its dictionary support suits MoQ's small, repetitive non-media payloads. deflate stays the mandatory baseline. Negotiation (both drafts): each endpoint advertises the algorithms it can de/compress in preference order; for a direction, the algorithm is the first in the receiver's list that the sender also lists. Both ends compute it identically from the two advertised lists, so a simultaneous SETUP can't disagree; the two directions are independent and may differ. deflate is mandatory, so a common algorithm always exists. moq-lite: the SETUP Compression parameter carries the ordered list (was a presence flag); the Compression section gains the algorithm table and per-algorithm framing trims (RFC 7692 for deflate; magicless, checksum-less frames for zstd). Still flagless and payload-only. moq-compression: rewritten to the same model — boolean COMPRESSION track Property (was an algorithm id) + COMPRESSION Setup Option carrying the ordered list + first-intersection selection + flagless inference + subgroup-scoped sliced stream. Drops the per-object override; the algorithms registry gains zstd. Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_01W8bLV6vHzucLNvDhPk3bMP --- draft-lcurley-moq-compression.md | 102 ++++++++++++++++++------------- draft-lcurley-moq-lite.md | 52 ++++++++++------ 2 files changed, 90 insertions(+), 64 deletions(-) diff --git a/draft-lcurley-moq-compression.md b/draft-lcurley-moq-compression.md index 13f7302..876c4e3 100644 --- a/draft-lcurley-moq-compression.md +++ b/draft-lcurley-moq-compression.md @@ -19,15 +19,17 @@ author: normative: moqt: I-D.ietf-moq-transport RFC1951: + RFC8878: informative: + RFC7692: --- abstract This document defines a payload compression extension for MoQ Transport {{moqt}}. -A track-level Compression property lets the original publisher signal that a track's object payloads are worth compressing, and with which algorithm. -Compression is then applied independently on each hop: a payload is compressed only on a hop that has negotiated the extension and whose receiver supports the algorithm, and is sent verbatim otherwise. -Each object is compressed independently so objects remain individually decodable, and the decompressed bytes — the actual object — are unchanged end to end. +A track-level Compression property is a boolean hint by which the original publisher marks a track's object payloads as good candidates for compression. +The algorithm is negotiated independently on each hop, and compression is applied only on a hop where the publisher's hint is set and the sender and receiver share an algorithm; otherwise payloads travel verbatim. +Compression is scoped to a subgroup: the object payloads of a subgroup form one compressed stream, sliced back into the individual object payloads so the object framing stays in the clear and the decompressed bytes — the actual objects — are unchanged end to end. --- middle @@ -43,17 +45,17 @@ But MoQ also carries non-media tracks — JSON, text, telemetry, captions, uncom For these tracks there is no standard, transport-visible way to compress payloads, so each application reinvents it, and relays cannot help. Like HTTP Transfer-Encoding, the on-wire compression is a hop-by-hop optimization: it does not conceptually change the object payload — the decompressed bytes *are* the object — it only changes how those bytes are carried over a single hop. -What this extension adds on top is an end-to-end *signal*: a track property by which the original publisher marks the content as worth compressing and names the algorithm. The signal travels end-to-end; the compression happens per hop. +What this extension adds is an end-to-end *signal* — a boolean track property by which the original publisher marks the content as worth compressing — plus a per-hop negotiation of the algorithm. -- **Publisher signals, hops apply**: the COMPRESSION track property is set by the original publisher and carried end-to-end, but a payload is only compressed on a hop that negotiated the extension and whose receiver supports the algorithm. Where the extension is not negotiated, the same payload travels verbatim. -- **Per object, independently**: each object payload is an independent compressed stream with no shared dictionary or state between objects. This keeps every object individually decodable and avoids head-of-line decoding within a group. +- **Publisher signals, hops apply**: the COMPRESSION track property is a boolean set by the original publisher and carried end to end, but a payload is only compressed on a hop that has negotiated a shared algorithm. Which algorithm is used is decided per hop, not by the publisher; a hop that has not negotiated compression forwards the property unchanged so a further-downstream hop can still act on it. +- **Per subgroup, sliced into objects**: within a subgroup the object payloads form one compressed stream, flushed at each object boundary so every object still carries its own payload slice, while the object headers and framing stay in the clear. This keeps the subgroup — the unit a receiver already takes as one ordered, reliable stream — as the unit of compression, while letting relays and caches store payloads compressed and re-frame them without recompressing. # Setup Negotiation The Payload Compression extension is negotiated during the SETUP exchange as defined in {{moqt}} Section 10.3. Unlike a purely additive property, compression MUST be negotiated: a receiver that does not understand the algorithm would otherwise pass the compressed bytes to the application as if they were plaintext. -Each endpoint advertises the algorithms it can decompress by including the following Setup Option: +Each endpoint advertises the algorithms it can decompress, in preference order (most-preferred first), by including the following Setup Option: ~~~ COMPRESSION Setup Option { @@ -64,88 +66,99 @@ COMPRESSION Setup Option { ~~~ **Algorithm**: -One or more Algorithm identifiers (see [Compression Algorithms](#compression-algorithms)) that the sender can decompress, each a varint, filling the Option Value. -The identifier `none` (0) MUST NOT be listed (it requires no negotiation). +One or more Algorithm identifiers (see [Compression Algorithms](#compression-algorithms)) that the sender can compress and decompress, each a varint, filling the Option Value, most-preferred first. +An endpoint that includes this option MUST list `deflate` (1); the identifier `none` (0) MUST NOT be listed (it requires no negotiation). +An endpoint that does not support the extension omits the option. -A sender MUST NOT compress with an algorithm the receiver did not advertise in its SETUP. -This makes the on-wire state unambiguous on every hop without any per-object signaling: a receiver decompresses a track's payloads **if and only if** the COMPRESSION track property is present and the receiver advertised that algorithm in its own SETUP. In every other case — the property absent, the extension not negotiated, or the algorithm not advertised by the receiver — the sender was not permitted to compress, so the receiver treats the payloads as verbatim. +Negotiation is per direction and per hop. For a given direction sender-to-receiver, the **selected algorithm** is the first identifier in the receiver's advertised list that also appears in the sender's advertised list; if the lists do not intersect, that direction is verbatim. +Because each endpoint holds both advertised lists once SETUP has been exchanged, both compute the same selection with no further handshake — the receiver's preference governs its own inbound direction, the two directions are independent and MAY select different algorithms, and a simultaneous SETUP exchange creates no ambiguity. +Since `deflate` is mandatory for any endpoint that advertises the option, two endpoints that both support the extension always share at least one algorithm. +A sender MUST NOT compress with an algorithm the receiver did not advertise. # COMPRESSION Track Property -The COMPRESSION property is the original publisher's signal that a track's object payloads are worth compressing, and which algorithm to use. +The COMPRESSION property is the original publisher's end-to-end signal that a track's object payloads are good candidates for compression. It is a track-level Key-Value-Pair carried with the track's properties (see {{moqt}} Section 2.5 and Section 12), set by the original publisher and forwarded unchanged by relays. Because the value is a single integer, COMPRESSION uses an even Type so the value is a bare varint: ~~~ COMPRESSION Track Property { Type (vi64) = 0xC03D0 - Value (vi64) ; Algorithm identifier + Value (vi64) ; boolean hint } ~~~ **Value**: -The Algorithm identifier the publisher recommends for this track's payloads. -The absence of the property, or a value of `none` (0), means the track is not marked for compression and its payloads are always transmitted verbatim. +A boolean hint: `1` means the track's payloads are good candidates for compression, `0` (or absence of the property) means they are not and are always transmitted verbatim. +Values greater than `1` are reserved for future use and MUST be treated as `1` by a receiver that does not understand them, so the hint stays additive. +The property names no algorithm; which algorithm is used, if any, is the per-hop [selected algorithm](#setup-negotiation). The property is fixed for the lifetime of the track and MUST NOT change. A relay MUST forward it unchanged on every hop, including a hop that has not negotiated the extension: there it is simply an ignored unknown Key-Value-Pair, but forwarding it lets a further-downstream hop that does negotiate the extension still act on the publisher's signal. -Compression is enabled only by the combination of this track property and the extension being negotiated on a hop. -A publisher MUST NOT compress object payloads on a track that does not carry the COMPRESSION property, and there is no way to enable compression on a per-object basis: the property governs the whole track, and on a compressing hop every non-empty payload is compressed. +Compression is enabled only by the combination of a non-zero hint and a shared algorithm being negotiated on a hop; there is no per-object or per-subgroup signal on the wire. +A receiver decompresses a track's object payloads **if and only if** the COMPRESSION hint is non-zero and the hop selected an algorithm for that direction, in which case it uses the selected algorithm. +In every other case — the hint absent or zero, the extension not negotiated, or the lists not intersecting — payloads are verbatim. -Whether payloads are actually compressed is decided per hop: +A publisher SHOULD set COMPRESSION only for payload types that benefit from it (e.g. JSON, text, uncompressed binary structures). +Already-compressed media SHOULD omit it (or use `0`). -- On a hop where the extension is negotiated and the receiver advertised the property's algorithm, every non-empty object payload MUST be compressed with that algorithm, and the receiver decompresses it. -- On any other hop — the extension not negotiated, or the receiver did not advertise that algorithm — payloads are sent verbatim. The receiver either never sees the property (an ignored unknown Key-Value-Pair) or sees it but knows the sender was not permitted to compress for it, so it treats the payloads as verbatim either way. -Compression applies to the object payload only; object properties and message framing are never compressed. -An empty payload (size 0) MUST NOT be compressed and remains empty on the wire. +# Compression {#compression} +Compression is applied to object payloads only — object headers, properties, and message framing are never compressed — and is **scoped to a subgroup**. +Within a subgroup the object payloads form a single compressed stream in the [selected algorithm](#setup-negotiation), reset at each subgroup boundary. +The stream's output is partitioned at object boundaries: the compressor flushes at the end of each object so that object's slice is exactly the bytes carried as its payload, and the payload length in the object header gives the on-wire (compressed) slice size. +Both algorithms provide a window-retaining flush (DEFLATE's sync flush; Zstandard's `ZSTD_e_flush`), so later objects in a subgroup reuse the compression context and retain cross-object redundancy. -A publisher SHOULD set COMPRESSION only for payload types that benefit from it. -Already-compressed media SHOULD omit it (or use `none`). +A receiver maintains a single decoder per subgroup, reset at each subgroup boundary, and feeds each object's payload through it in order: the first object of a subgroup starts the decoder fresh — so a receiver joining at a group boundary needs nothing earlier — while later objects build on it. +There is no shared state between subgroups; an empty payload (size 0) contributes nothing to the stream and remains empty on the wire. +An object delivered as a datagram is a single-object stream, compressed on its own. +Because the object framing already delimits each slice, an algorithm's own redundant boundary and container bytes are omitted: for `deflate`, the trailing four `00 00 FF FF` bytes a sync flush emits are removed from each payload and the decoder re-inserts them (as in {{RFC7692}}); for `zstd`, the per-subgroup stream uses the magicless frame format and omits the content checksum. -# Compression Algorithms {#compression-algorithms} +Leaving the framing uncompressed is deliberate. +A relay or cache can hold the object payloads compressed in memory and forward them without inflating, and can re-frame a subgroup — for example to bridge a transport version that changes the subgroup or object headers — without touching the compressed payloads. +Neither is possible if the framing is buried inside the compressed stream. + +## Compression Algorithms {#compression-algorithms} This document defines the following algorithms. -Further algorithms MAY be registered (see [IANA Considerations](#iana-considerations)). -| ID | Name | Description | -|---:|:--------|:--------------------------------------------------------| -| 0 | none | Payloads are transmitted verbatim. The default. | -| 1 | deflate | Raw DEFLATE {{RFC1951}}, with no zlib or gzip framing. | +| ID | Name | Requirement | Description | +|---:|:--------|:------------|:--------------------------------------------------------| +| 0 | none | — | Verbatim; the absence of compression. Never advertised. | +| 1 | deflate | mandatory | Raw DEFLATE {{RFC1951}}, with no zlib or gzip framing. | +| 2 | zstd | optional | Zstandard {{RFC8878}}. | -For `deflate`, each object payload is an independent raw DEFLATE stream. -There is no shared dictionary or state between objects, so each object decompresses on its own. +Every endpoint that advertises this extension MUST implement `deflate`, so a common algorithm always exists; `zstd` is optional. +Further algorithms MAY be registered (see [IANA Considerations](#iana-considerations)). # Relay Behavior -A relay forwards the COMPRESSION track property unchanged — it is the publisher's end-to-end signal — and applies compression independently on each hop. +A relay forwards the boolean COMPRESSION track property unchanged — it is the publisher's end-to-end signal — and applies compression independently on each hop, driven by each hop's negotiation rather than by its own initiative; a relay does not compress a track the publisher did not mark. -On its upstream subscription, the relay receives payloads compressed if and only if that hop compressed them (the extension negotiated and the relay advertised the algorithm); it decompresses them as needed. -On each downstream subscription the relay serves, it compresses payloads with the track's algorithm when that downstream negotiated the extension and advertised the algorithm, and sends them verbatim otherwise. - -Compression is thus driven by the publisher's track property, not by the relay: a relay does not compress a track the publisher did not mark. +On its upstream subscription the relay receives each subgroup compressed with that hop's selected algorithm (or verbatim) and decompresses as needed. +On each downstream subscription it (re)compresses each subgroup with that downstream's selected algorithm, or sends it verbatim when no algorithm is shared. +A relay MAY transcode between algorithms. In every case the decompressed bytes delivered to the application MUST be identical to what the origin published. - A relay or generic library MUST NOT inspect or modify the decompressed contents unless otherwise negotiated; only recompression that preserves the decompressed bytes exactly is permitted. # Security Considerations -Compressing data that mixes attacker-controlled and secret content in the same object can leak the secret through compressed size, as in the CRIME and BREACH attacks. -A publisher MUST NOT set COMPRESSION on a track whose object payloads combine secret material with attacker-influenced material. -Because compression here is per-object with no cross-object dictionary, the exposure is bounded to within a single object, but it is not eliminated. +Compressing data that mixes attacker-controlled and secret content can leak the secret through compressed size, as in the CRIME and BREACH attacks. +A publisher MUST NOT set a non-zero COMPRESSION hint on a track whose object payloads combine secret material with attacker-influenced material. +Because compression is scoped to a subgroup, the exposure is bounded to within a single subgroup — which may combine several objects, a wider window than a single object — but it is not eliminated. A malicious sender could emit a small compressed payload that decompresses to a very large buffer (a "decompression bomb"). -A receiver MUST bound the size of a decompressed object payload. If the bound is exceeded it MUST reset the affected Subscribe/Fetch stream (rather than allocate unbounded memory) and MAY close the session with a PROTOCOL_VIOLATION if it considers the peer abusive; the reset is stream-scoped so a single bad object does not tear down unrelated subscriptions. +A receiver MUST bound the size of a decompressed object payload. If the bound is exceeded it MUST reset the affected stream (rather than allocate unbounded memory) and MAY close the session with a PROTOCOL_VIOLATION if it considers the peer abusive; the reset is stream-scoped so a single bad subgroup does not tear down unrelated subscriptions. -Compression is orthogonal to {{moqt}} end-to-end encryption: an encrypted payload is effectively incompressible, so a publisher using end-to-end encryption SHOULD omit COMPRESSION (or use `none`). +Compression is orthogonal to {{moqt}} end-to-end encryption: an encrypted payload is effectively incompressible, so a publisher using end-to-end encryption SHOULD omit COMPRESSION (or use `0`). # IANA Considerations This document requests the following registrations. High, distinctive values are requested to avoid the low ranges reserved by {{moqt}} and to minimize collisions with provisional registrations by other extensions; they also avoid the greasing pattern (`0x7f * N + 0x9D`). -The parameter Type is even so that its value is a bare varint with no length prefix (see {{moqt}} Section 2.5). +Each Type is even so that its value is a bare varint with no length prefix (see {{moqt}} Section 2.5). ## MOQT Setup Options @@ -172,6 +185,7 @@ The initial contents are: |---:|:--------|:--------------| | 0 | none | This Document | | 1 | deflate | This Document | +| 2 | zstd | This Document | --- back diff --git a/draft-lcurley-moq-lite.md b/draft-lcurley-moq-lite.md index 32d3cf1..190a6a5 100644 --- a/draft-lcurley-moq-lite.md +++ b/draft-lcurley-moq-lite.md @@ -309,7 +309,7 @@ When the accepted track has already ended with no matching groups there is no st A rejection is a stream reset: if the publisher cannot serve the subscription — the track does not exist, or it otherwise refuses — it MUST reset the stream rather than leave it pending, and SHOULD do so promptly (within roughly a round trip) so the subscriber is not left waiting. A subscription the publisher accepts but has no groups for yet is not a rejection: for a live track the publisher MAY withhold SUBSCRIBE_OK until the first matching group resolves the start. A subscriber therefore distinguishes "pending" from "refused" by the stream reset, not by a timeout. The Subscribe Stream does not carry the track's publisher properties — those are immutable and fetched once via a [Track Stream](#track-stream) (see [TRACK_INFO](#track-info)). -The subscriber MUST have the track's TRACK_INFO before it can parse the FRAME messages that arrive on Group Streams, since the timescale determines the frame wire format and the compression hint determines whether frame payloads are DEFLATE-compressed; it MAY open the Track and Subscribe streams concurrently and buffer frames until TRACK_INFO arrives. +The subscriber MUST have the track's TRACK_INFO before it can parse the FRAME messages that arrive on Group Streams, since the timescale determines the frame wire format and the compression signal determines whether frame payloads are compressed; it MAY open the Track and Subscribe streams concurrently and buffer frames until TRACK_INFO arrives. The publisher sends SUBSCRIBE_OK once the absolute start group is resolved, and SUBSCRIBE_END once no further groups will be produced (see [SUBSCRIBE_OK](#subscribe-ok) and [SUBSCRIBE_END](#subscribe-end)). The publisher closes the stream (FIN) only once every group from start to end has been accounted for, either via a GROUP stream (completed or reset) or a SUBSCRIBE_DROP message. @@ -321,7 +321,7 @@ Either endpoint MAY reset/cancel the stream at any time. A subscriber opens a Fetch Stream (0x3) to request a single Group from a Track. The subscriber sends a FETCH message containing the broadcast path, track name, priority, and group sequence. -The publisher responds with FRAME messages directly on the same bidirectional stream — there is no response header. When the Track is compressed on this hop (see [Compression](#compression)), each frame's payload is a slice of the group's DEFLATE stream, as on a Group Stream. +The publisher responds with FRAME messages directly on the same bidirectional stream — there is no response header. When the Track is compressed on this hop (see [Compression](#compression)), each frame's payload is a slice of the group's compressed stream, as on a Group Stream. The Subscribe ID and Group Sequence for the returned FRAME messages are implicit, taken from the original FETCH request. As with a subscription, the subscriber MUST already have the track's [TRACK_INFO](#track-info) to parse the returned frames; because the properties are immutable, a single Track Stream lookup is reused across every FETCH of that track (group-by-group fetches do not re-fetch it). The publisher FINs the stream after the last frame, or resets the stream on error. @@ -473,7 +473,7 @@ See the [Session](#session) section for how an endpoint avoids waiting on the pe A publisher creates Group Streams in response to a Subscribe Stream. A Group Stream MUST start with a GROUP message and MAY be followed by any number of FRAME messages. -When the Track is compressed on this hop (see [Compression](#compression)), each frame's payload is a slice of the group's single DEFLATE stream, while the FRAME framing stays uncompressed. +When the Track is compressed on this hop (see [Compression](#compression)), each frame's payload is a slice of the group's single compressed stream, while the FRAME framing stays uncompressed. A Group MAY contain zero FRAME messages, potentially indicating a gap in the track. A frame MAY contain an empty payload, potentially indicating a gap in the group. @@ -520,7 +520,7 @@ Any varint value (including 0) is a valid absolute timestamp. **Payload**: The frame payload, extending to the end of the datagram. -A datagram is a single-frame group: when the Track is compressed on this hop (see [Compression](#compression)) the payload is a single raw DEFLATE stream, otherwise it is verbatim. +A datagram is a single-frame group: when the Track is compressed on this hop (see [Compression](#compression)) the payload is a single compressed stream in the negotiated algorithm, otherwise it is verbatim. The total datagram body (including all header fields above and the compressed payload if applicable) MUST NOT exceed 1200 bytes. This limit ensures the datagram fits within the minimum QUIC path MTU without IP-layer fragmentation. Payloads that would not fit MUST be sent as a Group Stream instead. @@ -594,7 +594,7 @@ The following Setup Parameters are defined: |------|-------------|-------------------| | 0x2 | Path | Path (s) | |------|-------------|-------------------| -| 0x3 | Compression | (empty) | +| 0x3 | Compression | Algorithm (i) ... | |------|-------------|-------------------| ### Probe Parameter {#probe-parameter} @@ -631,11 +631,13 @@ The remaining bindings convey the path in their own handshake. A relay MUST NOT forward the Path Parameter; like other per-hop setup metadata it applies only to this hop (see [Session](#session)). ### Compression Parameter {#compression-parameter} -The Compression Parameter advertises that the sender can *decompress* payloads on this hop (see [Compression](#compression)). -Its Parameter Value is empty; presence alone is the capability. +The Compression Parameter advertises the payload compression [algorithms](#compression) the sender can *decompress* on this hop, in preference order (most-preferred first). +The Parameter Value is a sequence of algorithm identifiers, each a variable-length integer, packed back-to-back to fill the Parameter Length. +An endpoint that supports compression MUST include `deflate` (1); `none` (0) MUST NOT be listed. An endpoint that does not support compression omits the parameter. -Negotiation is per direction and per hop: an endpoint's advertisement governs only what may be compressed when sending *to* it, and a relay MUST NOT forward the parameter (see [Session](#session)). -A sender MUST NOT compress when sending to a receiver that did not advertise this parameter, so an endpoint that omits it receives every Group Stream, Fetch Stream, and datagram verbatim regardless of the Track's `Publisher Compression` hint. +Negotiation is per direction and per hop, and a relay MUST NOT forward the parameter (see [Session](#session)). +For a given direction, the algorithm is the first entry in the **receiver's** list that also appears in the **sender's** list: the receiver's preference governs, and because both endpoints hold both lists once SETUP has been exchanged, each computes the same result with no further handshake. The two directions are independent and MAY select different algorithms. Since `deflate` is mandatory for any endpoint that supports compression, two compression-capable endpoints always share at least one algorithm. +A sender MUST NOT compress when sending to a receiver that advertised no list, so such a receiver gets every Group Stream, Fetch Stream, and datagram verbatim regardless of the Track's `Publisher Compression` signal. ## ANNOUNCE_REQUEST @@ -869,25 +871,35 @@ The original publisher's end-to-end signal that this Track's payloads are good c Values greater than `1` are reserved for future use; a subscriber that does not understand a reserved value MUST treat it as `1`, so the signal stays additive and never blocks the Track. -The signal is carried end to end and forwarded unchanged by relays, so even a hop that does not compress passes it along for a further-downstream hop to act on. It does not by itself cause compression: that also requires the receiving hop to advertise the [Compression Parameter](#compression-parameter), and there is no per-group flag on the wire — a receiver decompresses if and only if it holds a non-zero signal for the Track and advertised the capability itself (see [Compression](#compression)). +The signal is carried end to end and forwarded unchanged by relays, so even a hop that does not compress passes it along for a further-downstream hop to act on. It does not by itself cause compression: that also requires the receiving hop to advertise a shared algorithm via the [Compression Parameter](#compression-parameter), and there is no per-group flag on the wire — a receiver decompresses if and only if it holds a non-zero signal for the Track and the hop selected a shared algorithm (see [Compression](#compression)). The publisher SHOULD set it only for payload types that benefit from compression (e.g. JSON, text, uncompressed binary structures); already-compressed media (e.g. H.264, Opus, AV1) gains nothing and SHOULD leave it `0`. See [Security Considerations](#security-considerations) for content that MUST NOT be marked compressible. ## Compression {#compression} -moq-lite can compress payloads hop by hop, like HTTP Transfer-Encoding: the decompressed bytes *are* the payload and are identical end to end, but how they are carried MAY differ on each hop — a group compressed on one hop is decompressed and MAY be re-sent verbatim on the next. Compression is governed by two pieces, with no per-group flag on the wire: +moq-lite can compress payloads hop by hop, like HTTP Transfer-Encoding: the decompressed bytes *are* the payload and are identical end to end, but how they are carried MAY differ on each hop — a group compressed on one hop is decompressed and MAY be re-sent verbatim, or recompressed with a different algorithm, on the next. Compression is governed by two pieces, with no per-group flag on the wire: - the Track's `Publisher Compression` signal (see [TRACK_INFO](#track-info)) — the original publisher's end-to-end mark that the payloads are good candidates for compression; and -- the per-hop [Compression Parameter](#compression-parameter) — each endpoint advertises that it can decompress. +- the per-hop [Compression Parameter](#compression-parameter) — each endpoint advertises, in preference order, the algorithms it can decompress. -A hop compresses a Track's payloads when, and only when, the Track's signal is non-zero **and** the receiver advertised the Compression Parameter; otherwise they are verbatim. A receiver therefore needs no per-group flag: it decompresses if and only if it holds a non-zero signal for the Track (from cached [TRACK_INFO](#track-info)) and advertised the capability itself. Because the signal is carried end to end and forwarded unchanged, a hop that has not negotiated compression still passes it along, so a further-downstream hop that has can act on it. +A hop compresses a Track's payloads when, and only when, the Track's signal is non-zero **and** the sender and receiver share an algorithm (the [Compression Parameter](#compression-parameter) defines how the single per-direction algorithm is selected); otherwise they are verbatim. A receiver needs no per-group signal — neither a "compressed" flag nor an algorithm identifier — because it derives both from the Track's signal and the deterministic per-direction selection. Because the signal is carried end to end and forwarded unchanged, a hop that has not negotiated compression still passes it along, so a further-downstream hop that has can act on it. -Compression is **group-scoped** and applied only to frame payloads, never to the FRAME framing. Within a group the payloads form a single raw DEFLATE {{!RFC1951}} stream, reset at each group boundary, whose output is partitioned at frame boundaries: the compressor flushes at the end of each frame so that frame's portion of the stream is exactly the bytes stored in its `Payload` (delimited by `Message Length`). Because moq-lite delimits the slices itself, the trailing four `00 00 FF FF` bytes a flush emits are redundant and MUST be removed from each `Payload`; the decoder re-inserts them before inflating, as in {{?RFC7692}}. A receiver maintains a single DEFLATE decoder per group, reset at each group boundary, and feeds each frame's `Payload` through it in order: the first frame starts the decoder fresh, so a subscriber joining at a group boundary needs nothing earlier, while later frames reuse the decoder's window and so retain cross-frame redundancy across the group. There is no shared state between groups; a frame with an empty payload contributes nothing to the stream. +The following algorithms are defined: -Compressing only the payloads, and leaving the framing in the clear, is deliberate. A relay or cache can hold the payloads compressed in memory and forward them without inflating, and can re-frame a group — for example to bridge a future transport version that changes the GROUP or FRAME headers — without touching the compressed payloads. Neither is possible if the framing is buried inside the compressed stream. +| ID | Name | Requirement | Description | +|---:|:--------|:------------|:--------------------------------------------------------| +| 0 | none | — | Verbatim; the absence of compression. Never advertised. | +| 1 | deflate | mandatory | Raw DEFLATE {{!RFC1951}}, with no zlib or gzip framing. | +| 2 | zstd | optional | Zstandard {{!RFC8878}}. | + +Every endpoint that supports compression MUST implement `deflate`, so a common algorithm always exists; `zstd` is optional, and further algorithms MAY be defined by future extensions. + +Compression is **group-scoped** and applied only to frame payloads, never to the FRAME framing. Within a group the payloads form a single compressed stream in the negotiated algorithm, reset at each group boundary, whose output is partitioned at frame boundaries: the compressor flushes at the end of each frame so that frame's slice is exactly the bytes stored in its `Payload` (delimited by `Message Length`). Both algorithms provide such a flush that retains the window (DEFLATE's sync flush; Zstandard's `ZSTD_e_flush`), so later frames in a group reuse the compression context. A receiver maintains a single decoder per group, reset at each group boundary, and feeds each frame's `Payload` through it in order: the first frame starts the decoder fresh — so a subscriber joining at a group boundary needs nothing earlier — while later frames retain cross-frame redundancy across the group. There is no shared state between groups; a frame with an empty payload contributes nothing to the stream. -DEFLATE is the only scheme this version defines; a future extension MAY negotiate another, in which case at most one is in effect per hop. +Because moq-lite delimits the slices itself, each algorithm's own redundant boundary and container bytes are omitted: for `deflate`, the four `00 00 FF FF` bytes a sync flush emits are removed from each `Payload` and the decoder re-inserts them (as in {{?RFC7692}}); for `zstd`, the per-group stream uses the magicless frame format and omits the content checksum. + +Compressing only the payloads, and leaving the framing in the clear, is deliberate. A relay or cache can hold the payloads compressed in memory and forward them without inflating, and can re-frame a group — for example to bridge a future transport version that changes the GROUP or FRAME headers — without touching the compressed payloads. Neither is possible if the framing is buried inside the compressed stream. -A relay applies compression independently on each hop, driven by each downstream's negotiation — not by the relay's own initiative, and never on a Track the publisher did not mark. On its upstream subscription it decompresses each compressed group as needed; on each downstream subscription it compresses when that downstream advertised the capability, and sends verbatim otherwise. A relay MAY bridge protocol versions (e.g. a moq-lite-05 publisher to a moq-lite-04 subscriber) provided the decompressed bytes are identical to what the publisher produced. A relay or generic library MUST NOT inspect or modify the decompressed contents unless otherwise negotiated. +A relay applies compression independently on each hop, driven by each downstream's negotiation — not by the relay's own initiative, and never on a Track the publisher did not mark. On its upstream subscription it decompresses each compressed group as needed; on each downstream subscription it (re)compresses with that downstream's negotiated algorithm, or sends verbatim when no algorithm is shared. A relay MAY transcode between algorithms, and bridge protocol versions (e.g. a moq-lite-05 publisher to a moq-lite-04 subscriber), provided the decompressed bytes are identical to what the publisher produced. A relay or generic library MUST NOT inspect or modify the decompressed contents unless otherwise negotiated. ## SUBSCRIBE_OK {#subscribe-ok} A SUBSCRIBE_OK message confirms a subscription and resolves its absolute start group. @@ -984,7 +996,7 @@ See the [Prioritization](#prioritization) section for more information. **Group Sequence**: The sequence number of the group to fetch. -The publisher responds with FRAME messages directly on the same stream — there is no response header; when the Track is compressed on this hop each frame's payload is a slice of the group's DEFLATE stream (see [Compression](#compression)). +The publisher responds with FRAME messages directly on the same stream — there is no response header; when the Track is compressed on this hop each frame's payload is a slice of the group's compressed stream (see [Compression](#compression)). The subscriber parses them using the track's [TRACK_INFO](#track-info), which it MUST already have (see the [Track Stream](#track-stream)); the group sequence is implicit from the FETCH request. The publisher FINs the stream after the last frame, or resets on error. There is no FETCH_ERROR message — the publisher signals failure by resetting the stream. @@ -1076,7 +1088,7 @@ The first frame of a group is delta-encoded from `0`, so its `Timestamp Delta` i **Payload**: An application-specific payload. -When the group is compressed (see [Compression](#compression)) this holds this frame's slice of the group's DEFLATE stream and `Message Length` is its compressed size; otherwise it is verbatim. The framing around it is never compressed. +When the group is compressed (see [Compression](#compression)) this holds this frame's slice of the group's compressed stream and `Message Length` is its compressed size; otherwise it is verbatim. The framing around it is never compressed. A generic library or relay MUST NOT inspect or modify the decompressed contents unless otherwise negotiated; recompression that preserves the decompressed bytes exactly is allowed (see [Compression](#compression)). @@ -1100,7 +1112,7 @@ A generic library or relay MUST NOT inspect or modify the decompressed contents - Removed `Publisher Max Latency`. The publisher's retention guarantee is no longer part of the wire format; retention for FETCH and future subscriptions is best-effort and left to the publisher. - Timestamp-based expiration replaces wall-clock arrival time when a Track timescale is negotiated. - Added QUIC datagram delivery for groups, sharing Subscribe IDs with existing subscriptions (no separate control stream). -- Added payload compression, scoped per group. `Publisher Compression` in TRACK_INFO is the original publisher's end-to-end signal that a Track's payloads are good candidates for compression; a new SETUP `Compression` parameter negotiates the capability per hop. A hop compresses when, and only when, the signal is set and the receiver advertised the parameter — there is no per-group or per-frame flag on the wire. When compressed, a group's frame payloads (only the payloads, never the framing) form a single raw DEFLATE stream, reset at each group boundary and sliced per frame into each frame's opaque `Payload`; the redundant trailing `00 00 FF FF` of each flush is stripped (as in RFC 7692) since moq-lite frames the slices itself, and the decoder keeps one context per group. Keeping the framing uncompressed lets a relay store payloads compressed and re-frame across transport versions without recompressing. DEFLATE is the only scheme; others are left to future extensions (one in effect per hop). +- Added payload compression, scoped per group. `Publisher Compression` in TRACK_INFO is the original publisher's end-to-end signal that a Track's payloads are good candidates for compression; a new SETUP `Compression` parameter carries each endpoint's decompressable algorithms in preference order. For each direction the algorithm is the receiver's most-preferred that the sender also supports — computed identically by both ends, so the two directions may differ; `deflate` is mandatory (guaranteeing a common algorithm) and `zstd` is optional. There is no per-group or per-frame flag on the wire: the receiver infers both *whether* and *which* from the signal and the selection. When compressed, a group's frame payloads (only the payloads, never the framing) form a single stream in the negotiated algorithm, reset at each group boundary and sliced per frame into each frame's opaque `Payload`, with the algorithm's redundant container bytes omitted (the RFC 7692 trim for deflate; magicless, checksum-less frames for zstd) since moq-lite frames the slices itself; the decoder keeps one context per group. Keeping the framing uncompressed lets a relay store payloads compressed and re-frame across transport versions without recompressing. - Added Qmux [qmux] transport bindings for TCP/TLS and WebSocket, for environments where UDP is unavailable. The WebSocket binding uses the WebSocket message framing in place of the Qmux Record `Size` field. ## moq-lite-04 From 3bf34e58c1c27b187ef4b065ad88a0fe0bf62c77 Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 23 Jun 2026 00:47:10 +0000 Subject: [PATCH 06/10] compression: cumulative decompression bound; clarify negotiation MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Address review feedback on both drafts: - Decompression-bomb bound is now cumulative over the group (moq-lite) / subgroup (moq-compression), not per slice — since compression is stream-scoped, many small slices could otherwise accumulate without limit. - The advertised algorithm list denotes what an endpoint can BOTH compress and decompress, with a one-line rationale: flagless per-direction selection requires both sides to compute the algorithm from public info, so a decompress-only advertisement would force a per-message signal we deliberately don't have. (Unifies moq-lite, which had said "decompress", with moq-compression.) - A sender MUST NOT compress before it has received the peer's Compression Parameter / COMPRESSION option. Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_01W8bLV6vHzucLNvDhPk3bMP --- draft-lcurley-moq-compression.md | 5 +++-- draft-lcurley-moq-lite.md | 8 ++++---- 2 files changed, 7 insertions(+), 6 deletions(-) diff --git a/draft-lcurley-moq-compression.md b/draft-lcurley-moq-compression.md index 876c4e3..01ea29d 100644 --- a/draft-lcurley-moq-compression.md +++ b/draft-lcurley-moq-compression.md @@ -73,7 +73,8 @@ An endpoint that does not support the extension omits the option. Negotiation is per direction and per hop. For a given direction sender-to-receiver, the **selected algorithm** is the first identifier in the receiver's advertised list that also appears in the sender's advertised list; if the lists do not intersect, that direction is verbatim. Because each endpoint holds both advertised lists once SETUP has been exchanged, both compute the same selection with no further handshake — the receiver's preference governs its own inbound direction, the two directions are independent and MAY select different algorithms, and a simultaneous SETUP exchange creates no ambiguity. Since `deflate` is mandatory for any endpoint that advertises the option, two endpoints that both support the extension always share at least one algorithm. -A sender MUST NOT compress with an algorithm the receiver did not advertise. +Each endpoint lists only algorithms it can both produce and consume, so either side can compute the selection for either direction without a per-object signal. +A sender MUST NOT compress with an algorithm the receiver did not advertise, and MUST NOT compress at all before it has received the receiver's COMPRESSION option. # COMPRESSION Track Property @@ -149,7 +150,7 @@ A publisher MUST NOT set a non-zero COMPRESSION hint on a track whose object pay Because compression is scoped to a subgroup, the exposure is bounded to within a single subgroup — which may combine several objects, a wider window than a single object — but it is not eliminated. A malicious sender could emit a small compressed payload that decompresses to a very large buffer (a "decompression bomb"). -A receiver MUST bound the size of a decompressed object payload. If the bound is exceeded it MUST reset the affected stream (rather than allocate unbounded memory) and MAY close the session with a PROTOCOL_VIOLATION if it considers the peer abusive; the reset is stream-scoped so a single bad subgroup does not tear down unrelated subscriptions. +Because compression is subgroup-scoped, a receiver MUST bound the cumulative decompressed size of a subgroup — not merely each object's payload, since many small payloads can otherwise accumulate without limit. If the bound is exceeded it MUST reset the affected stream (rather than allocate unbounded memory) and MAY close the session with a PROTOCOL_VIOLATION if it considers the peer abusive; the reset is stream-scoped so a single bad subgroup does not tear down unrelated subscriptions. Compression is orthogonal to {{moqt}} end-to-end encryption: an encrypted payload is effectively incompressible, so a publisher using end-to-end encryption SHOULD omit COMPRESSION (or use `0`). diff --git a/draft-lcurley-moq-lite.md b/draft-lcurley-moq-lite.md index 190a6a5..ec472ac 100644 --- a/draft-lcurley-moq-lite.md +++ b/draft-lcurley-moq-lite.md @@ -631,13 +631,13 @@ The remaining bindings convey the path in their own handshake. A relay MUST NOT forward the Path Parameter; like other per-hop setup metadata it applies only to this hop (see [Session](#session)). ### Compression Parameter {#compression-parameter} -The Compression Parameter advertises the payload compression [algorithms](#compression) the sender can *decompress* on this hop, in preference order (most-preferred first). +The Compression Parameter advertises the payload compression [algorithms](#compression) the sender can both compress and decompress on this hop, in preference order (most-preferred first). The Parameter Value is a sequence of algorithm identifiers, each a variable-length integer, packed back-to-back to fill the Parameter Length. An endpoint that supports compression MUST include `deflate` (1); `none` (0) MUST NOT be listed. An endpoint that does not support compression omits the parameter. Negotiation is per direction and per hop, and a relay MUST NOT forward the parameter (see [Session](#session)). -For a given direction, the algorithm is the first entry in the **receiver's** list that also appears in the **sender's** list: the receiver's preference governs, and because both endpoints hold both lists once SETUP has been exchanged, each computes the same result with no further handshake. The two directions are independent and MAY select different algorithms. Since `deflate` is mandatory for any endpoint that supports compression, two compression-capable endpoints always share at least one algorithm. -A sender MUST NOT compress when sending to a receiver that advertised no list, so such a receiver gets every Group Stream, Fetch Stream, and datagram verbatim regardless of the Track's `Publisher Compression` signal. +For a given direction, the algorithm is the first entry in the **receiver's** list that also appears in the **sender's** list: the receiver's preference governs, and because both endpoints hold both lists once SETUP has been exchanged, each computes the same result with no further handshake. Listing only algorithms it can both produce and consume is what lets either endpoint make this selection for either direction without a per-message signal. The two directions are independent and MAY select different algorithms. Since `deflate` is mandatory for any endpoint that supports compression, two compression-capable endpoints always share at least one algorithm. +A sender MUST NOT compress until it has received the receiver's Compression Parameter; a receiver that advertised no list gets every Group Stream, Fetch Stream, and datagram verbatim regardless of the Track's `Publisher Compression` signal. ## ANNOUNCE_REQUEST @@ -1217,7 +1217,7 @@ A publisher MUST NOT set a non-zero `Publisher Compression` hint on a Track whos Because compression is group-scoped, the exposure is bounded to within a single group — which may combine several frames, a wider window than a single frame — but it is not eliminated. A malicious sender could emit a small compressed payload that decompresses to a very large buffer (a "decompression bomb"). -A receiver MUST bound the size of a decompressed payload; if the bound is exceeded it MUST reset the affected stream rather than allocate unbounded memory, and MAY close the session with a PROTOCOL_VIOLATION if it considers the peer abusive. +Because compression is group-scoped, a receiver MUST bound the cumulative decompressed size of a group stream — not merely each frame's slice, since many small slices can otherwise accumulate without limit; if the bound is exceeded it MUST reset the affected stream rather than allocate unbounded memory, and MAY close the session with a PROTOCOL_VIOLATION if it considers the peer abusive. Compression is orthogonal to end-to-end encryption: an encrypted payload is effectively incompressible, so a publisher using end-to-end encryption SHOULD leave `Publisher Compression` at `0`. From 63798e053f873a8f62121b86e6f64f8ef5f3e783 Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 23 Jun 2026 02:25:00 +0000 Subject: [PATCH 07/10] compression: publisher names the algorithm in the track property MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Per implementation feedback, drop the boolean hint + first-intersection selection. The track property (Publisher Compression / COMPRESSION) now names the algorithm the publisher used (none/deflate/zstd); SETUP advertises the decoders each endpoint supports, and the publisher MUST pick an algorithm its peer advertised (deflate mandatory, so always safe). Flagless inference is now off the property: a receiver decompresses iff the property names a non-none algorithm it advertised, else verbatim. Reverts the per-direction selection and the "compress and decompress" wording (the list is decode capability again). Group/subgroup-scoped sliced-stream mechanics, deflate+zstd, and the RFC 7692 / magicless framing trims are unchanged. Relays forward the property unchanged and may recompress only with the same algorithm. Added an explicit "Open issue" note: an immutable algorithm-naming property means a relay can't transcode (e.g. zstd-> deflate) for a downstream that supports only a different algorithm without rewriting the property, which moq-transport forbids — flagged for the working group. Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_01W8bLV6vHzucLNvDhPk3bMP --- draft-lcurley-moq-compression.md | 75 +++++++++++++++++--------------- draft-lcurley-moq-lite.md | 42 +++++++++--------- 2 files changed, 61 insertions(+), 56 deletions(-) diff --git a/draft-lcurley-moq-compression.md b/draft-lcurley-moq-compression.md index 01ea29d..4a79091 100644 --- a/draft-lcurley-moq-compression.md +++ b/draft-lcurley-moq-compression.md @@ -27,8 +27,8 @@ informative: --- abstract This document defines a payload compression extension for MoQ Transport {{moqt}}. -A track-level Compression property is a boolean hint by which the original publisher marks a track's object payloads as good candidates for compression. -The algorithm is negotiated independently on each hop, and compression is applied only on a hop where the publisher's hint is set and the sender and receiver share an algorithm; otherwise payloads travel verbatim. +A track-level Compression property names the algorithm the original publisher used for a track's object payloads. +Endpoints advertise the algorithms they can decode during SETUP, and a payload is compressed on a hop only when the receiver supports the named algorithm; otherwise it is sent verbatim. Compression is scoped to a subgroup: the object payloads of a subgroup form one compressed stream, sliced back into the individual object payloads so the object framing stays in the clear and the decompressed bytes — the actual objects — are unchanged end to end. --- middle @@ -45,17 +45,16 @@ But MoQ also carries non-media tracks — JSON, text, telemetry, captions, uncom For these tracks there is no standard, transport-visible way to compress payloads, so each application reinvents it, and relays cannot help. Like HTTP Transfer-Encoding, the on-wire compression is a hop-by-hop optimization: it does not conceptually change the object payload — the decompressed bytes *are* the object — it only changes how those bytes are carried over a single hop. -What this extension adds is an end-to-end *signal* — a boolean track property by which the original publisher marks the content as worth compressing — plus a per-hop negotiation of the algorithm. -- **Publisher signals, hops apply**: the COMPRESSION track property is a boolean set by the original publisher and carried end to end, but a payload is only compressed on a hop that has negotiated a shared algorithm. Which algorithm is used is decided per hop, not by the publisher; a hop that has not negotiated compression forwards the property unchanged so a further-downstream hop can still act on it. -- **Per subgroup, sliced into objects**: within a subgroup the object payloads form one compressed stream, flushed at each object boundary so every object still carries its own payload slice, while the object headers and framing stay in the clear. This keeps the subgroup — the unit a receiver already takes as one ordered, reliable stream — as the unit of compression, while letting relays and caches store payloads compressed and re-frame them without recompressing. +- **Publisher names, hops apply**: the COMPRESSION track property names the algorithm the original publisher used; it is carried end to end and forwarded unchanged. A payload is compressed on a hop only when the receiver advertised that algorithm; otherwise it travels verbatim. Each hop's behavior is fixed by the publisher's algorithm and that hop's negotiation, with no per-object signal. +- **Per subgroup, sliced into objects**: within a subgroup the object payloads form one compressed stream, flushed at each object boundary so every object still carries its own payload slice, while the object headers and framing stay in the clear. This keeps the subgroup — already one ordered, reliable stream — as the unit of compression, and lets relays and caches store payloads compressed and re-frame them without recompressing. # Setup Negotiation The Payload Compression extension is negotiated during the SETUP exchange as defined in {{moqt}} Section 10.3. Unlike a purely additive property, compression MUST be negotiated: a receiver that does not understand the algorithm would otherwise pass the compressed bytes to the application as if they were plaintext. -Each endpoint advertises the algorithms it can decompress, in preference order (most-preferred first), by including the following Setup Option: +Each endpoint advertises the algorithms it can decompress by including the following Setup Option: ~~~ COMPRESSION Setup Option { @@ -66,60 +65,61 @@ COMPRESSION Setup Option { ~~~ **Algorithm**: -One or more Algorithm identifiers (see [Compression Algorithms](#compression-algorithms)) that the sender can compress and decompress, each a varint, filling the Option Value, most-preferred first. +One or more Algorithm identifiers (see [Compression Algorithms](#compression-algorithms)) that the sender can decompress, each a varint, filling the Option Value. An endpoint that includes this option MUST list `deflate` (1); the identifier `none` (0) MUST NOT be listed (it requires no negotiation). An endpoint that does not support the extension omits the option. -Negotiation is per direction and per hop. For a given direction sender-to-receiver, the **selected algorithm** is the first identifier in the receiver's advertised list that also appears in the sender's advertised list; if the lists do not intersect, that direction is verbatim. -Because each endpoint holds both advertised lists once SETUP has been exchanged, both compute the same selection with no further handshake — the receiver's preference governs its own inbound direction, the two directions are independent and MAY select different algorithms, and a simultaneous SETUP exchange creates no ambiguity. -Since `deflate` is mandatory for any endpoint that advertises the option, two endpoints that both support the extension always share at least one algorithm. -Each endpoint lists only algorithms it can both produce and consume, so either side can compute the selection for either direction without a per-object signal. -A sender MUST NOT compress with an algorithm the receiver did not advertise, and MUST NOT compress at all before it has received the receiver's COMPRESSION option. +A sender MUST NOT compress with an algorithm the receiver did not advertise, and MUST NOT compress before it has received the receiver's COMPRESSION option. +This makes the on-wire state unambiguous with no per-object signaling: a receiver decompresses a track's object payloads **if and only if** the COMPRESSION property names a non-`none` algorithm and the receiver advertised that algorithm in its own SETUP. +In every other case — the property absent or `none`, the extension not negotiated, or the algorithm not advertised by the receiver — the sender was not permitted to compress, so the receiver treats the payloads as verbatim. # COMPRESSION Track Property -The COMPRESSION property is the original publisher's end-to-end signal that a track's object payloads are good candidates for compression. +The COMPRESSION property names the algorithm the original publisher applied to a track's object payloads. It is a track-level Key-Value-Pair carried with the track's properties (see {{moqt}} Section 2.5 and Section 12), set by the original publisher and forwarded unchanged by relays. Because the value is a single integer, COMPRESSION uses an even Type so the value is a bare varint: ~~~ COMPRESSION Track Property { Type (vi64) = 0xC03D0 - Value (vi64) ; boolean hint + Value (vi64) ; Algorithm identifier } ~~~ **Value**: -A boolean hint: `1` means the track's payloads are good candidates for compression, `0` (or absence of the property) means they are not and are always transmitted verbatim. -Values greater than `1` are reserved for future use and MUST be treated as `1` by a receiver that does not understand them, so the hint stays additive. -The property names no algorithm; which algorithm is used, if any, is the per-hop [selected algorithm](#setup-negotiation). +The Algorithm identifier the publisher used for this track's payloads (see [Compression Algorithms](#compression-algorithms)). +The absence of the property, or a value of `none` (0), means the track is uncompressed and its payloads are always transmitted verbatim. +The publisher MUST choose an algorithm that its peer advertised in the [COMPRESSION Setup Option](#setup-negotiation); since `deflate` is mandatory to implement, it is always a safe choice. The property is fixed for the lifetime of the track and MUST NOT change. -A relay MUST forward it unchanged on every hop, including a hop that has not negotiated the extension: there it is simply an ignored unknown Key-Value-Pair, but forwarding it lets a further-downstream hop that does negotiate the extension still act on the publisher's signal. +A relay MUST forward it unchanged on every hop, including a hop that has not negotiated the extension: there it is simply an ignored unknown Key-Value-Pair, but forwarding it lets a further-downstream hop that does negotiate the extension still act on the publisher's algorithm. -Compression is enabled only by the combination of a non-zero hint and a shared algorithm being negotiated on a hop; there is no per-object or per-subgroup signal on the wire. -A receiver decompresses a track's object payloads **if and only if** the COMPRESSION hint is non-zero and the hop selected an algorithm for that direction, in which case it uses the selected algorithm. -In every other case — the hint absent or zero, the extension not negotiated, or the lists not intersecting — payloads are verbatim. +Whether a payload is actually compressed is decided per hop: -A publisher SHOULD set COMPRESSION only for payload types that benefit from it (e.g. JSON, text, uncompressed binary structures). -Already-compressed media SHOULD omit it (or use `0`). +- On a hop where the receiver advertised the property's algorithm, each non-empty object payload is compressed with that algorithm, and the receiver decompresses it. +- On any other hop — the extension not negotiated, or the receiver did not advertise that algorithm — payloads are sent verbatim, and the receiver treats them as such. + +Compression applies to the object payload only; object headers, properties, and message framing are never compressed. +An empty payload (size 0) MUST NOT be compressed and remains empty on the wire. + +A publisher SHOULD set COMPRESSION only for payload types that benefit from it. +Already-compressed media SHOULD omit it (or use `none`). # Compression {#compression} -Compression is applied to object payloads only — object headers, properties, and message framing are never compressed — and is **scoped to a subgroup**. -Within a subgroup the object payloads form a single compressed stream in the [selected algorithm](#setup-negotiation), reset at each subgroup boundary. +Compression is **scoped to a subgroup**. +Within a subgroup the object payloads form a single compressed stream in the algorithm named by the [COMPRESSION property](#compression-track-property), reset at each subgroup boundary. The stream's output is partitioned at object boundaries: the compressor flushes at the end of each object so that object's slice is exactly the bytes carried as its payload, and the payload length in the object header gives the on-wire (compressed) slice size. Both algorithms provide a window-retaining flush (DEFLATE's sync flush; Zstandard's `ZSTD_e_flush`), so later objects in a subgroup reuse the compression context and retain cross-object redundancy. A receiver maintains a single decoder per subgroup, reset at each subgroup boundary, and feeds each object's payload through it in order: the first object of a subgroup starts the decoder fresh — so a receiver joining at a group boundary needs nothing earlier — while later objects build on it. -There is no shared state between subgroups; an empty payload (size 0) contributes nothing to the stream and remains empty on the wire. +There is no shared state between subgroups; an empty payload contributes nothing to the stream. An object delivered as a datagram is a single-object stream, compressed on its own. Because the object framing already delimits each slice, an algorithm's own redundant boundary and container bytes are omitted: for `deflate`, the trailing four `00 00 FF FF` bytes a sync flush emits are removed from each payload and the decoder re-inserts them (as in {{RFC7692}}); for `zstd`, the per-subgroup stream uses the magicless frame format and omits the content checksum. Leaving the framing uncompressed is deliberate. A relay or cache can hold the object payloads compressed in memory and forward them without inflating, and can re-frame a subgroup — for example to bridge a transport version that changes the subgroup or object headers — without touching the compressed payloads. -Neither is possible if the framing is buried inside the compressed stream. ## Compression Algorithms {#compression-algorithms} This document defines the following algorithms. @@ -130,29 +130,32 @@ This document defines the following algorithms. | 1 | deflate | mandatory | Raw DEFLATE {{RFC1951}}, with no zlib or gzip framing. | | 2 | zstd | optional | Zstandard {{RFC8878}}. | -Every endpoint that advertises this extension MUST implement `deflate`, so a common algorithm always exists; `zstd` is optional. +Every endpoint that advertises this extension MUST implement `deflate`, so the publisher always has a safe choice; `zstd` is optional. Further algorithms MAY be registered (see [IANA Considerations](#iana-considerations)). # Relay Behavior -A relay forwards the boolean COMPRESSION track property unchanged — it is the publisher's end-to-end signal — and applies compression independently on each hop, driven by each hop's negotiation rather than by its own initiative; a relay does not compress a track the publisher did not mark. +A relay forwards the COMPRESSION track property unchanged — it is the publisher's end-to-end signal — and applies compression independently on each hop, driven by each hop's negotiation rather than by its own initiative; a relay does not compress a track the publisher did not mark. + +On its upstream subscription the relay receives each subgroup compressed with the property's algorithm (if it advertised that algorithm) or verbatim, and decompresses as needed. +On a downstream subscription that advertised the property's algorithm, it sends each subgroup compressed with that algorithm (recompressing as needed); on one that did not, it sends the subgroup verbatim. +A relay MUST NOT recompress with an algorithm other than the one the property names, because the property tells the receiver how to decode and a relay MUST NOT rewrite it. +In every case the decompressed bytes delivered to the application MUST be identical to what the origin published, and a relay or generic library MUST NOT inspect or modify the decompressed contents unless otherwise negotiated. -On its upstream subscription the relay receives each subgroup compressed with that hop's selected algorithm (or verbatim) and decompresses as needed. -On each downstream subscription it (re)compresses each subgroup with that downstream's selected algorithm, or sends it verbatim when no algorithm is shared. -A relay MAY transcode between algorithms. -In every case the decompressed bytes delivered to the application MUST be identical to what the origin published. -A relay or generic library MUST NOT inspect or modify the decompressed contents unless otherwise negotiated; only recompression that preserves the decompressed bytes exactly is permitted. +Open issue: +because the COMPRESSION property both names the algorithm and is immutable, a downstream that supports a *different* algorithm than the publisher chose (for example only `deflate` when the publisher used `zstd`) receives the payloads verbatim rather than transcoded — a relay cannot offer it the algorithm it does support without rewriting the property, which {{moqt}} forbids. +Whether to relax this — by permitting a relay to rewrite this property for a downstream subscription, or by carrying the per-hop algorithm as transport metadata rather than an end-to-end track property — is an open question for the working group. # Security Considerations Compressing data that mixes attacker-controlled and secret content can leak the secret through compressed size, as in the CRIME and BREACH attacks. -A publisher MUST NOT set a non-zero COMPRESSION hint on a track whose object payloads combine secret material with attacker-influenced material. +A publisher MUST NOT set COMPRESSION on a track whose object payloads combine secret material with attacker-influenced material. Because compression is scoped to a subgroup, the exposure is bounded to within a single subgroup — which may combine several objects, a wider window than a single object — but it is not eliminated. A malicious sender could emit a small compressed payload that decompresses to a very large buffer (a "decompression bomb"). Because compression is subgroup-scoped, a receiver MUST bound the cumulative decompressed size of a subgroup — not merely each object's payload, since many small payloads can otherwise accumulate without limit. If the bound is exceeded it MUST reset the affected stream (rather than allocate unbounded memory) and MAY close the session with a PROTOCOL_VIOLATION if it considers the peer abusive; the reset is stream-scoped so a single bad subgroup does not tear down unrelated subscriptions. -Compression is orthogonal to {{moqt}} end-to-end encryption: an encrypted payload is effectively incompressible, so a publisher using end-to-end encryption SHOULD omit COMPRESSION (or use `0`). +Compression is orthogonal to {{moqt}} end-to-end encryption: an encrypted payload is effectively incompressible, so a publisher using end-to-end encryption SHOULD omit COMPRESSION (or use `none`). # IANA Considerations diff --git a/draft-lcurley-moq-lite.md b/draft-lcurley-moq-lite.md index ec472ac..5bb6129 100644 --- a/draft-lcurley-moq-lite.md +++ b/draft-lcurley-moq-lite.md @@ -309,7 +309,7 @@ When the accepted track has already ended with no matching groups there is no st A rejection is a stream reset: if the publisher cannot serve the subscription — the track does not exist, or it otherwise refuses — it MUST reset the stream rather than leave it pending, and SHOULD do so promptly (within roughly a round trip) so the subscriber is not left waiting. A subscription the publisher accepts but has no groups for yet is not a rejection: for a live track the publisher MAY withhold SUBSCRIBE_OK until the first matching group resolves the start. A subscriber therefore distinguishes "pending" from "refused" by the stream reset, not by a timeout. The Subscribe Stream does not carry the track's publisher properties — those are immutable and fetched once via a [Track Stream](#track-stream) (see [TRACK_INFO](#track-info)). -The subscriber MUST have the track's TRACK_INFO before it can parse the FRAME messages that arrive on Group Streams, since the timescale determines the frame wire format and the compression signal determines whether frame payloads are compressed; it MAY open the Track and Subscribe streams concurrently and buffer frames until TRACK_INFO arrives. +The subscriber MUST have the track's TRACK_INFO before it can parse the FRAME messages that arrive on Group Streams, since the timescale determines the frame wire format and the `Publisher Compression` value determines whether and how frame payloads are compressed; it MAY open the Track and Subscribe streams concurrently and buffer frames until TRACK_INFO arrives. The publisher sends SUBSCRIBE_OK once the absolute start group is resolved, and SUBSCRIBE_END once no further groups will be produced (see [SUBSCRIBE_OK](#subscribe-ok) and [SUBSCRIBE_END](#subscribe-end)). The publisher closes the stream (FIN) only once every group from start to end has been accounted for, either via a GROUP stream (completed or reset) or a SUBSCRIBE_DROP message. @@ -520,7 +520,7 @@ Any varint value (including 0) is a valid absolute timestamp. **Payload**: The frame payload, extending to the end of the datagram. -A datagram is a single-frame group: when the Track is compressed on this hop (see [Compression](#compression)) the payload is a single compressed stream in the negotiated algorithm, otherwise it is verbatim. +A datagram is a single-frame group: when the Track is compressed on this hop (see [Compression](#compression)) the payload is a single compressed stream in the Track's algorithm, otherwise it is verbatim. The total datagram body (including all header fields above and the compressed payload if applicable) MUST NOT exceed 1200 bytes. This limit ensures the datagram fits within the minimum QUIC path MTU without IP-layer fragmentation. Payloads that would not fit MUST be sent as a Group Stream instead. @@ -631,13 +631,13 @@ The remaining bindings convey the path in their own handshake. A relay MUST NOT forward the Path Parameter; like other per-hop setup metadata it applies only to this hop (see [Session](#session)). ### Compression Parameter {#compression-parameter} -The Compression Parameter advertises the payload compression [algorithms](#compression) the sender can both compress and decompress on this hop, in preference order (most-preferred first). +The Compression Parameter advertises the payload compression [algorithms](#compression) the sender can *decompress* on this hop. The Parameter Value is a sequence of algorithm identifiers, each a variable-length integer, packed back-to-back to fill the Parameter Length. An endpoint that supports compression MUST include `deflate` (1); `none` (0) MUST NOT be listed. An endpoint that does not support compression omits the parameter. -Negotiation is per direction and per hop, and a relay MUST NOT forward the parameter (see [Session](#session)). -For a given direction, the algorithm is the first entry in the **receiver's** list that also appears in the **sender's** list: the receiver's preference governs, and because both endpoints hold both lists once SETUP has been exchanged, each computes the same result with no further handshake. Listing only algorithms it can both produce and consume is what lets either endpoint make this selection for either direction without a per-message signal. The two directions are independent and MAY select different algorithms. Since `deflate` is mandatory for any endpoint that supports compression, two compression-capable endpoints always share at least one algorithm. -A sender MUST NOT compress until it has received the receiver's Compression Parameter; a receiver that advertised no list gets every Group Stream, Fetch Stream, and datagram verbatim regardless of the Track's `Publisher Compression` signal. +A relay MUST NOT forward the parameter (see [Session](#session)); it is negotiated independently on each hop. +The list constrains what a publisher may use: a publisher MUST set `Publisher Compression` to an algorithm its peer advertised here (see [TRACK_INFO](#track-info)), and more generally a sender MUST NOT compress with an algorithm the receiver did not advertise, nor compress at all before it has received the receiver's Compression Parameter. +A receiver decompresses a Track's payloads if and only if `Publisher Compression` names a non-`none` algorithm and the receiver advertised that algorithm; a receiver that advertised no list, or not the Track's algorithm, gets every Group Stream, Fetch Stream, and datagram verbatim. ## ANNOUNCE_REQUEST @@ -864,24 +864,24 @@ When `Publisher Timescale` is 0, the per-frame `Timestamp Delta` field is omitte Common values include `1000` (milliseconds), `1000000` (microseconds), `48000` (audio sample rate), and `90000` (RTP video clock). **Publisher Compression**: -The original publisher's end-to-end signal that this Track's payloads are good candidates for compression. +The compression algorithm the original publisher applied to this Track's payloads (see [Compression](#compression)). -- `0`: not a compression candidate; payloads are always sent verbatim (the default). -- `1`: a compression candidate; payloads are compressed on any hop that has negotiated compression, and verbatim elsewhere. +- `0` (`none`): payloads are uncompressed (the default). +- `1` (`deflate`) or `2` (`zstd`): payloads are compressed with that algorithm. -Values greater than `1` are reserved for future use; a subscriber that does not understand a reserved value MUST treat it as `1`, so the signal stays additive and never blocks the Track. - -The signal is carried end to end and forwarded unchanged by relays, so even a hop that does not compress passes it along for a further-downstream hop to act on. It does not by itself cause compression: that also requires the receiving hop to advertise a shared algorithm via the [Compression Parameter](#compression-parameter), and there is no per-group flag on the wire — a receiver decompresses if and only if it holds a non-zero signal for the Track and the hop selected a shared algorithm (see [Compression](#compression)). +The publisher MUST choose an algorithm its peer advertised in the [Compression Parameter](#compression-parameter); `deflate` is mandatory to support, so it is always a safe choice. +The value is fixed for the lifetime of the Track and forwarded unchanged by relays, so even a hop that does not compress passes it along for a further-downstream hop to act on. +It does not by itself cause compression: a receiver decompresses if and only if the value names a non-`none` algorithm and the receiver advertised that algorithm in its own [Compression Parameter](#compression-parameter); otherwise the payloads are verbatim (see [Compression](#compression)). A subscriber that does not recognize the value treats the payloads as verbatim, so an unknown future algorithm degrades to uncompressed rather than blocking the Track. The publisher SHOULD set it only for payload types that benefit from compression (e.g. JSON, text, uncompressed binary structures); already-compressed media (e.g. H.264, Opus, AV1) gains nothing and SHOULD leave it `0`. See [Security Considerations](#security-considerations) for content that MUST NOT be marked compressible. ## Compression {#compression} -moq-lite can compress payloads hop by hop, like HTTP Transfer-Encoding: the decompressed bytes *are* the payload and are identical end to end, but how they are carried MAY differ on each hop — a group compressed on one hop is decompressed and MAY be re-sent verbatim, or recompressed with a different algorithm, on the next. Compression is governed by two pieces, with no per-group flag on the wire: +moq-lite can compress payloads hop by hop, like HTTP Transfer-Encoding: the decompressed bytes *are* the payload and are identical end to end, but how they are carried MAY differ on each hop — a group compressed on one hop is decompressed and MAY be re-sent verbatim on the next. Compression is governed by two pieces, with no per-group flag on the wire: -- the Track's `Publisher Compression` signal (see [TRACK_INFO](#track-info)) — the original publisher's end-to-end mark that the payloads are good candidates for compression; and -- the per-hop [Compression Parameter](#compression-parameter) — each endpoint advertises, in preference order, the algorithms it can decompress. +- the Track's `Publisher Compression` value (see [TRACK_INFO](#track-info)) — the algorithm the original publisher used, carried end to end; and +- the per-hop [Compression Parameter](#compression-parameter) — each endpoint advertises the algorithms it can decompress. -A hop compresses a Track's payloads when, and only when, the Track's signal is non-zero **and** the sender and receiver share an algorithm (the [Compression Parameter](#compression-parameter) defines how the single per-direction algorithm is selected); otherwise they are verbatim. A receiver needs no per-group signal — neither a "compressed" flag nor an algorithm identifier — because it derives both from the Track's signal and the deterministic per-direction selection. Because the signal is carried end to end and forwarded unchanged, a hop that has not negotiated compression still passes it along, so a further-downstream hop that has can act on it. +A hop compresses a Track's payloads when, and only when, `Publisher Compression` names a non-`none` algorithm **and** the receiver advertised that algorithm; otherwise they are verbatim. A receiver needs no per-group signal — it takes the algorithm from `Publisher Compression`, and whether the payloads are compressed from its own advertisement. Because the value is carried end to end and forwarded unchanged, a hop that has not negotiated compression still passes it along, so a further-downstream hop that has can act on it. The following algorithms are defined: @@ -891,15 +891,17 @@ The following algorithms are defined: | 1 | deflate | mandatory | Raw DEFLATE {{!RFC1951}}, with no zlib or gzip framing. | | 2 | zstd | optional | Zstandard {{!RFC8878}}. | -Every endpoint that supports compression MUST implement `deflate`, so a common algorithm always exists; `zstd` is optional, and further algorithms MAY be defined by future extensions. +Every endpoint that supports compression MUST implement `deflate`, so the publisher always has a safe choice; `zstd` is optional, and further algorithms MAY be defined by future extensions. -Compression is **group-scoped** and applied only to frame payloads, never to the FRAME framing. Within a group the payloads form a single compressed stream in the negotiated algorithm, reset at each group boundary, whose output is partitioned at frame boundaries: the compressor flushes at the end of each frame so that frame's slice is exactly the bytes stored in its `Payload` (delimited by `Message Length`). Both algorithms provide such a flush that retains the window (DEFLATE's sync flush; Zstandard's `ZSTD_e_flush`), so later frames in a group reuse the compression context. A receiver maintains a single decoder per group, reset at each group boundary, and feeds each frame's `Payload` through it in order: the first frame starts the decoder fresh — so a subscriber joining at a group boundary needs nothing earlier — while later frames retain cross-frame redundancy across the group. There is no shared state between groups; a frame with an empty payload contributes nothing to the stream. +Compression is **group-scoped** and applied only to frame payloads, never to the FRAME framing. Within a group the payloads form a single compressed stream in the Track's algorithm, reset at each group boundary, whose output is partitioned at frame boundaries: the compressor flushes at the end of each frame so that frame's slice is exactly the bytes stored in its `Payload` (delimited by `Message Length`). Both algorithms provide such a flush that retains the window (DEFLATE's sync flush; Zstandard's `ZSTD_e_flush`), so later frames in a group reuse the compression context. A receiver maintains a single decoder per group, reset at each group boundary, and feeds each frame's `Payload` through it in order: the first frame starts the decoder fresh — so a subscriber joining at a group boundary needs nothing earlier — while later frames retain cross-frame redundancy across the group. There is no shared state between groups; a frame with an empty payload contributes nothing to the stream. Because moq-lite delimits the slices itself, each algorithm's own redundant boundary and container bytes are omitted: for `deflate`, the four `00 00 FF FF` bytes a sync flush emits are removed from each `Payload` and the decoder re-inserts them (as in {{?RFC7692}}); for `zstd`, the per-group stream uses the magicless frame format and omits the content checksum. Compressing only the payloads, and leaving the framing in the clear, is deliberate. A relay or cache can hold the payloads compressed in memory and forward them without inflating, and can re-frame a group — for example to bridge a future transport version that changes the GROUP or FRAME headers — without touching the compressed payloads. Neither is possible if the framing is buried inside the compressed stream. -A relay applies compression independently on each hop, driven by each downstream's negotiation — not by the relay's own initiative, and never on a Track the publisher did not mark. On its upstream subscription it decompresses each compressed group as needed; on each downstream subscription it (re)compresses with that downstream's negotiated algorithm, or sends verbatim when no algorithm is shared. A relay MAY transcode between algorithms, and bridge protocol versions (e.g. a moq-lite-05 publisher to a moq-lite-04 subscriber), provided the decompressed bytes are identical to what the publisher produced. A relay or generic library MUST NOT inspect or modify the decompressed contents unless otherwise negotiated. +A relay applies compression independently on each hop, driven by each downstream's negotiation — not by the relay's own initiative, and never on a Track the publisher did not mark. On its upstream subscription it decompresses each compressed group as needed; on a downstream subscription that advertised the Track's algorithm it (re)compresses with that algorithm, and on one that did not it sends the group verbatim. A relay MUST NOT recompress with an algorithm other than the one `Publisher Compression` names — that value tells the receiver how to decode and is immutable — though it MAY recompress with the same algorithm, and MAY bridge protocol versions (e.g. a moq-lite-05 publisher to a moq-lite-04 subscriber) provided the decompressed bytes are identical to what the publisher produced. A relay or generic library MUST NOT inspect or modify the decompressed contents unless otherwise negotiated. + +Open issue: because `Publisher Compression` both names the algorithm and is immutable, a downstream that supports only a *different* algorithm than the publisher chose (e.g. only `deflate` when the publisher used `zstd`) receives the payloads verbatim rather than transcoded — a relay cannot offer it an algorithm it does support without rewriting the immutable value. Whether to allow a per-hop algorithm distinct from the publisher's is left as an open question. ## SUBSCRIBE_OK {#subscribe-ok} A SUBSCRIBE_OK message confirms a subscription and resolves its absolute start group. @@ -1112,7 +1114,7 @@ A generic library or relay MUST NOT inspect or modify the decompressed contents - Removed `Publisher Max Latency`. The publisher's retention guarantee is no longer part of the wire format; retention for FETCH and future subscriptions is best-effort and left to the publisher. - Timestamp-based expiration replaces wall-clock arrival time when a Track timescale is negotiated. - Added QUIC datagram delivery for groups, sharing Subscribe IDs with existing subscriptions (no separate control stream). -- Added payload compression, scoped per group. `Publisher Compression` in TRACK_INFO is the original publisher's end-to-end signal that a Track's payloads are good candidates for compression; a new SETUP `Compression` parameter carries each endpoint's decompressable algorithms in preference order. For each direction the algorithm is the receiver's most-preferred that the sender also supports — computed identically by both ends, so the two directions may differ; `deflate` is mandatory (guaranteeing a common algorithm) and `zstd` is optional. There is no per-group or per-frame flag on the wire: the receiver infers both *whether* and *which* from the signal and the selection. When compressed, a group's frame payloads (only the payloads, never the framing) form a single stream in the negotiated algorithm, reset at each group boundary and sliced per frame into each frame's opaque `Payload`, with the algorithm's redundant container bytes omitted (the RFC 7692 trim for deflate; magicless, checksum-less frames for zstd) since moq-lite frames the slices itself; the decoder keeps one context per group. Keeping the framing uncompressed lets a relay store payloads compressed and re-frame across transport versions without recompressing. +- Added payload compression, scoped per group. `Publisher Compression` in TRACK_INFO now names the algorithm the publisher used (`none`/`deflate`/`zstd`), and a new SETUP `Compression` parameter carries the algorithms each endpoint can decompress on a hop. The publisher MUST pick an algorithm its peer advertised; `deflate` is mandatory (so always safe) and `zstd` is optional. There is no per-group or per-frame flag on the wire — a receiver decompresses iff `Publisher Compression` names a non-`none` algorithm it advertised, else treats payloads as verbatim. When compressed, a group's frame payloads (only the payloads, never the framing) form a single stream in that algorithm, reset at each group boundary and sliced per frame into each frame's opaque `Payload`, with the algorithm's redundant container bytes omitted (the RFC 7692 trim for deflate; magicless, checksum-less frames for zstd) since moq-lite frames the slices itself; the decoder keeps one context per group. Keeping the framing uncompressed lets a relay store payloads compressed and re-frame across transport versions without recompressing. - Added Qmux [qmux] transport bindings for TCP/TLS and WebSocket, for environments where UDP is unavailable. The WebSocket binding uses the WebSocket message framing in place of the Qmux Record `Size` field. ## moq-lite-04 From 58aeba44b4b43e1a91fbc3926908d39d4036ceb1 Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 23 Jun 2026 02:29:02 +0000 Subject: [PATCH 08/10] compression: no mandatory algorithm MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Drop deflate's MUST-implement status in both drafts. A mandatory algorithm is a conformance burden (a zstd-only endpoint shouldn't be non-conformant) and doesn't actually guarantee end-to-end compression on a relayed path anyway. Now both deflate and zstd are simply defined; endpoints advertise whichever they support, and a publisher uses an algorithm its peer advertised or `none` if they share none — verbatim, the same well-defined fallback as any unsupported case. Removes the "Requirement" column and the "always a safe choice" framing. Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_01W8bLV6vHzucLNvDhPk3bMP --- draft-lcurley-moq-compression.md | 16 ++++++++-------- draft-lcurley-moq-lite.md | 18 +++++++++--------- 2 files changed, 17 insertions(+), 17 deletions(-) diff --git a/draft-lcurley-moq-compression.md b/draft-lcurley-moq-compression.md index 4a79091..ee8a57d 100644 --- a/draft-lcurley-moq-compression.md +++ b/draft-lcurley-moq-compression.md @@ -66,7 +66,7 @@ COMPRESSION Setup Option { **Algorithm**: One or more Algorithm identifiers (see [Compression Algorithms](#compression-algorithms)) that the sender can decompress, each a varint, filling the Option Value. -An endpoint that includes this option MUST list `deflate` (1); the identifier `none` (0) MUST NOT be listed (it requires no negotiation). +The identifier `none` (0) MUST NOT be listed (it requires no negotiation); an endpoint lists whichever other algorithms it supports. An endpoint that does not support the extension omits the option. A sender MUST NOT compress with an algorithm the receiver did not advertise, and MUST NOT compress before it has received the receiver's COMPRESSION option. @@ -89,7 +89,7 @@ COMPRESSION Track Property { **Value**: The Algorithm identifier the publisher used for this track's payloads (see [Compression Algorithms](#compression-algorithms)). The absence of the property, or a value of `none` (0), means the track is uncompressed and its payloads are always transmitted verbatim. -The publisher MUST choose an algorithm that its peer advertised in the [COMPRESSION Setup Option](#setup-negotiation); since `deflate` is mandatory to implement, it is always a safe choice. +The publisher MUST choose an algorithm that its peer advertised in the [COMPRESSION Setup Option](#setup-negotiation), or `none` if the peer advertised none the publisher can produce. The property is fixed for the lifetime of the track and MUST NOT change. A relay MUST forward it unchanged on every hop, including a hop that has not negotiated the extension: there it is simply an ignored unknown Key-Value-Pair, but forwarding it lets a further-downstream hop that does negotiate the extension still act on the publisher's algorithm. @@ -124,13 +124,13 @@ A relay or cache can hold the object payloads compressed in memory and forward t ## Compression Algorithms {#compression-algorithms} This document defines the following algorithms. -| ID | Name | Requirement | Description | -|---:|:--------|:------------|:--------------------------------------------------------| -| 0 | none | — | Verbatim; the absence of compression. Never advertised. | -| 1 | deflate | mandatory | Raw DEFLATE {{RFC1951}}, with no zlib or gzip framing. | -| 2 | zstd | optional | Zstandard {{RFC8878}}. | +| ID | Name | Description | +|---:|:--------|:--------------------------------------------------------| +| 0 | none | Verbatim; the absence of compression. Never advertised. | +| 1 | deflate | Raw DEFLATE {{RFC1951}}, with no zlib or gzip framing. | +| 2 | zstd | Zstandard {{RFC8878}}. | -Every endpoint that advertises this extension MUST implement `deflate`, so the publisher always has a safe choice; `zstd` is optional. +Endpoints advertise whichever of these algorithms they support; none is mandatory, and a publisher uses one its peer advertised (or `none` if they share none). Further algorithms MAY be registered (see [IANA Considerations](#iana-considerations)). diff --git a/draft-lcurley-moq-lite.md b/draft-lcurley-moq-lite.md index 5bb6129..af1cc49 100644 --- a/draft-lcurley-moq-lite.md +++ b/draft-lcurley-moq-lite.md @@ -633,7 +633,7 @@ A relay MUST NOT forward the Path Parameter; like other per-hop setup metadata i ### Compression Parameter {#compression-parameter} The Compression Parameter advertises the payload compression [algorithms](#compression) the sender can *decompress* on this hop. The Parameter Value is a sequence of algorithm identifiers, each a variable-length integer, packed back-to-back to fill the Parameter Length. -An endpoint that supports compression MUST include `deflate` (1); `none` (0) MUST NOT be listed. An endpoint that does not support compression omits the parameter. +The identifier `none` (0) MUST NOT be listed. An endpoint that does not support compression omits the parameter. A relay MUST NOT forward the parameter (see [Session](#session)); it is negotiated independently on each hop. The list constrains what a publisher may use: a publisher MUST set `Publisher Compression` to an algorithm its peer advertised here (see [TRACK_INFO](#track-info)), and more generally a sender MUST NOT compress with an algorithm the receiver did not advertise, nor compress at all before it has received the receiver's Compression Parameter. @@ -869,7 +869,7 @@ The compression algorithm the original publisher applied to this Track's payload - `0` (`none`): payloads are uncompressed (the default). - `1` (`deflate`) or `2` (`zstd`): payloads are compressed with that algorithm. -The publisher MUST choose an algorithm its peer advertised in the [Compression Parameter](#compression-parameter); `deflate` is mandatory to support, so it is always a safe choice. +The publisher MUST choose an algorithm its peer advertised in the [Compression Parameter](#compression-parameter), or `0` if the peer advertised none the publisher can produce. The value is fixed for the lifetime of the Track and forwarded unchanged by relays, so even a hop that does not compress passes it along for a further-downstream hop to act on. It does not by itself cause compression: a receiver decompresses if and only if the value names a non-`none` algorithm and the receiver advertised that algorithm in its own [Compression Parameter](#compression-parameter); otherwise the payloads are verbatim (see [Compression](#compression)). A subscriber that does not recognize the value treats the payloads as verbatim, so an unknown future algorithm degrades to uncompressed rather than blocking the Track. @@ -885,13 +885,13 @@ A hop compresses a Track's payloads when, and only when, `Publisher Compression` The following algorithms are defined: -| ID | Name | Requirement | Description | -|---:|:--------|:------------|:--------------------------------------------------------| -| 0 | none | — | Verbatim; the absence of compression. Never advertised. | -| 1 | deflate | mandatory | Raw DEFLATE {{!RFC1951}}, with no zlib or gzip framing. | -| 2 | zstd | optional | Zstandard {{!RFC8878}}. | +| ID | Name | Description | +|---:|:--------|:--------------------------------------------------------| +| 0 | none | Verbatim; the absence of compression. Never advertised. | +| 1 | deflate | Raw DEFLATE {{!RFC1951}}, with no zlib or gzip framing. | +| 2 | zstd | Zstandard {{!RFC8878}}. | -Every endpoint that supports compression MUST implement `deflate`, so the publisher always has a safe choice; `zstd` is optional, and further algorithms MAY be defined by future extensions. +Endpoints advertise whichever of these algorithms they support; none is mandatory, and a publisher uses one its peer advertised, or `none` if they share none. Further algorithms MAY be defined by future extensions. Compression is **group-scoped** and applied only to frame payloads, never to the FRAME framing. Within a group the payloads form a single compressed stream in the Track's algorithm, reset at each group boundary, whose output is partitioned at frame boundaries: the compressor flushes at the end of each frame so that frame's slice is exactly the bytes stored in its `Payload` (delimited by `Message Length`). Both algorithms provide such a flush that retains the window (DEFLATE's sync flush; Zstandard's `ZSTD_e_flush`), so later frames in a group reuse the compression context. A receiver maintains a single decoder per group, reset at each group boundary, and feeds each frame's `Payload` through it in order: the first frame starts the decoder fresh — so a subscriber joining at a group boundary needs nothing earlier — while later frames retain cross-frame redundancy across the group. There is no shared state between groups; a frame with an empty payload contributes nothing to the stream. @@ -1114,7 +1114,7 @@ A generic library or relay MUST NOT inspect or modify the decompressed contents - Removed `Publisher Max Latency`. The publisher's retention guarantee is no longer part of the wire format; retention for FETCH and future subscriptions is best-effort and left to the publisher. - Timestamp-based expiration replaces wall-clock arrival time when a Track timescale is negotiated. - Added QUIC datagram delivery for groups, sharing Subscribe IDs with existing subscriptions (no separate control stream). -- Added payload compression, scoped per group. `Publisher Compression` in TRACK_INFO now names the algorithm the publisher used (`none`/`deflate`/`zstd`), and a new SETUP `Compression` parameter carries the algorithms each endpoint can decompress on a hop. The publisher MUST pick an algorithm its peer advertised; `deflate` is mandatory (so always safe) and `zstd` is optional. There is no per-group or per-frame flag on the wire — a receiver decompresses iff `Publisher Compression` names a non-`none` algorithm it advertised, else treats payloads as verbatim. When compressed, a group's frame payloads (only the payloads, never the framing) form a single stream in that algorithm, reset at each group boundary and sliced per frame into each frame's opaque `Payload`, with the algorithm's redundant container bytes omitted (the RFC 7692 trim for deflate; magicless, checksum-less frames for zstd) since moq-lite frames the slices itself; the decoder keeps one context per group. Keeping the framing uncompressed lets a relay store payloads compressed and re-frame across transport versions without recompressing. +- Added payload compression, scoped per group. `Publisher Compression` in TRACK_INFO now names the algorithm the publisher used (`none`/`deflate`/`zstd`), and a new SETUP `Compression` parameter carries the algorithms each endpoint can decompress on a hop. The publisher MUST pick an algorithm its peer advertised (or `none` if they share none); `deflate` and `zstd` are defined, neither mandatory. There is no per-group or per-frame flag on the wire — a receiver decompresses iff `Publisher Compression` names a non-`none` algorithm it advertised, else treats payloads as verbatim. When compressed, a group's frame payloads (only the payloads, never the framing) form a single stream in that algorithm, reset at each group boundary and sliced per frame into each frame's opaque `Payload`, with the algorithm's redundant container bytes omitted (the RFC 7692 trim for deflate; magicless, checksum-less frames for zstd) since moq-lite frames the slices itself; the decoder keeps one context per group. Keeping the framing uncompressed lets a relay store payloads compressed and re-frame across transport versions without recompressing. - Added Qmux [qmux] transport bindings for TCP/TLS and WebSocket, for environments where UDP is unavailable. The WebSocket binding uses the WebSocket message framing in place of the Qmux Record `Size` field. ## moq-lite-04 From 0c21c1924622f0af67c3599ac99d3ea6afecec49 Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 23 Jun 2026 02:32:49 +0000 Subject: [PATCH 09/10] compression: clarify the SETUP list is decode capability only The Compression parameter/option lists algorithms an endpoint can decompress; it does not advertise what it can produce. When sending, an endpoint compresses with an algorithm the receiver advertised. Resolves recurring ambiguity now that the publisher names the algorithm in the track property (so the receiver reads it rather than computing a mutual selection). Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_01W8bLV6vHzucLNvDhPk3bMP --- draft-lcurley-moq-compression.md | 4 ++-- draft-lcurley-moq-lite.md | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/draft-lcurley-moq-compression.md b/draft-lcurley-moq-compression.md index ee8a57d..d9ccc4d 100644 --- a/draft-lcurley-moq-compression.md +++ b/draft-lcurley-moq-compression.md @@ -65,8 +65,8 @@ COMPRESSION Setup Option { ~~~ **Algorithm**: -One or more Algorithm identifiers (see [Compression Algorithms](#compression-algorithms)) that the sender can decompress, each a varint, filling the Option Value. -The identifier `none` (0) MUST NOT be listed (it requires no negotiation); an endpoint lists whichever other algorithms it supports. +One or more Algorithm identifiers (see [Compression Algorithms](#compression-algorithms)) that the sender can decompress, each a varint, filling the Option Value — its decode capability. An endpoint does not advertise which algorithms it can *produce*; when sending, it compresses with one the receiver advertised. +The identifier `none` (0) MUST NOT be listed (it requires no negotiation). An endpoint that does not support the extension omits the option. A sender MUST NOT compress with an algorithm the receiver did not advertise, and MUST NOT compress before it has received the receiver's COMPRESSION option. diff --git a/draft-lcurley-moq-lite.md b/draft-lcurley-moq-lite.md index af1cc49..b04dd4f 100644 --- a/draft-lcurley-moq-lite.md +++ b/draft-lcurley-moq-lite.md @@ -631,7 +631,7 @@ The remaining bindings convey the path in their own handshake. A relay MUST NOT forward the Path Parameter; like other per-hop setup metadata it applies only to this hop (see [Session](#session)). ### Compression Parameter {#compression-parameter} -The Compression Parameter advertises the payload compression [algorithms](#compression) the sender can *decompress* on this hop. +The Compression Parameter advertises the payload compression [algorithms](#compression) the sender can *decompress* on this hop — its decode capability. An endpoint does not advertise which algorithms it can *produce*; when sending, it compresses with one the receiver advertised. The Parameter Value is a sequence of algorithm identifiers, each a variable-length integer, packed back-to-back to fill the Parameter Length. The identifier `none` (0) MUST NOT be listed. An endpoint that does not support compression omits the parameter. From c5d13f6a93c9094407c3b00b53105a4575b8458b Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 23 Jun 2026 21:22:29 +0000 Subject: [PATCH 10/10] moq-lite: add Decompressed Length; rename FRAME Message Length -> Payload Length MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit When a group is compressed, each FRAME and datagram now carries a Decompressed Length varint (the payload's size after decompression), present only when compressed. A receiver uses it to size the output buffer in one allocation and as a precise per-frame decompression-bomb bound: reject if it exceeds the receiver's limit, and reset if the decoder produces a different number of bytes than declared. The security section is tightened to this per-frame rule (plus a SHOULD cumulative-per-group bound for accumulation). A plain absolute varint, not a delta from Payload Length: effective compression makes the two lengths diverge, so a delta would be larger than the absolute compressed size exactly when compression works. FRAME's Message Length is renamed Payload Length, which is more accurate — it has only ever delimited the Payload, not the whole message (Timestamp Delta and now Decompressed Length precede it). moq-compression is left on its cumulative-subgroup bound; a per-object declared size would be an expensive object KVP in moq-transport. Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_01W8bLV6vHzucLNvDhPk3bMP --- draft-lcurley-moq-lite.md | 30 ++++++++++++++++++------------ 1 file changed, 18 insertions(+), 12 deletions(-) diff --git a/draft-lcurley-moq-lite.md b/draft-lcurley-moq-lite.md index b04dd4f..1fc468d 100644 --- a/draft-lcurley-moq-lite.md +++ b/draft-lcurley-moq-lite.md @@ -499,12 +499,13 @@ DATAGRAM Body { Subscribe ID (i) Group Sequence (i) [Timestamp (i)] + [Decompressed Length (i)] Payload (b) } ~~~ -`Timestamp` is present only when the Track's `Publisher Timescale` (see [TRACK_INFO](#track-info)) is non-zero. -When `Publisher Timescale` is 0, the field is omitted from the wire and the datagram body is just `Subscribe ID`, `Group Sequence`, and `Payload`. +`Timestamp` is present only when the Track's `Publisher Timescale` (see [TRACK_INFO](#track-info)) is non-zero, and `Decompressed Length` only when the Track is compressed on this hop (see [Compression](#compression)). +Each is independently omitted when its condition does not hold; with both absent the datagram body is just `Subscribe ID`, `Group Sequence`, and `Payload`. **Subscribe ID**: The Subscribe ID of an active subscription on the same session. @@ -520,7 +521,7 @@ Any varint value (including 0) is a valid absolute timestamp. **Payload**: The frame payload, extending to the end of the datagram. -A datagram is a single-frame group: when the Track is compressed on this hop (see [Compression](#compression)) the payload is a single compressed stream in the Track's algorithm, otherwise it is verbatim. +A datagram is a single-frame group: when the Track is compressed on this hop (see [Compression](#compression)) the payload is a single compressed stream in the Track's algorithm — with `Decompressed Length` giving its post-decompression size for the same allocation and bound role as in [FRAME](#frame) — otherwise it is verbatim. The total datagram body (including all header fields above and the compressed payload if applicable) MUST NOT exceed 1200 bytes. This limit ensures the datagram fits within the minimum QUIC path MTU without IP-layer fragmentation. Payloads that would not fit MUST be sent as a Group Stream instead. @@ -893,7 +894,7 @@ The following algorithms are defined: Endpoints advertise whichever of these algorithms they support; none is mandatory, and a publisher uses one its peer advertised, or `none` if they share none. Further algorithms MAY be defined by future extensions. -Compression is **group-scoped** and applied only to frame payloads, never to the FRAME framing. Within a group the payloads form a single compressed stream in the Track's algorithm, reset at each group boundary, whose output is partitioned at frame boundaries: the compressor flushes at the end of each frame so that frame's slice is exactly the bytes stored in its `Payload` (delimited by `Message Length`). Both algorithms provide such a flush that retains the window (DEFLATE's sync flush; Zstandard's `ZSTD_e_flush`), so later frames in a group reuse the compression context. A receiver maintains a single decoder per group, reset at each group boundary, and feeds each frame's `Payload` through it in order: the first frame starts the decoder fresh — so a subscriber joining at a group boundary needs nothing earlier — while later frames retain cross-frame redundancy across the group. There is no shared state between groups; a frame with an empty payload contributes nothing to the stream. +Compression is **group-scoped** and applied only to frame payloads, never to the FRAME framing. Within a group the payloads form a single compressed stream in the Track's algorithm, reset at each group boundary, whose output is partitioned at frame boundaries: the compressor flushes at the end of each frame so that frame's slice is exactly the bytes stored in its `Payload` (delimited by `Payload Length`, with the decompressed size in `Decompressed Length`). Both algorithms provide such a flush that retains the window (DEFLATE's sync flush; Zstandard's `ZSTD_e_flush`), so later frames in a group reuse the compression context. A receiver maintains a single decoder per group, reset at each group boundary, and feeds each frame's `Payload` through it in order: the first frame starts the decoder fresh — so a subscriber joining at a group boundary needs nothing earlier — while later frames retain cross-frame redundancy across the group. There is no shared state between groups; a frame with an empty payload contributes nothing to the stream. Because moq-lite delimits the slices itself, each algorithm's own redundant boundary and container bytes are omitted: for `deflate`, the four `00 00 FF FF` bytes a sync flush emits are removed from each `Payload` and the decoder re-inserts them (as in {{?RFC7692}}); for `zstd`, the per-group stream uses the magicless frame format and omits the content checksum. @@ -1069,14 +1070,15 @@ The FRAME message is a payload within a group. ~~~ FRAME Message { [Timestamp Delta (i)] - Message Length (i) + [Decompressed Length (i)] + Payload Length (i) Payload (b) } ~~~ -`Timestamp Delta` is present only when the Track's `Publisher Timescale` (see [TRACK_INFO](#track-info)) is non-zero. -When `Publisher Timescale` is 0, the field is omitted from the wire and the FRAME consists of just `Message Length` and `Payload`. -The framing is never compressed; only the `Payload` is (see [Compression](#compression)), so `Message Length` is always the on-wire `Payload` size. +`Timestamp Delta` is present only when the Track's `Publisher Timescale` (see [TRACK_INFO](#track-info)) is non-zero, and `Decompressed Length` only when the group is compressed (see [Compression](#compression)). +Each is independently omitted when its condition does not hold; with both absent the FRAME is just `Payload Length` and `Payload`. +The framing is never compressed; only the `Payload` is, so `Payload Length` is always the on-wire `Payload` size. **Timestamp Delta**: A signed delta from the previous frame's timestamp, in the Track's negotiated `Timescale`. @@ -1088,9 +1090,13 @@ Encoded as a zigzag-mapped variable-length integer: Zigzag interleaves non-negative and negative values (`0 → 0, -1 → 1, 1 → 2, -2 → 3, 2 → 4, ...`) so small magnitudes of either sign fit in a 1-byte varint and there is exactly one wire encoding for zero. The first frame of a group is delta-encoded from `0`, so its `Timestamp Delta` is the zigzag encoding of the absolute timestamp. +**Decompressed Length**: +The size in bytes of this frame's `Payload` after decompression, present only when the group is compressed. +A receiver uses it to size the output buffer and as a per-frame bound: it MUST reject a value larger than its configured limit, and MUST reset the stream if decompressing the `Payload` yields a different number of bytes than this declares (see [Security Considerations](#security-considerations)). + **Payload**: An application-specific payload. -When the group is compressed (see [Compression](#compression)) this holds this frame's slice of the group's compressed stream and `Message Length` is its compressed size; otherwise it is verbatim. The framing around it is never compressed. +When the group is compressed (see [Compression](#compression)) this holds this frame's slice of the group's compressed stream and `Payload Length` is its compressed (on-wire) size; otherwise it is verbatim. The framing around it is never compressed. A generic library or relay MUST NOT inspect or modify the decompressed contents unless otherwise negotiated; recompression that preserves the decompressed bytes exactly is allowed (see [Compression](#compression)). @@ -1114,7 +1120,7 @@ A generic library or relay MUST NOT inspect or modify the decompressed contents - Removed `Publisher Max Latency`. The publisher's retention guarantee is no longer part of the wire format; retention for FETCH and future subscriptions is best-effort and left to the publisher. - Timestamp-based expiration replaces wall-clock arrival time when a Track timescale is negotiated. - Added QUIC datagram delivery for groups, sharing Subscribe IDs with existing subscriptions (no separate control stream). -- Added payload compression, scoped per group. `Publisher Compression` in TRACK_INFO now names the algorithm the publisher used (`none`/`deflate`/`zstd`), and a new SETUP `Compression` parameter carries the algorithms each endpoint can decompress on a hop. The publisher MUST pick an algorithm its peer advertised (or `none` if they share none); `deflate` and `zstd` are defined, neither mandatory. There is no per-group or per-frame flag on the wire — a receiver decompresses iff `Publisher Compression` names a non-`none` algorithm it advertised, else treats payloads as verbatim. When compressed, a group's frame payloads (only the payloads, never the framing) form a single stream in that algorithm, reset at each group boundary and sliced per frame into each frame's opaque `Payload`, with the algorithm's redundant container bytes omitted (the RFC 7692 trim for deflate; magicless, checksum-less frames for zstd) since moq-lite frames the slices itself; the decoder keeps one context per group. Keeping the framing uncompressed lets a relay store payloads compressed and re-frame across transport versions without recompressing. +- Added payload compression, scoped per group. `Publisher Compression` in TRACK_INFO now names the algorithm the publisher used (`none`/`deflate`/`zstd`), and a new SETUP `Compression` parameter carries the algorithms each endpoint can decompress on a hop. The publisher MUST pick an algorithm its peer advertised (or `none` if they share none); `deflate` and `zstd` are defined, neither mandatory. There is no per-group or per-frame flag on the wire — a receiver decompresses iff `Publisher Compression` names a non-`none` algorithm it advertised, else treats payloads as verbatim. When compressed, a group's frame payloads (only the payloads, never the framing) form a single stream in that algorithm, reset at each group boundary and sliced per frame into each frame's opaque `Payload`, with the algorithm's redundant container bytes omitted (the RFC 7692 trim for deflate; magicless, checksum-less frames for zstd) since moq-lite frames the slices itself; the decoder keeps one context per group. Keeping the framing uncompressed lets a relay store payloads compressed and re-frame across transport versions without recompressing. A compressed frame also carries a `Decompressed Length` (post-decompression size, for buffer allocation and as a per-frame decompression bound), and FRAME's `Message Length` is renamed `Payload Length` (it has only ever delimited the payload). - Added Qmux [qmux] transport bindings for TCP/TLS and WebSocket, for environments where UDP is unavailable. The WebSocket binding uses the WebSocket message framing in place of the Qmux Record `Size` field. ## moq-lite-04 @@ -1215,11 +1221,11 @@ TODO: general security considerations. ## Payload Compression Compressing data that mixes attacker-controlled and secret material in the same payload can leak the secret through the compressed size, as in the CRIME and BREACH attacks. -A publisher MUST NOT set a non-zero `Publisher Compression` hint on a Track whose payloads combine secret material with attacker-influenced material. +A publisher MUST NOT compress a Track (set a non-`none` `Publisher Compression`) whose payloads combine secret material with attacker-influenced material. Because compression is group-scoped, the exposure is bounded to within a single group — which may combine several frames, a wider window than a single frame — but it is not eliminated. A malicious sender could emit a small compressed payload that decompresses to a very large buffer (a "decompression bomb"). -Because compression is group-scoped, a receiver MUST bound the cumulative decompressed size of a group stream — not merely each frame's slice, since many small slices can otherwise accumulate without limit; if the bound is exceeded it MUST reset the affected stream rather than allocate unbounded memory, and MAY close the session with a PROTOCOL_VIOLATION if it considers the peer abusive. +Each compressed frame declares its post-decompression size in `Decompressed Length` (see [FRAME](#frame)): a receiver MUST reject a frame whose `Decompressed Length` exceeds its configured limit, and MUST reset the stream if decompressing the `Payload` yields more bytes than declared, so it never allocates or emits more than that bounded size. A receiver SHOULD also bound the cumulative decompressed size across a group, since many in-bound frames can still accumulate. On any breach it MUST reset the affected stream rather than allocate unbounded memory, and MAY close the session with a PROTOCOL_VIOLATION if it considers the peer abusive. Compression is orthogonal to end-to-end encryption: an encrypted payload is effectively incompressible, so a publisher using end-to-end encryption SHOULD leave `Publisher Compression` at `0`.