Struct BytesGatherer 

pub struct BytesGatherer { /* private fields */ }

A scatter-gather buffer that threads large blobs through the response pipeline by reference, eliminating application-layer copies for bulk data such as post bundles.

Motivation

When serving a GetPostBundleV1 response the post bundle payload (potentially hundreds of KB, already brotli-compressed on disk) was previously copied multiple times before reaching the network. Then at the transport boundary the single Bytes was handed to HTTPS/TCP with no opportunity for vectored writes.

BytesGatherer fixes this by threading large blobs through the entire response path by reference. Small POD fields (header bytes, JSON metadata) accumulate into an internal BytesMut scratchpad, while large payloads are attached with push_bytes — a refcount bump only, no memcpy. At the transport boundary compact merges any adjacent small segments, sets Content-Length, and HTTP/2 receives a stream of Bytes chunks as separate DATA frames.

**Net result for GetPostBundle, the hottest-path of Hashiverse: ** multiple application-layer copies of the post bundle → 0. One kernel copy to the network card remains (unavoidable).

After compact(1024) the segment layout for a typical GetPostBundle response looks like:

Segment 0: [rpc_header (~300 B) + response_header (peers/token/counts) (~50 B) + flag (1 B)]
           ← merged by compact, one BytesMut flush
Segment 1: post_bundle Bytes (e.g. 500 KB)
           ← original Bytes from disk, zero copy

HTTP/2 sends these as 2 DATA frames. TCP concatenates to 1 contiguous buffer via to_bytes.

Implementations

impl BytesGatherer

pub fn from_bytes(bytes: Bytes) -> Self

pub fn put_u8(&mut self, v: u8)

pub fn put_u16_le(&mut self, v: u16)

pub fn put_u16(&mut self, v: u16)

pub fn put_u32_le(&mut self, v: u32)

pub fn put_u32(&mut self, v: u32)

pub fn put_u64(&mut self, v: u64)

pub fn put_slice(&mut self, s: &[u8])

pub fn put_bytes(&mut self, bytes: Bytes)

Push a large blob — flush accumulator first, store blob by reference (zero copy).

pub fn put_bytes_gatherer(&mut self, other: BytesGatherer)

Merge another gatherer — zero copy for blobs; fast path if both are still assembling.

If other has no committed segments yet (all bytes are still in its scratchpad), the scratchpad bytes are appended directly into self.current — no flush, no new segment.

pub fn is_empty(&self) -> bool

Returns true if the gatherer contains no bytes.

pub fn len(&self) -> usize

Total byte count across all segments + current accumulator.

pub fn compact(self, threshold: usize) -> Self

Merge adjacent segments smaller than threshold bytes into single segments. Large segments pass through untouched (zero copy). Generally, gall this at the transport boundary before streaming to optimise streaming window fragmentation.

pub fn finish(self) -> Vec<Bytes>

Consume into Vec<Bytes> for streaming / vectored I/O.

pub fn to_bytes(self) -> Bytes

Consume into one contiguous Bytes. Zero-copy fast path: single segment returned directly.

Trait Implementations

impl Default for BytesGatherer

fn default() -> BytesGatherer

Returns the “default value” for a type.