(Vibe?) building custom HTTP/2 and HTTP/3

I’m still reminding myself of this section from the vibeio announcement post:

But is it vibe-coded!?

Ah, we get it… 😅

We have named this asynchronous runtime “vibeio”, from “vibe” (from “vibe coding”, because this runtime was coded with help of AI) and a common suffix for Rust asynchronous runtimes, “-io”.

However, we have debugged some parts ourselves, like io_uring use-after-frees that led to memory corruption, some zombie process reaper-related test hangups on macOS or fixing UDP test failures on Windows (since ConnectEx function works on connection-bound sockets only; also, handles couldn’t be registered multiple times in I/O completion ports).

Well, I’m going to now write up on “vibe coding” (or rather “agentic engineering”, as I did write a proper spec, and also I was debugging some parts myself) custom HTTP/2 and HTTP/3 implementations, this time for vibeio-http (HTTP server library built on top of vibeio).

The process

First off, I wrote the spec for both implementation, referencing relevant RFCs (RFC 9113 for HTTP/2, RFC 9114 for HTTP/3), existing implementations (h2 crate for HTTP/2, h3 crate for HTTP/3). I extracted some information from h2 and h3 crates, for example:

fuzzing
HPACK and QPACK tests
h2spec and h3spec (both tools to test protocol compliance, for HTTP/2 and HTTP/3 respectively)

Also, I added interop tests with clients for h2 and h3 crates (both existing implementations from Hyperium).

Then I let an AI agent write an implementation plan, along with the broken-down checklist in a Markdown file. I added the RFCs, compliance test tools, and the resulting Markdown file in .git/info/exclude.

The initial prompt looked something like this:

Could you write a plan document in CUSTOM_HTTP2_IMPL.md file, planning out a custom, optimized, high-performance HTTP/2 server implementation in vibeio-http (to replace h2), along with custom HPACK implementation, step-by-step?

The custom implementation would include fuzzing, and fixtures with interop testing with h2. Also, there would be h2spec testing, and that can be committed as a GitHub Actions workflow.

Also, HPACK implementation would be tested as well, using real-world fixtures obtained from Go’s HPACK implementation, nghttp2’s HPACK implementation, and more.

Also, optionally, both custom HTTP server implementation and h2 can be benchmarked using cargo bench.

The changes would be committed one-by-one, while commits would be done with Assisted-by: OpenCode:deepseek-v4-flash trailer, and it can involve updating the plan document (although it, along with cloned copy of h2spec and rfc9113.txt, won’t be committed).

Resources:

https://github.com/summerwind/h2spec - GitHub repository for HTTP/2 specification test tool

rfc9113.txt in project root - the HTTP/2 specification

Afterwards, I let the AI agent implement, following the plan.

The AI agents

For the AI agent, I mainly used OpenCode, with these free models:

DeepSeek V4 Flash (this one’s most powerful model of all, at least according to Artificial Analysis)
Hy3
MiMo V2.5 (occasionally)
Big Pickle (occasionally)

I also used Codex agent with GPT-5.6 Terra for some performance optimization tasks, and I saw it worked really well for HTTP/2 and HPACK optimization!

Debugging protocol issues

Now here comes the most hyper-focusing part: I had to debug some protocol issues, since AI tends to produce rough code by default, although AI can successfully implement HTTP/2 and HTTP/3.

HTTP/2 breaking with Hyper’s HTTP client

This issue occurred, because the custom implementation sent WINDOW_UPDATE frame with a window increment of 0, which is not allowed, as I learned from an AI overview like this:

According to the HTTP/2 specification (RFC 7540), sending a WINDOW_UPDATE frame with a flow-control window increment of 0 is prohibited.

Stream-Level Errors: If the frame targets a specific stream (Stream ID != 0), the receiver MUST treat it as a stream error of type PROTOCOL_ERROR (Error Code 0x8).

Connection-Level Errors: If the frame targets the entire connection (Stream ID 0), the receiver MUST treat it as a connection error of type PROTOCOL_ERROR.

Consequently, implementations must ensure that the window_size_increment value is within the legal range of 1 to 2^31-1 octets. Sending a zero increment is considered a protocol violation and will trigger an immediate termination of the affected stream or connection.

I checked the code, and saw this:

#[inline]
pub(crate) async fn handle_data_frame(
    &mut self,
    stream_id: u32,
    end_stream: bool,
    data: Bytes,
) {
    self.writer
        .write_window_update(&mut self.out, stream_id, data.len() as u32);
    self.writer
        .write_window_update(&mut self.out, 0, data.len() as u32);

    // -- snip --
}

which I replaced with this (fixing the Hyper interoperability issue):

#[inline]
pub(crate) async fn handle_data_frame(
    &mut self,
    stream_id: u32,
    end_stream: bool,
    data: Bytes,
) {
    // Sending a WINDOW_UPDATE frame with a zero delta (increment) is explicitly prohibited
    // by the HTTP/2 specification and results in a STREAM_ERROR of type PROTOCOL_ERROR
    // (Error Code 23)
    if !data.is_empty() {
        self.writer
            .write_window_update(&mut self.out, stream_id, data.len() as u32);
        self.writer
            .write_window_update(&mut self.out, 0, data.len() as u32);
    }

    // -- snip --
}

I saw this issue when trying to debug HTTP/3 issues, so I spun up a simple nc <-> openssl s_client loop in a terminal, opened Wireshark, and saw the problematic packets, which Wireshark highlighted with a warning.

HTTP/3 hanging up with Brave (Chromium-based browser)

This issue is another one I saw with Wireshark, this time I saw hang-ups when QPACK decoding gets blocked (it works just fine when QPACK decoding isn’t blocked).

And I sent this prompt:

I got this:

When a QPACK decoder is blocked, it enters a blocked decoding state because it has received a header block with a Required Insert Count greater than its current Insert Count (the count of acknowledged dynamic table updates).

To resolve this and allow decoding to proceed, you must send data on the QPACK Encoder Stream containing the missing dynamic table insertions or duplications. Specifically:

Process Encoder Instructions: The decoder must receive and process the encoder instructions that update its dynamic table to satisfy the Required Insert Count.

Unblocking Condition: A stream becomes unblocked when the decoder’s Insert Count becomes greater than or equal to the Required Insert Count for all header blocks the decoder has started reading from that stream.

Flow Control: While blocked, the encoded header data SHOULD remain in the blocked stream’s flow-control window. Releasing flow control prematurely can lead to memory exhaustion attacks.

In summary, you cannot simply “send data” to unblock the decoder arbitrarily; you must send the specific QPACK encoder instructions (insertions/duplications) on the dedicated encoder stream that allow the decoder to update its state to match the Required Insert Count. Once the state is synchronized, the decoder can process the blocked header block and the stream becomes unblocked.

While in my HTTP/3 frame dump, there was ZERO instructions inside QPACK Encoder Stream data.

And it got fixed, with this summary:

Done. The fix is complete and verified.

Root cause: Insert With Literal Name (RFC 9204 §4.3.3) encodes the name length as a 5-bit prefix with the Huffman flag at bit 5. The code passed a 4-bit prefix to push_string/read_string, which placed the H flag at bit 4 — corrupting the QPACK encoder stream. Our own (buggy) decoder tolerated it, but strict decoders (Chromium) aborted the encoder stream, so referenced dynamic entries never got inserted → the client’s HEADERS stayed blocked forever.

Changes:

src/h3/qpack/encoder.rs:269 and :446 — name prefix 5 → 6 (one extra bit reserves the Huffman flag).

src/h3/qpack/decoder.rs:279 — matching fix read_string(..., 5, ...) → 6.

Added two regression tests in encoder.rs that parse the instruction with an independent strict RFC 9204 §4.3.3 decoder (proven to fail before the fix, pass after).

Verification: 67 QPACK unit tests pass, 10 h3_interop tests pass (including the dynamic-table client fixtures that simulate the Chromium-style strict decoder). The new tests fail on the unfixed code and pass with the fix.

When I tested again, the hangups are gone!

HTTP/3 intermittently erroring out with Firefox

This one was a lot more difficult issue… And AI actually did some non-fixes that didn’t target the root issue. And it was an intermittent issue that happens on a specific environment

So I added print statements, and saw that the response header sending failed. When I checked deeper, I found out that the QUIC connection was locally closed, and later that there’s a QPACK decoding error.

And later on, I found out that this is a bug in “feed decoder stream” function that led to QPACK decoder errors.

So I got Encoder::feed_decoder_stream QPACK decoder errors fixed… Also, AI suggested what I haven’t thought of - a possible bug in Decoder::feed_encoder_stream that can lead to QPACK encoder errors. This got fixed as well.

HPACK and QPACK bombs

When I was testing a HPACK bomb PoC against the custom implementation, I saw that the implementation was initially vulnerable to HPACK bombs (memory usage increased). So I read the code, and saw that SETTINGS_MAX_HEADER_LIST_SIZE was enforced per-field, not per-stream. But I saw from inserted debugging statements that the PoC sent many small header list fields (not one large field), which led to HPACK bombs.

I ended up fixing that, and it got fixed again when Codex + GPT-5.6 Terra was optimizing the HPACK implementation and ended up fixing a possible correctness issue.

Similar thing I did with QPACK (that HTTP/3 uses, while HTTP/2 uses HPACK), although I didn’t test any PoC against the custom implementation.

Huffman encoding performance issues

When I benchmarked the initial Huffman decoder implementation using LiteSpeed’s Huffman decoder benchmark setup, I saw it was slower than all other decoders, including “full LiteSpeed” (LiteSpeed alone uses optimized implementation of Huffman decoder that also wraps over “full” implementation).

So I let an AI agent adapt some code and 4-bit table from LiteSpeed’s HPACK library, and I tested again, and this time it got improved.

Also, I got it optimized even further with Codex + GPT-5.6 Terra xhigh…

Here’s are the performance benchmark results for Huffman decoder (I added ferron myself):

❯ time ./comp-dec idle.huff 500000 litespeed
./comp-dec idle.huff 500000 litespeed  4,07s user 0,01s system 99% cpu 4,108 total
❯ time ./comp-dec idle.huff 500000 litespeed-full
./comp-dec idle.huff 500000 litespeed-full  8,84s user 0,00s system 99% cpu 8,874 total
❯ time ./comp-dec idle.huff 500000 nginx
./comp-dec idle.huff 500000 nginx  7,29s user 0,00s system 99% cpu 7,316 total
❯ time ./comp-dec idle.huff 500000 ferron
./comp-dec idle.huff 500000 ferron  8,02s user 0,00s system 99% cpu 8,041 total

And Huffman encoder (I added ferron myself):

❯ time ./comp-enc idle.txt 2000000 h2o
./comp-enc idle.txt 2000000 h2o  10,01s user 0,00s system 99% cpu 10,040 total
❯ time ./comp-enc idle.txt 2000000 litespeed
./comp-enc idle.txt 2000000 litespeed  3,32s user 0,00s system 99% cpu 3,328 total
❯ time ./comp-enc idle.txt 2000000 nghttp2
./comp-enc idle.txt 2000000 nghttp2  13,99s user 0,00s system 99% cpu 14,027 total
❯ time ./comp-enc idle.txt 2000000 nginx
./comp-enc idle.txt 2000000 nginx  5,75s user 0,00s system 99% cpu 5,766 total
❯ time ./comp-enc idle.txt 2000000 ferron
./comp-enc idle.txt 2000000 ferron  6,67s user 0,00s system 99% cpu 6,695 total

They’re a bit slower than NGINX’s corresponding implementations, but at least the encoder seems to be “faster” than H2O’s and nghttp2’s.

The benefits

I got several benefits over previous h2 and h3 implementations, even though the original goal of this was to improve the server performance.

For HTTP/2, it’s improved HPACK header space savings (~95%, versus ~92% when using h2 crate; when benchmarking a Hello World application using h2load) and overall better network transfer-efficiency, as you can see in h2load benchmark results for a “Hello World” application proxied through new vibeio-http:

...
traffic: 26.72MB (28022841) total, 4.39MB (4606318) headers (space savings 95.64%), 9.36MB (9815429) data
...

versus the h2 implementation:

...
traffic: 31.46MB (32987333) total, 7.72MB (8096455) headers (space savings 92.79%), 9.95MB (10432344) data
...

For HTTP/3, it’s better h3spec compliance (one failure comes from QUIC implementation itself, quinn):

Failures:

  TransportError.hs:200:32:
  1) QUIC servers MUST send missing_extension TLS alert if the quic_transport_parameters extension does not included [TLS 8.2]
       did not get expected exception: QUICException

  To rerun use: --match "/QUIC servers/MUST send missing_extension TLS alert if the quic_transport_parameters extension does not included [TLS 8.2]/" --seed 691483842

Randomized with seed 691483842

Finished in 2.6137 seconds
49 examples, 1 failure

Versus h3:

Failures:

  TransportError.hs:200:32:
  1) QUIC servers MUST send missing_extension TLS alert if the quic_transport_parameters extension does not included [TLS 8.2]
       did not get expected exception: QUICException

  To rerun use: --match "/QUIC servers/MUST send missing_extension TLS alert if the quic_transport_parameters extension does not included [TLS 8.2]/" --seed 1385002365

  HTTP3Error.hs:63:17:
  2) HTTP/3 servers MUST send H3_MESSAGE_ERROR if a pseudo-header is duplicated [HTTP/3 4.1.1]
       did not get expected exception: QUICException

  To rerun use: --match "/HTTP/3 servers/MUST send H3_MESSAGE_ERROR if a pseudo-header is duplicated [HTTP/3 4.1.1]/" --seed 1385002365

  HTTP3Error.hs:125:21:
  3) HTTP/3 servers MUST send H3_FRAME_UNEXPECTED if CANCEL_PUSH is received in a request stream [HTTP/3 7.2.5]
       predicate failed on expected exception: QUICException
       ApplicationProtocolErrorIsReceived (ApplicationProtocolError 262) "received incomplete frame"

  To rerun use: --match "/HTTP/3 servers/MUST send H3_FRAME_UNEXPECTED if CANCEL_PUSH is received in a request stream [HTTP/3 7.2.5]/" --seed 1385002365

  HTTP3Error.hs:137:21:
  4) HTTP/3 servers MUST send QPACK_ENCODER_STREAM_ERROR if a new dynamic table capacity value exceeds the limit [QPACK 4.1.3]
       did not get expected exception: QUICException

  To rerun use: --match "/HTTP/3 servers/MUST send QPACK_ENCODER_STREAM_ERROR if a new dynamic table capacity value exceeds the limit [QPACK 4.1.3]/" --seed 1385002365

  HTTP3Error.hs:149:21:
  5) HTTP/3 servers MUST send QPACK_DECODER_STREAM_ERROR if Insert Count Increment is 0 [QPACK 4.4.3]
       did not get expected exception: QUICException

  To rerun use: --match "/HTTP/3 servers/MUST send QPACK_DECODER_STREAM_ERROR if Insert Count Increment is 0 [QPACK 4.4.3]/" --seed 1385002365

Randomized with seed 1385002365

Finished in 3.0860 seconds
49 examples, 5 failures

Also, since I saw h3 crate being declared “experimental” for quite some time, I think custom vibeio-http implementation would be an opportunity to mark HTTP/3 support in Ferron 3 as no longer experimental (Ferron 3 uses vibeio and vibeio-http by the way).

Conclusion

AI definitely helped me build custom implementation HTTP/2 and HTTP/3 protocol faster that have various benefits over h2 (better HPACK compression ratio) and h3 (better h3spec compliance), both from Hyperium (same organization that hosts the hyper crate, a popular HTTP library).

Although I had some struggles with debugging the implementation. I think it’s fair to say that AI shifted the bottleneck from writing code and initial implementation to architecting and debugging (of course, it’s possible to “vibe-debug”, but then vibe-coders would be stuck because of lack of fundamentals) systems.

The new implementation is available as vibeio-http 0.4.0, which you can use with Rust projects:

cargo add vibeio-http

Here are some links for you:

https://chat.qwen.ai/s/85387123-c3d4-4885-8cb7-4fe21c2ed3f3?fev=0.2.86 (Qwen Studio conversation which I shared some experiences with)
https://github.com/ferronweb/vibeio-http (the vibeio-http repository)
https://docs.rs/vibeio-http (the vibeio-http crate documentation)

Update (August 16, 2026): vibeio-http 0.4.0 is affected by HTTP/2 Rapid Reset, MadeYouReset, and CONTINUATION flood vulnerabilies. This has been addressed in vibeio-http 0.4.1.