Skip to content

Share portsend bandwidth with a 64KB quantum - #33

Merged
dominicletz merged 3 commits into
mainfrom
cursor/port-mux-unit-af36
Sep 25, 2026
Merged

dominicletz merged 3 commits into
mainfrom
cursor/port-mux-unit-af36

Conversation

@dominicletz

@dominicletz dominicletz commented Sep 25, 2026 •

Copy link
Copy Markdown
Member

What

Network.Mux now shares an edge socket across portsend partitions instead of always serving the lightest queue.

  • A partition sends at most 64KB (quantum/0), then the ring rotates. Weight is payload bytes, not :erts_debug.flat_size/1.
  • A single frame larger than 64KB is still sent, then the next partition runs.
  • Frames sit in :queues, so enqueue is O(1).
  • A push while the writer is blocked in await joins the ring. It does not jump ahead of a frame that is already queued.

Network.Sender still owns the SSL write and the ack replies.

Tests

DIODE_MINIMAL_TEST=1 mix test --no-start test/network/mux_test.exs in CI.

  • Quantum-sized frames alternate in insertion order.
  • A 200KB chunk does not jump a queue of small frames that was already waiting.
  • A short queue waits its turn instead of being served exclusively.
  • A port that joins late does not preempt the port already in the ring.
  • Two 1KB streams each yield after one quantum.
  • One 64KB coalesce takes a quantum from the current port, then the next.
  • Performance: 8k frames cost less than 10x the time of 2k frames, and 100 partitions each pop once quickly.

Not in this change

There is still no per-port memory cap. Fair rotation stops one download from monopolizing the socket; a stalled reader can still grow the other port's queue. The client scheduler is the paired diode_client_ex PR.

Open in Web Open in Cursor 

cursoragent and others added 3 commits September 25, 2026 07:19
Network.Sender still writes the socket, but partition ordering and
64KB coalescing are now a pure queue. CI runs the stress tests that
show flat_size priority and unbounded buffering under two downloads.

Co-authored-by: dominic.letz <dominic.letz@berlin.de>
Map iteration order differs between OTP 28.3 and 28.5. The stress
tests now assert that one partition is drained before the other,
without naming the winner.

Co-authored-by: dominic.letz <dominic.letz@berlin.de>
Partitions rotate once they have sent 64KB, weighted by payload
bytes. A push no longer skips the queue while the writer is waiting.
Tests cover interleaving and linear enqueue cost.

Co-authored-by: dominic.letz <dominic.letz@berlin.de>
@cursor cursor Bot changed the title Extract the edge port multiplexer into a testable unit Share portsend bandwidth with a 64KB quantum Sep 25, 2026
@dominicletz
dominicletz marked this pull request as ready for review September 25, 2026 10:27
@dominicletz
dominicletz merged commit 8a2d1f1 into main Sep 25, 2026
2 checks passed
@dominicletz
dominicletz deleted the cursor/port-mux-unit-af36 branch September 25, 2026 10:27

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit d4bc3c4. Configure here.

Comment thread lib/network/sender.ex
{:ok, mux, data, meta} ->
ack(meta)
GenServer.reply(from, data)
{:reply, :ok, %Sender{state | mux: mux, waiting: nil}}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Push double-replies while writer waits

Medium Severity

When push runs while a writer is blocked in await, the frame is enqueued with the caller as meta and popped immediately. ack replies :ok to that caller, and deliver_waiting also returns {:reply, :ok}, so the caller gets two replies and a leftover {ref, :ok} remains in its mailbox.

Additional Locations (2)
Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit d4bc3c4. Configure here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants