Skip to content

proxy_protocol: advertise the original client on all outbound paths - #4457

Open
NormB wants to merge 3 commits into
OpenSIPS:masterfrom
NormB:fix/proxy-protocol-orig-src
Open

NormB wants to merge 3 commits into
OpenSIPS:masterfrom
NormB:fix/proxy-protocol-orig-src

Conversation

@NormB

@NormB NormB commented Nov 14, 2026

Copy link
Copy Markdown
Member

Summary
Make send_proxy_protocol sockets advertise the original client in the PROXY header on every outbound path, not only for stateless UDP forwarding.

Details
Bug fix. A send_proxy_protocol socket puts OpenSIPS' own address in the PROXY header whenever the request is relayed statefully, or over a stream transport:

  • tm sends through send_pr_buffer(), which passes no message to the transport, so the header is built from the outbound socket.
  • On TCP/TLS/WS/WSS the header is written when the connect completes, in TCP main, which has no access to the request that triggered the connect.

Two further problems showed up while testing this:

  • tls_async_queue_proxy_header() dereferences proto_data (TCP main private memory) from a worker and crashes it. This is independent of the fix: it reproduces on unpatched master and 4.0 with a TLS send_proxy_protocol socket when several requests race towards a new destination.
  • A worker and TCP main could both send the header on a new connection, leaving a second header in the middle of the stream.

Solution
Three commits:

  1. proxy_protocol: advertise the original endpoints on all outbound paths
    • msg_send_rcv() lets the caller name the request whose receive info is advertised; msg_send() becomes a wrapper, so existing callers are unchanged.
    • tm passes the UAS request for relayed branches, including the CANCEL and hop-by-hop ACK it generates for them. Replies and local transactions are not changed.
    • For stream transports the original endpoints are stored on the connection when it is created (pp_out) and the header is built from there.
  2. proto_tls: do not access the TCP main private conn data from workers
  3. proxy_protocol: send the stream PROXY header only from the writer

Tested with a scripted harness that sends raw SIP through OpenSIPS and checks the PROXY header each next hop receives, in both async and sync TCP modes (179 checks):

  • UDP, TCP, TLS, WS and WSS, IPv4 and IPv6, stateless and stateful
  • inbound allow_proxy_protocol chained to outbound
  • parallel and serial forking, DNS failover, CANCEL and negative ACK
  • dialog + topology_hiding as in [BUG] HAProxy PROXY Protocol implementation doesn't forward the original Source IP. #4456: sequential requests in both directions and the dialog-generated BYE
  • one header per connection under connection reuse and moderate load
  • a real HAProxy 3.0 in front (send-proxy) and behind (accept-proxy) on every stream transport

The header checks fail without the fix and pass with it. The harness also passes under AddressSanitizer.

Limitations:

  • proto_sctp and proto_ipsec are compiled only.
  • HAProxy cannot validate the header on UDP; that path is checked by the harness' own listeners.
  • No long soak or performance comparison.

Compatibility
struct tcp_connection gains a pp_out member. Sockets without send_proxy_protocol are not affected. With it, relayed requests now carry the client's address where they used to carry OpenSIPS' own, which is the documented behaviour.

The reporter runs 4.0.3; the three commits cherry-pick cleanly onto 4.0 and the harness passes there too.

Closing issues
closes #4456

A socket flagged with send_proxy_protocol is expected to announce the
source and destination of the received request. This only happened for
stateless forwarding over UDP, in all the other cases the header was
carrying the address of the OpenSIPS outbound socket itself:

* tm never passes a SIP message to msg_send(), so the transport layer
  had no receive info for relayed requests, their retransmissions and
  the locally generated CANCELs and hop-by-hop ACKs. Add msg_send_rcv(),
  taking the message which provides the receive info separately from
  the one handed to the raw processing callbacks, and have tm use the
  request stored in the transaction. Replies and local transactions
  are not changed.

* for stream transports (TCP, TLS, WS, WSS) the header is sent when
  the connect completes, in TCP main, where there is no message at
  all. Remember the original endpoints in the connection when it is
  created and use them when the header is built.

Fixes OpenSIPS#4456
The proto_data of a connection is allocated by TCP main in its private
memory. When a worker was sending on a TLS connection that had already
been initialized by TCP main, but did not get to send its PROXY header
yet, tls_async_queue_proxy_header() was dereferencing that pointer in
the wrong process, crashing the worker. This was seen when several
requests were racing towards a new destination over a socket using
send_proxy_protocol.

Only TCP main is allowed to use it; the workers have nothing to queue
there anyway, as the header is built by TCP main upon the first write.
@NormB
NormB requested a review from razvancrainea November 14, 2026 20:25
@NormB
NormB marked this pull request as ready for review November 14, 2026 20:25
A worker sending on a connection whose connect was just completed by
TCP main was queueing the PROXY header as an async chunk, while TCP
main was about to send the very same header by itself, as none of
them had flagged the header as sent yet. With several requests racing
towards a new destination, the connection ended up with a second
header in the middle of the stream, or with the header following the
first request.

The header is now sent only by the process doing the actual write on
the connection, ahead of the first data. The other processes only
attach the original endpoints to the connections they create.
@NormB
NormB force-pushed the fix/proxy-protocol-orig-src branch from 77c0e48 to d20b290 Compare November 14, 2026 21:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] HAProxy PROXY Protocol implementation doesn't forward the original Source IP.

1 participant