[subhanyousafcwit]

Summary

Keep the dialog lifetime timer on a hashed timing wheel instead of one sorted list, so inserting, updating and removing a dialog timeout no longer depends on the number of active dialogs.

Details

insert_gen_timer_unsafe() places a new timeout by walking the sorted lifetime list back from the tail, with d_timer->lock held. When the new timeout is shorter than the ones already queued, the walk visits every active dialog. That is the normal case for scripts that set a short $DLG_timeout at the 200 OK (waiting for the ACK) and restore the call limit at the ACK, or that set per-call limits. Under load the walk becomes the bottleneck: at 1,000 calls/s with ~3,000 active dialogs it holds the lock for ~0.6 CPU and the other workers spin behind it (details and profile in #4458).

This is a generic problem (any script that sets dialog timeouts of different lengths), not tied to particular UACs.

Solution

• dlg_timer_wheel.h: a wheel of 16384 lists (one per second, about 4.5 hours per turn). A dialog due at second T is appended to list T & mask, so insert/update/remove are a few pointer writes.
• The timer routine (_get_gen_expired_dlgs()) looks only at the lists of the seconds since its previous run, all of them once if it is a whole turn or more behind, and takes the entries that are due. An entry more than a turn away stays in its list for a later turn.
• Entries are linked and marked exactly as in the sorted list (prev/next, FAKE_DIALOG_TL, prev = 0 and timeout = 0 when expired), so the callers of insert_dlg_timer(), update_dlg_timer(), remove_dlg_timer() and the timer handler are unchanged.
• A timeout of 0, or one whose second was already looked at, is due at the next second.
• dlg_db_handler.c writes tl.timeout of an already known dialog directly during a DB sync, without update_dlg_timer(). An entry found on a list that is not its own is moved to its own list when that list is looked at, so a raised timeout is honoured. (In the sorted list such an entry stays where it was and, while it is at the head, holds back the expiry of everything behind it.)
• Only the lifetime timer (d_timer) uses the wheel. The delete-delay timer, whose entries all share the same delay, keeps the list.

Unit tests (modules/dialog/test/test.c, run with make test-dialog):

• a model check with random inserts, updates and removals, lifetimes from 0 to several turns of the wheel, and the timer routine running every second, late, or more than a turn late: every entry comes out exactly once, at the first run at or after its second, with the expired marks set, and the wheel's lists stay consistent;
• a timeout raised in place expires on time, and one lowered in place expires no later than its old second.

All 20 dialog unit tests pass. Under load (4.0.3 with this change, same setup as in #4458): 6.4 CPUs at 1,000 calls/s instead of 14.2, normal setup times, and dialogs end at the same moments (60 s limit, 32 s ACK wait, normal calls).

Compatibility

No change to the script interface, the MI output or the DB/replication data: timeouts are stored and replicated as before. The wheel costs 16384 list heads in shared memory (384 KB on 64-bit). The expiry granularity stays one second, as before.

Closing issues

closes #4458