Code Map: LLUDPServer

Source: OpenSim/Region/ClientStack/Linden/UDP/LLUDPServer.cs
Namespace: OpenSim.Region.ClientStack.LindenUDP
Key classes: LLUDPServer, LLUDPServerShim

Overview

LLUDPServer is the UDP transport layer for a single region. It handles all inbound and outbound UDP traffic between the region and its connected viewers using the Linden Lab UDP (LLUDP) protocol.

One instance exists per region. It is not shared across regions even when multiple regions run in the same simulator process.

The region module entry point is LLUDPServerShim, a thin INonSharedRegionModule wrapper that instantiates and owns LLUDPServer. The shim handles the module lifecycle; the server handles the actual network I/O.

Threading Model

LLUDPServer runs three persistent threads per region:

Thread	Started by	Purpose
Inbound receive thread	`StartInbound()` via base class `OpenSimUDPBase`	Receives raw UDP datagrams from the OS socket and places them on `packetInbox`.
`IncomingPacketHandler`	`StartInbound()`	Drains `packetInbox`, dispatches packets to the correct `LLClientView` via `ProcessInPacket()`.
`OutgoingPacketHandler`	`StartOutbound()`	Drives per-client outbound queues, resends, ACKs, and pings on a timer loop.

A fourth engine, OqrEngine (Outgoing Queue Refill Engine), runs as a JobEngine thread pool to refill per-client outbound queues without blocking the outgoing packet handler.

Inbound path

OS socket
  → base class receive thread (OpenSimUDPBase)
    → PacketReceived()
      → decode + zerodecode
      → pending cache check (UseCircuitCode handling)
      → client lookup by endpoint
      → ACK processing (appended ACKs, PacketAck packets)
      → duplicate detection (PacketArchive)
      → packetInbox.Add()
        → IncomingPacketHandler thread
          → LLClientView.ProcessInPacket()

UseCircuitCode packets are handled specially: they are dispatched via FireAndForget to HandleUseCircuitCode(), which authenticates the circuit, creates a new LLClientView and LLUDPClient, and then reinjects any packets that arrived while the client was being set up (held in a per-endpoint pending queue).

Outbound path

Scene code / region modules
  → LLClientView.SendXxx()
    → LLUDPServer.SendPacket() / SendPacketData() / SendUDPPacket()
      → optional zerocoding
      → OutgoingPacket constructed
      → LLUDPClient.EnqueueOutgoing()
        if queue full or not throttled:
          → SendPacketFinal() → SyncSend() → OS socket
        else:
          → sits in per-client priority queue
            → OutgoingPacketHandler → DequeueOutgoing() → SendPacketFinal()

Key Structures

packetInbox

A BlockingCollection<IncomingPacket>. The receive thread produces; the IncomingPacketHandler thread consumes. Bounded only by memory.

TokenBucket / ThrottleRates

Throttle is a scene-wide token bucket capping total outbound bandwidth. ThrottleRates holds the per-client default rates for each traffic category (resend, texture, asset, land, wind, cloud, task, state, AvatarInfo). Both are configured from [ClientStack.LindenUDP].

Pending cache

An ExpiringCacheOS<IPEndPoint, Queue<UDPPacketBuffer>> keyed by endpoint. While a new client is being set up after a UseCircuitCode, all arriving packets for that endpoint are held here and reinjected once the client is ready. Entries expire after 60 seconds.

OqrEngine

JobEngine instance named "Outgoing Queue Refill Engine". When a client's outbound queue runs dry, a refill job is posted here rather than blocking the outgoing packet handler thread. This prevents one slow client from stalling outbound processing for all others.

Outgoing Packet Handler Timer Loop

The OutgoingPacketHandler thread runs a continuous loop with three chained timers:

Interval	Action
100ms	Check for unacked packets past their RTO; resend via `HandleUnacked()`.
500ms	Send accumulated ACKs for all clients via `SendAcks()`.
5000ms	Send ping checks to all clients via `SendPing()`.

If no packet was sent in a loop iteration and there are connected clients, the thread sleeps 15ms. If there are no clients, it sleeps 100ms.

ACK Handling

Inbound ACKs arrive in two forms:

Appended to the tail of any reliable packet (Header.AppendedAcks).
As explicit PacketAck packets.

Both are processed in PacketReceived() and passed to LLUDPClient.NeedAcks.Acknowledge().

Outbound ACKs are accumulated in LLUDPClient.PendingAcks and sent either appended to the next reliable outbound packet, or as a standalone PacketAck by the 500ms timer, whichever comes first. The threshold for immediate ACK sending is 2 * MTU bytes received since the last ACK.

Reliability and Resends

Reliable packets are added to LLUDPClient.NeedAcks after send. The 100ms timer calls HandleUnacked(), which retrieves all packets whose age exceeds the client's current RTO and invokes their UnackedMethod callback (default: ResendUnacked()).

ResendUnacked() sets the resent flag on the packet and re-enqueues it in the Resend throttle category.

A client is disconnected if no packet of any kind is received within the ack timeout (default 60 seconds; 300 seconds if the client is paused).

MTU and Zerocoding

MTU is fixed at 1400 bytes. Max payload is 1200 bytes.

Packets marked MSG_ZEROCODED are compressed by run-length encoding of zero bytes before send, and decoded on receive. If zerocoding makes a packet larger than the unencoded form, the flag is stripped and the unencoded data is sent instead.

Packets larger than the MTU are split into multiple datagrams via Packet.ToBytesMultiple() where the packet type supports variable blocks. CoarseLocationUpdate is explicitly excluded from splitting.

Config

All keys are in [ClientStack.LindenUDP] in OpenSim.ini:

Key	Default	Purpose
`client_socket_rcvbuf_size`	0 (OS default)	UDP receive buffer size passed to OS.
`scene_throttle_max_bps`	6250000	Scene-wide outbound bandwidth cap in bits/sec.
`TextureSendLimit`	20	Texture packets queued per OQR event.
`DefaultRTO`	0 (auto)	Default retransmission timeout in ms.
`MaxRTO`	0 (auto)	Maximum retransmission timeout in ms.
`AckTimeout`	60	Seconds before disconnecting a silent client.
`PausedAckTimeout`	300	Seconds before disconnecting a paused client.
`DisableFacelights`	false	Strip facelights from client updates.
`SupportViewerObjectsCache`	true	Enable viewer-side object cache support.

Statistics

LLUDPServer registers the following stats under the clientstack category (visible via the show stats console command):

InboxPacketsCount -- packets waiting in packetInbox.
IncomingUDPReceivesCount -- total UDP datagrams received.
IncomingPacketsProcessedCount -- packets dispatched to clients.
IncomingPacketsMalformedCount -- packets that could not be parsed.
IncomingPacketsOrphanedCount -- packets with no matching client.
IncomingPacketsResentCount -- inbound packets flagged as resends by clients.
OutgoingUDPSendsCount -- total UDP datagrams sent.
OutgoingPacketsResentCount -- packets resent due to missing ACK.
OutgoingPacketsQueuedCount -- packets queued across all clients.
OQRERequestsWaiting -- jobs pending in the OqrEngine.
AverageUDPProcessTime -- average ms per inbound UDP receive.
ClientLogoutsDueToNoReceives -- forced disconnects due to ack timeout.

OpenSimulator Internals/Code Map/LLUDPServer