Jwbshaw: First

2026-07-07T11:37:27Z

First

New page

= OpenSimulator Internals/Code Map/LLUDP =

== Overview ==

LLUDP is the network transport layer for viewer communication. It implements the Second Life UDP wire protocol -- packet framing, zero-encoding, reliable delivery with ACKs, resend handling, and per-client throttling. It sits below IClientAPI and LLClientView.

Source: OpenSim/Region/ClientStack/Linden/UDP/LLUDPServer.cs

Class hierarchy:

LLUDPServer → OpenSimUDPBase

One LLUDPServer instance exists per region, attached via LLUDPServerShim (an INonSharedRegionModule).

----

== LLUDPServerShim ==

Thin adapter that lets LLUDPServer participate in the region module system.

# AddRegion(scene) -- creates the LLUDPServer bound to the region's internal IP/port, sets scene.RegionInfo.InternalEndPoint.Port to whatever the server actually bound to
# RegionLoaded(scene) -- calls Start()
# RemoveRegion(scene) -- calls Stop()
# AddScene(scene) -- registers ~10 StatsManager stats (client logouts, UDP receives/sends, packets processed/malformed/orphaned/resent, average process time)

----

== Constants ==

* MTU = 1400 bytes (maximum transmission unit for a single UDP packet)
* MAXPAYLOAD = 1200 bytes

----

== Construction ==

Config read from [ClientStack.LindenUDP]:

{| class="wikitable"
! Setting !! Default !! Notes
|-
| client_socket_rcvbuf_size || 0 (OS default) || Receive buffer size passed to the socket
|-
| scene_throttle_max_bps || 6250000 || Total bandwidth throttle for the whole region
|-
| TextureSendLimit || 20 || Texture packets queued per OnQueueEmpty trigger
|-
| DefaultRTO || 0 || Default retransmission timeout
|-
| MaxRTO || 0 || Maximum retransmission timeout
|-
| AckTimeout || 60 (seconds) || Time without any packet before a client is disconnected
|-
| PausedAckTimeout || 300 (seconds) || Extended timeout when client is paused (e.g. file upload dialog open)
|-
| DisableFacelights || false ||
|-
| SupportViewerObjectsCache || true ||
|}

Constructor also measures the actual resolution of Environment.TickCount and Util.GetTimeStampMS() on this hardware -- logged at startup, used for timing accuracy elsewhere.

----

== Start / Stop ==

Start() launches three things:
# StartInbound() -- base class UDP receive loop, plus a dedicated "Incoming Packets (RegionName)" thread running IncomingPacketHandler()
# StartOutbound() -- base class UDP send loop, plus a dedicated "Outgoing Packets (RegionName)" thread running OutgoingPacketHandler()
# OqrEngine.Start() -- JobEngine for outgoing queue refills, decoupled from per-connection thread pool jobs to avoid performance problems with many connections

Stop() reverses all three.

----

== Incoming Packet Path ==

PacketReceived(buffer) -- called by OpenSimUDPBase for every UDP datagram

# Decoding: validates minimum length (7 bytes), computes header length (7/8/10 bytes depending on extra header flags), zero-decodes if MSG_ZEROCODED flag set, calls Packet.BuildPacket()
# Malformed packets are dropped and counted (IncomingMalformedPacketCount) -- logs a warning every 10000 as a probable network attack indicator
# Packet-to-client mapping:
#* If the endpoint has a pending queue (client still being created), the packet is queued rather than processed, except UseCircuitCode resends which are acked and dropped
#* If no client exists yet and the packet is UseCircuitCode, a pending queue is created and HandleUseCircuitCode() is dispatched via FireAndForget
#* Otherwise the client is looked up by endpoint via Scene.TryGetClient() -- unrecognized senders are counted as orphaned packets
# ACK receiving: appended ACKs and standalone PacketAck packets are applied to udpClient.NeedAcks
# ACK sending: reliable packets get their sequence number queued in PendingAcks; ACKs are flushed once 2*MTU bytes have been received since the last flush
# Duplicate detection: PacketArchive tracks recently seen reliable sequence numbers -- duplicates (resent or not) are dropped after the ack machinery has already run
# Ping handling: StartPingCheck/CompletePingCheck are handled inline and never reach the packet inbox
# Everything else is wrapped in an IncomingPacket and added to packetInbox (a BlockingCollection) for the IncomingPacketHandler thread to process

IncomingPacketHandler() thread: pulls from packetInbox with a 4500ms timeout, calls client.ProcessInPacket() for each. Exceptions in one client's packet do not crash the loop.

----

== Outgoing Packet Path ==

SendPacket() / SendPacketData()

# Packets larger than MTU with variable blocks are split via packet.ToBytesMultiple() (CoarseLocationUpdate is exempt -- cannot be split)
# Zero-encoding applied if MSG_ZEROCODED flag is set on the packet
# Wrapped in an OutgoingPacket and either queued via udpClient.EnqueueOutgoing() or sent immediately via SendPacketFinal()

SendPacketFinal():
# Appends pending ACKs to plain reliable packets if there's room (up to 256, then flags MSG_APPENDED_ACKS)
# Assigns a new sequence number (unless this is a resend, which keeps its original data but sets MSG_RESENT)
# Calls SyncSend() -- actual socket write
# If the packet has no custom UnackedMethod, the buffer is freed immediately (fire-and-forget). Otherwise it's added to udpClient.NeedAcks awaiting acknowledgment

OutgoingPacketHandler() thread loop (runs continuously):
# Every 100ms: triggers HandleUnacked() checks (resend expired packets)
# Every 500ms (5 x 100ms): triggers SendAcks()
# Every 5000ms (10 x 500ms): triggers SendPing()
# Calls Scene.ForEachClient() with ClientOutgoingPacketHandler, which dequeues throttled outgoing packets per client
# Sleeps 100ms if no clients connected, 15ms if nothing was sent this round (matches typical OS tick granularity), otherwise loops immediately

----

== Reliability ==

{| class="wikitable"
! Mechanism !! Notes
|-
| NeedAcks || Per-client collection of sent reliable packets awaiting acknowledgment, with expiry based on RTO (retransmission timeout)
|-
| PendingAcks || Per-client queue of sequence numbers to be acknowledged back to the sender
|-
| PacketArchive || Per-client record of recently received reliable sequence numbers, used to detect and drop duplicates
|-
| ResendUnacked() || Sets MSG_RESENT flag, increments ResendCount, requeues or resends immediately
|-
| HandleUnacked() || Called every 100ms per client. Disconnects the client if no packet received within AckTimeout (or PausedAckTimeout if paused). Otherwise resends expired unacked packets.
|}

----

== Connection Establishment (HandleUseCircuitCode) ==

Triggered when a UseCircuitCode packet arrives from an endpoint with no existing client:

# IsClientAuthorized() -- validates session/agent/circuit code via AgentCircuitManager.AuthenticateSession()
# On success: AddClient() creates the LLUDPClient and LLClientView, starts the client
# If AgentCircuitData is missing (shouldn't happen for a legitimately authorized circuit), the agent is force-closed
# Any packets that arrived and were queued in m_pendingCache while the client was being created are reinjected into PacketReceived()
# SendRegionHandshake() is called if this is not a teleport (teleportFlags <= 0)
# On authorization failure: pending cache entry removed, warning logged, no client created

AddClient() is synchronized across the whole scene (lock (this)) to avoid race conditions -- referenced Mantis #5365 in source comments as the reason.

----

== Client Disconnection ==

DeactivateClientDueToTimeout() -- called when HandleUnacked() detects no packets received within the timeout window. Logs a warning, then calls Scene.CloseAgent() (falls back to client.Close() if that fails).

LogoutHandler() -- sends the logout packet, then calls Scene.CloseAgent() if not already logging out.

----

== Zero-Encoding ==

ZeroEncode() implements run-length encoding of zero bytes in outgoing packets (a bandwidth optimization from the original SL protocol) -- runs of zero bytes become a 0x00 marker followed by a count byte. Applied conditionally: if the zero-encoded result is not smaller than the original, the MSG_ZEROCODED flag is stripped and the packet is sent unencoded instead.

----

== Notable Details ==

* Malformed packet warnings fire every 10000 occurrences with the message "probable network attack" -- this is the closest thing to built-in DoS detection at this layer
* The AddClient() lock is scene-wide, not per-agent -- a documented but still-present performance/correctness tradeoff (source comment references wanting to move to per-circuit locking eventually)
* BinaryStats packet logging exists (LogPacketHeader) but is disabled by default -- controlled by [Statistics.Binary] Enabled
* The outgoing loop's sleep constant (15ms) has a comment explicitly referencing Windows' ~16ms scheduler granularity to avoid the OS rounding up to 32ms

----

== See Also ==

* [[OpenSimulator Internals/Code Map]]
* [[OpenSimulator Internals/Code Map/Scene]]
* [[OpenSimulator Internals/Code Map/Shared]]
* [[OpenSimulator Internals/Walkthroughs/Bring Up Region]]

OpenSimulator Internals/Code Map/LLUDP - Revision history

Jwbshaw: First