Editing OpenSimulator Internals/Code Map/LLUDPServer

= Code Map: LLUDPServer =

'''Source:''' <code>OpenSim/Region/ClientStack/Linden/UDP/LLUDPServer.cs</code><br>
'''Namespace:''' <code>OpenSim.Region.ClientStack.LindenUDP</code><br>
'''Key classes:''' <code>LLUDPServer</code>, <code>LLUDPServerShim</code>

----

== Overview ==

<code>LLUDPServer</code> is the UDP transport layer for a single region. It handles
all inbound and outbound UDP traffic between the region and its connected viewers
using the Linden Lab UDP (LLUDP) protocol.

One instance exists per region. It is not shared across regions even when multiple
regions run in the same simulator process.

The region module entry point is <code>LLUDPServerShim</code>, a thin
<code>INonSharedRegionModule</code> wrapper that instantiates and owns
<code>LLUDPServer</code>. The shim handles the module lifecycle; the server
handles the actual network I/O.

----

== Threading Model ==

LLUDPServer runs three persistent threads per region:

{| class="wikitable"
|-
! Thread !! Started by !! Purpose
|-
| Inbound receive thread || <code>StartInbound()</code> via base class <code>OpenSimUDPBase</code> || Receives raw UDP datagrams from the OS socket and places them on <code>packetInbox</code>.
|-
| <code>IncomingPacketHandler</code> || <code>StartInbound()</code> || Drains <code>packetInbox</code>, dispatches packets to the correct <code>LLClientView</code> via <code>ProcessInPacket()</code>.
|-
| <code>OutgoingPacketHandler</code> || <code>StartOutbound()</code> || Drives per-client outbound queues, resends, ACKs, and pings on a timer loop.
|}

A fourth engine, <code>OqrEngine</code> (Outgoing Queue Refill Engine), runs as a
<code>JobEngine</code> thread pool to refill per-client outbound queues without
blocking the outgoing packet handler.

=== Inbound path ===

 OS socket
   → base class receive thread (OpenSimUDPBase)
     → PacketReceived()
       → decode + zerodecode
       → pending cache check (UseCircuitCode handling)
       → client lookup by endpoint
       → ACK processing (appended ACKs, PacketAck packets)
       → duplicate detection (PacketArchive)
       → packetInbox.Add()
         → IncomingPacketHandler thread
           → LLClientView.ProcessInPacket()

UseCircuitCode packets are handled specially: they are dispatched via
<code>FireAndForget</code> to <code>HandleUseCircuitCode()</code>, which
authenticates the circuit, creates a new <code>LLClientView</code> and
<code>LLUDPClient</code>, and then reinjects any packets that arrived while
the client was being set up (held in a per-endpoint pending queue).

=== Outbound path ===

 Scene code / region modules
   → LLClientView.SendXxx()
     → LLUDPServer.SendPacket() / SendPacketData() / SendUDPPacket()
       → optional zerocoding
       → OutgoingPacket constructed
       → LLUDPClient.EnqueueOutgoing()
         if queue full or not throttled:
           → SendPacketFinal() → SyncSend() → OS socket
         else:
           → sits in per-client priority queue
             → OutgoingPacketHandler → DequeueOutgoing() → SendPacketFinal()

----

== Key Structures ==

=== packetInbox ===

A <code>BlockingCollection&lt;IncomingPacket&gt;</code>. The receive thread
produces; the <code>IncomingPacketHandler</code> thread consumes. Bounded only
by memory.

=== TokenBucket / ThrottleRates ===

<code>Throttle</code> is a scene-wide token bucket capping total outbound
bandwidth. <code>ThrottleRates</code> holds the per-client default rates for
each traffic category (resend, texture, asset, land, wind, cloud, task,
state, AvatarInfo). Both are configured from <code>[ClientStack.LindenUDP]</code>.

=== Pending cache ===

An <code>ExpiringCacheOS&lt;IPEndPoint, Queue&lt;UDPPacketBuffer&gt;&gt;</code>
keyed by endpoint. While a new client is being set up after a UseCircuitCode,
all arriving packets for that endpoint are held here and reinjected once the
client is ready. Entries expire after 60 seconds.

=== OqrEngine ===

<code>JobEngine</code> instance named "Outgoing Queue Refill Engine". When a
client's outbound queue runs dry, a refill job is posted here rather than
blocking the outgoing packet handler thread. This prevents one slow client
from stalling outbound processing for all others.

----

== Outgoing Packet Handler Timer Loop ==

The <code>OutgoingPacketHandler</code> thread runs a continuous loop with
three chained timers:

{| class="wikitable"
|-
! Interval !! Action
|-
| 100ms || Check for unacked packets past their RTO; resend via <code>HandleUnacked()</code>.
|-
| 500ms || Send accumulated ACKs for all clients via <code>SendAcks()</code>.
|-
| 5000ms || Send ping checks to all clients via <code>SendPing()</code>.
|}

If no packet was sent in a loop iteration and there are connected clients,
the thread sleeps 15ms. If there are no clients, it sleeps 100ms.

----

== ACK Handling ==

Inbound ACKs arrive in two forms:
* Appended to the tail of any reliable packet (<code>Header.AppendedAcks</code>).
* As explicit <code>PacketAck</code> packets.

Both are processed in <code>PacketReceived()</code> and passed to
<code>LLUDPClient.NeedAcks.Acknowledge()</code>.

Outbound ACKs are accumulated in <code>LLUDPClient.PendingAcks</code> and
sent either appended to the next reliable outbound packet, or as a standalone
<code>PacketAck</code> by the 500ms timer, whichever comes first. The
threshold for immediate ACK sending is 2 * MTU bytes received since the last ACK.

----

== Reliability and Resends ==

Reliable packets are added to <code>LLUDPClient.NeedAcks</code> after send.
The 100ms timer calls <code>HandleUnacked()</code>, which retrieves all packets
whose age exceeds the client's current RTO and invokes their
<code>UnackedMethod</code> callback (default: <code>ResendUnacked()</code>).

<code>ResendUnacked()</code> sets the resent flag on the packet and re-enqueues
it in the <code>Resend</code> throttle category.

A client is disconnected if no packet of any kind is received within the ack
timeout (default 60 seconds; 300 seconds if the client is paused).

----

== MTU and Zerocoding ==

MTU is fixed at 1400 bytes. Max payload is 1200 bytes.

Packets marked <code>MSG_ZEROCODED</code> are compressed by run-length encoding
of zero bytes before send, and decoded on receive. If zerocoding makes a packet
larger than the unencoded form, the flag is stripped and the unencoded data
is sent instead.

Packets larger than the MTU are split into multiple datagrams via
<code>Packet.ToBytesMultiple()</code> where the packet type supports variable
blocks. <code>CoarseLocationUpdate</code> is explicitly excluded from splitting.

----

== Config ==

All keys are in <code>[ClientStack.LindenUDP]</code> in <code>OpenSim.ini</code>:

{| class="wikitable"
|-
! Key !! Default !! Purpose
|-
| <code>client_socket_rcvbuf_size</code> || 0 (OS default) || UDP receive buffer size passed to OS.
|-
| <code>scene_throttle_max_bps</code> || 6250000 || Scene-wide outbound bandwidth cap in bits/sec.
|-
| <code>TextureSendLimit</code> || 20 || Texture packets queued per OQR event.
|-
| <code>DefaultRTO</code> || 0 (auto) || Default retransmission timeout in ms.
|-
| <code>MaxRTO</code> || 0 (auto) || Maximum retransmission timeout in ms.
|-
| <code>AckTimeout</code> || 60 || Seconds before disconnecting a silent client.
|-
| <code>PausedAckTimeout</code> || 300 || Seconds before disconnecting a paused client.
|-
| <code>DisableFacelights</code> || false || Strip facelights from client updates.
|-
| <code>SupportViewerObjectsCache</code> || true || Enable viewer-side object cache support.
|}

----

== Statistics ==

LLUDPServer registers the following stats under the <code>clientstack</code>
category (visible via the <code>show stats</code> console command):

* <code>InboxPacketsCount</code> -- packets waiting in <code>packetInbox</code>.
* <code>IncomingUDPReceivesCount</code> -- total UDP datagrams received.
* <code>IncomingPacketsProcessedCount</code> -- packets dispatched to clients.
* <code>IncomingPacketsMalformedCount</code> -- packets that could not be parsed.
* <code>IncomingPacketsOrphanedCount</code> -- packets with no matching client.
* <code>IncomingPacketsResentCount</code> -- inbound packets flagged as resends by clients.
* <code>OutgoingUDPSendsCount</code> -- total UDP datagrams sent.
* <code>OutgoingPacketsResentCount</code> -- packets resent due to missing ACK.
* <code>OutgoingPacketsQueuedCount</code> -- packets queued across all clients.
* <code>OQRERequestsWaiting</code> -- jobs pending in the OqrEngine.
* <code>AverageUDPProcessTime</code> -- average ms per inbound UDP receive.
* <code>ClientLogoutsDueToNoReceives</code> -- forced disconnects due to ack timeout.

----

== See Also ==

* [[OpenSimulator_Internals/Package_Overview#OpenSim.Region.ClientStack|Package Overview: ClientStack]]
* [[OpenSimulator_Internals/Walkthroughs/Avatar_Rez_In_Region|Walkthrough: Avatar Rez In Region]]
* [[OpenSimulator_Internals/Walkthroughs/Avatar_Logs_Out|Walkthrough: Avatar Logs Out]]

[[Category:OpenSimulator Internals]]