Editing OpenSimulator Internals/Asset Connector

= OpenSimulator Internals/Asset Connector =

== Overview ==

The asset connector chain connects simulator code to the asset service. It handles local, grid, and HyperGrid asset retrieval transparently through a single interface.

In OpenSimulator 0.9.2, LocalAssetServiceConnector, RemoteAssetServiceConnector, and HGAssetBroker were consolidated into a single module: RegionAssetConnector.

See [[OpenSimulator Internals/Connector Architecture]] for the general connector pattern.

----

== Source Files ==

{| class="wikitable"
! File !! Location
|-
| RegionAssetConnectorModule.cs || OpenSim/Region/CoreModules/ServiceConnectorsOut/Asset/
|-
| LocalAssetServiceConnector.cs || OpenSim/Region/CoreModules/ServiceConnectorsOut/Asset/ (tests only)
|-
| AssetServicesConnector.cs || OpenSim/Services/Connectors/Asset/
|-
| HGAssetServiceConnector.cs || OpenSim/Services/Connectors/Asset/
|-
| AssetService.cs || OpenSim/Services/AssetService/
|-
| AssetServerConnector.cs || OpenSim/Server/Handlers/Asset/
|-
| AssetServerGetHandler.cs || OpenSim/Server/Handlers/Asset/
|-
| AssetServerPostHandler.cs || OpenSim/Server/Handlers/Asset/
|-
| AssetServerDeleteHandler.cs || OpenSim/Server/Handlers/Asset/
|-
| AssetsExistHandler.cs || OpenSim/Server/Handlers/Asset/
|}

----

== Configuration ==

=== Simulator side (OpenSim.ini) ===

RegionAssetConnector is selected via [Modules]:

 [Modules]
 AssetServices = "RegionAssetConnector"
 
 [AssetService]
 LocalGridAssetService = "OpenSim.Services.AssetService.dll:AssetService"
 HypergridAssetService = "OpenSim.Services.AssetService.dll:HGAssetService"

LocalGridAssetService is required -- connector throws on startup if missing.

HypergridAssetService is optional -- HyperGrid asset transfer is disabled if not configured.

HyperGrid import/export permissions are configured in [HGAssetService].

=== Server side (Robust.ini) ===

 [AssetService]
 LocalServiceModule = "OpenSim.Services.AssetService.dll:AssetService"
 AllowRemoteDelete = false
 AllowRemoteDeleteAllTypes = false
 ; RedirectURL = http://other.asset.server/

AllowRemoteDelete defaults false. If true, only MapTile assets may be deleted remotely unless AllowRemoteDeleteAllTypes is also true.

RedirectURL is optional. If set, GET requests that return 404 are redirected to this URL.

----

== Initialisation Sequence ==

=== Simulator side ===

# Reads AssetServices from [Modules] -- exits if name does not match
# Loads LocalGridAssetService plugin -- throws if missing or fails to load
# Loads HypergridAssetService plugin -- optional, skipped if not configured
# Loads AssetPermissions from [HGAssetService] if HyperGrid enabled
# Creates two worker queues:
#* m_localRequestsQueue ("GetAssetsWorkers", 2 threads, 2000 capacity)
#* m_remoteRequestsQueue ("GetRemoteAssetsWorkers", 2 threads, 2000 capacity)
# Cache (m_Cache) is acquired in RegionLoaded(), not Initialise(). Must implement ISharedRegionModule or is rejected.

=== Server side ===

AssetServiceConnector (OpenSim/Server/Handlers/Asset/AssetServerConnector.cs) runs inside ROBUST. On startup:

# Reads LocalServiceModule from config -- throws if missing
# Loads IAssetService plugin -- throws if load fails
# Reads AllowRemoteDelete and AllowRemoteDeleteAllTypes -- sets AllowedRemoteDeleteTypes enum
# Reads RedirectURL -- passed to GetHandler
# Creates IServiceAuth from config
# Registers four HTTP handlers:
#* AssetServerGetHandler -- GET /assets
#* AssetServerPostHandler -- POST /assets
#* AssetServerDeleteHandler -- DELETE /assets
#* AssetsExistHandler -- POST /get_assets_exist (no auth)
# Registers three console commands: show asset, delete asset, dump asset

----

== HyperGrid ID Detection ==

 private bool IsHG(string id)
 {
     return id.Length > 0 && (id[0] == 'h' || id[0] == 'H');
 }

HyperGrid asset IDs are HTTP URIs. IsHG() checks only the first character. All routing decisions flow from this check.

----

== Simulator-Side Call Paths ==

=== Synchronous Get -- local asset ===

# Scene.AssetService.Get(id)
# IsHG(id) = false
# Check m_Cache.Get() -- return asset if hit, return null if negatively cached
# GetFromLocal(id) -- calls m_localConnector.Get(id) directly (inlined)
# On miss: m_Cache.CacheNegative(id)
# On hit: m_Cache.Cache(asset), return asset

=== Synchronous Get -- HyperGrid asset ===

# Scene.AssetService.Get(id)
# IsHG(id) = true
# GetForeign(id):
#* Util.ParseForeignAssetID(id) -- extracts URI and UUID string
#* Check m_Cache.GetCached(uuidstr)
#* GetFromLocal(uuidstr) -- check local store first
#* If not found locally: GetFromForeign(uuidstr, uri) via m_HGConnector
# Check m_AssetPerms.AllowedImport() -- return null if not permitted
# Store(asset) -- persist to local grid if permitted
# Return asset

=== Synchronous Get -- with known foreign service URI ===

# Get(id, ForeignAssetService, StoreOnLocalGrid)
# Check m_Cache.GetCached(id)
# GetFromLocal(id)
# On local miss: GetFromForeign(id, ForeignAssetService)
#* Check AssetPerms.AllowedImport() -- CacheNegative and return null if denied
#* StoreLocal(asset) if StoreOnLocalGrid=true
#* Cache(asset)
# On foreign miss: CacheNegative(id)

=== Asynchronous Get -- local ===

# Get(id, sender, callBack)
# Check m_Cache.GetFromMemory() -- invoke callback immediately if found or negatively cached
# UUID.Zero check -- invoke callback with null and return false
# Lock m_AssetHandlers:
#* If request already in-flight for this id: append callback to handler list, return
#* Otherwise: create handler list, add to m_AssetHandlers, enqueue id to m_localRequestsQueue
# AssetRequestProcessor dequeues, calls Get(id), notifies all handlers via FireAndForget

=== Asynchronous Get -- HyperGrid ===

# Get(id, ForeignAssetService, StoreOnLocalGrid, callBack)
# Same cache and deduplication logic as async local
# If ForeignAssetService is empty: enqueue to m_localRequestsQueue
# Otherwise: enqueue ForeignAssetServiceGetData struct to m_remoteRequestsQueue
# AssetRequestProcessor calls Get(id, ForeignAssetService, StoreOnLocalGrid)

----

== Request Deduplication ==

m_AssetHandlers (Dictionary<string, List<SimpleAssetRetrieved>>) prevents duplicate in-flight requests for the same asset ID. If a second async request arrives for an asset already being fetched, its callback is appended to the existing handler list. When the fetch completes, all waiting callbacks are notified via FireAndForget.

----

== Simulator-Side Store Paths ==

{| class="wikitable"
! Condition !! Behaviour
|-
| HyperGrid asset, Local or Temporary flag set || Return null -- not stored
|-
| HyperGrid asset, export not permitted by AssetPermissions || Return empty string
|-
| HyperGrid asset, permitted || StoreForeign() via m_HGConnector, cache on success
|-
| Local asset, Local or Temporary flag set || Cache only, not persisted to database
|-
| Local asset, normal || Cache and StoreLocal() via m_localConnector
|}

UpdateContent() and Delete() always return false for HyperGrid assets.

AssetsExist() routes entirely to m_localConnector if no HG IDs present, entirely to m_HGConnector if any HG IDs present.

----

== Server-Side HTTP Endpoints ==

=== GET /assets/{id} ===

Returns full asset (metadata + data) serialised as XML. Returns 404 if not found.

If RedirectURL is configured and asset is not found, issues HTTP redirect to RedirectURL/assets/{id}.

Response content-type: text/xml

=== GET /assets/{id}/data ===

Returns raw asset data as application/octet-stream. Returns 404 if not found.

=== GET /assets/{id}/metadata ===

Returns AssetMetadata serialised as XML. Content-type set from SLUtil.SLAssetTypeToContentType(). Returns 404 if not found.

=== POST /assets ===

Body: AssetBase serialised as XML.

If no ID in path: calls Store(), returns new UUID as string.

If ID in path: calls UpdateContent(), returns bool.

=== DELETE /assets/{id} ===

Behaviour depends on AllowedRemoteDeleteTypes:

{| class="wikitable"
! Setting !! Behaviour
|-
| None (default) || Always returns false, no delete attempted
|-
| MapTile || Deletes only if asset has AssetFlags.Maptile set
|-
| All || Deletes any asset type
|}

Returns bool serialised as XML.

=== POST /get_assets_exist ===

Body: string[] of UUIDs serialised as XML.

Returns bool[] indicating which UUIDs exist in the asset store.

No authentication wrapper. (AssetServiceConnector passes no auth to AssetsExistHandler.)

----

== Console Commands ==

Registered by AssetServiceConnector on the ROBUST console:

{| class="wikitable"
! Command !! Usage !! Notes
|-
| show asset || show asset <ID> || Displays name, description, type, content-type, size, flags, first 80 bytes as hex
|-
| delete asset || delete asset <ID> || Deletes from store directly via IAssetService.Delete()
|-
| dump asset || dump asset <ID> || Writes raw asset data to a file named <ID> in the working directory
|}

----

== Shutdown ==

Close() disposes both worker queues and sets them to null.

----

== See Also ==

* [[OpenSimulator Internals/Connector Architecture]]
* [[OpenSimulator Internals/AssetService]]
* [[OpenSimulator Internals/Data Dictionaries#assets|assets table]]