Instance provisioning and networking

overview the instance provisioning and networking system enables multiple users to connect to shared virtual environments within the ir engine it handles the discovery, creation, and connection to server instances that host these shared spaces, establishing real time communication channels between users by leveraging webrtc technology and specialized server architectures, the system creates the foundation for multi user experiences where users can see and interact with each other in the same virtual space this chapter explores the implementation, workflow, and components involved in connecting users to shared instances within the ir engine client core concepts server instances a server instance represents a running server process that hosts a shared virtual environment definition a specific, running copy of a server that hosts a shared 3d world or media session isolation each instance is separate, with its own set of connected users and state types instances can be world instances (3d environments) or media instances (voice/video chat) identification each instance has a unique id and is associated with a specific location or channel instances serve as the "meeting places" where users gather to share experiences in the ir engine instance provisioning provisioning is the process of obtaining access to a server instance purpose locate or create a server instance for a specific location or channel process request instance details from a central provisioning service result obtain connection information (ip address, port, room code) types can provision existing public instances, create new private instances, or join specific instances by id this process ensures users can find and access the appropriate server for their desired virtual location real time networking networking establishes communication channels between the client and server instance technologies uses webrtc for real time communication, often with mediasoup as a media server connections establishes data channels for entity synchronization and media streams for audio/video topologies supports both client server (mediasoup) and peer to peer (p2p) architectures signaling uses websocket connections to exchange connection establishment information these connections enable the real time synchronization necessary for shared virtual experiences implementation instance provisioning service the locationinstanceconnectionservice handles the provisioning of world instances // simplified from src/common/services/locationinstanceconnectionservice ts import { api } from '@ir engine/common'; import { instanceprovisionpath, locationid, instanceid } from '@ir engine/common/src/schema type module'; import { getmutablestate, getstate } from '@ir engine/hyperflux'; import { authstate } from ' / /user/services/authservice'; import { locationinstancestate } from ' /locationinstanceconnectionservice'; export const locationinstanceconnectionservice = { // define the state for storing instance information state locationinstancestate, // provision a server for a specific location provisionserver async (locationid? locationid, instanceid? instanceid, sceneid? string) => { try { // get authentication token const token = getstate(authstate) authuser accesstoken; // request instance provisioning from the server const provisionresult = await api instance service(instanceprovisionpath) find({ query { locationid, instanceid, sceneid, token } }); // handle server based instance if (provisionresult ipaddress && provisionresult port) { // store instance details in state getmutablestate(locationinstancestate) instances merge({ \[provisionresult id] { ipaddress provisionresult ipaddress, port provisionresult port, locationid locationid || '', sceneid sceneid || '', roomcode provisionresult roomcode || '', instanceid provisionresult id, p2p false } }); console log(`server provisioned ${provisionresult ipaddress} ${provisionresult port}`); return provisionresult; } // handle peer to peer instance else if (provisionresult p2p) { // store p2p instance details getmutablestate(locationinstancestate) instances merge({ \[provisionresult id] { instanceid provisionresult id, locationid locationid || '', sceneid sceneid || '', roomcode provisionresult roomcode || '', p2p true } }); console log(`p2p instance provisioned ${provisionresult id}`); return provisionresult; } else { console error('failed to provision server invalid response'); throw new error('invalid provisioning response'); } } catch (error) { console error('failed to provision server ', error); throw error; } }, // remove an instance from state removeinstance (instanceid instanceid) => { const instances = getmutablestate(locationinstancestate) instances; if (instances\[instanceid]) { delete instances\[instanceid]; console log(`removed instance ${instanceid} from state`); } } }; this service sends a request to the server's instance provisioning endpoint receives connection details for an appropriate instance stores these details in the locationinstancestate provides methods to manage instance state instance state management instance information is stored using hyperflux state // simplified from src/common/services/locationinstanceconnectionservice ts import { definestate } from '@ir engine/hyperflux'; // define the structure of instance information export type locationinstancetype = { ipaddress? string; port? number; locationid locationid; sceneid string; roomcode string; instanceid instanceid; p2p boolean; }; // define the state for storing instance information export const locationinstancestate = definestate({ name 'locationinstancestate', initial { instances {} as record\<instanceid, locationinstancetype> } }); this state stores information about all provisioned instances associates each instance with its unique id contains connection details like ip address and port indicates whether the instance is server based or p2p is observed by components that need to establish connections network instance provisioning the networkinstanceprovisioning component observes instance state and establishes connections // simplified from src/networking/networkinstanceprovisioning tsx import react, { useeffect } from 'react'; import { usehookstate } from '@hookstate/core'; import { getmutablestate, getstate } from '@ir engine/hyperflux'; import { locationinstancestate } from ' /common/services/locationinstanceconnectionservice'; import { connecttoinstance } from ' /transports/mediasoup/mediasoupclientfunctions'; import { connecttop2pinstance } from ' /transports/p2p/peertopeernetworkstate'; // component for managing a single world instance connection function worldinstance({ instanceid }) { useeffect(() => { // get instance details from state const worldinstance = getstate(locationinstancestate) instances\[instanceid]; // skip if instance details are not available if (!worldinstance) return; // handle p2p instance if (worldinstance p2p) { console log(`connecting to p2p instance ${instanceid}`); // connect to p2p instance const cleanup = connecttop2pinstance({ instanceid instanceid, locationid worldinstance locationid, roomcode worldinstance roomcode }); // return cleanup function return cleanup; } // handle server based instance else if (worldinstance ipaddress && worldinstance port) { console log(`connecting to server instance ${worldinstance ipaddress} ${worldinstance port}`); // connect to mediasoup instance const cleanup = connecttoinstance( instanceid, worldinstance ipaddress, worldinstance port, worldinstance locationid, undefined, // channelid (not used for world instances) worldinstance roomcode ); // return cleanup function return cleanup; } }, \[instanceid]); // this component doesn't render anything visible return null; } // component that manages all world instance connections export function worldinstanceprovisioning() { // get all instances from state const locationinstances = usehookstate(getmutablestate(locationinstancestate) instances); return ( <> {/ create a worldinstance component for each instance /} {locationinstances keys map((id) => ( \<worldinstance key={id} instanceid={id} /> ))} \</> ); } this component observes the locationinstancestate for changes creates a worldinstance component for each provisioned instance establishes connections to instances based on their type (p2p or server based) provides cleanup functions to properly disconnect when unmounting mediasoup connection the connecttoinstance function establishes webrtc connections to server instances // simplified from src/transports/mediasoup/mediasoupclientfunctions ts import primus from 'primus client'; import mediasoupclient from 'mediasoup client'; import { getmutablestate, getstate, dispatchaction } from '@ir engine/hyperflux'; import { authstate } from ' / /user/services/authservice'; import { networkstate, createnetwork, addnetwork } from '@ir engine/network'; import { mediasouptransportactions } from '@ir engine/common/src/transports/mediasoup/mediasouptransportstate'; export const connecttoinstance = ( instanceid, ipaddress, port, locationid, channelid, roomcode ) => { // get authentication token const token = getstate(authstate) authuser accesstoken; // prepare query parameters const query = { peerid engine instance store peerid, instanceid instanceid, token, locationid, channelid, roomcode }; // create websocket connection for signaling const primus = new primus( `https //${ipaddress} ${port}?${new urlsearchparams(query) tostring()}` ); // handle websocket connection open primus on('incoming open', async () => { console log(`websocket connected to instance ${instanceid}`); try { // authenticate with the instance server const { routerrtpcapabilities, hostpeerid } = await authenticateprimus( primus, token, instanceid ); // initialize mediasoup device const mediasoupdevice = new mediasoupclient device(); await mediasoupdevice load({ routerrtpcapabilities }); // create network object const network = createnetwork( instanceid, hostpeerid, locationid || channelid, primus, mediasoupdevice ); // add network to state addnetwork(network); // request webrtc transports dispatchaction(mediasouptransportactions requesttransport({ network network id, direction 'send' })); dispatchaction(mediasouptransportactions requesttransport({ network network id, direction 'recv' })); console log(`successfully connected to instance ${instanceid}`); } catch (error) { console error(`failed to initialize connection to ${instanceid} `, error); } }); // handle websocket disconnection primus on('incoming end', () => { console log(`websocket disconnected from instance ${instanceid}`); }); // return cleanup function return () => { console log(`cleaning up connection to instance ${instanceid}`); primus end(); removenetwork(instanceid); }; }; this function establishes a websocket connection to the instance server for signaling authenticates with the instance server initializes a mediasoup device with the server's capabilities creates a network object to manage the connection requests webrtc transports for sending and receiving data provides a cleanup function to properly disconnect connection workflow the complete instance connection workflow follows this sequence sequencediagram participant user as user participant client as client application participant api as main api server participant instance as instance server user >>client request to join location client >>api provisionserver(locationid) api >>api find or create instance api >>client return instance details client >>client store in locationinstancestate client >>instance websocket connection instance >>client router capabilities client >>instance request webrtc transport instance >>client transport parameters client >>instance connect transport instance >>client connection established client >>client update ui to show connected user >>user sees shared environment this diagram illustrates the user requests to join a specific location the client requests instance provisioning from the api server the api server finds or creates an appropriate instance the client receives and stores the instance details the client establishes a websocket connection to the instance the client and instance exchange webrtc parameters the webrtc connection is established the user enters the shared environment p2p networking for peer to peer connections, the system uses a different approach // simplified from src/transports/p2p/peertopeernetworkstate tsx import { api } from '@ir engine/common'; import { instancesignalingpath } from '@ir engine/common/src/schema type module'; import { getmutablestate, getstate } from '@ir engine/hyperflux'; import { authstate } from ' / /user/services/authservice'; export const connecttop2pinstance = ({ instanceid, locationid, roomcode }) => { // get authentication token const token = getstate(authstate) authuser accesstoken; const peerid = engine instance store peerid; // create signaling client const signalingclient = api instance service(instancesignalingpath); // join the signaling room signalingclient join({ instanceid, peerid, token, roomcode }) then(async (result) => { console log(`joined p2p signaling room for ${instanceid}`); // get list of peers already in the room const peers = result peers || \[]; // create p2p connections to each peer for (const remotepeerid of peers) { if (remotepeerid !== peerid) { createp2pconnection(remotepeerid, signalingclient, instanceid); } } // listen for new peers joining signalingclient on('peer join', (data) => { if (data peerid !== peerid) { createp2pconnection(data peerid, signalingclient, instanceid); } }); // listen for peers leaving signalingclient on('peer leave', (data) => { if (data peerid !== peerid) { removep2pconnection(data peerid); } }); // listen for signaling messages signalingclient on('signal', (data) => { handlesignalingmessage(data, signalingclient); }); }) catch((error) => { console error(`failed to join p2p signaling room for ${instanceid} `, error); }); // return cleanup function return () => { signalingclient leave({ instanceid, peerid }); cleanupp2pconnections(); }; }; this function joins a signaling room for the p2p instance creates p2p connections to existing peers sets up listeners for new peers joining and leaving handles signaling messages for webrtc connection establishment provides a cleanup function to leave the signaling room media instance connections similar to world instances, media instances (for voice/video chat) are provisioned and connected // simplified from src/common/services/mediainstanceconnectionservice ts import { api } from '@ir engine/common'; import { mediainstanceprovisionpath } from '@ir engine/common/src/schema type module'; import { getmutablestate, getstate } from '@ir engine/hyperflux'; import { authstate } from ' / /user/services/authservice'; import { mediainstancestate } from ' /mediainstanceconnectionservice'; export const mediainstanceconnectionservice = { // provision a media server for a channel provisionserver async (channelid) => { try { // get authentication token const token = getstate(authstate) authuser accesstoken; // request media instance provisioning const provisionresult = await api instance service(mediainstanceprovisionpath) find({ query { channelid, token } }); // store instance details in state if (provisionresult ipaddress && provisionresult port) { getmutablestate(mediainstancestate) instances merge({ \[provisionresult id] { ipaddress provisionresult ipaddress, port provisionresult port, channelid, instanceid provisionresult id } }); console log(`media server provisioned ${provisionresult ipaddress} ${provisionresult port}`); return provisionresult; } else { console error('failed to provision media server invalid response'); throw new error('invalid media provisioning response'); } } catch (error) { console error('failed to provision media server ', error); throw error; } } }; this service requests media instance provisioning for a specific channel stores the instance details in mediainstancestate follows a similar pattern to the locationinstanceconnectionservice integration with other components the instance provisioning and networking system integrates with several other components authentication system instance connections require authentication // example of authentication integration import { getstate } from '@ir engine/hyperflux'; import { authstate } from ' / /user/services/authservice'; function getauthenticationtoken() { const authstate = getstate(authstate); // check if user is authenticated if (!authstate isauthenticated value) { throw new error('user is not authenticated'); } // get authentication token const token = authstate authuser accesstoken value; if (!token) { throw new error('authentication token not available'); } return token; } this integration retrieves the authentication token from authstate ensures the user is authenticated before attempting to connect includes the token in provisioning and connection requests avatar system avatars are spawned in the shared instance // example of avatar integration import { spawnlocalavatarinworld } from '@ir engine/engine/src/avatar/functions/spawnlocalavatarinworld'; import { getstate } from '@ir engine/hyperflux'; import { authstate } from ' / /user/services/authservice'; function spawnavatarininstance(instanceid, sceneid) { // get user information const user = getstate(authstate) user; const username = user name value; // get avatar url from user preferences const avatarurl = user avatarurl value; // spawn avatar in the world const avatarentity = spawnlocalavatarinworld({ parentuuid sceneid, avatarurl avatarurl, name username }); console log(`avatar spawned in instance ${instanceid}`); return avatarentity; } this integration spawns the user's avatar in the shared instance uses the user's name and avatar url from their profile creates a visual representation of the user in the shared space hyperflux state management instance and network state is managed through hyperflux // example of hyperflux integration import { definestate, getmutablestate } from '@ir engine/hyperflux'; // define network state export const networkstate = definestate({ name 'networkstate', initial { networks {} as record\<string, networktype>, connected false } }); // update network connection status function updatenetworkstatus(instanceid, connected) { const networkstate = getmutablestate(networkstate); // update specific network status if (networkstate networks\[instanceid]) { networkstate networks\[instanceid] connected set(connected); } // update overall connection status const anyconnected = object values(networkstate networks value) some(network => network connected); networkstate connected set(anyconnected); } this integration defines state structures for network information provides reactive state updates when connection status changes enables components to respond to network events benefits of instance provisioning and networking the instance provisioning and networking system provides several key advantages multi user experiences enables multiple users to share the same virtual environment dynamic instance allocation automatically finds or creates appropriate server instances real time communication establishes low latency connections for data and media sharing scalability supports both server based and peer to peer architectures seamless integration works with authentication, avatars, and other system components connection management handles the complex webrtc connection establishment process fault tolerance provides error handling and reconnection capabilities these benefits make instance provisioning and networking essential components for creating shared virtual experiences in the ir engine next steps with an understanding of how users connect to shared virtual environments, the next chapter explores how administrators can manage the system through a dedicated interface next admin panel system docid\ df4gycvmg6gi3bfblsnus