iR Engine on MicroK8s (Linux)

this guide has been tested on ubuntu, and it is intended for local deployment only install python 3 if python 3 ( pip3 version ) is not already installed on your device, you can do so by running following commands from your ubuntu terminal sudo apt get update y sudo apt get install y python3 pip you can verify that python3 is installed correctly with the command python3 version install make if make ( make version ) is not already installed on your device, you can do so by running following commands from your ubuntu terminal sudo apt get update y sudo apt get install y build essential you can verify that make is installed correctly with the command make version install kubectl, helm and docker install kubectl https //kubernetes io/docs/tasks/tools/ , helm https //helm sh/docs/intro/install/ and docker https //docs docker com/get docker/ on your device if they aren't already installed you may also need to install docker compose https //docs docker com/compose/install/ download and install microk8s sudo snap install microk8s classic channel=1 26/stable note it is recommended to use microk8s version >=1 26 due to an issue with host storage access in version 1 25 another alternative is to follow the instructions for how to install kubernetes with microk8s https //ubuntu com/tutorials/install a local kubernetes with microk8s#1 overview in a local environment this will help you learn more about how microk8s deployment works download ir engine to build the ir engine docker image locally, and to have a pre tested way to run various local services, you'll need to download the ir engine repository to your device the easiest way to do this is by running the following command in your ubuntu terminal git clone https //github com/ir engine/ir engine git you can create an env local file by duplicating env local default if it does not already exist in the root folder of ir engine's repository start minio & mariadb we recommend running minio & mariadb server on your local machine via docker and outside of microk8s running the command docker compose up from the top level /scripts directory of the ir engine repository will also start minio and multiple mariadb docker containers (as well as an optional redis server) port 3306 server for local development port 3305 server for automated testing port 3304 server for minikube/microk8s testing you can start the docker container by running npm run dev docker the next time you need to start it alternatively, you could run minio & mariadb on their own without docker you'll have to configure the helm config file to have the appropriate s3 & sql server configuration, and possibly also change the /scripts/build microk8s sh script enable microk8s addons the following command will enable the required microk8s addons sudo microk8s enable dashboard dns registry host access ingress rbac hostpath storage helm3 add microk8s to kubectl first make sure there is no existing configuration for microk8s in your kubectl config to do so, run the following command in your ubuntu terminal and check if the output contains microk8s kubectl config get contexts you can remove any existing configurations using the following commands kubectl config delete context microk8s kubectl config delete cluster microk8s cluster kubectl config delete user microk8s admin next we will add microk8s configuration to kubectl config by running the following commands (from reference https //discuss kubernetes io/t/use kubectl with microk8s/5313/6 ) kubectl config set cluster microk8s server=https //127 0 0 1 16443/ certificate authority=/var/snap/microk8s/current/certs/ca crt kubectl config set credentials microk8s admin token="$(sudo microk8s kubectl config view raw o 'jsonpath={ users\[0] user token}')" kubectl config set context microk8s cluster=microk8s namespace=default user=microk8s admin afterwards you can use the newly created context by executing kubectl config use context microk8s check that microk8s is the current context with kubectl config get contexts add microk8s to lens (optional) you should now see the microk8s cluster in the k8s lens https //k8slens dev/ gui tool troubleshooting if you don't see it, it means that the previous step was not performed successfully you can print the current configuration using the command microk8s config option 1 if you have kubectl already installed, use sudo gedit / kube/config and add the output of microk8s config into the file option 2 in lens https //k8slens dev/ , goto file > add cluster and paste the output of microk8s config enable microk8s access for local docker for microk8s we will use microk8s local registry https //microk8s io/docs/registry built in add the following lines to /etc/docker/daemon json on linux, this is can be done by running sudo gedit /etc/docker/daemon json { "insecure registries" \["localhost 32000"] } afterwards, restart docker with sudo systemctl restart docker troubleshooting microk8s its recommended to fix all microk8s warnings for it to work properly you can check if there are any warnings by running sudo microk8s inspect these are some possible warnings and their potential fixes warning this machine's hostname contains capital letters and/or underscores this is not a valid name for a kubernetes node, causing node registration to fail please change the machine's hostname or refer to the documentation for more details possible fix https //askubuntu com/a/87687/1558816 https //askubuntu com/a/87687/1558816 warning the memory cgroup is not enabled the cluster may not be functioning properly please ensure cgroups are enabled possible fix https //github com/canonical/microk8s/issues/1691#issuecomment 1265788228 https //github com/canonical/microk8s/issues/1691#issuecomment 1265788228 warning iptables forward policy is drop consider enabling traffic forwarding with sudo iptables p forward accept the change can be made persistent with sudo apt get install iptables persistent microk8s is not running use microk8s inspect for a deeper inspection possible fix https //lightrun com/answers/canonical microk8s microk8s is not running microk8sinspect showing no error https //lightrun com/answers/canonical microk8s microk8s is not running microk8sinspect showing no error here this error cloud be due to conflicting kubectl being installed use this command to remove kubectl sudo rm rf /usr/local/bin/kubectl update the hosts file you'll need to edit your hosts file to redirect certain domains to your host machine ip address add or update the following line into your /etc/hosts file 127 0 0 1 local etherealengine org api local theinfinitereality io instanceserver local theinfinitereality io 00000 instanceserver local theinfinitereality io 00001 instanceserver local theinfinitereality io 00002 instanceserver local theinfinitereality io 00003 instanceserver local theinfinitereality io on linux this can be done by running sudo gedit /etc/hosts make sure to save the file after editing you will need administrator permissions to do so this will redirect several local etherealengine org domains internally to the host machine nginx ingress server will now be able to redirect the traffic to the appropriate ir engine pod add helm repositories you'll need to add a few helm chart repositories you can do so by running the following commands helm repo add agones https //agones dev/chart/stable helm repo add redis https //charts bitnami com/bitnami helm repo add etherealengine https //helm etherealengine org this will add the helm charts for agones, redis and ir engine, respectively install agones and redis deployments from here on, deployments will be installed using helm repositories make sure that kubectl is pointed at microk8s by running this command kubectl config current context it should say microk8s you can also run this command to get all contexts that kubectl has been configured to run kubectl config get contexts the current context will have a under the left most current column once kubectl is pointed to microk8s, run these commands from the top of the ir engine repo to install agones and redis helm install f \</path/to/agones default values yaml> agones agones/agones helm install local redis redis/redis important make sure to change /path/to/agones default values yaml with the actual path of the file the agones default values yaml https //github com/ir engine/ir engine ops/blob/master/configs/agones default values yaml file can be found in the ir engine ops https //github com/ir engine/ir engine ops/ repository after a minute or so, all of these pods should be in the running state you can run this command to list all of the pods running in microk8s kubectl get pods a install elastic search and kibana (optional) install elasticsearch by adding the elastic repository with helm helm repo add elastic https //helm elastic co download the values yaml file containing elasticsearch's configuration information using curl curl o https //raw\ githubusercontent com/elastic/helm charts/master/elasticsearch/examples/minikube/values yaml install the elasticsearch helm chart with helm install and pass the values yaml file that we just downloaded to the command helm install elasticsearch elastic/elasticsearch f /values yaml the f option allows specifying the yaml file with the template you can add the n option, followed by a name, to install elasticsearch in a specific namespace helm install elasticsearch elastic/elasticsearch n \[namespace] f /values yaml check if the cluster members are running kubectl get pods namespace=default l app=elasticsearch master w another option is to use the helm test command to examine the cluster’s health helm test elasticsearch install kibana on top of elasticsearch helm install kibana elastic/kibana check if all of the pods are ready kubectl get pods after port forwarding has been setup, elasticsearch and the kibana gui can be accessed by navigating to https //localhost 5601 https //localhost 5601 in your browser edit the file local microk8s template values yaml and update the env variable api extraenv elastic host to connect the logger with elasticsearch https //\<username> \<password>@\<host> \<port> the file local microk8s template values yaml https //github com/ir engine/ir engine ops/blob/master/configs/local microk8s template values yaml can be found in the ir engine ops https //github com/ir engine/ir engine ops/ repo build microk8s run the following command from the root of the ir engine repository after microk8s is running /scripts/build microk8s sh note if you get the error "packages/projects/projects/" does not exist , run the following commands in your terminal export mysql host=localhost npm run dev docker npm run dev reinit npx ts node swc scripts/install projects js this build microk8s script will build an image of the entire ir engine repository into a single docker file vite builds the client files using some information from the mariadb database created for microk8s deployments to fill in some variables, and it needs database credentials the script will supply default values for all of the mysql variables if they are not provided to the script, as well as vite client host , vite server host , and vite instanceserver host the latter three will make your microk8s deployment accessible on (local/api local/instanceserver local) etherealengine org ; if you want to run it on a different domain, then you'll have to set those three environment variables to what you want them to be (and also change the hosts file records you made to redirect to those subdomains this process may take up to 15 minutes when run for the first time consecutive builds will take less time the process has caching support when built for targeting multiple services, the resulting docker file will only deploy and run the parts needed for each individual service once the images are built, the script will push the result to the microk8s local registry you can verify that the images are pushed correctly by visiting https //localhost 32000/v2/ catalog https //localhost 32000/v2/ catalog update helm values file this will use a helm config file titled local values yaml to configure the deployment there is a template https //github com/ir engine/ir engine ops/blob/master/configs/local microk8s template values yaml for this file in ir engine ops https //github com/ir engine/ir engine ops/ repo if you are using a local file server, as explained in one of the previous steps, you will need to update the variable api fileserver hostuploadfolder in the local values yaml file with a value similar to engine full path/packages/server/upload e g /home/username/ir engine/packages/server/upload it is mandatory that it points to the /packages/server/upload folder of your ir engine folder deploy ir engine helm chart run the following command helm install f \</path/to/local values yaml> f \</path/to/db refresh true values yaml> local ir engine/ir engine important make sure to change /path/to/local values yaml and /path/to/db refresh true values yaml with the actual path of the files the file db refresh true values yaml https //github com/ir engine/ir engine ops/blob/master/configs/db refresh true values yaml can be found in the ir engine ops https //github com/ir engine/ir engine ops/ repository after a minute or so, running kubectl get pods should show one or more instanceservers, one or more api servers, and one client server in the running state setting the option force db refresh=true made the api servers (re)initialize the database since you don't want that to happen every time a new api pod starts, run the following command to restart the api pods and configure them to not reinit the database on boot helm upgrade reuse values f \</path/to/db refresh false values yaml> local ir engine/ir engine important make sure to change /path/to/db refresh true values yaml with the actual path of the file the file db refresh false values yaml https //github com/ir engine/ir engine ops/blob/master/configs/db refresh false values yaml can be found in the ir engine ops https //github com/ir engine/ir engine ops/ repository accept invalid certs when loading the engine's website for the first time you'll have to tell your browser to ignore insecure connections open the developer tools of your browser by clicking the side menu with three dots, then go to more tools > developer tools (or use either ctrl+shift+i or f12 ) and then go to the console tab you will see some errors in url addresses starting with wss replace wss with https and open that url in a new tab accept the certificate reload your ir engine's tab you will see some errors in url addresses starting with https //localhost 9000 open the url linked in one of those errors accept the certificate reload your ir engine's tab you need to do this for the following domains wss\ //api local theinfinitereality io > https //api local theinfinitereality io https //api local theinfinitereality io wss\ //instanceserver local theinfinitereality io > https //instanceserver local theinfinitereality io https //instanceserver local theinfinitereality iohttps //localhost 9000 https //localhost 9000 if the engine's website keeps displaying loading routes progress for a long time, it means that you have to allow the engine's certificates web browsers will throw warnings when navigating to pages with unknown certificates (aka insecure pages) you should be able to tell the browser to ignore these warnings by opening your browser's advanced options , but during development it is easier to just ignore the browser's warnings and accept the default certificates note you will be able to create signed certificates to replace the default ones when you deploy your own ir engine stack