iR Engine on MicroK8s (Windows)

this guide is intended for local environment and currently tested on windows 11 install windows subsystem for linux (wsl) install ubuntu distribution of linux from microsoft store by using guide here https //learn microsoft com/en us/windows/wsl/install alternatively, you can follow these instructions as well how to install wsl https //pureinfotech com/install wsl windows 11/manual installation steps for wsl https //learn microsoft com/en us/windows/wsl/install manual once wsl is installed, make sure to set up your linux username and password https //learn microsoft com/en us/windows/wsl/setup/environment#set up your linux username and passwordupdate and upgrade packages https //learn microsoft com/en us/windows/wsl/setup/environment#update and upgrade packages set ubuntu as default wsl distribution in powershell/cmd run following command to see the list of distributions wsl l in the list you should be able to see ubuntu listed afterwards, run following command to set ubuntu as default distribution wsl s ubuntuwsl ubuntu default distribution install docker desktop install docker desktop with wsl 2 backend you can find the instructions here https //docs docker com/desktop/install/windows install/ once docker desktop is installed and running make sure to enable your wsl distribution you can do so from docker desktop app by visiting settings > resources > wsl integration make sure to hit 'apply & restart' docker desktop wsl distro enable systemd in wsl inside your ubuntu instance, add the following modification to /etc/wsl conf \[boot] systemd=true then restart your instance by running wsl shutdown in powershell and relaunching ubuntu upon launch you should have systemd running you can check this with the command systemctl list unit files type=service which should show your services status you can read more about this on ubuntu blog https //ubuntu com/blog/ubuntu wsl enable systemd & microsoft blog https //devblogs microsoft com/commandline/systemd support is now available in wsl/ enable localhostforwarding in wsl create or update wslconfig file located at c \users\\{user name}\\ wslconfig (or %userprofile%\\ wslconfig ) with following content \[wsl2] localhostforwarding=true this requires wsl shutdown and reboot shutting down your terminal is insufficient also machine boot is not required simply run wsl shutdown (in powershell) or wsl exe shutdown (within ubuntu) reference custom hostname for servers running in wsl 2 https //stackoverflow\ com/a/65707003/2077741 install node in your wsl ubuntu terminal, if node ( node version ) isn't already installed on your machine you can do so by first installing nvm by running following commands curl https //raw\ githubusercontent com/creationix/nvm/master/install sh | bash source / profile export nvm dir="$home/ nvm" \[ s "$nvm dir/nvm sh" ] && \\ "$nvm dir/nvm sh" # this loads nvm \[ s "$nvm dir/bash completion" ] && \\ "$nvm dir/bash completion" # this loads nvm bash completion you can verify nvm by using nvm version command afterwards, install node by using nvm install lts you can verify nvm by using node version command install python 3 in your wsl ubuntu terminal, if python 3 ( pip3 version ) isn't already installed on your machine you can do so by running following commands sudo apt get update y sudo apt get install y python3 pip you can verify python3 by using python3 version command install make in your wsl ubuntu terminal, if make ( make version ) isn't already installed on your machine you can do so by running following commands sudo apt get update y sudo apt get install y build essential you can verify make by using make version command install kubectl and helm in your wsl ubuntu terminal, if kubectl https //kubernetes io/docs/tasks/tools/ & helm https //helm sh/docs/intro/install/ aren't already installed on your machine, install them docker & docker compose should be installed if you successfully completed " install docker desktop /#install docker desktop " step you can verify by running docker version & docker compose version commands in wsl ubuntu terminal download and install microk8s make sure to install microk8s in your wsl ubuntu terminal instructions can be found here https //ubuntu com/tutorials/install a local kubernetes with microk8s#1 overviewsudo snap install microk8s classic channel=1 26/stable due to some ongoing issue with host storage access in microk8s 1 25 version, it is recommended to use version 1 26 while you can follow the demo instructions there about starting microk8s, deploying some demo deployments, etc to get a feel for it clone ir engine repo to your local machine to build the ir engine docker image locally, and to have a pre tested way to run various local services, you'll need to get the ir engine repo on your machine this is most easily done by running following command in wsl ubuntu terminal git clone https //github com/ir engine/ir engine git etherealengine if env local file does not exist in the root of your repo folder then create it by duplicating env local default start minio & mariadb server locally via docker for simplicity, we recommend running minio & mariadb server on your local machine outside of microk8s if you run docker compose up from the top level /scripts directory in the ir engine repo, it will start up minio & multiple mariadb docker containers (as well as a redis server, which is not needed) for mariadb containers, one is intended for local development, runs on port 3306; another, intended for automated testing purposes, runs on port 3305; and the last one, intended for minikube/microk8s testing, runs on port 3304 once the docker container is stopped, you can start it again by running npm run dev docker alternatively, if you want to just run minio & mariadb on its own without docker, that's fine too you'll just have to configure the helm config file to have the appropriate s3 & sql server configuration, and possibly change the script /scripts/build microk8s sh enabling microk8s addons execute following command in your wsl ubuntu terminal to enable microk8s addons sudo microk8s enable dashboard dns registry host access ingress rbac hostpath storage helm3 add microk8s to kubectl first make sure there is no existing configuration for microk8s in your kubectl config to do so you run kubectl config get contexts command in wsl ubuntu terminal and see if the output contains microk8s you can remove the existing configurations using following commands kubectl config delete context microk8s kubectl config delete cluster microk8s cluster kubectl config delete user microk8s admin now, we will add microk8s configuration to kubectl config we can do this by using following commands in wsl ubuntu terminal reference https //discuss kubernetes io/t/use kubectl with microk8s/5313/6kubectl config set cluster microk8s server=https //127 0 0 1 16443/ certificate authority=/var/snap/microk8s/current/certs/ca crt kubectl config set credentials microk8s admin token="$(sudo microk8s kubectl config view raw o 'jsonpath={ users\[0] user token}')" kubectl config set context microk8s cluster=microk8s namespace=default user=microk8s admin afterwards you can use this newly create context by executing kubectl config use context microk8s now if you run kubectl config get contexts command then microk8s should be current context (optional) add microk8s to lens if the previous step was performed successfully then you should be able to see microk8s cluster in gui tool lens https //k8slens dev/ else you can print the configuration using following command microk8s config in lens, goto file > add cluster and paste the output of above command to add cluster enable microk8s access for local docker for microk8s we will be using microk8s local registry https //microk8s io/docs/registry built in option 1 in windows, add the following lines to %userprofile%\\ docker\daemon json create this file if it does not already exists { "insecure registries" \["https //microk8s registry 32000", "microk8s registry 32000"] } afterwards, restart docker from powershell restart service docker option 2 edit configuration as shown in below image make sure to hit 'apply & restart' after making changes docker desktop configuration reference daemon json file in w1indows https //stackoverflow\ com/a/55352883/2077741when using buildkit, http needs to be added https //github com/docker/docs/blob/62adddbb6b1f8d861c72f6ade2c50977fd57f481/registry/insecure md#troubleshoot insecure registryrestart docker service from command line https //forums docker com/t/restart docker service from command line/27331/2 verify and troubleshoot microk8s run following command and check if there is any warning its recommended to fix the warnings for microk8s to work properly sudo microk8s inspect update system hostfile to point to microk8s you'll need to edit your hostfile to point certain domains to host machine ip address first you need to find the ip address of your wsl run wsl hostname i in powershell/cmd for example c \users\hanzl>wsl hostname i 172 31 89 133 10 1 215 0 note if you face issue while running above command, make sure that 'ubuntu' distribution is selected as default you can do so by running wsl /l to view distributions and then run wslconfig /s ubuntu to select distribution from the above output, use 172 31 89 133 as {wsl ip} note your ip would be different, this is just for example next, edit your windows hostfile, this is done by editing c \windows\system32\drivers\etc\hosts add/update the following lines {wsl ip} local etherealengine org api local theinfinitereality io instanceserver local theinfinitereality io 00000 instanceserver local theinfinitereality io 00001 instanceserver local theinfinitereality io 00002 instanceserver local theinfinitereality io 00003 instanceserver local theinfinitereality io {wsl ip} microk8s registry make sure to replace {wsl ip} with ip address from wsl hostname i command the first line says to point several local etherealengine org domains internally to the host machine, where the nginx ingress server will redirect the traffic to the appropriate pod make sure to save this file after you've edited it also, you will need to update hostfile with updated ip address after every windows/wsl reboot add helm repos you'll need to add a few helm repos run the following helm repo add agones https //agones dev/chart/stable helm repo add redis https //charts bitnami com/bitnami helm repo add etherealengine https //helm etherealengine org this will add the helm charts for agones, redis, and ir engine, respectively install agones and redis deployments after adding those helm repos, you'll start installing deployments using helm repos make sure that kubectl is pointed at microk8s by running kubectl config current context , which should say 'microk8s' you can also run kubectl config get contexts to get all contexts that kubectl has been configured to run; the current one will have a ' ' under the left most 'current' column once kubectl is pointed to microk8s, from the top of the ir engine repo, run helm install f \</path/to/agones default values yaml> agones agones/agones to install agones and helm install local redis redis/redis to install redis agones default values yaml https //github com/ir engine/ir engine ops/blob/master/configs/agones default values yaml can be found in ir engine ops https //github com/ir engine/ir engine ops/ repo you can run kubectl get pods a to list all of the pods running in microk8s after a minute or so, all of these pods should be in the running state (optional) install elastic search and kibana using helm for server logs to install elasticsearch, add the elastic repository in helm helm repo add elastic https //helm elastic co now, use the curl command to download the values yaml file containing configuration information curl o https //raw\ githubusercontent com/elastic/helm charts/master/elasticsearch/examples/minikube/values yaml use the helm install command and the values yaml file to install the elasticsearch helm chart helm install elasticsearch elastic/elasticsearch f /values yaml the f option allows specifying the yaml file with the template if you wish to install elasticsearch in a specific namespace, add the n option followed by the name of the namespace helm install elasticsearch elastic/elasticsearch n \[namespace] f /values yaml now check if the cluster members are up kubectl get pods namespace=default l app=elasticsearch master w the other option is to use the helm test command to examine the cluster’s health helm test elasticsearch to install kibana on top of elasticsearch helm install kibana elastic/kibana check if all the pods are ready kubectl get pods after you set up port forwarding, access elasticsearch, and the kibana gui by typing https //localhost 5601 in your browser in order to connect logger with elasticsearch, update local microk8s template values yaml env api extraenv elastic host for e g https //\<username> \<password>@\<host> \<port> local microk8s template values yaml https //github com/ir engine/ir engine ops/blob/master/configs/local microk8s template values yaml can be found in ir engine ops https //github com/ir engine/ir engine ops/ repo run build microk8s sh when microk8s is running, run the following command from the root of the ir engine repo in wsl ubuntu terminal export registry host=microk8s registry; export mysql host=kubernetes docker internal;bash /scripts/build microk8s sh if you face issue related to "packages/projects/projects/" does not exist then run following commands in your wsl terminal export mysql host=localhost npm run dev docker npm run dev reinit npx ts node swc scripts/install projects js the script builds the full repo docker image using several build arguments vite, which builds he client files, uses some information from the mariadb database created for microk8s deployments to fill in some variables, and needs database credentials the script will supply default values for all of the mysql variables if they are not provided to the script, as well as vite client host, vite server host, and vite instanceserver host the latter three will make your microk8s deployment accessible on (local/api local/instanceserver local) etherealengine org ; if you want to run it on a different domain, then you'll have to set those three environment variables to what you want them to be (and also change the hostfile records you made pointing those subdomains) this will build an image of the entire ir engine repo into a single docker file when deployed for different services, it will only run the parts needed for that service this may take up to 15 minutes, though later builds should take less time as things are cached once the images are build it will push it to microk8s local registry you can verify that images are pushed to registry by visiting https //microk8s registry 32000/v2/ catalog https //microk8s registry 32000/v2/ catalog update helm values file this will use a helm config file titled 'local values yaml' to configure the deployment there is a template https //github com/ir engine/ir engine ops/blob/master/configs/local microk8s template values yaml for this file in ir engine ops https //github com/ir engine/ir engine ops/ repo if you are using local file server as explained couple of steps earlier then, update 'local values yaml' variable api fileserver hostuploadfolder with value similar to '\<engine full path>/packages/server/upload' e g '/home/\<os user name>/\<engine folder>/packages/server/upload' its mandatory to point to /packages/server/upload folder of your engine folder deploy ir engine helm chart before this step, ensure that all the agones and redis pods are in "running" state you can check pods status using the below command kubectl get pods run the following command helm install f \</path/to/local values yaml> f \</path/to/db refresh true values yaml> local ir engine/ir enginedb refresh true values yaml https //github com/ir engine/ir engine ops/blob/master/configs/db refresh true values yaml can be found in ir engine ops https //github com/ir engine/ir engine ops/ repo after a minute or so, running kubectl get pods should show one or more instanceservers, one or more api servers, and one client server in the running state setting force db refresh=true made the api servers (re)initialize the database since you don't want that to happen every time a new api pod starts, run helm upgrade reuse values f \</path/to/db refresh false values yaml> local ir engine/ir engine the api pods will restart and will now not attempt to reinit the database on boot db refresh false values yaml https //github com/ir engine/ir engine ops/blob/master/configs/db refresh false values yaml can be found in ir engine ops https //github com/ir engine/ir engine ops/ repo accept invalid certs since there are no valid certificates for this domain, you'll have to tell your browser to ignore the insecure connections when you try to load the application go to < https //local etherealengine org/> https //local etherealengine org/> , you should see a warning about an invalid certificate; accept this invalid cert to get to the home page next if it keeps displaying 'loading routes' progress for a long time, it is due to the fact that you have to allow certificates you'll have to open the dev tools for your browser and go to the 'console' tab you will see some errors in url address starting with 'wss' replace 'wss' with 'https' and open it in new tab accept the certificate and reload your ir engine tab you need to do this for following domains wss\ //api local theinfinitereality io > https //api local theinfinitereality io https //api local theinfinitereality io wss\ //instanceserver local theinfinitereality io > https //instanceserver local theinfinitereality io https //instanceserver local theinfinitereality iohttps //localhost 9000 https //localhost 9000 you can open developer tools in chrome by clicking the side menu with three dots, then more tools > developer tools (or use ctrl+shift+i)