Visual scripting system

overview the visual scripting system provides a graphical programming interface for creating interactive behaviors in the ir engine without writing traditional code it enables developers, designers, and artists to build complex logic by connecting visual nodes in a graph based editor by representing programming concepts as visual elements, the system makes game logic more accessible and easier to understand this chapter explores the concepts, structure, and implementation of the visual scripting system within the ir engine core concepts visual scripting fundamentals visual scripting is a programming paradigm that uses graphical elements instead of text based code nodes visual blocks that represent specific functions, events, data, or operations connections lines (or "wires") that connect nodes to define execution flow and data flow execution flow determines the sequence in which nodes are processed data flow defines how information passes between nodes this approach offers several advantages makes programming more accessible to non programmers provides visual clarity for complex logic flows enables rapid prototyping and iteration reduces syntax errors common in text based programming visual script component the visualscriptcomponent serves as the container for visual scripts attached to entities // simplified from src/visualscript/components/visualscriptcomponent tsx import { definecomponent, s } from '@ir engine/ecs'; export const visualscriptcomponent = definecomponent({ name 'visualscriptcomponent', schema s object({ visualscript s object({}), // the script graph data (json) run s bool({ default true }), // is the script currently running? disabled s bool({ default false }) // is the script temporarily disabled? }) }); this component stores the complete visual script graph as a json structure controls whether the script is currently active provides a way to temporarily disable the script without removing it serves as the entry point for the visual script execution system node types visual scripts are composed of different types of nodes, each serving a specific purpose event nodes event nodes serve as entry points for script execution // conceptual example of an event node definition const oninteracteventnode = { type 'oninteract', category 'events', outputs \[ { name 'exec', type 'execution' }, // execution output { name 'interactingentity', type 'entity' } // data output ], // internal implementation that connects to the interaction system onevent (entity, interactingentity) => { // trigger execution flow starting from this node return { interactingentity }; } }; event nodes respond to specific game events (e g , interaction, collision, timer) provide the starting point for execution flow often output relevant data about the event (e g , the entity that triggered it) connect to system events from other parts of the engine action nodes action nodes perform operations that affect the game world // conceptual example of an action node definition const setmaterialcolornode = { type 'setmaterialcolor', category 'rendering', inputs \[ { name 'exec', type 'execution' }, // execution input { name 'entity', type 'entity', defaultvalue 'self' }, // target entity { name 'color', type 'color' } // color value ], outputs \[ { name 'exec', type 'execution' } // execution output ], // internal implementation execute (inputs) => { const { entity, color } = inputs; // get the material component const material = getcomponent(entity, materialcomponent); if (material) { // update the color setcomponent(entity, materialcomponent, { material, color color }); } // continue execution flow return {}; } }; action nodes perform specific operations when executed take input data to configure their behavior often modify component data or trigger engine functions continue the execution flow to the next node data nodes data nodes provide values to other nodes // conceptual example of a data node definition const colorvaluenode = { type 'color', category 'values', inputs \[ { name 'r', type 'number', defaultvalue 1 0 }, { name 'g', type 'number', defaultvalue 0 0 }, { name 'b', type 'number', defaultvalue 0 0 }, { name 'a', type 'number', defaultvalue 1 0 } ], outputs \[ { name 'color', type 'color' } // data output ], // internal implementation getvalue (inputs) => { const { r, g, b, a } = inputs; return { color { r, g, b, a } }; } }; data nodes provide constant values or computed data can take inputs to calculate their output values connect to input pins of action or logic nodes do not participate in execution flow, only data flow flow control nodes flow control nodes manage the execution path // conceptual example of a flow control node definition const branchnode = { type 'branch', category 'flow', inputs \[ { name 'exec', type 'execution' }, // execution input { name 'condition', type 'boolean' } // condition value ], outputs \[ { name 'true', type 'execution' }, // execution output if condition is true { name 'false', type 'execution' } // execution output if condition is false ], // internal implementation execute (inputs) => { const { condition } = inputs; // determine which execution path to follow return { outputexec condition ? 'true' 'false' }; } }; flow control nodes control the path of execution based on conditions include branches, loops, sequences, and delays often take data inputs to determine flow decisions direct execution to different output pins based on logic visual script execution the process of executing a visual script involves several components working together visual script system the visualscriptsystem initializes and manages the visual scripting environment // simplified concept from src/visualscript/systems/visualscriptsystem ts const visualscriptsystem = definesystem({ uuid 'ir engine visualscriptsystem', execute () => { // register node profiles if not already registered if (!visualscriptstate isprofileregistered(visualscriptdomain ecs)) { visualscriptstate registerprofile(registerengineprofile, visualscriptdomain ecs); } // process global visual script actions processvisualscriptactions(); } }); this system registers available node types from different profiles processes global actions related to visual scripts initializes the visual scripting environment provides the foundation for script execution visual script runner the usevisualscriptrunner hook manages the execution of individual scripts // simplified concept from src/visualscript/systems/usevisualscriptrunner ts function usevisualscriptrunner({ visualscriptjson, autorun, registry }) { const \[engine, setengine] = usestate(null); const \[running, setrunning] = usestate(autorun); // initialize the engine when the script changes useeffect(() => { if (!visualscriptjson || !registry || !running) return; // parse the json into a node graph const nodes = readgraphfromjson({ graphjson visualscriptjson, registry }) nodes; // create a new engine instance const newengine = new visualscriptengine(nodes); setengine(newengine); // execute initial nodes newengine executeallsync(); // emit start event for event nodes registry dependencies? ilifecycleeventemitter? startevent emit(); // clean up when unmounted return () => { newengine dispose(); }; }, \[visualscriptjson, registry, running]); // play/pause functions const play = () => setrunning(true); const pause = () => setrunning(false); return { engine, playing running, play, pause }; } this hook parses the visual script json into a node graph creates and manages a visualscriptengine instance executes initial nodes and emits start events provides controls for playing and pausing the script handles cleanup when the script is no longer needed execution flow the execution of a visual script follows this general process sequencediagram participant event as game event participant vscomponent as visualscriptcomponent participant vsrunner as visual script runner participant vsengine as visual script engine participant gamesystems as game systems event >>vscomponent trigger event (e g , interaction) vscomponent >>vsrunner notify of event vsrunner >>vsengine find event node and start execution loop node execution vsengine >>vsengine process current node vsengine >>gamesystems perform actions (if action node) gamesystems >>vsengine return results vsengine >>vsengine determine next node(s) end this diagram illustrates an event in the game world triggers execution the visual script component receives the event the visual script runner finds the corresponding event node the visual script engine processes nodes in sequence action nodes interact with game systems to affect the world the execution continues until all paths are complete practical example let's create a visual script for an interactive button that changes color and plays a sound when activated // create a button entity const buttonentity = createentity(); // add visual components setcomponent(buttonentity, transformcomponent, { position { x 0, y 1, z 2 }, rotation { x 0, y 0, z 0 }, scale { x 0 2, y 0 2, z 0 05 } }); setcomponent(buttonentity, meshcomponent, { geometrytype 'box', materialprops { color 'blue' } }); // make the button interactable setcomponent(buttonentity, interactablecomponent, { label "press e to activate", activationdistance 2, clickinteract true }); // add a visual script component setcomponent(buttonentity, visualscriptcomponent, { visualscript { // this would be a json representation of the node graph // oninteract event node // setmaterialcolor action node (set to red) // playsound action node // with appropriate connections between them nodes \[ { id "node1", type "oninteract", position { x 100, y 100 } }, { id "node2", type "setmaterialcolor", position { x 300, y 100 }, data { entity "self", color { r 1, g 0, b 0, a 1 } } }, { id "node3", type "playsound", position { x 500, y 100 }, data { soundfile "sounds/button click mp3" } } ], connections \[ { source { nodeid "node1", pinid "exec" }, target { nodeid "node2", pinid "exec" }, type "execution" }, { source { nodeid "node2", pinid "exec" }, target { nodeid "node3", pinid "exec" }, type "execution" } ] }, run true, disabled false }); this example demonstrates creating a button entity with visual and interactive components adding a visual script component with a graph of nodes defining the execution flow from interaction to color change to sound configuring the script to run automatically when a player interacts with this button the oninteract event node activates execution flows to the setmaterialcolor node, changing the button to red execution continues to the playsound node, playing the click sound the script completes until the next interaction implementation details node registration the engine registers available node types through profiles // simplified concept from src/visualscript/nodes/profiles/engine/registerengineprofile ts function registerengineprofile(registry) { // register component getters registercomponentgetters(registry); // register component setters registercomponentsetters(registry); // register event nodes registereventnodes(registry); // register action nodes registeractionnodes(registry); // register flow control nodes registerflowcontrolnodes(registry); // register value nodes registervaluenodes(registry); } function registercomponentsetters(registry) { // register a node for setting material color registry registernodetype({ type 'setmaterialcolor', category 'components', inputs \[ { name 'exec', type 'execution' }, { name 'entity', type 'entity', defaultvalue 'self' }, { name 'color', type 'color' } ], outputs \[ { name 'exec', type 'execution' } ], execute (inputs, context) => { const { entity, color } = inputs; const targetentity = entity === 'self' ? context entity entity; // implementation that uses setcomponent to update material color // return {}; } }); // register other component setters // } this function registers different categories of nodes with the registry defines the inputs, outputs, and behavior of each node provides the implementation that connects nodes to engine functionality makes nodes available for use in visual scripts script serialization visual scripts are serialized to and from json for storage and editing // simplified concept of script serialization function serializevisualscript(graph) { const nodes = \[]; const connections = \[]; // serialize each node for (const node of graph nodes) { nodes push({ id node id, type node type, position node position, data node data }); } // serialize each connection for (const connection of graph connections) { connections push({ source { nodeid connection source nodeid, pinid connection source pinid }, target { nodeid connection target nodeid, pinid connection target pinid }, type connection type }); } return { nodes, connections }; } function deserializevisualscript(json, registry) { const graph = { nodes \[], connections \[] }; // create nodes from json for (const nodedata of json nodes) { const nodetype = registry getnodetype(nodedata type); if (!nodetype) continue; const node = createnode(nodetype, nodedata id, nodedata position, nodedata data); graph nodes push(node); } // create connections from json for (const connectiondata of json connections) { const connection = createconnection( connectiondata source, connectiondata target, connectiondata type ); graph connections push(connection); } return graph; } these functions convert between in memory graph representations and json preserve node types, positions, data, and connections allow scripts to be saved, loaded, and edited work with the node registry to instantiate the correct node types benefits of visual scripting the visual scripting system provides several key benefits accessibility makes programming accessible to team members without coding experience visualization provides a clear visual representation of logic flow rapid iteration enables quick changes and experimentation without recompiling reduced errors eliminates many syntax errors common in text based programming component integration seamlessly works with the ecs architecture extensibility can be extended with custom node types for specific game features these benefits make the visual scripting system a powerful tool for creating interactive behaviors in the ir engine next steps with an understanding of how to create complex behaviors using visual scripting, the next chapter explores how all these systems are organized and integrated into the engine's modular architecture next engine module system docid 1ac1t6b9dstg9jsmwkmhh