QuaternionEngine/docs/Scene.md

Scene System: Cameras, DrawContext, and Instances

Thin scene layer that produces RenderObjects for the renderer. It gathers opaque/transparent surfaces, maintains the main camera, and exposes simple runtime instance APIs.

Components

• SceneManager (src/scene/vk_scene.h/.cpp)

  • Owns the main Camera, GPUSceneData, and DrawContext.
  • Loads GLTF scenes via AssetManager/LoadedGLTF and creates dynamic mesh/GLTF instances.
  • Updates per‑frame transforms, camera, and GPUSceneData (view/proj/viewproj, sun/ambient).

• DrawContext

  • Two lists: OpaqueSurfaces and TransparentSurfaces of RenderObject.
  • Populated by scene graph traversal and dynamic instances each frame.

• RenderObject

  • Geometry: indexBuffer, vertexBuffer (for RG tracking), vertexBufferAddress (device address used by shaders).
  • Material: MaterialInstance* material with bound set and pipeline.
  • Transform and bounds for optional culling.

Frame Flow

1. SceneManager::update_scene() clears the draw lists and rebuilds them by drawing all active scene/instance nodes.
2. Renderer consumes the lists:
   • Geometry pass sorts opaque by material and index buffer to improve locality.
   • Transparent pass sorts back‑to‑front against camera and blends to the HDR target.
3. Uniforms: Passes allocate a small per‑frame UBO (GPUSceneData) and bind it via a shared layout.

Sorting / Culling

• Opaque (geometry): stable sort by material then indexBuffer (see src/render/passes/geometry.cpp).
• Transparent: sort by camera‑space depth far→near (see src/render/passes/transparent.cpp).
• An example frustum test exists in passes/geometry.cpp (is_visible) and can be enabled to cull meshes.

Dynamic Instances

• Mesh instances

  • addMeshInstance(name, mesh, transform), removeMeshInstance(name), clearMeshInstances().
  • Useful for spawning primitives or asset meshes at runtime.

• GLTF instances

  • addGLTFInstance(name, LoadedGLTF, transform), removeGLTFInstance(name), clearGLTFInstances().

GLTF Animation / “Actions”

GLTF files can contain one or more animation clips (e.g. Idle, Walk, Run). The loader (LoadedGLTF) parses these into LoadedGLTF::Animation objects. Animation state (which clip, time, loop flag) is stored outside the glTF asset:

• One AnimationState per named static scene (for loadScene).
• One AnimationState per runtime glTF instance (SceneManager::GLTFInstance).

This means that animation is independent per scene and per instance, even if they share the same underlying LoadedGLTF asset and meshes.

Static scenes (loaded via loadScene)

Example: engine default scene in VulkanEngine::init():

• structure is loaded and registered via:
  • sceneManager->loadScene("structure", structureFile);

To control its animation state:

• By index (per‑scene state):
  • scene->setSceneAnimation("structure", 0); // first clip
  • scene->setSceneAnimation("structure", 1, true); // second clip, reset time
• By name (per‑scene state; matches glTF animation name):
  • scene->setSceneAnimation("structure", "Idle");
  • scene->setSceneAnimation("structure", "Run");
• Looping (per‑scene state):
  • scene->setSceneAnimationLoop("structure", true); // enable loop
  • scene->setSceneAnimationLoop("structure", false); // play once and stop at end

All functions return bool to indicate whether the scene name was found. A negative index (e.g. -1) disables animation for that scene (pose stays at the last evaluated frame).

Runtime GLTF instances

GLTF instances are created via:

• scene->addGLTFInstance("player", playerGltf, playerTransform);

You can treat each instance as an “actor” and drive its current action from your game state. Each instance has its own AnimationState, even if multiple instances share the same LoadedGLTF.

• By index (per‑instance state):
  • scene->setGLTFInstanceAnimation("player", 0);
  • scene->setGLTFInstanceAnimation("player", -1); // disable animation for this actor
• By name (per‑instance state):
  • scene->setGLTFInstanceAnimation("player", "Idle");
  • scene->setGLTFInstanceAnimation("player", "Run");
• Looping (per‑instance state):
  • scene->setGLTFInstanceAnimationLoop("player", true);

These helpers update the instance’s AnimationState. SceneManager::update_scene() advances each instance’s state every frame using a per‑frame dt before drawing, so once you select an action, it will keep playing automatically until you change it or disable looping for that instance.

Per‑Instance Node / Joint Overrides

For non‑skinned models (rigid parts), you can apply local‑space pose offsets to specific glTF nodes on a per‑instance basis. This is useful for things like control surfaces, doors, or turrets layered on top of an existing animation.

• API (on SceneManager):
  • bool setGLTFInstanceNodeOffset(const std::string &instanceName, const std::string &nodeName, const glm::mat4 &offset);
  • bool clearGLTFInstanceNodeOffset(const std::string &instanceName, const std::string &nodeName);
  • void clearGLTFInstanceNodeOffsets(const std::string &instanceName);

Notes:

• Offsets are local‑space post‑multipliers:
  • Effective local transform = node.localTransform * offset.
• Offsets are per instance:
  • Different instances of the same glTF can have different joint poses at the same animation time.
• Overrides are applied during draw via DrawContext::gltfNodeLocalOverrides and MeshNode::Draw, without modifying the shared glTF asset.

GPU Scene Data

• GPUSceneData carries camera matrices and lighting constants for the frame.
• Passes map and fill it into a per-frame UBO, bindable with DescriptorManager::gpuSceneDataLayout().

Point Lights

• SceneManager::PointLight
  • position – world‑space position.
  • radius – approximate influence radius (used for falloff).
  • color – RGB color.
  • intensity – scalar brightness.
• API
  • addPointLight(const PointLight &light)
  • clearPointLights()
  • getPointLightCount(), getPointLight(index, outLight), setPointLight(index, light), removePointLight(index)
• Usage pattern
  • On level load: call addPointLight for each baked/runtime point light.
  • At runtime (e.g. gameplay): read/modify lights via the indexed helpers.

Picking & Selection (Game‑Facing)

The scene system exposes CPU ray‑based picking and rectangle selection that the engine uses for editor tools, but you can also call them directly from game code.

• Single‑object ray pick

  • Function: bool SceneManager::pick(const glm::vec2 &mousePosPixels, RenderObject &outObject, glm::vec3 &outWorldPos)
  • Input: mousePosPixels in window coordinates (SDL style), origin at top‑left.
  • Output:
    • outObject – the closest hit RenderObject (opaque or transparent) along the camera ray.
    • outWorldPos – world‑space hit position (mesh BVH‑refined when available).
  • Returns true when something was hit, false otherwise.

• ID‑buffer picking

  • In addition to CPU ray picks, the engine can render an object‑ID buffer and read back a single pixel.
  • Core API:
    • SceneManager::resolveObjectID(uint32_t id, RenderObject &outObject) const
      • Takes an ID from the ID buffer and resolves it back to the RenderObject in the latest DrawContext.
  • Engine integration:
    • RenderObject::objectID is assigned in MeshNode::Draw and consumed by the geometry pass to write the ID buffer.
    • VulkanEngine wires a small PickReadback render‑graph pass that copies one pixel from the ID buffer into a CPU readback buffer when ID‑buffer picking is enabled.

• Rectangle selection

  • Function: void SceneManager::selectRect(const glm::vec2 &p0, const glm::vec2 &p1, std::vector<RenderObject> &outObjects) const
  • Input: p0, p1 – opposite corners of a screen‑space rectangle in window coordinates (top‑left origin).
  • Output: outObjects – all RenderObjects whose projected bounds intersect the rectangle.
  • Internals:
    • Uses sceneData.viewproj and box_overlaps_ndc_rect to test each object’s bounds in clip space.

Engine‑Level Picking Helpers

VulkanEngine wraps the scene picking functions and keeps a small amount of per‑frame state that game/editor code can query:

• Structures on VulkanEngine

  • struct PickInfo
    • MeshAsset *mesh, LoadedGLTF *scene, Node *node
    • RenderObject::OwnerType ownerType, std::string ownerName
    • glm::vec3 worldPos, glm::mat4 worldTransform
    • uint32_t indexCount, firstIndex, surfaceIndex
    • bool valid
  • Fields:
    • _lastPick – result of the last click selection (via CPU ray or ID‑buffer).
    • _hoverPick – result of the last per‑frame hover raycast (under the mouse cursor).
    • _dragSelection – list of PickInfo filled by SceneManager::selectRect when a drag‑select completes.
    • _useIdBufferPicking – toggles between CPU ray picking and ID‑buffer picking.

• How the engine fills them (for reference)

  • Mouse hover:
    • The frame loop tracks _mousePosPixels from SDL mouse motion events.
    • Each frame, VulkanEngine::draw() calls SceneManager::pick(_mousePosPixels, ...) and stores the result in _hoverPick.
  • Click selection:
    • On mouse button release, the engine either:
      • Queues a pick request for the ID buffer (if _useIdBufferPicking is true), or
      • Calls SceneManager::pick(...) directly (CPU ray).
    • The resolved result is stored in _lastPick.
  • Drag selection:
    • The engine tracks a drag rectangle in screen space and, on release, calls SceneManager::selectRect(...) and converts each RenderObject into a PickInfo stored in _dragSelection.

• Typical game‑side usage

  • Hover tooltips:
    • Read engine->_hoverPick each frame; if valid, use ownerName, worldPos, or ownerType to drive UI.
  • Click selection:
    • After input handling, inspect engine->_lastPick for the most recent clicked object.
  • Multi‑select:
    • After a drag, iterate engine->_dragSelection for group actions.

Tips

• Treat DrawContext as immutable during rendering; build it fully in update_scene().
• Keep RenderObject small; use device addresses for vertex data to avoid per-draw vertex buffer binds.
• For custom sorting/culling, modify only the scene layer; render passes stay simple.