QuaternionEngine/docs/Picking.md

Picking System: Object Selection and Interaction

Unified picking system that handles single-click selection, hover detection, and drag-box multi-select. Supports both CPU ray-casting (via BVH) and GPU ID-buffer picking.

Components

• PickingSystem (src/core/picking/picking_system.h/.cpp)

  • Main entry point for all picking operations.
  • Processes SDL mouse events (click, drag, release).
  • Maintains per-frame hover picks and last click selection.
  • Integrates with RenderGraph for async ID-buffer readback.

• SceneManager picking helpers (src/scene/vk_scene_picking.cpp)

  • CPU ray-casting against RenderObject bounds.
  • BVH-accelerated mesh picking for precise triangle-level hits.
  • Rectangle selection in NDC space.

PickInfo Structure

Result of any pick operation:

struct PickInfo
{
    MeshAsset *mesh = nullptr;           // Source mesh asset
    LoadedGLTF *scene = nullptr;         // Source glTF scene
    Node *node = nullptr;                // glTF node that owns this surface
    RenderObject::OwnerType ownerType;   // StaticGLTF, GLTFInstance, MeshInstance
    std::string ownerName;               // Logical name (e.g., "player", "cube01")
    WorldVec3 worldPos;                  // Hit position in world space (double-precision)
    glm::mat4 worldTransform;            // Object's world transform
    uint32_t indexCount;                 // Index count of picked surface
    uint32_t firstIndex;                 // First index of picked surface
    uint32_t surfaceIndex;               // Surface index within mesh
    bool valid = false;                  // True if pick hit something
};

Picking Modes

1. CPU Ray Picking (Default)

Uses camera ray against object bounds with BVH acceleration:

• Sphere bounds: Quick bounding sphere test.
• Box bounds: Ray-box intersection in local space.
• Capsule bounds: Combined cylinder + sphere caps intersection.
• Mesh bounds: Full BVH traversal for triangle-level precision.

Advantages:

• Immediate results (same frame).
• No GPU readback latency.
• Precise triangle-level hits with mesh BVH.

2. ID-Buffer Picking

Renders object IDs to a GPU buffer and reads back single pixel:

• Each RenderObject has a unique objectID assigned during draw.
• Geometry pass writes IDs to an R32_UINT attachment.
• PickReadback render pass copies single pixel to CPU-readable buffer.
• Result resolved on next frame after GPU fence wait.

Advantages:

• Pixel-perfect accuracy (no false positives from bounds).
• Handles complex overlapping geometry.
• Works with any object shape.

Enable via:

picking_system.set_use_id_buffer_picking(true);

PickingSystem API

Initialization:

void init(EngineContext *context);
void cleanup();

Frame Lifecycle:

void begin_frame();  // Resolve pending async picks after fence wait
void process_event(const SDL_Event &event, bool ui_want_capture_mouse);
void update_hover(); // Update hover pick each frame

Results Access:

const PickInfo& last_pick() const;           // Last click selection
const PickInfo& hover_pick() const;          // Current hover (under cursor)
const std::vector<PickInfo>& drag_selection() const;  // Multi-select results
uint32_t last_pick_object_id() const;        // Raw object ID of last pick

Configuration:

bool use_id_buffer_picking() const;
void set_use_id_buffer_picking(bool enabled);

bool debug_draw_bvh() const;
void set_debug_draw_bvh(bool enabled);

Utilities:

// Clear picks for removed objects (call when deleting instances)
void clear_owner_picks(RenderObject::OwnerType owner_type, const std::string &owner_name);

// Mutable access for engine integration
PickInfo* mutable_last_pick();
PickInfo* mutable_hover_pick();

RenderGraph Integration:

// Called during RenderGraph build when ID-buffer picking is enabled
void register_id_buffer_readback(RenderGraph &graph,
                                 RGImageHandle id_buffer,
                                 VkExtent2D draw_extent,
                                 VkExtent2D swapchain_extent);

SceneManager Picking API

Lower-level picking functions for custom use:

// Single-object ray pick
bool pick(const glm::vec2 &mousePosPixels, RenderObject &outObject, WorldVec3 &outWorldPos);

// Resolve ID from ID-buffer back to RenderObject
bool resolveObjectID(uint32_t id, RenderObject &outObject) const;

// Rectangle selection (drag-box)
void selectRect(const glm::vec2 &p0, const glm::vec2 &p1,
                std::vector<RenderObject> &outObjects) const;

Usage Examples

Basic Click Selection:

void Game::handle_selection(PickingSystem& picking)
{
    const PickingSystem::PickInfo& pick = picking.last_pick();

    if (pick.valid)
    {
        fmt::println("Selected: {} at ({}, {}, {})",
                     pick.ownerName,
                     pick.worldPos.x, pick.worldPos.y, pick.worldPos.z);

        // Handle different owner types
        switch (pick.ownerType)
        {
            case RenderObject::OwnerType::GLTFInstance:
                select_actor(pick.ownerName);
                break;
            case RenderObject::OwnerType::MeshInstance:
                select_prop(pick.ownerName);
                break;
            default:
                break;
        }
    }
}

Hover Tooltips:

void Game::update_ui(PickingSystem& picking)
{
    const PickingSystem::PickInfo& hover = picking.hover_pick();

    if (hover.valid)
    {
        show_tooltip(hover.ownerName);
    }
}

Multi-Select with Drag Box:

void Game::process_drag_selection(PickingSystem& picking)
{
    const auto& selection = picking.drag_selection();

    for (const PickingSystem::PickInfo& pick : selection)
    {
        if (pick.valid)
        {
            add_to_selection(pick.ownerName);
        }
    }
}

Custom Ray Pick:

void Game::custom_pick(SceneManager& scene, const glm::vec2& screen_pos)
{
    RenderObject hit_object{};
    WorldVec3 hit_pos{};

    if (scene.pick(screen_pos, hit_object, hit_pos))
    {
        // hit_object contains the picked render object
        // hit_pos is the precise world-space hit position
        spawn_effect_at(hit_pos);
    }
}

Bounds Types

Objects can use different bounds types for picking:

enum class BoundsType : uint8_t
{
    None = 0,    // Not pickable
    Box = 1,     // AABB (default for most objects)
    Sphere = 2,  // Bounding sphere
    Capsule = 3, // Cylinder + hemisphere caps
    Mesh = 4,    // Full BVH mesh intersection
};

Set bounds type when adding instances:

scene->addMeshInstance("capsule_enemy", mesh, transform, BoundsType::Capsule);

Mesh BVH Picking

For precise triangle-level picking on complex meshes:

1. Mesh assets automatically build a BVH during loading.
2. BoundsType::Mesh triggers BVH traversal instead of simple bounds test.
3. Returns exact triangle hit position, not just bounding box intersection.
4. PickInfo::firstIndex and indexCount are refined to the exact triangle.

Debug BVH visualization:

picking.set_debug_draw_bvh(true);  // Shows BVH nodes in debug overlay

Coordinate Space Handling

The picking system handles multiple coordinate transformations:

1. Window pixels (SDL event coordinates, top-left origin)
2. Swapchain pixels (scaled for HiDPI displays)
3. Logical render pixels (internal render resolution)
4. NDC (normalized device coordinates, -1 to 1)
5. World space (double-precision WorldVec3)

The window_to_swapchain_pixels() helper handles HiDPI scaling. Letterboxing is accounted for when render resolution differs from swapchain size.

Frame Flow

1. begin_frame() — Resolve pending async ID-buffer picks from previous frame.
2. process_event() — Handle mouse events (click start/end, motion).
3. update_hover() — CPU ray-cast for current hover under cursor.
4. RenderGraph build — If ID-buffer picking enabled, register readback pass.
5. Next frame — Async pick result becomes available.

Integration with ImGui

The picking system respects ImGui's input capture:

// In event loop
bool ui_capture = imgui_system.want_capture_mouse();
picking_system.process_event(event, ui_capture);

When ui_want_capture_mouse is true:

• Click events are ignored (no picks started).
• Mouse motion still updates cursor position for future picks.
• Hover picking is not affected.

Tips

• Use CPU ray picking (set_use_id_buffer_picking(false)) for immediate feedback.
• Use ID-buffer picking for pixel-perfect selection in dense scenes.
• Call clear_owner_picks() when removing instances to avoid stale picks.
• For mesh BVH to work, ensure the mesh was loaded with BVH generation enabled.
• The drag selection threshold is 3 pixels — smaller motions are treated as clicks.