ADD: Update Docs
This commit is contained in:
191
docs/FloatingOrigin.md
Normal file
191
docs/FloatingOrigin.md
Normal file
@@ -0,0 +1,191 @@
|
||||
## Floating Origin & Double-Precision Coordinates
|
||||
|
||||
Precision-safe coordinate system that stores authoritative world positions in double precision and dynamically shifts the rendering origin to keep local float coordinates near zero.
|
||||
|
||||
### Problem
|
||||
|
||||
Single-precision floats (32-bit) have ~7 significant digits. At large world positions (e.g., 100km from origin), sub-meter precision is lost, causing:
|
||||
|
||||
- Vertex jitter / z-fighting
|
||||
- Camera stutter during movement
|
||||
- Picking inaccuracy
|
||||
- Physics instability
|
||||
|
||||
### Solution
|
||||
|
||||
1. **Double-precision world coordinates**: Store authoritative positions as `glm::dvec3` (`WorldVec3`).
|
||||
2. **Floating origin**: Maintain a world-space origin point that follows the camera.
|
||||
3. **Local-space rendering**: Convert world positions to local float coordinates relative to the origin.
|
||||
|
||||
This keeps all render-local coordinates within a few hundred meters of (0,0,0), preserving full float precision.
|
||||
|
||||
### Core Types (`src/core/world.h`)
|
||||
|
||||
```c++
|
||||
// Authoritative world-space coordinates (double precision)
|
||||
using WorldVec3 = glm::dvec3;
|
||||
|
||||
// Convert world position to local float (for rendering)
|
||||
glm::vec3 world_to_local(const WorldVec3 &world, const WorldVec3 &origin_world);
|
||||
|
||||
// Convert local float back to world position
|
||||
WorldVec3 local_to_world(const glm::vec3 &local, const WorldVec3 &origin_world);
|
||||
|
||||
// Snap a world position to a grid (for stable origin placement)
|
||||
WorldVec3 snap_world(const WorldVec3 &p, double grid_size);
|
||||
```
|
||||
|
||||
### Origin Management (`SceneManager`)
|
||||
|
||||
The `SceneManager` automatically recenters the origin when the camera drifts too far:
|
||||
|
||||
```c++
|
||||
// Private members in SceneManager
|
||||
WorldVec3 _origin_world{0.0, 0.0, 0.0};
|
||||
glm::vec3 _camera_position_local{0.0f, 0.0f, 0.0f};
|
||||
double _floating_origin_recenter_threshold = 1000.0; // meters
|
||||
double _floating_origin_snap_size = 100.0; // grid snap size
|
||||
```
|
||||
|
||||
**Recentering Logic** (in `update_scene()`):
|
||||
|
||||
1. Compute distance from camera to current origin.
|
||||
2. If distance exceeds threshold, snap camera position to grid and use as new origin.
|
||||
3. Recompute all local positions relative to new origin.
|
||||
|
||||
```c++
|
||||
if (_floating_origin_recenter_threshold > 0.0) {
|
||||
const WorldVec3 d = mainCamera.position_world - _origin_world;
|
||||
if (glm::length2(d) > threshold2) {
|
||||
_origin_world = snap_world(mainCamera.position_world, _floating_origin_snap_size);
|
||||
}
|
||||
}
|
||||
_camera_position_local = world_to_local(mainCamera.position_world, _origin_world);
|
||||
```
|
||||
|
||||
### Coordinate Storage
|
||||
|
||||
| Component | Type | Description |
|
||||
|-----------|------|-------------|
|
||||
| `Camera::position_world` | `WorldVec3` | Camera world position (double) |
|
||||
| `MeshInstance::translation_world` | `WorldVec3` | Instance world position |
|
||||
| `GLTFInstance::translation_world` | `WorldVec3` | glTF instance world position |
|
||||
| `PointLight::position_world` | `WorldVec3` | Light world position |
|
||||
| `SceneManager::_origin_world` | `WorldVec3` | Current floating origin |
|
||||
| `SceneManager::_camera_position_local` | `glm::vec3` | Camera in local/render space |
|
||||
|
||||
### Rendering Pipeline
|
||||
|
||||
During `update_scene()`, all world-space objects are converted to local coordinates:
|
||||
|
||||
1. **Static glTF scenes**: Apply `world_to_local_root` transform that offsets by `-origin`.
|
||||
2. **Dynamic instances**: Convert `translation_world` to local position, build TRS matrix.
|
||||
3. **Point lights**: Convert `position_world` to local for GPU upload.
|
||||
4. **Camera**: Store both world and local positions; view matrix uses local.
|
||||
|
||||
```c++
|
||||
// Root transform for static world content
|
||||
const glm::mat4 world_to_local_root =
|
||||
glm::translate(glm::mat4{1.f}, world_to_local(WorldVec3(0.0, 0.0, 0.0), _origin_world));
|
||||
|
||||
// Dynamic instance positioning
|
||||
glm::vec3 tLocal = world_to_local(inst.translation_world, _origin_world);
|
||||
glm::mat4 instanceTransform = make_trs_matrix(tLocal, inst.rotation, inst.scale);
|
||||
```
|
||||
|
||||
### GameAPI Double-Precision Variants
|
||||
|
||||
The `GameAPI` exposes both float and double-precision transform types:
|
||||
|
||||
```c++
|
||||
// Float-precision (relative/local usage)
|
||||
struct Transform {
|
||||
glm::vec3 position{0.0f};
|
||||
glm::quat rotation{1.0f, 0.0f, 0.0f, 0.0f};
|
||||
glm::vec3 scale{1.0f};
|
||||
};
|
||||
|
||||
// Double-precision (world-space usage)
|
||||
struct TransformD {
|
||||
glm::dvec3 position{0.0};
|
||||
glm::quat rotation{1.0f, 0.0f, 0.0f, 0.0f};
|
||||
glm::vec3 scale{1.0f};
|
||||
};
|
||||
|
||||
// API accepts both variants
|
||||
bool add_gltf_instance(const std::string& name, const std::string& modelPath,
|
||||
const Transform& transform, bool preloadTextures = true);
|
||||
bool add_gltf_instance(const std::string& name, const std::string& modelPath,
|
||||
const TransformD& transform, bool preloadTextures = true);
|
||||
```
|
||||
|
||||
Similarly for `PointLight` / `PointLightD` and `IBLVolume` / `IBLVolumeD`.
|
||||
|
||||
### Picking Integration
|
||||
|
||||
Picking operates in local coordinates but returns world positions:
|
||||
|
||||
```c++
|
||||
// Ray origin in local space
|
||||
glm::vec3 rayOrigin = world_to_local(mainCamera.position_world, _origin_world);
|
||||
|
||||
// ... perform ray-object intersection in local space ...
|
||||
|
||||
// Convert hit position back to world
|
||||
outWorldPos = local_to_world(bestHitPos, _origin_world);
|
||||
```
|
||||
|
||||
### Query Functions
|
||||
|
||||
```c++
|
||||
// Get current floating origin (world coordinates)
|
||||
WorldVec3 SceneManager::get_world_origin() const;
|
||||
|
||||
// Get camera position in local/render coordinates
|
||||
glm::vec3 SceneManager::get_camera_local_position() const;
|
||||
```
|
||||
|
||||
### Configuration Parameters
|
||||
|
||||
| Parameter | Default | Description |
|
||||
|-----------|---------|-------------|
|
||||
| `_floating_origin_recenter_threshold` | 1000.0 | Distance (m) before recentering |
|
||||
| `_floating_origin_snap_size` | 100.0 | Grid snap size (m) for new origin |
|
||||
|
||||
Setting `_floating_origin_recenter_threshold` to 0 or negative disables floating origin.
|
||||
|
||||
### Best Practices
|
||||
|
||||
1. **Store world positions in double**: Use `WorldVec3` / `TransformD` for anything that persists or needs absolute positioning.
|
||||
|
||||
2. **Work in local space for rendering**: All GPU-visible transforms should be in local coordinates.
|
||||
|
||||
3. **Convert at boundaries**: Convert world↔local at the scene update boundary, not per-frame in shaders.
|
||||
|
||||
4. **Use snap size**: Grid-aligned origins prevent micro-shifts that could cause subtle jitter.
|
||||
|
||||
5. **Consider threshold tuning**:
|
||||
- Smaller threshold (500m): More frequent recenters, tighter precision
|
||||
- Larger threshold (2000m): Fewer recenters, slightly reduced precision at edges
|
||||
|
||||
### Debugging
|
||||
|
||||
The engine UI displays origin and camera positions:
|
||||
|
||||
```
|
||||
Origin (world): (1200.000, 0.000, 3400.000)
|
||||
Camera (local): (23.456, 1.234, -12.789)
|
||||
```
|
||||
|
||||
If you observe jitter at large world coordinates, verify:
|
||||
- Positions are stored as `WorldVec3`, not `glm::vec3`
|
||||
- `world_to_local()` is applied before GPU upload
|
||||
- Origin recentering is enabled (threshold > 0)
|
||||
|
||||
### Related Files
|
||||
|
||||
- `src/core/world.h` — `WorldVec3` type and conversion functions
|
||||
- `src/scene/vk_scene.h` — `SceneManager` origin management
|
||||
- `src/scene/vk_scene.cpp` — Recentering logic in `update_scene()`
|
||||
- `src/scene/camera.h` — `Camera::position_world` double-precision position
|
||||
- `src/core/game_api.h` — `Transform` / `TransformD` API types
|
||||
361
docs/GameAPI.md
361
docs/GameAPI.md
@@ -7,6 +7,9 @@ For details on the underlying systems, see also:
|
||||
- `docs/Scene.md` – cameras, draw context, instances, picking.
|
||||
- `docs/EngineContext.md` – access to managers and per‑frame state.
|
||||
- `docs/RenderGraph.md` – render‑graph API for custom passes.
|
||||
- `docs/InputSystem.md` – keyboard, mouse, and cursor mode handling.
|
||||
- `docs/Picking.md` – object selection, hover detection, and drag-box multi-select.
|
||||
- `docs/ImGuiSystem.md` – immediate-mode UI integration and debug widgets.
|
||||
|
||||
---
|
||||
|
||||
@@ -24,6 +27,7 @@ Implementation: `src/core/game_api.cpp`
|
||||
- Post‑processing (tonemap, bloom, FXAA).
|
||||
- Camera control.
|
||||
- Picking and render‑graph pass toggles.
|
||||
- Input handling (keyboard, mouse, cursor modes).
|
||||
|
||||
Typical creation:
|
||||
|
||||
@@ -184,6 +188,152 @@ This writes into `VulkanEngine::_rgPassToggles` and is applied during RenderGrap
|
||||
|
||||
---
|
||||
|
||||
## Input System
|
||||
|
||||
Header: `src/core/input/input_system.h`
|
||||
Docs: `docs/InputSystem.md`
|
||||
|
||||
The engine provides a unified input abstraction layer that wraps SDL2 events. Access it via `VulkanEngine::input()` or store a reference during initialization.
|
||||
|
||||
### Polled State (Recommended for Games)
|
||||
|
||||
Query current keyboard/mouse state each frame:
|
||||
|
||||
```cpp
|
||||
void Game::update(VulkanEngine& engine)
|
||||
{
|
||||
const InputState& input = engine.input().state();
|
||||
|
||||
// Movement (held keys)
|
||||
glm::vec3 move{0.0f};
|
||||
if (input.key_down(Key::W)) move.z -= 1.0f;
|
||||
if (input.key_down(Key::S)) move.z += 1.0f;
|
||||
if (input.key_down(Key::A)) move.x -= 1.0f;
|
||||
if (input.key_down(Key::D)) move.x += 1.0f;
|
||||
|
||||
// Sprint modifier
|
||||
float speed = input.modifiers().shift ? 10.0f : 5.0f;
|
||||
player.move(move * speed * dt);
|
||||
|
||||
// Fire on left click (just pressed this frame)
|
||||
if (input.mouse_pressed(MouseButton::Left))
|
||||
{
|
||||
player.fire();
|
||||
}
|
||||
|
||||
// Toggle menu on Escape (just pressed)
|
||||
if (input.key_pressed(Key::Escape))
|
||||
{
|
||||
ui.toggle_menu();
|
||||
}
|
||||
|
||||
// Mouse look (right button held)
|
||||
if (input.mouse_down(MouseButton::Right))
|
||||
{
|
||||
engine.input().set_cursor_mode(CursorMode::Relative);
|
||||
glm::vec2 delta = input.mouse_delta();
|
||||
camera.rotate(delta.x * 0.1f, delta.y * 0.1f);
|
||||
}
|
||||
else
|
||||
{
|
||||
engine.input().set_cursor_mode(CursorMode::Normal);
|
||||
}
|
||||
|
||||
// Scroll wheel for zoom
|
||||
float scroll = input.wheel_delta().y;
|
||||
if (scroll != 0.0f)
|
||||
{
|
||||
camera.zoom(scroll * 0.5f);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Key State Types
|
||||
|
||||
- `key_down(Key)` / `mouse_down(MouseButton)` — true while held (continuous actions)
|
||||
- `key_pressed(Key)` / `mouse_pressed(MouseButton)` — true only on the frame it was pressed (one-shot)
|
||||
- `key_released(Key)` / `mouse_released(MouseButton)` — true only on the frame it was released
|
||||
|
||||
### Available Keys
|
||||
|
||||
Letters: `Key::A` through `Key::Z`
|
||||
Numbers: `Key::Num0` through `Key::Num9`
|
||||
Special: `Key::Enter`, `Key::Escape`, `Key::Space`, `Key::Tab`, `Key::Backspace`
|
||||
Modifiers: `Key::LeftShift`, `Key::LeftCtrl`, `Key::LeftAlt`, `Key::LeftSuper` (and Right variants)
|
||||
|
||||
### Mouse Buttons
|
||||
|
||||
`MouseButton::Left`, `MouseButton::Middle`, `MouseButton::Right`, `MouseButton::X1`, `MouseButton::X2`
|
||||
|
||||
### Cursor Modes
|
||||
|
||||
```cpp
|
||||
enum class CursorMode : uint8_t
|
||||
{
|
||||
Normal = 0, // Default visible cursor
|
||||
Hidden = 1, // Hidden but not captured
|
||||
Relative = 2, // FPS-style: hidden + captured, only delta matters
|
||||
};
|
||||
|
||||
// Set via:
|
||||
engine.input().set_cursor_mode(CursorMode::Relative);
|
||||
```
|
||||
|
||||
### Mouse Position and Motion
|
||||
|
||||
```cpp
|
||||
glm::vec2 pos = input.mouse_position(); // Screen coordinates (pixels)
|
||||
glm::vec2 delta = input.mouse_delta(); // Frame motion delta
|
||||
glm::vec2 wheel = input.wheel_delta(); // Scroll wheel delta
|
||||
```
|
||||
|
||||
### Modifier Keys
|
||||
|
||||
```cpp
|
||||
InputModifiers mods = input.modifiers();
|
||||
if (mods.ctrl && input.key_pressed(Key::S))
|
||||
{
|
||||
save_game();
|
||||
}
|
||||
```
|
||||
|
||||
### Event-Driven Access (For UI)
|
||||
|
||||
For text input or other event-driven needs:
|
||||
|
||||
```cpp
|
||||
for (const InputEvent& ev : engine.input().events())
|
||||
{
|
||||
if (ev.type == InputEvent::Type::KeyDown)
|
||||
{
|
||||
if (ev.key == Key::Backspace)
|
||||
{
|
||||
text_field.delete_char();
|
||||
}
|
||||
}
|
||||
else if (ev.type == InputEvent::Type::MouseWheel)
|
||||
{
|
||||
scroll_view.scroll(ev.wheel_delta.y);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Window State
|
||||
|
||||
```cpp
|
||||
if (engine.input().quit_requested())
|
||||
{
|
||||
// Handle window close
|
||||
}
|
||||
|
||||
if (engine.input().window_minimized())
|
||||
{
|
||||
// Skip heavy rendering
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## VulkanEngine Helpers
|
||||
|
||||
Header: `src/core/engine.h`
|
||||
@@ -300,66 +450,183 @@ Typical usage:
|
||||
|
||||
---
|
||||
|
||||
## Picking & Selection (Interaction)
|
||||
## Picking System
|
||||
|
||||
Picking lives in `SceneManager` and is wired into `VulkanEngine`’s frame loop.
|
||||
Header: `src/core/picking/picking_system.h`
|
||||
Docs: `docs/Picking.md`
|
||||
|
||||
Header: `src/scene/vk_scene.h`
|
||||
Implementation: `src/scene/vk_scene_picking.cpp`
|
||||
The engine provides a unified picking system that handles click selection, hover detection, and drag-box multi-select. Access it via `VulkanEngine::picking()`.
|
||||
|
||||
### Single‑Object Ray Picking
|
||||
### Accessing Pick Results
|
||||
|
||||
- `bool pick(const glm::vec2 &mousePosPixels, RenderObject &outObject, glm::vec3 &outWorldPos);`
|
||||
- Input:
|
||||
- `mousePosPixels` – window coordinates (SDL style), origin at top‑left.
|
||||
- Output on success:
|
||||
- `outObject` – closest `RenderObject` hit by the camera ray.
|
||||
- `outWorldPos` – precise world‑space hit position (uses mesh BVH when available).
|
||||
- Returns:
|
||||
- `true` if any object was hit, otherwise `false`.
|
||||
```cpp
|
||||
void Game::handle_interaction(VulkanEngine& engine)
|
||||
{
|
||||
PickingSystem& picking = engine.picking();
|
||||
|
||||
### ID‑Buffer Picking
|
||||
// Last click selection
|
||||
const PickingSystem::PickInfo& pick = picking.last_pick();
|
||||
if (pick.valid)
|
||||
{
|
||||
fmt::println("Selected: {}", pick.ownerName);
|
||||
interact_with(pick.ownerName, pick.worldPos);
|
||||
}
|
||||
|
||||
- `bool resolveObjectID(uint32_t id, RenderObject &outObject) const;`
|
||||
- Takes an ID read back from the ID buffer and resolves it to the corresponding `RenderObject` in the latest `DrawContext`.
|
||||
- Used by the engine when `_useIdBufferPicking` is enabled to implement asynchronous picking.
|
||||
// Current hover (for tooltips)
|
||||
const PickingSystem::PickInfo& hover = picking.hover_pick();
|
||||
if (hover.valid)
|
||||
{
|
||||
show_tooltip(hover.ownerName);
|
||||
}
|
||||
|
||||
### Rectangle Selection (Drag Box)
|
||||
// Drag-box multi-select
|
||||
for (const auto& sel : picking.drag_selection())
|
||||
{
|
||||
if (sel.valid)
|
||||
{
|
||||
add_to_selection(sel.ownerName);
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
- `void selectRect(const glm::vec2 &p0, const glm::vec2 &p1, std::vector<RenderObject> &outObjects) const;`
|
||||
- Inputs:
|
||||
- `p0`, `p1` – opposite corners of a window‑space rectangle (top‑left origin).
|
||||
- Output:
|
||||
- `outObjects` – appended with all `RenderObject`s whose projected bounds intersect the rectangle.
|
||||
- Internals:
|
||||
- Uses `sceneData.viewproj` and an NDC‑space bounds test to determine overlap.
|
||||
### PickInfo Structure
|
||||
|
||||
### Typical Game Usage
|
||||
```cpp
|
||||
struct PickInfo
|
||||
{
|
||||
MeshAsset *mesh; // Source mesh
|
||||
LoadedGLTF *scene; // Source glTF
|
||||
Node *node; // glTF node
|
||||
RenderObject::OwnerType ownerType; // StaticGLTF, GLTFInstance, MeshInstance
|
||||
std::string ownerName; // Logical name (e.g., "player")
|
||||
WorldVec3 worldPos; // Hit position (double-precision)
|
||||
glm::mat4 worldTransform; // Object transform
|
||||
uint32_t indexCount, firstIndex; // Picked surface indices
|
||||
uint32_t surfaceIndex; // Surface index in mesh
|
||||
bool valid; // True if hit something
|
||||
};
|
||||
```
|
||||
|
||||
- Hover tooltips:
|
||||
- Track mouse position in window coordinates.
|
||||
- Use `SceneManager::pick` directly, or read `VulkanEngine::get_last_pick()` and/or `_hoverPick` as documented in `Scene.md`.
|
||||
- Object selection / interaction:
|
||||
- On mouse click release, inspect `engine->get_last_pick()`:
|
||||
- If `valid`, dispatch interaction based on `ownerType` / `ownerName` (e.g. select unit, open door).
|
||||
- Multi‑select:
|
||||
- When implementing drag selection, call `SceneManager::selectRect` with the drag rectangle to get all hits, or reuse the engine’s `_dragSelection` mechanism.
|
||||
### Picking Modes
|
||||
|
||||
```cpp
|
||||
// CPU ray picking (default) - immediate, BVH-accelerated
|
||||
picking.set_use_id_buffer_picking(false);
|
||||
|
||||
// ID-buffer picking - pixel-perfect, 1-frame latency
|
||||
picking.set_use_id_buffer_picking(true);
|
||||
```
|
||||
|
||||
### Owner Types
|
||||
|
||||
```cpp
|
||||
enum class OwnerType
|
||||
{
|
||||
None,
|
||||
StaticGLTF, // Loaded via loadScene()
|
||||
GLTFInstance, // Runtime glTF instance
|
||||
MeshInstance, // Runtime mesh instance
|
||||
};
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## ImGui / ImGuizmo Editor Utilities
|
||||
## ImGui System
|
||||
|
||||
File: `src/core/engine_ui.cpp`
|
||||
Header: `src/core/ui/imgui_system.h`
|
||||
Docs: `docs/ImGuiSystem.md`
|
||||
|
||||
These are primarily debug/editor features but can be kept in a game build to provide in‑game tools.
|
||||
The engine integrates Dear ImGui for debug UI and editor tools. Access it via `VulkanEngine::imgui()`.
|
||||
|
||||
- Main entry:
|
||||
- `void vk_engine_draw_debug_ui(VulkanEngine *eng);`
|
||||
- Called once per frame by the engine to build the “Debug” window tabs.
|
||||
- Useful tools for games:
|
||||
- Scene tab:
|
||||
- Spawn glTF instances at runtime using `VulkanEngine::addGLTFInstance(...)`.
|
||||
- Spawn primitive mesh instances (cube/sphere) using `SceneManager::addMeshInstance(...)`.
|
||||
- Point‑light editor UI built on `SceneManager` light APIs.
|
||||
- Object gizmo (ImGuizmo):
|
||||
- Uses last pick / hover pick as the current target and manipulates transforms via `setMeshInstanceTransform` / `setGLTFInstanceTransform`.
|
||||
### Adding Custom UI
|
||||
|
||||
```cpp
|
||||
void Game::init(VulkanEngine& engine)
|
||||
{
|
||||
// Register your UI callback
|
||||
engine.imgui().add_draw_callback([this]() {
|
||||
draw_game_ui();
|
||||
});
|
||||
}
|
||||
|
||||
void Game::draw_game_ui()
|
||||
{
|
||||
if (ImGui::Begin("Game HUD"))
|
||||
{
|
||||
ImGui::Text("Score: %d", _score);
|
||||
ImGui::Text("Health: %.0f%%", _health * 100.0f);
|
||||
|
||||
if (ImGui::Button("Pause"))
|
||||
{
|
||||
toggle_pause();
|
||||
}
|
||||
}
|
||||
ImGui::End();
|
||||
}
|
||||
```
|
||||
|
||||
### Input Capture
|
||||
|
||||
Always check ImGui input capture before processing game input:
|
||||
|
||||
```cpp
|
||||
void Game::update(VulkanEngine& engine)
|
||||
{
|
||||
// Skip game mouse input when ImGui wants it
|
||||
if (!engine.imgui().want_capture_mouse())
|
||||
{
|
||||
handle_mouse_input();
|
||||
}
|
||||
|
||||
// Skip game keyboard input when ImGui wants it
|
||||
if (!engine.imgui().want_capture_keyboard())
|
||||
{
|
||||
handle_keyboard_input();
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Built-in Debug UI
|
||||
|
||||
The engine provides comprehensive debug widgets in `src/core/engine_ui.cpp`:
|
||||
|
||||
- **Window Tab**: Monitor selection, fullscreen modes, HiDPI info
|
||||
- **Stats Tab**: Frame time, FPS, draw calls, triangle count
|
||||
- **Scene Tab**: Instance spawning, point light editor, ImGuizmo gizmos
|
||||
- **Render Graph Tab**: Pass toggles, resource tracking
|
||||
- **Texture Streaming Tab**: VRAM budget, cache stats
|
||||
- **Shadows Tab**: Shadow mode, cascade visualization
|
||||
- **Post Processing Tab**: Tonemapping, bloom, FXAA, SSR settings
|
||||
|
||||
### ImGuizmo Integration
|
||||
|
||||
For 3D transform gizmos on selected objects:
|
||||
|
||||
```cpp
|
||||
#include "ImGuizmo.h"
|
||||
|
||||
void draw_gizmo(const PickingSystem::PickInfo& pick,
|
||||
const glm::mat4& view, const glm::mat4& proj)
|
||||
{
|
||||
if (!pick.valid) return;
|
||||
|
||||
glm::mat4 transform = pick.worldTransform;
|
||||
|
||||
ImGuizmo::SetOrthographic(false);
|
||||
ImGuizmo::SetDrawlist();
|
||||
|
||||
ImGuiIO& io = ImGui::GetIO();
|
||||
ImGuizmo::SetRect(0, 0, io.DisplaySize.x, io.DisplaySize.y);
|
||||
|
||||
if (ImGuizmo::Manipulate(glm::value_ptr(view),
|
||||
glm::value_ptr(proj),
|
||||
ImGuizmo::TRANSLATE,
|
||||
ImGuizmo::WORLD,
|
||||
glm::value_ptr(transform)))
|
||||
{
|
||||
// Apply transform to the picked object
|
||||
scene->setGLTFInstanceTransform(pick.ownerName, transform);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
314
docs/ImGuiSystem.md
Normal file
314
docs/ImGuiSystem.md
Normal file
@@ -0,0 +1,314 @@
|
||||
## ImGui System: Immediate-Mode UI Integration
|
||||
|
||||
Manages Dear ImGui lifecycle, event processing, and rendering within the Vulkan engine. Provides DPI-aware font scaling and callback-based UI composition.
|
||||
|
||||
### Components
|
||||
|
||||
- `ImGuiSystem` (src/core/ui/imgui_system.h/.cpp)
|
||||
- Initializes ImGui context with Vulkan backend.
|
||||
- Processes SDL events for ImGui input.
|
||||
- Manages draw callback registration.
|
||||
- Handles DPI scaling and font rebuilding.
|
||||
|
||||
- `ImGuiPass` (src/render/passes/imgui_pass.h/.cpp)
|
||||
- RenderGraph pass that records ImGui draw commands.
|
||||
- Renders to swapchain image using dynamic rendering.
|
||||
|
||||
- `engine_ui.cpp`
|
||||
- Built-in debug UI widgets (stats, render graph, texture streaming, etc.).
|
||||
- Example of using the draw callback system.
|
||||
|
||||
### ImGuiSystem API
|
||||
|
||||
**Initialization:**
|
||||
|
||||
```cpp
|
||||
void init(EngineContext *context);
|
||||
void cleanup();
|
||||
```
|
||||
|
||||
**Frame Lifecycle:**
|
||||
|
||||
```cpp
|
||||
void begin_frame(); // NewFrame + invoke draw callbacks
|
||||
void end_frame(); // ImGui::Render()
|
||||
```
|
||||
|
||||
**Event Processing:**
|
||||
|
||||
```cpp
|
||||
void process_event(const SDL_Event &event);
|
||||
```
|
||||
|
||||
**Draw Callbacks:**
|
||||
|
||||
```cpp
|
||||
void add_draw_callback(DrawCallback callback);
|
||||
void clear_draw_callbacks();
|
||||
|
||||
// DrawCallback type
|
||||
using DrawCallback = std::function<void()>;
|
||||
```
|
||||
|
||||
**Input Capture Queries:**
|
||||
|
||||
```cpp
|
||||
bool want_capture_mouse() const;
|
||||
bool want_capture_keyboard() const;
|
||||
```
|
||||
|
||||
**Swapchain Events:**
|
||||
|
||||
```cpp
|
||||
void on_swapchain_recreated(); // Update image count after resize
|
||||
```
|
||||
|
||||
### Usage Examples
|
||||
|
||||
**Basic Setup (Engine Internal):**
|
||||
|
||||
```cpp
|
||||
// In VulkanEngine::init()
|
||||
_imgui_system.init(&_context);
|
||||
|
||||
// Register debug UI callback
|
||||
_imgui_system.add_draw_callback([this]() {
|
||||
vk_engine_draw_debug_ui(this);
|
||||
});
|
||||
```
|
||||
|
||||
**Event Processing:**
|
||||
|
||||
```cpp
|
||||
// In event loop
|
||||
for (const SDL_Event& event : events)
|
||||
{
|
||||
_imgui_system.process_event(event);
|
||||
}
|
||||
```
|
||||
|
||||
**Frame Integration:**
|
||||
|
||||
```cpp
|
||||
// Start of frame (after input processing)
|
||||
_imgui_system.begin_frame();
|
||||
|
||||
// ... game update, scene rendering ...
|
||||
|
||||
// End of frame (before RenderGraph execution)
|
||||
_imgui_system.end_frame();
|
||||
```
|
||||
|
||||
**Custom UI Callback:**
|
||||
|
||||
```cpp
|
||||
void Game::init()
|
||||
{
|
||||
engine.imgui_system().add_draw_callback([this]() {
|
||||
draw_game_ui();
|
||||
});
|
||||
}
|
||||
|
||||
void Game::draw_game_ui()
|
||||
{
|
||||
if (ImGui::Begin("Game Stats"))
|
||||
{
|
||||
ImGui::Text("Score: %d", _score);
|
||||
ImGui::Text("Health: %.0f%%", _health * 100.0f);
|
||||
|
||||
if (ImGui::Button("Pause"))
|
||||
{
|
||||
toggle_pause();
|
||||
}
|
||||
}
|
||||
ImGui::End();
|
||||
}
|
||||
```
|
||||
|
||||
**Respecting Input Capture:**
|
||||
|
||||
```cpp
|
||||
void Game::update()
|
||||
{
|
||||
// Don't process game input when ImGui wants it
|
||||
if (!engine.imgui_system().want_capture_mouse())
|
||||
{
|
||||
handle_mouse_input();
|
||||
}
|
||||
|
||||
if (!engine.imgui_system().want_capture_keyboard())
|
||||
{
|
||||
handle_keyboard_input();
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### DPI Scaling
|
||||
|
||||
The system automatically handles HiDPI displays:
|
||||
|
||||
1. **DPI Detection**: Computed from swapchain extent vs window size ratio.
|
||||
2. **Font Scaling**: Base font size (16px) scaled by DPI factor.
|
||||
3. **Global Scale**: `FontGlobalScale` set to 1/DPI for proper sizing.
|
||||
4. **Dynamic Updates**: Fonts rebuilt when DPI changes (e.g., monitor switch).
|
||||
|
||||
DPI scale range: 0.5x to 4.0x (clamped for stability).
|
||||
|
||||
### Vulkan Integration
|
||||
|
||||
ImGui is initialized with:
|
||||
- **Dynamic Rendering**: No render pass objects, uses `VK_KHR_dynamic_rendering`.
|
||||
- **Dedicated Descriptor Pool**: Separate pool with generous limits for ImGui textures.
|
||||
- **Swapchain Format**: Renders directly to swapchain image format.
|
||||
|
||||
```cpp
|
||||
ImGui_ImplVulkan_InitInfo init_info{};
|
||||
init_info.Instance = device->instance();
|
||||
init_info.PhysicalDevice = device->physicalDevice();
|
||||
init_info.Device = device->device();
|
||||
init_info.QueueFamily = device->graphicsQueueFamily();
|
||||
init_info.Queue = device->graphicsQueue();
|
||||
init_info.DescriptorPool = _imgui_pool;
|
||||
init_info.MinImageCount = swapchain_image_count;
|
||||
init_info.ImageCount = swapchain_image_count;
|
||||
init_info.UseDynamicRendering = true;
|
||||
init_info.PipelineRenderingCreateInfo.colorAttachmentCount = 1;
|
||||
init_info.PipelineRenderingCreateInfo.pColorAttachmentFormats = &swapchain_format;
|
||||
init_info.MSAASamples = VK_SAMPLE_COUNT_1_BIT;
|
||||
```
|
||||
|
||||
### RenderGraph Integration
|
||||
|
||||
ImGui rendering is handled by `ImGuiPass`:
|
||||
|
||||
```cpp
|
||||
// In RenderGraph build
|
||||
_imgui_pass.register_graph(graph, swapchain_image_handle);
|
||||
|
||||
// Pass executes after all other rendering
|
||||
// Renders ImGui draw data to swapchain image
|
||||
```
|
||||
|
||||
The pass:
|
||||
1. Begins dynamic rendering on swapchain image.
|
||||
2. Calls `ImGui_ImplVulkan_RenderDrawData()`.
|
||||
3. Ends rendering.
|
||||
|
||||
### Built-in Debug UI
|
||||
|
||||
The engine provides comprehensive debug widgets in `engine_ui.cpp`:
|
||||
|
||||
**Window Tab:**
|
||||
- Monitor selection and fullscreen modes.
|
||||
- HiDPI status and size information.
|
||||
- GPU information display.
|
||||
|
||||
**Stats Tab:**
|
||||
- Frame time and FPS.
|
||||
- Draw call and triangle counts.
|
||||
- Memory usage statistics.
|
||||
|
||||
**Scene Tab:**
|
||||
- GLTF instance spawning.
|
||||
- Primitive mesh spawning.
|
||||
- Point light editor.
|
||||
- Object transform manipulation (ImGuizmo).
|
||||
|
||||
**Render Graph Tab:**
|
||||
- Pass list with toggle controls.
|
||||
- Resource tracking visualization.
|
||||
- Barrier inspection.
|
||||
|
||||
**Texture Streaming Tab:**
|
||||
- VRAM budget and usage.
|
||||
- Texture load queue status.
|
||||
- Cache statistics.
|
||||
|
||||
**Shadows Tab:**
|
||||
- Shadow mode selection.
|
||||
- Cascade visualization.
|
||||
- Ray-tracing hybrid controls.
|
||||
|
||||
**Post Processing Tab:**
|
||||
- Tonemapping settings.
|
||||
- Bloom controls.
|
||||
- FXAA parameters.
|
||||
- SSR configuration.
|
||||
|
||||
### Draw Callback Order
|
||||
|
||||
Callbacks are invoked in registration order during `begin_frame()`:
|
||||
|
||||
```cpp
|
||||
void ImGuiSystem::begin_frame()
|
||||
{
|
||||
ImGui_ImplVulkan_NewFrame();
|
||||
ImGui_ImplSDL2_NewFrame();
|
||||
ImGui::NewFrame();
|
||||
|
||||
// Invoke all registered callbacks
|
||||
for (auto& cb : _draw_callbacks)
|
||||
{
|
||||
if (cb) cb();
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Register order-dependent callbacks carefully:
|
||||
```cpp
|
||||
// Engine debug UI first
|
||||
imgui.add_draw_callback([]{ draw_engine_ui(); });
|
||||
|
||||
// Game UI on top
|
||||
imgui.add_draw_callback([]{ draw_game_ui(); });
|
||||
|
||||
// Editor overlays last
|
||||
imgui.add_draw_callback([]{ draw_editor_overlays(); });
|
||||
```
|
||||
|
||||
### ImGuizmo Integration
|
||||
|
||||
The engine integrates ImGuizmo for 3D gizmo manipulation:
|
||||
|
||||
```cpp
|
||||
#include "ImGuizmo.h"
|
||||
|
||||
void draw_object_gizmo(const glm::mat4& view, const glm::mat4& proj,
|
||||
glm::mat4& object_transform)
|
||||
{
|
||||
ImGuizmo::SetOrthographic(false);
|
||||
ImGuizmo::SetDrawlist();
|
||||
|
||||
ImGuiIO& io = ImGui::GetIO();
|
||||
ImGuizmo::SetRect(0, 0, io.DisplaySize.x, io.DisplaySize.y);
|
||||
|
||||
static ImGuizmo::OPERATION op = ImGuizmo::TRANSLATE;
|
||||
static ImGuizmo::MODE mode = ImGuizmo::WORLD;
|
||||
|
||||
ImGuizmo::Manipulate(
|
||||
glm::value_ptr(view),
|
||||
glm::value_ptr(proj),
|
||||
op,
|
||||
mode,
|
||||
glm::value_ptr(object_transform));
|
||||
}
|
||||
```
|
||||
|
||||
### Tips
|
||||
|
||||
- Always check `want_capture_mouse()` before processing game mouse input.
|
||||
- Use `want_capture_keyboard()` before processing game keyboard input.
|
||||
- Register draw callbacks during initialization, not every frame.
|
||||
- Call `on_swapchain_recreated()` after window resize/mode change.
|
||||
- The descriptor pool is sized for 1000 sets of each type — sufficient for most debug UIs.
|
||||
- For production games, consider conditionally compiling out debug UI.
|
||||
- ImGui windows are persistent between frames — state is preserved automatically.
|
||||
|
||||
### Frame Flow
|
||||
|
||||
1. **Event Processing**: `process_event()` for each SDL event.
|
||||
2. **Begin Frame**: `begin_frame()` starts new ImGui frame and invokes callbacks.
|
||||
3. **UI Building**: All `ImGui::*` calls happen inside draw callbacks.
|
||||
4. **End Frame**: `end_frame()` calls `ImGui::Render()` to finalize draw data.
|
||||
5. **RenderGraph**: `ImGuiPass` executes, recording draw commands to GPU.
|
||||
6. **Present**: Swapchain presents the final image with ImGui overlay.
|
||||
292
docs/InputSystem.md
Normal file
292
docs/InputSystem.md
Normal file
@@ -0,0 +1,292 @@
|
||||
## Input System: Cross-Platform Input Handling
|
||||
|
||||
Unified input abstraction layer that wraps SDL2 events into a clean, game-friendly API. Provides both polled state queries and event-based access.
|
||||
|
||||
### Components
|
||||
|
||||
- `InputSystem` (src/core/input/input_system.h/.cpp)
|
||||
- Main entry point for all input handling.
|
||||
- Pumps SDL2 events, tracks window state (quit, minimize, resize).
|
||||
- Maintains per-frame `InputState` and `InputEvent` list.
|
||||
|
||||
- `InputState`
|
||||
- Polled snapshot of keyboard and mouse state.
|
||||
- Distinguishes between "down" (held), "pressed" (just pressed this frame), and "released" (just released this frame).
|
||||
|
||||
- `InputEvent`
|
||||
- Discrete input event with timestamp and modifiers.
|
||||
- Types: `KeyDown`, `KeyUp`, `MouseButtonDown`, `MouseButtonUp`, `MouseMove`, `MouseWheel`.
|
||||
|
||||
### Key Codes
|
||||
|
||||
Cross-platform key codes based on USB HID usage IDs (compatible with SDL scancodes):
|
||||
|
||||
```cpp
|
||||
enum class Key : uint16_t
|
||||
{
|
||||
Unknown = 0,
|
||||
|
||||
// Letters (A-Z): 4-29
|
||||
A = 4, B = 5, C = 6, D = 7, E = 8, F = 9, G = 10, H = 11, I = 12,
|
||||
J = 13, K = 14, L = 15, M = 16, N = 17, O = 18, P = 19, Q = 20,
|
||||
R = 21, S = 22, T = 23, U = 24, V = 25, W = 26, X = 27, Y = 28, Z = 29,
|
||||
|
||||
// Numbers (0-9): 30-39
|
||||
Num1 = 30, Num2 = 31, Num3 = 32, Num4 = 33, Num5 = 34,
|
||||
Num6 = 35, Num7 = 36, Num8 = 37, Num9 = 38, Num0 = 39,
|
||||
|
||||
// Special keys
|
||||
Enter = 40,
|
||||
Escape = 41,
|
||||
Backspace = 42,
|
||||
Tab = 43,
|
||||
Space = 44,
|
||||
|
||||
// Modifiers
|
||||
LeftCtrl = 224,
|
||||
LeftShift = 225,
|
||||
LeftAlt = 226,
|
||||
LeftSuper = 227,
|
||||
RightCtrl = 228,
|
||||
RightShift = 229,
|
||||
RightAlt = 230,
|
||||
RightSuper = 231,
|
||||
};
|
||||
```
|
||||
|
||||
### Mouse Buttons
|
||||
|
||||
```cpp
|
||||
enum class MouseButton : uint8_t
|
||||
{
|
||||
Left = 0,
|
||||
Middle = 1,
|
||||
Right = 2,
|
||||
X1 = 3, // Extra button 1
|
||||
X2 = 4, // Extra button 2
|
||||
};
|
||||
```
|
||||
|
||||
### Cursor Modes
|
||||
|
||||
```cpp
|
||||
enum class CursorMode : uint8_t
|
||||
{
|
||||
Normal = 0, // Default cursor, visible
|
||||
Hidden = 1, // Cursor hidden but not captured
|
||||
Relative = 2, // FPS-style: cursor hidden, motion is relative delta only
|
||||
};
|
||||
```
|
||||
|
||||
### InputState API
|
||||
|
||||
Polled state for keyboard and mouse. Updated each frame before game logic runs.
|
||||
|
||||
**Keyboard:**
|
||||
|
||||
```cpp
|
||||
bool key_down(Key key) const; // True if key is currently held
|
||||
bool key_pressed(Key key) const; // True only on the frame key was pressed
|
||||
bool key_released(Key key) const; // True only on the frame key was released
|
||||
```
|
||||
|
||||
**Mouse:**
|
||||
|
||||
```cpp
|
||||
bool mouse_down(MouseButton button) const; // True if button is held
|
||||
bool mouse_pressed(MouseButton button) const; // True only on frame pressed
|
||||
bool mouse_released(MouseButton button) const; // True only on frame released
|
||||
|
||||
glm::vec2 mouse_position() const; // Current cursor position (pixels)
|
||||
glm::vec2 mouse_delta() const; // Motion delta this frame (pixels)
|
||||
glm::vec2 wheel_delta() const; // Scroll wheel delta this frame
|
||||
```
|
||||
|
||||
**Modifiers:**
|
||||
|
||||
```cpp
|
||||
InputModifiers modifiers() const;
|
||||
|
||||
struct InputModifiers
|
||||
{
|
||||
bool shift = false;
|
||||
bool ctrl = false;
|
||||
bool alt = false;
|
||||
bool super = false; // Windows key / Command key
|
||||
};
|
||||
```
|
||||
|
||||
### InputEvent Structure
|
||||
|
||||
For event-driven input handling:
|
||||
|
||||
```cpp
|
||||
struct InputEvent
|
||||
{
|
||||
enum class Type : uint8_t
|
||||
{
|
||||
KeyDown,
|
||||
KeyUp,
|
||||
MouseButtonDown,
|
||||
MouseButtonUp,
|
||||
MouseMove,
|
||||
MouseWheel,
|
||||
};
|
||||
|
||||
Type type;
|
||||
uint32_t timestamp_ms; // SDL timestamp
|
||||
InputModifiers mods; // Active modifiers
|
||||
|
||||
Key key; // Valid for KeyDown/KeyUp
|
||||
MouseButton mouse_button; // Valid for MouseButtonDown/Up
|
||||
|
||||
glm::vec2 mouse_pos; // Valid for mouse events
|
||||
glm::vec2 mouse_delta; // Valid for MouseMove
|
||||
glm::vec2 wheel_delta; // Valid for MouseWheel
|
||||
};
|
||||
```
|
||||
|
||||
### InputSystem API
|
||||
|
||||
**Frame Lifecycle:**
|
||||
|
||||
```cpp
|
||||
void begin_frame(); // Clear per-frame state (pressed/released, deltas)
|
||||
void pump_events(); // Poll and process all pending SDL events
|
||||
```
|
||||
|
||||
**State Access:**
|
||||
|
||||
```cpp
|
||||
const InputState& state() const; // Get current polled state
|
||||
std::span<const InputEvent> events() const; // Get all events this frame
|
||||
```
|
||||
|
||||
**Window State:**
|
||||
|
||||
```cpp
|
||||
bool quit_requested() const; // True if window close requested
|
||||
bool window_minimized() const; // True if window is minimized
|
||||
|
||||
bool resize_requested() const; // True if resize/move occurred
|
||||
uint32_t last_resize_event_ms() const;
|
||||
void clear_resize_request(); // Clear after handling resize
|
||||
```
|
||||
|
||||
**Cursor Control:**
|
||||
|
||||
```cpp
|
||||
CursorMode cursor_mode() const;
|
||||
void set_cursor_mode(CursorMode mode);
|
||||
```
|
||||
|
||||
**Native Event Access (Engine Internal):**
|
||||
|
||||
```cpp
|
||||
// For ImGui or other systems that need raw SDL events
|
||||
void for_each_native_event(NativeEventCallback callback, void* user) const;
|
||||
```
|
||||
|
||||
### Usage Example
|
||||
|
||||
**Polled State (Most Common):**
|
||||
|
||||
```cpp
|
||||
void Game::update(InputSystem& input)
|
||||
{
|
||||
const InputState& state = input.state();
|
||||
|
||||
// Movement
|
||||
glm::vec3 move{0.0f};
|
||||
if (state.key_down(Key::W)) move.z -= 1.0f;
|
||||
if (state.key_down(Key::S)) move.z += 1.0f;
|
||||
if (state.key_down(Key::A)) move.x -= 1.0f;
|
||||
if (state.key_down(Key::D)) move.x += 1.0f;
|
||||
|
||||
// Sprint (shift held)
|
||||
float speed = 5.0f;
|
||||
if (state.modifiers().shift) speed = 10.0f;
|
||||
|
||||
// Camera look (relative mode)
|
||||
if (state.mouse_down(MouseButton::Right))
|
||||
{
|
||||
input.set_cursor_mode(CursorMode::Relative);
|
||||
glm::vec2 delta = state.mouse_delta();
|
||||
camera.rotate(delta.x * 0.1f, delta.y * 0.1f);
|
||||
}
|
||||
else
|
||||
{
|
||||
input.set_cursor_mode(CursorMode::Normal);
|
||||
}
|
||||
|
||||
// Fire on click
|
||||
if (state.mouse_pressed(MouseButton::Left))
|
||||
{
|
||||
player.fire();
|
||||
}
|
||||
|
||||
// Toggle inventory on I press
|
||||
if (state.key_pressed(Key::I))
|
||||
{
|
||||
ui.toggle_inventory();
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Event-Driven (For UI / Text Input):**
|
||||
|
||||
```cpp
|
||||
void TextBox::process_input(InputSystem& input)
|
||||
{
|
||||
for (const InputEvent& ev : input.events())
|
||||
{
|
||||
if (ev.type == InputEvent::Type::KeyDown)
|
||||
{
|
||||
if (ev.key == Key::Backspace && !text.empty())
|
||||
{
|
||||
text.pop_back();
|
||||
}
|
||||
else if (ev.key == Key::Enter)
|
||||
{
|
||||
submit();
|
||||
}
|
||||
}
|
||||
else if (ev.type == InputEvent::Type::MouseWheel)
|
||||
{
|
||||
scroll_offset -= ev.wheel_delta.y * 20.0f;
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Frame Flow
|
||||
|
||||
1. Engine calls `InputSystem::begin_frame()` at frame start.
|
||||
2. Engine calls `InputSystem::pump_events()` to process SDL events.
|
||||
3. Game code queries `state()` for polled input or iterates `events()` for event-driven logic.
|
||||
4. Render loop proceeds.
|
||||
|
||||
### Integration with ImGui
|
||||
|
||||
The engine provides native event access for ImGui integration:
|
||||
|
||||
```cpp
|
||||
input.for_each_native_event(
|
||||
[](void* user, InputSystem::NativeEventView event)
|
||||
{
|
||||
if (event.backend == InputSystem::NativeBackend::SDL2)
|
||||
{
|
||||
ImGui_ImplSDL2_ProcessEvent(
|
||||
static_cast<const SDL_Event*>(event.data));
|
||||
}
|
||||
},
|
||||
nullptr);
|
||||
```
|
||||
|
||||
### Tips
|
||||
|
||||
- Prefer polled state (`key_down`, `mouse_down`) for continuous actions like movement.
|
||||
- Use pressed/released (`key_pressed`, `mouse_pressed`) for one-shot actions like firing or toggling.
|
||||
- Use `CursorMode::Relative` for FPS-style camera control; mouse position becomes meaningless, only delta matters.
|
||||
- Check `quit_requested()` each frame to handle window close gracefully.
|
||||
- The `resize_requested()` flag includes window move events to trigger swapchain recreation on multi-monitor setups.
|
||||
301
docs/Picking.md
Normal file
301
docs/Picking.md
Normal file
@@ -0,0 +1,301 @@
|
||||
## Picking System: Object Selection and Interaction
|
||||
|
||||
Unified picking system that handles single-click selection, hover detection, and drag-box multi-select. Supports both CPU ray-casting (via BVH) and GPU ID-buffer picking.
|
||||
|
||||
### Components
|
||||
|
||||
- `PickingSystem` (src/core/picking/picking_system.h/.cpp)
|
||||
- Main entry point for all picking operations.
|
||||
- Processes SDL mouse events (click, drag, release).
|
||||
- Maintains per-frame hover picks and last click selection.
|
||||
- Integrates with RenderGraph for async ID-buffer readback.
|
||||
|
||||
- `SceneManager` picking helpers (src/scene/vk_scene_picking.cpp)
|
||||
- CPU ray-casting against `RenderObject` bounds.
|
||||
- BVH-accelerated mesh picking for precise triangle-level hits.
|
||||
- Rectangle selection in NDC space.
|
||||
|
||||
### PickInfo Structure
|
||||
|
||||
Result of any pick operation:
|
||||
|
||||
```cpp
|
||||
struct PickInfo
|
||||
{
|
||||
MeshAsset *mesh = nullptr; // Source mesh asset
|
||||
LoadedGLTF *scene = nullptr; // Source glTF scene
|
||||
Node *node = nullptr; // glTF node that owns this surface
|
||||
RenderObject::OwnerType ownerType; // StaticGLTF, GLTFInstance, MeshInstance
|
||||
std::string ownerName; // Logical name (e.g., "player", "cube01")
|
||||
WorldVec3 worldPos; // Hit position in world space (double-precision)
|
||||
glm::mat4 worldTransform; // Object's world transform
|
||||
uint32_t indexCount; // Index count of picked surface
|
||||
uint32_t firstIndex; // First index of picked surface
|
||||
uint32_t surfaceIndex; // Surface index within mesh
|
||||
bool valid = false; // True if pick hit something
|
||||
};
|
||||
```
|
||||
|
||||
### Picking Modes
|
||||
|
||||
#### 1. CPU Ray Picking (Default)
|
||||
|
||||
Uses camera ray against object bounds with BVH acceleration:
|
||||
|
||||
- **Sphere bounds**: Quick bounding sphere test.
|
||||
- **Box bounds**: Ray-box intersection in local space.
|
||||
- **Capsule bounds**: Combined cylinder + sphere caps intersection.
|
||||
- **Mesh bounds**: Full BVH traversal for triangle-level precision.
|
||||
|
||||
Advantages:
|
||||
- Immediate results (same frame).
|
||||
- No GPU readback latency.
|
||||
- Precise triangle-level hits with mesh BVH.
|
||||
|
||||
#### 2. ID-Buffer Picking
|
||||
|
||||
Renders object IDs to a GPU buffer and reads back single pixel:
|
||||
|
||||
- Each `RenderObject` has a unique `objectID` assigned during draw.
|
||||
- Geometry pass writes IDs to an R32_UINT attachment.
|
||||
- `PickReadback` render pass copies single pixel to CPU-readable buffer.
|
||||
- Result resolved on next frame after GPU fence wait.
|
||||
|
||||
Advantages:
|
||||
- Pixel-perfect accuracy (no false positives from bounds).
|
||||
- Handles complex overlapping geometry.
|
||||
- Works with any object shape.
|
||||
|
||||
Enable via:
|
||||
```cpp
|
||||
picking_system.set_use_id_buffer_picking(true);
|
||||
```
|
||||
|
||||
### PickingSystem API
|
||||
|
||||
**Initialization:**
|
||||
|
||||
```cpp
|
||||
void init(EngineContext *context);
|
||||
void cleanup();
|
||||
```
|
||||
|
||||
**Frame Lifecycle:**
|
||||
|
||||
```cpp
|
||||
void begin_frame(); // Resolve pending async picks after fence wait
|
||||
void process_event(const SDL_Event &event, bool ui_want_capture_mouse);
|
||||
void update_hover(); // Update hover pick each frame
|
||||
```
|
||||
|
||||
**Results Access:**
|
||||
|
||||
```cpp
|
||||
const PickInfo& last_pick() const; // Last click selection
|
||||
const PickInfo& hover_pick() const; // Current hover (under cursor)
|
||||
const std::vector<PickInfo>& drag_selection() const; // Multi-select results
|
||||
uint32_t last_pick_object_id() const; // Raw object ID of last pick
|
||||
```
|
||||
|
||||
**Configuration:**
|
||||
|
||||
```cpp
|
||||
bool use_id_buffer_picking() const;
|
||||
void set_use_id_buffer_picking(bool enabled);
|
||||
|
||||
bool debug_draw_bvh() const;
|
||||
void set_debug_draw_bvh(bool enabled);
|
||||
```
|
||||
|
||||
**Utilities:**
|
||||
|
||||
```cpp
|
||||
// Clear picks for removed objects (call when deleting instances)
|
||||
void clear_owner_picks(RenderObject::OwnerType owner_type, const std::string &owner_name);
|
||||
|
||||
// Mutable access for engine integration
|
||||
PickInfo* mutable_last_pick();
|
||||
PickInfo* mutable_hover_pick();
|
||||
```
|
||||
|
||||
**RenderGraph Integration:**
|
||||
|
||||
```cpp
|
||||
// Called during RenderGraph build when ID-buffer picking is enabled
|
||||
void register_id_buffer_readback(RenderGraph &graph,
|
||||
RGImageHandle id_buffer,
|
||||
VkExtent2D draw_extent,
|
||||
VkExtent2D swapchain_extent);
|
||||
```
|
||||
|
||||
### SceneManager Picking API
|
||||
|
||||
Lower-level picking functions for custom use:
|
||||
|
||||
```cpp
|
||||
// Single-object ray pick
|
||||
bool pick(const glm::vec2 &mousePosPixels, RenderObject &outObject, WorldVec3 &outWorldPos);
|
||||
|
||||
// Resolve ID from ID-buffer back to RenderObject
|
||||
bool resolveObjectID(uint32_t id, RenderObject &outObject) const;
|
||||
|
||||
// Rectangle selection (drag-box)
|
||||
void selectRect(const glm::vec2 &p0, const glm::vec2 &p1,
|
||||
std::vector<RenderObject> &outObjects) const;
|
||||
```
|
||||
|
||||
### Usage Examples
|
||||
|
||||
**Basic Click Selection:**
|
||||
|
||||
```cpp
|
||||
void Game::handle_selection(PickingSystem& picking)
|
||||
{
|
||||
const PickingSystem::PickInfo& pick = picking.last_pick();
|
||||
|
||||
if (pick.valid)
|
||||
{
|
||||
fmt::println("Selected: {} at ({}, {}, {})",
|
||||
pick.ownerName,
|
||||
pick.worldPos.x, pick.worldPos.y, pick.worldPos.z);
|
||||
|
||||
// Handle different owner types
|
||||
switch (pick.ownerType)
|
||||
{
|
||||
case RenderObject::OwnerType::GLTFInstance:
|
||||
select_actor(pick.ownerName);
|
||||
break;
|
||||
case RenderObject::OwnerType::MeshInstance:
|
||||
select_prop(pick.ownerName);
|
||||
break;
|
||||
default:
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Hover Tooltips:**
|
||||
|
||||
```cpp
|
||||
void Game::update_ui(PickingSystem& picking)
|
||||
{
|
||||
const PickingSystem::PickInfo& hover = picking.hover_pick();
|
||||
|
||||
if (hover.valid)
|
||||
{
|
||||
show_tooltip(hover.ownerName);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Multi-Select with Drag Box:**
|
||||
|
||||
```cpp
|
||||
void Game::process_drag_selection(PickingSystem& picking)
|
||||
{
|
||||
const auto& selection = picking.drag_selection();
|
||||
|
||||
for (const PickingSystem::PickInfo& pick : selection)
|
||||
{
|
||||
if (pick.valid)
|
||||
{
|
||||
add_to_selection(pick.ownerName);
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Custom Ray Pick:**
|
||||
|
||||
```cpp
|
||||
void Game::custom_pick(SceneManager& scene, const glm::vec2& screen_pos)
|
||||
{
|
||||
RenderObject hit_object{};
|
||||
WorldVec3 hit_pos{};
|
||||
|
||||
if (scene.pick(screen_pos, hit_object, hit_pos))
|
||||
{
|
||||
// hit_object contains the picked render object
|
||||
// hit_pos is the precise world-space hit position
|
||||
spawn_effect_at(hit_pos);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Bounds Types
|
||||
|
||||
Objects can use different bounds types for picking:
|
||||
|
||||
```cpp
|
||||
enum class BoundsType : uint8_t
|
||||
{
|
||||
None = 0, // Not pickable
|
||||
Box = 1, // AABB (default for most objects)
|
||||
Sphere = 2, // Bounding sphere
|
||||
Capsule = 3, // Cylinder + hemisphere caps
|
||||
Mesh = 4, // Full BVH mesh intersection
|
||||
};
|
||||
```
|
||||
|
||||
Set bounds type when adding instances:
|
||||
```cpp
|
||||
scene->addMeshInstance("capsule_enemy", mesh, transform, BoundsType::Capsule);
|
||||
```
|
||||
|
||||
### Mesh BVH Picking
|
||||
|
||||
For precise triangle-level picking on complex meshes:
|
||||
|
||||
1. Mesh assets automatically build a BVH during loading.
|
||||
2. `BoundsType::Mesh` triggers BVH traversal instead of simple bounds test.
|
||||
3. Returns exact triangle hit position, not just bounding box intersection.
|
||||
4. `PickInfo::firstIndex` and `indexCount` are refined to the exact triangle.
|
||||
|
||||
Debug BVH visualization:
|
||||
```cpp
|
||||
picking.set_debug_draw_bvh(true); // Shows BVH nodes in debug overlay
|
||||
```
|
||||
|
||||
### Coordinate Space Handling
|
||||
|
||||
The picking system handles multiple coordinate transformations:
|
||||
|
||||
1. **Window pixels** (SDL event coordinates, top-left origin)
|
||||
2. **Swapchain pixels** (scaled for HiDPI displays)
|
||||
3. **Logical render pixels** (internal render resolution)
|
||||
4. **NDC** (normalized device coordinates, -1 to 1)
|
||||
5. **World space** (double-precision `WorldVec3`)
|
||||
|
||||
The `window_to_swapchain_pixels()` helper handles HiDPI scaling. Letterboxing is accounted for when render resolution differs from swapchain size.
|
||||
|
||||
### Frame Flow
|
||||
|
||||
1. `begin_frame()` — Resolve pending async ID-buffer picks from previous frame.
|
||||
2. `process_event()` — Handle mouse events (click start/end, motion).
|
||||
3. `update_hover()` — CPU ray-cast for current hover under cursor.
|
||||
4. RenderGraph build — If ID-buffer picking enabled, register readback pass.
|
||||
5. Next frame — Async pick result becomes available.
|
||||
|
||||
### Integration with ImGui
|
||||
|
||||
The picking system respects ImGui's input capture:
|
||||
|
||||
```cpp
|
||||
// In event loop
|
||||
bool ui_capture = imgui_system.want_capture_mouse();
|
||||
picking_system.process_event(event, ui_capture);
|
||||
```
|
||||
|
||||
When `ui_want_capture_mouse` is true:
|
||||
- Click events are ignored (no picks started).
|
||||
- Mouse motion still updates cursor position for future picks.
|
||||
- Hover picking is not affected.
|
||||
|
||||
### Tips
|
||||
|
||||
- Use CPU ray picking (`set_use_id_buffer_picking(false)`) for immediate feedback.
|
||||
- Use ID-buffer picking for pixel-perfect selection in dense scenes.
|
||||
- Call `clear_owner_picks()` when removing instances to avoid stale picks.
|
||||
- For mesh BVH to work, ensure the mesh was loaded with BVH generation enabled.
|
||||
- The drag selection threshold is 3 pixels — smaller motions are treated as clicks.
|
||||
Reference in New Issue
Block a user