Image-Based Lighting (IBL) Overview - IBL assets (environment maps + BRDF LUT + SH coefficients) are managed by `IBLManager` (`src/core/ibl_manager.{h,cpp}`) and exposed to passes via `EngineContext::ibl`. - Shaders share a common include, `shaders/ibl_common.glsl`, which defines the IBL bindings for descriptor set 3 and helper functions used by deferred, forward, and background passes. - The engine currently supports: - Specular environment from an equirectangular 2D texture with prefiltered mips (`sampler2D iblSpec2D`). - Diffuse irradiance from 2nd‑order SH (9 coefficients baked on the CPU). - A 2D BRDF integration LUT used for the split‑sum approximation. Data Flow - Init: - `VulkanEngine::init_vulkan()` creates an `IBLManager`, calls `init(context)`, and publishes it via `EngineContext::ibl`. - The engine optionally loads default IBL assets (`IBLPaths` in `src/core/vk_engine.cpp`), typically a BRDF LUT plus a specular environment `.ktx2`. - Loading (IBLManager): - `IBLManager::load(const IBLPaths&)`: - Specular: - Tries `ktxutil::load_ktx2_cubemap` first. If successful, uploads via `ResourceManager::create_image_compressed_layers` with `VK_IMAGE_CREATE_CUBE_COMPATIBLE_BIT`. - If cubemap loading fails, falls back to 2D `.ktx2` via `ktxutil::load_ktx2_2d` and `create_image_compressed`. The image is treated as equirectangular with prefiltered mips. - When the specular `.ktx2` is HDR (`R16G16B16A16_SFLOAT` or `R32G32B32A32_SFLOAT`) and 2:1 aspect, `IBLManager` computes 9 SH coefficients on the CPU: - Integrates the environment over the sphere using real SH basis functions (L2) with solid‑angle weighting. - Applies Lambert band scaling (A0 = π, A1 = 2π/3, A2 = π/4). - Uploads the result as `vec4 sh[9]` in a uniform buffer (`_shBuffer`). - Diffuse: - If `IBLPaths::diffuseCube` is provided and valid, loads it as a cubemap via `load_ktx2_cubemap` + `create_image_compressed_layers`. - Current shaders only use the SH buffer for diffuse; the diffuse cubemap is reserved for future variants. - BRDF LUT: - Loaded as 2D `.ktx2` via `ktxutil::load_ktx2_2d` and uploaded with `create_image_compressed`. - Fallbacks: - If `diffuseCube` is missing but a specular env exists, `_diff` is aliased to `_spec`. - `IBLManager::unload()` releases GPU images, the SH buffer, and the descriptor set layout. - Descriptor layout: - `IBLManager::ensureLayout()` builds a descriptor set layout (set=3) with: - binding 0: `COMBINED_IMAGE_SAMPLER` — specular environment (2D equirect). - binding 1: `COMBINED_IMAGE_SAMPLER` — BRDF LUT 2D. - binding 2: `UNIFORM_BUFFER` — SH coefficients (`vec4 sh[9]`). - Passes request this layout from `EngineContext::ibl` and plug it into their pipeline set layouts: - Background: `vk_renderpass_background.cpp` (set 3 used for env background). - Lighting: `vk_renderpass_lighting.cpp` (deferred lighting pass, set 3). - Transparent: `vk_renderpass_transparent.cpp` (forward/transparent materials, set 3). Shader Side (`shaders/ibl_common.glsl`) - Bindings: - `layout(set=3, binding=0) uniform sampler2D iblSpec2D;` - `layout(set=3, binding=1) uniform sampler2D iblBRDF;` - `layout(std140, set=3, binding=2) uniform IBL_SH { vec4 sh[9]; } iblSH;` - Helpers: - `vec3 sh_eval_irradiance(vec3 n)`: - Evaluates the 9 SH basis functions (L2) at direction `n` using the same real SH basis as the CPU bake. - Multiplies each basis value by the corresponding `iblSH.sh[i].rgb` coefficient and sums the result. - Coefficients are already convolved with the Lambert kernel on the CPU; the function returns diffuse irradiance directly. - `vec2 dir_to_equirect(vec3 d)`: - Normalizes `d`, computes `(phi, theta)` and returns equirectangular UV in `[0,1]²`. - Used consistently by background, deferred, and forward pipelines. - `float ibl_lod_from_roughness(float roughness, float levels)`: - Computes the mip LOD for specular IBL using `roughness² * (levels - 1)`. - This biases mid‑roughness reflections towards blurrier mips and avoids overly sharp reflections. Usage in Passes - Deferred lighting (`shaders/deferred_lighting.frag` and `shaders/deferred_lighting_nort.frag`): - Include: - `#include "input_structures.glsl"` - `#include "ibl_common.glsl"` - IBL contribution (per pixel): - Specular: - `vec3 R = reflect(-V, N);` - `float levels = float(textureQueryLevels(iblSpec2D));` - `float lod = ibl_lod_from_roughness(roughness, levels);` - `vec2 uv = dir_to_equirect(R);` - `vec3 prefiltered = textureLod(iblSpec2D, uv, lod).rgb;` - `vec2 brdf = texture(iblBRDF, vec2(max(dot(N,V),0.0), roughness)).rg;` - `vec3 specIBL = prefiltered * (F0 * brdf.x + brdf.y);` - Diffuse: - `vec3 diffIBL = (1.0 - metallic) * albedo * sh_eval_irradiance(N);` - Combined: - `color += diffIBL + specIBL;` - Forward/transparent (`shaders/mesh.frag`): - Same include and IBL logic as deferred, applied after direct lighting. - Uses the same `ibl_lod_from_roughness` helper for LOD selection. - Background (`shaders/background_env.frag`): - Includes `ibl_common.glsl` and uses `dir_to_equirect(worldDir)` + `textureLod(iblSpec2D, uv, 0.0)` to render the environment at LOD 0. Authoring IBL Assets - Specular environment: - Preferred: prefiltered HDR cubemap in `.ktx2` (BC6H or `R16G16B16A16_SFLOAT`) with multiple mips. - Alternative: prefiltered equirectangular 2D `.ktx2` with width = 2 × height and full mip chain. - Make sure the mip chain is generated with a GGX importance sampling tool so the BRDF LUT + mip chain match. - BRDF LUT: - A standard 2D preintegrated GGX LUT (RG), usually stored as `R8G8_UNORM` or BC5. - The LUT is sampled with `(NoV, roughness)` coordinates. - Diffuse: - The engine currently uses SH coefficients baked from the specular equirectangular map. If you provide a separate diffuse cubemap, the CPU SH bake still uses the specular HDR; you can adjust this in `IBLManager` if you want SH to come from a different source. Implementation Notes - CPU SH bake: - Implemented in `IBLManager::load` using libktx to access raw HDR pixel data from `.ktx2`. - Uses a simple nested loop over pixels with solid‑angle weighting and the same SH basis as `sh_eval_irradiance`. - Fallbacks: - Lighting and transparent passes create small fallback textures so that the IBL descriptor set is always valid, even when no IBL assets are loaded. - Background pass builds a 1×1×6 black cube as a fallback env.