Support for VK_EXT_frame_boundary and usage in offscreen swapchain (#…

…1354) * Support for VK_EXT_frame_boundary Add support for the frame boundary extension VK_EXT_frame_boundary. This includes: - `VkFrameBoundaryEXT` structures found in the pNext chains of `vkSubmitInfo` and `vkSubmitInfo2` are now interpreted as frame delimiters at capture time if `flags` member has the `VK_FRAME_BOUNDARY_FRAME_END_BIT_EXT` bit set. - The same structures in the same calls support screenshots at replay time by interpreting `pImages` as the rendered frames. - `VK_EXT_frame_boundary` is now listed as a supported layer device extension (programs that use the layer have access to the extension even if the driver does not support it) Extended support for this extension that is NOT implemented by this commit could include: - Interpretation of `VkFrameBoundaryEXT` structure when found in `VkPresentInfoKHR` and `VkBindSparseInfo` (eg. by considering the *real* render frames to be the ones specified by the structures instead of the one presented) - Support for `VkFrameBoundaryEXT` even when the `VK_FRAME_BOUNDARY_FRAME_END_BIT_EXT` bit is not set (eg. by saving the resources associated in a map/vector to be taken into account when the real frame end is encountered) * Add `--offscreen-swapchain-frame-boundary` option This option inserts a `VkFrameBoundaryEXT` from `VK_EXT_frame_boundary` into a command buffer submission called where `vkQueuePresentKHR` should have been called by an offscreen swapchain. By doing this, it is now possible to capture a trace replayed with the option `--swapchain offscreen` AND to still have frame numbering and the possibility to take screenshots. This is very important for automated pipelines that run on a server without WSI context and still need to take screenshots, or do fast forwarding... --------- Co-authored-by: Charles Giessen <[email protected]>
LunarG · Mar 6, 2024 · 9847057 · 9847057
1 parent 93c5a3d
commit 9847057
Show file tree

Hide file tree

Showing 15 changed files with 308 additions and 22 deletions.
diff --git a/USAGE_android.md b/USAGE_android.md
@@ -867,6 +867,15 @@ optional arguments:
                         Swap the swapchain color space if unsupported by replay device.
                         Check if color space is not supported by replay device and swap
                         to VK_COLOR_SPACE_SRGB_NONLINEAR_KHR. (forwarded to replay tool).
+  --offscreen-swapchain-frame-boundary
+                        Should only be used with offscreen swapchain. Activates
+                        the extension VK_EXT_frame_boundary (always supported
+                        if trimming, checks for driver support otherwise) and
+                        inserts command buffer submission with
+                        VkFrameBoundaryEXT where vkQueuePresentKHR was called
+                        in the original capture. This allows preserving frames
+                        when capturing a replay that uses. offscreen swapchain.
+                        (forwarded to replay tool)
 ```
 
 The command will force-stop an active replay process before starting the replay

diff --git a/USAGE_desktop_Vulkan.md b/USAGE_desktop_Vulkan.md
@@ -541,6 +541,14 @@ Optional arguments:
               Swap the swapchain color space if unsupported by replay device.
               Check if color space is not supported by replay device and
               fallback to VK_COLOR_SPACE_SRGB_NONLINEAR_KHR.
+  --offscreen-swapchain-frame-boundary
+              Should only be used with offscreen swapchain.
+              Activates the extension VK_EXT_frame_boundary (always supported if
+              trimming, checks for driver support otherwise) and inserts command
+              buffer submission with VkFrameBoundaryEXT where vkQueuePresentKHR
+              was called in the original capture.
+              This allows preserving frames when capturing a replay that uses.
+              offscreen swapchain.
 ```
 
 ### Key Controls

diff --git a/android/scripts/gfxrecon.py b/android/scripts/gfxrecon.py
@@ -85,6 +85,7 @@ def CreateReplayParser():
     parser.add_argument('--validate', action='store_true', default=False, help='Enables the Khronos Vulkan validation layer (forwarded to replay tool)')
     parser.add_argument('--onhb', '--omit-null-hardware-buffers', action='store_true', default=False, help='Omit Vulkan calls that would pass a NULL AHardwareBuffer* (forwarded to replay tool)')
     parser.add_argument('--use-colorspace-fallback', action='store_true', default=False, help='Swap the swapchain color space if unsupported by replay device. Check if color space is not supported by replay device and swap to VK_COLOR_SPACE_SRGB_NONLINEAR_KHR. (forwarded to replay tool).')
+    parser.add_argument('--offscreen-swapchain-frame-boundary', action='store_true', default=False, help='Should only be used with offscreen swapchain. Activates the extension VK_EXT_frame_boundary (always supported if trimming, checks for driver support otherwise) and inserts command buffer submission with VkFrameBoundaryEXT where vkQueuePresentKHR was called in the original capture. This allows preserving frames when capturing a replay that uses. offscreen swapchain. (forwarded to replay tool)')
     parser.add_argument('--mfr', '--measurement-frame-range', metavar='START-END', help='Custom framerange to measure FPS for. This range will include the start frame but not the end frame. The measurement frame range defaults to all frames except the loading frame but can be configured for any range. If the end frame is past the last frame in the trace it will be clamped to the frame after the last (so in that case the results would include the last frame). (forwarded to replay tool)')
     parser.add_argument('--measurement-file', metavar='DEVICE_FILE', help='Write measurements to a file at the specified path. Default is: \'/sdcard/gfxrecon-measurements.json\' on android and \'./gfxrecon-measurements.json\' on desktop. (forwarded to replay tool)')
     parser.add_argument('--quit-after-measurement-range', action='store_true', default=False, help='If this is specified the replayer will abort when it reaches the <end_frame> specified in the --measurement-frame-range argument. (forwarded to replay tool)')
@@ -194,6 +195,9 @@ def MakeExtrasString(args):
     if args.swapchain:
         arg_list.append('--swapchain')
         arg_list.append('{}'.format(args.swapchain))
+
+    if args.offscreen_swapchain_frame_boundary:
+        arg_list.append('--offscreen-swapchain-frame-boundary')
 
     if args.memory_translation:
         arg_list.append('-m')

diff --git a/framework/decode/vulkan_offscreen_swapchain.cpp b/framework/decode/vulkan_offscreen_swapchain.cpp
@@ -33,13 +33,14 @@ VkResult VulkanOffscreenSwapchain::CreateSurface(VkResult
                                                  HandlePointerDecoder<VkSurfaceKHR>* surface,
                                                  const encode::InstanceTable*        instance_table,
                                                  application::Application*           application,
-                                                 int32_t                             options_surface_index)
+                                                 const VulkanReplayOptions&          replay_options)
 {
     GFXRECON_ASSERT(surface);
 
     instance_table_        = instance_table;
     application_           = application;
-    options_surface_index_ = options_surface_index;
+    options_surface_index_ = replay_options.surface_index;
+    insert_frame_boundary_ = replay_options.offscreen_swapchain_frame_boundary;
 
     // For multi-surface captures, when replay is restricted to a specific surface, only create a surface for
     // the specified index.
@@ -96,9 +97,72 @@ VkResult VulkanOffscreenSwapchain::CreateSwapchainKHR(VkResult
         return VK_ERROR_OUT_OF_HOST_MEMORY;
     }
 
-    VkDevice device = device = device_info->handle;
+    VkDevice device = device_info->handle;
     device_table_->GetDeviceQueue(device, default_queue_family_index_, 0, &default_queue_);
 
+    // If this option is set, a command buffer submission with a `VkFrameBoundaryEXT` must be called each time
+    // `vkQueuePresentKHR` should have been called by the offscreen swapchain. So a maximum of work must be done at
+    // swapchain creation: Allocation and recording of an empty command buffer, initialization of a `VkFrameBoundaryEXT`
+    // structure... (Don't forget to free everything at swapchain destruction)
+    if (insert_frame_boundary_)
+    {
+        VkResult result;
+        command_pools_.resize(device_info->queue_family_index_enabled.size());
+        command_buffers_.resize(device_info->queue_family_index_enabled.size());
+
+        for (uint32_t family_index = 0; family_index < device_info->queue_family_index_enabled.size(); family_index++)
+        {
+            if (!device_info->queue_family_index_enabled.at(family_index))
+            {
+                continue;
+            }
+
+            VkCommandPoolCreateInfo commandPoolCreateInfo;
+            commandPoolCreateInfo.sType            = VK_STRUCTURE_TYPE_COMMAND_POOL_CREATE_INFO;
+            commandPoolCreateInfo.pNext            = nullptr;
+            commandPoolCreateInfo.queueFamilyIndex = family_index;
+            commandPoolCreateInfo.flags            = 0;
+
+            result = device_table_->CreateCommandPool(
+                device, &commandPoolCreateInfo, nullptr, &command_pools_.at(family_index));
+            GFXRECON_ASSERT(result == VK_SUCCESS);
+
+            VkCommandBufferAllocateInfo commandBufferAllocateInfo;
+            commandBufferAllocateInfo.sType              = VK_STRUCTURE_TYPE_COMMAND_BUFFER_ALLOCATE_INFO;
+            commandBufferAllocateInfo.pNext              = nullptr;
+            commandBufferAllocateInfo.commandPool        = command_pools_.at(family_index);
+            commandBufferAllocateInfo.level              = VK_COMMAND_BUFFER_LEVEL_PRIMARY;
+            commandBufferAllocateInfo.commandBufferCount = 1;
+
+            result = device_table_->AllocateCommandBuffers(
+                device, &commandBufferAllocateInfo, &command_buffers_.at(family_index));
+            GFXRECON_ASSERT(result == VK_SUCCESS);
+
+            VkCommandBufferBeginInfo commandBufferBeginInfo;
+            commandBufferBeginInfo.sType            = VK_STRUCTURE_TYPE_COMMAND_BUFFER_BEGIN_INFO;
+            commandBufferBeginInfo.pNext            = nullptr;
+            commandBufferBeginInfo.flags            = 0;
+            commandBufferBeginInfo.pInheritanceInfo = nullptr;
+
+            result = device_table_->BeginCommandBuffer(command_buffers_.at(family_index), &commandBufferBeginInfo);
+            GFXRECON_ASSERT(result == VK_SUCCESS);
+
+            result = device_table_->EndCommandBuffer(command_buffers_.at(family_index));
+            GFXRECON_ASSERT(result == VK_SUCCESS);
+        }
+        frame_boundary_.sType       = VK_STRUCTURE_TYPE_FRAME_BOUNDARY_EXT;
+        frame_boundary_.pNext       = nullptr;
+        frame_boundary_.flags       = VK_FRAME_BOUNDARY_FRAME_END_BIT_EXT;
+        frame_boundary_.frameID     = 0;
+        frame_boundary_.imageCount  = 0;
+        frame_boundary_.pImages     = nullptr;
+        frame_boundary_.bufferCount = 0;
+        frame_boundary_.pBuffers    = nullptr;
+        frame_boundary_.tagName     = 0;
+        frame_boundary_.tagSize     = 0;
+        frame_boundary_.pTag        = nullptr;
+    }
+
     return original_result;
 }
 
@@ -111,6 +175,14 @@ void VulkanOffscreenSwapchain::DestroySwapchainKHR(PFN_vkDestroySwapchainKHR
     {
         CleanSwapchainResourceData(device_info, swapchain_info);
     }
+
+    if (insert_frame_boundary_ && command_pools_.size() > 0)
+    {
+        for (const auto& command_pool : command_pools_)
+        {
+            device_table_->DestroyCommandPool(device_info->handle, command_pool, nullptr);
+        }
+    }
 }
 
 VkResult VulkanOffscreenSwapchain::GetSwapchainImagesKHR(VkResult                    original_result,
@@ -222,7 +294,36 @@ VkResult VulkanOffscreenSwapchain::QueuePresentKHR(VkResult
                                                    const QueueInfo*                      queue_info,
                                                    const VkPresentInfoKHR*               present_info)
 {
-    if (present_info->waitSemaphoreCount > 0)
+    if (insert_frame_boundary_ && command_buffers_.size() > 0)
+    {
+        std::vector<VkImage> images(present_info->swapchainCount);
+        for (uint32_t i = 0; i < images.size(); ++i)
+        {
+            images[i] = swapchain_resources_[present_info->pSwapchains[i]]
+                            ->virtual_swapchain_images[present_info->pImageIndices[i]]
+                            .image;
+        }
+        frame_boundary_.imageCount = images.size();
+        frame_boundary_.pImages    = images.data();
+        ++frame_boundary_.frameID;
+
+        std::vector<VkPipelineStageFlags> dstStageFlags(present_info->waitSemaphoreCount,
+                                                        VK_PIPELINE_STAGE_ALL_COMMANDS_BIT);
+
+        VkSubmitInfo submitInfo;
+        submitInfo.sType                = VK_STRUCTURE_TYPE_SUBMIT_INFO;
+        submitInfo.pNext                = &frame_boundary_;
+        submitInfo.waitSemaphoreCount   = present_info->waitSemaphoreCount;
+        submitInfo.pWaitSemaphores      = present_info->pWaitSemaphores;
+        submitInfo.pWaitDstStageMask    = dstStageFlags.data();
+        submitInfo.commandBufferCount   = 1;
+        submitInfo.pCommandBuffers      = &command_buffers_.at(queue_info->family_index);
+        submitInfo.signalSemaphoreCount = 0;
+        submitInfo.pSignalSemaphores    = nullptr;
+
+        device_table_->QueueSubmit(queue_info->handle, 1, &submitInfo, VK_NULL_HANDLE);
+    }
+    else if (present_info->waitSemaphoreCount > 0)
     {
         VkResult result = SignalSemaphoresFence(
             queue_info, present_info->waitSemaphoreCount, present_info->pWaitSemaphores, 0, nullptr, VK_NULL_HANDLE);
@@ -235,6 +336,7 @@ VkResult VulkanOffscreenSwapchain::QueuePresentKHR(VkResult
             return result;
         }
     }
+
     return original_result;
 }
 

diff --git a/framework/decode/vulkan_offscreen_swapchain.h b/framework/decode/vulkan_offscreen_swapchain.h
@@ -42,7 +42,7 @@ class VulkanOffscreenSwapchain : public VulkanVirtualSwapchain
                                    HandlePointerDecoder<VkSurfaceKHR>* surface,
                                    const encode::InstanceTable*        instance_table,
                                    application::Application*           application,
-                                   int32_t                             options_surface_index) override;
+                                   const VulkanReplayOptions&          replay_options) override;
 
     virtual void DestroySurface(PFN_vkDestroySurfaceKHR      func,
                                 const InstanceInfo*          instance_info,
@@ -105,6 +105,11 @@ class VulkanOffscreenSwapchain : public VulkanVirtualSwapchain
                                    uint32_t           signal_semaphore_count,
                                    const VkSemaphore* signal_semaphores,
                                    VkFence            fence);
+
+    bool                         insert_frame_boundary_{ false };
+    std::vector<VkCommandPool>   command_pools_{ VK_NULL_HANDLE };
+    std::vector<VkCommandBuffer> command_buffers_{ VK_NULL_HANDLE };
+    VkFrameBoundaryEXT           frame_boundary_;
 };
 
 GFXRECON_END_NAMESPACE(decode)