Qt
Internal/Contributor docs for the Qt SDK. Note: These are NOT official API docs; those are found at https://doc.qt.io/
Loading...
Searching...
No Matches
qt3dxr-multiview.qdoc
Go to the documentation of this file.
1// Copyright (C) 2024 The Qt Company Ltd.
2// SPDX-License-Identifier: LicenseRef-Qt-Commercial OR GFDL-1.3-no-invariants-only
3
4/*!
5 \page qt3dxr-multiview.html
6 \title Multiview Rendering
7
8 \brief This page describes the concepts and practical implications of
9 multiview rendering support in Qt Quick 3D.
10
11 Multiview rendering refers to instancing draw calls into layers of a 2D
12 texture array. In Qt, it is relevant, in particular for VR/AR applications
13 built with \qxr. Instead of independently doing scene traversal, rendering
14 preparations, and render pass recording for the left and right eye's
15 content, multiview rendering allows doing it once, with the draw calls in
16 the single render pass being instanced into layers 0 and 1 of a texture
17 array. The vertex shader uses a special variable indicating the view index
18 and computes per-view values based on that. Uniforms that contain
19 view-dependent data, such as camera matrices, must be provided for both
20 eyes. Multiview rendering is expected to decrease the renderer's load on the
21 system, potentially leading to better performance. It comes at the expense
22 of having to make the renderer and shaders aware of working with
23 view-dependent data and texture array as appropriate.
24
25 \section1 Low-Level Overview
26
27 Qt application developers do not necessarily need a full understanding of
28 how multiview rendering is enabled on the lower levels of the Qt rendering
29 stack. The following links are provided for developers wishing to look more
30 into the details under the hood.
31
32 \section2 Multiview support in 3D APIs
33
34 Multiview rendering is available only when the underlying 3D API supports it
35 at run time. For details, see the appropriate specifications and documentation:
36
37 \list
38
39 \li OpenGL:
40 \l{https://registry.khronos.org/OpenGL/extensions/OVR/OVR_multiview.txt}{OVR_multiview1} and
41 \l{https://registry.khronos.org/OpenGL/extensions/OVR/OVR_multiview2.txt}{OVR_multiview2}
42
43 \li Vulkan:
44 \l{https://registry.khronos.org/vulkan/specs/1.3-extensions/man/html/VK_KHR_multiview.html}{VK_KHR_multiview}
45
46 \li Direct 3D 12:
47 \l{https://microsoft.github.io/DirectX-Specs/d3d/ViewInstancing.html}{ViewInstancing}
48
49 \li Metal:
50 \l{https://developer.apple.com/documentation/metal/render_passes/rendering_to_multiple_texture_slices_in_a_draw_command}{rendering_to_multiple_texture_slices}
51
52 \endlist
53
54 \section2 Multiview support in Qt's rendering hardware interface (RHI)
55
56 In Qt, the 3D graphics APIs are abstracted by the \l QRhi class. Qt Quick
57 and Qt Quick 3D uses this infrastructure for all their accelerated
58 rendering. See the following for further low-level information on multiview
59 rendering support:
60
61 \list
62 \li \l QRhi::MultiView
63 \li \l QRhiColorAttachment::setMultiViewCount()
64 \li \l QRhiGraphicsPipeline::setMultiViewCount()
65 \endlist
66
67 \section1 Multiview Support in the Qt Quick - Quick 3D - Quick 3D XR Stack
68
69 \section2 Multiview support in Qt Quick
70
71 Qt Quick is multiview-aware, but is never using multiview rendering on its
72 own. Supporting multiview rendering becomes important in combination with Qt
73 Quick 3D when 2D elements are embedded into the 3D scene. To render the 2D
74 content correctly when the 3D scene is output to a multiview render target,
75 the 2D renderer and its materials (shaders) must be prepared for multiview
76 support.
77
78 Developers of Qt-based applications do not need to take this into
79 consideration in many cases because Qt Quick's built-in materials that
80 items such as \l Rectangle, \l Image, or \l Text are built on are all
81 multiview compatible.
82
83 However, when developing custom materials (\l QSGMaterial, \l
84 QSGMaterialShader) or writing shaders for \l ShaderEffect, and the intention
85 is to use that 2D content within a VR/AR scene in a \qxr application, the
86 custom content must be multiview-aware. see \l QSGMaterial::viewCount() for
87 details on this.
88
89 Writing multiview-aware shaders is enabled by Qt's shader conditioning
90 pipeline. See the Multiview sections in \l{QSB Manual} and \l{Qt Shader
91 Tools Build System Integration} for details.
92
93 \section2 Multiview support in Qt Quick 3D
94
95 Qt Quick 3D applications that do not use \qxr, meaning they are not VR/AR
96 applications, cannot currently use multiview rendering. The 3D renderer is
97 fully multiview-capable, however, since \qxr is built on the same
98 infrastructure. All standard, built-in features, such as \l Model or \l
99 PrincipledMaterial are fully multiview compatible. Some deprecated
100 functionality, such as the old standalone effects module, may not support
101 multiview rendering.
102
103 When custom shader snippets are involved in a \l CustomMaterial or \l
104 Effect, the application-provided shader code needs to be written with
105 multiview support in mind in order to function correctly in a \qxr
106 application with multiview rendering enabled. See the documentation of these
107 types on how to achieve this. The special keywords for which this is
108 particularly important are \c VIEW_INDEX, \c INPUT, \c SCREEN_TEXTURE, \c
109 DEPTH_TEXTURE, \c AO_TEXTURE.
110
111 For example, the following postprocessing effect is multiview compatible,
112 because it is prepared for the case when \c INPUT is a \c sampler2DArray
113 instead of a \c sampler2D:
114
115 \badcode
116 void MAIN()
117 {
118 vec4 c = texture(someTexture, TEXTURE_UV);
119 // ...
120 #if QSHADER_VIEW_COUNT >= 2
121 FRAGCOLOR = c * texture(INPUT, vec3(INPUT_UV, VIEW_INDEX));
122 #else
123 FRAGCOLOR = c * texture(INPUT, INPUT_UV);
124 #endif
125 }
126 \endcode
127
128 \section2 Multiview support in \qxr
129
130 Multiview rendering is \b enabled by default, as long as the underlying graphics API
131 supports it. This is done to ensure the best possible performance. To query if multiview
132 rendering is supported, check the
133 \l{QtQuick3D.Xr::XrView::multiViewRenderingSupported}{multiViewRenderingSupported}
134 property.
135
136 For development and testing purposes, it can be useful to disable the usage of
137 multiview rendering. This is done by setting the environment variable
138 \c QT_QUICK3D_XR_DISABLE_MULTIVIEW to a non-zero value.
139
140 To query if multiview rendering is enabled, use the
141 \l{QtQuick3D.Xr::XrView::multiViewRenderingEnabled}{multiViewRenderingEnabled}
142 property. The value stays false always if multiview rendering cannot be
143 enabled.
144
145 In general, it is recommended that VR/AR applications leave multiview
146 rendering enabled when it's supported, and only disable it when experiencing problems,
147 or for testing purposes. Multiview rendering is expected to improve performance,
148 reducing the GPU and especially the CPU load.
149*/