580.65.06

(cherry picked from commit 1795a8bb20)

CHANGELOG.md

@@ -1,210 +0,0 @@
-# Changelog
-
-## Release 550 Entries
-
-### [550.54.14] 2024-02-23
-
-#### Added
-
-- Added vGPU Host and vGPU Guest support. For vGPU Host, please refer to the README.vgpu packaged in the vGPU Host Package for more details.
-
-### [550.40.07] 2024-01-24
-
-#### Fixed
-
-- Set INSTALL_MOD_DIR only if it's not defined, [#570](https://github.com/NVIDIA/open-gpu-kernel-modules/pull/570) by @keelung-yang
-
-## Release 545 Entries
-
-### [545.29.06] 2023-11-22
-
-#### Fixed
-
-- The brightness control of NVIDIA seems to be broken, [#573](https://github.com/NVIDIA/open-gpu-kernel-modules/issues/573)
-
-### [545.29.02] 2023-10-31
-
-### [545.23.06] 2023-10-17
-
-#### Fixed
-
-- Fix always-false conditional, [#493](https://github.com/NVIDIA/open-gpu-kernel-modules/pull/493) by @meme8383
-
-#### Added
-
-- Added beta-quality support for GeForce and Workstation GPUs. Please see the "Open Linux Kernel Modules" chapter in the NVIDIA GPU driver end user README for details.
-
-## Release 535 Entries
-
-### [535.129.03] 2023-10-31
-
-### [535.113.01] 2023-09-21
-
-#### Fixed
-
-- Fixed building main against current centos stream 8 fails, [#550](https://github.com/NVIDIA/open-gpu-kernel-modules/issues/550) by @airlied
-
-### [535.104.05] 2023-08-22
-
-### [535.98] 2023-08-08
-
-### [535.86.10] 2023-07-31
-
-### [535.86.05] 2023-07-18
-
-### [535.54.03] 2023-06-14
-
-### [535.43.02] 2023-05-30
-
-#### Fixed
-
-- Fixed console restore with traditional VGA consoles.
-
-#### Added
-
-- Added support for Run Time D3 (RTD3) on Ampere and later GPUs.
-- Added support for G-Sync on desktop GPUs.
-
-## Release 530 Entries
-
-### [530.41.03] 2023-03-23
-
-### [530.30.02] 2023-02-28
-
-#### Changed
-
-- GSP firmware is now distributed as `gsp_tu10x.bin` and `gsp_ga10x.bin` to better reflect the GPU architectures supported by each firmware file in this release.
-    - The .run installer will continue to install firmware to /lib/firmware/nvidia/<version> and the nvidia.ko kernel module will load the appropriate firmware for each GPU at runtime.
-  
-#### Fixed
-
-- Add support for resizable BAR on Linux when NVreg_EnableResizableBar=1 module param is set. [#3](https://github.com/NVIDIA/open-gpu-kernel-modules/pull/3) by @sjkelly
-
-#### Added
-
-- Support for power management features like Suspend, Hibernate and Resume.
-
-## Release 525 Entries
-
-### [525.147.05] 2023-10-31
-
-#### Fixed
-
-- Fix nvidia_p2p_get_pages(): Fix double-free in register-callback error path, [#557](https://github.com/NVIDIA/open-gpu-kernel-modules/pull/557) by @BrendanCunningham
-
-### [525.125.06] 2023-06-26
-
-### [525.116.04] 2023-05-09
-
-### [525.116.03] 2023-04-25
-
-### [525.105.17] 2023-03-30
-
-### [525.89.02] 2023-02-08
-
-### [525.85.12] 2023-01-30
-
-### [525.85.05] 2023-01-19
-
-#### Fixed
-
-- Fix build problems with Clang 15.0, [#377](https://github.com/NVIDIA/open-gpu-kernel-modules/issues/377) by @ptr1337
-
-### [525.78.01] 2023-01-05
-
-### [525.60.13] 2022-12-05
-
-### [525.60.11] 2022-11-28
-
-#### Fixed
-
-- Fixed nvenc compatibility with usermode clients [#104](https://github.com/NVIDIA/open-gpu-kernel-modules/issues/104)
-
-### [525.53] 2022-11-10
-
-#### Changed
-
-- GSP firmware is now distributed as multiple firmware files: this release has `gsp_tu10x.bin` and `gsp_ad10x.bin` replacing `gsp.bin` from previous releases.
-    - Each file is named after a GPU architecture and supports GPUs from one or more architectures. This allows GSP firmware to better leverage each architecture's capabilities.
-    - The .run installer will continue to install firmware to `/lib/firmware/nvidia/<version>` and the `nvidia.ko` kernel module will load the appropriate firmware for each GPU at runtime.
-
-#### Fixed
-
-- Add support for IBT (indirect branch tracking) on supported platforms, [#256](https://github.com/NVIDIA/open-gpu-kernel-modules/issues/256) by @rnd-ash
-- Return EINVAL when [failing to] allocating memory, [#280](https://github.com/NVIDIA/open-gpu-kernel-modules/pull/280) by @YusufKhan-gamedev
-- Fix various typos in nvidia/src/kernel, [#16](https://github.com/NVIDIA/open-gpu-kernel-modules/pull/16) by @alexisgeoffrey
-- Added support for rotation in X11, Quadro Sync, Stereo, and YUV 4:2:0 on Turing.
-
-## Release 520 Entries
-
-### [520.61.07] 2022-10-20
-
-### [520.56.06] 2022-10-12
-
-#### Added
-
-- Introduce support for GeForce RTX 4090 GPUs.
-
-### [520.61.05] 2022-10-10
-
-#### Added
-
-- Introduce support for NVIDIA H100 GPUs.
-
-#### Fixed
-
-- Fix/Improve Makefile, [#308](https://github.com/NVIDIA/open-gpu-kernel-modules/pull/308/) by @izenynn
-- Make nvLogBase2 more efficient, [#177](https://github.com/NVIDIA/open-gpu-kernel-modules/pull/177/) by @DMaroo
-- nv-pci: fixed always true expression, [#195](https://github.com/NVIDIA/open-gpu-kernel-modules/pull/195/) by @ValZapod
-
-## Release 515 Entries
-
-### [515.76] 2022-09-20
-
-#### Fixed
-
-- Improved compatibility with new Linux kernel releases
-- Fixed possible excessive GPU power draw on an idle X11 or Wayland desktop when driving high resolutions or refresh rates
-
-### [515.65.07] 2022-10-19
-
-### [515.65.01] 2022-08-02
-
-#### Fixed
-
-- Collection of minor fixes to issues, [#6](https://github.com/NVIDIA/open-gpu-kernel-modules/pull/61) by @Joshua-Ashton
-- Remove unnecessary use of acpi_bus_get_device().
-
-### [515.57] 2022-06-28
-
-#### Fixed
-
-- Backtick is deprecated, [#273](https://github.com/NVIDIA/open-gpu-kernel-modules/pull/273) by @arch-user-france1
-
-### [515.48.07] 2022-05-31
-
-#### Added
-
-- List of compatible GPUs in README.md.
-
-#### Fixed
-
-- Fix various README capitalizations, [#8](https://github.com/NVIDIA/open-gpu-kernel-modules/pull/8) by @27lx 
-- Automatically tag bug report issues, [#15](https://github.com/NVIDIA/open-gpu-kernel-modules/pull/15) by @thebeanogamer
-- Improve conftest.sh Script, [#37](https://github.com/NVIDIA/open-gpu-kernel-modules/pull/37) by @Nitepone
-- Update HTTP link to HTTPS, [#101](https://github.com/NVIDIA/open-gpu-kernel-modules/pull/101) by @alcaparra
-- moved array sanity check to before the array access, [#117](https://github.com/NVIDIA/open-gpu-kernel-modules/pull/117) by @RealAstolfo
-- Fixed some typos, [#122](https://github.com/NVIDIA/open-gpu-kernel-modules/pull/122) by @FEDOyt
-- Fixed capitalization, [#123](https://github.com/NVIDIA/open-gpu-kernel-modules/pull/123) by @keroeslux
-- Fix typos in NVDEC Engine Descriptor, [#126](https://github.com/NVIDIA/open-gpu-kernel-modules/pull/126) from @TrickyDmitriy
-- Extranous apostrohpes in a makefile script [sic], [#14](https://github.com/NVIDIA/open-gpu-kernel-modules/issues/14) by @kiroma
-- HDMI no audio @ 4K above 60Hz, [#75](https://github.com/NVIDIA/open-gpu-kernel-modules/issues/75) by @adolfotregosa
-- dp_configcaps.cpp:405: array index sanity check in wrong place?, [#110](https://github.com/NVIDIA/open-gpu-kernel-modules/issues/110) by @dcb314
-- NVRM kgspInitRm_IMPL: missing NVDEC0 engine, cannot initialize GSP-RM, [#116](https://github.com/NVIDIA/open-gpu-kernel-modules/issues/116) by @kfazz
-- ERROR: modpost: "backlight_device_register" [...nvidia-modeset.ko] undefined, [#135](https://github.com/NVIDIA/open-gpu-kernel-modules/issues/135) by @sndirsch
-- aarch64 build fails, [#151](https://github.com/NVIDIA/open-gpu-kernel-modules/issues/151) by @frezbo
-
-### [515.43.04] 2022-05-11
-
-- Initial release.
-

kernel-open/Kbuild

@@ -8,7 +8,7 @@
 # NV_KERNEL_SOURCES : The root of the kernel source tree.
 # NV_KERNEL_OUTPUT : The kernel's output tree.
 # NV_KERNEL_MODULES : A whitespace-separated list of modules to build.
-# ARCH : The target CPU architecture: x86_64|arm64|powerpc
+# ARCH : The target CPU architecture: x86_64|arm64
 #
 # Kbuild provides the variables:
 #
@@ -57,25 +57,42 @@ ifeq ($(NV_UNDEF_BEHAVIOR_SANITIZER),1)
  UBSAN_SANITIZE := y
 endif
 
+#
+# Command to create a symbolic link, explicitly resolving the symlink target
+# to an absolute path to abstract away the difference between Linux < 6.13,
+# where the CWD is the Linux kernel source tree for Kbuild extmod builds, and
+# Linux >= 6.13, where the CWD is the external module source tree.
+#
+# This is used to create the nv*-kernel.o -> nv*-kernel.o_binary symlinks for
+# kernel modules which use precompiled binary object files.
+#
+
+quiet_cmd_symlink = SYMLINK $@
+ cmd_symlink = ln -sf $(abspath $<) $@
+
+
 $(foreach _module, $(NV_KERNEL_MODULES), \
  $(eval include $(src)/$(_module)/$(_module).Kbuild))
 
 
-#
-# Define CFLAGS that apply to all the NVIDIA kernel modules. EXTRA_CFLAGS
-# is deprecated since 2.6.24 in favor of ccflags-y, but we need to support
-# older kernels which do not have ccflags-y. Newer kernels append
-# $(EXTRA_CFLAGS) to ccflags-y for compatibility.
-#
+ccflags-y += -I$(src)/common/inc
+ccflags-y += -I$(src)
+ccflags-y += -Wall $(DEFINES) $(INCLUDES) -Wno-cast-qual -Wno-format-extra-args
+ccflags-y += -D__KERNEL__ -DMODULE -DNVRM
+ccflags-y += -DNV_VERSION_STRING=\"580.65.06\"
 
-EXTRA_CFLAGS += -I$(src)/common/inc
-EXTRA_CFLAGS += -I$(src)
-EXTRA_CFLAGS += -Wall $(DEFINES) $(INCLUDES) -Wno-cast-qual -Wno-format-extra-args
-EXTRA_CFLAGS += -D__KERNEL__ -DMODULE -DNVRM
-EXTRA_CFLAGS += -DNV_VERSION_STRING=\"550.54.14\"
+# Include and link Tegra out-of-tree modules.
+ifneq ($(wildcard /usr/src/nvidia/nvidia-oot),)
+ SYSSRCNVOOT ?= /usr/src/nvidia/nvidia-oot
+endif
 
 ifneq ($(SYSSRCHOST1X),)
- EXTRA_CFLAGS += -I$(SYSSRCHOST1X)
+ ccflags-y += -I$(SYSSRCHOST1X)
+endif
+
+ifneq ($(SYSSRCNVOOT),)
+ ccflags-y += -I$(SYSSRCNVOOT)/include
+ KBUILD_EXTRA_SYMBOLS = $(SYSSRCNVOOT)/Module.symvers
 endif
 
 # Some Android kernels prohibit driver use of filesystem functions like
@@ -85,57 +102,53 @@ endif
 PLATFORM_IS_ANDROID ?= 0
 
 ifeq ($(PLATFORM_IS_ANDROID),1)
- EXTRA_CFLAGS += -DNV_FILESYSTEM_ACCESS_AVAILABLE=0
+ ccflags-y += -DNV_FILESYSTEM_ACCESS_AVAILABLE=0
 else
- EXTRA_CFLAGS += -DNV_FILESYSTEM_ACCESS_AVAILABLE=1
+ ccflags-y += -DNV_FILESYSTEM_ACCESS_AVAILABLE=1
 endif
 
-EXTRA_CFLAGS += -Wno-unused-function
+ccflags-y += -Wno-unused-function
 
 ifneq ($(NV_BUILD_TYPE),debug)
- EXTRA_CFLAGS += -Wuninitialized
+ ccflags-y += -Wuninitialized
 endif
 
-EXTRA_CFLAGS += -fno-strict-aliasing
+ccflags-y += -fno-strict-aliasing
 
 ifeq ($(ARCH),arm64)
- EXTRA_CFLAGS += -mstrict-align
+ ccflags-y += -mstrict-align
 endif
 
 ifeq ($(NV_BUILD_TYPE),debug)
- EXTRA_CFLAGS += -g
+ ccflags-y += -g
 endif
 
-EXTRA_CFLAGS += -ffreestanding
+ccflags-y += -ffreestanding
 
 ifeq ($(ARCH),arm64)
- EXTRA_CFLAGS += -mgeneral-regs-only -march=armv8-a
- EXTRA_CFLAGS += $(call cc-option,-mno-outline-atomics,)
+ ccflags-y += -mgeneral-regs-only -march=armv8-a
+ ccflags-y += $(call cc-option,-mno-outline-atomics,)
 endif
 
 ifeq ($(ARCH),x86_64)
- EXTRA_CFLAGS += -mno-red-zone -mcmodel=kernel
+ ccflags-y += -mno-red-zone -mcmodel=kernel
 endif
 
-ifeq ($(ARCH),powerpc)
- EXTRA_CFLAGS += -mlittle-endian -mno-strict-align -mno-altivec
-endif
-
-EXTRA_CFLAGS += -DNV_UVM_ENABLE
-EXTRA_CFLAGS += $(call cc-option,-Werror=undef,)
-EXTRA_CFLAGS += -DNV_SPECTRE_V2=$(NV_SPECTRE_V2)
-EXTRA_CFLAGS += -DNV_KERNEL_INTERFACE_LAYER
+ccflags-y += -DNV_UVM_ENABLE
+ccflags-y += $(call cc-option,-Werror=undef,)
+ccflags-y += -DNV_SPECTRE_V2=$(NV_SPECTRE_V2)
+ccflags-y += -DNV_KERNEL_INTERFACE_LAYER
 
 #
 # Detect SGI UV systems and apply system-specific optimizations.
 #
 
 ifneq ($(wildcard /proc/sgi_uv),)
- EXTRA_CFLAGS += -DNV_CONFIG_X86_UV
+ ccflags-y += -DNV_CONFIG_X86_UV
 endif
 
 ifdef VGX_FORCE_VFIO_PCI_CORE
- EXTRA_CFLAGS += -DNV_VGPU_FORCE_VFIO_PCI_CORE
+ ccflags-y += -DNV_VGPU_FORCE_VFIO_PCI_CORE
 endif
 
 WARNINGS_AS_ERRORS ?=
@@ -169,7 +182,11 @@ NV_CONFTEST_CMD := /bin/sh $(NV_CONFTEST_SCRIPT) \
 
 NV_CFLAGS_FROM_CONFTEST := $(shell $(NV_CONFTEST_CMD) build_cflags)
 
-NV_CONFTEST_CFLAGS = $(NV_CFLAGS_FROM_CONFTEST) $(EXTRA_CFLAGS) -fno-pie
+NV_CONFTEST_CFLAGS = $(NV_CFLAGS_FROM_CONFTEST) $(ccflags-y) -fno-pie
+NV_CONFTEST_CFLAGS += $(filter -std=%,$(KBUILD_CFLAGS))
+NV_CONFTEST_CFLAGS += $(call cc-disable-warning,pointer-sign)
+NV_CONFTEST_CFLAGS += $(call cc-option,-fshort-wchar,)
+NV_CONFTEST_CFLAGS += $(call cc-option,-Werror=incompatible-pointer-types,)
 NV_CONFTEST_CFLAGS += -Wno-error
 
 NV_CONFTEST_COMPILE_TEST_HEADERS := $(obj)/conftest/macros.h

kernel-open/Makefile

@@ -28,7 +28,7 @@ else
   else
     KERNEL_UNAME ?= $(shell uname -r)
     KERNEL_MODLIB := /lib/modules/$(KERNEL_UNAME)
-    KERNEL_SOURCES := $(shell test -d $(KERNEL_MODLIB)/source && echo $(KERNEL_MODLIB)/source || echo $(KERNEL_MODLIB)/build)
+    KERNEL_SOURCES := $(shell ((test -d $(KERNEL_MODLIB)/source && echo $(KERNEL_MODLIB)/source) || (test -d $(KERNEL_MODLIB)/build/source && echo $(KERNEL_MODLIB)/build/source)) || echo $(KERNEL_MODLIB)/build)
   endif
 
   KERNEL_OUTPUT := $(KERNEL_SOURCES)
@@ -42,25 +42,78 @@ else
   else
     KERNEL_UNAME ?= $(shell uname -r)
     KERNEL_MODLIB := /lib/modules/$(KERNEL_UNAME)
-    ifeq ($(KERNEL_SOURCES), $(KERNEL_MODLIB)/source)
+    # $(filter patter...,text) - Returns all whitespace-separated words in text that
+    # do match any of the pattern words, removing any words that do not match.
+    # Set the KERNEL_OUTPUT only if either $(KERNEL_MODLIB)/source or
+    # $(KERNEL_MODLIB)/build/source path matches the KERNEL_SOURCES.
+    ifneq ($(filter $(KERNEL_SOURCES),$(KERNEL_MODLIB)/source $(KERNEL_MODLIB)/build/source),)
       KERNEL_OUTPUT := $(KERNEL_MODLIB)/build
       KBUILD_PARAMS := KBUILD_OUTPUT=$(KERNEL_OUTPUT)
     endif
   endif
 
+  # If CC hasn't been set explicitly, check the value of CONFIG_CC_VERSION_TEXT.
+  # Look for the compiler specified there, and use it by default, if found.
+  ifeq ($(origin CC),default)
+    cc_version_text=$(firstword $(shell . $(KERNEL_OUTPUT)/.config; \
+                      echo "$$CONFIG_CC_VERSION_TEXT"))
+
+    ifneq ($(cc_version_text),)
+      ifeq ($(shell command -v $(cc_version_text)),)
+          $(warning WARNING: Unable to locate the compiler $(cc_version_text) \
+            from CONFIG_CC_VERSION_TEXT in the kernel configuration.)
+      else
+          CC=$(cc_version_text)
+      endif
+    endif
+  endif
+
   CC ?= cc
   LD ?= ld
   OBJDUMP ?= objdump
+  AWK ?= awk
+  # Bake the following awk program in a string. The program is needed to add C++
+  # to the languages excluded from BTF generation.
+  #
+  # Also, unconditionally return success (0) from the awk program, rather than
+  # propagating pahole's return status (with 'exit system(pahole_cmd)'), to
+  # workaround an DW_TAG_rvalue_reference_type error in
+  # kernel/nvidia-modeset.ko.
+  #
+  # BEGIN {
+  #     pahole_cmd = "pahole"
+  #     for (i = 1; i < ARGC; i++) {
+  #         if (ARGV[i] ~ /--lang_exclude=/) {
+  #             pahole_cmd = pahole_cmd sprintf(" %s,c++", ARGV[i])
+  #         } else {
+  #             pahole_cmd = pahole_cmd sprintf(" %s", ARGV[i])
+  #         }
+  #     }
+  #     system(pahole_cmd)
+  # }
+  PAHOLE_AWK_PROGRAM = BEGIN { pahole_cmd = \"pahole\"; for (i = 1; i < ARGC; i++) { if (ARGV[i] ~ /--lang_exclude=/) { pahole_cmd = pahole_cmd sprintf(\" %s,c++\", ARGV[i]); } else { pahole_cmd = pahole_cmd sprintf(\" %s\", ARGV[i]); } } system(pahole_cmd); }
+  # If scripts/pahole-flags.sh is not present in the kernel tree, add PAHOLE and
+  # PAHOLE_AWK_PROGRAM assignments to PAHOLE_VARIABLES; otherwise assign the
+  # empty string to PAHOLE_VARIABLES.
+  PAHOLE_VARIABLES=$(if $(wildcard $(KERNEL_SOURCES)/scripts/pahole-flags.sh),,"PAHOLE=$(AWK) '$(PAHOLE_AWK_PROGRAM)'")
 
   ifndef ARCH
     ARCH := $(shell uname -m | sed -e 's/i.86/i386/' \
-      -e 's/armv[0-7]\w\+/arm/' \
       -e 's/aarch64/arm64/' \
-      -e 's/ppc64le/powerpc/' \
       -e 's/riscv64/riscv/' \
     )
   endif
 
+  KERNEL_ARCH = $(ARCH)
+
+  ifneq ($(filter $(ARCH),i386 x86_64),)
+    KERNEL_ARCH = x86
+  else
+    ifeq ($(filter $(ARCH),arm64 riscv),)
+        $(error Unsupported architecture $(ARCH))
+    endif
+  endif
+
   NV_KERNEL_MODULES ?= $(wildcard nvidia nvidia-uvm nvidia-vgpu-vfio nvidia-modeset nvidia-drm nvidia-peermem)
   NV_KERNEL_MODULES := $(filter-out $(NV_EXCLUDE_KERNEL_MODULES), \
                                     $(NV_KERNEL_MODULES))
@@ -82,7 +135,8 @@ else
 
   .PHONY: modules module clean clean_conftest modules_install
   modules clean modules_install:
-	@$(MAKE) "LD=$(LD)" "CC=$(CC)" "OBJDUMP=$(OBJDUMP)" $(KBUILD_PARAMS) $@
+	@$(MAKE) "LD=$(LD)" "CC=$(CC)" "OBJDUMP=$(OBJDUMP)" \
+	  $(PAHOLE_VARIABLES) $(KBUILD_PARAMS) $@
 	@if [ "$@" = "modules" ]; then \
 	  for module in $(NV_KERNEL_MODULES); do \
 	    if [ -x split-object-file.sh ]; then \
@@ -102,8 +156,9 @@ else
   # module symbols on which the Linux kernel's module resolution is dependent
   # and hence must be used whenever present.
 
-  LD_SCRIPT ?= $(KERNEL_SOURCES)/scripts/module-common.lds      \
-               $(KERNEL_SOURCES)/arch/$(ARCH)/kernel/module.lds \
+  LD_SCRIPT ?= $(KERNEL_SOURCES)/scripts/module-common.lds             \
+               $(KERNEL_SOURCES)/arch/$(KERNEL_ARCH)/kernel/module.lds \
+               $(KERNEL_OUTPUT)/arch/$(KERNEL_ARCH)/module.lds         \
                $(KERNEL_OUTPUT)/scripts/module.lds
   NV_MODULE_COMMON_SCRIPTS := $(foreach s, $(wildcard $(LD_SCRIPT)), -T $(s))
 

kernel-open/common/inc/dce_rm_client_ipc.h

@@ -1,5 +1,5 @@
 /*
- * SPDX-FileCopyrightText: Copyright (c) 2021-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+ * SPDX-FileCopyrightText: Copyright (c) 2020-2021 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
  * SPDX-License-Identifier: MIT
  *
  * Permission is hereby granted, free of charge, to any person obtaining a
@@ -20,20 +20,16 @@
  * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
  * DEALINGS IN THE SOFTWARE.
  */
+#ifndef _OS_DCE_CLIENT_IPC_H_
+#define _OS_DCE_CLIENT_IPC_H_
 
-#include "core/core.h"
-#include "os/os.h"
-#include "gpu/mmu/vidmem_access_bit_buffer.h"
-#include "gpu/gpu.h"
+// RM IPC Client Types
 
-NV_STATUS
-vidmemAccessBitBufConstructHelper_KERNEL
-(
-    VidmemAccessBitBuffer  *pVidmemAccessBitBuffer,
-    CALL_CONTEXT           *pCallContext,
-    RS_RES_ALLOC_PARAMS_INTERNAL *pParams
-)
-{
-    OBJGPU *pGpu = GPU_RES_GET_GPU(pVidmemAccessBitBuffer);
-    return IS_VGPU_GSP_PLUGIN_OFFLOAD_ENABLED(pGpu) && !IS_VIRTUAL(pGpu)? NV_OK : NV_ERR_NOT_SUPPORTED;
-}
+#define DCE_CLIENT_RM_IPC_TYPE_SYNC             0x0
+#define DCE_CLIENT_RM_IPC_TYPE_EVENT            0x1
+#define DCE_CLIENT_RM_IPC_TYPE_MAX              0x2
+
+void dceclientHandleAsyncRpcCallback(NvU32 handle, NvU32 interfaceType,
+                                     NvU32 msgLength, void *data,
+                                     void *usrCtx);
+#endif

kernel-open/common/inc/nv-firmware.h

@@ -1,5 +1,5 @@
 /*
- * SPDX-FileCopyrightText: Copyright (c) 2022-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+ * SPDX-FileCopyrightText: Copyright (c) 2022-2025 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
  * SPDX-License-Identifier: MIT
  *
  * Permission is hereby granted, free of charge, to any person obtaining a
@@ -32,7 +32,10 @@
 typedef enum
 {
     NV_FIRMWARE_TYPE_GSP,
-    NV_FIRMWARE_TYPE_GSP_LOG
+    NV_FIRMWARE_TYPE_GSP_LOG,
+#if defined(NV_VMWARE)
+    NV_FIRMWARE_TYPE_BINDATA
+#endif
 } nv_firmware_type_t;
 
 typedef enum
@@ -44,6 +47,10 @@ typedef enum
     NV_FIRMWARE_CHIP_FAMILY_GA10X = 4,
     NV_FIRMWARE_CHIP_FAMILY_AD10X = 5,
     NV_FIRMWARE_CHIP_FAMILY_GH100 = 6,
+    NV_FIRMWARE_CHIP_FAMILY_GB10X = 8,
+    NV_FIRMWARE_CHIP_FAMILY_GB20X = 9,
+    NV_FIRMWARE_CHIP_FAMILY_GB10Y = 11,
+    NV_FIRMWARE_CHIP_FAMILY_GB20Y = 12,
     NV_FIRMWARE_CHIP_FAMILY_END,
 } nv_firmware_chip_family_t;
 
@@ -52,6 +59,10 @@ static inline const char *nv_firmware_chip_family_to_string(
 )
 {
     switch (fw_chip_family) {
+        case NV_FIRMWARE_CHIP_FAMILY_GB10X: return "gb10x";
+        case NV_FIRMWARE_CHIP_FAMILY_GB10Y: return "gb10y";
+        case NV_FIRMWARE_CHIP_FAMILY_GB20Y: return "gb20y";
+        case NV_FIRMWARE_CHIP_FAMILY_GB20X: return "gb20x";
         case NV_FIRMWARE_CHIP_FAMILY_GH100: return "gh100";
         case NV_FIRMWARE_CHIP_FAMILY_AD10X: return "ad10x";
         case NV_FIRMWARE_CHIP_FAMILY_GA10X: return "ga10x";
@@ -61,18 +72,18 @@ static inline const char *nv_firmware_chip_family_to_string(
 
         case NV_FIRMWARE_CHIP_FAMILY_END:  // fall through
         case NV_FIRMWARE_CHIP_FAMILY_NULL:
-            return NULL;
+            return "";
     }
-    return NULL;
+    return "";
 }
 
-// The includer (presumably nv.c) may optionally define
-// NV_FIRMWARE_PATH_FOR_FILENAME(filename)
-// to return a string "path" given a gsp_*.bin or gsp_log_*.bin filename.
+// The includer may optionally define
+// NV_FIRMWARE_FOR_NAME(name)
+// to return a platform-defined string for a given a gsp_* or gsp_log_* name.
 //
-// The function nv_firmware_path will then be available.
-#if defined(NV_FIRMWARE_PATH_FOR_FILENAME)
-static inline const char *nv_firmware_path(
+// The function nv_firmware_for_chip_family will then be available.
+#if defined(NV_FIRMWARE_FOR_NAME)
+static inline const char *nv_firmware_for_chip_family(
     nv_firmware_type_t fw_type,
     nv_firmware_chip_family_t fw_chip_family
 )
@@ -81,15 +92,19 @@ static inline const char *nv_firmware_path(
     {
         switch (fw_chip_family)
         {
+            case NV_FIRMWARE_CHIP_FAMILY_GB10X:  // fall through
+            case NV_FIRMWARE_CHIP_FAMILY_GB10Y:  // fall through
+            case NV_FIRMWARE_CHIP_FAMILY_GB20Y:  // fall through
+            case NV_FIRMWARE_CHIP_FAMILY_GB20X:  // fall through
             case NV_FIRMWARE_CHIP_FAMILY_GH100:  // fall through
             case NV_FIRMWARE_CHIP_FAMILY_AD10X:  // fall through
             case NV_FIRMWARE_CHIP_FAMILY_GA10X:
-                return NV_FIRMWARE_PATH_FOR_FILENAME("gsp_ga10x.bin");
+                return NV_FIRMWARE_FOR_NAME("gsp_ga10x");
 
             case NV_FIRMWARE_CHIP_FAMILY_GA100:  // fall through
             case NV_FIRMWARE_CHIP_FAMILY_TU11X:  // fall through
             case NV_FIRMWARE_CHIP_FAMILY_TU10X:
-                return NV_FIRMWARE_PATH_FOR_FILENAME("gsp_tu10x.bin");
+                return NV_FIRMWARE_FOR_NAME("gsp_tu10x");
 
             case NV_FIRMWARE_CHIP_FAMILY_END:  // fall through
             case NV_FIRMWARE_CHIP_FAMILY_NULL:
@@ -100,33 +115,42 @@ static inline const char *nv_firmware_path(
     {
         switch (fw_chip_family)
         {
+            case NV_FIRMWARE_CHIP_FAMILY_GB10X:  // fall through
+            case NV_FIRMWARE_CHIP_FAMILY_GB10Y:  // fall through
+            case NV_FIRMWARE_CHIP_FAMILY_GB20Y:  // fall through
+            case NV_FIRMWARE_CHIP_FAMILY_GB20X:  // fall through
             case NV_FIRMWARE_CHIP_FAMILY_GH100:  // fall through
             case NV_FIRMWARE_CHIP_FAMILY_AD10X:  // fall through
             case NV_FIRMWARE_CHIP_FAMILY_GA10X:
-                return NV_FIRMWARE_PATH_FOR_FILENAME("gsp_log_ga10x.bin");
+                return NV_FIRMWARE_FOR_NAME("gsp_log_ga10x");
 
             case NV_FIRMWARE_CHIP_FAMILY_GA100:  // fall through
             case NV_FIRMWARE_CHIP_FAMILY_TU11X:  // fall through
             case NV_FIRMWARE_CHIP_FAMILY_TU10X:
-                return NV_FIRMWARE_PATH_FOR_FILENAME("gsp_log_tu10x.bin");
+                return NV_FIRMWARE_FOR_NAME("gsp_log_tu10x");
 
             case NV_FIRMWARE_CHIP_FAMILY_END:  // fall through
             case NV_FIRMWARE_CHIP_FAMILY_NULL:
                 return "";
         }
     }
-
+#if defined(NV_VMWARE)
+    else if (fw_type == NV_FIRMWARE_TYPE_BINDATA)
+    {
+        return NV_FIRMWARE_FOR_NAME("bindata_image");
+    }
+#endif
     return "";
 }
-#endif  // defined(NV_FIRMWARE_PATH_FOR_FILENAME)
+#endif  // defined(NV_FIRMWARE_FOR_NAME)
 
-// The includer (presumably nv.c) may optionally define
-// NV_FIRMWARE_DECLARE_GSP_FILENAME(filename)
+// The includer may optionally define
+// NV_FIRMWARE_DECLARE_GSP(name)
 // which will then be invoked (at the top-level) for each
-// gsp_*.bin (but not gsp_log_*.bin)
-#if defined(NV_FIRMWARE_DECLARE_GSP_FILENAME)
-NV_FIRMWARE_DECLARE_GSP_FILENAME("gsp_ga10x.bin")
-NV_FIRMWARE_DECLARE_GSP_FILENAME("gsp_tu10x.bin")
-#endif  // defined(NV_FIRMWARE_DECLARE_GSP_FILENAME)
+// gsp_* (but not gsp_log_*)
+#if defined(NV_FIRMWARE_DECLARE_GSP)
+NV_FIRMWARE_DECLARE_GSP("gsp_ga10x")
+NV_FIRMWARE_DECLARE_GSP("gsp_tu10x")
+#endif  // defined(NV_FIRMWARE_DECLARE_GSP)
 
-#endif  // NV_FIRMWARE_DECLARE_GSP_FILENAME
+#endif  // NV_FIRMWARE_DECLARE_GSP

kernel-open/common/inc/nv-hash.h

@@ -29,17 +29,9 @@
 #include <linux/kernel.h>
 #include <linux/hash.h>
 
-#if defined(NV_LINUX_STRINGHASH_H_PRESENT)
 #include <linux/stringhash.h>       /* full_name_hash() */
-#else
-#include <linux/dcache.h>
-#endif
 
-#if (NV_FULL_NAME_HASH_ARGUMENT_COUNT == 3)
 #define nv_string_hash(_str) full_name_hash(NULL, _str, strlen(_str))
-#else
-#define nv_string_hash(_str) full_name_hash(_str, strlen(_str))
-#endif
 
 /**
  * This naive hashtable was introduced by commit d9b482c8ba19 (v3.7, 2012-10-31).

kernel-open/common/inc/nv-hypervisor.h

@@ -37,13 +37,11 @@ typedef enum _HYPERVISOR_TYPE
     OS_HYPERVISOR_UNKNOWN
 } HYPERVISOR_TYPE;
 
-#define CMD_VGPU_VFIO_WAKE_WAIT_QUEUE         0
-#define CMD_VGPU_VFIO_INJECT_INTERRUPT        1
-#define CMD_VGPU_VFIO_REGISTER_MDEV           2
-#define CMD_VGPU_VFIO_PRESENT                 3
-#define CMD_VFIO_PCI_CORE_PRESENT             4
+#define CMD_VFIO_WAKE_REMOVE_GPU              1
+#define CMD_VGPU_VFIO_PRESENT                 2
+#define CMD_VFIO_PCI_CORE_PRESENT             3
 
-#define MAX_VF_COUNT_PER_GPU 64
+#define MAX_VF_COUNT_PER_GPU                  64
 
 typedef enum _VGPU_TYPE_INFO
 {
@@ -54,17 +52,11 @@ typedef enum _VGPU_TYPE_INFO
 
 typedef struct
 {
-    void  *vgpuVfioRef;
-    void  *waitQueue;
     void  *nv;
-    NvU32 *vgpuTypeIds;
-    NvU8 **vgpuNames;
-    NvU32  numVgpuTypes;
-    NvU32  domain;
-    NvU8   bus;
-    NvU8   slot;
-    NvU8   function;
-    NvBool is_virtfn;
+    NvU32 domain;
+    NvU32 bus;
+    NvU32 device;
+    NvU32 return_status;
 } vgpu_vfio_info;
 
 typedef struct

kernel-open/common/inc/nv-ioctl.h

@@ -1,5 +1,5 @@
 /*
- * SPDX-FileCopyrightText: Copyright (c) 2020-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+ * SPDX-FileCopyrightText: Copyright (c) 2020-2025 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
  * SPDX-License-Identifier: MIT
  *
  * Permission is hereby granted, free of charge, to any person obtaining a
@@ -128,6 +128,9 @@ typedef struct nv_ioctl_register_fd
 
 #define NV_DMABUF_EXPORT_MAX_HANDLES 128
 
+#define NV_DMABUF_EXPORT_MAPPING_TYPE_DEFAULT        0
+#define NV_DMABUF_EXPORT_MAPPING_TYPE_FORCE_PCIE     1
+
 typedef struct nv_ioctl_export_to_dma_buf_fd
 {
     int         fd;
@@ -136,6 +139,8 @@ typedef struct nv_ioctl_export_to_dma_buf_fd
     NvU32       numObjects;
     NvU32       index;
     NvU64       totalSize NV_ALIGN_BYTES(8);
+    NvU8        mappingType;
+    NvBool      bAllowMmap;
     NvHandle    handles[NV_DMABUF_EXPORT_MAX_HANDLES];
     NvU64       offsets[NV_DMABUF_EXPORT_MAX_HANDLES] NV_ALIGN_BYTES(8);
     NvU64       sizes[NV_DMABUF_EXPORT_MAX_HANDLES] NV_ALIGN_BYTES(8);

kernel-open/common/inc/nv-kthread-q-os.h

@@ -1,5 +1,5 @@
 /*
- * SPDX-FileCopyrightText: Copyright (c) 2016 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+ * SPDX-FileCopyrightText: Copyright (c) 2016-2024 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
  * SPDX-License-Identifier: MIT
  *
  * Permission is hereby granted, free of charge, to any person obtaining a
@@ -43,6 +43,8 @@ struct nv_kthread_q
     atomic_t main_loop_should_exit;
 
     struct task_struct *q_kthread;
+
+    bool is_unload_flush_ongoing;
 };
 
 struct nv_kthread_q_item

kernel-open/common/inc/nv-lock.h

@@ -28,12 +28,9 @@
 
 #include <linux/spinlock.h>
 #include <linux/rwsem.h>
-#include <linux/sched.h> /* signal_pending, cond_resched */
+#include <linux/sched.h> /* cond_resched */
 #include <linux/semaphore.h>
-
-#if defined(NV_LINUX_SCHED_SIGNAL_H_PRESENT)
-#include <linux/sched/signal.h>     /* signal_pending for kernels >= 4.11 */
-#endif
+#include <linux/sched/signal.h> /* signal_pending */
 
 #if defined(CONFIG_PREEMPT_RT) || defined(CONFIG_PREEMPT_RT_FULL)
 typedef raw_spinlock_t            nv_spinlock_t;

kernel-open/common/inc/nv-mm.h

@@ -1,5 +1,5 @@
 /*
- * SPDX-FileCopyrightText: Copyright (c) 2016-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+ * SPDX-FileCopyrightText: Copyright (c) 2016-2025 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
  * SPDX-License-Identifier: MIT
  *
  * Permission is hereby granted, free of charge, to any person obtaining a
@@ -29,17 +29,17 @@
 typedef int vm_fault_t;
 #endif
 
-/* pin_user_pages
+/*
+ * pin_user_pages()
+ *
  * Presence of pin_user_pages() also implies the presence of unpin-user_page().
- * Both were added in the v5.6-rc1
+ * Both were added in the v5.6.
  *
- * pin_user_pages() was added by commit eddb1c228f7951d399240
- * ("mm/gup: introduce pin_user_pages*() and FOLL_PIN") in v5.6-rc1 (2020-01-30)
- *
- * Removed vmas parameter from pin_user_pages() by commit 40896a02751
- * ("mm/gup: remove vmas parameter from pin_user_pages()")
- * in linux-next, expected in v6.5-rc1 (2023-05-17)
+ * pin_user_pages() was added by commit eddb1c228f79
+ * ("mm/gup: introduce pin_user_pages*() and FOLL_PIN") in v5.6.
  *
+ * Removed vmas parameter from pin_user_pages() by commit 4c630f307455
+ * ("mm/gup: remove vmas parameter from pin_user_pages()") in v6.5.
  */
 
 #include <linux/mm.h>
@@ -63,67 +63,34 @@ typedef int vm_fault_t;
     #define NV_UNPIN_USER_PAGE put_page
 #endif // NV_PIN_USER_PAGES_PRESENT
 
-/* get_user_pages
+/*
+ * get_user_pages()
  *
- * The 8-argument version of get_user_pages was deprecated by commit
- * (2016 Feb 12: cde70140fed8429acf7a14e2e2cbd3e329036653)for the non-remote case
- * (calling get_user_pages with current and current->mm).
- *
- * Completely moved to the 6 argument version of get_user_pages -
- * 2016 Apr 4: c12d2da56d0e07d230968ee2305aaa86b93a6832
- *
- * write and force parameters were replaced with gup_flags by -
- * 2016 Oct 12: 768ae309a96103ed02eb1e111e838c87854d8b51
- *
- * A 7-argument version of get_user_pages was introduced into linux-4.4.y by
- * commit 8e50b8b07f462ab4b91bc1491b1c91bd75e4ad40 which cherry-picked the
- * replacement of the write and force parameters with gup_flags
- *
- * Removed vmas parameter from get_user_pages() by commit 7bbf9c8c99
- * ("mm/gup: remove unused vmas parameter from get_user_pages()")
- * in linux-next, expected in v6.5-rc1 (2023-05-17)
+ * Removed vmas parameter from get_user_pages() by commit 54d020692b34
+ * ("mm/gup: remove unused vmas parameter from get_user_pages()") in v6.5.
  *
  */
 
-#if defined(NV_GET_USER_PAGES_HAS_ARGS_FLAGS)
+#if !defined(NV_GET_USER_PAGES_HAS_VMAS_ARG)
     #define NV_GET_USER_PAGES get_user_pages
-#elif defined(NV_GET_USER_PAGES_HAS_ARGS_FLAGS_VMAS)
+#else
     #define NV_GET_USER_PAGES(start, nr_pages, flags, pages) \
         get_user_pages(start, nr_pages, flags, pages, NULL)
-#elif defined(NV_GET_USER_PAGES_HAS_ARGS_TSK_FLAGS_VMAS)
-    #define NV_GET_USER_PAGES(start, nr_pages, flags, pages) \
-        get_user_pages(current, current->mm, start, nr_pages, flags, pages, NULL)
-#else
-    static inline long NV_GET_USER_PAGES(unsigned long start,
-                                         unsigned long nr_pages,
-                                         unsigned int flags,
-                                         struct page **pages)
-    {
-        int write = flags & FOLL_WRITE;
-        int force = flags & FOLL_FORCE;
+#endif
 
-    #if defined(NV_GET_USER_PAGES_HAS_ARGS_WRITE_FORCE_VMAS)
-        return get_user_pages(start, nr_pages, write, force, pages, NULL);
-    #else
-        // NV_GET_USER_PAGES_HAS_ARGS_TSK_WRITE_FORCE_VMAS
-        return get_user_pages(current, current->mm, start, nr_pages, write,
-                              force, pages, NULL);
-    #endif // NV_GET_USER_PAGES_HAS_ARGS_WRITE_FORCE_VMAS
-    }
-#endif // NV_GET_USER_PAGES_HAS_ARGS_FLAGS
-
-/* pin_user_pages_remote
+/*
+ * pin_user_pages_remote()
  *
- * pin_user_pages_remote() was added by commit eddb1c228f7951d399240
- * ("mm/gup: introduce pin_user_pages*() and FOLL_PIN") in v5.6 (2020-01-30)
+ * pin_user_pages_remote() was added by commit eddb1c228f79
+ * ("mm/gup: introduce pin_user_pages*() and FOLL_PIN") in v5.6.
  *
  * pin_user_pages_remote() removed 'tsk' parameter by commit
- * 64019a2e467a ("mm/gup: remove task_struct pointer for  all gup code")
- * in v5.9-rc1 (2020-08-11). *
+ * 64019a2e467a ("mm/gup: remove task_struct pointer for all gup code")
+ * in v5.9.
  *
  * Removed unused vmas parameter from pin_user_pages_remote() by commit
- * 83bcc2e132("mm/gup: remove unused vmas parameter from pin_user_pages_remote()")
- * in linux-next, expected in v6.5-rc1 (2023-05-14)
+ * 0b295316b3a9 ("mm/gup: remove unused vmas parameter from
+ * pin_user_pages_remote()") in v6.5.
  *
  */
 
@@ -142,104 +109,31 @@ typedef int vm_fault_t;
 #endif // NV_PIN_USER_PAGES_REMOTE_PRESENT
 
 /*
- * get_user_pages_remote() was added by commit 1e9877902dc7
- * ("mm/gup: Introduce get_user_pages_remote()") in v4.6 (2016-02-12).
- *
  * Note that get_user_pages_remote() requires the caller to hold a reference on
- * the task_struct (if non-NULL and if this API has tsk argument) and the mm_struct.
+ * the mm_struct.
  * This will always be true when using current and current->mm. If the kernel passes
  * the driver a vma via driver callback, the kernel holds a reference on vma->vm_mm
  * over that callback.
  *
- * get_user_pages_remote() write/force parameters were replaced
- * with gup_flags by commit 9beae1ea8930 ("mm: replace get_user_pages_remote()
- * write/force parameters with gup_flags") in v4.9 (2016-10-13).
- *
- * get_user_pages_remote() added 'locked' parameter by commit 5b56d49fc31d
- * ("mm: add locked parameter to get_user_pages_remote()") in
- * v4.10 (2016-12-14).
- *
  * get_user_pages_remote() removed 'tsk' parameter by
  * commit 64019a2e467a ("mm/gup: remove task_struct pointer for
- * all gup code") in v5.9-rc1 (2020-08-11).
+ * all gup code") in v5.9.
  *
- * Removed vmas parameter from get_user_pages_remote() by commit a4bde14d549 
- * ("mm/gup: remove vmas parameter from get_user_pages_remote()")
- * in linux-next, expected in v6.5-rc1 (2023-05-14)
+ * Removed vmas parameter from get_user_pages_remote() by commit ca5e863233e8
+ * ("mm/gup: remove vmas parameter from get_user_pages_remote()") in v6.5.
  *
  */
 
-#if defined(NV_GET_USER_PAGES_REMOTE_PRESENT)
-    #if defined(NV_GET_USER_PAGES_REMOTE_HAS_ARGS_FLAGS_LOCKED)
-        #define NV_GET_USER_PAGES_REMOTE get_user_pages_remote
+#if defined(NV_GET_USER_PAGES_REMOTE_HAS_ARGS_TSK_VMAS)
+    #define NV_GET_USER_PAGES_REMOTE(mm, start, nr_pages, flags, pages, locked) \
+        get_user_pages_remote(NULL, mm, start, nr_pages, flags, pages, NULL, locked)
+#elif defined(NV_GET_USER_PAGES_REMOTE_HAS_ARGS_TSK)
+    #define NV_GET_USER_PAGES_REMOTE(mm, start, nr_pages, flags, pages, locked) \
+        get_user_pages_remote(mm, start, nr_pages, flags, pages, NULL, locked)
 
-    #elif defined(NV_GET_USER_PAGES_REMOTE_HAS_ARGS_FLAGS_LOCKED_VMAS)
-        #define NV_GET_USER_PAGES_REMOTE(mm, start, nr_pages, flags, pages, locked) \
-            get_user_pages_remote(mm, start, nr_pages, flags, pages, NULL, locked)
-
-    #elif defined(NV_GET_USER_PAGES_REMOTE_HAS_ARGS_TSK_FLAGS_LOCKED_VMAS)
-        #define NV_GET_USER_PAGES_REMOTE(mm, start, nr_pages, flags, pages, locked) \
-            get_user_pages_remote(NULL, mm, start, nr_pages, flags, pages, NULL, locked)
-
-    #elif defined(NV_GET_USER_PAGES_REMOTE_HAS_ARGS_TSK_FLAGS_VMAS)
-        #define NV_GET_USER_PAGES_REMOTE(mm, start, nr_pages, flags, pages, locked) \
-            get_user_pages_remote(NULL, mm, start, nr_pages, flags, pages, NULL)
-
-    #else
-        // NV_GET_USER_PAGES_REMOTE_HAS_ARGS_TSK_WRITE_FORCE_VMAS
-        static inline long NV_GET_USER_PAGES_REMOTE(struct mm_struct *mm,
-                                                    unsigned long start,
-                                                    unsigned long nr_pages,
-                                                    unsigned int flags,
-                                                    struct page **pages,
-                                                    int *locked)
-        {
-            int write = flags & FOLL_WRITE;
-            int force = flags & FOLL_FORCE;
-
-            return get_user_pages_remote(NULL, mm, start, nr_pages, write, force,
-                                         pages, NULL);
-        }
-    #endif // NV_GET_USER_PAGES_REMOTE_HAS_ARGS_FLAGS_LOCKED
 #else
-    #if defined(NV_GET_USER_PAGES_HAS_ARGS_TSK_WRITE_FORCE_VMAS)
-        static inline long NV_GET_USER_PAGES_REMOTE(struct mm_struct *mm,
-                                                    unsigned long start,
-                                                    unsigned long nr_pages,
-                                                    unsigned int flags,
-                                                    struct page **pages,
-                                                    int *locked)
-        {
-            int write = flags & FOLL_WRITE;
-            int force = flags & FOLL_FORCE;
-
-            return get_user_pages(NULL, mm, start, nr_pages, write, force, pages, NULL);
-        }
-
-    #else
-        #define NV_GET_USER_PAGES_REMOTE(mm, start, nr_pages, flags, pages, locked) \
-            get_user_pages(NULL, mm, start, nr_pages, flags, pages, NULL)
-    #endif // NV_GET_USER_PAGES_HAS_ARGS_TSK_WRITE_FORCE_VMAS
-#endif // NV_GET_USER_PAGES_REMOTE_PRESENT
-
-/*
- * The .virtual_address field was effectively renamed to .address, by these
- * two commits:
- *
- *  struct vm_fault: .address was added by:
- *   2016-12-14  82b0f8c39a3869b6fd2a10e180a862248736ec6f
- *
- *  struct vm_fault: .virtual_address was removed by:
- *   2016-12-14  1a29d85eb0f19b7d8271923d8917d7b4f5540b3e
- */
-static inline unsigned long nv_page_fault_va(struct vm_fault *vmf)
-{
-#if defined(NV_VM_FAULT_HAS_ADDRESS)
-    return vmf->address;
-#else
-    return (unsigned long)(vmf->virtual_address);
+    #define NV_GET_USER_PAGES_REMOTE get_user_pages_remote
 #endif
-}
 
 static inline void nv_mmap_read_lock(struct mm_struct *mm)
 {
@@ -295,9 +189,21 @@ static inline struct rw_semaphore *nv_mmap_get_lock(struct mm_struct *mm)
 #endif
 }
 
+#define NV_CAN_CALL_VMA_START_WRITE 1
+
+#if !NV_CAN_CALL_VMA_START_WRITE
+/*
+ * Commit 45ad9f5290dc updated vma_start_write() to call __vma_start_write().
+ */
+void nv_vma_start_write(struct vm_area_struct *);
+#endif
+
 static inline void nv_vm_flags_set(struct vm_area_struct *vma, vm_flags_t flags)
 {
-#if defined(NV_VM_AREA_STRUCT_HAS_CONST_VM_FLAGS)
+#if !NV_CAN_CALL_VMA_START_WRITE
+    nv_vma_start_write(vma);
+    ACCESS_PRIVATE(vma, __vm_flags) |= flags;
+#elif defined(NV_VM_AREA_STRUCT_HAS_CONST_VM_FLAGS)
     vm_flags_set(vma, flags);
 #else
     vma->vm_flags |= flags;
@@ -306,7 +212,10 @@ static inline void nv_vm_flags_set(struct vm_area_struct *vma, vm_flags_t flags)
 
 static inline void nv_vm_flags_clear(struct vm_area_struct *vma, vm_flags_t flags)
 {
-#if defined(NV_VM_AREA_STRUCT_HAS_CONST_VM_FLAGS)
+#if !NV_CAN_CALL_VMA_START_WRITE
+    nv_vma_start_write(vma);
+    ACCESS_PRIVATE(vma, __vm_flags) &= ~flags;
+#elif defined(NV_VM_AREA_STRUCT_HAS_CONST_VM_FLAGS)
     vm_flags_clear(vma, flags);
 #else
     vma->vm_flags &= ~flags;

kernel-open/common/inc/nv-msi.h

@@ -26,8 +26,7 @@
 
 #include "nv-linux.h"
 
-#if (defined(CONFIG_X86_LOCAL_APIC) || defined(NVCPU_AARCH64) || \
-     defined(NVCPU_PPC64LE)) && \
+#if (defined(CONFIG_X86_LOCAL_APIC) || defined(NVCPU_AARCH64)) && \
     (defined(CONFIG_PCI_MSI) || defined(CONFIG_PCI_USE_VECTOR))
 #define NV_LINUX_PCIE_MSI_SUPPORTED
 #endif
@@ -87,12 +86,6 @@ static inline int nv_pci_enable_msix(nv_linux_state_t *nvl, int nvec)
 {
     int rc = 0;
 
-    /*
-     * pci_enable_msix_range() replaced pci_enable_msix() in 3.14-rc1:
-     * 2014-01-03  302a2523c277bea0bbe8340312b09507905849ed
-     */
-
-#if defined(NV_PCI_ENABLE_MSIX_RANGE_PRESENT)
     // We require all the vectors we are requesting so use the same min and max
     rc = pci_enable_msix_range(nvl->pci_dev, nvl->msix_entries, nvec, nvec);
     if (rc < 0)
@@ -100,13 +93,6 @@ static inline int nv_pci_enable_msix(nv_linux_state_t *nvl, int nvec)
         return NV_ERR_OPERATING_SYSTEM;
     }
     WARN_ON(nvec != rc);
-#else
-    rc = pci_enable_msix(nvl->pci_dev, nvl->msix_entries, nvec);
-    if (rc != 0)
-    {
-        return NV_ERR_OPERATING_SYSTEM;
-    }
-#endif
 
     nvl->num_intr = nvec;
     return NV_OK;

kernel-open/common/inc/nv-pci.h

@@ -1,5 +1,5 @@
 /*
- * SPDX-FileCopyrightText: Copyright (c) 2019 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+ * SPDX-FileCopyrightText: Copyright (c) 2019-2025 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
  * SPDX-License-Identifier: MIT
  *
  * Permission is hereby granted, free of charge, to any person obtaining a
@@ -36,5 +36,6 @@ int nv_pci_count_devices(void);
 NvU8 nv_find_pci_capability(struct pci_dev *, NvU8);
 int nvidia_dev_get_pci_info(const NvU8 *, struct pci_dev **, NvU64 *, NvU64 *);
 nv_linux_state_t * find_pci(NvU32, NvU8, NvU8, NvU8);
-
+NvBool nv_pci_is_valid_topology_for_direct_pci(nv_state_t *, struct pci_dev *);
+NvBool nv_pci_has_common_pci_switch(nv_state_t *nv, struct pci_dev *);
 #endif

kernel-open/common/inc/nv-pgprot.h

@@ -1,5 +1,5 @@
 /*
- * SPDX-FileCopyrightText: Copyright (c) 2015-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+ * SPDX-FileCopyrightText: Copyright (c) 2015-2025 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
  * SPDX-License-Identifier: MIT
  *
  * Permission is hereby granted, free of charge, to any person obtaining a
@@ -63,16 +63,10 @@ static inline pgprot_t pgprot_modify_writecombine(pgprot_t old_prot)
 extern NvBool nvos_is_chipset_io_coherent(void);
 /*
  * Don't rely on the kernel's definition of pgprot_noncached(), as on 64-bit
- * ARM that's not for system memory, but device memory instead. For I/O cache
- * coherent systems, use cached mappings instead of uncached.
+ * ARM that's not for system memory, but device memory instead.
  */
 #define NV_PGPROT_UNCACHED(old_prot)   \
-    ((nvos_is_chipset_io_coherent()) ? \
-     (old_prot) :                      \
-     __pgprot_modify((old_prot), PTE_ATTRINDX_MASK, PTE_ATTRINDX(MT_NORMAL_NC)))
-#elif defined(NVCPU_PPC64LE)
-/* Don't attempt to mark sysmem pages as uncached on ppc64le */
-#define NV_PGPROT_UNCACHED(old_prot)          old_prot
+     __pgprot_modify((old_prot), PTE_ATTRINDX_MASK, PTE_ATTRINDX(MT_NORMAL_NC))
 #else
 #define NV_PGPROT_UNCACHED(old_prot)          pgprot_noncached(old_prot)
 #endif
@@ -94,32 +88,6 @@ extern NvBool nvos_is_chipset_io_coherent(void);
     NV_PGPROT_WRITE_COMBINED_DEVICE(old_prot)
 #define NV_PGPROT_READ_ONLY(old_prot)                                         \
     __pgprot(pgprot_val((old_prot)) & ~_PAGE_RW)
-#elif defined(NVCPU_PPC64LE)
-/*
- * Some kernels use H_PAGE instead of _PAGE
- */
-#if defined(_PAGE_RW)
-#define NV_PAGE_RW _PAGE_RW
-#elif defined(H_PAGE_RW)
-#define NV_PAGE_RW H_PAGE_RW
-#else
-#warning "The kernel does not provide page protection defines!"
-#endif
-
-#if defined(_PAGE_4K_PFN)
-#define NV_PAGE_4K_PFN _PAGE_4K_PFN
-#elif defined(H_PAGE_4K_PFN)
-#define NV_PAGE_4K_PFN H_PAGE_4K_PFN
-#else
-#undef NV_PAGE_4K_PFN
-#endif
-
-#define NV_PGPROT_WRITE_COMBINED_DEVICE(old_prot)                             \
-    pgprot_writecombine(old_prot)
-/* Don't attempt to mark sysmem pages as write combined on ppc64le */
-#define NV_PGPROT_WRITE_COMBINED(old_prot)    old_prot
-#define NV_PGPROT_READ_ONLY(old_prot)                                         \
-    __pgprot(pgprot_val((old_prot)) & ~NV_PAGE_RW)
 #elif defined(NVCPU_RISCV64)
 #define NV_PGPROT_WRITE_COMBINED_DEVICE(old_prot)                             \
     pgprot_writecombine(old_prot)

kernel-open/common/inc/nv-platform.h

@@ -0,0 +1,48 @@
+/*
+ * SPDX-FileCopyrightText: Copyright (c) 2019-2025 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+ * SPDX-License-Identifier: MIT
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
+ * DEALINGS IN THE SOFTWARE.
+ */
+
+#ifndef NV_PLATFORM_H
+#define NV_PLATFORM_H
+
+#include "nv-linux.h"
+
+irqreturn_t     nvidia_isr              (int, void *);
+irqreturn_t     nvidia_isr_kthread_bh   (int, void *);
+
+int       nv_platform_register_driver(void);
+void      nv_platform_unregister_driver(void);
+int       nv_platform_count_devices(void);
+int       nv_soc_register_irqs(nv_state_t *nv);
+void      nv_soc_free_irqs(nv_state_t *nv);
+
+#define NV_SUPPORTS_PLATFORM_DEVICE NV_IS_EXPORT_SYMBOL_PRESENT___platform_driver_register
+
+#if defined(NV_LINUX_PLATFORM_TEGRA_DCE_DCE_CLIENT_IPC_H_PRESENT)
+#define NV_SUPPORTS_DCE_CLIENT_IPC 1
+#else
+#define NV_SUPPORTS_DCE_CLIENT_IPC 0
+#endif
+
+#define NV_SUPPORTS_PLATFORM_DISPLAY_DEVICE (NV_SUPPORTS_PLATFORM_DEVICE && NV_SUPPORTS_DCE_CLIENT_IPC)
+
+#endif

kernel-open/common/inc/nv-proto.h

@@ -1,5 +1,5 @@
 /*
- * SPDX-FileCopyrightText: Copyright (c) 1999-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+ * SPDX-FileCopyrightText: Copyright (c) 1999-2024 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
  * SPDX-License-Identifier: MIT
  *
  * Permission is hereby granted, free of charge, to any person obtaining a
@@ -25,6 +25,7 @@
 #define _NV_PROTO_H_
 
 #include "nv-pci.h"
+#include "nv-platform.h"
 
 extern const char *nv_device_name;
 
@@ -41,7 +42,7 @@ void        nv_procfs_remove_gpu        (nv_linux_state_t *);
 
 int         nvidia_mmap                 (struct file *, struct vm_area_struct *);
 int         nvidia_mmap_helper          (nv_state_t *, nv_linux_file_private_t *, nvidia_stack_t *, struct vm_area_struct *, void *);
-int         nv_encode_caching           (pgprot_t *, NvU32, NvU32);
+int         nv_encode_caching           (pgprot_t *, NvU32, nv_memory_type_t);
 void        nv_revoke_gpu_mappings_locked(nv_state_t *);
 
 NvUPtr      nv_vm_map_pages             (struct page **, NvU32, NvBool, NvBool);
@@ -59,6 +60,8 @@ NV_STATUS   nv_uvm_resume               (void);
 void        nv_uvm_notify_start_device  (const NvU8 *uuid);
 void        nv_uvm_notify_stop_device   (const NvU8 *uuid);
 NV_STATUS   nv_uvm_event_interrupt      (const NvU8 *uuid);
+NV_STATUS   nv_uvm_drain_P2P            (const NvU8 *uuid);
+NV_STATUS   nv_uvm_resume_P2P           (const NvU8 *uuid);
 
 /* Move these to nv.h once implemented by other UNIX platforms */
 NvBool      nvidia_get_gpuid_list       (NvU32 *gpu_ids, NvU32 *gpu_count);

kernel-open/common/inc/nv-time.h

@@ -36,13 +36,6 @@
 #define NV_MAX_ISR_DELAY_MS           (NV_MAX_ISR_DELAY_US / 1000)
 #define NV_NSECS_TO_JIFFIES(nsec)     ((nsec) * HZ / 1000000000)
 
-#if !defined(NV_TIMESPEC64_PRESENT)
-struct timespec64 {
-    __s64 tv_sec;
-    long  tv_nsec;
-};
-#endif
-
 #if !defined(NV_KTIME_GET_RAW_TS64_PRESENT)
 static inline void ktime_get_raw_ts64(struct timespec64 *ts64)
 {
@@ -53,16 +46,6 @@ static inline void ktime_get_raw_ts64(struct timespec64 *ts64)
 }
 #endif
 
-#if !defined(NV_KTIME_GET_REAL_TS64_PRESENT)
-static inline void ktime_get_real_ts64(struct timespec64 *ts64)
-{
-    struct timeval tv;
-    do_gettimeofday(&tv);
-    ts64->tv_sec = tv.tv_sec;
-    ts64->tv_nsec = tv.tv_usec * (NvU64) NSEC_PER_USEC;
-}
-#endif
-
 static NvBool nv_timer_less_than
 (
     const struct timespec64 *a,
@@ -73,49 +56,6 @@ static NvBool nv_timer_less_than
                                     : (a->tv_sec < b->tv_sec);
 }
 
-#if !defined(NV_TIMESPEC64_PRESENT)
-static inline struct timespec64 timespec64_add
-(
-    const struct timespec64    a,
-    const struct timespec64    b
-)
-{
-    struct timespec64 result;
-
-    result.tv_sec = a.tv_sec + b.tv_sec;
-    result.tv_nsec = a.tv_nsec + b.tv_nsec;
-    while (result.tv_nsec >= NSEC_PER_SEC)
-    {
-        ++result.tv_sec;
-        result.tv_nsec -= NSEC_PER_SEC;
-    }
-    return result;
-}
-
-static inline struct timespec64  timespec64_sub
-(
-    const struct timespec64    a,
-    const struct timespec64    b
-)
-{
-    struct timespec64 result;
-
-    result.tv_sec = a.tv_sec - b.tv_sec;
-    result.tv_nsec = a.tv_nsec - b.tv_nsec;
-    while (result.tv_nsec < 0)
-    {
-        --(result.tv_sec);
-        result.tv_nsec += NSEC_PER_SEC;
-    }
-    return result;
-}
-
-static inline s64 timespec64_to_ns(struct timespec64 *ts)
-{
-    return ((s64) ts->tv_sec *  NSEC_PER_SEC) + ts->tv_nsec;
-}
-#endif
-
 static inline NvU64 nv_ktime_get_raw_ns(void)
 {
     struct timespec64 ts;

kernel-open/common/inc/nv-timer.h

@@ -1,5 +1,5 @@
 /*
- * SPDX-FileCopyrightText: Copyright (c) 2017 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+ * SPDX-FileCopyrightText: Copyright (c) 2017-2025 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
  * SPDX-License-Identifier: MIT
  *
  * Permission is hereby granted, free of charge, to any person obtaining a
@@ -42,24 +42,20 @@ static inline void nv_timer_callback_typed_data(struct timer_list *timer)
     nv_timer->nv_timer_callback(nv_timer);
 }
 
-static inline void nv_timer_callback_anon_data(unsigned long arg)
-{
-    struct nv_timer *nv_timer = (struct nv_timer *)arg;
-
-    nv_timer->nv_timer_callback(nv_timer);
-}
-
 static inline void nv_timer_setup(struct nv_timer *nv_timer,
                                   void (*callback)(struct nv_timer *nv_timer))
 {
     nv_timer->nv_timer_callback = callback;
 
-#if defined(NV_TIMER_SETUP_PRESENT)
     timer_setup(&nv_timer->kernel_timer, nv_timer_callback_typed_data, 0);
+}
+
+static inline void nv_timer_delete_sync(struct timer_list *timer)
+{
+#if !defined(NV_BSD) && NV_IS_EXPORT_SYMBOL_PRESENT_timer_delete_sync
+    timer_delete_sync(timer);
 #else
-    init_timer(&nv_timer->kernel_timer);
-    nv_timer->kernel_timer.function = nv_timer_callback_anon_data;
-    nv_timer->kernel_timer.data = (unsigned long)nv_timer;
+    del_timer_sync(timer);
 #endif
 }
 

kernel-open/common/inc/nv.h

@@ -1,5 +1,5 @@
 /*
- * SPDX-FileCopyrightText: Copyright (c) 1999-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+ * SPDX-FileCopyrightText: Copyright (c) 1999-2025 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
  * SPDX-License-Identifier: MIT
  *
  * Permission is hereby granted, free of charge, to any person obtaining a
@@ -44,11 +44,15 @@
 #include <nv-ioctl.h>
 #include <nv-ioctl-numa.h>
 #include <nvmisc.h>
+#include <os/nv_memory_area.h>
 
 extern nv_cap_t *nvidia_caps_root;
 
 extern const NvBool nv_is_rm_firmware_supported_os;
 
+#include <nvi2c.h>
+#include <nvimpshared.h>
+
 #include <nv-kernel-interface-api.h>
 
 #define GPU_UUID_LEN    (16)
@@ -82,6 +86,18 @@ extern const NvBool nv_is_rm_firmware_supported_os;
 
 #define NV_RM_DEVICE_INTR_ADDRESS 0x100
 
+/*
+ * Clock domain identifier, which is used for fetching the engine
+ * load backed by the specified clock domain for Tegra platforms
+ * conforming linux devfreq framework to realize dynamic frequency
+ * scaling.
+ */
+typedef enum _TEGRASOC_DEVFREQ_CLK
+{
+    TEGRASOC_DEVFREQ_CLK_GPC,
+    TEGRASOC_DEVFREQ_CLK_NVD,
+} TEGRASOC_DEVFREQ_CLK;
+
 /*!
  * @brief The order of the display clocks in the below defined enum
  * should be synced with below mapping array and macro.
@@ -104,27 +120,45 @@ typedef enum _TEGRASOC_WHICH_CLK
     TEGRASOC_WHICH_CLK_NVDISPLAY_DISP,
     TEGRASOC_WHICH_CLK_NVDISPLAY_P0,
     TEGRASOC_WHICH_CLK_NVDISPLAY_P1,
+    TEGRASOC_WHICH_CLK_NVDISPLAY_P2,
+    TEGRASOC_WHICH_CLK_NVDISPLAY_P3,
+    TEGRASOC_WHICH_CLK_NVDISPLAY_P4,
+    TEGRASOC_WHICH_CLK_NVDISPLAY_P5,
+    TEGRASOC_WHICH_CLK_NVDISPLAY_P6,
+    TEGRASOC_WHICH_CLK_NVDISPLAY_P7,
     TEGRASOC_WHICH_CLK_DPAUX0,
     TEGRASOC_WHICH_CLK_FUSE,
     TEGRASOC_WHICH_CLK_DSIPLL_VCO,
     TEGRASOC_WHICH_CLK_DSIPLL_CLKOUTPN,
     TEGRASOC_WHICH_CLK_DSIPLL_CLKOUTA,
     TEGRASOC_WHICH_CLK_SPPLL0_VCO,
-    TEGRASOC_WHICH_CLK_SPPLL0_CLKOUTPN,
     TEGRASOC_WHICH_CLK_SPPLL0_CLKOUTA,
     TEGRASOC_WHICH_CLK_SPPLL0_CLKOUTB,
+    TEGRASOC_WHICH_CLK_SPPLL0_CLKOUTPN,
+    TEGRASOC_WHICH_CLK_SPPLL1_CLKOUTPN,
+    TEGRASOC_WHICH_CLK_SPPLL0_DIV27,
+    TEGRASOC_WHICH_CLK_SPPLL1_DIV27,
     TEGRASOC_WHICH_CLK_SPPLL0_DIV10,
     TEGRASOC_WHICH_CLK_SPPLL0_DIV25,
-    TEGRASOC_WHICH_CLK_SPPLL0_DIV27,
     TEGRASOC_WHICH_CLK_SPPLL1_VCO,
-    TEGRASOC_WHICH_CLK_SPPLL1_CLKOUTPN,
-    TEGRASOC_WHICH_CLK_SPPLL1_DIV27,
     TEGRASOC_WHICH_CLK_VPLL0_REF,
     TEGRASOC_WHICH_CLK_VPLL0,
     TEGRASOC_WHICH_CLK_VPLL1,
+    TEGRASOC_WHICH_CLK_VPLL2,
+    TEGRASOC_WHICH_CLK_VPLL3,
+    TEGRASOC_WHICH_CLK_VPLL4,
+    TEGRASOC_WHICH_CLK_VPLL5,
+    TEGRASOC_WHICH_CLK_VPLL6,
+    TEGRASOC_WHICH_CLK_VPLL7,
     TEGRASOC_WHICH_CLK_NVDISPLAY_P0_REF,
     TEGRASOC_WHICH_CLK_RG0,
     TEGRASOC_WHICH_CLK_RG1,
+    TEGRASOC_WHICH_CLK_RG2,
+    TEGRASOC_WHICH_CLK_RG3,
+    TEGRASOC_WHICH_CLK_RG4,
+    TEGRASOC_WHICH_CLK_RG5,
+    TEGRASOC_WHICH_CLK_RG6,
+    TEGRASOC_WHICH_CLK_RG7,
     TEGRASOC_WHICH_CLK_DISPPLL,
     TEGRASOC_WHICH_CLK_DISPHUBPLL,
     TEGRASOC_WHICH_CLK_DSI_LP,
@@ -132,9 +166,20 @@ typedef enum _TEGRASOC_WHICH_CLK
     TEGRASOC_WHICH_CLK_DSI_PIXEL,
     TEGRASOC_WHICH_CLK_PRE_SOR0,
     TEGRASOC_WHICH_CLK_PRE_SOR1,
-    TEGRASOC_WHICH_CLK_DP_LINK_REF,
+    TEGRASOC_WHICH_CLK_PRE_SOR2,
+    TEGRASOC_WHICH_CLK_PRE_SOR3,
+    TEGRASOC_WHICH_CLK_DP_LINKA_REF,
+    TEGRASOC_WHICH_CLK_DP_LINKB_REF,
+    TEGRASOC_WHICH_CLK_DP_LINKC_REF,
+    TEGRASOC_WHICH_CLK_DP_LINKD_REF,
     TEGRASOC_WHICH_CLK_SOR_LINKA_INPUT,
+    TEGRASOC_WHICH_CLK_SOR_LINKB_INPUT,
+    TEGRASOC_WHICH_CLK_SOR_LINKC_INPUT,
+    TEGRASOC_WHICH_CLK_SOR_LINKD_INPUT,
     TEGRASOC_WHICH_CLK_SOR_LINKA_AFIFO,
+    TEGRASOC_WHICH_CLK_SOR_LINKB_AFIFO,
+    TEGRASOC_WHICH_CLK_SOR_LINKC_AFIFO,
+    TEGRASOC_WHICH_CLK_SOR_LINKD_AFIFO,
     TEGRASOC_WHICH_CLK_SOR_LINKA_AFIFO_M,
     TEGRASOC_WHICH_CLK_RG0_M,
     TEGRASOC_WHICH_CLK_RG1_M,
@@ -143,17 +188,36 @@ typedef enum _TEGRASOC_WHICH_CLK
     TEGRASOC_WHICH_CLK_PLLHUB,
     TEGRASOC_WHICH_CLK_SOR0,
     TEGRASOC_WHICH_CLK_SOR1,
-    TEGRASOC_WHICH_CLK_SOR_PAD_INPUT,
+    TEGRASOC_WHICH_CLK_SOR2,
+    TEGRASOC_WHICH_CLK_SOR3,
+    TEGRASOC_WHICH_CLK_SOR_PADA_INPUT,
+    TEGRASOC_WHICH_CLK_SOR_PADB_INPUT,
+    TEGRASOC_WHICH_CLK_SOR_PADC_INPUT,
+    TEGRASOC_WHICH_CLK_SOR_PADD_INPUT,
+    TEGRASOC_WHICH_CLK_SOR0_PAD,
+    TEGRASOC_WHICH_CLK_SOR1_PAD,
+    TEGRASOC_WHICH_CLK_SOR2_PAD,
+    TEGRASOC_WHICH_CLK_SOR3_PAD,
     TEGRASOC_WHICH_CLK_PRE_SF0,
     TEGRASOC_WHICH_CLK_SF0,
     TEGRASOC_WHICH_CLK_SF1,
+    TEGRASOC_WHICH_CLK_SF2,
+    TEGRASOC_WHICH_CLK_SF3,
+    TEGRASOC_WHICH_CLK_SF4,
+    TEGRASOC_WHICH_CLK_SF5,
+    TEGRASOC_WHICH_CLK_SF6,
+    TEGRASOC_WHICH_CLK_SF7,
     TEGRASOC_WHICH_CLK_DSI_PAD_INPUT,
     TEGRASOC_WHICH_CLK_PRE_SOR0_REF,
     TEGRASOC_WHICH_CLK_PRE_SOR1_REF,
     TEGRASOC_WHICH_CLK_SOR0_PLL_REF,
     TEGRASOC_WHICH_CLK_SOR1_PLL_REF,
+    TEGRASOC_WHICH_CLK_SOR2_PLL_REF,
+    TEGRASOC_WHICH_CLK_SOR3_PLL_REF,
     TEGRASOC_WHICH_CLK_SOR0_REF,
     TEGRASOC_WHICH_CLK_SOR1_REF,
+    TEGRASOC_WHICH_CLK_SOR2_REF,
+    TEGRASOC_WHICH_CLK_SOR3_REF,
     TEGRASOC_WHICH_CLK_OSC,
     TEGRASOC_WHICH_CLK_DSC,
     TEGRASOC_WHICH_CLK_MAUD,
@@ -167,6 +231,27 @@ typedef enum _TEGRASOC_WHICH_CLK
     TEGRASOC_WHICH_CLK_PLLA_DISP,
     TEGRASOC_WHICH_CLK_PLLA_DISPHUB,
     TEGRASOC_WHICH_CLK_PLLA,
+    TEGRASOC_WHICH_CLK_VPLLX_SOR0_MUXED,
+    TEGRASOC_WHICH_CLK_VPLLX_SOR1_MUXED,
+    TEGRASOC_WHICH_CLK_VPLLX_SOR2_MUXED,
+    TEGRASOC_WHICH_CLK_VPLLX_SOR3_MUXED,
+    TEGRASOC_WHICH_CLK_SF0_SOR,
+    TEGRASOC_WHICH_CLK_SF1_SOR,
+    TEGRASOC_WHICH_CLK_SF2_SOR,
+    TEGRASOC_WHICH_CLK_SF3_SOR,
+    TEGRASOC_WHICH_CLK_SF4_SOR,
+    TEGRASOC_WHICH_CLK_SF5_SOR,
+    TEGRASOC_WHICH_CLK_SF6_SOR,
+    TEGRASOC_WHICH_CLK_SF7_SOR,
+    TEGRASOC_WHICH_CLK_EMC,
+    TEGRASOC_WHICH_CLK_GPU_FIRST,
+    TEGRASOC_WHICH_CLK_GPU_SYS = TEGRASOC_WHICH_CLK_GPU_FIRST,
+    TEGRASOC_WHICH_CLK_GPU_NVD,
+    TEGRASOC_WHICH_CLK_GPU_UPROC,
+    TEGRASOC_WHICH_CLK_GPU_GPC0,
+    TEGRASOC_WHICH_CLK_GPU_GPC1,
+    TEGRASOC_WHICH_CLK_GPU_GPC2,
+    TEGRASOC_WHICH_CLK_GPU_LAST = TEGRASOC_WHICH_CLK_GPU_GPC2,
     TEGRASOC_WHICH_CLK_MAX, // TEGRASOC_WHICH_CLK_MAX is defined for boundary checks only.
 } TEGRASOC_WHICH_CLK;
 
@@ -279,11 +364,9 @@ typedef struct nv_usermap_access_params_s
     NvU64    offset;
     NvU64   *page_array;
     NvU64    num_pages;
-    NvU64    mmap_start;
-    NvU64    mmap_size;
+    MemoryArea memArea;
     NvU64    access_start;
     NvU64    access_size;
-    NvU64    remap_prot_extra;
     NvBool   contig;
     NvU32    caching;
 } nv_usermap_access_params_t;
@@ -296,11 +379,9 @@ typedef struct nv_alloc_mapping_context_s {
     NvU64  page_index;
     NvU64 *page_array;
     NvU64  num_pages;
-    NvU64  mmap_start;
-    NvU64  mmap_size;
+    MemoryArea memArea;
     NvU64  access_start;
     NvU64  access_size;
-    NvU64  remap_prot_extra;
     NvU32  prot;
     NvBool valid;
     NvU32  caching;
@@ -330,13 +411,11 @@ typedef struct nv_soc_irq_info_s {
     NvS32 ref_count;
 } nv_soc_irq_info_t;
 
-#define NV_MAX_SOC_IRQS              6
+#define NV_MAX_SOC_IRQS              10
 #define NV_MAX_DPAUX_NUM_DEVICES     4
-#define NV_MAX_SOC_DPAUX_NUM_DEVICES 2 // From SOC_DEV_MAPPING
 
-#define NV_IGPU_LEGACY_STALL_IRQ     70
-#define NV_IGPU_MAX_STALL_IRQS       3
-#define NV_IGPU_MAX_NONSTALL_IRQS    1
+#define NV_MAX_SOC_DPAUX_NUM_DEVICES 4
+
 /*
  * per device state
  */
@@ -367,6 +446,8 @@ typedef struct nv_state_t
     {
         NvBool         valid;
         NvU8           uuid[GPU_UUID_LEN];
+        NvBool         pci_uuid_read_attempted;
+        NV_STATUS      pci_uuid_status;
     } nv_uuid_cache;
     void *handle;
 
@@ -380,7 +461,6 @@ typedef struct nv_state_t
     nv_aperture_t *mipical_regs;
     nv_aperture_t *fb, ud;
     nv_aperture_t *simregs;
-    nv_aperture_t *emc_regs;
 
     NvU32  num_dpaux_instance;
     NvU32  interrupt_line;
@@ -394,12 +474,14 @@ typedef struct nv_state_t
     NvU32 soc_dcb_size;
     NvU32 disp_sw_soc_chip_id;
     NvBool soc_is_dpalt_mode_supported;
+    NvBool soc_is_hfrp_supported;
 
-    NvU32 igpu_stall_irq[NV_IGPU_MAX_STALL_IRQS];
-    NvU32 igpu_nonstall_irq;
-    NvU32 num_stall_irqs;
     NvU64 dma_mask;
-    
+
+    NvBool is_tegra_pci_igpu;
+    NvBool supports_tegra_igpu_rg;
+    NvBool is_tegra_pci_igpu_rg_enabled;
+
     NvBool primary_vga;
 
     NvU32 sim_env;
@@ -478,6 +560,15 @@ typedef struct nv_state_t
     /* Bool to check if the GPU has a coherent sysmem link */
     NvBool coherent;
 
+    /*
+     * Bool to check if GPU memory is backed by struct page.
+     * False for non-coherent platforms. May also be false
+     * on coherent platforms if GPU memory is not onlined to the kernel.
+     */
+    NvBool mem_has_struct_page;
+
+    /* OS detected GPU has ATS capability */
+    NvBool ats_support;
     /*
      * NUMA node ID of the CPU to which the GPU is attached.
      * Holds NUMA_NO_NODE on platforms that don't support NUMA configuration.
@@ -493,13 +584,13 @@ typedef struct nv_state_t
         NvU32 dispIsoStreamId;
         NvU32 dispNisoStreamId;
     } iommus;
-} nv_state_t;
 
-// These define need to be in sync with defines in system.h
-#define OS_TYPE_LINUX   0x1
-#define OS_TYPE_FREEBSD 0x2
-#define OS_TYPE_SUNOS   0x3
-#define OS_TYPE_VMWARE  0x4
+    /* Console is managed by drm drivers or NVKMS */
+    NvBool client_managed_console;
+
+    /* Bool to check if power management is supported */
+    NvBool is_pm_supported;
+} nv_state_t;
 
 #define NVFP_TYPE_NONE       0x0
 #define NVFP_TYPE_REFCOUNTED 0x1
@@ -539,16 +630,18 @@ typedef struct UvmGpuAddressSpaceInfo_tag           *nvgpuAddressSpaceInfo_t;
 typedef struct UvmGpuAllocInfo_tag                  *nvgpuAllocInfo_t;
 typedef struct UvmGpuP2PCapsParams_tag              *nvgpuP2PCapsParams_t;
 typedef struct UvmGpuFbInfo_tag                     *nvgpuFbInfo_t;
+typedef struct UvmGpuNvlinkInfo_tag                 *nvgpuNvlinkInfo_t;
 typedef struct UvmGpuEccInfo_tag                    *nvgpuEccInfo_t;
 typedef struct UvmGpuFaultInfo_tag                  *nvgpuFaultInfo_t;
 typedef struct UvmGpuAccessCntrInfo_tag             *nvgpuAccessCntrInfo_t;
-typedef struct UvmGpuAccessCntrConfig_tag           *nvgpuAccessCntrConfig_t;
-typedef struct UvmGpuInfo_tag                       nvgpuInfo_t;
-typedef struct UvmGpuClientInfo_tag                 nvgpuClientInfo_t;
+typedef struct UvmGpuAccessCntrConfig_tag            nvgpuAccessCntrConfig_t;
+typedef struct UvmGpuInfo_tag                        nvgpuInfo_t;
+typedef struct UvmGpuClientInfo_tag                  nvgpuClientInfo_t;
 typedef struct UvmPmaAllocationOptions_tag          *nvgpuPmaAllocationOptions_t;
 typedef struct UvmPmaStatistics_tag                 *nvgpuPmaStatistics_t;
 typedef struct UvmGpuMemoryInfo_tag                 *nvgpuMemoryInfo_t;
 typedef struct UvmGpuExternalMappingInfo_tag        *nvgpuExternalMappingInfo_t;
+typedef struct UvmGpuExternalPhysAddrInfo_tag       *nvgpuExternalPhysAddrInfo_t;
 typedef struct UvmGpuChannelResourceInfo_tag        *nvgpuChannelResourceInfo_t;
 typedef struct UvmGpuChannelInstanceInfo_tag        *nvgpuChannelInstanceInfo_t;
 typedef struct UvmGpuChannelResourceBindParams_tag  *nvgpuChannelResourceBindParams_t;
@@ -563,23 +656,23 @@ typedef NV_STATUS (*nvPmaEvictRangeCallback)(void *, NvU64, NvU64, nvgpuGpuMemor
  * flags
  */
 
-#define NV_FLAG_OPEN                   0x0001
-#define NV_FLAG_EXCLUDE                0x0002
-#define NV_FLAG_CONTROL                0x0004
-// Unused                              0x0008
-#define NV_FLAG_SOC_DISPLAY            0x0010
-#define NV_FLAG_USES_MSI               0x0020
-#define NV_FLAG_USES_MSIX              0x0040
-#define NV_FLAG_PASSTHRU               0x0080
-#define NV_FLAG_SUSPENDED              0x0100
-#define NV_FLAG_SOC_IGPU               0x0200
-// Unused                              0x0400
-#define NV_FLAG_PERSISTENT_SW_STATE    0x0800
-#define NV_FLAG_IN_RECOVERY            0x1000
-// Unused                              0x2000
-#define NV_FLAG_UNBIND_LOCK            0x4000
+#define NV_FLAG_OPEN                        0x0001
+#define NV_FLAG_EXCLUDE                     0x0002
+#define NV_FLAG_CONTROL                     0x0004
+#define NV_FLAG_PCI_P2P_UNSUPPORTED_CHIPSET 0x0008
+#define NV_FLAG_SOC_DISPLAY                 0x0010
+#define NV_FLAG_USES_MSI                    0x0020
+#define NV_FLAG_USES_MSIX                   0x0040
+#define NV_FLAG_PASSTHRU                    0x0080
+#define NV_FLAG_SUSPENDED                   0x0100
+/* To be set when an FLR needs to be triggered after device shut down. */
+#define NV_FLAG_TRIGGER_FLR                 0x0400
+#define NV_FLAG_PERSISTENT_SW_STATE         0x0800
+#define NV_FLAG_IN_RECOVERY                 0x1000
+#define NV_FLAG_PCI_REMOVE_IN_PROGRESS      0x2000
+#define NV_FLAG_UNBIND_LOCK                 0x4000
 /* To be set when GPU is not present on the bus, to help device teardown */
-#define NV_FLAG_IN_SURPRISE_REMOVAL    0x8000
+#define NV_FLAG_IN_SURPRISE_REMOVAL         0x8000
 
 typedef enum
 {
@@ -609,15 +702,30 @@ typedef enum
     NV_POWER_STATE_RUNNING
 } nv_power_state_t;
 
+typedef struct
+{
+    const char *vidmem_power_status;
+    const char *dynamic_power_status;
+    const char *gc6_support;
+    const char *gcoff_support;
+    const char *s0ix_status;
+    const char *db_support;
+} nv_power_info_t;
+
+typedef enum
+{
+    NV_MEMORY_TYPE_SYSTEM,      /* Memory mapped for ROM, SBIOS and physical RAM. */
+    NV_MEMORY_TYPE_REGISTERS,
+    NV_MEMORY_TYPE_FRAMEBUFFER,
+    NV_MEMORY_TYPE_DEVICE_MMIO, /* All kinds of MMIO referred by NVRM e.g. BARs and MCFG of device */
+} nv_memory_type_t;
+
 #define NV_PRIMARY_VGA(nv)      ((nv)->primary_vga)
 
 #define NV_IS_CTL_DEVICE(nv)    ((nv)->flags & NV_FLAG_CONTROL)
 #define NV_IS_SOC_DISPLAY_DEVICE(nv)    \
         ((nv)->flags & NV_FLAG_SOC_DISPLAY)
 
-#define NV_IS_SOC_IGPU_DEVICE(nv)    \
-        ((nv)->flags & NV_FLAG_SOC_IGPU)
-
 #define NV_IS_DEVICE_IN_SURPRISE_REMOVAL(nv)    \
         (((nv)->flags & NV_FLAG_IN_SURPRISE_REMOVAL) != 0)
 
@@ -752,6 +860,7 @@ static inline NvBool IS_IMEM_OFFSET(nv_state_t *nv, NvU64 offset, NvU64 length)
 #define NV_ALIGN_DOWN(v,g) ((v) & ~((g) - 1))
 #endif
 
+
 /*
  * driver internal interfaces
  */
@@ -766,9 +875,9 @@ static inline NvBool IS_IMEM_OFFSET(nv_state_t *nv, NvU64 offset, NvU64 length)
 
 NvU32      NV_API_CALL  nv_get_dev_minor         (nv_state_t *);
 void*      NV_API_CALL  nv_alloc_kernel_mapping  (nv_state_t *, void *, NvU64, NvU32, NvU64, void **);
-NV_STATUS  NV_API_CALL  nv_free_kernel_mapping   (nv_state_t *, void *, void *, void *);
+void       NV_API_CALL  nv_free_kernel_mapping   (nv_state_t *, void *, void *, void *);
 NV_STATUS  NV_API_CALL  nv_alloc_user_mapping    (nv_state_t *, void *, NvU64, NvU32, NvU64, NvU32, NvU64 *, void **);
-NV_STATUS  NV_API_CALL  nv_free_user_mapping     (nv_state_t *, void *, NvU64, void *);
+void       NV_API_CALL  nv_free_user_mapping     (nv_state_t *, void *, NvU64, void *);
 NV_STATUS  NV_API_CALL  nv_add_mapping_context_to_file (nv_state_t *, nv_usermap_access_params_t*, NvU32, void *, NvU64, NvU32);
 
 NvU64  NV_API_CALL  nv_get_kern_phys_address     (NvU64);
@@ -778,11 +887,11 @@ nv_state_t*  NV_API_CALL  nv_get_ctl_state       (void);
 
 void   NV_API_CALL  nv_set_dma_address_size      (nv_state_t *, NvU32 );
 
-NV_STATUS  NV_API_CALL  nv_alias_pages           (nv_state_t *, NvU32, NvU32, NvU32, NvU64, NvU64 *, void **);
+NV_STATUS  NV_API_CALL  nv_alias_pages           (nv_state_t *, NvU32, NvU64, NvU32, NvU32, NvU64, NvU64 *, NvBool, void **);
 NV_STATUS  NV_API_CALL  nv_alloc_pages           (nv_state_t *, NvU32, NvU64, NvBool, NvU32, NvBool, NvBool, NvS32, NvU64 *, void **);
 NV_STATUS  NV_API_CALL  nv_free_pages            (nv_state_t *, NvU32, NvBool, NvU32, void *);
 
-NV_STATUS  NV_API_CALL  nv_register_user_pages   (nv_state_t *, NvU64, NvU64 *, void *, void **);
+NV_STATUS  NV_API_CALL  nv_register_user_pages   (nv_state_t *, NvU64, NvU64 *, void *, void **, NvBool);
 void       NV_API_CALL  nv_unregister_user_pages (nv_state_t *, NvU64, void **, void **);
 
 NV_STATUS NV_API_CALL   nv_register_peer_io_mem  (nv_state_t *, NvU64 *, NvU64, void **);
@@ -790,7 +899,8 @@ void      NV_API_CALL   nv_unregister_peer_io_mem(nv_state_t *, void *);
 
 struct sg_table;
 
-NV_STATUS NV_API_CALL   nv_register_sgt          (nv_state_t *, NvU64 *, NvU64, NvU32, void **, struct sg_table *, void *);
+NV_STATUS NV_API_CALL   nv_register_sgt          (nv_state_t *, NvU64 *, NvU64, NvU32, void **,
+                                                  struct sg_table *, void *, NvBool);
 void      NV_API_CALL   nv_unregister_sgt        (nv_state_t *, struct sg_table **, void **, void *);
 NV_STATUS NV_API_CALL   nv_register_phys_pages   (nv_state_t *, NvU64 *, NvU64, NvU32, void **);
 void      NV_API_CALL   nv_unregister_phys_pages (nv_state_t *, void *);
@@ -801,13 +911,14 @@ NV_STATUS  NV_API_CALL  nv_dma_map_alloc         (nv_dma_device_t *, NvU64, NvU6
 NV_STATUS  NV_API_CALL  nv_dma_unmap_alloc       (nv_dma_device_t *, NvU64, NvU64 *, void **);
 
 NV_STATUS  NV_API_CALL  nv_dma_map_peer          (nv_dma_device_t *, nv_dma_device_t *, NvU8, NvU64, NvU64 *);
+NV_STATUS  NV_API_CALL  nv_dma_map_non_pci_peer  (nv_dma_device_t *, NvU64, NvU64 *);
 void       NV_API_CALL  nv_dma_unmap_peer        (nv_dma_device_t *, NvU64, NvU64);
 
 NV_STATUS  NV_API_CALL  nv_dma_map_mmio          (nv_dma_device_t *, NvU64, NvU64 *);
 void       NV_API_CALL  nv_dma_unmap_mmio        (nv_dma_device_t *, NvU64, NvU64);
 
 void       NV_API_CALL  nv_dma_cache_invalidate  (nv_dma_device_t *, void *);
-void       NV_API_CALL  nv_dma_enable_nvlink     (nv_dma_device_t *);
+NvBool     NV_API_CALL  nv_grdma_pci_topology_supported(nv_state_t *, nv_dma_device_t *);
 
 NvS32  NV_API_CALL  nv_start_rc_timer            (nv_state_t *);
 NvS32  NV_API_CALL  nv_stop_rc_timer             (nv_state_t *);
@@ -822,6 +933,7 @@ void   NV_API_CALL  nv_acpi_methods_init         (NvU32 *);
 void   NV_API_CALL  nv_acpi_methods_uninit       (void);
 
 NV_STATUS  NV_API_CALL  nv_acpi_method           (NvU32, NvU32, NvU32, void *, NvU16, NvU32 *, void *, NvU16 *);
+NV_STATUS  NV_API_CALL  nv_acpi_d3cold_dsm_for_upstream_port (nv_state_t *, NvU8 *, NvU32, NvU32, NvU32 *);
 NV_STATUS  NV_API_CALL  nv_acpi_dsm_method       (nv_state_t *, NvU8 *, NvU32, NvBool, NvU32, void *, NvU16, NvU32 *, void *, NvU16 *);
 NV_STATUS  NV_API_CALL  nv_acpi_ddc_method       (nv_state_t *, void *, NvU32 *, NvBool);
 NV_STATUS  NV_API_CALL  nv_acpi_dod_method       (nv_state_t *, NvU32 *, NvU32 *);
@@ -833,9 +945,7 @@ NV_STATUS  NV_API_CALL  nv_acpi_mux_method       (nv_state_t *, NvU32 *, NvU32,
 
 NV_STATUS  NV_API_CALL  nv_log_error             (nv_state_t *, NvU32, const char *, va_list);
 
-NvU64      NV_API_CALL  nv_get_dma_start_address (nv_state_t *);
 NV_STATUS  NV_API_CALL  nv_set_primary_vga_status(nv_state_t *);
-NV_STATUS  NV_API_CALL  nv_pci_trigger_recovery  (nv_state_t *);
 NvBool     NV_API_CALL  nv_requires_dma_remap    (nv_state_t *);
 
 NvBool     NV_API_CALL  nv_is_rm_firmware_active(nv_state_t *);
@@ -848,19 +958,8 @@ void               NV_API_CALL nv_put_file_private(void *);
 NV_STATUS NV_API_CALL nv_get_device_memory_config(nv_state_t *, NvU64 *, NvU64 *, NvU64 *, NvU32 *, NvS32 *);
 NV_STATUS NV_API_CALL nv_get_egm_info(nv_state_t *, NvU64 *, NvU64 *, NvS32 *);
 
-NV_STATUS NV_API_CALL nv_get_ibmnpu_genreg_info(nv_state_t *, NvU64 *, NvU64 *, void**);
-NV_STATUS NV_API_CALL nv_get_ibmnpu_relaxed_ordering_mode(nv_state_t *nv, NvBool *mode);
-
-void      NV_API_CALL nv_wait_for_ibmnpu_rsync(nv_state_t *nv);
-
-void      NV_API_CALL nv_ibmnpu_cache_flush_range(nv_state_t *nv, NvU64, NvU64);
-
 void      NV_API_CALL nv_p2p_free_platform_data(void *data);
 
-#if defined(NVCPU_PPC64LE)
-NV_STATUS NV_API_CALL nv_get_nvlink_line_rate    (nv_state_t *, NvU32 *);
-#endif
-
 NV_STATUS NV_API_CALL nv_revoke_gpu_mappings     (nv_state_t *);
 void      NV_API_CALL nv_acquire_mmap_lock       (nv_state_t *);
 void      NV_API_CALL nv_release_mmap_lock       (nv_state_t *);
@@ -883,8 +982,6 @@ void      NV_API_CALL nv_cap_drv_exit(void);
 NvBool    NV_API_CALL nv_is_gpu_accessible(nv_state_t *);
 NvBool    NV_API_CALL nv_match_gpu_os_info(nv_state_t *, void *);
 
-NvU32     NV_API_CALL nv_get_os_type(void);
-
 void      NV_API_CALL nv_get_updated_emu_seg(NvU32 *start, NvU32 *end);
 void      NV_API_CALL nv_get_screen_info(nv_state_t *, NvU64 *, NvU32 *, NvU32 *, NvU32 *, NvU32 *, NvU64 *);
 
@@ -894,15 +991,22 @@ struct drm_gem_object;
 
 NV_STATUS NV_API_CALL nv_dma_import_sgt  (nv_dma_device_t *, struct sg_table *, struct drm_gem_object *);
 void NV_API_CALL nv_dma_release_sgt(struct sg_table *, struct drm_gem_object *);
-NV_STATUS NV_API_CALL nv_dma_import_dma_buf      (nv_dma_device_t *, struct dma_buf *, NvU32 *, struct sg_table **, nv_dma_buf_t **);
-NV_STATUS NV_API_CALL nv_dma_import_from_fd      (nv_dma_device_t *, NvS32, NvU32 *, struct sg_table **, nv_dma_buf_t **);
+NV_STATUS NV_API_CALL nv_dma_import_dma_buf      (nv_dma_device_t *, struct dma_buf *, NvBool, NvU32 *, struct sg_table **, nv_dma_buf_t **);
+NV_STATUS NV_API_CALL nv_dma_import_from_fd      (nv_dma_device_t *, NvS32, NvBool, NvU32 *, struct sg_table **, nv_dma_buf_t **);
 void      NV_API_CALL nv_dma_release_dma_buf     (nv_dma_buf_t *);
 
 void      NV_API_CALL nv_schedule_uvm_isr        (nv_state_t *);
 
+NV_STATUS NV_API_CALL nv_schedule_uvm_drain_p2p  (NvU8 *);
+void      NV_API_CALL nv_schedule_uvm_resume_p2p (NvU8 *);
+
 NvBool    NV_API_CALL nv_platform_supports_s0ix  (void);
 NvBool    NV_API_CALL nv_s2idle_pm_configured    (void);
 
+NvBool    NV_API_CALL nv_pci_tegra_register_power_domain    (nv_state_t *, NvBool);
+NvBool    NV_API_CALL nv_pci_tegra_pm_init        (nv_state_t *);
+void      NV_API_CALL nv_pci_tegra_pm_deinit      (nv_state_t *);
+
 NvBool    NV_API_CALL nv_is_chassis_notebook      (void);
 void      NV_API_CALL nv_allow_runtime_suspend    (nv_state_t *nv);
 void      NV_API_CALL nv_disallow_runtime_suspend (nv_state_t *nv);
@@ -911,9 +1015,67 @@ typedef void (*nvTegraDceClientIpcCallback)(NvU32, NvU32, NvU32, void *, void *)
 
 NV_STATUS NV_API_CALL nv_get_num_phys_pages      (void *, NvU32 *);
 NV_STATUS NV_API_CALL nv_get_phys_pages          (void *, void *, NvU32 *);
-
 void      NV_API_CALL nv_get_disp_smmu_stream_ids (nv_state_t *, NvU32 *, NvU32 *);
 
+typedef struct TEGRA_IMP_IMPORT_DATA TEGRA_IMP_IMPORT_DATA;
+typedef struct nv_i2c_msg_s nv_i2c_msg_t;
+
+NV_STATUS NV_API_CALL nv_bpmp_send_mrq           (nv_state_t *, NvU32, const void *, NvU32, void *, NvU32, NvS32 *, NvS32 *);
+NV_STATUS NV_API_CALL nv_i2c_transfer(nv_state_t *, NvU32, NvU8, nv_i2c_msg_t *, int);
+void      NV_API_CALL nv_i2c_unregister_clients(nv_state_t *);
+NV_STATUS NV_API_CALL nv_i2c_bus_status(nv_state_t *, NvU32, NvS32 *, NvS32 *);
+NV_STATUS NV_API_CALL nv_imp_get_import_data     (TEGRA_IMP_IMPORT_DATA *);
+NV_STATUS NV_API_CALL nv_imp_enable_disable_rfl  (nv_state_t *nv, NvBool bEnable);
+NV_STATUS NV_API_CALL nv_imp_icc_set_bw          (nv_state_t *nv, NvU32 avg_bw_kbps, NvU32 floor_bw_kbps);
+NV_STATUS NV_API_CALL nv_get_num_dpaux_instances(nv_state_t *nv, NvU32 *num_instances);
+NV_STATUS NV_API_CALL nv_get_tegra_brightness_level(nv_state_t *, NvU32 *);
+NV_STATUS NV_API_CALL nv_set_tegra_brightness_level(nv_state_t *, NvU32);
+
+NV_STATUS NV_API_CALL nv_soc_device_reset        (nv_state_t *);
+NV_STATUS NV_API_CALL nv_soc_pm_powergate        (nv_state_t *);
+NV_STATUS NV_API_CALL nv_soc_pm_unpowergate      (nv_state_t *);
+NV_STATUS NV_API_CALL nv_gpio_get_pin_state(nv_state_t *, NvU32, NvU32 *);
+void      NV_API_CALL nv_gpio_set_pin_state(nv_state_t *, NvU32, NvU32);
+NV_STATUS NV_API_CALL nv_gpio_set_pin_direction(nv_state_t *, NvU32, NvU32);
+NV_STATUS NV_API_CALL nv_gpio_get_pin_direction(nv_state_t *, NvU32, NvU32 *);
+NV_STATUS NV_API_CALL nv_gpio_get_pin_number(nv_state_t *, NvU32, NvU32 *);
+NvBool    NV_API_CALL nv_gpio_get_pin_interrupt_status(nv_state_t *, NvU32, NvU32);
+NV_STATUS NV_API_CALL nv_gpio_set_pin_interrupt(nv_state_t *, NvU32, NvU32);
+NvU32     NV_API_CALL nv_tegra_get_rm_interface_type(NvU32);
+NV_STATUS NV_API_CALL nv_tegra_dce_register_ipc_client(NvU32, void *, nvTegraDceClientIpcCallback, NvU32 *);
+NV_STATUS NV_API_CALL nv_tegra_dce_client_ipc_send_recv(NvU32, void *, NvU32);
+NV_STATUS NV_API_CALL nv_tegra_dce_unregister_ipc_client(NvU32);
+NV_STATUS NV_API_CALL nv_dsi_parse_panel_props(nv_state_t *, void *);
+NvBool    NV_API_CALL nv_dsi_is_panel_connected(nv_state_t *);
+NV_STATUS NV_API_CALL nv_dsi_panel_enable(nv_state_t *, void *);
+NV_STATUS NV_API_CALL nv_dsi_panel_reset(nv_state_t *, void *);
+void      NV_API_CALL nv_dsi_panel_disable(nv_state_t *, void *);
+void      NV_API_CALL nv_dsi_panel_cleanup(nv_state_t *, void *);
+NV_STATUS NV_API_CALL nv_soc_mipi_cal_reset(nv_state_t *);
+NvU32     NV_API_CALL nv_soc_fuse_register_read (NvU32 addr);
+NvBool    NV_API_CALL nv_get_hdcp_enabled(nv_state_t *nv);
+NV_STATUS NV_API_CALL nv_get_valid_window_head_mask(nv_state_t *nv, NvU64 *);
+NV_STATUS NV_API_CALL nv_dp_uphy_pll_init(nv_state_t *, NvU32, NvU32);
+NV_STATUS NV_API_CALL nv_dp_uphy_pll_deinit(nv_state_t *);
+NV_STATUS NV_API_CALL nv_soc_i2c_hsp_semaphore_acquire(NvU32 ownerId, NvBool bAcquire, NvU64 timeout);
+typedef void (*nv_soc_tsec_cb_func_t)(void*, void*);
+NvU32     NV_API_CALL nv_soc_tsec_send_cmd(void* cmd, nv_soc_tsec_cb_func_t cb_func, void* cb_context);
+NvU32     NV_API_CALL nv_soc_tsec_event_register(nv_soc_tsec_cb_func_t cb_func, void* cb_context, NvBool is_init_event);
+NvU32     NV_API_CALL nv_soc_tsec_event_unregister(NvBool is_init_event);
+void*     NV_API_CALL nv_soc_tsec_alloc_mem_desc(NvU32 num_bytes, NvU32 *flcn_addr);
+void      NV_API_CALL nv_soc_tsec_free_mem_desc(void *mem_desc);
+NvBool    NV_API_CALL nv_is_clk_enabled          (nv_state_t *, TEGRASOC_WHICH_CLK);
+NV_STATUS NV_API_CALL nv_set_parent              (nv_state_t *, TEGRASOC_WHICH_CLK, TEGRASOC_WHICH_CLK);
+NV_STATUS NV_API_CALL nv_get_parent              (nv_state_t *, TEGRASOC_WHICH_CLK, TEGRASOC_WHICH_CLK*);
+NV_STATUS NV_API_CALL nv_clk_get_handles         (nv_state_t *);
+void      NV_API_CALL nv_clk_clear_handles       (nv_state_t *);
+NV_STATUS NV_API_CALL nv_enable_clk              (nv_state_t *, TEGRASOC_WHICH_CLK);
+void      NV_API_CALL nv_disable_clk             (nv_state_t *, TEGRASOC_WHICH_CLK);
+NV_STATUS NV_API_CALL nv_get_curr_freq           (nv_state_t *, TEGRASOC_WHICH_CLK, NvU32 *);
+NV_STATUS NV_API_CALL nv_get_max_freq            (nv_state_t *, TEGRASOC_WHICH_CLK, NvU32 *);
+NV_STATUS NV_API_CALL nv_get_min_freq            (nv_state_t *, TEGRASOC_WHICH_CLK, NvU32 *);
+NV_STATUS NV_API_CALL nv_set_freq                (nv_state_t *, TEGRASOC_WHICH_CLK, NvU32);
+
 /*
  * ---------------------------------------------------------------------------
  *
@@ -941,6 +1103,7 @@ NvBool     NV_API_CALL  rm_isr                   (nvidia_stack_t *, nv_state_t *
 void       NV_API_CALL  rm_isr_bh                (nvidia_stack_t *, nv_state_t *);
 void       NV_API_CALL  rm_isr_bh_unlocked       (nvidia_stack_t *, nv_state_t *);
 NvBool     NV_API_CALL  rm_is_msix_allowed       (nvidia_stack_t *, nv_state_t *);
+NV_STATUS  NV_API_CALL  rm_pmu_perfmon_get_load  (nvidia_stack_t *, nv_state_t *, NvU32 *, TEGRASOC_DEVFREQ_CLK);
 NV_STATUS  NV_API_CALL  rm_power_management      (nvidia_stack_t *, nv_state_t *, nv_pm_action_t);
 NV_STATUS  NV_API_CALL  rm_stop_user_channels    (nvidia_stack_t *, nv_state_t *);
 NV_STATUS  NV_API_CALL  rm_restart_user_channels (nvidia_stack_t *, nv_state_t *);
@@ -988,20 +1151,28 @@ void       NV_API_CALL  rm_request_dnotifier_state          (nvidia_stack_t *, n
 void       NV_API_CALL  rm_disable_gpu_state_persistence    (nvidia_stack_t *sp, nv_state_t *);
 NV_STATUS  NV_API_CALL  rm_p2p_init_mapping       (nvidia_stack_t *, NvU64, NvU64 *, NvU64 *, NvU64 *, NvU64 *, NvU64, NvU64, NvU64, NvU64, void (*)(void *), void *);
 NV_STATUS  NV_API_CALL  rm_p2p_destroy_mapping    (nvidia_stack_t *, NvU64);
-NV_STATUS  NV_API_CALL  rm_p2p_get_pages          (nvidia_stack_t *, NvU64, NvU32, NvU64, NvU64, NvU64 *, NvU32 *, NvU32 *, NvU32 *, NvU8 **, void *);
+NV_STATUS  NV_API_CALL  rm_p2p_get_pages          (nvidia_stack_t *, NvU64, NvU32, NvU64, NvU64, NvU64 *, NvU32 *, NvU32 *, NvU32 *, NvU8 **, void *, NvBool *);
 NV_STATUS  NV_API_CALL  rm_p2p_get_gpu_info       (nvidia_stack_t *, NvU64, NvU64, NvU8 **, void **);
-NV_STATUS  NV_API_CALL  rm_p2p_get_pages_persistent (nvidia_stack_t *,  NvU64, NvU64, void **, NvU64 *, NvU32 *, void *, void *);
+NV_STATUS  NV_API_CALL  rm_p2p_get_pages_persistent (nvidia_stack_t *,  NvU64, NvU64, void **, NvU64 *, NvU32 *, NvBool, void *, void *, void **, NvBool *);
 NV_STATUS  NV_API_CALL  rm_p2p_register_callback  (nvidia_stack_t *, NvU64, NvU64, NvU64, void *, void (*)(void *), void *);
 NV_STATUS  NV_API_CALL  rm_p2p_put_pages          (nvidia_stack_t *, NvU64, NvU32, NvU64, void *);
-NV_STATUS  NV_API_CALL  rm_p2p_put_pages_persistent(nvidia_stack_t *, void *, void *);
+NV_STATUS  NV_API_CALL  rm_p2p_put_pages_persistent(nvidia_stack_t *, void *, void *, void *);
 NV_STATUS  NV_API_CALL  rm_p2p_dma_map_pages      (nvidia_stack_t *, nv_dma_device_t *, NvU8 *, NvU64, NvU32, NvU64 *, void **);
-NV_STATUS  NV_API_CALL  rm_dma_buf_dup_mem_handle (nvidia_stack_t *, nv_state_t *, NvHandle, NvHandle, NvHandle, NvHandle, void *, NvHandle, NvU64, NvU64, NvHandle *, void **);
+NV_STATUS  NV_API_CALL  rm_dma_buf_dup_mem_handle (nvidia_stack_t *, nv_state_t *, NvHandle, NvHandle, NvHandle,
+                                                   NvHandle, void *, NvHandle, NvU64, NvU64, NvHandle *, void **,
+                                                   NvBool *, NvU32 *, NvBool *, nv_memory_type_t *);
 void       NV_API_CALL  rm_dma_buf_undup_mem_handle(nvidia_stack_t *, nv_state_t *, NvHandle, NvHandle);
-NV_STATUS  NV_API_CALL  rm_dma_buf_map_mem_handle (nvidia_stack_t *, nv_state_t *, NvHandle, NvHandle, NvU64, NvU64, void *, nv_phys_addr_range_t **, NvU32 *);
-void       NV_API_CALL  rm_dma_buf_unmap_mem_handle(nvidia_stack_t *, nv_state_t *, NvHandle, NvHandle, NvU64, nv_phys_addr_range_t **, NvU32);
-NV_STATUS  NV_API_CALL  rm_dma_buf_get_client_and_device(nvidia_stack_t *, nv_state_t *, NvHandle, NvHandle, NvHandle *, NvHandle *, NvHandle *, void **, NvBool *);
+NV_STATUS  NV_API_CALL  rm_dma_buf_map_mem_handle (nvidia_stack_t *, nv_state_t *,
+                                                   NvHandle, NvHandle, MemoryRange,
+                                                   NvU8, void *, NvBool, MemoryArea *);
+void       NV_API_CALL  rm_dma_buf_unmap_mem_handle(nvidia_stack_t *, nv_state_t *,
+                                                    NvHandle, NvHandle, NvU8, void *,
+                                                    NvBool, MemoryArea);
+NV_STATUS  NV_API_CALL  rm_dma_buf_get_client_and_device(nvidia_stack_t *,
+                                                nv_state_t *, NvHandle, NvHandle,
+                                                NvU8, NvHandle *, NvHandle *,
+                                                NvHandle *, void **, NvBool *, NvBool *);
 void       NV_API_CALL  rm_dma_buf_put_client_and_device(nvidia_stack_t *, nv_state_t *, NvHandle, NvHandle, NvHandle, void *);
-NV_STATUS  NV_API_CALL  rm_log_gpu_crash          (nv_stack_t *, nv_state_t *);
 
 void       NV_API_CALL rm_kernel_rmapi_op(nvidia_stack_t *sp, void *ops_cmd);
 NvBool     NV_API_CALL rm_get_device_remove_flag(nvidia_stack_t *sp, NvU32 gpu_id);
@@ -1018,38 +1189,40 @@ NvBool     NV_API_CALL rm_is_device_sequestered(nvidia_stack_t *, nv_state_t *);
 void       NV_API_CALL rm_check_for_gpu_surprise_removal(nvidia_stack_t *, nv_state_t *);
 NV_STATUS  NV_API_CALL rm_set_external_kernel_client_count(nvidia_stack_t *, nv_state_t *, NvBool);
 NV_STATUS  NV_API_CALL rm_schedule_gpu_wakeup(nvidia_stack_t *, nv_state_t *);
-NvBool     NV_API_CALL rm_is_iommu_needed_for_sriov(nvidia_stack_t *, nv_state_t *);
 NvBool     NV_API_CALL rm_disable_iomap_wc(void);
 
+void       NV_API_CALL rm_init_tegra_dynamic_power_management(nvidia_stack_t *, nv_state_t *);
 void       NV_API_CALL rm_init_dynamic_power_management(nvidia_stack_t *, nv_state_t *, NvBool);
 void       NV_API_CALL rm_cleanup_dynamic_power_management(nvidia_stack_t *, nv_state_t *);
 void       NV_API_CALL rm_enable_dynamic_power_management(nvidia_stack_t *, nv_state_t *);
 NV_STATUS  NV_API_CALL rm_ref_dynamic_power(nvidia_stack_t *, nv_state_t *, nv_dynamic_power_mode_t);
 void       NV_API_CALL rm_unref_dynamic_power(nvidia_stack_t *, nv_state_t *, nv_dynamic_power_mode_t);
 NV_STATUS  NV_API_CALL rm_transition_dynamic_power(nvidia_stack_t *, nv_state_t *, NvBool, NvBool *);
-const char* NV_API_CALL rm_get_vidmem_power_status(nvidia_stack_t *, nv_state_t *);
-const char* NV_API_CALL rm_get_dynamic_power_management_status(nvidia_stack_t *, nv_state_t *);
-const char* NV_API_CALL rm_get_gpu_gcx_support(nvidia_stack_t *, nv_state_t *, NvBool);
+void       NV_API_CALL rm_get_power_info(nvidia_stack_t *, nv_state_t *, nv_power_info_t *);
 
 void       NV_API_CALL rm_acpi_notify(nvidia_stack_t *, nv_state_t *, NvU32);
 void       NV_API_CALL rm_acpi_nvpcf_notify(nvidia_stack_t *);
 
 NvBool     NV_API_CALL rm_is_altstack_in_use(void);
 
+void       NV_API_CALL rm_notify_gpu_addition(nvidia_stack_t *, nv_state_t *);
+void       NV_API_CALL rm_notify_gpu_removal(nvidia_stack_t *, nv_state_t *);
+
 /* vGPU VFIO specific functions */
-NV_STATUS  NV_API_CALL  nv_vgpu_create_request(nvidia_stack_t *, nv_state_t *, const NvU8 *, NvU32, NvU16 *, NvU32);
+NV_STATUS  NV_API_CALL  nv_vgpu_create_request(nvidia_stack_t *, nv_state_t *, const NvU8 *, NvU32, NvU16 *,
+                                               NvU32 *, NvU32 *, NvU32);
 NV_STATUS  NV_API_CALL  nv_vgpu_delete(nvidia_stack_t *, const NvU8 *, NvU16);
 NV_STATUS  NV_API_CALL  nv_vgpu_get_type_ids(nvidia_stack_t *, nv_state_t *, NvU32 *, NvU32 *, NvBool, NvU8, NvBool);
 NV_STATUS  NV_API_CALL  nv_vgpu_get_type_info(nvidia_stack_t *, nv_state_t *, NvU32, char *, int, NvU8);
-NV_STATUS  NV_API_CALL  nv_vgpu_get_bar_info(nvidia_stack_t *, nv_state_t *, const NvU8 *, NvU64 *, NvU32, void *, NvBool *);
+NV_STATUS  NV_API_CALL  nv_vgpu_get_bar_info(nvidia_stack_t *, nv_state_t *, const NvU8 *, NvU64 *,
+                                             NvU64 *, NvU64 *, NvU32 *, NvBool *, NvU8 *);
+NV_STATUS  NV_API_CALL  nv_vgpu_update_sysfs_info(nvidia_stack_t *, nv_state_t *, const NvU8 *, NvU32, NvU32 *);
 NV_STATUS  NV_API_CALL  nv_vgpu_get_hbm_info(nvidia_stack_t *, nv_state_t *, const NvU8 *, NvU64 *, NvU64 *);
-NV_STATUS  NV_API_CALL  nv_vgpu_start(nvidia_stack_t *, const NvU8 *, void *, NvS32 *, NvU8 *, NvU32);
-NV_STATUS  NV_API_CALL  nv_vgpu_get_sparse_mmap(nvidia_stack_t *, nv_state_t *, const NvU8 *, NvU64 **, NvU64 **, NvU32 *);
 NV_STATUS  NV_API_CALL  nv_vgpu_process_vf_info(nvidia_stack_t *, nv_state_t *, NvU8, NvU32, NvU8, NvU8, NvU8, NvBool, void *);
-NV_STATUS  NV_API_CALL  nv_vgpu_update_request(nvidia_stack_t *, const NvU8 *, NvU32, NvU64 *, NvU64 *, const char *);
-NV_STATUS  NV_API_CALL  nv_gpu_bind_event(nvidia_stack_t *);
+NV_STATUS  NV_API_CALL  nv_gpu_bind_event(nvidia_stack_t *, NvU32, NvBool *);
+NV_STATUS  NV_API_CALL  nv_gpu_unbind_event(nvidia_stack_t *, NvU32, NvBool *);
 
-NV_STATUS NV_API_CALL nv_get_usermap_access_params(nv_state_t*, nv_usermap_access_params_t*);
+NV_STATUS NV_API_CALL nv_check_usermap_access_params(nv_state_t*, const nv_usermap_access_params_t*);
 nv_soc_irq_type_t NV_API_CALL nv_get_current_irq_type(nv_state_t*);
 void       NV_API_CALL  nv_flush_coherent_cpu_cache_range(nv_state_t *nv, NvU64 cpu_virtual, NvU64 size);
 
@@ -1078,6 +1251,9 @@ NV_STATUS   NV_API_CALL rm_run_nano_timer_callback(nvidia_stack_t *, nv_state_t
 void        NV_API_CALL nv_cancel_nano_timer(nv_state_t *, nv_nano_timer_t *);
 void        NV_API_CALL nv_destroy_nano_timer(nv_state_t *nv, nv_nano_timer_t *);
 
+// Host1x specific functions.
+NV_STATUS NV_API_CALL nv_get_syncpoint_aperture(NvU32, NvU64 *, NvU64 *, NvU32 *);
+
 #if defined(NVCPU_X86_64)
 
 static inline NvU64 nv_rdtsc(void)

kernel-open/common/inc/nv_common_utils.h

@@ -0,0 +1,120 @@
+/*
+ * SPDX-FileCopyrightText: Copyright (c) 2015 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+ * SPDX-License-Identifier: MIT
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
+ * DEALINGS IN THE SOFTWARE.
+ */
+
+#ifndef __NV_COMMON_UTILS_H__
+#define __NV_COMMON_UTILS_H__
+
+#include "nvtypes.h"
+#include "nvmisc.h"
+
+#if !defined(TRUE)
+#define TRUE NV_TRUE
+#endif
+
+#if !defined(FALSE)
+#define FALSE NV_FALSE
+#endif
+
+#define NV_IS_UNSIGNED(x) ((__typeof__(x))-1 > 0)
+
+/* Get the length of a statically-sized array. */
+#define ARRAY_LEN(_arr) (sizeof(_arr) / sizeof(_arr[0]))
+
+#define NV_INVALID_HEAD         0xFFFFFFFF
+
+#define NV_INVALID_CONNECTOR_PHYSICAL_INFORMATION (~0)
+
+#if !defined(NV_MIN)
+# define NV_MIN(a,b) (((a)<(b))?(a):(b))
+#endif
+
+#define NV_MIN3(a,b,c) NV_MIN(NV_MIN(a, b), c)
+#define NV_MIN4(a,b,c,d) NV_MIN3(NV_MIN(a,b),c,d)
+
+#if !defined(NV_MAX)
+# define NV_MAX(a,b) (((a)>(b))?(a):(b))
+#endif
+
+#define NV_MAX3(a,b,c) NV_MAX(NV_MAX(a, b), c)
+#define NV_MAX4(a,b,c,d) NV_MAX3(NV_MAX(a,b),c,d)
+
+static inline int NV_LIMIT_VAL_TO_MIN_MAX(int val, int min, int max)
+{
+    if (val < min) {
+        return min;
+    }
+    if (val > max) {
+        return max;
+    }
+    return val;
+}
+
+#define NV_ROUNDUP_DIV(x,y) ((x) / (y) + (((x) % (y)) ? 1 : 0))
+
+/*
+ * Macros used for computing palette entries:
+ *
+ * NV_UNDER_REPLICATE(val, source_size, result_size) expands a value
+ * of source_size bits into a value of target_size bits by shifting
+ * the source value into the high bits and replicating the high bits
+ * of the value into the low bits of the result.
+ *
+ * PALETTE_DEPTH_SHIFT(val, w) maps a colormap entry for a component
+ * that has w bits to an appropriate entry in a LUT of 256 entries.
+ */
+static inline unsigned int NV_UNDER_REPLICATE(unsigned short val,
+                                              int source_size,
+                                              int result_size)
+{
+    return (val << (result_size - source_size)) |
+        (val >> ((source_size << 1) - result_size));
+}
+
+
+static inline unsigned short PALETTE_DEPTH_SHIFT(unsigned short val, int depth)
+{
+    return NV_UNDER_REPLICATE(val, depth, 8);
+}
+
+/*
+ *  Use __builtin_ffs where it is supported, or provide an equivalent
+ *  implementation for platforms like riscv where it is not.
+ */
+#if defined(__GNUC__) && !NVCPU_IS_RISCV64
+static inline int nv_ffs(int x)
+{
+    return __builtin_ffs(x);
+}
+#else
+static inline int nv_ffs(int x)
+{
+    if (x == 0)
+        return 0;
+
+    LOWESTBITIDX_32(x);
+
+    return 1 + x;
+}
+#endif
+
+#endif /* __NV_COMMON_UTILS_H__ */

kernel-open/common/inc/nv_dpy_id.h

@@ -0,0 +1,370 @@
+/*
+ * SPDX-FileCopyrightText: Copyright (c) 2010-2014 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+ * SPDX-License-Identifier: MIT
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
+ * DEALINGS IN THE SOFTWARE.
+ */
+/*
+ * This header file defines the types NVDpyId and NVDpyIdList, as well
+ * as inline functions to manipulate these types.  NVDpyId and
+ * NVDpyIdList should be treated as opaque by includers of this header
+ * file.
+ */
+
+#ifndef __NV_DPY_ID_H__
+#define __NV_DPY_ID_H__
+
+#include "nvtypes.h"
+#include "nvmisc.h"
+#include "nv_common_utils.h"
+#include <nvlimits.h> /* NV_MAX_SUBDEVICES */
+
+typedef struct {
+    NvU32 opaqueDpyId;
+} NVDpyId;
+
+typedef struct {
+    NvU32 opaqueDpyIdList;
+} NVDpyIdList;
+
+#define NV_DPY_ID_MAX_SUBDEVICES NV_MAX_SUBDEVICES
+#define NV_DPY_ID_MAX_DPYS_IN_LIST 32
+
+/*
+ * For use in combination with nvDpyIdToPrintFormat(); e.g.,
+ *
+ * printf("dpy id: " NV_DPY_ID_PRINT_FORMAT "\n",
+ *        nvDpyIdToPrintFormat(dpyId));
+ *
+ * The includer should not make assumptions about the return type of
+ * nvDpyIdToPrintFormat().
+ */
+#define NV_DPY_ID_PRINT_FORMAT "0x%08x"
+
+/* functions to return an invalid DpyId and empty DpyIdList */
+
+static inline NVDpyId nvInvalidDpyId(void)
+{
+    NVDpyId dpyId = { 0 };
+    return dpyId;
+}
+
+static inline NVDpyIdList nvEmptyDpyIdList(void)
+{
+    NVDpyIdList dpyIdList = { 0 };
+    return dpyIdList;
+}
+
+static inline NVDpyIdList nvAllDpyIdList(void)
+{
+    NVDpyIdList dpyIdList = { ~0U };
+    return dpyIdList;
+}
+
+static inline void
+nvEmptyDpyIdListSubDeviceArray(NVDpyIdList dpyIdList[NV_DPY_ID_MAX_SUBDEVICES])
+{
+    int dispIndex;
+    for (dispIndex = 0; dispIndex < NV_DPY_ID_MAX_SUBDEVICES; dispIndex++) {
+        dpyIdList[dispIndex] = nvEmptyDpyIdList();
+    }
+}
+
+/* set operations on DpyIds and DpyIdLists: Add, Subtract, Intersect, Xor */
+
+static inline __attribute__ ((warn_unused_result))
+NVDpyIdList nvAddDpyIdToDpyIdList(NVDpyId dpyId, NVDpyIdList dpyIdList)
+{
+    NVDpyIdList tmpDpyIdList;
+    tmpDpyIdList.opaqueDpyIdList = dpyIdList.opaqueDpyIdList |
+        dpyId.opaqueDpyId;
+    return tmpDpyIdList;
+}
+
+/* Passing an invalid display ID makes this function return an empty list. */
+static inline __attribute__ ((warn_unused_result))
+NVDpyIdList nvAddDpyIdToEmptyDpyIdList(NVDpyId dpyId)
+{
+    NVDpyIdList tmpDpyIdList;
+    tmpDpyIdList.opaqueDpyIdList =  dpyId.opaqueDpyId;
+    return tmpDpyIdList;
+}
+
+static inline __attribute__ ((warn_unused_result))
+NVDpyIdList nvAddDpyIdListToDpyIdList(NVDpyIdList dpyIdListA,
+                                      NVDpyIdList dpyIdListB)
+{
+    NVDpyIdList tmpDpyIdList;
+    tmpDpyIdList.opaqueDpyIdList = dpyIdListB.opaqueDpyIdList |
+        dpyIdListA.opaqueDpyIdList;
+    return tmpDpyIdList;
+}
+
+/* Returns: dpyIdList - dpyId */
+static inline __attribute__ ((warn_unused_result))
+NVDpyIdList nvDpyIdListMinusDpyId(NVDpyIdList dpyIdList, NVDpyId dpyId)
+{
+    NVDpyIdList tmpDpyIdList;
+    tmpDpyIdList.opaqueDpyIdList = dpyIdList.opaqueDpyIdList &
+        (~dpyId.opaqueDpyId);
+    return tmpDpyIdList;
+}
+
+/* Returns: dpyIdListA - dpyIdListB */
+static inline __attribute__ ((warn_unused_result))
+NVDpyIdList nvDpyIdListMinusDpyIdList(NVDpyIdList dpyIdListA,
+                                      NVDpyIdList dpyIdListB)
+{
+    NVDpyIdList tmpDpyIdList;
+    tmpDpyIdList.opaqueDpyIdList = dpyIdListA.opaqueDpyIdList &
+        (~dpyIdListB.opaqueDpyIdList);
+    return tmpDpyIdList;
+}
+
+static inline __attribute__ ((warn_unused_result))
+NVDpyIdList nvIntersectDpyIdAndDpyIdList(NVDpyId dpyId, NVDpyIdList dpyIdList)
+{
+    NVDpyIdList tmpDpyIdList;
+    tmpDpyIdList.opaqueDpyIdList = dpyIdList.opaqueDpyIdList &
+        dpyId.opaqueDpyId;
+    return tmpDpyIdList;
+}
+
+static inline __attribute__ ((warn_unused_result))
+NVDpyIdList nvIntersectDpyIdListAndDpyIdList(NVDpyIdList dpyIdListA,
+                                             NVDpyIdList dpyIdListB)
+{
+    NVDpyIdList tmpDpyIdList;
+    tmpDpyIdList.opaqueDpyIdList = dpyIdListA.opaqueDpyIdList &
+        dpyIdListB.opaqueDpyIdList;
+    return tmpDpyIdList;
+}
+
+static inline __attribute__ ((warn_unused_result))
+NVDpyIdList nvXorDpyIdAndDpyIdList(NVDpyId dpyId, NVDpyIdList dpyIdList)
+{
+    NVDpyIdList tmpDpyIdList;
+    tmpDpyIdList.opaqueDpyIdList = dpyIdList.opaqueDpyIdList ^
+        dpyId.opaqueDpyId;
+    return tmpDpyIdList;
+}
+
+static inline __attribute__ ((warn_unused_result))
+NVDpyIdList nvXorDpyIdListAndDpyIdList(NVDpyIdList dpyIdListA,
+                                       NVDpyIdList dpyIdListB)
+{
+    NVDpyIdList tmpDpyIdList;
+    tmpDpyIdList.opaqueDpyIdList = dpyIdListA.opaqueDpyIdList ^
+        dpyIdListB.opaqueDpyIdList;
+    return tmpDpyIdList;
+}
+
+
+/* boolean checks */
+
+static inline NvBool nvDpyIdIsInDpyIdList(NVDpyId dpyId,
+                                          NVDpyIdList dpyIdList)
+{
+    return !!(dpyIdList.opaqueDpyIdList & dpyId.opaqueDpyId);
+}
+
+static inline NvBool nvDpyIdIsInvalid(NVDpyId dpyId)
+{
+    return (dpyId.opaqueDpyId == 0);
+}
+
+static inline NvBool nvDpyIdListIsEmpty(NVDpyIdList dpyIdList)
+{
+    return (dpyIdList.opaqueDpyIdList == 0);
+}
+
+static inline NvBool
+nvDpyIdListSubDeviceArrayIsEmpty(NVDpyIdList
+                                 dpyIdList[NV_DPY_ID_MAX_SUBDEVICES])
+{
+    int dispIndex;
+    for (dispIndex = 0; dispIndex < NV_DPY_ID_MAX_SUBDEVICES; dispIndex++) {
+        if (!nvDpyIdListIsEmpty(dpyIdList[dispIndex])) {
+            return NV_FALSE;
+        }
+    }
+    return NV_TRUE;
+}
+
+
+static inline NvBool nvDpyIdsAreEqual(NVDpyId dpyIdA, NVDpyId dpyIdB)
+{
+    return (dpyIdA.opaqueDpyId == dpyIdB.opaqueDpyId);
+}
+
+static inline NvBool nvDpyIdListsAreEqual(NVDpyIdList dpyIdListA,
+                                          NVDpyIdList dpyIdListB)
+{
+    return (dpyIdListA.opaqueDpyIdList == dpyIdListB.opaqueDpyIdList);
+}
+
+static inline NvBool nvDpyIdListIsASubSetofDpyIdList(NVDpyIdList dpyIdListA,
+                                                     NVDpyIdList dpyIdListB)
+{
+    NVDpyIdList intersectedDpyIdList =
+        nvIntersectDpyIdListAndDpyIdList(dpyIdListA, dpyIdListB);
+
+    return nvDpyIdListsAreEqual(intersectedDpyIdList, dpyIdListA);
+}
+
+
+/*
+ * retrieve the individual dpyIds from dpyIdList; if dpyId is invalid,
+ * start at the beginning of the list; otherwise, start at the dpyId
+ * after the specified dpyId
+ */
+
+static inline __attribute__ ((warn_unused_result))
+NVDpyId nvNextDpyIdInDpyIdListUnsorted(NVDpyId dpyId, NVDpyIdList dpyIdList)
+{
+    if (nvDpyIdIsInvalid(dpyId)) {
+        dpyId.opaqueDpyId = 1;
+    } else {
+        dpyId.opaqueDpyId <<= 1;
+    }
+
+    while (dpyId.opaqueDpyId) {
+
+        if (nvDpyIdIsInDpyIdList(dpyId, dpyIdList)) {
+            return dpyId;
+        }
+
+        dpyId.opaqueDpyId <<= 1;
+    }
+
+    /* no dpyIds left in dpyIdlist; return the invalid dpyId */
+
+    return nvInvalidDpyId();
+}
+
+#define FOR_ALL_DPY_IDS(_dpyId, _dpyIdList)                           \
+    for ((_dpyId) = nvNextDpyIdInDpyIdListUnsorted(nvInvalidDpyId(),  \
+                                                   (_dpyIdList));     \
+         !nvDpyIdIsInvalid(_dpyId);                                   \
+         (_dpyId) = nvNextDpyIdInDpyIdListUnsorted((_dpyId),          \
+                                                   (_dpyIdList)))
+
+/* report how many dpyIds are in the dpyIdList */
+
+static inline int nvCountDpyIdsInDpyIdList(NVDpyIdList dpyIdList)
+{
+    return nvPopCount32(dpyIdList.opaqueDpyIdList);
+}
+
+static inline int
+nvCountDpyIdsInDpyIdListSubDeviceArray(NVDpyIdList
+                                       dpyIdList[NV_DPY_ID_MAX_SUBDEVICES])
+{
+    int dispIndex, n = 0;
+
+    for (dispIndex = 0; dispIndex < NV_DPY_ID_MAX_SUBDEVICES; dispIndex++) {
+        n += nvCountDpyIdsInDpyIdList(dpyIdList[dispIndex]);
+    }
+
+    return n;
+}
+
+/* convert between dpyId/dpyIdList and NV-CONTROL values */
+
+static inline int nvDpyIdToNvControlVal(NVDpyId dpyId)
+{
+    return (int) dpyId.opaqueDpyId;
+}
+
+static inline int nvDpyIdListToNvControlVal(NVDpyIdList dpyIdList)
+{
+    return (int) dpyIdList.opaqueDpyIdList;
+}
+
+static inline NVDpyId nvNvControlValToDpyId(int val)
+{
+    NVDpyId dpyId;
+    dpyId.opaqueDpyId = (val == 0) ? 0 : 1 << (nv_ffs(val)-1);
+    return dpyId;
+}
+
+static inline NVDpyIdList nvNvControlValToDpyIdList(int val)
+{
+    NVDpyIdList dpyIdList;
+    dpyIdList.opaqueDpyIdList = val;
+    return dpyIdList;
+}
+
+
+/* convert between dpyId and NvU32 */
+
+static inline NVDpyId nvNvU32ToDpyId(NvU32 val)
+{
+    NVDpyId dpyId;
+    dpyId.opaqueDpyId = (val == 0) ? 0 : 1 << (nv_ffs(val)-1);
+    return dpyId;
+}
+
+static inline NVDpyIdList nvNvU32ToDpyIdList(NvU32 val)
+{
+    NVDpyIdList dpyIdList;
+    dpyIdList.opaqueDpyIdList = val;
+    return dpyIdList;
+}
+
+static inline NvU32 nvDpyIdToNvU32(NVDpyId dpyId)
+{
+    return dpyId.opaqueDpyId;
+}
+
+static inline NvU32 nvDpyIdListToNvU32(NVDpyIdList dpyIdList)
+{
+    return dpyIdList.opaqueDpyIdList;
+}
+
+/* Return the bit position of dpyId: a number in the range [0..31]. */
+static inline NvU32 nvDpyIdToIndex(NVDpyId dpyId)
+{
+    return nv_ffs(dpyId.opaqueDpyId) - 1;
+}
+
+/* Return a display ID that is not in the list passed in. */
+
+static inline NVDpyId nvNewDpyId(NVDpyIdList excludeList)
+{
+    NVDpyId dpyId;
+    if (~excludeList.opaqueDpyIdList == 0) {
+        return nvInvalidDpyId();
+    }
+    dpyId.opaqueDpyId =
+        1U << (nv_ffs(~excludeList.opaqueDpyIdList) - 1);
+    return dpyId;
+}
+
+/* See comment for NV_DPY_ID_PRINT_FORMAT. */
+static inline NvU32 nvDpyIdToPrintFormat(NVDpyId dpyId)
+{
+    return nvDpyIdToNvU32(dpyId);
+}
+
+/* Prevent usage of opaque values. */
+#define opaqueDpyId __ERROR_ACCESS_ME_VIA_NV_DPY_ID_H
+#define opaqueDpyIdList __ERROR_ACCESS_ME_VIA_NV_DPY_ID_H
+
+#endif /* __NV_DPY_ID_H__ */

kernel-open/common/inc/nv_mig_types.h

@@ -1,5 +1,5 @@
 /*
- * SPDX-FileCopyrightText: Copyright (c) 2021 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+ * SPDX-FileCopyrightText: Copyright (c) 2024 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
  * SPDX-License-Identifier: MIT
  *
  * Permission is hereby granted, free of charge, to any person obtaining a
@@ -20,16 +20,21 @@
  * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
  * DEALINGS IN THE SOFTWARE.
  */
+#ifndef __NV_MIG_TYPES_H__
+#define __NV_MIG_TYPES_H__
 
-#ifndef GSP_ERROR_H
-#define GSP_ERROR_H
+#ifdef __cplusplus
+extern "C" {
+#endif
 
-// Definitions for GSP-RM to report errors to CPU-RM via mailbox
-#define NV_GSP_ERROR_CODE     7:0
-#define NV_GSP_ERROR_REASON  15:8
-#define NV_GSP_ERROR_TASK    23:16
-#define NV_GSP_ERROR_SKIPPED 27:24
-#define NV_GSP_ERROR_TAG     31:28
-#define NV_GSP_ERROR_TAG_VAL  0xE
+#include "nvtypes.h"
 
-#endif // GSP_ERROR_H
+typedef NvU32 MIGDeviceId;
+
+#define NO_MIG_DEVICE 0L
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif /* __NV_MIG_TYPES_H__ */

kernel-open/common/inc/nv_uvm_interface.h

@@ -1,5 +1,5 @@
 /*
- * SPDX-FileCopyrightText: Copyright (c) 2013-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+ * SPDX-FileCopyrightText: Copyright (c) 2013-2024 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
  * SPDX-License-Identifier: MIT
  *
  * Permission is hereby granted, free of charge, to any person obtaining a
@@ -46,6 +46,7 @@ struct UvmOpsUvmEvents;
 #include "nvgputypes.h"
 #include "nvstatus.h"
 #include "nv_uvm_types.h"
+#include "nv_uvm_user_types.h"
 
 
 // Define the type here as it's Linux specific, used only by the Linux specific
@@ -592,6 +593,14 @@ void nvUvmInterfaceChannelDestroy(uvmGpuChannelHandle channel);
     Error codes:
       NV_ERR_GENERIC
       NV_ERR_NO_MEMORY
+      NV_ERR_INVALID_STATE
+      NV_ERR_NOT_SUPPORTED
+      NV_ERR_NOT_READY
+      NV_ERR_INVALID_LOCK_STATE
+      NV_ERR_INVALID_STATE
+      NV_ERR_NVLINK_FABRIC_NOT_READY
+      NV_ERR_NVLINK_FABRIC_FAILURE
+      NV_ERR_GPU_MEMORY_ONLINING_FAILURE
 */
 NV_STATUS nvUvmInterfaceQueryCaps(uvmGpuDeviceHandle device,
                                   UvmGpuCaps *caps);
@@ -652,14 +661,20 @@ NV_STATUS nvUvmInterfaceServiceDeviceInterruptsRM(uvmGpuDeviceHandle device);
     RM will propagate the update to all channels using the provided VA space.
     All channels must be idle when this call is made.
 
+    If the pageDirectory is in system memory then a CPU physical address must be
+    provided. RM will establish and manage the DMA mapping for the
+    pageDirectory.
+
     Arguments:
       vaSpace[IN}         - VASpace Object
-      physAddress[IN]     - Physical address of new page directory
+      physAddress[IN]     - Physical address of new page directory. If
+                            !bVidMemAperture this is a CPU physical address.
       numEntries[IN]      - Number of entries including previous PDE which will be copied
       bVidMemAperture[IN] - If set pageDirectory will reside in VidMem aperture else sysmem
       pasid[IN]           - PASID (Process Address Space IDentifier) of the process
                             corresponding to the VA space. Ignored unless the VA space
                             object has ATS enabled.
+      dmaAddress[OUT]     - DMA mapping created for physAddress.
 
     Error codes:
       NV_ERR_GENERIC
@@ -667,7 +682,8 @@ NV_STATUS nvUvmInterfaceServiceDeviceInterruptsRM(uvmGpuDeviceHandle device);
 */
 NV_STATUS nvUvmInterfaceSetPageDirectory(uvmGpuAddressSpaceHandle vaSpace,
                                          NvU64 physAddress, unsigned numEntries,
-                                         NvBool bVidMemAperture, NvU32 pasid);
+                                         NvBool bVidMemAperture, NvU32 pasid,
+                                         NvU64 *dmaAddress);
 
 /*******************************************************************************
     nvUvmInterfaceUnsetPageDirectory
@@ -1048,7 +1064,7 @@ NV_STATUS nvUvmInterfaceDestroyAccessCntrInfo(uvmGpuDeviceHandle device,
 */
 NV_STATUS nvUvmInterfaceEnableAccessCntr(uvmGpuDeviceHandle device,
                                          UvmGpuAccessCntrInfo *pAccessCntrInfo,
-                                         UvmGpuAccessCntrConfig *pAccessCntrConfig);
+                                         const UvmGpuAccessCntrConfig *pAccessCntrConfig);
 
 /*******************************************************************************
     nvUvmInterfaceDisableAccessCntr
@@ -1085,6 +1101,22 @@ NV_STATUS nvUvmInterfaceRegisterUvmCallbacks(struct UvmOpsUvmEvents *importedUvm
 //
 void nvUvmInterfaceDeRegisterUvmOps(void);
 
+/*******************************************************************************
+    nvUvmInterfaceGetNvlinkInfo
+
+    Gets NVLINK information from RM.
+
+    Arguments:
+        device[IN]        - GPU device handle
+        nvlinkInfo [OUT]     - Pointer to NvlinkInfo structure
+
+    Error codes:
+      NV_ERROR
+      NV_ERR_INVALID_ARGUMENT
+*/
+NV_STATUS nvUvmInterfaceGetNvlinkInfo(uvmGpuDeviceHandle device,
+                                      UvmGpuNvlinkInfo *nvlinkInfo);
+
 /*******************************************************************************
     nvUvmInterfaceP2pObjectCreate
 
@@ -1161,6 +1193,48 @@ NV_STATUS nvUvmInterfaceGetExternalAllocPtes(uvmGpuAddressSpaceHandle vaSpace,
                                              NvU64 size,
                                              UvmGpuExternalMappingInfo *gpuExternalMappingInfo);
 
+/*******************************************************************************
+    nvUvmInterfaceGetExternalAllocPhysAddrs
+
+    The interface builds the RM physical addrs using the provided input parameters.
+
+    Arguments:
+        vaSpace[IN]                     -  vaSpace handle.
+        hMemory[IN]                     -  Memory handle.
+        offset [IN]                     -  Offset from the beginning of the allocation
+                                           where PTE mappings should begin.
+                                           Should be aligned with mappingPagesize
+                                           in gpuExternalMappingInfo associated
+                                           with the allocation.
+        size [IN]                       -  Length of the allocation for which PhysAddrs
+                                           should be built.
+                                           Should be aligned with mappingPagesize
+                                           in gpuExternalMappingInfo associated
+                                           with the allocation.
+                                           size = 0 will be interpreted as the total size
+                                           of the allocation.
+        gpuExternalMappingInfo[IN/OUT]  -  See nv_uvm_types.h for more information.
+
+   Error codes:
+        NV_ERR_INVALID_ARGUMENT         - Invalid parameter/s is passed.
+        NV_ERR_INVALID_OBJECT_HANDLE    - Invalid memory handle is passed.
+        NV_ERR_NOT_SUPPORTED            - Functionality is not supported (see comments in nv_gpu_ops.c)
+        NV_ERR_INVALID_BASE             - offset is beyond the allocation size
+        NV_ERR_INVALID_LIMIT            - (offset + size) is beyond the allocation size.
+        NV_ERR_BUFFER_TOO_SMALL         - gpuExternalMappingInfo.physAddrBufferSize is insufficient to
+                                          store single physAddr.
+        NV_ERR_NOT_READY                - Returned when querying the physAddrs requires a deferred setup
+                                          which has not yet completed. It is expected that the caller
+                                          will reattempt the call until a different code is returned.
+                                          As an example, multi-node systems which require querying
+                                          physAddrs from the Fabric Manager may return this code.
+*/
+NV_STATUS nvUvmInterfaceGetExternalAllocPhysAddrs(uvmGpuAddressSpaceHandle vaSpace,
+                                                  NvHandle hMemory,
+                                                  NvU64 offset,
+                                                  NvU64 size,
+                                                  UvmGpuExternalPhysAddrInfo *gpuExternalPhysAddrsInfo);
+
 /*******************************************************************************
     nvUvmInterfaceRetainChannel
 
@@ -1462,6 +1536,16 @@ NV_STATUS nvUvmInterfacePagingChannelPushStream(UvmGpuPagingChannelHandle channe
                                                 char *methodStream,
                                                 NvU32 methodStreamSize);
 
+/*******************************************************************************
+    nvUvmInterfaceReportFatalError
+
+    Reports a global fatal error so RM can inform the clients that a node reboot
+    is necessary to recover from this error. This function can be called from
+    any lock environment, bottom half or non-interrupt context.
+
+*/
+void nvUvmInterfaceReportFatalError(NV_STATUS error);
+
 /*******************************************************************************
     Cryptography Services Library (CSL) Interface
 */
@@ -1505,23 +1589,35 @@ NV_STATUS nvUvmInterfaceCslInitContext(UvmCslContext *uvmCslContext,
 void nvUvmInterfaceDeinitCslContext(UvmCslContext *uvmCslContext);
 
 /*******************************************************************************
-    nvUvmInterfaceCslUpdateContext
+    nvUvmInterfaceCslRotateKey
 
-    Updates a context after a key rotation event and can only be called once per
-    key rotation event. Following a key rotation event, and before
-    nvUvmInterfaceCslUpdateContext is called, data encrypted by the GPU with the
-    previous key can be decrypted with nvUvmInterfaceCslDecrypt.
+    Disables channels and rotates keys.
 
-    Locking: This function acquires an API lock.
-    Memory : This function does not dynamically allocate memory.
+    This function disables channels and rotates associated keys. The channels
+    associated with the given CSL contexts must be idled before this function is
+    called. To trigger key rotation all allocated channels for a given key must
+    be present in the list. If the function returns successfully then the CSL
+    contexts have been updated with the new key.
+
+    Locking: This function attempts to acquire the GPU lock. In case of failure
+             to acquire the return code is NV_ERR_STATE_IN_USE. The caller must
+             guarantee that no CSL function, including this one, is invoked
+             concurrently with the CSL contexts in contextList.
+    Memory : This function dynamically allocates memory.
 
     Arguments:
-        uvmCslContext[IN] - The CSL context associated with a channel.
-
+        contextList[IN/OUT]  - An array of pointers to CSL contexts.
+        contextListCount[IN] - Number of CSL contexts in contextList. Its value
+                               must be greater than 0.
     Error codes:
-        NV_ERR_INVALID_ARGUMENT - The CSL context is not associated with a channel.
+        NV_ERR_INVALID_ARGUMENT - contextList is NULL or contextListCount is 0.
+        NV_ERR_STATE_IN_USE     - Unable to acquire lock / resource. Caller
+                                  can retry at a later time.
+        NV_ERR_GENERIC          - A failure other than _STATE_IN_USE occurred
+                                  when attempting to acquire a lock.
 */
-NV_STATUS nvUvmInterfaceCslUpdateContext(UvmCslContext *uvmCslContext);
+NV_STATUS nvUvmInterfaceCslRotateKey(UvmCslContext *contextList[],
+                                     NvU32 contextListCount);
 
 /*******************************************************************************
     nvUvmInterfaceCslRotateIv
@@ -1529,17 +1625,13 @@ NV_STATUS nvUvmInterfaceCslUpdateContext(UvmCslContext *uvmCslContext);
     Rotates the IV for a given channel and operation.
 
     This function will rotate the IV on both the CPU and the GPU.
-    Outstanding messages that have been encrypted by the GPU should first be
-    decrypted before calling this function with operation equal to
-    UVM_CSL_OPERATION_DECRYPT. Similarly, outstanding messages that have been
-    encrypted by the CPU should first be decrypted before calling this function
-    with operation equal to UVM_CSL_OPERATION_ENCRYPT. For a given operation
-    the channel must be idle before calling this function. This function can be
-    called regardless of the value of the IV's message counter.
+    For a given operation the channel must be idle before calling this function.
+    This function can be called regardless of the value of the IV's message counter.
 
-    Locking: This function attempts to acquire the GPU lock.
-             In case of failure to acquire the return code
-             is NV_ERR_STATE_IN_USE.
+    Locking: This function attempts to acquire the GPU lock. In case of failure to
+             acquire the return code is NV_ERR_STATE_IN_USE. The caller must guarantee
+             that no CSL function, including this one, is invoked concurrently with
+             the same CSL context.
     Memory : This function does not dynamically allocate memory.
 
 Arguments:
@@ -1573,8 +1665,8 @@ NV_STATUS nvUvmInterfaceCslRotateIv(UvmCslContext *uvmCslContext,
     However, it is optional. If it is NULL, the next IV in line will be used.
 
     Locking: This function does not acquire an API or GPU lock.
-             If called concurrently in different threads with the same UvmCslContext
-             the caller must guarantee exclusion.
+             The caller must guarantee that no CSL function, including this one,
+             is invoked concurrently with the same CSL context.
     Memory : This function does not dynamically allocate memory.
 
 Arguments:
@@ -1610,9 +1702,14 @@ NV_STATUS nvUvmInterfaceCslEncrypt(UvmCslContext *uvmCslContext,
     maximized when the input and output buffers are 16-byte aligned. This is
     natural alignment for AES block.
 
+    During a key rotation event the previous key is stored in the CSL context.
+    This allows data encrypted by the GPU to be decrypted with the previous key.
+    The keyRotationId parameter identifies which key is used. The first key rotation
+    ID has a value of 0 that increments by one for each key rotation event.
+
     Locking: This function does not acquire an API or GPU lock.
-             If called concurrently in different threads with the same UvmCslContext
-             the caller must guarantee exclusion.
+             The caller must guarantee that no CSL function, including this one,
+             is invoked concurrently with the same CSL context.
     Memory : This function does not dynamically allocate memory.
 
     Arguments:
@@ -1622,6 +1719,8 @@ NV_STATUS nvUvmInterfaceCslEncrypt(UvmCslContext *uvmCslContext,
         decryptIv[IN]         - IV used to decrypt the ciphertext. Its value can either be given by
                                 nvUvmInterfaceCslIncrementIv, or, if NULL, the CSL context's
                                 internal counter is used.
+        keyRotationId[IN]     - Specifies the key that is used for decryption.
+                                A value of NV_U32_MAX specifies the current key.
         inputBuffer[IN]       - Address of ciphertext input buffer.
         outputBuffer[OUT]     - Address of plaintext output buffer.
         addAuthData[IN]       - Address of the plaintext additional authenticated data used to
@@ -1642,6 +1741,7 @@ NV_STATUS nvUvmInterfaceCslDecrypt(UvmCslContext *uvmCslContext,
                                    NvU32 bufferSize,
                                    NvU8 const *inputBuffer,
                                    UvmCslIv const *decryptIv,
+                                   NvU32 keyRotationId,
                                    NvU8 *outputBuffer,
                                    NvU8 const *addAuthData,
                                    NvU32 addAuthDataSize,
@@ -1656,8 +1756,8 @@ NV_STATUS nvUvmInterfaceCslDecrypt(UvmCslContext *uvmCslContext,
     undefined behavior.
 
     Locking: This function does not acquire an API or GPU lock.
-             If called concurrently in different threads with the same UvmCslContext
-             the caller must guarantee exclusion.
+             The caller must guarantee that no CSL function, including this one,
+             is invoked concurrently with the same CSL context.
     Memory : This function does not dynamically allocate memory.
 
     Arguments:
@@ -1685,8 +1785,8 @@ NV_STATUS nvUvmInterfaceCslSign(UvmCslContext *uvmCslContext,
 
     Locking: This function does not acquire an API or GPU lock.
     Memory : This function does not dynamically allocate memory.
-             If called concurrently in different threads with the same UvmCslContext
-             the caller must guarantee exclusion.
+             The caller must guarantee that no CSL function, including this one,
+             is invoked concurrently with the same CSL context.
 
     Arguments:
         uvmCslContext[IN/OUT] - The CSL context.
@@ -1711,8 +1811,8 @@ NV_STATUS nvUvmInterfaceCslQueryMessagePool(UvmCslContext *uvmCslContext,
     the returned IV can be used in nvUvmInterfaceCslDecrypt.
 
     Locking: This function does not acquire an API or GPU lock.
-             If called concurrently in different threads with the same UvmCslContext
-             the caller must guarantee exclusion.
+             The caller must guarantee that no CSL function, including this one,
+             is invoked concurrently with the same CSL context.
     Memory : This function does not dynamically allocate memory.
 
 Arguments:
@@ -1734,28 +1834,40 @@ NV_STATUS nvUvmInterfaceCslIncrementIv(UvmCslContext *uvmCslContext,
                                        UvmCslIv *iv);
 
 /*******************************************************************************
-    nvUvmInterfaceCslLogExternalEncryption
+    nvUvmInterfaceCslLogEncryption
 
-    Checks and logs information about non-CSL encryptions, such as those that
-    originate from the GPU.
+    Checks and logs information about encryptions associated with the given
+    CSL context.
 
-    This function does not modify elements of the UvmCslContext.
+    For contexts associated with channels, this function does not modify elements of
+    the UvmCslContext, and must be called for every CPU/GPU encryption.
+
+    For the context associated with fault buffers, bufferSize can encompass multiple
+    encryption invocations, and the UvmCslContext will be updated following a key
+    rotation event.
+
+    In either case the IV remains unmodified after this function is called.
 
     Locking: This function does not acquire an API or GPU lock.
     Memory : This function does not dynamically allocate memory.
-             If called concurrently in different threads with the same UvmCslContext
-             the caller must guarantee exclusion.
+             The caller must guarantee that no CSL function, including this one,
+             is invoked concurrently with the same CSL context.
 
     Arguments:
         uvmCslContext[IN/OUT] - The CSL context.
-        bufferSize[OUT]       - The size of the buffer encrypted by the
+        operation[IN]         - If the CSL context is associated with a fault
+                                buffer, this argument is ignored. If it is
+                                associated with a channel, it must be either
+                                - UVM_CSL_OPERATION_ENCRYPT
+                                - UVM_CSL_OPERATION_DECRYPT
+        bufferSize[IN]        - The size of the buffer(s) encrypted by the
                                 external entity in units of bytes.
 
     Error codes:
-      NV_ERR_INSUFFICIENT_RESOURCES - The device encryption would cause a counter
+      NV_ERR_INSUFFICIENT_RESOURCES - The encryption would cause a counter
                                       to overflow.
 */
-NV_STATUS nvUvmInterfaceCslLogExternalEncryption(UvmCslContext *uvmCslContext,
-                                                 NvU32 bufferSize);
-
+NV_STATUS nvUvmInterfaceCslLogEncryption(UvmCslContext *uvmCslContext,
+                                         UvmCslOperation operation,
+                                         NvU32 bufferSize);
 #endif // _NV_UVM_INTERFACE_H_

kernel-open/common/inc/nv_uvm_types.h

@@ -1,5 +1,5 @@
 /*
- * SPDX-FileCopyrightText: Copyright (c) 2014-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+ * SPDX-FileCopyrightText: Copyright (c) 2014-2025 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
  * SPDX-License-Identifier: MIT
  *
  * Permission is hereby granted, free of charge, to any person obtaining a
@@ -22,7 +22,8 @@
  */
 
 //
-// This file provides common types for both UVM driver and RM's UVM interface.
+// This file provides common types for both the UVM kernel driver and RM's UVM
+// interface.
 //
 
 #ifndef _NV_UVM_TYPES_H_
@@ -32,20 +33,9 @@
 #include "nvstatus.h"
 #include "nvgputypes.h"
 #include "nvCpuUuid.h"
+#include "nv_uvm_user_types.h" // For UvmGpuCachingType, UvmGpuMappingType, etc
 
 
-//
-// Default Page Size if left "0" because in RM BIG page size is default & there
-// are multiple BIG page sizes in RM. These defines are used as flags to "0"
-// should be OK when user is not sure which pagesize allocation it wants
-//
-#define UVM_PAGE_SIZE_DEFAULT    0x0
-#define UVM_PAGE_SIZE_4K         0x1000
-#define UVM_PAGE_SIZE_64K        0x10000
-#define UVM_PAGE_SIZE_128K       0x20000
-#define UVM_PAGE_SIZE_2M         0x200000
-#define UVM_PAGE_SIZE_512M       0x20000000
-
 //
 // When modifying flags, make sure they are compatible with the mirrored
 // PMA_* flags in phys_mem_allocator.h.
@@ -80,9 +70,6 @@
 //
 #define UVM_PMA_CALLED_FROM_PMA_EVICTION        16384
 
-#define UVM_UUID_LEN 16
-#define UVM_SW_OBJ_SUBCHANNEL 5
-
 typedef unsigned long long UvmGpuPointer;
 
 //
@@ -268,6 +255,8 @@ typedef struct UvmGpuChannelInfo_tag
     // The errorNotifier is filled out when the channel hits an RC error.
     NvNotification    *errorNotifier;
 
+    NvNotification    *keyRotationNotifier;
+
     NvU32              hwRunlistId;
     NvU32              hwChannelId;
 
@@ -292,13 +281,14 @@ typedef struct UvmGpuChannelInfo_tag
 
     // GPU VAs of both GPFIFO and GPPUT are needed in Confidential Computing
     // so a channel can be controlled via another channel (SEC2 or WLC/LCIC)
-    NvU64             gpFifoGpuVa;
-    NvU64             gpPutGpuVa;
-    NvU64             gpGetGpuVa;
+    NvU64              gpFifoGpuVa;
+    NvU64              gpPutGpuVa;
+    NvU64              gpGetGpuVa;
+
     // GPU VA of work submission offset is needed in Confidential Computing
     // so CE channels can ring doorbell of other channels as required for
     // WLC/LCIC work submission
-    NvU64             workSubmissionOffsetGpuVa;
+    NvU64              workSubmissionOffsetGpuVa;
 } UvmGpuChannelInfo;
 
 typedef enum
@@ -372,6 +362,9 @@ typedef struct
     // True if the CE can be used for P2P transactions
     NvBool p2p:1;
 
+    // True if the CE supports encryption
+    NvBool secure:1;
+
     // Mask of physical CEs assigned to this LCE
     //
     // The value returned by RM for this field may change when a GPU is
@@ -394,6 +387,7 @@ typedef enum
     UVM_LINK_TYPE_NVLINK_2,
     UVM_LINK_TYPE_NVLINK_3,
     UVM_LINK_TYPE_NVLINK_4,
+    UVM_LINK_TYPE_NVLINK_5,
     UVM_LINK_TYPE_C2C,
 } UVM_LINK_TYPE;
 
@@ -439,80 +433,22 @@ typedef struct UvmGpuAllocInfo_tag
                                       // SEV or GPU CC modes are enabled. Ignored otherwise
 } UvmGpuAllocInfo;
 
-typedef enum
-{
-    UVM_VIRT_MODE_NONE = 0,             // Baremetal or passthrough virtualization
-    UVM_VIRT_MODE_LEGACY = 1,           // Virtualization without SRIOV support
-    UVM_VIRT_MODE_SRIOV_HEAVY = 2,      // Virtualization with SRIOV Heavy configured
-    UVM_VIRT_MODE_SRIOV_STANDARD = 3,   // Virtualization with SRIOV Standard configured
-    UVM_VIRT_MODE_COUNT = 4,
-} UVM_VIRT_MODE;
-
-// !!! The following enums (with UvmRm prefix) are defined and documented in
-// mm/uvm/interface/uvm_types.h and must be mirrored. Please refer to that file
-// for more details.
-
-// UVM GPU mapping types
-typedef enum
-{
-    UvmRmGpuMappingTypeDefault = 0,
-    UvmRmGpuMappingTypeReadWriteAtomic = 1,
-    UvmRmGpuMappingTypeReadWrite = 2,
-    UvmRmGpuMappingTypeReadOnly = 3,
-    UvmRmGpuMappingTypeCount = 4
-} UvmRmGpuMappingType;
-
-// UVM GPU caching types
-typedef enum
-{
-    UvmRmGpuCachingTypeDefault = 0,
-    UvmRmGpuCachingTypeForceUncached = 1,
-    UvmRmGpuCachingTypeForceCached = 2,
-    UvmRmGpuCachingTypeCount = 3
-} UvmRmGpuCachingType;
-
-// UVM GPU format types
-typedef enum {
-   UvmRmGpuFormatTypeDefault = 0,
-   UvmRmGpuFormatTypeBlockLinear = 1,
-   UvmRmGpuFormatTypeCount = 2
-} UvmRmGpuFormatType;
-
-// UVM GPU Element bits types
-typedef enum {
-   UvmRmGpuFormatElementBitsDefault = 0,
-   UvmRmGpuFormatElementBits8 = 1,
-   UvmRmGpuFormatElementBits16 = 2,
-   // Cuda does not support 24-bit width
-   UvmRmGpuFormatElementBits32 = 4,
-   UvmRmGpuFormatElementBits64 = 5,
-   UvmRmGpuFormatElementBits128 = 6,
-   UvmRmGpuFormatElementBitsCount = 7
-} UvmRmGpuFormatElementBits;
-
-// UVM GPU Compression types
-typedef enum {
-    UvmRmGpuCompressionTypeDefault = 0,
-    UvmRmGpuCompressionTypeEnabledNoPlc = 1,
-    UvmRmGpuCompressionTypeCount = 2
-} UvmRmGpuCompressionType;
-
 typedef struct UvmGpuExternalMappingInfo_tag
 {
     // In: GPU caching ability.
-    UvmRmGpuCachingType cachingType;
+    UvmGpuCachingType cachingType;
 
     // In: Virtual permissions.
-    UvmRmGpuMappingType mappingType;
+    UvmGpuMappingType mappingType;
 
     // In: RM virtual mapping memory format
-    UvmRmGpuFormatType formatType;
+    UvmGpuFormatType formatType;
 
     // In: RM virtual mapping element bits
-    UvmRmGpuFormatElementBits elementBits;
+    UvmGpuFormatElementBits elementBits;
 
     // In: RM virtual compression type
-    UvmRmGpuCompressionType compressionType;
+    UvmGpuCompressionType compressionType;
 
     // In: Size of the buffer to store PTEs (in bytes).
     NvU64 pteBufferSize;
@@ -538,8 +474,41 @@ typedef struct UvmGpuExternalMappingInfo_tag
 
     // Out: PTE size (in bytes)
     NvU32 pteSize;
+
+    // Out: UVM needs to invalidate L2 at unmap
+    NvBool bNeedL2InvalidateAtUnmap;
 } UvmGpuExternalMappingInfo;
 
+typedef struct UvmGpuExternalPhysAddrInfo_tag
+{
+    // In: Virtual permissions. Returns
+    // NV_ERR_INVALID_ACCESS_TYPE if input is
+    // inaccurate
+    UvmGpuMappingType mappingType;
+
+    // In: Size of the buffer to store PhysAddrs (in bytes).
+    NvU64 physAddrBufferSize;
+
+    // In: Page size for mapping
+    //     If this field is passed as 0, the page size
+    //     of the allocation is used for mapping.
+    //     nvUvmInterfaceGetExternalAllocPtes must pass
+    //     this field as zero.
+    NvU64 mappingPageSize;
+
+    // In: Pointer to a buffer to store PhysAddrs.
+    // Out: The interface will fill the buffer with PhysAddrs
+    NvU64 *physAddrBuffer;
+
+    // Out: Number of PhysAddrs filled in to the buffer.
+    NvU64 numWrittenPhysAddrs;
+
+    // Out: Number of PhysAddrs remaining to be filled
+    //      if the buffer is not sufficient to accommodate
+    //      requested PhysAddrs.
+    NvU64 numRemainingPhysAddrs;
+} UvmGpuExternalPhysAddrInfo;
+
 typedef struct UvmGpuP2PCapsParams_tag
 {
     // Out: peerId[i] contains gpu[i]'s peer id of gpu[1 - i]. Only defined if
@@ -566,10 +535,10 @@ typedef struct UvmGpuP2PCapsParams_tag
     // bandwidth for indirect peers is zero.
     NvU32 totalLinkLineRateMBps;
 
-    // Out: True if the peers have a indirect link to communicate. On P9
-    // systems, this is true if peers are connected to different NPUs that
-    // forward the requests between them.
-    NvU32 indirectAccess      : 1;
+    // Out: IOMMU/DMA mappings of bar1 of the respective peer vidmem.
+    // Size is 0 if bar1 p2p is not supported.
+    NvU64 bar1DmaAddress[2];
+    NvU64 bar1DmaSize[2];
 } UvmGpuP2PCapsParams;
 
 // Platform-wide information
@@ -592,18 +561,13 @@ typedef struct UvmGpuClientInfo_tag
     NvHandle hSmcPartRef;
 } UvmGpuClientInfo;
 
-typedef enum
-{
-    UVM_GPU_CONF_COMPUTE_MODE_NONE,
-    UVM_GPU_CONF_COMPUTE_MODE_APM,
-    UVM_GPU_CONF_COMPUTE_MODE_HCC,
-    UVM_GPU_CONF_COMPUTE_MODE_COUNT
-} UvmGpuConfComputeMode;
-
 typedef struct UvmGpuConfComputeCaps_tag
 {
-    // Out: GPU's confidential compute mode
-    UvmGpuConfComputeMode mode;
+    // Out: true if Confidential Computing is enabled on the GPU
+    NvBool bConfComputingEnabled;
+
+    // Out: true if key rotation is enabled (for UVM keys) on the GPU
+    NvBool bKeyRotationEnabled;
 } UvmGpuConfComputeCaps;
 
 #define UVM_GPU_NAME_LENGTH 0x40
@@ -660,6 +624,9 @@ typedef struct UvmGpuInfo_tag
     // Maximum number of TPCs per GPC
     NvU32 maxTpcPerGpcCount;
 
+    // Number of access counter buffers.
+    NvU32 accessCntrBufferCount;
+
     // NV_TRUE if SMC is enabled on this GPU.
     NvBool smcEnabled;
 
@@ -706,6 +673,18 @@ typedef struct UvmGpuInfo_tag
 
     // EGM base address to offset in the GMMU PTE entry for EGM mappings
     NvU64    egmBaseAddr;
+
+    // If connectedToSwitch is NV_TRUE,
+    // nvswitchEgmMemoryWindowStart tells the base address for the GPU's EGM memory in the
+    // NVSwitch address space. It is used when creating PTEs of GPU memory mappings
+    // to NVSwitch peers.
+    NvU64 nvswitchEgmMemoryWindowStart;
+
+    // GPU supports ATS capability
+    NvBool atsSupport;
+
+    // GPU supports Non-PASID ATS capability
+    NvBool nonPasidAtsSupport;
 } UvmGpuInfo;
 
 typedef struct UvmGpuFbInfo_tag
@@ -714,10 +693,16 @@ typedef struct UvmGpuFbInfo_tag
     // RM regions that are not registered with PMA either.
     NvU64 maxAllocatableAddress;
 
-    NvU32 heapSize;          // RAM in KB available for user allocations
-    NvU32 reservedHeapSize;  // RAM in KB reserved for internal RM allocation
-    NvBool bZeroFb;          // Zero FB mode enabled.
-    NvU64 maxVidmemPageSize; // Largest GPU page size to access vidmem.
+    NvU32  heapSize;           // RAM in KB available for user allocations
+    NvU32  reservedHeapSize;   // RAM in KB reserved for internal RM allocation
+    NvBool bZeroFb;            // Zero FB mode enabled.
+    NvU64  maxVidmemPageSize;  // Largest GPU page size to access vidmem.
+    NvBool bStaticBar1Enabled; // Static BAR1 mode is enabled
+    NvBool bStaticBar1WriteCombined; // Write combined is enabled
+    NvU64  staticBar1StartOffset;  // The start offset of the the static mapping
+    NvU64  staticBar1Size;     // The size of the static mapping
+    NvU32  heapStart;          // The start offset of heap in KB, helpful for MIG
+                               // systems
 } UvmGpuFbInfo;
 
 typedef struct UvmGpuEccInfo_tag
@@ -729,6 +714,15 @@ typedef struct UvmGpuEccInfo_tag
     NvBool   bEccEnabled;
 } UvmGpuEccInfo;
 
+typedef struct UvmGpuNvlinkInfo_tag
+{
+    unsigned nvlinkMask;
+    unsigned nvlinkOffset;
+    void    *nvlinkReadLocation;
+    NvBool  *nvlinkErrorNotifier;
+    NvBool   bNvlinkRecoveryEnabled;
+} UvmGpuNvlinkInfo;
+
 typedef struct UvmPmaAllocationOptions_tag
 {
     NvU32 flags;
@@ -741,19 +735,6 @@ typedef struct UvmPmaAllocationOptions_tag
     NvU32 resultFlags;          // valid if the allocation function returns NV_OK
 } UvmPmaAllocationOptions;
 
-//
-// Mirrored in PMA (PMA_STATS)
-//
-typedef struct UvmPmaStatistics_tag
-{
-    volatile NvU64 numPages2m;                // PMA-wide 2MB pages count across all regions
-    volatile NvU64 numFreePages64k;           // PMA-wide free 64KB page count across all regions
-    volatile NvU64 numFreePages2m;            // PMA-wide free 2MB pages count across all regions
-    volatile NvU64 numPages2mProtected;       // PMA-wide 2MB pages count in protected memory
-    volatile NvU64 numFreePages64kProtected;  // PMA-wide free 64KB page count in protected memory
-    volatile NvU64 numFreePages2mProtected;   // PMA-wide free 2MB pages count in protected memory
-} UvmPmaStatistics;
-
 /*******************************************************************************
     uvmEventSuspend
     This function will be called by the GPU driver to signal to UVM that the
@@ -845,6 +826,41 @@ typedef NV_STATUS (*uvmEventIsrTopHalf_t) (const NvProcessorUuid *pGpuUuidStruct
 typedef void (*uvmEventIsrTopHalf_t) (void);
 #endif
 
+/*******************************************************************************
+    uvmEventDrainP2P
+    This function will be called by the GPU driver to signal to UVM that the
+    GPU has encountered an uncontained error, and all peer work must be drained
+    to recover.  When it is called, the following assumptions/guarantees are
+    valid/made:
+
+      * Impacted user channels have been preempted and disabled
+      * UVM channels are still running normally and will continue to do
+        so unless an unrecoverable error is hit on said channels
+      * UVM must not return from this function until all enqueued work on
+      * peer channels has drained
+      * In the context of this function call, RM will still service faults
+      * UVM must prevent new peer work from being enqueued until the
+        uvmEventResumeP2P callback is issued
+
+    Returns:
+        NV_OK if UVM has idled peer work and will prevent new peer workloads.
+        NV_ERR_TIMEOUT if peer work was unable to be drained within a timeout
+        XXX NV_ERR_* for any other failure (TBD)
+
+*/
+typedef NV_STATUS (*uvmEventDrainP2P_t) (const NvProcessorUuid *pGpuUuidStruct);
+
+/*******************************************************************************
+    uvmEventResumeP2P
+    This function will be called by the GPU driver to signal to UVM that the
+    GPU has recovered from the previously reported uncontained NVLINK error.
+    When it is called, the following assumptions/guarantees are valid/made:
+
+      * UVM is again allowed to enqueue peer work
+      * UVM channels are still running normally
+*/
+typedef NV_STATUS (*uvmEventResumeP2P_t) (const NvProcessorUuid *pGpuUuidStruct);
+
 struct UvmOpsUvmEvents
 {
     uvmEventSuspend_t     suspend;
@@ -857,6 +873,8 @@ struct UvmOpsUvmEvents
     uvmEventWddmRestartAfterTimeout_t wddmRestartAfterTimeout;
     uvmEventServiceInterrupt_t serviceInterrupt;
 #endif
+    uvmEventDrainP2P_t drainP2P;
+    uvmEventResumeP2P_t resumeP2P;
 };
 
 #define UVM_CSL_SIGN_AUTH_TAG_SIZE_BYTES 32
@@ -921,17 +939,17 @@ typedef struct UvmGpuFaultInfo_tag
         NvU32 replayableFaultMask;
 
         // Fault buffer CPU mapping
-        void*  bufferAddress;
-        //
         // When Confidential Computing is disabled, the mapping points to the
         // actual HW fault buffer.
         //
         // When Confidential Computing is enabled, the mapping points to a
         // copy of the HW fault buffer. This "shadow buffer" is maintained
         // by GSP-RM.
+        void*  bufferAddress;
 
         // Size, in bytes, of the fault buffer pointed by bufferAddress.
         NvU32  bufferSize;
+
         // Mapping pointing to the start of the fault buffer metadata containing
         // a 16Byte authentication tag and a valid byte. Always NULL when
         // Confidential Computing is disabled.
@@ -1009,32 +1027,9 @@ typedef struct UvmGpuAccessCntrInfo_tag
     NvHandle accessCntrBufferHandle;
 } UvmGpuAccessCntrInfo;
 
-typedef enum
-{
-    UVM_ACCESS_COUNTER_GRANULARITY_64K = 1,
-    UVM_ACCESS_COUNTER_GRANULARITY_2M  = 2,
-    UVM_ACCESS_COUNTER_GRANULARITY_16M = 3,
-    UVM_ACCESS_COUNTER_GRANULARITY_16G = 4,
-} UVM_ACCESS_COUNTER_GRANULARITY;
-
-typedef enum
-{
-    UVM_ACCESS_COUNTER_USE_LIMIT_NONE = 1,
-    UVM_ACCESS_COUNTER_USE_LIMIT_QTR  = 2,
-    UVM_ACCESS_COUNTER_USE_LIMIT_HALF = 3,
-    UVM_ACCESS_COUNTER_USE_LIMIT_FULL = 4,
-} UVM_ACCESS_COUNTER_USE_LIMIT;
-
 typedef struct UvmGpuAccessCntrConfig_tag
 {
-    NvU32 mimcGranularity;
-
-    NvU32 momcGranularity;
-
-    NvU32 mimcUseLimit;
-
-    NvU32 momcUseLimit;
-
+    NvU32 granularity;
     NvU32 threshold;
 } UvmGpuAccessCntrConfig;
 
@@ -1064,11 +1059,13 @@ typedef UvmGpuAccessCntrConfig gpuAccessCntrConfig;
 typedef UvmGpuFaultInfo gpuFaultInfo;
 typedef UvmGpuMemoryInfo gpuMemoryInfo;
 typedef UvmGpuExternalMappingInfo gpuExternalMappingInfo;
+typedef UvmGpuExternalPhysAddrInfo gpuExternalPhysAddrInfo;
 typedef UvmGpuChannelResourceInfo gpuChannelResourceInfo;
 typedef UvmGpuChannelInstanceInfo gpuChannelInstanceInfo;
 typedef UvmGpuChannelResourceBindParams gpuChannelResourceBindParams;
 typedef UvmGpuFbInfo gpuFbInfo;
 typedef UvmGpuEccInfo gpuEccInfo;
+typedef UvmGpuNvlinkInfo gpuNvlinkInfo;
 typedef UvmGpuPagingChannel *gpuPagingChannelHandle;
 typedef UvmGpuPagingChannelInfo gpuPagingChannelInfo;
 typedef UvmGpuPagingChannelAllocParams gpuPagingChannelAllocParams;
@@ -1086,4 +1083,21 @@ typedef enum UvmCslOperation
     UVM_CSL_OPERATION_DECRYPT
 } UvmCslOperation;
 
+typedef enum UVM_KEY_ROTATION_STATUS {
+    // Key rotation complete/not in progress
+    UVM_KEY_ROTATION_STATUS_IDLE = 0,
+    // RM is waiting for clients to report their channels are idle for key rotation
+    UVM_KEY_ROTATION_STATUS_PENDING = 1,
+    // Key rotation is in progress
+    UVM_KEY_ROTATION_STATUS_IN_PROGRESS = 2,
+    // Key rotation timeout failure, RM will RC non-idle channels.
+    // UVM should never see this status value.
+    UVM_KEY_ROTATION_STATUS_FAILED_TIMEOUT = 3,
+    // Key rotation failed because upper threshold was crossed, RM will RC non-idle channels
+    UVM_KEY_ROTATION_STATUS_FAILED_THRESHOLD = 4,
+    // Internal RM failure while rotating keys for a certain channel, RM will RC the channel.
+    UVM_KEY_ROTATION_STATUS_FAILED_ROTATION = 5,
+    UVM_KEY_ROTATION_STATUS_MAX_COUNT = 6,
+} UVM_KEY_ROTATION_STATUS;
+
 #endif // _NV_UVM_TYPES_H_

kernel-open/common/inc/nv_uvm_user_types.h

@@ -0,0 +1,166 @@
+/*
+ * SPDX-FileCopyrightText: Copyright (c) 2025 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+ * SPDX-License-Identifier: MIT
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
+ * DEALINGS IN THE SOFTWARE.
+ */
+
+//
+// This file provides common types for the UVM kernel driver, UVM user layer,
+// and RM's UVM interface.
+//
+
+#ifndef _NV_UVM_USER_TYPES_H_
+#define _NV_UVM_USER_TYPES_H_
+
+#include "nvtypes.h"
+
+//
+// Default Page Size if left "0" because in RM BIG page size is default & there
+// are multiple BIG page sizes in RM. These defines are used as flags to "0"
+// should be OK when user is not sure which pagesize allocation it wants
+//
+#define UVM_PAGE_SIZE_DEFAULT    0x0ULL
+#define UVM_PAGE_SIZE_4K         0x1000ULL
+#define UVM_PAGE_SIZE_64K        0x10000ULL
+#define UVM_PAGE_SIZE_128K       0x20000ULL
+#define UVM_PAGE_SIZE_2M         0x200000ULL
+#define UVM_PAGE_SIZE_512M       0x20000000ULL
+#define UVM_PAGE_SIZE_256G       0x4000000000ULL
+
+typedef enum
+{
+    UVM_VIRT_MODE_NONE = 0,             // Baremetal or passthrough virtualization
+    UVM_VIRT_MODE_LEGACY = 1,           // Virtualization without SRIOV support
+    UVM_VIRT_MODE_SRIOV_HEAVY = 2,      // Virtualization with SRIOV Heavy configured
+    UVM_VIRT_MODE_SRIOV_STANDARD = 3,   // Virtualization with SRIOV Standard configured
+    UVM_VIRT_MODE_COUNT = 4,
+} UVM_VIRT_MODE;
+
+//------------------------------------------------------------------------------
+// UVM GPU mapping types
+//
+// These types indicate the kinds of accesses allowed from a given GPU at the
+// specified virtual address range. There are 3 basic kinds of accesses: read,
+// write and atomics. Each type indicates what kinds of accesses are allowed.
+// Accesses of any disallowed kind are fatal. The "Default" type specifies that
+// the UVM driver should decide on the types of accesses allowed.
+//------------------------------------------------------------------------------
+typedef enum
+{
+    UvmGpuMappingTypeDefault = 0,
+    UvmGpuMappingTypeReadWriteAtomic = 1,
+    UvmGpuMappingTypeReadWrite = 2,
+    UvmGpuMappingTypeReadOnly = 3,
+    UvmGpuMappingTypeCount = 4
+} UvmGpuMappingType;
+
+//------------------------------------------------------------------------------
+// UVM GPU caching types
+//
+// These types indicate the cacheability of the specified virtual address range
+// from a given GPU. The "Default" type specifies that the UVM driver should
+// set caching on or off as required to follow the UVM coherence model. The
+// "ForceUncached" and "ForceCached" types will always turn caching off or on
+// respectively. These two types override the cacheability specified by the UVM
+// coherence model.
+//------------------------------------------------------------------------------
+typedef enum
+{
+    UvmGpuCachingTypeDefault = 0,
+    UvmGpuCachingTypeForceUncached = 1,
+    UvmGpuCachingTypeForceCached = 2,
+    UvmGpuCachingTypeCount = 3
+} UvmGpuCachingType;
+
+//------------------------------------------------------------------------------
+// UVM GPU format types
+//
+// These types indicate the memory format of the specified virtual address
+// range for a given GPU. The "Default" type specifies that the UVM driver will
+// detect the format based on the allocation and is mutually inclusive with
+// UvmGpuFormatElementBitsDefault.
+//------------------------------------------------------------------------------
+typedef enum {
+   UvmGpuFormatTypeDefault = 0,
+   UvmGpuFormatTypeBlockLinear = 1,
+   UvmGpuFormatTypeCount = 2
+} UvmGpuFormatType;
+
+//------------------------------------------------------------------------------
+// UVM GPU Element bits types
+//
+// These types indicate the element size of the specified virtual address range
+// for a given GPU. The "Default" type specifies that the UVM driver will
+// detect the element size based on the allocation and is mutually inclusive
+// with UvmGpuFormatTypeDefault. The element size is specified in bits:
+// UvmGpuFormatElementBits8 uses the 8-bits format.
+//------------------------------------------------------------------------------
+typedef enum {
+   UvmGpuFormatElementBitsDefault = 0,
+   UvmGpuFormatElementBits8 = 1,
+   UvmGpuFormatElementBits16 = 2,
+   // Cuda does not support 24-bit width
+   UvmGpuFormatElementBits32 = 4,
+   UvmGpuFormatElementBits64 = 5,
+   UvmGpuFormatElementBits128 = 6,
+   UvmGpuFormatElementBitsCount = 7
+} UvmGpuFormatElementBits;
+
+//------------------------------------------------------------------------------
+// UVM GPU Compression types
+//
+// These types indicate the compression type of the specified virtual address
+// range for a given GPU. The "Default" type specifies that the UVM driver will
+// detect the compression attributes based on the allocation. Any type other
+// than the default will override the compression behavior of the physical
+// allocation. UvmGpuCompressionTypeEnabledNoPlc will disable PLC but enables
+// generic compression. UvmGpuCompressionTypeEnabledNoPlc type is only supported
+// on Turing plus GPUs. Since UvmGpuCompressionTypeEnabledNoPlc type enables
+// generic compression, it can only be used when the compression attribute of
+// the underlying physical allocation is enabled.
+//------------------------------------------------------------------------------
+typedef enum {
+    UvmGpuCompressionTypeDefault = 0,
+    UvmGpuCompressionTypeEnabledNoPlc = 1,
+    UvmGpuCompressionTypeCount = 2
+} UvmGpuCompressionType;
+
+//
+// Mirrored in PMA (PMA_STATS)
+//
+typedef struct UvmPmaStatistics_tag
+{
+    volatile NvU64 numPages2m;                // PMA-wide 2MB pages count across all regions
+    volatile NvU64 numFreePages64k;           // PMA-wide free 64KB page count across all regions
+    volatile NvU64 numFreePages2m;            // PMA-wide free 2MB pages count across all regions
+    volatile NvU64 numPages2mProtected;       // PMA-wide 2MB pages count in protected memory
+    volatile NvU64 numFreePages64kProtected;  // PMA-wide free 64KB page count in protected memory
+    volatile NvU64 numFreePages2mProtected;   // PMA-wide free 2MB pages count in protected memory
+} UvmPmaStatistics;
+
+typedef enum
+{
+    UVM_ACCESS_COUNTER_GRANULARITY_64K = 1,
+    UVM_ACCESS_COUNTER_GRANULARITY_2M  = 2,
+    UVM_ACCESS_COUNTER_GRANULARITY_16M = 3,
+    UVM_ACCESS_COUNTER_GRANULARITY_16G = 4,
+} UVM_ACCESS_COUNTER_GRANULARITY;
+
+#endif // _NV_UVM_USER_TYPES_H_

kernel-open/common/inc/nvi2c.h

@@ -0,0 +1,37 @@
+/*
+ * SPDX-FileCopyrightText: Copyright (c) 2019 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+ * SPDX-License-Identifier: MIT
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
+ * DEALINGS IN THE SOFTWARE.
+ */
+#ifndef _NV_I2C_H_
+#define _NV_I2C_H_
+
+#define NV_I2C_MSG_WR 0x0000
+#define NV_I2C_MSG_RD 0x0001
+
+typedef struct nv_i2c_msg_s
+{
+    NvU16 addr;
+    NvU16 flags;
+    NvU16 len;
+    NvU8* buf;
+} nv_i2c_msg_t;
+
+#endif

kernel-open/common/inc/nvimpshared.h

@@ -0,0 +1,96 @@
+/*
+ * SPDX-FileCopyrightText: Copyright (c) 2020-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+ * SPDX-License-Identifier: MIT
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
+ * DEALINGS IN THE SOFTWARE.
+ */
+
+/******************************************************************************\
+*                                                                              *
+*  Description:                                                                *
+*    Accommodates sharing of IMP-related structures between kernel interface   *
+*    files and core RM.                                                        *
+*                                                                              *
+\******************************************************************************/
+
+#pragma once
+
+#include <nvtypes.h>
+#if defined(_MSC_VER)
+#pragma warning(disable:4324)
+#endif
+
+//
+// This file was generated with FINN, an NVIDIA coding tool.
+// Source file:      nvimpshared.finn
+//
+
+
+
+
+//
+// There are only a small number of discrete dramclk frequencies available on
+// the system.  This structure contains IMP-relevant information associated
+// with a specific dramclk frequency.
+//
+typedef struct DRAM_CLK_INSTANCE {
+    NvU32 dram_clk_freq_khz;
+
+    NvU32 mchub_clk_khz;
+
+    NvU32 mc_clk_khz;
+
+    NvU32 max_iso_bw_kbps;
+
+    //
+    // switch_latency_ns is the maximum time required to switch the dramclk
+    // frequency to the frequency specified in dram_clk_freq_khz.
+    //
+    NvU32 switch_latency_ns;
+} DRAM_CLK_INSTANCE;
+
+//
+// This table is used to collect information from other modules that is needed
+// for RM IMP calculations.  (Used on Tegra only.)
+//
+typedef struct TEGRA_IMP_IMPORT_DATA {
+    //
+    // max_iso_bw_kbps stores the maximum possible ISO bandwidth available to
+    // display, assuming display is the only active ISO client.  (Note that ISO
+    // bandwidth will typically be allocated to multiple clients, so display
+    // will generally not have access to the maximum possible bandwidth.)
+    //
+    NvU32             max_iso_bw_kbps;
+
+    // On Orin, each dram channel is 16 bits wide.
+    NvU32             num_dram_channels;
+
+    //
+    // dram_clk_instance stores entries for all possible dramclk frequencies,
+    // sorted by dramclk frequency in increasing order.
+    //
+    // "24" is expected to be larger than the actual number of required entries
+    // (which is provided by a BPMP API), but it can be increased if necessary.
+    //
+    // num_dram_clk_entries is filled in with the actual number of distinct
+    // dramclk entries.
+    //
+    NvU32             num_dram_clk_entries;
+    DRAM_CLK_INSTANCE dram_clk_instance[24];
+} TEGRA_IMP_IMPORT_DATA;

kernel-open/common/inc/nvkms-api-types.h

@@ -50,6 +50,8 @@
 #define NVKMS_LOG2_LUT_ARRAY_SIZE             10
 #define NVKMS_LUT_ARRAY_SIZE                  (1 << NVKMS_LOG2_LUT_ARRAY_SIZE)
 
+#define NVKMS_OLUT_FP_NORM_SCALE_DEFAULT      0xffffffff
+
 typedef NvU32 NvKmsDeviceHandle;
 typedef NvU32 NvKmsDispHandle;
 typedef NvU32 NvKmsConnectorHandle;
@@ -245,6 +247,80 @@ struct NvKmsLutRamps {
     NvU16 blue[NVKMS_LUT_ARRAY_SIZE];  /*! in */
 };
 
+/* Datatypes for LUT capabilities */
+enum NvKmsLUTFormat {
+    /*
+     * Normalized fixed-point format mapping [0, 1] to [0x0, 0xFFFF].
+     */
+    NVKMS_LUT_FORMAT_UNORM16,
+
+    /*
+     * Half-precision floating point.
+     */
+    NVKMS_LUT_FORMAT_FP16,
+
+    /*
+     * 14-bit fixed-point format required to work around hardware bug 813188.
+     *
+     * To convert from UNORM16 to UNORM14_WAR_813188:
+     * unorm14_war_813188 = ((unorm16 >> 2) & ~7) + 0x6000
+     */
+    NVKMS_LUT_FORMAT_UNORM14_WAR_813188
+};
+
+enum NvKmsLUTVssSupport {
+    NVKMS_LUT_VSS_NOT_SUPPORTED,
+    NVKMS_LUT_VSS_SUPPORTED,
+    NVKMS_LUT_VSS_REQUIRED,
+};
+
+enum NvKmsLUTVssType {
+    NVKMS_LUT_VSS_TYPE_NONE,
+    NVKMS_LUT_VSS_TYPE_LINEAR,
+    NVKMS_LUT_VSS_TYPE_LOGARITHMIC,
+};
+
+struct NvKmsLUTCaps {
+    /*! Whether this layer or head on this device supports this LUT stage. */
+    NvBool supported;
+
+    /*! Whether this LUT supports VSS. */
+    enum NvKmsLUTVssSupport vssSupport;
+
+    /*!
+     * The type of VSS segmenting this LUT uses.
+     */
+    enum NvKmsLUTVssType vssType;
+
+    /*!
+     * Expected number of VSS segments.
+     */
+    NvU32 vssSegments;
+
+    /*!
+     * Expected number of LUT entries.
+     */
+    NvU32 lutEntries;
+
+    /*!
+     * Format for each of the LUT entries.
+     */
+    enum NvKmsLUTFormat entryFormat;
+};
+
+/* each LUT entry uses this many bytes */
+#define NVKMS_LUT_CAPS_LUT_ENTRY_SIZE (4 * sizeof(NvU16))
+
+/* if the LUT surface uses VSS, size of the VSS header */
+#define NVKMS_LUT_VSS_HEADER_SIZE (4 * NVKMS_LUT_CAPS_LUT_ENTRY_SIZE)
+
+struct NvKmsLUTSurfaceParams {
+    NvKmsSurfaceHandle surfaceHandle;
+    NvU64 offset NV_ALIGN_BYTES(8);
+    NvU32 vssSegments;
+    NvU32 lutEntries;
+};
+
 /*
  * A 3x4 row-major colorspace conversion matrix.
  *
@@ -440,9 +516,9 @@ struct NvKmsLayerCapabilities {
     NvBool supportsWindowMode              :1;
 
     /*!
-     * Whether layer supports HDR pipe.
+     * Whether layer supports ICtCp pipe.
      */
-    NvBool supportsHDR                     :1;
+    NvBool supportsICtCp                   :1;
 
 
     /*!
@@ -463,6 +539,10 @@ struct NvKmsLayerCapabilities {
      * still expected to honor the NvKmsUsageBounds for each head.
      */
     NvU64 supportedSurfaceMemoryFormats NV_ALIGN_BYTES(8);
+
+    /* Capabilities for each LUT stage in the EVO3 precomp pipeline. */
+    struct NvKmsLUTCaps ilut;
+    struct NvKmsLUTCaps tmo;
 };
 
 /*!
@@ -560,22 +640,28 @@ enum NvKmsInputColorRange {
      * If DEFAULT is provided, driver will assume full range for RGB formats
      * and limited range for YUV formats.
      */
-    NVKMS_INPUT_COLORRANGE_DEFAULT = 0,
+    NVKMS_INPUT_COLOR_RANGE_DEFAULT = 0,
 
-    NVKMS_INPUT_COLORRANGE_LIMITED = 1,
+    NVKMS_INPUT_COLOR_RANGE_LIMITED = 1,
 
-    NVKMS_INPUT_COLORRANGE_FULL = 2,
+    NVKMS_INPUT_COLOR_RANGE_FULL = 2,
 };
 
 enum NvKmsInputColorSpace {
-    /* Unknown colorspace; no de-gamma will be applied */
-    NVKMS_INPUT_COLORSPACE_NONE = 0,
+    /* Unknown colorspace */
+    NVKMS_INPUT_COLOR_SPACE_NONE = 0,
 
-    /* Linear, Rec.709 [-0.5, 7.5) */
-    NVKMS_INPUT_COLORSPACE_SCRGB_LINEAR = 1,
+    NVKMS_INPUT_COLOR_SPACE_BT601 = 1,
+    NVKMS_INPUT_COLOR_SPACE_BT709 = 2,
+    NVKMS_INPUT_COLOR_SPACE_BT2020 = 3,
+    NVKMS_INPUT_COLOR_SPACE_BT2100 = NVKMS_INPUT_COLOR_SPACE_BT2020,
 
-    /* PQ, Rec.2020 unity */
-    NVKMS_INPUT_COLORSPACE_BT2100_PQ = 2,
+    NVKMS_INPUT_COLOR_SPACE_SCRGB = 4
+};
+
+enum NvKmsInputTf {
+    NVKMS_INPUT_TF_LINEAR = 0,
+    NVKMS_INPUT_TF_PQ = 1
 };
 
 enum NvKmsOutputColorimetry {
@@ -683,4 +769,20 @@ struct NvKmsSuperframeInfo {
     } view[NVKMS_MAX_SUPERFRAME_VIEWS];
 };
 
+/* Fields within NvKmsVblankSemControlDataOneHead::flags */
+#define NVKMS_VBLANK_SEM_CONTROL_SWAP_INTERVAL          15:0
+
+struct NvKmsVblankSemControlDataOneHead {
+    NvU32 requestCounterAccel;
+    NvU32 requestCounter;
+    NvU32 flags;
+
+    NvU32 semaphore;
+    NvU64 vblankCount NV_ALIGN_BYTES(8);
+};
+
+struct NvKmsVblankSemControlData {
+    struct NvKmsVblankSemControlDataOneHead head[NV_MAX_HEADS];
+};
+
 #endif /* NVKMS_API_TYPES_H */

kernel-open/common/inc/nvkms-kapi.h

@@ -24,8 +24,10 @@
 #if !defined(__NVKMS_KAPI_H__)
 
 #include "nvtypes.h"
+#include "nv_mig_types.h"
 
 #include "nv-gpu-info.h"
+#include "nv_dpy_id.h"
 #include "nvkms-api-types.h"
 #include "nvkms-format.h"
 
@@ -124,6 +126,14 @@ struct NvKmsKapiDisplayMode {
 #define NVKMS_KAPI_LAYER_INVALID_IDX           0xff
 #define NVKMS_KAPI_LAYER_PRIMARY_IDX              0
 
+struct NvKmsKapiLutCaps {
+    struct {
+        struct NvKmsLUTCaps ilut;
+        struct NvKmsLUTCaps tmo;
+    } layer[NVKMS_KAPI_LAYER_MAX];
+    struct NvKmsLUTCaps olut;
+};
+
 struct NvKmsKapiDeviceResourcesInfo {
 
     NvU32 numHeads;
@@ -158,13 +168,25 @@ struct NvKmsKapiDeviceResourcesInfo {
 
         NvU32 hasVideoMemory;
 
+        NvU32 numDisplaySemaphores;
+
         NvU8  genericPageKind;
 
         NvBool  supportsSyncpts;
+
+        NvBool requiresVrrSemaphores;
+
+        NvBool  supportsInputColorRange;
+        NvBool  supportsInputColorSpace;
     } caps;
 
     NvU64 supportedSurfaceMemoryFormats[NVKMS_KAPI_LAYER_MAX];
-    NvBool supportsHDR[NVKMS_KAPI_LAYER_MAX];
+    NvBool supportsICtCp[NVKMS_KAPI_LAYER_MAX];
+
+    struct NvKmsKapiLutCaps lutCaps;
+
+    NvU64 vtFbBaseAddress;
+    NvU64 vtFbSize;
 };
 
 #define NVKMS_KAPI_LAYER_MASK(layerType) (1 << (layerType))
@@ -190,6 +212,7 @@ struct NvKmsKapiConnectorInfo {
     NvU32        numIncompatibleConnectors;
     NvKmsKapiConnector incompatibleConnectorHandles[NVKMS_KAPI_MAX_CONNECTORS];
 
+    NVDpyIdList dynamicDpyIdList;
 };
 
 struct NvKmsKapiStaticDisplayInfo {
@@ -208,20 +231,30 @@ struct NvKmsKapiStaticDisplayInfo {
     NvKmsKapiDisplay possibleCloneHandles[NVKMS_KAPI_MAX_CLONE_DISPLAYS];
 
     NvU32 headMask;
+
+    NvBool isDpMST;
 };
 
-struct NvKmsKapiSyncpt {
+struct NvKmsKapiSyncParams {
+    union {
+        struct {
+            /*!
+             * Possible syncpt use case in kapi.
+             * For pre-syncpt, use only id and value
+             * and for post-syncpt, use only fd.
+             */
+            NvU32   preSyncptId;
+            NvU32   preSyncptValue;
+        } syncpt;
 
-    /*!
-     * Possible syncpt use case in kapi.
-     * For pre-syncpt, use only id and value
-     * and for post-syncpt, use only fd.
-     */
-    NvBool  preSyncptSpecified;
-    NvU32   preSyncptId;
-    NvU32   preSyncptValue;
+        struct {
+            NvU32 index;
+        } semaphore;
+    } u;
 
-    NvBool  postSyncptRequested;
+    NvBool preSyncptSpecified;
+    NvBool postSyncptRequested;
+    NvBool semaphoreSpecified;
 };
 
 struct NvKmsKapiLayerConfig {
@@ -231,14 +264,15 @@ struct NvKmsKapiLayerConfig {
         NvU8 surfaceAlpha;
     } compParams;
     struct NvKmsRRParams rrParams;
-    struct NvKmsKapiSyncpt syncptParams;
+    struct NvKmsKapiSyncParams syncParams;
 
     struct {
         struct NvKmsHDRStaticMetadata val;
         NvBool enabled;
     } hdrMetadata;
 
-    enum NvKmsOutputTf tf;
+    enum NvKmsInputTf inputTf;
+    enum NvKmsOutputTf outputTf;
 
     NvU8 minPresentInterval;
     NvBool tearing;
@@ -250,21 +284,58 @@ struct NvKmsKapiLayerConfig {
     NvU16 dstWidth, dstHeight;
 
     enum NvKmsInputColorSpace inputColorSpace;
+    enum NvKmsInputColorRange inputColorRange;
+
+    struct {
+        NvBool enabled;
+        struct NvKmsKapiSurface *lutSurface;
+        NvU64 offset;
+        NvU32 vssSegments;
+        NvU32 lutEntries;
+    } ilut;
+
+    struct {
+        NvBool enabled;
+        struct NvKmsKapiSurface *lutSurface;
+        NvU64 offset;
+        NvU32 vssSegments;
+        NvU32 lutEntries;
+    } tmo;
+
     struct NvKmsCscMatrix csc;
     NvBool cscUseMain;
+
+    struct {
+        struct NvKmsCscMatrix lmsCtm;
+        struct NvKmsCscMatrix lmsToItpCtm;
+        struct NvKmsCscMatrix itpToLmsCtm;
+        struct NvKmsCscMatrix blendCtm;
+        struct {
+            NvBool lmsCtm      : 1;
+            NvBool lmsToItpCtm : 1;
+            NvBool itpToLmsCtm : 1;
+            NvBool blendCtm    : 1;
+        } enabled;
+    } matrixOverrides;
 };
 
 struct NvKmsKapiLayerRequestedConfig {
     struct NvKmsKapiLayerConfig config;
     struct {
-        NvBool surfaceChanged     : 1;
-        NvBool srcXYChanged       : 1;
-        NvBool srcWHChanged       : 1;
-        NvBool dstXYChanged       : 1;
-        NvBool dstWHChanged       : 1;
-        NvBool cscChanged         : 1;
-        NvBool tfChanged          : 1;
-        NvBool hdrMetadataChanged : 1;
+        NvBool surfaceChanged          : 1;
+        NvBool srcXYChanged            : 1;
+        NvBool srcWHChanged            : 1;
+        NvBool dstXYChanged            : 1;
+        NvBool dstWHChanged            : 1;
+        NvBool cscChanged              : 1;
+        NvBool inputTfChanged          : 1;
+        NvBool outputTfChanged         : 1;
+        NvBool inputColorSpaceChanged  : 1;
+        NvBool inputColorRangeChanged  : 1;
+        NvBool hdrMetadataChanged      : 1;
+        NvBool matrixOverridesChanged  : 1;
+        NvBool ilutChanged             : 1;
+        NvBool tmoChanged              : 1;
     } flags;
 };
 
@@ -319,7 +390,6 @@ struct NvKmsKapiHeadModeSetConfig {
 
     struct {
         struct {
-            NvBool specified;
             NvU32 depth;
             NvU32 start;
             NvU32 end;
@@ -327,22 +397,34 @@ struct NvKmsKapiHeadModeSetConfig {
         } input;
 
         struct {
-            NvBool specified;
             NvBool enabled;
             struct NvKmsLutRamps *pRamps;
         } output;
     } lut;
+
+    struct {
+        NvBool enabled;
+        struct NvKmsKapiSurface *lutSurface;
+        NvU64 offset;
+        NvU32 vssSegments;
+        NvU32 lutEntries;
+    } olut;
+
+    NvU32 olutFpNormScale;
 };
 
 struct NvKmsKapiHeadRequestedConfig {
     struct NvKmsKapiHeadModeSetConfig modeSetConfig;
     struct {
-        NvBool activeChanged       : 1;
-        NvBool displaysChanged     : 1;
-        NvBool modeChanged         : 1;
-        NvBool hdrInfoFrameChanged : 1;
-        NvBool colorimetryChanged  : 1;
-        NvBool lutChanged      : 1;
+        NvBool activeChanged          : 1;
+        NvBool displaysChanged        : 1;
+        NvBool modeChanged            : 1;
+        NvBool hdrInfoFrameChanged    : 1;
+        NvBool colorimetryChanged     : 1;
+        NvBool legacyIlutChanged      : 1;
+        NvBool legacyOlutChanged      : 1;
+        NvBool olutChanged            : 1;
+        NvBool olutFpNormScaleChanged : 1;
     } flags;
 
     struct NvKmsKapiCursorRequestedConfig cursorRequestedConfig;
@@ -368,6 +450,8 @@ struct NvKmsKapiHeadReplyConfig {
 
 struct NvKmsKapiModeSetReplyConfig {
     enum NvKmsFlipResult flipResult;
+    NvBool vrrFlip;
+    NvS32 vrrSemaphoreIndex;
     struct NvKmsKapiHeadReplyConfig
         headReplyConfig[NVKMS_KAPI_MAX_HEADS];
 };
@@ -413,6 +497,8 @@ struct NvKmsKapiEvent {
 struct NvKmsKapiAllocateDeviceParams {
     /* [IN] GPU ID obtained from enumerateGpus() */
     NvU32 gpuId;
+    /* [IN] MIG device if requested */
+    MIGDeviceId migDevice;
 
     /* [IN] Private data of device allocator */
     void *privateData;
@@ -484,6 +570,28 @@ enum NvKmsKapiAllocationType {
     NVKMS_KAPI_ALLOCATION_TYPE_OFFSCREEN = 2,
 };
 
+struct NvKmsKapiAllocateMemoryParams {
+    /* [IN] BlockLinear or Pitch */
+    enum NvKmsSurfaceMemoryLayout layout;
+
+    /* [IN] Allocation type */
+    enum NvKmsKapiAllocationType type;
+
+    /* [IN] Size, in bytes, of the memory to allocate */
+    NvU64 size;
+
+    /* [IN] Whether memory can be updated directly on the screen */
+    NvBool noDisplayCaching;
+
+    /* [IN] Whether to allocate memory from video memory or system memory */
+    NvBool useVideoMemory;
+
+    /* [IN/OUT] For input, non-zero if compression backing store should be
+     * allocated for the memory, for output, non-zero if compression backing
+     * store was allocated for the memory */
+    NvU8 *compressible;
+};
+
 typedef enum NvKmsKapiRegisterWaiterResultRec {
     NVKMS_KAPI_REG_WAITER_FAILED,
     NVKMS_KAPI_REG_WAITER_SUCCESS,
@@ -492,6 +600,11 @@ typedef enum NvKmsKapiRegisterWaiterResultRec {
 
 typedef void NvKmsKapiSuspendResumeCallbackFunc(NvBool suspend);
 
+struct NvKmsKapiGpuInfo {
+    nv_gpu_info_t gpuInfo;
+    MIGDeviceId   migDevice;
+};
+
 struct NvKmsKapiFunctionsTable {
 
     /*!
@@ -508,14 +621,19 @@ struct NvKmsKapiFunctionsTable {
     } systemInfo;
 
     /*!
-     * Enumerate the available physical GPUs that can be used with NVKMS.
+     * Enumerate the available GPUs that can be used with NVKMS.
      *
-     * \param [out]  gpuInfo  The information of the enumerated GPUs.
-     *                        It is an array of NVIDIA_MAX_GPUS elements.
+     * The gpuCallback will be called with a NvKmsKapiGpuInfo for each
+     * physical and MIG GPU currently available in the system.
+     *
+     * \param [in] gpuCallback          Client function to handle each GPU.
      *
      * \return  Count of enumerated gpus.
      */
-    NvU32 (*enumerateGpus)(nv_gpu_info_t *gpuInfo);
+    NvU32 (*enumerateGpus)
+    (
+        void (*gpuCallback)(const struct NvKmsKapiGpuInfo *info)
+    );
 
     /*!
      * Allocate an NVK device using which you can query/allocate resources on
@@ -745,66 +863,22 @@ struct NvKmsKapiFunctionsTable {
     );
 
     /*!
-     * Allocate some unformatted video memory of the specified size.
+     * Allocate some unformatted video or system memory of the specified size.
      *
-     * This function allocates video memory on the specified GPU.
-     * It should be suitable for mapping on the CPU as a pitch
-     * linear or block-linear surface.
+     * This function allocates video or system memory on the specified GPU. It
+     * should be suitable for mapping on the CPU as a pitch linear or
+     * block-linear surface.
      *
-     * \param [in] device  A device allocated using allocateDevice().
+     * \param [in]     device  A device allocated using allocateDevice().
      *
-     * \param [in] layout  BlockLinear or Pitch.
-     * 
-     * \param [in] type    Allocation type.
-     *
-     * \param [in] size    Size, in bytes, of the memory to allocate.
-     *
-     * \param [in/out] compressible For input, non-zero if compression
-     *                              backing store should be allocated for
-     *                              the memory, for output, non-zero if
-     *                              compression backing store was
-     *                              allocated for the memory.
+     * \param [in/out] params  Parameters required for memory allocation.
      *
      * \return An valid memory handle on success, NULL on failure.
      */
-    struct NvKmsKapiMemory* (*allocateVideoMemory)
+    struct NvKmsKapiMemory* (*allocateMemory)
     (
         struct NvKmsKapiDevice *device,
-        enum NvKmsSurfaceMemoryLayout layout,
-        enum NvKmsKapiAllocationType type,
-        NvU64 size,
-        NvU8 *compressible
-    );
-
-    /*!
-     * Allocate some unformatted system memory of the specified size.
-     *
-     * This function allocates system memory . It should be suitable
-     * for mapping on the CPU as a pitch linear or block-linear surface.
-     *
-     * \param [in] device  A device allocated using allocateDevice().
-     *
-     * \param [in] layout  BlockLinear or Pitch.
-     * 
-     * \param [in] type    Allocation type.
-     *
-     * \param [in] size    Size, in bytes, of the memory to allocate.
-     *
-     * \param [in/out] compressible For input, non-zero if compression
-     *                              backing store should be allocated for
-     *                              the memory, for output, non-zero if
-     *                              compression backing store was
-     *                              allocated for the memory.
-     *
-     * \return An valid memory handle on success, NULL on failure.
-     */
-    struct NvKmsKapiMemory* (*allocateSystemMemory)
-    (
-        struct NvKmsKapiDevice *device,
-        enum NvKmsSurfaceMemoryLayout layout,
-        enum NvKmsKapiAllocationType type,
-        NvU64 size,
-        NvU8 *compressible
+        struct NvKmsKapiAllocateMemoryParams *params
     );
 
     /*!
@@ -943,6 +1017,17 @@ struct NvKmsKapiFunctionsTable {
         const void *pLinearAddress
     );
 
+    /*!
+     * Check if memory object allocated is video memory.
+     *
+     * \param [in]  memory           Memory allocated using allocateMemory()
+     *
+     * \return NV_TRUE if memory is vidmem, NV_FALSE otherwise.
+     */
+    NvBool (*isVidmem)(
+        const struct NvKmsKapiMemory *memory
+    );
+
     /*!
      * Create a formatted surface from an NvKmsKapiMemory object.
      *
@@ -1159,21 +1244,6 @@ struct NvKmsKapiFunctionsTable {
         NvU64 *pPages
     );
 
-     /*!
-     * Check if this memory object can be scanned out for display.
-     *
-     * \param [in]  device  A device allocated using allocateDevice().
-     *
-     * \param [in]  memory  The memory object to check for display support.
-     *
-     * \return NV_TRUE if this memory can be displayed, NV_FALSE if not.
-     */
-    NvBool (*isMemoryValidForDisplay)
-    (
-        const struct NvKmsKapiDevice *device,
-        const struct NvKmsKapiMemory *memory
-    );
-
     /*
      * Import SGT as a memory handle.
      *
@@ -1410,6 +1480,117 @@ struct NvKmsKapiFunctionsTable {
     (
         NvKmsKapiSuspendResumeCallbackFunc *function
     );
+
+    /*!
+     * Immediately initialize the specified display semaphore to the pending state.
+     *
+     * Must be called prior to applying a mode set that utilizes the specified
+     * display semaphore for synchronization.
+     *
+     * \param [in] device         The device which will utilize the semaphore.
+     *
+     * \param [in] semaphoreIndex Index of the desired semaphore within the
+     *                            NVKMS semaphore pool. Must be less than
+     *                            NvKmsKapiDeviceResourcesInfo::caps::numDisplaySemaphores
+     *                            for the specified device.
+     */
+    NvBool
+    (*tryInitDisplaySemaphore)
+    (
+        struct NvKmsKapiDevice *device,
+        NvU32 semaphoreIndex
+    );
+
+    /*!
+     * Immediately set the specified display semaphore to the displayable state.
+     *
+     * Must be called after \ref tryInitDisplaySemaphore to indicate a mode
+     * configuration change that utilizes the specified display semaphore for
+     * synchronization may proceed.
+     *
+     * \param [in] device         The device which will utilize the semaphore.
+     *
+     * \param [in] semaphoreIndex Index of the desired semaphore within the
+     *                            NVKMS semaphore pool. Must be less than
+     *                            NvKmsKapiDeviceResourcesInfo::caps::numDisplaySemaphores
+     *                            for the specified device.
+     */
+    void
+    (*signalDisplaySemaphore)
+    (
+        struct NvKmsKapiDevice *device,
+        NvU32 semaphoreIndex
+    );
+
+    /*!
+     * Immediately cancel use of a display semaphore by resetting its value to
+     * its initial state.
+     *
+     * This can be used by clients to restore a semaphore to a consistent state
+     * when they have prepared it for use by previously calling
+     * \ref tryInitDisplaySemaphore() on it, but are then prevented from
+     * submitting the associated hardware operations to consume it due to the
+     * subsequent failure of some software or hardware operation.
+     *
+     * \param [in] device         The device which will utilize the semaphore.
+     *
+     * \param [in] semaphoreIndex Index of the desired semaphore within the
+     *                            NVKMS semaphore pool. Must be less than
+     *                            NvKmsKapiDeviceResourcesInfo::caps::numDisplaySemaphores
+     *                            for the specified device.
+     */
+    void
+    (*cancelDisplaySemaphore)
+    (
+        struct NvKmsKapiDevice *device,
+        NvU32 semaphoreIndex
+    );
+
+    /*!
+     * Signal the VRR semaphore at the specified index from the CPU.
+     * If device does not support VRR semaphores, this is a no-op.
+     * Returns true if signal is success or no-op, otherwise returns false.
+     *
+     * \param [in]  device  A device allocated using allocateDevice().
+     *
+     * \param [in]  index   The VRR semaphore index to be signalled.
+     */
+    NvBool
+    (*signalVrrSemaphore)
+    (
+        struct NvKmsKapiDevice *device,
+        NvS32 index
+    );
+
+    /*!
+     * Check or wait on a head's LUT notifier.
+     *
+     * \param [in]  device              A device allocated using allocateDevice().
+     *
+     * \param [in]  head                The head to check for LUT completion.
+     *
+     * \param [in]  waitForCompletion   If true, wait for the notifier in NvKms
+     *                                  before returning.
+     *
+     * \param [out] complete            Returns whether the notifier has completed.
+     */
+    NvBool
+    (*checkLutNotifier)
+    (
+        struct NvKmsKapiDevice *device,
+        NvU32 head,
+        NvBool waitForCompletion
+    );
+
+    /*
+     * Notify NVKMS that the system's framebuffer console has been disabled and
+     * the reserved allocation for the old framebuffer console can be unmapped.
+     */
+    void
+    (*framebufferConsoleDisabled)
+    (
+        struct NvKmsKapiDevice *device
+    );
 };
 
 /** @} */
@@ -1424,6 +1605,20 @@ NvBool nvKmsKapiGetFunctionsTable
     struct NvKmsKapiFunctionsTable *funcsTable
 );
 
+NvU32 nvKmsKapiF16ToF32(NvU16 a);
+
+NvU16 nvKmsKapiF32ToF16(NvU32 a);
+
+NvU32 nvKmsKapiF32Mul(NvU32 a, NvU32 b);
+
+NvU32 nvKmsKapiF32Div(NvU32 a, NvU32 b);
+
+NvU32 nvKmsKapiF32Add(NvU32 a, NvU32 b);
+
+NvU32 nvKmsKapiF32ToUI32RMinMag(NvU32 a, NvBool exact);
+
+NvU32 nvKmsKapiUI32ToF32(NvU32 a);
+
 /** @} */
 
 #endif /* defined(__NVKMS_KAPI_H__) */

kernel-open/common/inc/nvlimits.h

@@ -1,5 +1,5 @@
 /*
- * SPDX-FileCopyrightText: Copyright (c) 2017 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+ * SPDX-FileCopyrightText: Copyright (c) 2017-2024 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
  * SPDX-License-Identifier: MIT
  *
  * Permission is hereby granted, free of charge, to any person obtaining a
@@ -34,19 +34,25 @@
 /*
  * This is the maximum number of GPUs supported in a single system.
  */
-#define NV_MAX_DEVICES          32
+#define NV_MAX_DEVICES                32
 
 /*
  * This is the maximum number of subdevices within a single device.
  */
-#define NV_MAX_SUBDEVICES       8
+#define NV_MAX_SUBDEVICES             8
 
 /*
  * This is the maximum length of the process name string.
  */
-#define NV_PROC_NAME_MAX_LENGTH 100U
+#define NV_PROC_NAME_MAX_LENGTH       100U
 
 /*
  * This is the maximum number of heads per GPU.
  */
-#define NV_MAX_HEADS            4
+#define NV_MAX_HEADS                  4
+
+/*
+ * Maximum length of a MIG device UUID. It is a 36-byte UUID string plus a
+ * 4-byte prefix and NUL terminator: 'M' 'I' 'G' '-' UUID '\0x0'
+ */
+#define NV_MIG_DEVICE_UUID_STR_LENGTH 41U

kernel-open/common/inc/nvmisc.h

@@ -1,5 +1,5 @@
 /*
- * SPDX-FileCopyrightText: Copyright (c) 1993-2020 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+ * SPDX-FileCopyrightText: Copyright (c) 1993-2024 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
  * SPDX-License-Identifier: MIT
  *
  * Permission is hereby granted, free of charge, to any person obtaining a
@@ -33,43 +33,26 @@ extern "C" {
 
 #include "nvtypes.h"
 
-#if !defined(NVIDIA_UNDEF_LEGACY_BIT_MACROS)
-//
-// Miscellaneous macros useful for bit field manipulations
-//
-// STUPID HACK FOR CL 19434692.  Will revert when fix CL is delivered bfm -> chips_a.
-#ifndef BIT
-#define BIT(b)                  (1U<<(b))
-#endif
-#ifndef BIT32
-#define BIT32(b)                ((NvU32)1U<<(b))
-#endif
-#ifndef BIT64
-#define BIT64(b)                ((NvU64)1U<<(b))
-#endif
-
-#endif
-
-//
-// It is recommended to use the following bit macros to avoid macro name
-// collisions with other src code bases.
-//
+// Miscellaneous macros useful for bit field manipulations.
 #ifndef NVBIT
-#define NVBIT(b)                  (1U<<(b))
+#define NVBIT(b)         (1U<<(b))
 #endif
 #ifndef NVBIT_TYPE
-#define NVBIT_TYPE(b, t)          (((t)1U)<<(b))
+#define NVBIT_TYPE(b, t) (((t)1U)<<(b))
 #endif
 #ifndef NVBIT32
-#define NVBIT32(b)                NVBIT_TYPE(b, NvU32)
+#define NVBIT32(b)       NVBIT_TYPE(b, NvU32)
 #endif
 #ifndef NVBIT64
-#define NVBIT64(b)                NVBIT_TYPE(b, NvU64)
+#define NVBIT64(b)       NVBIT_TYPE(b, NvU64)
 #endif
 
+//Concatenate 2 32bit values to a 64bit value
+#define NV_CONCAT_32_TO_64(hi, lo) ((((NvU64)hi) << 32) | ((NvU64)lo))
+
 // Helper macro's for 32 bit bitmasks
 #define NV_BITMASK32_ELEMENT_SIZE            (sizeof(NvU32) << 3)
-#define NV_BITMASK32_IDX(chId)               (((chId) & ~(0x1F)) >> 5)  
+#define NV_BITMASK32_IDX(chId)               (((chId) & ~(0x1F)) >> 5)
 #define NV_BITMASK32_OFFSET(chId)            ((chId) & (0x1F))
 #define NV_BITMASK32_SET(pChannelMask, chId) \
         (pChannelMask)[NV_BITMASK32_IDX(chId)] |= NVBIT(NV_BITMASK32_OFFSET(chId))
@@ -494,6 +477,23 @@ do                                                      \
 //
 #define NV_TWO_N_MINUS_ONE(n) (((1ULL<<(n/2))<<((n+1)/2))-1)
 
+//
+// Create a 64b bitmask with n bits set
+// This is the same as ((1ULL<<n) - 1), but it doesn't overflow for n=64
+//
+// ...
+// n=-1, 0x0000000000000000
+// n=0,  0x0000000000000000
+// n=1,  0x0000000000000001
+// ...
+// n=63, 0x7FFFFFFFFFFFFFFF
+// n=64, 0xFFFFFFFFFFFFFFFF
+// n=65, 0xFFFFFFFFFFFFFFFF
+// n=66, 0xFFFFFFFFFFFFFFFF
+// ...
+//
+#define NV_BITMASK64(n) ((n<1) ? 0ULL : (NV_U64_MAX>>((n>64) ? 0 : (64-n))))
+
 #define DRF_READ_1WORD_BS(d,r,f,v) \
     ((DRF_EXTENT_MW(NV##d##r##f)<8)?DRF_READ_1BYTE_BS(NV##d##r##f,(v)): \
     ((DRF_EXTENT_MW(NV##d##r##f)<16)?DRF_READ_2BYTE_BS(NV##d##r##f,(v)): \
@@ -574,6 +574,12 @@ nvMaskPos32(const NvU32 mask, const NvU32 bitIdx)
     n32 = BIT_IDX_32(LOWESTBIT(n32));\
 }
 
+// Destructive operation on n64
+#define LOWESTBITIDX_64(n64)         \
+{                                    \
+    n64 = BIT_IDX_64(LOWESTBIT(n64));\
+}
+
 // Destructive operation on n32
 #define HIGHESTBITIDX_32(n32)   \
 {                               \
@@ -585,6 +591,17 @@ nvMaskPos32(const NvU32 mask, const NvU32 bitIdx)
     n32 = count;                \
 }
 
+// Destructive operation on n64
+#define HIGHESTBITIDX_64(n64)   \
+{                               \
+    NvU64 count = 0;            \
+    while (n64 >>= 1)           \
+    {                           \
+        count++;                \
+    }                           \
+    n64 = count;                \
+}
+
 // Destructive operation on n32
 #define ROUNDUP_POW2(n32) \
 {                         \
@@ -694,6 +711,35 @@ nvPrevPow2_U64(const NvU64 x )
     }                                                       \
 }
 
+/*!
+ * Returns the position of nth set bit in the given mask.
+ *
+ * Returns -1 if mask has fewer than n bits set.
+ *
+ * n is 0 indexed and has valid values 0..31 inclusive, so "zeroth" set bit is
+ * the first set LSB.
+ *
+ * Example, if mask = 0x000000F0u and n = 1, the return value will be 5.
+ * Example, if mask = 0x000000F0u and n = 4, the return value will be -1.
+ */
+static NV_FORCEINLINE NvS32
+nvGetNthSetBitIndex32(NvU32 mask, NvU32 n)
+{
+    NvU32 seenSetBitsCount = 0;
+    NvS32 index;
+    FOR_EACH_INDEX_IN_MASK(32, index, mask)
+    {
+        if (seenSetBitsCount == n)
+        {
+            return index;
+        }
+        ++seenSetBitsCount;
+    }
+    FOR_EACH_INDEX_IN_MASK_END;
+
+    return -1;
+}
+
 //
 // Size to use when declaring variable-sized arrays
 //
@@ -737,12 +783,15 @@ nvPrevPow2_U64(const NvU64 x )
 // Returns the offset (in bytes) of 'member' in struct 'type'.
 #ifndef NV_OFFSETOF
     #if defined(__GNUC__) && (__GNUC__ > 3)
-        #define NV_OFFSETOF(type, member)   ((NvU32)__builtin_offsetof(type, member))
+        #define NV_OFFSETOF(type, member)   ((NvUPtr) __builtin_offsetof(type, member))
     #else
-        #define NV_OFFSETOF(type, member)    ((NvU32)(NvU64)&(((type *)0)->member)) // shouldn't we use PtrToUlong? But will need to include windows header.
+        #define NV_OFFSETOF(type, member)    ((NvUPtr) &(((type *)0)->member))
     #endif
 #endif
 
+// Given a pointer and the member it is of the parent struct, return a pointer to the parent struct
+#define NV_CONTAINEROF(ptr, type, member) ((type *) (((NvUPtr) ptr) - NV_OFFSETOF(type, member)))
+
 //
 // Performs a rounded division of b into a (unsigned). For SIGNED version of
 // NV_ROUNDED_DIV() macro check the comments in bug 769777.
@@ -918,10 +967,31 @@ static NV_FORCEINLINE void *NV_NVUPTR_TO_PTR(NvUPtr address)
 // Use (lo) if (b) is less than 64, and (hi) if >= 64.
 //
 #define NV_BIT_SET_128(b, lo, hi)              { nvAssert( (b) < 128 ); if ( (b) < 64 ) (lo) |= NVBIT64(b); else (hi) |= NVBIT64( b & 0x3F ); }
+//
+// Clear the bit at pos (b) for U64 which is < 128.
+// Use (lo) if (b) is less than 64, and (hi) if >= 64.
+//
+#define NV_BIT_CLEAR_128(b, lo, hi)            { nvAssert( (b) < 128 ); if ( (b) < 64 ) (lo) &= ~NVBIT64(b); else (hi) &= ~NVBIT64( b & 0x3F ); }
 
 // Get the number of elements the specified fixed-size array
 #define NV_ARRAY_ELEMENTS(x)                   ((sizeof(x)/sizeof((x)[0])))
 
+#if !defined(NVIDIA_UNDEF_LEGACY_BIT_MACROS)
+//
+// Deprecated macros whose definition can be removed once the code base no longer references them.
+// Use the NVBIT* macros instead of these macros.
+//
+#ifndef BIT
+#define BIT(b)   (1U<<(b))
+#endif
+#ifndef BIT32
+#define BIT32(b) ((NvU32)1U<<(b))
+#endif
+#ifndef BIT64
+#define BIT64(b) ((NvU64)1U<<(b))
+#endif
+#endif
+
 #ifdef __cplusplus
 }
 #endif //__cplusplus

kernel-open/common/inc/nvstatuscodes.h

@@ -1,5 +1,5 @@
 /*
- * SPDX-FileCopyrightText: Copyright (c) 2014-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+ * SPDX-FileCopyrightText: Copyright (c) 2014-2025 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
  * SPDX-License-Identifier: MIT
  *
  * Permission is hereby granted, free of charge, to any person obtaining a
@@ -151,6 +151,20 @@ NV_STATUS_CODE(NV_ERR_RISCV_ERROR,                     0x00000079, "Generic RISC
 NV_STATUS_CODE(NV_ERR_FABRIC_MANAGER_NOT_PRESENT,      0x0000007A, "Fabric Manager is not loaded")
 NV_STATUS_CODE(NV_ERR_ALREADY_SIGNALLED,               0x0000007B, "Semaphore Surface value already >= requested wait value")
 NV_STATUS_CODE(NV_ERR_QUEUE_TASK_SLOT_NOT_AVAILABLE,   0x0000007C, "PMU RPC error due to no queue slot available for this event")
+NV_STATUS_CODE(NV_ERR_KEY_ROTATION_IN_PROGRESS,        0x0000007D, "Operation not allowed as key rotation is in progress")
+NV_STATUS_CODE(NV_ERR_TEST_ONLY_CODE_NOT_ENABLED,      0x0000007E, "Test-only code path not enabled")
+NV_STATUS_CODE(NV_ERR_SECURE_BOOT_FAILED,              0x0000007F, "GFW secure boot failed")
+NV_STATUS_CODE(NV_ERR_INSUFFICIENT_ZBC_ENTRY,          0x00000080, "No more ZBC entry for the client")
+NV_STATUS_CODE(NV_ERR_NVLINK_FABRIC_NOT_READY,         0x00000081, "Nvlink Fabric Status or Fabric Probe is not yet complete, caller needs to retry")
+NV_STATUS_CODE(NV_ERR_NVLINK_FABRIC_FAILURE,           0x00000082, "Nvlink Fabric Probe failed")
+NV_STATUS_CODE(NV_ERR_GPU_MEMORY_ONLINING_FAILURE,     0x00000083, "GPU Memory Onlining failed")
+NV_STATUS_CODE(NV_ERR_REDUCTION_MANAGER_NOT_AVAILABLE, 0x00000084, "Reduction Manager is not available")
+NV_STATUS_CODE(NV_ERR_THRESHOLD_CROSSED,               0x00000085, "A fatal threshold has been crossed")
+NV_STATUS_CODE(NV_ERR_RESOURCE_RETIREMENT_ERROR,       0x00000086, "An error occurred while trying to retire a resource")
+NV_STATUS_CODE(NV_ERR_FABRIC_STATE_OUT_OF_SYNC,        0x00000087, "NVLink fabric state cached by the driver is out of sync")
+NV_STATUS_CODE(NV_ERR_BUFFER_FULL,                     0x00000088, "Buffer is full")
+NV_STATUS_CODE(NV_ERR_BUFFER_EMPTY,                    0x00000089, "Buffer is empty")
+NV_STATUS_CODE(NV_ERR_MC_FLA_OFFSET_TABLE_FULL,        0x0000008A, "Multicast FLA offset table has no available slots")
 
 // Warnings:
 NV_STATUS_CODE(NV_WARN_HOT_SWITCH,                     0x00010001, "WARNING Hot switch")
@@ -161,5 +175,6 @@ NV_STATUS_CODE(NV_WARN_MORE_PROCESSING_REQUIRED,       0x00010005, "WARNING More
 NV_STATUS_CODE(NV_WARN_NOTHING_TO_DO,                  0x00010006, "WARNING Nothing to do")
 NV_STATUS_CODE(NV_WARN_NULL_OBJECT,                    0x00010007, "WARNING NULL object found")
 NV_STATUS_CODE(NV_WARN_OUT_OF_RANGE,                   0x00010008, "WARNING value out of range")
+NV_STATUS_CODE(NV_WARN_THRESHOLD_CROSSED,              0x00010009, "WARNING Threshold has been crossed")
 
 #endif /* SDK_NVSTATUSCODES_H */

kernel-open/common/inc/nvtypes.h

@@ -1,5 +1,5 @@
 /*
- * SPDX-FileCopyrightText: Copyright (c) 1993-2020 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+ * SPDX-FileCopyrightText: Copyright (c) 1993-2025 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
  * SPDX-License-Identifier: MIT
  *
  * Permission is hereby granted, free of charge, to any person obtaining a
@@ -24,10 +24,6 @@
 #ifndef NVTYPES_INCLUDED
 #define NVTYPES_INCLUDED
 
-#ifdef __cplusplus
-extern "C" {
-#endif
-
 #include "cpuopsys.h"
 
 #ifndef NVTYPES_USE_STDINT
@@ -55,6 +51,10 @@ extern "C" {
 #endif
 #endif // __cplusplus
 
+#ifdef __cplusplus
+extern "C" {
+#endif
+
 #if defined(MAKE_NV64TYPES_8BYTES_ALIGNED) && defined(__i386__)
 // ensure or force 8-bytes alignment of NV 64-bit types
 #define OPTIONAL_ALIGN8_ATTR __attribute__((aligned(8)))
@@ -152,6 +152,12 @@ typedef   signed short     NvS16; /* -32768 to 32767                         */
      (((NvU32)(c) & 0xff) << 8)  | \
      (((NvU32)(d) & 0xff))))
 
+// Macro to build an NvU64 from two DWORDS, listed from msb to lsb
+#define NvU64_BUILD(a, b) \
+    ((NvU64)( \
+     (((NvU64)(a) & ~0U) << 32) | \
+     (((NvU64)(b) & ~0U))))
+
 #if NVTYPES_USE_STDINT
 typedef uint32_t           NvV32; /* "void": enumerated or multiple fields   */
 typedef uint32_t           NvU32; /* 0 to 4294967295                         */

kernel-open/common/inc/os-interface.h

@@ -1,5 +1,5 @@
 /*
- * SPDX-FileCopyrightText: Copyright (c) 1999-2024 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+ * SPDX-FileCopyrightText: Copyright (c) 1999-2025 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
  * SPDX-License-Identifier: MIT
  *
  * Permission is hereby granted, free of charge, to any person obtaining a
@@ -40,8 +40,11 @@
 #include "nv_stdarg.h"
 #include <nv-kernel-interface-api.h>
 #include <os/nv_memory_type.h>
+#include <os/nv_memory_area.h>
 #include <nv-caps.h>
 
+#include "rs_access.h"
+
 
 
 typedef struct
@@ -67,145 +70,152 @@ typedef struct os_wait_queue os_wait_queue;
  * ---------------------------------------------------------------------------
  */
 
-NV_STATUS   NV_API_CALL  os_alloc_mem                (void **, NvU64);
-void        NV_API_CALL  os_free_mem                 (void *);
-NV_STATUS   NV_API_CALL  os_get_current_time         (NvU32 *, NvU32 *);
-NvU64       NV_API_CALL  os_get_current_tick         (void);
-NvU64       NV_API_CALL  os_get_current_tick_hr      (void);
-NvU64       NV_API_CALL  os_get_tick_resolution      (void);
-NV_STATUS   NV_API_CALL  os_delay                    (NvU32);
-NV_STATUS   NV_API_CALL  os_delay_us                 (NvU32);
-NvU64       NV_API_CALL  os_get_cpu_frequency        (void);
-NvU32       NV_API_CALL  os_get_current_process      (void);
-void        NV_API_CALL  os_get_current_process_name (char *, NvU32);
-NV_STATUS   NV_API_CALL  os_get_current_thread       (NvU64 *);
-char*       NV_API_CALL  os_string_copy              (char *, const char *);
-NvU32       NV_API_CALL  os_string_length            (const char *);
-NvU32       NV_API_CALL  os_strtoul                  (const char *, char **, NvU32);
-NvS32       NV_API_CALL  os_string_compare           (const char *, const char *);
-NvS32       NV_API_CALL  os_snprintf                 (char *, NvU32, const char *, ...);
-NvS32       NV_API_CALL  os_vsnprintf                (char *, NvU32, const char *, va_list);
-void        NV_API_CALL  os_log_error                (const char *, va_list);
-void*       NV_API_CALL  os_mem_copy                 (void *, const void *, NvU32);
-NV_STATUS   NV_API_CALL  os_memcpy_from_user         (void *, const void *, NvU32);
-NV_STATUS   NV_API_CALL  os_memcpy_to_user           (void *, const void *, NvU32);
-void*       NV_API_CALL  os_mem_set                  (void *, NvU8, NvU32);
-NvS32       NV_API_CALL  os_mem_cmp                  (const NvU8 *, const NvU8 *, NvU32);
-void*       NV_API_CALL  os_pci_init_handle          (NvU32, NvU8, NvU8, NvU8, NvU16 *, NvU16 *);
-NV_STATUS   NV_API_CALL  os_pci_read_byte            (void *, NvU32, NvU8 *);
-NV_STATUS   NV_API_CALL  os_pci_read_word            (void *, NvU32, NvU16 *);
-NV_STATUS   NV_API_CALL  os_pci_read_dword           (void *, NvU32, NvU32 *);
-NV_STATUS   NV_API_CALL  os_pci_write_byte           (void *, NvU32, NvU8);
-NV_STATUS   NV_API_CALL  os_pci_write_word           (void *, NvU32, NvU16);
-NV_STATUS   NV_API_CALL  os_pci_write_dword          (void *, NvU32, NvU32);
-NvBool      NV_API_CALL  os_pci_remove_supported     (void);
-void        NV_API_CALL  os_pci_remove               (void *);
-void*       NV_API_CALL  os_map_kernel_space         (NvU64, NvU64, NvU32);
-void        NV_API_CALL  os_unmap_kernel_space       (void *, NvU64);
-void*       NV_API_CALL  os_map_user_space           (NvU64, NvU64, NvU32, NvU32, void **);
-void        NV_API_CALL  os_unmap_user_space         (void *, NvU64, void *);
-NV_STATUS   NV_API_CALL  os_flush_cpu_cache_all      (void);
-NV_STATUS   NV_API_CALL  os_flush_user_cache         (void);
+NV_STATUS   NV_API_CALL  os_alloc_mem                     (void **, NvU64);
+void        NV_API_CALL  os_free_mem                      (void *);
+NV_STATUS   NV_API_CALL  os_get_system_time               (NvU32 *, NvU32 *);
+NvU64       NV_API_CALL  os_get_monotonic_time_ns         (void);
+NvU64       NV_API_CALL  os_get_monotonic_time_ns_hr      (void);
+NvU64       NV_API_CALL  os_get_monotonic_tick_resolution_ns (void);
+NV_STATUS   NV_API_CALL  os_delay                         (NvU32);
+NV_STATUS   NV_API_CALL  os_delay_us                      (NvU32);
+NvU64       NV_API_CALL  os_get_cpu_frequency             (void);
+NvU32       NV_API_CALL  os_get_current_process           (void);
+void        NV_API_CALL  os_get_current_process_name      (char *, NvU32);
+NV_STATUS   NV_API_CALL  os_get_current_thread            (NvU64 *);
+char*       NV_API_CALL  os_string_copy                   (char *, const char *);
+NvU32       NV_API_CALL  os_string_length                 (const char *);
+NvU32       NV_API_CALL  os_strtoul                       (const char *, char **, NvU32);
+NvS32       NV_API_CALL  os_string_compare                (const char *, const char *);
+NvS32       NV_API_CALL  os_snprintf                      (char *, NvU32, const char *, ...);
+NvS32       NV_API_CALL  os_vsnprintf                     (char *, NvU32, const char *, va_list);
+void        NV_API_CALL  os_log_error                     (const char *, va_list);
+void*       NV_API_CALL  os_mem_copy                      (void *, const void *, NvU32);
+NV_STATUS   NV_API_CALL  os_memcpy_from_user              (void *, const void *, NvU32);
+NV_STATUS   NV_API_CALL  os_memcpy_to_user                (void *, const void *, NvU32);
+void*       NV_API_CALL  os_mem_set                       (void *, NvU8, NvU32);
+NvS32       NV_API_CALL  os_mem_cmp                       (const NvU8 *, const NvU8 *, NvU32);
+void*       NV_API_CALL  os_pci_init_handle               (NvU32, NvU8, NvU8, NvU8, NvU16 *, NvU16 *);
+NV_STATUS   NV_API_CALL  os_pci_read_byte                 (void *, NvU32, NvU8 *);
+NV_STATUS   NV_API_CALL  os_pci_read_word                 (void *, NvU32, NvU16 *);
+NV_STATUS   NV_API_CALL  os_pci_read_dword                (void *, NvU32, NvU32 *);
+NV_STATUS   NV_API_CALL  os_pci_write_byte                (void *, NvU32, NvU8);
+NV_STATUS   NV_API_CALL  os_pci_write_word                (void *, NvU32, NvU16);
+NV_STATUS   NV_API_CALL  os_pci_write_dword               (void *, NvU32, NvU32);
+NvBool      NV_API_CALL  os_pci_remove_supported          (void);
+void        NV_API_CALL  os_pci_remove                    (void *);
+void*       NV_API_CALL  os_map_kernel_space              (NvU64, NvU64, NvU32);
+void        NV_API_CALL  os_unmap_kernel_space            (void *, NvU64);
+#if defined(NV_VMWARE)
+void*       NV_API_CALL  os_map_user_space                (MemoryArea *, NvU32, NvU32, void **);
+void        NV_API_CALL  os_unmap_user_space              (void *, NvU64, void *);
+#endif
+NV_STATUS   NV_API_CALL  os_flush_cpu_cache_all           (void);
+NV_STATUS   NV_API_CALL  os_flush_user_cache              (void);
 void        NV_API_CALL  os_flush_cpu_write_combine_buffer(void);
-NvU8        NV_API_CALL  os_io_read_byte             (NvU32);
-NvU16       NV_API_CALL  os_io_read_word             (NvU32);
-NvU32       NV_API_CALL  os_io_read_dword            (NvU32);
-void        NV_API_CALL  os_io_write_byte            (NvU32, NvU8);
-void        NV_API_CALL  os_io_write_word            (NvU32, NvU16);
-void        NV_API_CALL  os_io_write_dword           (NvU32, NvU32);
-NvBool      NV_API_CALL  os_is_administrator         (void);
-NvBool      NV_API_CALL  os_allow_priority_override  (void);
-void        NV_API_CALL  os_dbg_init                 (void);
-void        NV_API_CALL  os_dbg_breakpoint           (void);
-void        NV_API_CALL  os_dbg_set_level            (NvU32);
-NvU32       NV_API_CALL  os_get_cpu_count            (void);
-NvU32       NV_API_CALL  os_get_cpu_number           (void);
-void        NV_API_CALL  os_disable_console_access   (void);
-void        NV_API_CALL  os_enable_console_access    (void);
-NV_STATUS   NV_API_CALL  os_registry_init            (void);
-NvU64       NV_API_CALL  os_get_max_user_va          (void);
-NV_STATUS   NV_API_CALL  os_schedule                 (void);
-NV_STATUS   NV_API_CALL  os_alloc_spinlock           (void **);
-void        NV_API_CALL  os_free_spinlock            (void *);
-NvU64       NV_API_CALL  os_acquire_spinlock         (void *);
-void        NV_API_CALL  os_release_spinlock         (void *, NvU64);
-NV_STATUS   NV_API_CALL  os_queue_work_item          (struct os_work_queue *, void *);
-NV_STATUS   NV_API_CALL  os_flush_work_queue         (struct os_work_queue *);
-NV_STATUS   NV_API_CALL  os_alloc_mutex              (void **);
-void        NV_API_CALL  os_free_mutex               (void *);
-NV_STATUS   NV_API_CALL  os_acquire_mutex            (void *);
-NV_STATUS   NV_API_CALL  os_cond_acquire_mutex       (void *);
-void        NV_API_CALL  os_release_mutex            (void *);
-void*       NV_API_CALL  os_alloc_semaphore          (NvU32);
-void        NV_API_CALL  os_free_semaphore           (void *);
-NV_STATUS   NV_API_CALL  os_acquire_semaphore        (void *);
-NV_STATUS   NV_API_CALL  os_cond_acquire_semaphore   (void *);
-NV_STATUS   NV_API_CALL  os_release_semaphore        (void *);
-void*       NV_API_CALL  os_alloc_rwlock             (void);
-void        NV_API_CALL  os_free_rwlock              (void *);
-NV_STATUS   NV_API_CALL  os_acquire_rwlock_read      (void *);
-NV_STATUS   NV_API_CALL  os_acquire_rwlock_write     (void *);
-NV_STATUS   NV_API_CALL  os_cond_acquire_rwlock_read (void *);
-NV_STATUS   NV_API_CALL  os_cond_acquire_rwlock_write(void *);
-void        NV_API_CALL  os_release_rwlock_read      (void *);
-void        NV_API_CALL  os_release_rwlock_write     (void *);
-NvBool      NV_API_CALL  os_semaphore_may_sleep      (void);
-NV_STATUS   NV_API_CALL  os_get_version_info         (os_version_info*);
-NvBool      NV_API_CALL  os_is_isr                   (void);
-NvBool      NV_API_CALL  os_pat_supported            (void);
-void        NV_API_CALL  os_dump_stack               (void);
-NvBool      NV_API_CALL  os_is_efi_enabled           (void);
-NvBool      NV_API_CALL  os_is_xen_dom0              (void);
-NvBool      NV_API_CALL  os_is_vgx_hyper             (void);
-NV_STATUS   NV_API_CALL  os_inject_vgx_msi           (NvU16, NvU64, NvU32);
-NvBool      NV_API_CALL  os_is_grid_supported        (void);
-NvU32       NV_API_CALL  os_get_grid_csp_support     (void);
-void        NV_API_CALL  os_bug_check                (NvU32, const char *);
-NV_STATUS   NV_API_CALL  os_lock_user_pages          (void *, NvU64, void **, NvU32);
-NV_STATUS   NV_API_CALL  os_lookup_user_io_memory    (void *, NvU64, NvU64 **);
-NV_STATUS   NV_API_CALL  os_unlock_user_pages        (NvU64, void *);
-NV_STATUS   NV_API_CALL  os_match_mmap_offset        (void *, NvU64, NvU64 *);
-NV_STATUS   NV_API_CALL  os_get_euid                 (NvU32 *);
-NV_STATUS   NV_API_CALL  os_get_smbios_header        (NvU64 *pSmbsAddr);
-NV_STATUS   NV_API_CALL  os_get_acpi_rsdp_from_uefi  (NvU32 *);
-void        NV_API_CALL  os_add_record_for_crashLog  (void *, NvU32);
-void        NV_API_CALL  os_delete_record_for_crashLog (void *);
-NV_STATUS   NV_API_CALL  os_call_vgpu_vfio           (void *, NvU32);
-NV_STATUS   NV_API_CALL  os_numa_memblock_size       (NvU64 *);
-NV_STATUS   NV_API_CALL  os_alloc_pages_node         (NvS32, NvU32, NvU32, NvU64 *);
-NV_STATUS   NV_API_CALL  os_get_page                 (NvU64 address);
-NV_STATUS   NV_API_CALL  os_put_page                 (NvU64 address);
-NvU32       NV_API_CALL  os_get_page_refcount        (NvU64 address);
-NvU32       NV_API_CALL  os_count_tail_pages         (NvU64 address);
-void        NV_API_CALL  os_free_pages_phys          (NvU64, NvU32);
-NV_STATUS   NV_API_CALL  os_open_temporary_file      (void **);
-void        NV_API_CALL  os_close_file               (void *);
-NV_STATUS   NV_API_CALL  os_write_file               (void *, NvU8 *, NvU64, NvU64);
-NV_STATUS   NV_API_CALL  os_read_file                (void *, NvU8 *, NvU64, NvU64);
-NV_STATUS   NV_API_CALL  os_open_readonly_file       (const char *, void **);
-NV_STATUS   NV_API_CALL  os_open_and_read_file       (const char *, NvU8 *, NvU64);
-NvBool      NV_API_CALL  os_is_nvswitch_present      (void);
-NV_STATUS   NV_API_CALL  os_get_random_bytes         (NvU8 *, NvU16);
-NV_STATUS   NV_API_CALL  os_alloc_wait_queue         (os_wait_queue **);
-void        NV_API_CALL  os_free_wait_queue          (os_wait_queue *);
-void        NV_API_CALL  os_wait_uninterruptible     (os_wait_queue *);
-void        NV_API_CALL  os_wait_interruptible       (os_wait_queue *);
-void        NV_API_CALL  os_wake_up                  (os_wait_queue *);
-nv_cap_t*   NV_API_CALL  os_nv_cap_init               (const char *);
-nv_cap_t*   NV_API_CALL  os_nv_cap_create_dir_entry   (nv_cap_t *, const char *, int);
-nv_cap_t*   NV_API_CALL  os_nv_cap_create_file_entry  (nv_cap_t *, const char *, int);
-void        NV_API_CALL  os_nv_cap_destroy_entry      (nv_cap_t *);
-int         NV_API_CALL  os_nv_cap_validate_and_dup_fd(const nv_cap_t *, int);
-void        NV_API_CALL  os_nv_cap_close_fd           (int);
-NvS32       NV_API_CALL  os_imex_channel_get          (NvU64);
-NvS32       NV_API_CALL  os_imex_channel_count        (void);
+NvU8        NV_API_CALL  os_io_read_byte                  (NvU32);
+NvU16       NV_API_CALL  os_io_read_word                  (NvU32);
+NvU32       NV_API_CALL  os_io_read_dword                 (NvU32);
+void        NV_API_CALL  os_io_write_byte                 (NvU32, NvU8);
+void        NV_API_CALL  os_io_write_word                 (NvU32, NvU16);
+void        NV_API_CALL  os_io_write_dword                (NvU32, NvU32);
+NvBool      NV_API_CALL  os_is_administrator              (void);
+NvBool      NV_API_CALL  os_check_access                  (RsAccessRight accessRight);
+void        NV_API_CALL  os_dbg_init                      (void);
+void        NV_API_CALL  os_dbg_breakpoint                (void);
+void        NV_API_CALL  os_dbg_set_level                 (NvU32);
+NvU32       NV_API_CALL  os_get_cpu_count                 (void);
+NvU32       NV_API_CALL  os_get_cpu_number                (void);
+void        NV_API_CALL  os_disable_console_access        (void);
+void        NV_API_CALL  os_enable_console_access         (void);
+NV_STATUS   NV_API_CALL  os_registry_init                 (void);
+NvU64       NV_API_CALL  os_get_max_user_va               (void);
+NV_STATUS   NV_API_CALL  os_schedule                      (void);
+NV_STATUS   NV_API_CALL  os_alloc_spinlock                (void **);
+void        NV_API_CALL  os_free_spinlock                 (void *);
+NvU64       NV_API_CALL  os_acquire_spinlock              (void *);
+void        NV_API_CALL  os_release_spinlock              (void *, NvU64);
+NV_STATUS   NV_API_CALL  os_queue_work_item               (struct os_work_queue *, void *);
+NV_STATUS   NV_API_CALL  os_flush_work_queue              (struct os_work_queue *, NvBool);
+NvBool      NV_API_CALL  os_is_queue_flush_ongoing        (struct os_work_queue *);
+NV_STATUS   NV_API_CALL  os_alloc_mutex                   (void **);
+void        NV_API_CALL  os_free_mutex                    (void *);
+NV_STATUS   NV_API_CALL  os_acquire_mutex                 (void *);
+NV_STATUS   NV_API_CALL  os_cond_acquire_mutex            (void *);
+void        NV_API_CALL  os_release_mutex                 (void *);
+void*       NV_API_CALL  os_alloc_semaphore               (NvU32);
+void        NV_API_CALL  os_free_semaphore                (void *);
+NV_STATUS   NV_API_CALL  os_acquire_semaphore             (void *);
+NV_STATUS   NV_API_CALL  os_cond_acquire_semaphore        (void *);
+NV_STATUS   NV_API_CALL  os_release_semaphore             (void *);
+void*       NV_API_CALL  os_alloc_rwlock                  (void);
+void        NV_API_CALL  os_free_rwlock                   (void *);
+NV_STATUS   NV_API_CALL  os_acquire_rwlock_read           (void *);
+NV_STATUS   NV_API_CALL  os_acquire_rwlock_write          (void *);
+NV_STATUS   NV_API_CALL  os_cond_acquire_rwlock_read      (void *);
+NV_STATUS   NV_API_CALL  os_cond_acquire_rwlock_write     (void *);
+void        NV_API_CALL  os_release_rwlock_read           (void *);
+void        NV_API_CALL  os_release_rwlock_write          (void *);
+NvBool      NV_API_CALL  os_semaphore_may_sleep           (void);
+NV_STATUS   NV_API_CALL  os_get_version_info              (os_version_info*);
+NV_STATUS   NV_API_CALL  os_get_is_openrm                 (NvBool *);
+NvBool      NV_API_CALL  os_is_isr                        (void);
+NvBool      NV_API_CALL  os_pat_supported                 (void);
+void        NV_API_CALL  os_dump_stack                    (void);
+NvBool      NV_API_CALL  os_is_efi_enabled                (void);
+NvBool      NV_API_CALL  os_is_xen_dom0                   (void);
+NvBool      NV_API_CALL  os_is_vgx_hyper                  (void);
+NV_STATUS   NV_API_CALL  os_inject_vgx_msi                (NvU16, NvU64, NvU32);
+NvBool      NV_API_CALL  os_is_grid_supported             (void);
+NvU32       NV_API_CALL  os_get_grid_csp_support          (void);
+void        NV_API_CALL  os_bug_check                     (NvU32, const char *);
+NV_STATUS   NV_API_CALL  os_lock_user_pages               (void *, NvU64, void **, NvU32);
+NV_STATUS   NV_API_CALL  os_lookup_user_io_memory         (void *, NvU64, NvU64 **);
+NV_STATUS   NV_API_CALL  os_unlock_user_pages             (NvU64, void *, NvU32);
+NV_STATUS   NV_API_CALL  os_match_mmap_offset             (void *, NvU64, NvU64 *);
+NV_STATUS   NV_API_CALL  os_get_euid                      (NvU32 *);
+NV_STATUS   NV_API_CALL  os_get_smbios_header             (NvU64 *pSmbsAddr);
+NV_STATUS   NV_API_CALL  os_get_acpi_rsdp_from_uefi       (NvU32 *);
+void        NV_API_CALL  os_add_record_for_crashLog       (void *, NvU32);
+void        NV_API_CALL  os_delete_record_for_crashLog    (void *);
+NV_STATUS   NV_API_CALL  os_call_vgpu_vfio                (void *, NvU32);
+NV_STATUS   NV_API_CALL  os_device_vm_present             (void);
+NV_STATUS   NV_API_CALL  os_numa_memblock_size            (NvU64 *);
+NV_STATUS   NV_API_CALL  os_alloc_pages_node              (NvS32, NvU32, NvU32, NvU64 *);
+NV_STATUS   NV_API_CALL  os_get_page                      (NvU64 address);
+NV_STATUS   NV_API_CALL  os_put_page                      (NvU64 address);
+NvU32       NV_API_CALL  os_get_page_refcount             (NvU64 address);
+NvU32       NV_API_CALL  os_count_tail_pages              (NvU64 address);
+void        NV_API_CALL  os_free_pages_phys               (NvU64, NvU32);
+NV_STATUS   NV_API_CALL  os_open_temporary_file           (void **);
+void        NV_API_CALL  os_close_file                    (void *);
+NV_STATUS   NV_API_CALL  os_write_file                    (void *, NvU8 *, NvU64, NvU64);
+NV_STATUS   NV_API_CALL  os_read_file                     (void *, NvU8 *, NvU64, NvU64);
+NV_STATUS   NV_API_CALL  os_open_readonly_file            (const char *, void **);
+NV_STATUS   NV_API_CALL  os_open_and_read_file            (const char *, NvU8 *, NvU64);
+NvBool      NV_API_CALL  os_is_nvswitch_present           (void);
+NV_STATUS   NV_API_CALL  os_get_random_bytes              (NvU8 *, NvU16);
+NV_STATUS   NV_API_CALL  os_alloc_wait_queue              (os_wait_queue **);
+void        NV_API_CALL  os_free_wait_queue               (os_wait_queue *);
+void        NV_API_CALL  os_wait_uninterruptible          (os_wait_queue *);
+void        NV_API_CALL  os_wait_interruptible            (os_wait_queue *);
+void        NV_API_CALL  os_wake_up                       (os_wait_queue *);
+nv_cap_t*   NV_API_CALL  os_nv_cap_init                   (const char *);
+nv_cap_t*   NV_API_CALL  os_nv_cap_create_dir_entry       (nv_cap_t *, const char *, int);
+nv_cap_t*   NV_API_CALL  os_nv_cap_create_file_entry      (nv_cap_t *, const char *, int);
+void        NV_API_CALL  os_nv_cap_destroy_entry          (nv_cap_t *);
+int         NV_API_CALL  os_nv_cap_validate_and_dup_fd    (const nv_cap_t *, int);
+void        NV_API_CALL  os_nv_cap_close_fd               (int);
+NvS32       NV_API_CALL  os_imex_channel_get              (NvU64);
+NvS32       NV_API_CALL  os_imex_channel_count            (void);
 
+NV_STATUS   NV_API_CALL  os_get_tegra_platform       (NvU32 *);
 enum os_pci_req_atomics_type {
     OS_INTF_PCIE_REQ_ATOMICS_32BIT,
     OS_INTF_PCIE_REQ_ATOMICS_64BIT,
     OS_INTF_PCIE_REQ_ATOMICS_128BIT
 };
 NV_STATUS   NV_API_CALL  os_enable_pci_req_atomics   (void *, enum os_pci_req_atomics_type);
+void        NV_API_CALL  os_pci_trigger_flr(void *handle);
 NV_STATUS   NV_API_CALL  os_get_numa_node_memory_usage (NvS32, NvU64 *, NvU64 *);
 NV_STATUS   NV_API_CALL  os_numa_add_gpu_memory      (void *, NvU64, NvU64, NvU32 *);
 NV_STATUS   NV_API_CALL  os_numa_remove_gpu_memory   (void *, NvU64, NvU64, NvU32); 
@@ -213,11 +223,18 @@ NV_STATUS   NV_API_CALL  os_offline_page_at_address(NvU64 address);
 void*       NV_API_CALL  os_get_pid_info(void);
 void        NV_API_CALL  os_put_pid_info(void *pid_info);
 NV_STATUS   NV_API_CALL  os_find_ns_pid(void *pid_info, NvU32 *ns_pid);
+NvBool      NV_API_CALL  os_is_init_ns(void);
+NV_STATUS   NV_API_CALL  os_iommu_sva_bind(void *arg, void **handle, NvU32 *pasid);
+void        NV_API_CALL  os_iommu_sva_unbind(void *handle);
 
-extern NvU32 os_page_size;
+extern NvU64 os_page_size;
+extern NvU64 os_max_page_size;
 extern NvU64 os_page_mask;
 extern NvU8  os_page_shift;
 extern NvBool os_cc_enabled;
+extern NvBool os_cc_sev_snp_enabled;
+extern NvBool os_cc_sme_enabled;
+extern NvBool os_cc_snp_vtom_enabled;
 extern NvBool os_cc_tdx_enabled;
 extern NvBool os_dma_buf_enabled;
 extern NvBool os_imex_channel_is_supported;
@@ -253,4 +270,9 @@ int  NV_API_CALL  nv_printf(NvU32 debuglevel, const char *printf_format, ...);
 #define NV_LOCK_USER_PAGES_FLAGS_WRITE_NO                  0x00000000
 #define NV_LOCK_USER_PAGES_FLAGS_WRITE_YES                 0x00000001
 
+// NV OS Tegra platform type defines
+#define NV_OS_TEGRA_PLATFORM_SIM        0
+#define NV_OS_TEGRA_PLATFORM_FPGA       1
+#define NV_OS_TEGRA_PLATFORM_SILICON    2
+
 #endif /* OS_INTERFACE_H */

kernel-open/common/inc/os/nv_memory_area.h

@@ -0,0 +1,104 @@
+/*
+ * SPDX-FileCopyrightText: Copyright (c) 2024 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+ * SPDX-License-Identifier: MIT
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
+ * DEALINGS IN THE SOFTWARE.
+ */
+
+#ifndef NV_MEMORY_AREA_H
+#define NV_MEMORY_AREA_H
+
+typedef struct MemoryRange
+{
+    NvU64 start;
+    NvU64 size;
+} MemoryRange;
+
+typedef struct MemoryArea
+{
+    MemoryRange *pRanges;
+    NvU64 numRanges;
+} MemoryArea;
+
+static inline NvU64 memareaSize(MemoryArea memArea)
+{
+    NvU64 size = 0;
+    NvU64 idx = 0;
+    for (idx = 0; idx < memArea.numRanges; idx++)
+    {
+        size += memArea.pRanges[idx].size;
+    }
+    return size;
+}
+
+static inline MemoryRange
+mrangeMake
+(
+    NvU64 start,
+    NvU64 size
+)
+{
+    MemoryRange range;
+    range.start = start;
+    range.size = size;
+    return range;
+}
+
+static inline NvU64
+mrangeLimit
+(
+    MemoryRange a
+)
+{
+    return a.start + a.size;
+}
+
+static inline NvBool
+mrangeIntersects
+(
+    MemoryRange a,
+    MemoryRange b
+)
+{
+    return ((a.start >= b.start) && (a.start < mrangeLimit(b))) ||
+        ((b.start >= a.start) && (b.start < mrangeLimit(a)));
+}
+
+static inline NvBool
+mrangeContains
+(
+    MemoryRange outer,
+    MemoryRange inner
+)
+{
+    return (inner.start >= outer.start) && (mrangeLimit(inner) <= mrangeLimit(outer));
+}
+
+static inline MemoryRange
+mrangeOffset
+(
+    MemoryRange range,
+    NvU64 amt
+)
+{
+    range.start += amt;
+    return range;
+}
+
+#endif /* NV_MEMORY_AREA_H */

kernel-open/common/inc/os_dsi_panel_props.h

@@ -0,0 +1,387 @@
+/*
+ * SPDX-FileCopyrightText: Copyright (c) 2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+ * SPDX-License-Identifier: MIT
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
+ * DEALINGS IN THE SOFTWARE.
+ */
+#ifndef _OS_DSI_PANEL_PARAMS_H_
+#define _OS_DSI_PANEL_PARAMS_H_
+
+#define DSI_GENERIC_LONG_WRITE                  0x29
+#define DSI_DCS_LONG_WRITE                      0x39
+#define DSI_GENERIC_SHORT_WRITE_1_PARAMS        0x13
+#define DSI_GENERIC_SHORT_WRITE_2_PARAMS        0x23
+#define DSI_DCS_WRITE_0_PARAM                   0x05
+#define DSI_DCS_WRITE_1_PARAM                   0x15
+#define DSI_DCS_READ_PARAM                      0x06
+#define DSI_DCS_COMPRESSION_MODE		        0x07
+#define DSI_DCS_PPS_LONG_WRITE			        0x0A
+
+#define DSI_DCS_SET_ADDR_MODE                   0x36
+#define DSI_DCS_EXIT_SLEEP_MODE                 0x11
+#define DSI_DCS_ENTER_SLEEP_MODE                0x10
+#define DSI_DCS_SET_DISPLAY_ON                  0x29
+#define DSI_DCS_SET_DISPLAY_OFF                 0x28
+#define DSI_DCS_SET_TEARING_EFFECT_OFF          0x34
+#define DSI_DCS_SET_TEARING_EFFECT_ON           0x35
+#define DSI_DCS_NO_OP                           0x0
+#define DSI_NULL_PKT_NO_DATA                    0x9
+#define DSI_BLANKING_PKT_NO_DATA                0x19
+#define DSI_DCS_SET_COMPRESSION_METHOD          0xC0
+
+/* DCS commands for command mode */
+#define DSI_ENTER_PARTIAL_MODE                  0x12
+#define DSI_SET_PIXEL_FORMAT                    0x3A
+#define DSI_AREA_COLOR_MODE                     0x4C
+#define DSI_SET_PARTIAL_AREA                    0x30
+#define DSI_SET_PAGE_ADDRESS                    0x2B
+#define DSI_SET_ADDRESS_MODE                    0x36
+#define DSI_SET_COLUMN_ADDRESS                  0x2A
+#define DSI_WRITE_MEMORY_START                  0x2C
+#define DSI_WRITE_MEMORY_CONTINUE               0x3C
+
+#define PKT_ID0(id)     ((((id) & 0x3f) << 3) | \
+                        (((DSI_ENABLE) & 0x1) << 9))
+#define PKT_LEN0(len)   (((len) & 0x7) << 0)
+#define PKT_ID1(id)     ((((id) & 0x3f) << 13) | \
+                        (((DSI_ENABLE) & 0x1) << 19))
+#define PKT_LEN1(len)   (((len) & 0x7) << 10)
+#define PKT_ID2(id)     ((((id) & 0x3f) << 23) | \
+                        (((DSI_ENABLE) & 0x1) << 29))
+#define PKT_LEN2(len)   (((len) & 0x7) << 20)
+#define PKT_ID3(id)     ((((id) & 0x3f) << 3) | \
+                        (((DSI_ENABLE) & 0x1) << 9))
+#define PKT_LEN3(len)   (((len) & 0x7) << 0)
+#define PKT_ID4(id)     ((((id) & 0x3f) << 13) | \
+                        (((DSI_ENABLE) & 0x1) << 19))
+#define PKT_LEN4(len)   (((len) & 0x7) << 10)
+#define PKT_ID5(id)     ((((id) & 0x3f) << 23) | \
+                        (((DSI_ENABLE) & 0x1) << 29))
+#define PKT_LEN5(len)   (((len) & 0x7) << 20)
+#define PKT_LP          (((DSI_ENABLE) & 0x1) << 30)
+#define NUMOF_PKT_SEQ   12
+
+/* DSI pixel data format, enum values should match with dt-bindings in tegra-panel.h */
+typedef enum
+{
+    DSI_PIXEL_FORMAT_16BIT_P,
+    DSI_PIXEL_FORMAT_18BIT_P,
+    DSI_PIXEL_FORMAT_18BIT_NP,
+    DSI_PIXEL_FORMAT_24BIT_P,
+    DSI_PIXEL_FORMAT_8BIT_DSC,
+    DSI_PIXEL_FORMAT_12BIT_DSC,
+    DSI_PIXEL_FORMAT_16BIT_DSC,
+    DSI_PIXEL_FORMAT_10BIT_DSC,
+    DSI_PIXEL_FORMAT_30BIT_P,
+    DSI_PIXEL_FORMAT_36BIT_P,
+} DSIPIXELFORMAT;
+
+/* DSI virtual channel number */
+typedef enum
+{
+    DSI_VIRTUAL_CHANNEL_0,
+    DSI_VIRTUAL_CHANNEL_1,
+    DSI_VIRTUAL_CHANNEL_2,
+    DSI_VIRTUAL_CHANNEL_3,
+} DSIVIRTUALCHANNEL;
+
+/* DSI transmit method for video data */
+typedef enum
+{
+    DSI_VIDEO_TYPE_VIDEO_MODE,
+    DSI_VIDEO_TYPE_COMMAND_MODE,
+} DSIVIDEODATAMODE;
+
+/* DSI HS clock mode */
+typedef enum
+{
+    DSI_VIDEO_CLOCK_CONTINUOUS,
+    DSI_VIDEO_CLOCK_TX_ONLY,
+} DSICLOCKMODE;
+
+/* DSI burst mode setting in video mode. Each mode is assigned with a
+ * fixed value. The rationale behind this is to avoid change of these
+ * values, since the calculation of dsi clock depends on them. */
+typedef enum
+{
+    DSI_VIDEO_NON_BURST_MODE = 0,
+    DSI_VIDEO_NON_BURST_MODE_WITH_SYNC_END = 1,
+    DSI_VIDEO_BURST_MODE_LOWEST_SPEED = 2,
+    DSI_VIDEO_BURST_MODE_LOW_SPEED = 3,
+    DSI_VIDEO_BURST_MODE_MEDIUM_SPEED = 4,
+    DSI_VIDEO_BURST_MODE_FAST_SPEED = 5,
+    DSI_VIDEO_BURST_MODE_FASTEST_SPEED = 6,
+} DSIVIDEOBURSTMODE;
+
+/* DSI Ganged Mode */
+typedef enum
+{
+    DSI_GANGED_SYMMETRIC_LEFT_RIGHT = 1,
+    DSI_GANGED_SYMMETRIC_EVEN_ODD = 2,
+    DSI_GANGED_SYMMETRIC_LEFT_RIGHT_OVERLAP = 3,
+} DSIGANGEDTYPE;
+
+typedef enum
+{
+    DSI_LINK0,
+    DSI_LINK1,
+} DSILINKNUM;
+
+/* DSI Command Packet type */
+typedef enum
+{
+    DSI_PACKET_CMD,
+    DSI_DELAY_MS,
+    DSI_GPIO_SET,
+    DSI_SEND_FRAME,
+    DSI_PACKET_VIDEO_VBLANK_CMD,
+    DSI_DELAY_US,
+} DSICMDPKTTYPE;
+
+/* DSI Phy type */
+typedef enum
+{
+    DSI_DPHY,
+    DSI_CPHY,
+} DSIPHYTYPE;
+
+enum {
+    DSI_GPIO_LCD_RESET,
+    DSI_GPIO_PANEL_EN,
+    DSI_GPIO_PANEL_EN_1,
+    DSI_GPIO_BL_ENABLE,
+    DSI_GPIO_BL_PWM,
+    DSI_GPIO_AVDD_AVEE_EN,
+    DSI_GPIO_VDD_1V8_LCD_EN,
+    DSI_GPIO_TE,
+    DSI_GPIO_BRIDGE_EN_0,
+    DSI_GPIO_BRIDGE_EN_1,
+    DSI_GPIO_BRIDGE_REFCLK_EN,
+    DSI_N_GPIO_PANEL, /* add new gpio above this entry */
+};
+
+enum
+{
+    DSI_DISABLE,
+    DSI_ENABLE,
+};
+
+
+typedef struct
+{
+    NvU8      cmd_type;
+    NvU8      data_id;
+    union
+    {
+        NvU16 data_len;
+        NvU16 delay_ms;
+        NvU16 delay_us;
+        NvU32 gpio;
+        NvU16 frame_cnt;
+        struct
+        {
+            NvU8 data0;
+            NvU8 data1;
+        } sp;
+    } sp_len_dly;
+    NvU32      *pdata;
+    NvU8      link_id;
+    NvBool    club_cmd;
+} DSI_CMD, *PDSICMD;
+
+typedef struct
+{
+    NvU16             t_hsdexit_ns;
+    NvU16             t_hstrail_ns;
+    NvU16             t_datzero_ns;
+    NvU16             t_hsprepare_ns;
+    NvU16             t_hsprebegin_ns;
+    NvU16             t_hspost_ns;
+
+    NvU16             t_clktrail_ns;
+    NvU16             t_clkpost_ns;
+    NvU16             t_clkzero_ns;
+    NvU16             t_tlpx_ns;
+
+    NvU16             t_clkprepare_ns;
+    NvU16             t_clkpre_ns;
+    NvU16             t_wakeup_ns;
+
+    NvU16             t_taget_ns;
+    NvU16             t_tasure_ns;
+    NvU16             t_tago_ns;
+} DSI_PHY_TIMING_IN_NS;
+
+typedef struct
+{
+    NvU32 hActive;
+    NvU32 vActive;
+    NvU32 hFrontPorch;
+    NvU32 vFrontPorch;
+    NvU32 hBackPorch;
+    NvU32 vBackPorch;
+    NvU32 hSyncWidth;
+    NvU32 vSyncWidth;
+    NvU32 hPulsePolarity;
+    NvU32 vPulsePolarity;
+    NvU32 pixelClkRate;
+} DSITIMINGS, *PDSITIMINGS;
+
+typedef struct
+{
+    NvU8              n_data_lanes;                   /* required */
+    NvU8              pixel_format;                   /* required */
+    NvU8              refresh_rate;                   /* required */
+    NvU8              rated_refresh_rate;
+    NvU8              panel_reset;                    /* required */
+    NvU8              virtual_channel;                /* required */
+    NvU8              dsi_instance;
+    NvU16             dsi_panel_rst_gpio;
+    NvU16             dsi_panel_bl_en_gpio;
+    NvU16             dsi_panel_bl_pwm_gpio;
+    NvU16             even_odd_split_width;
+    NvU8              controller_vs;
+
+    NvBool            panel_has_frame_buffer; /* required*/
+
+    /* Deprecated. Use DSI_SEND_FRAME panel command instead. */
+    NvBool            panel_send_dc_frames;
+
+    DSI_CMD           *dsi_init_cmd;          /* required */
+    NvU16             n_init_cmd;             /* required */
+    NvU32             *dsi_init_cmd_array;
+    NvU32             init_cmd_array_size;
+    NvBool            sendInitCmdsEarly;
+
+    DSI_CMD           *dsi_early_suspend_cmd;
+    NvU16             n_early_suspend_cmd;
+    NvU32             *dsi_early_suspend_cmd_array;
+    NvU32             early_suspend_cmd_array_size;
+
+    DSI_CMD           *dsi_late_resume_cmd;
+    NvU16             n_late_resume_cmd;
+    NvU32             *dsi_late_resume_cmd_array;
+    NvU32             late_resume_cmd_array_size;
+
+    DSI_CMD           *dsi_postvideo_cmd;
+    NvU16             n_postvideo_cmd;
+    NvU32             *dsi_postvideo_cmd_array;
+    NvU32             postvideo_cmd_array_size;
+
+    DSI_CMD           *dsi_suspend_cmd;               /* required */
+    NvU16             n_suspend_cmd;                  /* required */
+    NvU32             *dsi_suspend_cmd_array;
+    NvU32             suspend_cmd_array_size;
+
+    NvU8              video_data_type;                /* required */
+    NvU8              video_clock_mode;
+    NvU8              video_burst_mode;
+    NvU8              ganged_type;
+    NvU16             ganged_overlap;
+    NvBool            ganged_swap_links;
+    NvBool            ganged_write_to_all_links;
+    NvU8              split_link_type;
+
+    NvU8              suspend_aggr;
+
+    NvU16             panel_buffer_size_byte;
+    NvU16             panel_reset_timeout_msec;
+
+    NvBool            hs_cmd_mode_supported;
+    NvBool            hs_cmd_mode_on_blank_supported;
+    NvBool            enable_hs_clock_on_lp_cmd_mode;
+    NvBool            no_pkt_seq_eot; /* 1st generation panel may not
+                                       * support eot. Don't set it for
+                                       * most panels.*/
+    const NvU32       *pktSeq;
+    NvU32             *pktSeq_array;
+    NvU32             pktSeq_array_size;
+    NvBool            skip_dsi_pkt_header;
+    NvBool            power_saving_suspend;
+    NvBool            suspend_stop_stream_late;
+    NvBool            dsi2lvds_bridge_enable;
+    NvBool            dsi2edp_bridge_enable;
+
+    NvU32             max_panel_freq_khz;
+    NvU32             lp_cmd_mode_freq_khz;
+    NvU32             lp_read_cmd_mode_freq_khz;
+    NvU32             hs_clk_in_lp_cmd_mode_freq_khz;
+    NvU32             burst_mode_freq_khz;
+    NvU32             fpga_freq_khz;
+
+    NvU32             te_gpio;
+    NvBool            te_polarity_low;
+    NvBool            dsiEnVRR;
+    NvBool            dsiVrrPanelSupportsTe;
+    NvBool            dsiForceSetTePin;
+
+    int               panel_gpio[DSI_N_GPIO_PANEL];
+    NvBool            panel_gpio_populated;
+
+    NvU32             dpd_dsi_pads;
+
+    DSI_PHY_TIMING_IN_NS phyTimingNs;
+
+    NvU8              *bl_name;
+
+    NvBool            lp00_pre_panel_wakeup;
+    NvBool            ulpm_not_supported;
+    NvBool            use_video_host_fifo_for_cmd;
+    NvBool            dsi_csi_loopback;
+    NvBool            set_max_timeout;
+    NvBool            use_legacy_dphy_core;
+    // Swap P/N pins polarity of all data lanes
+    NvBool            swap_data_lane_polarity;
+    // Swap P/N pins polarity of clock lane
+    NvBool            swap_clock_lane_polarity;
+    // Reverse clock polarity for partition A/B. 1st SOT bit goes on negedge of Clock lane
+    NvBool            reverse_clock_polarity;
+    // DSI Lane Crossbar. Allocating xbar array for max number of lanes
+    NvBool            lane_xbar_exists;
+    NvU32             lane_xbar_ctrl[8];
+    NvU32             refresh_rate_adj;
+
+    NvU8              dsiPhyType;
+    NvBool            en_data_scrambling;
+
+    NvU32             dsipll_vco_rate_hz;
+    NvU32             dsipll_clkoutpn_rate_hz;
+    NvU32             dsipll_clkouta_rate_hz;
+    NvU32             vpll0_rate_hz;
+
+    DSITIMINGS        dsiTimings;
+
+    // DSC Parameters
+    NvBool            dsiDscEnable;
+    NvU32             dsiDscBpp;
+    NvU32             dsiDscNumSlices;
+    NvU32             dsiDscSliceWidth;
+    NvU32             dsiDscSliceHeight;
+    NvBool            dsiDscEnBlockPrediction;
+    NvBool            dsiDscEnDualDsc;
+    NvU32             dsiDscDecoderMajorVersion;
+    NvU32             dsiDscDecoderMinorVersion;
+    NvBool            dsiDscUseCustomPPS;
+    NvU32             dsiDscCustomPPSData[32];
+
+    // Driver allocates memory for PPS cmd to be sent to Panel
+    NvBool            ppsCmdMemAllocated;
+} DSI_PANEL_INFO;
+
+#endif

kernel-open/common/inc/os_gpio.h

@@ -0,0 +1,34 @@
+/*
+ * SPDX-FileCopyrightText: Copyright (c) 2020 - 2024 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+ * SPDX-License-Identifier: MIT
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
+ * DEALINGS IN THE SOFTWARE.
+ */
+#ifndef _OS_GPIO_H_
+#define _OS_GPIO_H_
+
+typedef enum
+{
+    NV_OS_GPIO_FUNC_HOTPLUG_A,
+    NV_OS_GPIO_FUNC_HOTPLUG_B,
+    NV_OS_GPIO_FUNC_HOTPLUG_C,
+    NV_OS_GPIO_FUNC_HOTPLUG_D,
+} NV_OS_GPIO_FUNC_NAMES;
+
+#endif

kernel-open/common/inc/rm-gpu-ops.h

@@ -1,5 +1,5 @@
 /*
- * SPDX-FileCopyrightText: Copyright (c) 1999-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+ * SPDX-FileCopyrightText: Copyright (c) 1999-2024 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
  * SPDX-License-Identifier: MIT
  *
  * Permission is hereby granted, free of charge, to any person obtaining a
@@ -81,13 +81,15 @@ NV_STATUS  NV_API_CALL rm_gpu_ops_has_pending_non_replayable_faults(nvidia_stack
 NV_STATUS  NV_API_CALL rm_gpu_ops_init_access_cntr_info(nvidia_stack_t *, nvgpuDeviceHandle_t, nvgpuAccessCntrInfo_t, NvU32);
 NV_STATUS  NV_API_CALL rm_gpu_ops_destroy_access_cntr_info(nvidia_stack_t *, nvgpuDeviceHandle_t, nvgpuAccessCntrInfo_t);
 NV_STATUS  NV_API_CALL rm_gpu_ops_own_access_cntr_intr(nvidia_stack_t *, nvgpuSessionHandle_t, nvgpuAccessCntrInfo_t, NvBool);
-NV_STATUS  NV_API_CALL rm_gpu_ops_enable_access_cntr(nvidia_stack_t *, nvgpuDeviceHandle_t, nvgpuAccessCntrInfo_t, nvgpuAccessCntrConfig_t);
+NV_STATUS  NV_API_CALL rm_gpu_ops_enable_access_cntr(nvidia_stack_t *, nvgpuDeviceHandle_t, nvgpuAccessCntrInfo_t, const nvgpuAccessCntrConfig_t *);
 NV_STATUS  NV_API_CALL rm_gpu_ops_disable_access_cntr(nvidia_stack_t *, nvgpuDeviceHandle_t, nvgpuAccessCntrInfo_t);
-NV_STATUS  NV_API_CALL  rm_gpu_ops_set_page_directory (nvidia_stack_t *, nvgpuAddressSpaceHandle_t, NvU64, unsigned, NvBool, NvU32);
+NV_STATUS  NV_API_CALL  rm_gpu_ops_set_page_directory (nvidia_stack_t *, nvgpuAddressSpaceHandle_t, NvU64, unsigned, NvBool, NvU32, NvU64 *);
 NV_STATUS  NV_API_CALL  rm_gpu_ops_unset_page_directory (nvidia_stack_t *, nvgpuAddressSpaceHandle_t);
+NV_STATUS  NV_API_CALL rm_gpu_ops_get_nvlink_info(nvidia_stack_t *, nvgpuDeviceHandle_t, nvgpuNvlinkInfo_t);
 NV_STATUS  NV_API_CALL rm_gpu_ops_p2p_object_create(nvidia_stack_t *, nvgpuDeviceHandle_t, nvgpuDeviceHandle_t, NvHandle *);
 void       NV_API_CALL rm_gpu_ops_p2p_object_destroy(nvidia_stack_t *, nvgpuSessionHandle_t, NvHandle);
 NV_STATUS  NV_API_CALL rm_gpu_ops_get_external_alloc_ptes(nvidia_stack_t*, nvgpuAddressSpaceHandle_t, NvHandle, NvU64, NvU64, nvgpuExternalMappingInfo_t);
+NV_STATUS  NV_API_CALL rm_gpu_ops_get_external_alloc_phys_addrs(nvidia_stack_t*, nvgpuAddressSpaceHandle_t, NvHandle, NvU64, NvU64, nvgpuExternalPhysAddrInfo_t);
 NV_STATUS  NV_API_CALL rm_gpu_ops_retain_channel(nvidia_stack_t *, nvgpuAddressSpaceHandle_t, NvHandle, NvHandle, void **, nvgpuChannelInstanceInfo_t);
 NV_STATUS  NV_API_CALL rm_gpu_ops_bind_channel_resources(nvidia_stack_t *, void *, nvgpuChannelResourceBindParams_t);
 void       NV_API_CALL rm_gpu_ops_release_channel(nvidia_stack_t *, void *);
@@ -100,17 +102,18 @@ void       NV_API_CALL rm_gpu_ops_paging_channel_destroy(nvidia_stack_t *, nvgpu
 NV_STATUS  NV_API_CALL rm_gpu_ops_paging_channels_map(nvidia_stack_t *, nvgpuAddressSpaceHandle_t, NvU64, nvgpuDeviceHandle_t, NvU64 *);
 void       NV_API_CALL rm_gpu_ops_paging_channels_unmap(nvidia_stack_t *, nvgpuAddressSpaceHandle_t, NvU64, nvgpuDeviceHandle_t);
 NV_STATUS  NV_API_CALL rm_gpu_ops_paging_channel_push_stream(nvidia_stack_t *, nvgpuPagingChannelHandle_t, char *, NvU32);
+void       NV_API_CALL rm_gpu_ops_report_fatal_error(nvidia_stack_t *, NV_STATUS error);
 
 NV_STATUS  NV_API_CALL rm_gpu_ops_ccsl_context_init(nvidia_stack_t *, struct ccslContext_t **, nvgpuChannelHandle_t);
 NV_STATUS  NV_API_CALL rm_gpu_ops_ccsl_context_clear(nvidia_stack_t *, struct ccslContext_t *);
-NV_STATUS  NV_API_CALL rm_gpu_ops_ccsl_context_update(nvidia_stack_t *, struct ccslContext_t *);
+NV_STATUS  NV_API_CALL rm_gpu_ops_ccsl_rotate_key(nvidia_stack_t *, UvmCslContext *[], NvU32);
 NV_STATUS  NV_API_CALL rm_gpu_ops_ccsl_rotate_iv(nvidia_stack_t *, struct ccslContext_t *, NvU8);
 NV_STATUS  NV_API_CALL rm_gpu_ops_ccsl_encrypt(nvidia_stack_t *, struct ccslContext_t *, NvU32, NvU8 const *, NvU8 *, NvU8 *);
 NV_STATUS  NV_API_CALL rm_gpu_ops_ccsl_encrypt_with_iv(nvidia_stack_t *, struct ccslContext_t *, NvU32, NvU8 const *, NvU8*, NvU8 *, NvU8 *);
-NV_STATUS  NV_API_CALL rm_gpu_ops_ccsl_decrypt(nvidia_stack_t *, struct ccslContext_t *, NvU32, NvU8 const *, NvU8 const *, NvU8 *, NvU8 const *, NvU32, NvU8 const *);
+NV_STATUS  NV_API_CALL rm_gpu_ops_ccsl_decrypt(nvidia_stack_t *, struct ccslContext_t *, NvU32, NvU8 const *, NvU8 const *, NvU32, NvU8 *, NvU8 const *, NvU32, NvU8 const *);
 NV_STATUS  NV_API_CALL rm_gpu_ops_ccsl_sign(nvidia_stack_t *, struct ccslContext_t *, NvU32, NvU8 const *, NvU8 *);
 NV_STATUS  NV_API_CALL rm_gpu_ops_ccsl_query_message_pool(nvidia_stack_t *, struct ccslContext_t *, NvU8, NvU64 *);
 NV_STATUS  NV_API_CALL rm_gpu_ops_ccsl_increment_iv(nvidia_stack_t *, struct ccslContext_t *, NvU8, NvU64, NvU8 *);
-NV_STATUS  NV_API_CALL rm_gpu_ops_ccsl_log_device_encryption(nvidia_stack_t *, struct ccslContext_t *, NvU32);
+NV_STATUS  NV_API_CALL rm_gpu_ops_ccsl_log_encryption(nvidia_stack_t *, struct ccslContext_t *, NvU8, NvU32);
 
 #endif

kernel-open/common/inc/rs_access.h

@@ -0,0 +1,276 @@
+/*
+ * SPDX-FileCopyrightText: Copyright (c) 2019-2024 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+ * SPDX-License-Identifier: MIT
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
+ * DEALINGS IN THE SOFTWARE.
+ */
+#pragma once
+
+#include <nvtypes.h>
+#if defined(_MSC_VER)
+#pragma warning(disable:4324)
+#endif
+
+//
+// This file was generated with FINN, an NVIDIA coding tool.
+// Source file:      rs_access.finn
+//
+
+
+
+
+#include "nvtypes.h"
+#include "nvmisc.h"
+
+
+/****************************************************************************/
+/*                       Access right definitions                           */
+/****************************************************************************/
+
+//
+// The meaning of each access right is documented in
+//   resman/docs/rmapi/resource_server/rm_capabilities.adoc
+//
+// RS_ACCESS_COUNT is the number of access rights that have been defined
+// and are in use. All integers in the range [0, RS_ACCESS_COUNT) should
+// represent valid access rights.
+//
+// When adding a new access right, don't forget to update
+//   1) The descriptions in the resman/docs/rmapi/resource_server/rm_capabilities.adoc
+//   2) RS_ACCESS_COUNT, defined below
+//   3) The declaration of g_rsAccessMetadata in rs_access_rights.c
+//   4) The list of access rights in drivers/common/chip-config/Chipcontrols.pm
+//   5) Any relevant access right callbacks
+//
+
+#define RS_ACCESS_DUP_OBJECT 0U
+#define RS_ACCESS_NICE       1U
+#define RS_ACCESS_DEBUG      2U
+#define RS_ACCESS_PERFMON    3U
+#define RS_ACCESS_COUNT      4U
+
+
+/****************************************************************************/
+/*                     Access right data structures                         */
+/****************************************************************************/
+
+/*!
+ * @brief A type that can be used to represent any access right.
+ */
+typedef NvU16 RsAccessRight;
+
+/*!
+ * @brief An internal type used to represent one limb in an access right mask.
+ */
+typedef NvU32 RsAccessLimb;
+#define SDK_RS_ACCESS_LIMB_BITS 32
+
+/*!
+ * @brief The number of limbs in the RS_ACCESS_MASK struct.
+ */
+#define SDK_RS_ACCESS_MAX_LIMBS 1
+
+/*!
+ * @brief The maximum number of possible access rights supported by the
+ *        current data structure definition.
+ *
+ * You probably want RS_ACCESS_COUNT instead, which is the number of actual
+ * access rights defined.
+ */
+#define SDK_RS_ACCESS_MAX_COUNT (0x20) /* finn: Evaluated from "(SDK_RS_ACCESS_LIMB_BITS * SDK_RS_ACCESS_MAX_LIMBS)" */
+
+/**
+ * @brief A struct representing a set of access rights.
+ *
+ * Note that the values of bit positions larger than RS_ACCESS_COUNT is
+ * undefined, and should not be assumed to be 0 (see RS_ACCESS_MASK_FILL).
+ */
+typedef struct RS_ACCESS_MASK {
+    RsAccessLimb limbs[SDK_RS_ACCESS_MAX_LIMBS];
+} RS_ACCESS_MASK;
+
+/**
+ * @brief A struct representing auxiliary information about each access right.
+ */
+typedef struct RS_ACCESS_INFO {
+    NvU32 flags;
+} RS_ACCESS_INFO;
+
+
+/****************************************************************************/
+/*                           Access right macros                            */
+/****************************************************************************/
+
+#define SDK_RS_ACCESS_LIMB_INDEX(index) ((index) / SDK_RS_ACCESS_LIMB_BITS)
+#define SDK_RS_ACCESS_LIMB_POS(index)   ((index) % SDK_RS_ACCESS_LIMB_BITS)
+
+#define SDK_RS_ACCESS_LIMB_ELT(pAccessMask, index) \
+    ((pAccessMask)->limbs[SDK_RS_ACCESS_LIMB_INDEX(index)])
+#define SDK_RS_ACCESS_OFFSET_MASK(index) \
+    NVBIT_TYPE(SDK_RS_ACCESS_LIMB_POS(index), RsAccessLimb)
+
+/*!
+ * @brief Checks that accessRight represents a valid access right.
+ *
+ * The valid range of access rights is [0, RS_ACCESS_COUNT).
+ *
+ * @param[in] accessRight The access right value to check
+ *
+ * @return true if accessRight is valid
+ * @return false otherwise
+ */
+#define RS_ACCESS_BOUNDS_CHECK(accessRight) \
+    (accessRight < RS_ACCESS_COUNT)
+
+/*!
+ * @brief Test whether an access right is present in a set
+ *
+ * @param[in] pAccessMask The set of access rights to read
+ * @param[in] index The access right to examine
+ *
+ * @return NV_TRUE if the access right specified by index was present in the set,
+ *         and NV_FALSE otherwise
+ */
+#define RS_ACCESS_MASK_TEST(pAccessMask, index) \
+    (RS_ACCESS_BOUNDS_CHECK(index) && \
+        (SDK_RS_ACCESS_LIMB_ELT(pAccessMask, index) & SDK_RS_ACCESS_OFFSET_MASK(index)) != 0)
+
+/*!
+ * @brief Add an access right to a mask
+ *
+ * @param[in] pAccessMask The set of access rights to modify
+ * @param[in] index The access right to set
+ */
+#define RS_ACCESS_MASK_ADD(pAccessMask, index) \
+    do \
+    { \
+        if (RS_ACCESS_BOUNDS_CHECK(index)) { \
+            SDK_RS_ACCESS_LIMB_ELT(pAccessMask, index) |= SDK_RS_ACCESS_OFFSET_MASK(index); \
+        } \
+    } while (NV_FALSE)
+
+/*!
+ * @brief Remove an access right from a mask
+ *
+ * @param[in] pAccessMask The set of access rights to modify
+ * @param[in] index The access right to unset
+ */
+#define RS_ACCESS_MASK_REMOVE(pAccessMask, index) \
+    do \
+    { \
+        if (RS_ACCESS_BOUNDS_CHECK(index)) { \
+            SDK_RS_ACCESS_LIMB_ELT(pAccessMask, index) &= ~SDK_RS_ACCESS_OFFSET_MASK(index); \
+        } \
+    } while (NV_FALSE)
+
+/*!
+ * @brief Performs an in-place union between two access right masks
+ *
+ * @param[in,out] pMaskOut The access rights mask to be updated
+ * @param[in] pMaskIn The set of access rights to be added to pMaskOut
+ */
+#define RS_ACCESS_MASK_UNION(pMaskOut, pMaskIn) \
+    do \
+    { \
+        NvLength limb; \
+        for (limb = 0; limb < SDK_RS_ACCESS_MAX_LIMBS; limb++) \
+        { \
+            SDK_RS_ACCESS_LIMB_ELT(pMaskOut, limb) |= SDK_RS_ACCESS_LIMB_ELT(pMaskIn, limb); \
+        } \
+    } while (NV_FALSE)
+
+/*!
+ * @brief Performs an in-place subtract of one mask's rights from another
+ *
+ * @param[in,out] pMaskOut The access rights mask to be updated
+ * @param[in] pMaskIn The set of access rights to be removed from pMaskOut
+ */
+#define RS_ACCESS_MASK_SUBTRACT(pMaskOut, pMaskIn) \
+    do \
+    { \
+        NvLength limb; \
+        for (limb = 0; limb < SDK_RS_ACCESS_MAX_LIMBS; limb++) \
+        { \
+            SDK_RS_ACCESS_LIMB_ELT(pMaskOut, limb) &= ~SDK_RS_ACCESS_LIMB_ELT(pMaskIn, limb); \
+        } \
+    } while (NV_FALSE)
+
+/*!
+ * @brief Removes all rights from an access rights mask
+ *
+ * @param[in,out] pAccessMask The access rights mask to be updated
+ */
+#define RS_ACCESS_MASK_CLEAR(pAccessMask) \
+    do \
+    { \
+        portMemSet(pAccessMask, 0, sizeof(*pAccessMask)); \
+    } while (NV_FALSE)
+
+/*!
+ * @brief Adds all rights to an access rights mask
+ *
+ * @param[in,out] pAccessMask The access rights mask to be updated
+ */
+#define RS_ACCESS_MASK_FILL(pAccessMask) \
+    do \
+    { \
+        portMemSet(pAccessMask, 0xff, sizeof(*pAccessMask)); \
+    } while (NV_FALSE)
+
+
+/****************************************************************************/
+/*                           Share definitions                              */
+/****************************************************************************/
+
+//
+// The usage of Share Policy and the meaning of each share type is documented in
+//   resman/docs/rmapi/resource_server/rm_capabilities.adoc
+//
+#define RS_SHARE_TYPE_NONE              (0U)
+#define RS_SHARE_TYPE_ALL               (1U)
+#define RS_SHARE_TYPE_OS_SECURITY_TOKEN (2U)
+#define RS_SHARE_TYPE_CLIENT            (3U)
+#define RS_SHARE_TYPE_PID               (4U)
+#define RS_SHARE_TYPE_SMC_PARTITION     (5U)
+#define RS_SHARE_TYPE_GPU               (6U)
+#define RS_SHARE_TYPE_FM_CLIENT         (7U)
+// Must be last. Update when a new SHARE_TYPE is added
+#define RS_SHARE_TYPE_MAX               (8U)
+
+
+//
+// Use Revoke to remove an existing policy from the list.
+// Allow is based on OR logic, Require is based on AND logic.
+// To share a right, at least one Allow (non-Require) must match, and all Require must pass.
+// If Compose is specified, policies will be added to the list. Otherwise, they will replace the list.
+//
+#define RS_SHARE_ACTION_FLAG_REVOKE      NVBIT(0)
+#define RS_SHARE_ACTION_FLAG_REQUIRE     NVBIT(1)
+#define RS_SHARE_ACTION_FLAG_COMPOSE     NVBIT(2)
+
+/****************************************************************************/
+/*                       Share flag data structures                         */
+/****************************************************************************/
+
+typedef struct RS_SHARE_POLICY {
+    NvU32          target;
+    RS_ACCESS_MASK accessMask;
+    NvU16          type;                         ///< RS_SHARE_TYPE_
+    NvU8           action;                        ///< RS_SHARE_ACTION_
+} RS_SHARE_POLICY;

kernel-open/header-presence-tests.mk

@@ -4,98 +4,39 @@ NV_HEADER_PRESENCE_TESTS = \
   asm/system.h \
   drm/drmP.h \
   drm/drm_aperture.h \
-  drm/drm_auth.h \
-  drm/drm_gem.h \
-  drm/drm_crtc.h \
-  drm/drm_color_mgmt.h \
-  drm/drm_atomic.h \
-  drm/drm_atomic_helper.h \
   drm/drm_atomic_state_helper.h \
-  drm/drm_encoder.h \
   drm/drm_atomic_uapi.h \
-  drm/drm_drv.h \
   drm/drm_fbdev_generic.h \
-  drm/drm_framebuffer.h \
-  drm/drm_connector.h \
+  drm/drm_fbdev_ttm.h \
+  drm/drm_client_setup.h \
   drm/drm_probe_helper.h \
-  drm/drm_blend.h \
-  drm/drm_fourcc.h \
-  drm/drm_prime.h \
-  drm/drm_plane.h \
-  drm/drm_vblank.h \
-  drm/drm_file.h \
-  drm/drm_ioctl.h \
-  drm/drm_device.h \
-  drm/drm_mode_config.h \
-  drm/drm_modeset_lock.h \
+  drm/clients/drm_client_setup.h \
   dt-bindings/interconnect/tegra_icc_id.h \
   generated/autoconf.h \
   generated/compile.h \
   generated/utsrelease.h \
-  linux/efi.h \
-  linux/kconfig.h \
+  linux/pfn_t.h \
+  linux/aperture.h \
+  linux/dma-direct.h \
   linux/platform/tegra/mc_utils.h \
-  linux/printk.h \
-  linux/ratelimit.h \
-  linux/prio_tree.h \
-  linux/log2.h \
-  linux/of.h \
-  linux/bug.h \
-  linux/sched.h \
-  linux/sched/mm.h \
-  linux/sched/signal.h \
-  linux/sched/task.h \
-  linux/sched/task_stack.h \
   xen/ioemu.h \
   linux/fence.h \
-  linux/dma-fence.h \
   linux/dma-resv.h \
-  soc/tegra/chip-id.h \
-  soc/tegra/fuse.h \
   soc/tegra/tegra_bpmp.h \
-  video/nv_internal.h \
   linux/platform/tegra/dce/dce-client-ipc.h \
   linux/nvhost.h \
   linux/nvhost_t194.h \
   linux/host1x-next.h \
-  asm/book3s/64/hash-64k.h \
   asm/set_memory.h \
-  asm/prom.h \
-  asm/powernv.h \
-  linux/atomic.h \
-  asm/barrier.h \
-  asm/opal-api.h \
-  sound/hdaudio.h \
   asm/pgtable_types.h \
-  asm/page.h \
-  linux/stringhash.h \
   linux/dma-map-ops.h \
-  rdma/peer_mem.h \
   sound/hda_codec.h \
-  linux/dma-buf.h \
-  linux/time.h \
-  linux/platform_device.h \
-  linux/mutex.h \
-  linux/reset.h \
-  linux/of_platform.h \
-  linux/of_device.h \
-  linux/of_gpio.h \
-  linux/gpio.h \
-  linux/gpio/consumer.h \
   linux/interconnect.h \
-  linux/pm_runtime.h \
-  linux/clk.h \
-  linux/clk-provider.h \
   linux/ioasid.h \
   linux/stdarg.h \
   linux/iosys-map.h \
-  asm/coco.h \
   linux/vfio_pci_core.h \
-  linux/mdev.h \
-  soc/tegra/bpmp-abi.h \
-  soc/tegra/bpmp.h \
-  linux/sync_file.h \
   linux/cc_platform.h \
-  asm/cpufeature.h \
-  linux/mpi.h
+  asm/mshyperv.h \
+  crypto/sig.h
 

kernel-open/nvidia-drm/nv-kthread-q.c

@@ -1,5 +1,5 @@
 /*
- * SPDX-FileCopyrightText: Copyright (c) 2016 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+ * SPDX-FileCopyrightText: Copyright (c) 2016-2024 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
  * SPDX-License-Identifier: MIT
  *
  * Permission is hereby granted, free of charge, to any person obtaining a
@@ -29,12 +29,7 @@
 #include <linux/completion.h>
 #include <linux/module.h>
 #include <linux/mm.h>
-
-#if defined(NV_LINUX_BUG_H_PRESENT)
-    #include <linux/bug.h>
-#else
-    #include <asm/bug.h>
-#endif
+#include <linux/bug.h>
 
 // Today's implementation is a little simpler and more limited than the
 // API description allows for in nv-kthread-q.h. Details include:
@@ -176,7 +171,7 @@ static struct task_struct *thread_create_on_node(int (*threadfn)(void *data),
 {
 
     unsigned i, j;
-    const static unsigned attempts = 3;
+    static const unsigned attempts = 3;
     struct task_struct *thread[3];
 
     for (i = 0;; i++) {
@@ -201,7 +196,7 @@ static struct task_struct *thread_create_on_node(int (*threadfn)(void *data),
 
         // Ran out of attempts - return thread even if its stack may not be
         // allocated on the preferred node
-        if ((i == (attempts - 1)))
+        if (i == (attempts - 1))
             break;
 
         // Get the NUMA node where the first page of the stack is resident. If

kernel-open/nvidia-drm/nv_common_utils.h

@@ -0,0 +1,120 @@
+/*
+ * SPDX-FileCopyrightText: Copyright (c) 2015 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+ * SPDX-License-Identifier: MIT
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
+ * DEALINGS IN THE SOFTWARE.
+ */
+
+#ifndef __NV_COMMON_UTILS_H__
+#define __NV_COMMON_UTILS_H__
+
+#include "nvtypes.h"
+#include "nvmisc.h"
+
+#if !defined(TRUE)
+#define TRUE NV_TRUE
+#endif
+
+#if !defined(FALSE)
+#define FALSE NV_FALSE
+#endif
+
+#define NV_IS_UNSIGNED(x) ((__typeof__(x))-1 > 0)
+
+/* Get the length of a statically-sized array. */
+#define ARRAY_LEN(_arr) (sizeof(_arr) / sizeof(_arr[0]))
+
+#define NV_INVALID_HEAD         0xFFFFFFFF
+
+#define NV_INVALID_CONNECTOR_PHYSICAL_INFORMATION (~0)
+
+#if !defined(NV_MIN)
+# define NV_MIN(a,b) (((a)<(b))?(a):(b))
+#endif
+
+#define NV_MIN3(a,b,c) NV_MIN(NV_MIN(a, b), c)
+#define NV_MIN4(a,b,c,d) NV_MIN3(NV_MIN(a,b),c,d)
+
+#if !defined(NV_MAX)
+# define NV_MAX(a,b) (((a)>(b))?(a):(b))
+#endif
+
+#define NV_MAX3(a,b,c) NV_MAX(NV_MAX(a, b), c)
+#define NV_MAX4(a,b,c,d) NV_MAX3(NV_MAX(a,b),c,d)
+
+static inline int NV_LIMIT_VAL_TO_MIN_MAX(int val, int min, int max)
+{
+    if (val < min) {
+        return min;
+    }
+    if (val > max) {
+        return max;
+    }
+    return val;
+}
+
+#define NV_ROUNDUP_DIV(x,y) ((x) / (y) + (((x) % (y)) ? 1 : 0))
+
+/*
+ * Macros used for computing palette entries:
+ *
+ * NV_UNDER_REPLICATE(val, source_size, result_size) expands a value
+ * of source_size bits into a value of target_size bits by shifting
+ * the source value into the high bits and replicating the high bits
+ * of the value into the low bits of the result.
+ *
+ * PALETTE_DEPTH_SHIFT(val, w) maps a colormap entry for a component
+ * that has w bits to an appropriate entry in a LUT of 256 entries.
+ */
+static inline unsigned int NV_UNDER_REPLICATE(unsigned short val,
+                                              int source_size,
+                                              int result_size)
+{
+    return (val << (result_size - source_size)) |
+        (val >> ((source_size << 1) - result_size));
+}
+
+
+static inline unsigned short PALETTE_DEPTH_SHIFT(unsigned short val, int depth)
+{
+    return NV_UNDER_REPLICATE(val, depth, 8);
+}
+
+/*
+ *  Use __builtin_ffs where it is supported, or provide an equivalent
+ *  implementation for platforms like riscv where it is not.
+ */
+#if defined(__GNUC__) && !NVCPU_IS_RISCV64
+static inline int nv_ffs(int x)
+{
+    return __builtin_ffs(x);
+}
+#else
+static inline int nv_ffs(int x)
+{
+    if (x == 0)
+        return 0;
+
+    LOWESTBITIDX_32(x);
+
+    return 1 + x;
+}
+#endif
+
+#endif /* __NV_COMMON_UTILS_H__ */

kernel-open/nvidia-drm/nvidia-dma-fence-helper.h

@@ -1,156 +0,0 @@
-/*
- * Copyright (c) 2017, NVIDIA CORPORATION. All rights reserved.
- *
- * Permission is hereby granted, free of charge, to any person obtaining a
- * copy of this software and associated documentation files (the "Software"),
- * to deal in the Software without restriction, including without limitation
- * the rights to use, copy, modify, merge, publish, distribute, sublicense,
- * and/or sell copies of the Software, and to permit persons to whom the
- * Software is furnished to do so, subject to the following conditions:
- *
- * The above copyright notice and this permission notice shall be included in
- * all copies or substantial portions of the Software.
- *
- * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
- * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
- * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
- * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
- * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
- * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
- * DEALINGS IN THE SOFTWARE.
- */
-
-#ifndef __NVIDIA_DMA_FENCE_HELPER_H__
-#define __NVIDIA_DMA_FENCE_HELPER_H__
-
-#include "nvidia-drm-conftest.h"
-
-#if defined(NV_DRM_FENCE_AVAILABLE)
-
-/*
- * Fence headers are moved to file dma-fence.h and struct fence has
- * been renamed to dma_fence by commit -
- *
- *      2016-10-25 : f54d1867005c3323f5d8ad83eed823e84226c429
- */
-
-#if defined(NV_LINUX_FENCE_H_PRESENT)
-#include <linux/fence.h>
-#else
-#include <linux/dma-fence.h>
-#endif
-
-#if defined(NV_LINUX_FENCE_H_PRESENT)
-typedef struct fence nv_dma_fence_t;
-typedef struct fence_ops nv_dma_fence_ops_t;
-typedef struct fence_cb nv_dma_fence_cb_t;
-typedef fence_func_t nv_dma_fence_func_t;
-#else
-typedef struct dma_fence nv_dma_fence_t;
-typedef struct dma_fence_ops nv_dma_fence_ops_t;
-typedef struct dma_fence_cb nv_dma_fence_cb_t;
-typedef dma_fence_func_t nv_dma_fence_func_t;
-#endif
-
-#if defined(NV_LINUX_FENCE_H_PRESENT)
-#define NV_DMA_FENCE_FLAG_ENABLE_SIGNAL_BIT FENCE_FLAG_ENABLE_SIGNAL_BIT
-#else
-#define NV_DMA_FENCE_FLAG_ENABLE_SIGNAL_BIT DMA_FENCE_FLAG_ENABLE_SIGNAL_BIT
-#endif
-
-static inline bool nv_dma_fence_is_signaled(nv_dma_fence_t *fence) {
-#if defined(NV_LINUX_FENCE_H_PRESENT)
-    return fence_is_signaled(fence);
-#else
-    return dma_fence_is_signaled(fence);
-#endif
-}
-
-static inline nv_dma_fence_t *nv_dma_fence_get(nv_dma_fence_t *fence)
-{
-#if defined(NV_LINUX_FENCE_H_PRESENT)
-    return fence_get(fence);
-#else
-    return dma_fence_get(fence);
-#endif
-}
-
-static inline void nv_dma_fence_put(nv_dma_fence_t *fence) {
-#if defined(NV_LINUX_FENCE_H_PRESENT)
-    fence_put(fence);
-#else
-    dma_fence_put(fence);
-#endif
-}
-
-static inline signed long
-nv_dma_fence_default_wait(nv_dma_fence_t *fence,
-                          bool intr, signed long timeout) {
-#if defined(NV_LINUX_FENCE_H_PRESENT)
-    return fence_default_wait(fence, intr, timeout);
-#else
-    return dma_fence_default_wait(fence, intr, timeout);
-#endif
-}
-
-static inline int nv_dma_fence_signal(nv_dma_fence_t *fence) {
-#if defined(NV_LINUX_FENCE_H_PRESENT)
-    return fence_signal(fence);
-#else
-    return dma_fence_signal(fence);
-#endif
-}
-
-static inline int nv_dma_fence_signal_locked(nv_dma_fence_t *fence) {
-#if defined(NV_LINUX_FENCE_H_PRESENT)
-    return fence_signal_locked(fence);
-#else
-    return dma_fence_signal_locked(fence);
-#endif
-}
-
-static inline u64 nv_dma_fence_context_alloc(unsigned num) {
-#if defined(NV_LINUX_FENCE_H_PRESENT)
-    return fence_context_alloc(num);
-#else
-    return dma_fence_context_alloc(num);
-#endif
-}
-
-static inline void
-nv_dma_fence_init(nv_dma_fence_t *fence,
-                  const nv_dma_fence_ops_t *ops,
-                  spinlock_t *lock, u64 context, uint64_t seqno) {
-#if defined(NV_LINUX_FENCE_H_PRESENT)
-    fence_init(fence, ops, lock, context, seqno);
-#else
-    dma_fence_init(fence, ops, lock, context, seqno);
-#endif
-}
-
-static inline void
-nv_dma_fence_set_error(nv_dma_fence_t *fence,
-                       int error) {
-#if defined(NV_DMA_FENCE_SET_ERROR_PRESENT)
-    return dma_fence_set_error(fence, error);
-#elif defined(NV_FENCE_SET_ERROR_PRESENT)
-    return fence_set_error(fence, error);
-#else
-    fence->status = error;
-#endif
-}
-
-static inline int
-nv_dma_fence_add_callback(nv_dma_fence_t *fence,
-                          nv_dma_fence_cb_t *cb,
-                          nv_dma_fence_func_t func) {
-#if defined(NV_LINUX_FENCE_H_PRESENT)
-    return fence_add_callback(fence, cb, func);
-#else
-    return dma_fence_add_callback(fence, cb, func);
-#endif
-}
-
-#endif /* defined(NV_DRM_FENCE_AVAILABLE) */
-
-#endif /* __NVIDIA_DMA_FENCE_HELPER_H__ */

kernel-open/nvidia-drm/nvidia-dma-resv-helper.h

@@ -25,8 +25,6 @@
 
 #include "nvidia-drm-conftest.h"
 
-#if defined(NV_DRM_FENCE_AVAILABLE)
-
 /*
  * linux/reservation.h is renamed to linux/dma-resv.h, by commit
  * 52791eeec1d9 (dma-buf: rename reservation_object to dma_resv)
@@ -39,7 +37,7 @@
 #include <linux/reservation.h>
 #endif
 
-#include <nvidia-dma-fence-helper.h>
+#include <linux/dma-fence.h>
 
 #if defined(NV_LINUX_DMA_RESV_H_PRESENT)
 typedef struct dma_resv nv_dma_resv_t;
@@ -108,7 +106,7 @@ static inline int nv_dma_resv_reserve_fences(nv_dma_resv_t *obj,
 }
 
 static inline void nv_dma_resv_add_excl_fence(nv_dma_resv_t *obj,
-                                              nv_dma_fence_t *fence)
+                                              struct dma_fence *fence)
 {
 #if defined(NV_LINUX_DMA_RESV_H_PRESENT)
 #if defined(NV_DMA_RESV_ADD_FENCE_PRESENT)
@@ -122,7 +120,7 @@ static inline void nv_dma_resv_add_excl_fence(nv_dma_resv_t *obj,
 }
 
 static inline void nv_dma_resv_add_shared_fence(nv_dma_resv_t *obj,
-                                                nv_dma_fence_t *fence)
+                                                struct dma_fence *fence)
 {
 #if defined(NV_LINUX_DMA_RESV_H_PRESENT)
 #if defined(NV_DMA_RESV_ADD_FENCE_PRESENT)
@@ -135,6 +133,4 @@ static inline void nv_dma_resv_add_shared_fence(nv_dma_resv_t *obj,
 #endif
 }
 
-#endif /* defined(NV_DRM_FENCE_AVAILABLE) */
-
 #endif /* __NVIDIA_DMA_RESV_HELPER_H__ */

kernel-open/nvidia-drm/nvidia-drm-conftest.h

@@ -55,22 +55,35 @@
 
 #endif
 
-#if defined(NV_DRM_DRIVER_HAS_GEM_PRIME_RES_OBJ) || \
-    defined(NV_DRM_GEM_OBJECT_HAS_RESV)
-#define NV_DRM_FENCE_AVAILABLE
-#else
-#undef NV_DRM_FENCE_AVAILABLE
+#include <linux/kconfig.h> // for IS_ENABLED()
+
+#if IS_ENABLED(CONFIG_DRM) || defined(__FreeBSD__)
+#define NV_DRM_AVAILABLE
 #endif
 
-/*
- * We can support color management if either drm_helper_crtc_enable_color_mgmt()
- * or drm_crtc_enable_color_mgmt() exist.
- */
-#if defined(NV_DRM_HELPER_CRTC_ENABLE_COLOR_MGMT_PRESENT) || \
-    defined(NV_DRM_CRTC_ENABLE_COLOR_MGMT_PRESENT)
-#define NV_DRM_COLOR_MGMT_AVAILABLE
-#else
-#undef NV_DRM_COLOR_MGMT_AVAILABLE
+#if defined(NV_DRM_FBDEV_GENERIC_SETUP_PRESENT) &&                             \
+    defined(NV_DRM_APERTURE_REMOVE_CONFLICTING_PCI_FRAMEBUFFERS_PRESENT)
+#define NV_DRM_FBDEV_AVAILABLE
+#define NV_DRM_FBDEV_GENERIC_AVAILABLE
+#endif
+
+#if defined(NV_DRM_FBDEV_TTM_SETUP_PRESENT) &&                                 \
+    defined(NV_DRM_APERTURE_REMOVE_CONFLICTING_PCI_FRAMEBUFFERS_PRESENT)
+#if IS_ENABLED(CONFIG_DRM_TTM_HELPER)
+#define NV_DRM_FBDEV_AVAILABLE
+#define NV_DRM_FBDEV_TTM_AVAILABLE
+#endif
+#endif
+
+#if defined(NV_DRM_CLIENT_SETUP_PRESENT) &&                                    \
+    (defined(NV_DRM_APERTURE_REMOVE_CONFLICTING_PCI_FRAMEBUFFERS_PRESENT) ||   \
+     defined(NV_APERTURE_REMOVE_CONFLICTING_PCI_DEVICES_PRESENT))
+// XXX remove dependency on DRM_TTM_HELPER by implementing nvidia-drm's own
+// .fbdev_probe callback that uses NVKMS kapi
+#if IS_ENABLED(CONFIG_DRM_TTM_HELPER)
+#define NV_DRM_FBDEV_AVAILABLE
+#define NV_DRM_CLIENT_AVAILABLE
+#endif
 #endif
 
 /*
@@ -85,7 +98,11 @@
 
 /* For nv_drm_gem_prime_force_fence_signal */
 #ifndef spin_is_locked
+#if ((__FreeBSD_version >= 1500000) && (__FreeBSD_version < 1500018)) || (__FreeBSD_version < 1401501)
 #define spin_is_locked(lock) mtx_owned(lock.m)
+#else
+#define spin_is_locked(lock) mtx_owned(lock)
+#endif
 #endif
 
 #ifndef rwsem_is_locked

kernel-open/nvidia-drm/nvidia-drm-connector.c

@@ -20,9 +20,9 @@
  * DEALINGS IN THE SOFTWARE.
  */
 
-#include "nvidia-drm-conftest.h" /* NV_DRM_ATOMIC_MODESET_AVAILABLE */
+#include "nvidia-drm-conftest.h" /* NV_DRM_AVAILABLE */
 
-#if defined(NV_DRM_ATOMIC_MODESET_AVAILABLE)
+#if defined(NV_DRM_AVAILABLE)
 
 #include "nvidia-drm-helper.h"
 #include "nvidia-drm-priv.h"
@@ -228,9 +228,6 @@ nv_drm_connector_detect(struct drm_connector *connector, bool force)
 }
 
 static struct drm_connector_funcs nv_connector_funcs = {
-#if defined NV_DRM_ATOMIC_HELPER_CONNECTOR_DPMS_PRESENT
-    .dpms                   = drm_atomic_helper_connector_dpms,
-#endif
     .destroy                = nv_drm_connector_destroy,
     .reset                  = drm_atomic_helper_connector_reset,
     .force                  = __nv_drm_connector_force,
@@ -314,7 +311,11 @@ static int nv_drm_connector_get_modes(struct drm_connector *connector)
 }
 
 static int nv_drm_connector_mode_valid(struct drm_connector    *connector,
+#if defined(NV_DRM_CONNECTOR_HELPER_FUNCS_MODE_VALID_HAS_CONST_MODE_ARG)
+                                       const struct drm_display_mode *mode)
+#else
                                        struct drm_display_mode *mode)
+#endif
 {
     struct drm_device *dev = connector->dev;
     struct nv_drm_device *nv_dev = to_nv_device(dev);
@@ -584,16 +585,11 @@ nv_drm_get_connector(struct drm_device *dev,
                      char dpAddress[NVKMS_DP_ADDRESS_STRING_LENGTH])
 {
     struct drm_connector *connector = NULL;
-#if defined(NV_DRM_CONNECTOR_LIST_ITER_PRESENT)
     struct drm_connector_list_iter conn_iter;
-    nv_drm_connector_list_iter_begin(dev, &conn_iter);
-#else
-    struct drm_mode_config *config = &dev->mode_config;
-    mutex_lock(&config->mutex);
-#endif
+    drm_connector_list_iter_begin(dev, &conn_iter);
 
     /* Lookup for existing connector with same physical index */
-    nv_drm_for_each_connector(connector, &conn_iter, dev) {
+    drm_for_each_connector_iter(connector, &conn_iter) {
         struct nv_drm_connector *nv_connector = to_nv_connector(connector);
 
         if (nv_connector->physicalIndex == physicalIndex) {
@@ -608,11 +604,7 @@ nv_drm_get_connector(struct drm_device *dev,
     connector = NULL;
 
 done:
-#if defined(NV_DRM_CONNECTOR_LIST_ITER_PRESENT)
-    nv_drm_connector_list_iter_end(&conn_iter);
-#else
-    mutex_unlock(&config->mutex);
-#endif
+    drm_connector_list_iter_end(&conn_iter);
 
     if (!connector) {
         connector = nv_drm_connector_new(dev,

kernel-open/nvidia-drm/nvidia-drm-connector.h

@@ -25,15 +25,13 @@
 
 #include "nvidia-drm-conftest.h"
 
-#if defined(NV_DRM_ATOMIC_MODESET_AVAILABLE)
+#if defined(NV_DRM_AVAILABLE)
 
 #if defined(NV_DRM_DRMP_H_PRESENT)
 #include <drm/drmP.h>
 #endif
 
-#if defined(NV_DRM_DRM_CONNECTOR_H_PRESENT)
 #include <drm/drm_connector.h>
-#endif
 
 #include "nvtypes.h"
 #include "nvkms-api-types.h"
@@ -101,6 +99,6 @@ nv_drm_get_connector(struct drm_device *dev,
 bool nv_drm_connector_revoke_permissions(struct drm_device *dev,
                                          struct nv_drm_connector *nv_connector);
 
-#endif /* NV_DRM_ATOMIC_MODESET_AVAILABLE */
+#endif /* NV_DRM_AVAILABLE */
 
 #endif /* __NVIDIA_DRM_CONNECTOR_H__ */

kernel-open/nvidia-drm/nvidia-drm-crtc.h

@@ -25,7 +25,7 @@
 
 #include "nvidia-drm-conftest.h"
 
-#if defined(NV_DRM_ATOMIC_MODESET_AVAILABLE)
+#if defined(NV_DRM_AVAILABLE)
 
 #include "nvidia-drm-helper.h"
 
@@ -38,6 +38,13 @@
 #include "nvtypes.h"
 #include "nvkms-kapi.h"
 
+enum nv_drm_transfer_function {
+    NV_DRM_TRANSFER_FUNCTION_DEFAULT,
+    NV_DRM_TRANSFER_FUNCTION_LINEAR,
+    NV_DRM_TRANSFER_FUNCTION_PQ,
+    NV_DRM_TRANSFER_FUNCTION_MAX,
+};
+
 struct nv_drm_crtc {
     NvU32 head;
 
@@ -63,6 +70,8 @@ struct nv_drm_crtc {
      */
     struct drm_file *modeset_permission_filep;
 
+    struct NvKmsLUTCaps olut_caps;
+
     struct drm_crtc base;
 };
 
@@ -129,9 +138,6 @@ struct nv_drm_crtc_state {
      */
     struct NvKmsKapiHeadRequestedConfig req_config;
 
-    struct NvKmsLutRamps *ilut_ramps;
-    struct NvKmsLutRamps *olut_ramps;
-
     /**
      * @nv_flip:
      *
@@ -145,6 +151,12 @@ struct nv_drm_crtc_state {
      * nv_drm_atomic_crtc_destroy_state().
      */
     struct nv_drm_flip *nv_flip;
+
+    enum nv_drm_transfer_function regamma_tf;
+    struct drm_property_blob *regamma_lut;
+    uint64_t regamma_divisor;
+    struct nv_drm_lut_surface *regamma_drm_lut_surface;
+    NvBool regamma_changed;
 };
 
 static inline struct nv_drm_crtc_state *to_nv_crtc_state(struct drm_crtc_state *state)
@@ -152,6 +164,11 @@ static inline struct nv_drm_crtc_state *to_nv_crtc_state(struct drm_crtc_state *
     return container_of(state, struct nv_drm_crtc_state, base);
 }
 
+static inline const struct nv_drm_crtc_state *to_nv_crtc_state_const(const struct drm_crtc_state *state)
+{
+    return container_of(state, struct nv_drm_crtc_state, base);
+}
+
 struct nv_drm_plane {
     /**
      * @base:
@@ -173,6 +190,16 @@ struct nv_drm_plane {
      * Index of this plane in the per head array of layers.
      */
     uint32_t layer_idx;
+
+    /**
+     * @supportsColorProperties
+     *
+     * If true, supports the COLOR_ENCODING and COLOR_RANGE properties.
+     */
+    bool supportsColorProperties;
+
+    struct NvKmsLUTCaps ilut_caps;
+    struct NvKmsLUTCaps tmo_caps;
 };
 
 static inline struct nv_drm_plane *to_nv_plane(struct drm_plane *plane)
@@ -183,13 +210,54 @@ static inline struct nv_drm_plane *to_nv_plane(struct drm_plane *plane)
     return container_of(plane, struct nv_drm_plane, base);
 }
 
+struct nv_drm_nvkms_surface {
+    struct NvKmsKapiDevice *pDevice;
+    struct NvKmsKapiMemory *nvkms_memory;
+    struct NvKmsKapiSurface *nvkms_surface;
+    void *buffer;
+    struct kref refcount;
+};
+
+struct nv_drm_nvkms_surface_params {
+    NvU32 width;
+    NvU32 height;
+    size_t surface_size;
+    enum NvKmsSurfaceMemoryFormat format;
+};
+
+struct nv_drm_lut_surface {
+    struct nv_drm_nvkms_surface base;
+    struct {
+        NvU32 vssSegments;
+        enum NvKmsLUTVssType vssType;
+
+        NvU32 lutEntries;
+        enum NvKmsLUTFormat entryFormat;
+
+    } properties;
+};
+
 struct nv_drm_plane_state {
     struct drm_plane_state base;
     s32 __user *fd_user_ptr;
-    enum NvKmsInputColorSpace input_colorspace;
+    enum nv_drm_input_color_space input_colorspace;
 #if defined(NV_DRM_HAS_HDR_OUTPUT_METADATA)
     struct drm_property_blob *hdr_output_metadata;
 #endif
+    struct drm_property_blob *lms_ctm;
+    struct drm_property_blob *lms_to_itp_ctm;
+    struct drm_property_blob *itp_to_lms_ctm;
+    struct drm_property_blob *blend_ctm;
+
+    enum nv_drm_transfer_function degamma_tf;
+    struct drm_property_blob *degamma_lut;
+    uint64_t degamma_multiplier; /* S31.32 Sign-Magnitude Format */
+    struct nv_drm_lut_surface *degamma_drm_lut_surface;
+    NvBool degamma_changed;
+
+    struct drm_property_blob *tmo_lut;
+    struct nv_drm_lut_surface *tmo_drm_lut_surface;
+    NvBool tmo_changed;
 };
 
 static inline struct nv_drm_plane_state *to_nv_drm_plane_state(struct drm_plane_state *state)
@@ -281,6 +349,6 @@ int nv_drm_get_crtc_crc32_ioctl(struct drm_device *dev,
 int nv_drm_get_crtc_crc32_v2_ioctl(struct drm_device *dev,
                                    void *data, struct drm_file *filep);
 
-#endif /* NV_DRM_ATOMIC_MODESET_AVAILABLE */
+#endif /* NV_DRM_AVAILABLE */
 
 #endif /* __NVIDIA_DRM_CRTC_H__ */

kernel-open/nvidia-drm/nvidia-drm-drv.h

@@ -27,13 +27,15 @@
 
 #if defined(NV_DRM_AVAILABLE)
 
+struct NvKmsKapiGpuInfo;
+
 int nv_drm_probe_devices(void);
 
 void nv_drm_remove_devices(void);
 
 void nv_drm_suspend_resume(NvBool suspend);
 
-void nv_drm_register_drm_device(const nv_gpu_info_t *);
+void nv_drm_register_drm_device(const struct NvKmsKapiGpuInfo *);
 
 void nv_drm_update_drm_driver_features(void);
 

kernel-open/nvidia-drm/nvidia-drm-encoder.c

@@ -20,9 +20,9 @@
  * DEALINGS IN THE SOFTWARE.
  */
 
-#include "nvidia-drm-conftest.h" /* NV_DRM_ATOMIC_MODESET_AVAILABLE */
+#include "nvidia-drm-conftest.h" /* NV_DRM_AVAILABLE */
 
-#if defined(NV_DRM_ATOMIC_MODESET_AVAILABLE)
+#if defined(NV_DRM_AVAILABLE)
 
 #include "nvidia-drm-priv.h"
 #include "nvidia-drm-encoder.h"
@@ -139,12 +139,8 @@ nv_drm_encoder_new(struct drm_device *dev,
 
     ret = drm_encoder_init(dev,
                            &nv_encoder->base, &nv_encoder_funcs,
-                           nvkms_connector_signal_to_drm_encoder_signal(format)
-#if defined(NV_DRM_ENCODER_INIT_HAS_NAME_ARG)
-                           , NULL
-#endif
-                           );
-
+                           nvkms_connector_signal_to_drm_encoder_signal(format),
+                           NULL);
     if (ret != 0) {
         nv_drm_free(nv_encoder);
 
@@ -319,7 +315,7 @@ void nv_drm_handle_dynamic_display_connected(struct nv_drm_device *nv_dev,
     nv_encoder = get_nv_encoder_from_nvkms_display(dev, hDisplay);
 
     if (nv_encoder != NULL) {
-        NV_DRM_DEV_LOG_ERR(
+        NV_DRM_DEV_LOG_INFO(
             nv_dev,
             "Encoder with NvKmsKapiDisplay 0x%08x already exists.",
             hDisplay);
@@ -336,17 +332,6 @@ void nv_drm_handle_dynamic_display_connected(struct nv_drm_device *nv_dev,
         return;
     }
 
-    /*
-     * On some kernels, DRM has the notion of a "primary group" that
-     * tracks the global mode setting state for the device.
-     *
-     * On kernels where DRM has a primary group, we need to reinitialize
-     * after adding encoders and connectors.
-     */
-#if defined(NV_DRM_REINIT_PRIMARY_MODE_GROUP_PRESENT)
-    drm_reinit_primary_mode_group(dev);
-#endif
-
     schedule_delayed_work(&nv_dev->hotplug_event_work, 0);
 }
 #endif

kernel-open/nvidia-drm/nvidia-drm-encoder.h

@@ -25,15 +25,11 @@
 
 #include "nvidia-drm-conftest.h"
 
-#if defined(NV_DRM_ATOMIC_MODESET_AVAILABLE)
+#if defined(NV_DRM_AVAILABLE)
 
 #include "nvidia-drm-priv.h"
 
-#if defined(NV_DRM_DRM_ENCODER_H_PRESENT)
 #include <drm/drm_encoder.h>
-#else
-#include <drm/drmP.h>
-#endif
 
 #include "nvkms-kapi.h"
 
@@ -63,6 +59,6 @@ void nv_drm_handle_display_change(struct nv_drm_device *nv_dev,
 void nv_drm_handle_dynamic_display_connected(struct nv_drm_device *nv_dev,
                                              NvKmsKapiDisplay hDisplay);
 
-#endif /* NV_DRM_ATOMIC_MODESET_AVAILABLE */
+#endif /* NV_DRM_AVAILABLE */
 
 #endif /* __NVIDIA_DRM_ENCODER_H__ */

kernel-open/nvidia-drm/nvidia-drm-fb.c

@@ -20,9 +20,9 @@
  * DEALINGS IN THE SOFTWARE.
  */
 
-#include "nvidia-drm-conftest.h" /* NV_DRM_ATOMIC_MODESET_AVAILABLE */
+#include "nvidia-drm-conftest.h" /* NV_DRM_AVAILABLE */
 
-#if defined(NV_DRM_ATOMIC_MODESET_AVAILABLE)
+#if defined(NV_DRM_AVAILABLE)
 
 #include "nvidia-drm-priv.h"
 #include "nvidia-drm-ioctl.h"
@@ -36,12 +36,15 @@
 
 static void __nv_drm_framebuffer_free(struct nv_drm_framebuffer *nv_fb)
 {
+    struct drm_framebuffer *fb = &nv_fb->base;
     uint32_t i;
 
     /* Unreference gem object */
-    for (i = 0; i < ARRAY_SIZE(nv_fb->nv_gem); i++) {
-        if (nv_fb->nv_gem[i] != NULL) {
-            nv_drm_gem_object_unreference_unlocked(nv_fb->nv_gem[i]);
+    for (i = 0; i < NVKMS_MAX_PLANES_PER_SURFACE; i++) {
+        struct drm_gem_object *gem = fb->obj[i];
+        if (gem != NULL) {
+            struct nv_drm_gem_object *nv_gem = to_nv_gem_object(gem);
+            nv_drm_gem_object_unreference_unlocked(nv_gem);
         }
     }
 
@@ -69,10 +72,8 @@ static int
 nv_drm_framebuffer_create_handle(struct drm_framebuffer *fb,
                                  struct drm_file *file, unsigned int *handle)
 {
-    struct nv_drm_framebuffer *nv_fb = to_nv_framebuffer(fb);
-
     return nv_drm_gem_handle_create(file,
-                                    nv_fb->nv_gem[0],
+                                    to_nv_gem_object(fb->obj[0]),
                                     handle);
 }
 
@@ -82,12 +83,12 @@ static struct drm_framebuffer_funcs nv_framebuffer_funcs = {
 };
 
 static struct nv_drm_framebuffer *nv_drm_framebuffer_alloc(
-    struct drm_device *dev,
+    struct nv_drm_device *nv_dev,
     struct drm_file *file,
-    struct drm_mode_fb_cmd2 *cmd)
+    const struct drm_mode_fb_cmd2 *cmd)
 {
-    struct nv_drm_device *nv_dev = to_nv_device(dev);
     struct nv_drm_framebuffer *nv_fb;
+    struct nv_drm_gem_object *nv_gem;
     const int num_planes = nv_drm_format_num_planes(cmd->pixel_format);
     uint32_t i;
 
@@ -101,21 +102,22 @@ static struct nv_drm_framebuffer *nv_drm_framebuffer_alloc(
         return ERR_PTR(-ENOMEM);
     }
 
-    if (num_planes > ARRAY_SIZE(nv_fb->nv_gem)) {
+    if (num_planes > NVKMS_MAX_PLANES_PER_SURFACE) {
         NV_DRM_DEV_DEBUG_DRIVER(nv_dev, "Unsupported number of planes");
         goto failed;
     }
 
     for (i = 0; i < num_planes; i++) {
-        if ((nv_fb->nv_gem[i] = nv_drm_gem_object_lookup(
-                        dev,
-                        file,
-                        cmd->handles[i])) == NULL) {
+        nv_gem = nv_drm_gem_object_lookup(file, cmd->handles[i]);
+
+        if (nv_gem == NULL) {
             NV_DRM_DEV_DEBUG_DRIVER(
                 nv_dev,
                 "Failed to find gem object of type nvkms memory");
             goto failed;
         }
+
+        nv_fb->base.obj[i] = &nv_gem->base;
     }
 
      return nv_fb;
@@ -135,12 +137,14 @@ static int nv_drm_framebuffer_init(struct drm_device *dev,
 {
     struct nv_drm_device *nv_dev = to_nv_device(dev);
     struct NvKmsKapiCreateSurfaceParams params = { };
+    struct nv_drm_gem_object *nv_gem;
+    struct drm_framebuffer *fb = &nv_fb->base;
     uint32_t i;
     int ret;
 
     /* Initialize the base framebuffer object and add it to drm subsystem */
 
-    ret = drm_framebuffer_init(dev, &nv_fb->base, &nv_framebuffer_funcs);
+    ret = drm_framebuffer_init(dev, fb, &nv_framebuffer_funcs);
     if (ret != 0) {
         NV_DRM_DEV_DEBUG_DRIVER(
             nv_dev,
@@ -148,23 +152,18 @@ static int nv_drm_framebuffer_init(struct drm_device *dev,
         return ret;
     }
 
-    for (i = 0; i < ARRAY_SIZE(nv_fb->nv_gem); i++) {
-        if (nv_fb->nv_gem[i] != NULL) {
-            if (!nvKms->isMemoryValidForDisplay(nv_dev->pDevice,
-                                                nv_fb->nv_gem[i]->pMemory)) {
-                NV_DRM_DEV_LOG_INFO(
-                        nv_dev,
-                        "Framebuffer memory not appropriate for scanout");
-                goto fail;
-            }
+    for (i = 0; i < NVKMS_MAX_PLANES_PER_SURFACE; i++) {
+        struct drm_gem_object *gem = fb->obj[i];
+        if (gem != NULL) {
+            nv_gem = to_nv_gem_object(gem);
 
-            params.planes[i].memory = nv_fb->nv_gem[i]->pMemory;
-            params.planes[i].offset = nv_fb->base.offsets[i];
-            params.planes[i].pitch = nv_fb->base.pitches[i];
+            params.planes[i].memory = nv_gem->pMemory;
+            params.planes[i].offset = fb->offsets[i];
+            params.planes[i].pitch = fb->pitches[i];
         }
     }
-    params.height = nv_fb->base.height;
-    params.width = nv_fb->base.width;
+    params.height = fb->height;
+    params.width = fb->width;
     params.format = format;
 
     if (have_modifier) {
@@ -188,6 +187,43 @@ static int nv_drm_framebuffer_init(struct drm_device *dev,
         params.explicit_layout = false;
     }
 
+    /*
+     * XXX work around an invalid pitch assumption in DRM.
+     *
+     * The smallest pitch the display hardware allows is 256.
+     *
+     * If a DRM client allocates a 32x32 cursor surface through
+     * DRM_IOCTL_MODE_CREATE_DUMB, we'll correctly round the pitch to 256:
+     *
+     *     pitch = round(32width * 4Bpp, 256) = 256
+     *
+     * and then allocate an 8k surface:
+     *
+     *     size = pitch * 32height = 8196
+     *
+     * and report the rounded pitch and size back to the client through the
+     * struct drm_mode_create_dumb ioctl params.
+     *
+     * But when the DRM client passes that buffer object handle to
+     * DRM_IOCTL_MODE_CURSOR, the client has no way to specify the pitch.  This
+     * path in drm:
+     *
+     *    DRM_IOCTL_MODE_CURSOR
+     *     drm_mode_cursor_ioctl()
+     *      drm_mode_cursor_common()
+     *       drm_mode_cursor_universal()
+     *
+     * will implicitly create a framebuffer from the buffer object, and compute
+     * the pitch as width x 32 (without aligning to our minimum pitch).
+     *
+     * Intercept this case and force the pitch back to 256.
+     */
+    if ((params.width == 32) &&
+        (params.height == 32) &&
+        (params.planes[0].pitch == 128)) {
+        params.planes[0].pitch = 256;
+    }
+
     /* Create NvKmsKapiSurface */
 
     nv_fb->pSurface = nvKms->createSurface(nv_dev->pDevice, &params);
@@ -199,23 +235,21 @@ static int nv_drm_framebuffer_init(struct drm_device *dev,
     return 0;
 
 fail:
-    drm_framebuffer_cleanup(&nv_fb->base);
+    drm_framebuffer_cleanup(fb);
     return -EINVAL;
 }
 
-struct drm_framebuffer *nv_drm_internal_framebuffer_create(
+struct drm_framebuffer *nv_drm_framebuffer_create(
     struct drm_device *dev,
     struct drm_file *file,
-    struct drm_mode_fb_cmd2 *cmd)
+    const struct drm_mode_fb_cmd2 *cmd)
 {
     struct nv_drm_device *nv_dev = to_nv_device(dev);
     struct nv_drm_framebuffer *nv_fb;
     uint64_t modifier = 0;
     int ret;
     enum NvKmsSurfaceMemoryFormat format;
-#if defined(NV_DRM_FORMAT_MODIFIERS_PRESENT)
     int i;
-#endif
     bool have_modifier = false;
 
     /* Check whether NvKms supports the given pixel format */
@@ -226,7 +260,6 @@ struct drm_framebuffer *nv_drm_internal_framebuffer_create(
         return ERR_PTR(-EINVAL);
     }
 
-#if defined(NV_DRM_FORMAT_MODIFIERS_PRESENT)
     if (cmd->flags & DRM_MODE_FB_MODIFIERS) {
         have_modifier = true;
         modifier = cmd->modifier[0];
@@ -245,9 +278,8 @@ struct drm_framebuffer *nv_drm_internal_framebuffer_create(
             return ERR_PTR(-EINVAL);
         }
     }
-#endif
 
-    nv_fb = nv_drm_framebuffer_alloc(dev, file, cmd);
+    nv_fb = nv_drm_framebuffer_alloc(nv_dev, file, cmd);
     if (IS_ERR(nv_fb)) {
         return (struct drm_framebuffer *)nv_fb;
     }
@@ -255,9 +287,7 @@ struct drm_framebuffer *nv_drm_internal_framebuffer_create(
     /* Fill out framebuffer metadata from the userspace fb creation request */
 
     drm_helper_mode_fill_fb_struct(
-        #if defined(NV_DRM_HELPER_MODE_FILL_FB_STRUCT_HAS_DEV_ARG)
         dev,
-        #endif
         &nv_fb->base,
         cmd);
 

kernel-open/nvidia-drm/nvidia-drm-fb.h

@@ -25,25 +25,19 @@
 
 #include "nvidia-drm-conftest.h"
 
-#if defined(NV_DRM_ATOMIC_MODESET_AVAILABLE)
+#if defined(NV_DRM_AVAILABLE)
 
 #if defined(NV_DRM_DRMP_H_PRESENT)
 #include <drm/drmP.h>
 #endif
 
-#if defined(NV_DRM_DRM_FRAMEBUFFER_H_PRESENT)
 #include <drm/drm_framebuffer.h>
-#endif
 
-#include "nvidia-drm-gem-nvkms-memory.h"
 #include "nvkms-kapi.h"
 
 struct nv_drm_framebuffer {
     struct NvKmsKapiSurface *pSurface;
 
-    struct nv_drm_gem_object*
-        nv_gem[NVKMS_MAX_PLANES_PER_SURFACE];
-
     struct drm_framebuffer base;
 };
 
@@ -56,11 +50,11 @@ static inline struct nv_drm_framebuffer *to_nv_framebuffer(
     return container_of(fb, struct nv_drm_framebuffer, base);
 }
 
-struct drm_framebuffer *nv_drm_internal_framebuffer_create(
+struct drm_framebuffer *nv_drm_framebuffer_create(
     struct drm_device *dev,
     struct drm_file *file,
-    struct drm_mode_fb_cmd2 *cmd);
+    const struct drm_mode_fb_cmd2 *cmd);
 
-#endif /* NV_DRM_ATOMIC_MODESET_AVAILABLE */
+#endif /* NV_DRM_AVAILABLE */
 
 #endif /* __NVIDIA_DRM_FB_H__ */

kernel-open/nvidia-drm/nvidia-drm-fence.c

@@ -1,5 +1,5 @@
 /*
- * Copyright (c) 2016, NVIDIA CORPORATION. All rights reserved.
+ * Copyright (c) 2016-2025, NVIDIA CORPORATION. All rights reserved.
  *
  * Permission is hereby granted, free of charge, to any person obtaining a
  * copy of this software and associated documentation files (the "Software"),
@@ -34,9 +34,7 @@
 #include "nvidia-drm-fence.h"
 #include "nvidia-dma-resv-helper.h"
 
-#if defined(NV_DRM_FENCE_AVAILABLE)
-
-#include "nvidia-dma-fence-helper.h"
+#include <linux/dma-fence.h>
 
 #define NV_DRM_SEMAPHORE_SURFACE_FENCE_MAX_TIMEOUT_MS 5000
 
@@ -83,42 +81,42 @@ struct nv_drm_prime_fence_context {
 
 struct nv_drm_prime_fence {
     struct list_head list_entry;
-    nv_dma_fence_t base;
+    struct dma_fence base;
     spinlock_t lock;
 };
 
 static inline
-struct nv_drm_prime_fence *to_nv_drm_prime_fence(nv_dma_fence_t *fence)
+struct nv_drm_prime_fence *to_nv_drm_prime_fence(struct dma_fence *fence)
 {
     return container_of(fence, struct nv_drm_prime_fence, base);
 }
 
 static const char*
-nv_drm_gem_fence_op_get_driver_name(nv_dma_fence_t *fence)
+nv_drm_gem_fence_op_get_driver_name(struct dma_fence *fence)
 {
     return "NVIDIA";
 }
 
 static const char*
-nv_drm_gem_prime_fence_op_get_timeline_name(nv_dma_fence_t *fence)
+nv_drm_gem_prime_fence_op_get_timeline_name(struct dma_fence *fence)
 {
     return "nvidia.prime";
 }
 
-static bool nv_drm_gem_prime_fence_op_enable_signaling(nv_dma_fence_t *fence)
+static bool nv_drm_gem_prime_fence_op_enable_signaling(struct dma_fence *fence)
 {
     // DO NOTHING
     return true;
 }
 
-static void nv_drm_gem_prime_fence_op_release(nv_dma_fence_t *fence)
+static void nv_drm_gem_prime_fence_op_release(struct dma_fence *fence)
 {
     struct nv_drm_prime_fence *nv_fence = to_nv_drm_prime_fence(fence);
     nv_drm_free(nv_fence);
 }
 
 static signed long
-nv_drm_gem_prime_fence_op_wait(nv_dma_fence_t *fence,
+nv_drm_gem_prime_fence_op_wait(struct dma_fence *fence,
                                bool intr, signed long timeout)
 {
     /*
@@ -131,12 +129,12 @@ nv_drm_gem_prime_fence_op_wait(nv_dma_fence_t *fence,
      * that it should never get hit during normal operation, but not so long
      * that the system becomes unresponsive.
      */
-    return nv_dma_fence_default_wait(fence, intr,
+    return dma_fence_default_wait(fence, intr,
                               (timeout == MAX_SCHEDULE_TIMEOUT) ?
                                   msecs_to_jiffies(96) : timeout);
 }
 
-static const nv_dma_fence_ops_t nv_drm_gem_prime_fence_ops = {
+static const struct dma_fence_ops nv_drm_gem_prime_fence_ops = {
     .get_driver_name = nv_drm_gem_fence_op_get_driver_name,
     .get_timeline_name = nv_drm_gem_prime_fence_op_get_timeline_name,
     .enable_signaling = nv_drm_gem_prime_fence_op_enable_signaling,
@@ -148,8 +146,8 @@ static inline void
 __nv_drm_prime_fence_signal(struct nv_drm_prime_fence *nv_fence)
 {
     list_del(&nv_fence->list_entry);
-    nv_dma_fence_signal(&nv_fence->base);
-    nv_dma_fence_put(&nv_fence->base);
+    dma_fence_signal(&nv_fence->base);
+    dma_fence_put(&nv_fence->base);
 }
 
 static void nv_drm_gem_prime_force_fence_signal(
@@ -289,18 +287,16 @@ __nv_drm_prime_fence_context_new(
     }
 
     /*
-     * nv_dma_fence_context_alloc() cannot fail, so we do not need
+     * dma_fence_context_alloc() cannot fail, so we do not need
      * to check a return value.
      */
 
-    *nv_prime_fence_context = (struct nv_drm_prime_fence_context) {
-        .base.ops = &nv_drm_prime_fence_context_ops,
-        .base.nv_dev = nv_dev,
-        .base.context = nv_dma_fence_context_alloc(1),
-        .base.fenceSemIndex = p->index,
-        .pSemSurface = pSemSurface,
-        .pLinearAddress = pLinearAddress,
-    };
+    nv_prime_fence_context->base.ops = &nv_drm_prime_fence_context_ops;
+    nv_prime_fence_context->base.nv_dev = nv_dev;
+    nv_prime_fence_context->base.context = dma_fence_context_alloc(1);
+    nv_prime_fence_context->base.fenceSemIndex = p->index;
+    nv_prime_fence_context->pSemSurface = pSemSurface;
+    nv_prime_fence_context->pLinearAddress = pLinearAddress;
 
     INIT_LIST_HEAD(&nv_prime_fence_context->pending);
 
@@ -345,7 +341,7 @@ failed:
     return NULL;
 }
 
-static nv_dma_fence_t *__nv_drm_prime_fence_context_create_fence(
+static struct dma_fence *__nv_drm_prime_fence_context_create_fence(
     struct nv_drm_prime_fence_context *nv_prime_fence_context,
     unsigned int seqno)
 {
@@ -371,12 +367,12 @@ static nv_dma_fence_t *__nv_drm_prime_fence_context_create_fence(
 
     spin_lock_init(&nv_fence->lock);
 
-    nv_dma_fence_init(&nv_fence->base, &nv_drm_gem_prime_fence_ops,
-                      &nv_fence->lock, nv_prime_fence_context->base.context,
-                      seqno);
+    dma_fence_init(&nv_fence->base, &nv_drm_gem_prime_fence_ops,
+                   &nv_fence->lock, nv_prime_fence_context->base.context,
+                   seqno);
 
     /* The context maintains a reference to any pending fences. */
-    nv_dma_fence_get(&nv_fence->base);
+    dma_fence_get(&nv_fence->base);
 
     list_add_tail(&nv_fence->list_entry, &nv_prime_fence_context->pending);
 
@@ -426,12 +422,11 @@ const struct nv_drm_gem_object_funcs nv_fence_context_gem_ops = {
 static inline
 struct nv_drm_fence_context *
 __nv_drm_fence_context_lookup(
-    struct drm_device *dev,
     struct drm_file *filp,
     u32 handle)
 {
     struct nv_drm_gem_object *nv_gem =
-            nv_drm_gem_object_lookup(dev, filp, handle);
+            nv_drm_gem_object_lookup(filp, handle);
 
     if (nv_gem != NULL && nv_gem->ops != &nv_fence_context_gem_ops) {
         nv_drm_gem_object_unreference_unlocked(nv_gem);
@@ -465,10 +460,15 @@ int nv_drm_prime_fence_context_create_ioctl(struct drm_device *dev,
 {
     struct nv_drm_device *nv_dev = to_nv_device(dev);
     struct drm_nvidia_prime_fence_context_create_params *p = data;
-    struct nv_drm_prime_fence_context *nv_prime_fence_context =
-        __nv_drm_prime_fence_context_new(nv_dev, p);
+    struct nv_drm_prime_fence_context *nv_prime_fence_context;
     int err;
 
+    if (nv_dev->pDevice == NULL) {
+        return -EOPNOTSUPP;
+    }
+
+    nv_prime_fence_context = __nv_drm_prime_fence_context_new(nv_dev, p);
+
     if (!nv_prime_fence_context) {
         goto done;
     }
@@ -488,7 +488,7 @@ done:
 }
 
 static int __nv_drm_gem_attach_fence(struct nv_drm_gem_object *nv_gem,
-                                     nv_dma_fence_t *fence,
+                                     struct dma_fence *fence,
                                      bool shared)
 {
     nv_dma_resv_t *resv = nv_drm_gem_res_obj(nv_gem);
@@ -521,14 +521,19 @@ int nv_drm_gem_prime_fence_attach_ioctl(struct drm_device *dev,
 
     struct nv_drm_gem_object *nv_gem;
     struct nv_drm_fence_context *nv_fence_context;
-    nv_dma_fence_t *fence;
+    struct dma_fence *fence;
+
+    if (nv_dev->pDevice == NULL) {
+        ret = -EOPNOTSUPP;
+        goto done;
+    }
 
     if (p->__pad != 0) {
         NV_DRM_DEV_LOG_ERR(nv_dev, "Padding fields must be zeroed");
         goto done;
     }
 
-    nv_gem = nv_drm_gem_object_lookup(nv_dev->dev, filep, p->handle);
+    nv_gem = nv_drm_gem_object_lookup(filep, p->handle);
 
     if (!nv_gem) {
         NV_DRM_DEV_LOG_ERR(
@@ -540,7 +545,6 @@ int nv_drm_gem_prime_fence_attach_ioctl(struct drm_device *dev,
     }
 
     if((nv_fence_context = __nv_drm_fence_context_lookup(
-                nv_dev->dev,
                 filep,
                 p->fence_context_handle)) == NULL) {
 
@@ -579,7 +583,7 @@ int nv_drm_gem_prime_fence_attach_ioctl(struct drm_device *dev,
 
     ret = __nv_drm_gem_attach_fence(nv_gem, fence, true /* exclusive */);
 
-    nv_dma_fence_put(fence);
+    dma_fence_put(fence);
 
 fence_context_create_fence_failed:
     nv_drm_gem_object_unreference_unlocked(&nv_fence_context->base);
@@ -592,7 +596,7 @@ done:
 }
 
 struct nv_drm_semsurf_fence {
-    nv_dma_fence_t base;
+    struct dma_fence base;
     spinlock_t lock;
 
     /*
@@ -620,7 +624,7 @@ struct nv_drm_semsurf_fence_callback {
 };
 
 struct nv_drm_sync_fd_wait_data {
-    nv_dma_fence_cb_t dma_fence_cb;
+    struct dma_fence_cb dma_fence_cb;
     struct nv_drm_semsurf_fence_ctx *ctx;
     nv_drm_work work; /* Deferred second half of fence wait callback */
 
@@ -751,15 +755,15 @@ __nv_drm_semsurf_force_complete_pending(struct nv_drm_semsurf_fence_ctx *ctx)
             &ctx->pending_fences,
             typeof(*nv_fence),
             pending_node);
-        nv_dma_fence_t *fence = &nv_fence->base;
+        struct dma_fence *fence = &nv_fence->base;
 
         list_del(&nv_fence->pending_node);
 
-        nv_dma_fence_set_error(fence, -ETIMEDOUT);
-        nv_dma_fence_signal(fence);
+        dma_fence_set_error(fence, -ETIMEDOUT);
+        dma_fence_signal(fence);
 
         /* Remove the pending list's reference */
-        nv_dma_fence_put(fence);
+        dma_fence_put(fence);
     }
 
     /*
@@ -816,7 +820,7 @@ __nv_drm_semsurf_ctx_process_completed(struct nv_drm_semsurf_fence_ctx *ctx,
     struct list_head finished;
     struct list_head timed_out;
     struct nv_drm_semsurf_fence *nv_fence;
-    nv_dma_fence_t *fence;
+    struct dma_fence *fence;
     NvU64 currentSeqno = __nv_drm_get_semsurf_ctx_seqno(ctx);
     NvU64 fenceSeqno = 0;
     unsigned long flags;
@@ -880,8 +884,8 @@ __nv_drm_semsurf_ctx_process_completed(struct nv_drm_semsurf_fence_ctx *ctx,
         nv_fence = list_first_entry(&finished, typeof(*nv_fence), pending_node);
         list_del_init(&nv_fence->pending_node);
         fence = &nv_fence->base;
-        nv_dma_fence_signal(fence);
-        nv_dma_fence_put(fence); /* Drops the pending list's reference */
+        dma_fence_signal(fence);
+        dma_fence_put(fence); /* Drops the pending list's reference */
     }
 
     while (!list_empty(&timed_out)) {
@@ -889,9 +893,9 @@ __nv_drm_semsurf_ctx_process_completed(struct nv_drm_semsurf_fence_ctx *ctx,
                                     pending_node);
         list_del_init(&nv_fence->pending_node);
         fence = &nv_fence->base;
-        nv_dma_fence_set_error(fence, -ETIMEDOUT);
-        nv_dma_fence_signal(fence);
-        nv_dma_fence_put(fence); /* Drops the pending list's reference */
+        dma_fence_set_error(fence, -ETIMEDOUT);
+        dma_fence_signal(fence);
+        dma_fence_put(fence); /* Drops the pending list's reference */
     }
 }
 
@@ -1124,7 +1128,7 @@ static void __nv_drm_semsurf_fence_ctx_destroy(
      */
     nv_drm_workthread_shutdown(&ctx->worker);
 
-    nv_drm_del_timer_sync(&ctx->timer);
+    nv_timer_delete_sync(&ctx->timer.kernel_timer);
 
     /*
      * The semaphore surface could still be sending callbacks, so it is still
@@ -1257,22 +1261,20 @@ __nv_drm_semsurf_fence_ctx_new(
     }
 
     /*
-     * nv_dma_fence_context_alloc() cannot fail, so we do not need
+     * dma_fence_context_alloc() cannot fail, so we do not need
      * to check a return value.
      */
 
-    *ctx = (struct nv_drm_semsurf_fence_ctx) {
-        .base.ops = &nv_drm_semsurf_fence_ctx_ops,
-        .base.nv_dev = nv_dev,
-        .base.context = nv_dma_fence_context_alloc(1),
-        .base.fenceSemIndex = p->index,
-        .pSemSurface = pSemSurface,
-        .pSemMapping.pVoid = semMapping,
-        .pMaxSubmittedMapping = (volatile NvU64 *)maxSubmittedMapping,
-        .callback.local = NULL,
-        .callback.nvKms = NULL,
-        .current_wait_value = 0,
-    };
+    ctx->base.ops = &nv_drm_semsurf_fence_ctx_ops;
+    ctx->base.nv_dev = nv_dev;
+    ctx->base.context = dma_fence_context_alloc(1);
+    ctx->base.fenceSemIndex = p->index;
+    ctx->pSemSurface = pSemSurface;
+    ctx->pSemMapping.pVoid = semMapping;
+    ctx->pMaxSubmittedMapping = (volatile NvU64 *)maxSubmittedMapping;
+    ctx->callback.local = NULL;
+    ctx->callback.nvKms = NULL;
+    ctx->current_wait_value = 0;
 
     spin_lock_init(&ctx->lock);
     INIT_LIST_HEAD(&ctx->pending_fences);
@@ -1312,6 +1314,10 @@ int nv_drm_semsurf_fence_ctx_create_ioctl(struct drm_device *dev,
     struct nv_drm_semsurf_fence_ctx *ctx;
     int err;
 
+    if (nv_dev->pDevice == NULL) {
+        return -EOPNOTSUPP;
+    }
+
     if (p->__pad != 0) {
         NV_DRM_DEV_LOG_ERR(nv_dev, "Padding fields must be zeroed");
         return -EINVAL;
@@ -1333,26 +1339,26 @@ int nv_drm_semsurf_fence_ctx_create_ioctl(struct drm_device *dev,
 }
 
 static inline struct nv_drm_semsurf_fence*
-to_nv_drm_semsurf_fence(nv_dma_fence_t *fence)
+to_nv_drm_semsurf_fence(struct dma_fence *fence)
 {
     return container_of(fence, struct nv_drm_semsurf_fence, base);
 }
 
 static const char*
-__nv_drm_semsurf_fence_op_get_timeline_name(nv_dma_fence_t *fence)
+__nv_drm_semsurf_fence_op_get_timeline_name(struct dma_fence *fence)
 {
     return "nvidia.semaphore_surface";
 }
 
 static bool
-__nv_drm_semsurf_fence_op_enable_signaling(nv_dma_fence_t *fence)
+__nv_drm_semsurf_fence_op_enable_signaling(struct dma_fence *fence)
 {
     // DO NOTHING - Could defer RM callback registration until this point
     return true;
 }
 
 static void
-__nv_drm_semsurf_fence_op_release(nv_dma_fence_t *fence)
+__nv_drm_semsurf_fence_op_release(struct dma_fence *fence)
 {
     struct nv_drm_semsurf_fence *nv_fence =
         to_nv_drm_semsurf_fence(fence);
@@ -1360,12 +1366,12 @@ __nv_drm_semsurf_fence_op_release(nv_dma_fence_t *fence)
     nv_drm_free(nv_fence);
 }
 
-static const nv_dma_fence_ops_t nv_drm_semsurf_fence_ops = {
+static const struct dma_fence_ops nv_drm_semsurf_fence_ops = {
     .get_driver_name = nv_drm_gem_fence_op_get_driver_name,
     .get_timeline_name = __nv_drm_semsurf_fence_op_get_timeline_name,
     .enable_signaling = __nv_drm_semsurf_fence_op_enable_signaling,
     .release = __nv_drm_semsurf_fence_op_release,
-    .wait = nv_dma_fence_default_wait,
+    .wait = dma_fence_default_wait,
 #if defined(NV_DMA_FENCE_OPS_HAS_USE_64BIT_SEQNO)
     .use_64bit_seqno = true,
 #endif
@@ -1391,7 +1397,7 @@ __nv_drm_semsurf_ctx_add_pending(struct nv_drm_semsurf_fence_ctx *ctx,
     }
 
     /* Add a reference to the fence for the list */
-    nv_dma_fence_get(&nv_fence->base);
+    dma_fence_get(&nv_fence->base);
     INIT_LIST_HEAD(&nv_fence->pending_node);
 
     nv_fence->timeout = nv_drm_timeout_from_ms(timeoutMS);
@@ -1424,14 +1430,14 @@ __nv_drm_semsurf_ctx_add_pending(struct nv_drm_semsurf_fence_ctx *ctx,
     __nv_drm_semsurf_ctx_reg_callbacks(ctx);
 }
 
-static nv_dma_fence_t *__nv_drm_semsurf_fence_ctx_create_fence(
+static struct dma_fence *__nv_drm_semsurf_fence_ctx_create_fence(
     struct nv_drm_device *nv_dev,
     struct nv_drm_semsurf_fence_ctx *ctx,
     NvU64 wait_value,
     NvU64 timeout_value_ms)
 {
     struct nv_drm_semsurf_fence *nv_fence;
-    nv_dma_fence_t *fence;
+    struct dma_fence *fence;
     int ret = 0;
 
     if (timeout_value_ms == 0 ||
@@ -1451,9 +1457,9 @@ static nv_dma_fence_t *__nv_drm_semsurf_fence_ctx_create_fence(
 #endif
 
     /* Initializes the fence with one reference (for the caller) */
-    nv_dma_fence_init(fence, &nv_drm_semsurf_fence_ops,
-                      &nv_fence->lock,
-                      ctx->base.context, wait_value);
+    dma_fence_init(fence, &nv_drm_semsurf_fence_ops,
+                   &nv_fence->lock,
+                   ctx->base.context, wait_value);
 
     __nv_drm_semsurf_ctx_add_pending(ctx, nv_fence, timeout_value_ms);
 
@@ -1469,17 +1475,21 @@ int nv_drm_semsurf_fence_create_ioctl(struct drm_device *dev,
     struct nv_drm_device *nv_dev = to_nv_device(dev);
     struct drm_nvidia_semsurf_fence_create_params *p = data;
     struct nv_drm_fence_context *nv_fence_context;
-    nv_dma_fence_t *fence;
+    struct dma_fence *fence;
     int ret = -EINVAL;
     int fd;
 
+    if (nv_dev->pDevice == NULL) {
+        ret = -EOPNOTSUPP;
+        goto done;
+    }
+
     if (p->__pad != 0) {
         NV_DRM_DEV_LOG_ERR(nv_dev, "Padding fields must be zeroed");
         goto done;
     }
 
     if ((nv_fence_context = __nv_drm_fence_context_lookup(
-                                nv_dev->dev,
                                 filep,
                                 p->fence_context_handle)) == NULL) {
         NV_DRM_DEV_LOG_ERR(
@@ -1535,7 +1545,7 @@ fence_context_create_sync_failed:
      * FD will still hold a reference, and the pending list (if the fence hasn't
      * already been signaled) will also retain a reference.
      */
-    nv_dma_fence_put(fence);
+    dma_fence_put(fence);
 
 fence_context_create_fence_failed:
     nv_drm_gem_object_unreference_unlocked(&nv_fence_context->base);
@@ -1593,8 +1603,8 @@ __nv_drm_semsurf_wait_fence_work_cb
 static void
 __nv_drm_semsurf_wait_fence_cb
 (
-    nv_dma_fence_t *fence,
-    nv_dma_fence_cb_t *cb
+    struct dma_fence *fence,
+    struct dma_fence_cb *cb
 )
 {
     struct nv_drm_sync_fd_wait_data *wait_data =
@@ -1619,7 +1629,7 @@ __nv_drm_semsurf_wait_fence_cb
     }
 
     /* Don't need to reference the fence anymore, just the fence context. */
-    nv_dma_fence_put(fence);
+    dma_fence_put(fence);
 }
 
 int nv_drm_semsurf_fence_wait_ioctl(struct drm_device *dev,
@@ -1631,10 +1641,14 @@ int nv_drm_semsurf_fence_wait_ioctl(struct drm_device *dev,
     struct nv_drm_fence_context *nv_fence_context;
     struct nv_drm_semsurf_fence_ctx *ctx;
     struct nv_drm_sync_fd_wait_data *wait_data = NULL;
-    nv_dma_fence_t *fence;
+    struct dma_fence *fence;
     unsigned long flags;
     int ret = -EINVAL;
 
+    if (nv_dev->pDevice == NULL) {
+        return -EOPNOTSUPP;
+    }
+
     if (p->pre_wait_value >= p->post_wait_value) {
         NV_DRM_DEV_LOG_ERR(
             nv_dev,
@@ -1644,7 +1658,6 @@ int nv_drm_semsurf_fence_wait_ioctl(struct drm_device *dev,
     }
 
     if ((nv_fence_context = __nv_drm_fence_context_lookup(
-                                nv_dev->dev,
                                 filep,
                                 p->fence_context_handle)) == NULL) {
         NV_DRM_DEV_LOG_ERR(
@@ -1697,9 +1710,9 @@ int nv_drm_semsurf_fence_wait_ioctl(struct drm_device *dev,
     list_add(&wait_data->pending_node, &ctx->pending_waits);
     spin_unlock_irqrestore(&ctx->lock, flags);
 
-    ret = nv_dma_fence_add_callback(fence,
-                                    &wait_data->dma_fence_cb,
-                                    __nv_drm_semsurf_wait_fence_cb);
+    ret = dma_fence_add_callback(fence,
+                                 &wait_data->dma_fence_cb,
+                                 __nv_drm_semsurf_wait_fence_cb);
 
     if (ret) {
        if (ret == -ENOENT) {
@@ -1711,7 +1724,7 @@ int nv_drm_semsurf_fence_wait_ioctl(struct drm_device *dev,
        }
 
        /* Execute second half of wait immediately, avoiding the worker thread */
-       nv_dma_fence_put(fence);
+       dma_fence_put(fence);
         __nv_drm_semsurf_wait_fence_work_cb(wait_data);
     }
 
@@ -1740,10 +1753,15 @@ int nv_drm_semsurf_fence_attach_ioctl(struct drm_device *dev,
     struct drm_nvidia_semsurf_fence_attach_params *p = data;
     struct nv_drm_gem_object *nv_gem = NULL;
     struct nv_drm_fence_context *nv_fence_context = NULL;
-    nv_dma_fence_t *fence;
+    struct dma_fence *fence;
     int ret = -EINVAL;
 
-    nv_gem = nv_drm_gem_object_lookup(nv_dev->dev, filep, p->handle);
+    if (nv_dev->pDevice == NULL) {
+        ret = -EOPNOTSUPP;
+        goto done;
+    }
+
+    nv_gem = nv_drm_gem_object_lookup(filep, p->handle);
 
     if (!nv_gem) {
         NV_DRM_DEV_LOG_ERR(
@@ -1755,7 +1773,6 @@ int nv_drm_semsurf_fence_attach_ioctl(struct drm_device *dev,
     }
 
     nv_fence_context = __nv_drm_fence_context_lookup(
-        nv_dev->dev,
         filep,
         p->fence_context_handle);
 
@@ -1795,7 +1812,7 @@ int nv_drm_semsurf_fence_attach_ioctl(struct drm_device *dev,
 
     ret = __nv_drm_gem_attach_fence(nv_gem, fence, p->shared);
 
-    nv_dma_fence_put(fence);
+    dma_fence_put(fence);
 
 done:
     if (nv_fence_context) {
@@ -1809,6 +1826,4 @@ done:
     return ret;
 }
 
-#endif /* NV_DRM_FENCE_AVAILABLE */
-
 #endif /* NV_DRM_AVAILABLE */

kernel-open/nvidia-drm/nvidia-drm-fence.h

@@ -30,8 +30,6 @@
 struct drm_file;
 struct drm_device;
 
-#if defined(NV_DRM_FENCE_AVAILABLE)
-
 int nv_drm_fence_supported_ioctl(struct drm_device *dev,
                                  void *data, struct drm_file *filep);
 
@@ -57,8 +55,6 @@ int nv_drm_semsurf_fence_attach_ioctl(struct drm_device *dev,
                                       void *data,
                                       struct drm_file *filep);
 
-#endif /* NV_DRM_FENCE_AVAILABLE */
-
 #endif /* NV_DRM_AVAILABLE */
 
 #endif /* __NVIDIA_DRM_PRIME_FENCE_H__ */

kernel-open/nvidia-drm/nvidia-drm-format.c

@@ -20,9 +20,9 @@
  * DEALINGS IN THE SOFTWARE.
  */
 
-#include "nvidia-drm-conftest.h" /* NV_DRM_ATOMIC_MODESET_AVAILABLE */
+#include "nvidia-drm-conftest.h" /* NV_DRM_AVAILABLE */
 
-#if defined(NV_DRM_ATOMIC_MODESET_AVAILABLE)
+#if defined(NV_DRM_AVAILABLE)
 
 #if defined(NV_DRM_DRMP_H_PRESENT)
 #include <drm/drmP.h>
@@ -166,4 +166,37 @@ uint32_t *nv_drm_format_array_alloc(
     return array;
 }
 
+bool nv_drm_format_is_yuv(u32 format)
+{
+#if defined(NV_DRM_FORMAT_INFO_HAS_IS_YUV)
+    const struct drm_format_info *format_info = drm_format_info(format);
+    return (format_info != NULL) && format_info->is_yuv;
+#else
+    switch (format) {
+        case DRM_FORMAT_YUYV:
+        case DRM_FORMAT_UYVY:
+
+        case DRM_FORMAT_NV24:
+        case DRM_FORMAT_NV42:
+        case DRM_FORMAT_NV16:
+        case DRM_FORMAT_NV61:
+        case DRM_FORMAT_NV12:
+        case DRM_FORMAT_NV21:
+
+#if defined(DRM_FORMAT_P210)
+        case DRM_FORMAT_P210:
+#endif
+#if defined(DRM_FORMAT_P010)
+        case DRM_FORMAT_P010:
+#endif
+#if defined(DRM_FORMAT_P012)
+        case DRM_FORMAT_P012:
+#endif
+            return true;
+        default:
+            return false;
+    }
+#endif
+}
+
 #endif

kernel-open/nvidia-drm/nvidia-drm-format.h

@@ -25,7 +25,7 @@
 
 #include "nvidia-drm-conftest.h"
 
-#if defined(NV_DRM_ATOMIC_MODESET_AVAILABLE)
+#if defined(NV_DRM_AVAILABLE)
 
 #include <drm/drm_fourcc.h>
 
@@ -38,6 +38,8 @@ uint32_t *nv_drm_format_array_alloc(
     unsigned int *count,
     const long unsigned int nvkms_format_mask);
 
-#endif /* NV_DRM_ATOMIC_MODESET_AVAILABLE */
+bool nv_drm_format_is_yuv(u32 format);
+
+#endif /* NV_DRM_AVAILABLE */
 
 #endif /* __NVIDIA_DRM_FORMAT_H__ */

kernel-open/nvidia-drm/nvidia-drm-gem-dma-buf.c

@@ -24,17 +24,13 @@
 
 #if defined(NV_DRM_AVAILABLE)
 
-#if defined(NV_DRM_DRM_PRIME_H_PRESENT)
 #include <drm/drm_prime.h>
-#endif
 
 #if defined(NV_DRM_DRMP_H_PRESENT)
 #include <drm/drmP.h>
 #endif
 
-#if defined(NV_DRM_DRM_DRV_H_PRESENT)
 #include <drm/drm_drv.h>
-#endif
 
 #include "nvidia-drm-gem-dma-buf.h"
 #include "nvidia-drm-ioctl.h"
@@ -47,12 +43,10 @@ void __nv_drm_gem_dma_buf_free(struct nv_drm_gem_object *nv_gem)
     struct nv_drm_device *nv_dev = nv_gem->nv_dev;
     struct nv_drm_gem_dma_buf *nv_dma_buf = to_nv_dma_buf(nv_gem);
 
-#if defined(NV_DRM_ATOMIC_MODESET_AVAILABLE)
     if (nv_dma_buf->base.pMemory) {
         /* Free NvKmsKapiMemory handle associated with this gem object */
         nvKms->freeMemory(nv_dev->pDevice, nv_dma_buf->base.pMemory);
     }
-#endif
 
     drm_prime_gem_destroy(&nv_gem->base, nv_dma_buf->sgt);
 
@@ -157,13 +151,11 @@ nv_drm_gem_prime_import_sg_table(struct drm_device *dev,
     BUG_ON(dma_buf->size % PAGE_SIZE);
 
     pMemory = NULL;
-#if defined(NV_DRM_ATOMIC_MODESET_AVAILABLE)
     if (drm_core_check_feature(dev, DRIVER_MODESET)) {
         pMemory = nvKms->getSystemMemoryHandleFromDmaBuf(nv_dev->pDevice,
                                                   (NvP64)(NvUPtr)dma_buf,
                                                   dma_buf->size - 1);
     }
-#endif
 
     nv_drm_gem_object_init(nv_dev, &nv_dma_buf->base,
                            &__nv_gem_dma_buf_ops, dma_buf->size, pMemory);
@@ -194,7 +186,7 @@ int nv_drm_gem_export_dmabuf_memory_ioctl(struct drm_device *dev,
     }
 
     if ((nv_dma_buf = nv_drm_gem_object_dma_buf_lookup(
-             dev, filep, p->handle)) == NULL) {
+             filep, p->handle)) == NULL) {
         ret = -EINVAL;
         NV_DRM_DEV_LOG_ERR(
             nv_dev,
@@ -203,7 +195,6 @@ int nv_drm_gem_export_dmabuf_memory_ioctl(struct drm_device *dev,
         goto done;
     }
 
-#if defined(NV_DRM_ATOMIC_MODESET_AVAILABLE)
     if (drm_core_check_feature(dev, DRIVER_MODESET)) {
         if (!nv_dma_buf->base.pMemory) {
             /*
@@ -218,7 +209,6 @@ int nv_drm_gem_export_dmabuf_memory_ioctl(struct drm_device *dev,
                              nv_dma_buf->base.base.size - 1);
         }
     }
-#endif
 
     if (!nv_dma_buf->base.pMemory && !pTmpMemory) {
         ret = -ENOMEM;

kernel-open/nvidia-drm/nvidia-drm-gem-dma-buf.h

@@ -48,12 +48,11 @@ static inline struct nv_drm_gem_dma_buf *to_nv_dma_buf(
 
 static inline
 struct nv_drm_gem_dma_buf *nv_drm_gem_object_dma_buf_lookup(
-    struct drm_device *dev,
     struct drm_file *filp,
     u32 handle)
 {
     struct nv_drm_gem_object *nv_gem =
-            nv_drm_gem_object_lookup(dev, filp, handle);
+            nv_drm_gem_object_lookup(filp, handle);
 
     if (nv_gem != NULL && nv_gem->ops != &__nv_gem_dma_buf_ops) {
         nv_drm_gem_object_unreference_unlocked(nv_gem);

kernel-open/nvidia-drm/nvidia-drm-gem-nvkms-memory.c

@@ -22,27 +22,20 @@
 
 #include "nvidia-drm-conftest.h"
 
-#if defined(NV_DRM_ATOMIC_MODESET_AVAILABLE)
+#if defined(NV_DRM_AVAILABLE)
 
 #include "nvidia-drm-gem-nvkms-memory.h"
 #include "nvidia-drm-helper.h"
 #include "nvidia-drm-ioctl.h"
 
-#if defined(NV_DRM_DRM_DRV_H_PRESENT)
 #include <drm/drm_drv.h>
-#endif
-
-#if defined(NV_DRM_DRM_PRIME_H_PRESENT)
 #include <drm/drm_prime.h>
-#endif
 
 #include <linux/io.h>
 #if defined(NV_BSD)
 #include <vm/vm_pageout.h>
 #endif
 
-#include "nv-mm.h"
-
 static void __nv_drm_gem_nvkms_memory_free(struct nv_drm_gem_object *nv_gem)
 {
     struct nv_drm_device *nv_dev = nv_gem->nv_dev;
@@ -71,9 +64,20 @@ static void __nv_drm_gem_nvkms_memory_free(struct nv_drm_gem_object *nv_gem)
     nv_drm_free(nv_nvkms_memory);
 }
 
+static int __nv_drm_gem_nvkms_map(
+    struct nv_drm_gem_nvkms_memory *nv_nvkms_memory);
+
 static int __nv_drm_gem_nvkms_mmap(struct nv_drm_gem_object *nv_gem,
                                    struct vm_area_struct *vma)
 {
+    struct nv_drm_gem_nvkms_memory *nv_nvkms_memory =
+        to_nv_nvkms_memory(nv_gem);
+
+    int ret = __nv_drm_gem_nvkms_map(nv_nvkms_memory);
+    if (ret) {
+       return ret;
+    }
+
     return drm_gem_mmap_obj(&nv_gem->base,
                 drm_vma_node_size(&nv_gem->base.vma_node) << PAGE_SHIFT, vma);
 }
@@ -83,10 +87,9 @@ static vm_fault_t __nv_drm_gem_nvkms_handle_vma_fault(
     struct vm_area_struct *vma,
     struct vm_fault *vmf)
 {
-#if defined(NV_DRM_ATOMIC_MODESET_AVAILABLE)
     struct nv_drm_gem_nvkms_memory *nv_nvkms_memory =
         to_nv_nvkms_memory(nv_gem);
-    unsigned long address = nv_page_fault_va(vmf);
+    unsigned long address = vmf->address;
     struct drm_gem_object *gem = vma->vm_private_data;
     unsigned long page_offset, pfn;
     vm_fault_t ret;
@@ -135,8 +138,6 @@ static vm_fault_t __nv_drm_gem_nvkms_handle_vma_fault(
     }
 #endif /* defined(NV_VMF_INSERT_PFN_PRESENT) */
     return ret;
-#endif /* defined(NV_DRM_ATOMIC_MODESET_AVAILABLE) */
-    return VM_FAULT_SIGBUS;
 }
 
 static struct drm_gem_object *__nv_drm_gem_nvkms_prime_dup(
@@ -146,11 +147,18 @@ static struct drm_gem_object *__nv_drm_gem_nvkms_prime_dup(
 static int __nv_drm_gem_nvkms_map(
     struct nv_drm_gem_nvkms_memory *nv_nvkms_memory)
 {
+    int ret = 0;
     struct nv_drm_device *nv_dev = nv_nvkms_memory->base.nv_dev;
     struct NvKmsKapiMemory *pMemory = nv_nvkms_memory->base.pMemory;
 
-    if (!nv_dev->hasVideoMemory) {
-        return 0;
+    mutex_lock(&nv_nvkms_memory->map_lock);
+
+    if (nv_nvkms_memory->physically_mapped) {
+        goto done;
+    }
+
+    if (!nvKms->isVidmem(pMemory)) {
+        goto done;
     }
 
     if (!nvKms->mapMemory(nv_dev->pDevice,
@@ -161,7 +169,8 @@ static int __nv_drm_gem_nvkms_map(
             nv_dev,
             "Failed to map NvKmsKapiMemory 0x%p",
             pMemory);
-        return -ENOMEM;
+        ret = -ENOMEM;
+        goto done;
     }
 
     nv_nvkms_memory->pWriteCombinedIORemapAddress = ioremap_wc(
@@ -177,7 +186,9 @@ static int __nv_drm_gem_nvkms_map(
 
     nv_nvkms_memory->physically_mapped = true;
 
-    return 0;
+done:
+    mutex_unlock(&nv_nvkms_memory->map_lock);
+    return ret;
 }
 
 static void *__nv_drm_gem_nvkms_prime_vmap(
@@ -186,14 +197,40 @@ static void *__nv_drm_gem_nvkms_prime_vmap(
     struct nv_drm_gem_nvkms_memory *nv_nvkms_memory =
         to_nv_nvkms_memory(nv_gem);
 
-    if (!nv_nvkms_memory->physically_mapped) {
-        int ret = __nv_drm_gem_nvkms_map(nv_nvkms_memory);
-        if (ret) {
-           return ERR_PTR(ret);
-        }
+    int ret = __nv_drm_gem_nvkms_map(nv_nvkms_memory);
+    if (ret) {
+       return ERR_PTR(ret);
     }
 
-    return nv_nvkms_memory->pWriteCombinedIORemapAddress;
+    if (nv_nvkms_memory->physically_mapped) {
+        return nv_nvkms_memory->pWriteCombinedIORemapAddress;
+    }
+
+    /*
+     * If this buffer isn't physically mapped, it might be backed by struct
+     * pages. Use vmap in that case. Do a noncached mapping for system memory
+     * as display is non io-coherent device in case of Tegra.
+     */
+    if (nv_nvkms_memory->pages_count > 0) {
+         return nv_drm_vmap(nv_nvkms_memory->pages,
+                            nv_nvkms_memory->pages_count,
+                            false);
+    }
+
+    return ERR_PTR(-ENOMEM);
+}
+
+static void __nv_drm_gem_nvkms_prime_vunmap(
+    struct nv_drm_gem_object *nv_gem,
+    void *address)
+{
+    struct nv_drm_gem_nvkms_memory *nv_nvkms_memory =
+        to_nv_nvkms_memory(nv_gem);
+
+    if (!nv_nvkms_memory->physically_mapped &&
+        nv_nvkms_memory->pages_count > 0) {
+        nv_drm_vunmap(address);
+    }
 }
 
 static int __nv_drm_gem_map_nvkms_memory_offset(
@@ -201,17 +238,7 @@ static int __nv_drm_gem_map_nvkms_memory_offset(
     struct nv_drm_gem_object *nv_gem,
     uint64_t *offset)
 {
-    struct nv_drm_gem_nvkms_memory *nv_nvkms_memory =
-        to_nv_nvkms_memory(nv_gem);
-
-    if (!nv_nvkms_memory->physically_mapped) {
-        int ret = __nv_drm_gem_nvkms_map(nv_nvkms_memory);
-        if (ret) {
-           return ret;
-        }
-    }
-
-    return nv_drm_gem_create_mmap_offset(&nv_nvkms_memory->base, offset);
+    return nv_drm_gem_create_mmap_offset(nv_gem, offset);
 }
 
 static struct sg_table *__nv_drm_gem_nvkms_memory_prime_get_sg_table(
@@ -223,7 +250,7 @@ static struct sg_table *__nv_drm_gem_nvkms_memory_prime_get_sg_table(
     struct sg_table *sg_table;
 
     if (nv_nvkms_memory->pages_count == 0) {
-        NV_DRM_DEV_LOG_ERR(
+        NV_DRM_DEV_DEBUG_DRIVER(
                 nv_dev,
                 "Cannot create sg_table for NvKmsKapiMemory 0x%p",
                 nv_gem->pMemory);
@@ -241,6 +268,7 @@ const struct nv_drm_gem_object_funcs nv_gem_nvkms_memory_ops = {
     .free = __nv_drm_gem_nvkms_memory_free,
     .prime_dup = __nv_drm_gem_nvkms_prime_dup,
     .prime_vmap = __nv_drm_gem_nvkms_prime_vmap,
+    .prime_vunmap = __nv_drm_gem_nvkms_prime_vunmap,
     .mmap = __nv_drm_gem_nvkms_mmap,
     .handle_vma_fault = __nv_drm_gem_nvkms_handle_vma_fault,
     .create_mmap_offset = __nv_drm_gem_map_nvkms_memory_offset,
@@ -265,16 +293,17 @@ static int __nv_drm_nvkms_gem_obj_init(
         return -EINVAL;
     }
 
+    mutex_init(&nv_nvkms_memory->map_lock);
     nv_nvkms_memory->pPhysicalAddress = NULL;
     nv_nvkms_memory->pWriteCombinedIORemapAddress = NULL;
     nv_nvkms_memory->physically_mapped = false;
 
-    if (!nvKms->getMemoryPages(nv_dev->pDevice,
+    if (!nvKms->isVidmem(pMemory) &&
+        !nvKms->getMemoryPages(nv_dev->pDevice,
                                pMemory,
                                &pages,
-                               &numPages) &&
-        !nv_dev->hasVideoMemory) {
-        /* GetMemoryPages may fail for vidmem allocations,
+                               &numPages)) {
+        /* GetMemoryPages will fail for vidmem allocations,
          * but it should not fail for sysmem allocations. */
         NV_DRM_DEV_LOG_ERR(nv_dev,
             "Failed to get memory pages for NvKmsKapiMemory 0x%p",
@@ -301,6 +330,7 @@ int nv_drm_dumb_create(
     struct nv_drm_gem_nvkms_memory *nv_nvkms_memory;
     uint8_t compressible = 0;
     struct NvKmsKapiMemory *pMemory;
+    struct NvKmsKapiAllocateMemoryParams allocParams = { };
     int ret = 0;
 
     args->pitch = roundup(args->width * ((args->bpp + 7) >> 3),
@@ -318,20 +348,14 @@ int nv_drm_dumb_create(
         goto fail;
     }
 
-    if (nv_dev->hasVideoMemory) {
-        pMemory = nvKms->allocateVideoMemory(nv_dev->pDevice,
-                                             NvKmsSurfaceMemoryLayoutPitch,
-                                             NVKMS_KAPI_ALLOCATION_TYPE_SCANOUT,
-                                             args->size,
-                                             &compressible);
-    } else {
-        pMemory = nvKms->allocateSystemMemory(nv_dev->pDevice,
-                                              NvKmsSurfaceMemoryLayoutPitch,
-                                              NVKMS_KAPI_ALLOCATION_TYPE_SCANOUT,
-                                              args->size,
-                                              &compressible);
-    }
+    allocParams.layout = NvKmsSurfaceMemoryLayoutPitch;
+    allocParams.type = NVKMS_KAPI_ALLOCATION_TYPE_SCANOUT;
+    allocParams.size = args->size;
+    allocParams.noDisplayCaching = true;
+    allocParams.useVideoMemory = nv_dev->hasVideoMemory;
+    allocParams.compressible = &compressible;
 
+    pMemory = nvKms->allocateMemory(nv_dev->pDevice, &allocParams);
     if (pMemory == NULL) {
         ret = -ENOMEM;
         NV_DRM_DEV_LOG_ERR(
@@ -380,7 +404,7 @@ int nv_drm_gem_import_nvkms_memory_ioctl(struct drm_device *dev,
     int ret;
 
     if (!drm_core_check_feature(dev, DRIVER_MODESET)) {
-        ret = -EINVAL;
+        ret = -EOPNOTSUPP;
         goto failed;
     }
 
@@ -430,7 +454,7 @@ int nv_drm_gem_export_nvkms_memory_ioctl(struct drm_device *dev,
     int ret = 0;
 
     if (!drm_core_check_feature(dev, DRIVER_MODESET)) {
-        ret = -EINVAL;
+        ret = -EOPNOTSUPP;
         goto done;
     }
 
@@ -441,7 +465,6 @@ int nv_drm_gem_export_nvkms_memory_ioctl(struct drm_device *dev,
     }
 
     if ((nv_nvkms_memory = nv_drm_gem_object_nvkms_memory_lookup(
-                    dev,
                     filep,
                     p->handle)) == NULL) {
         ret = -EINVAL;
@@ -478,12 +501,11 @@ int nv_drm_gem_alloc_nvkms_memory_ioctl(struct drm_device *dev,
     struct drm_nvidia_gem_alloc_nvkms_memory_params *p = data;
     struct nv_drm_gem_nvkms_memory *nv_nvkms_memory = NULL;
     struct NvKmsKapiMemory *pMemory;
-    enum NvKmsSurfaceMemoryLayout layout;
-    enum NvKmsKapiAllocationType type;
+    struct NvKmsKapiAllocateMemoryParams allocParams = { };
     int ret = 0;
 
     if (!drm_core_check_feature(dev, DRIVER_MODESET)) {
-        ret = -EINVAL;
+        ret = -EOPNOTSUPP;
         goto failed;
     }
 
@@ -499,25 +521,15 @@ int nv_drm_gem_alloc_nvkms_memory_ioctl(struct drm_device *dev,
         goto failed;
     }
 
-    layout = p->block_linear ?
+    allocParams.layout = p->block_linear ?
         NvKmsSurfaceMemoryLayoutBlockLinear : NvKmsSurfaceMemoryLayoutPitch;
-    type = (p->flags & NV_GEM_ALLOC_NO_SCANOUT) ?
+    allocParams.type = (p->flags & NV_GEM_ALLOC_NO_SCANOUT) ?
         NVKMS_KAPI_ALLOCATION_TYPE_OFFSCREEN : NVKMS_KAPI_ALLOCATION_TYPE_SCANOUT;
+    allocParams.size = p->memory_size;
+    allocParams.useVideoMemory = nv_dev->hasVideoMemory;
+    allocParams.compressible = &p->compressible;
 
-    if (nv_dev->hasVideoMemory) {
-        pMemory = nvKms->allocateVideoMemory(nv_dev->pDevice,
-                                             layout,
-                                             type,
-                                             p->memory_size,
-                                             &p->compressible);
-    } else {
-        pMemory = nvKms->allocateSystemMemory(nv_dev->pDevice,
-                                              layout,
-                                              type,
-                                              p->memory_size,
-                                              &p->compressible);
-    }
-
+    pMemory = nvKms->allocateMemory(nv_dev->pDevice, &allocParams);
     if (pMemory == NULL) {
         ret = -EINVAL;
         NV_DRM_DEV_LOG_ERR(nv_dev,
@@ -551,14 +563,12 @@ static struct drm_gem_object *__nv_drm_gem_nvkms_prime_dup(
 {
     struct nv_drm_device *nv_dev = to_nv_device(dev);
     const struct nv_drm_device *nv_dev_src;
-    const struct nv_drm_gem_nvkms_memory *nv_nvkms_memory_src;
     struct nv_drm_gem_nvkms_memory *nv_nvkms_memory;
     struct NvKmsKapiMemory *pMemory;
 
     BUG_ON(nv_gem_src == NULL || nv_gem_src->ops != &nv_gem_nvkms_memory_ops);
 
     nv_dev_src = to_nv_device(nv_gem_src->base.dev);
-    nv_nvkms_memory_src = to_nv_nvkms_memory_const(nv_gem_src);
 
     if ((nv_nvkms_memory =
             nv_drm_calloc(1, sizeof(*nv_nvkms_memory))) == NULL) {
@@ -601,7 +611,6 @@ int nv_drm_dumb_map_offset(struct drm_file *file,
     int ret = -EINVAL;
 
     if ((nv_nvkms_memory = nv_drm_gem_object_nvkms_memory_lookup(
-                    dev,
                     file,
                     handle)) == NULL) {
         NV_DRM_DEV_LOG_ERR(

kernel-open/nvidia-drm/nvidia-drm-gem-nvkms-memory.h

@@ -25,15 +25,22 @@
 
 #include "nvidia-drm-conftest.h"
 
-#if defined(NV_DRM_ATOMIC_MODESET_AVAILABLE)
+#if defined(NV_DRM_AVAILABLE)
 
 #include "nvidia-drm-gem.h"
 
 struct nv_drm_gem_nvkms_memory {
     struct nv_drm_gem_object base;
 
+    /*
+     * Lock to protect concurrent writes to physically_mapped, pPhysicalAddress,
+     * and pWriteCombinedIORemapAddress.
+     *
+     * __nv_drm_gem_nvkms_map(), the sole writer, is structured such that
+     * readers are not required to hold the lock.
+     */
+    struct mutex map_lock;
     bool physically_mapped;
-
     void *pPhysicalAddress;
     void *pWriteCombinedIORemapAddress;
 
@@ -65,12 +72,11 @@ static inline struct nv_drm_gem_nvkms_memory *to_nv_nvkms_memory_const(
 
 static inline
 struct nv_drm_gem_nvkms_memory *nv_drm_gem_object_nvkms_memory_lookup(
-    struct drm_device *dev,
     struct drm_file *filp,
     u32 handle)
 {
     struct nv_drm_gem_object *nv_gem =
-            nv_drm_gem_object_lookup(dev, filp, handle);
+            nv_drm_gem_object_lookup(filp, handle);
 
     if (nv_gem != NULL && nv_gem->ops != &nv_gem_nvkms_memory_ops) {
         nv_drm_gem_object_unreference_unlocked(nv_gem);

kernel-open/nvidia-drm/nvidia-drm-gem-user-memory.c

@@ -24,9 +24,7 @@
 
 #if defined(NV_DRM_AVAILABLE)
 
-#if defined(NV_DRM_DRM_PRIME_H_PRESENT)
 #include <drm/drm_prime.h>
-#endif
 
 #include "nvidia-drm-gem-user-memory.h"
 #include "nvidia-drm-helper.h"
@@ -36,6 +34,10 @@
 #include "linux/mm.h"
 #include "nv-mm.h"
 
+#if defined(NV_LINUX_PFN_T_H_PRESENT)
+#include "linux/pfn_t.h"
+#endif
+
 #if defined(NV_BSD)
 #include <vm/vm_pageout.h>
 #endif
@@ -68,7 +70,8 @@ static void *__nv_drm_gem_user_memory_prime_vmap(
     struct nv_drm_gem_user_memory *nv_user_memory = to_nv_user_memory(nv_gem);
 
     return nv_drm_vmap(nv_user_memory->pages,
-                           nv_user_memory->pages_count);
+                           nv_user_memory->pages_count,
+                           true);
 }
 
 static void __nv_drm_gem_user_memory_prime_vunmap(
@@ -103,25 +106,16 @@ static int __nv_drm_gem_user_memory_mmap(struct nv_drm_gem_object *nv_gem,
     return 0;
 }
 
-static vm_fault_t __nv_drm_gem_user_memory_handle_vma_fault(
-    struct nv_drm_gem_object *nv_gem,
+#if defined(NV_LINUX) && !defined(NV_VMF_INSERT_MIXED_PRESENT)
+static vm_fault_t __nv_vm_insert_mixed_helper(
     struct vm_area_struct *vma,
-    struct vm_fault *vmf)
+    unsigned long address,
+    unsigned long pfn)
 {
-    struct nv_drm_gem_user_memory *nv_user_memory = to_nv_user_memory(nv_gem);
-    unsigned long address = nv_page_fault_va(vmf);
-    struct drm_gem_object *gem = vma->vm_private_data;
-    unsigned long page_offset;
-    vm_fault_t ret;
+    int ret;
 
-    page_offset = vmf->pgoff - drm_vma_node_start(&gem->vma_node);
+    ret = vm_insert_mixed(vma, address, pfn_to_pfn_t(pfn));
 
-    BUG_ON(page_offset >= nv_user_memory->pages_count);
-
-#if !defined(NV_LINUX)
-    ret = vmf_insert_pfn(vma, address, page_to_pfn(nv_user_memory->pages[page_offset]));
-#else /* !defined(NV_LINUX) */
-    ret = vm_insert_page(vma, address, nv_user_memory->pages[page_offset]);
     switch (ret) {
         case 0:
         case -EBUSY:
@@ -129,19 +123,44 @@ static vm_fault_t __nv_drm_gem_user_memory_handle_vma_fault(
              * EBUSY indicates that another thread already handled
              * the faulted range.
              */
-            ret = VM_FAULT_NOPAGE;
-            break;
+            return VM_FAULT_NOPAGE;
         case -ENOMEM:
-            ret = VM_FAULT_OOM;
-            break;
+            return VM_FAULT_OOM;
         default:
             WARN_ONCE(1, "Unhandled error in %s: %d\n", __FUNCTION__, ret);
-            ret = VM_FAULT_SIGBUS;
-            break;
+            return VM_FAULT_SIGBUS;
     }
-#endif /* !defined(NV_LINUX) */
+}
+#endif
 
-    return ret;
+static vm_fault_t __nv_drm_gem_user_memory_handle_vma_fault(
+    struct nv_drm_gem_object *nv_gem,
+    struct vm_area_struct *vma,
+    struct vm_fault *vmf)
+{
+    struct nv_drm_gem_user_memory *nv_user_memory = to_nv_user_memory(nv_gem);
+    unsigned long address = vmf->address;
+    struct drm_gem_object *gem = vma->vm_private_data;
+    unsigned long page_offset;
+    unsigned long pfn;
+
+    page_offset = vmf->pgoff - drm_vma_node_start(&gem->vma_node);
+    BUG_ON(page_offset >= nv_user_memory->pages_count);
+    pfn = page_to_pfn(nv_user_memory->pages[page_offset]);
+
+#if !defined(NV_LINUX)
+    return vmf_insert_pfn(vma, address, pfn);
+#elif defined(NV_VMF_INSERT_MIXED_PRESENT)
+
+#if defined(NV_LINUX_PFN_T_H_PRESENT)
+    return vmf_insert_mixed(vma, address, pfn_to_pfn_t(pfn));
+#else
+    return vmf_insert_mixed(vma, address, pfn);
+#endif
+
+#else
+    return __nv_vm_insert_mixed_helper(vma, address, pfn);
+#endif
 }
 
 static int __nv_drm_gem_user_create_mmap_offset(

kernel-open/nvidia-drm/nvidia-drm-gem-user-memory.h

@@ -52,12 +52,11 @@ int nv_drm_gem_import_userspace_memory_ioctl(struct drm_device *dev,
 
 static inline
 struct nv_drm_gem_user_memory *nv_drm_gem_object_user_memory_lookup(
-    struct drm_device *dev,
     struct drm_file *filp,
     u32 handle)
 {
     struct nv_drm_gem_object *nv_gem =
-            nv_drm_gem_object_lookup(dev, filp, handle);
+            nv_drm_gem_object_lookup(filp, handle);
 
     if (nv_gem != NULL && nv_gem->ops != &__nv_gem_user_memory_ops) {
         nv_drm_gem_object_unreference_unlocked(nv_gem);

kernel-open/nvidia-drm/nvidia-drm-gem.c

@@ -35,17 +35,10 @@
 #include "nvidia-drm-gem-dma-buf.h"
 #include "nvidia-drm-gem-nvkms-memory.h"
 
-#if defined(NV_DRM_DRM_DRV_H_PRESENT)
 #include <drm/drm_drv.h>
-#endif
-
-#if defined(NV_DRM_DRM_PRIME_H_PRESENT)
 #include <drm/drm_prime.h>
-#endif
-
-#if defined(NV_DRM_DRM_FILE_H_PRESENT)
 #include <drm/drm_file.h>
-#endif
+#include <drm/drm_vma_manager.h>
 
 #include "linux/dma-buf.h"
 
@@ -58,7 +51,7 @@ void nv_drm_gem_free(struct drm_gem_object *gem)
     /* Cleanup core gem object */
     drm_gem_object_release(&nv_gem->base);
 
-#if defined(NV_DRM_FENCE_AVAILABLE) && !defined(NV_DRM_GEM_OBJECT_HAS_RESV)
+#if !defined(NV_DRM_GEM_OBJECT_HAS_RESV)
     nv_dma_resv_fini(&nv_gem->resv);
 #endif
 
@@ -135,7 +128,7 @@ void nv_drm_gem_object_init(struct nv_drm_device *nv_dev,
 
     /* Initialize the gem object */
 
-#if defined(NV_DRM_FENCE_AVAILABLE) && !defined(NV_DRM_GEM_OBJECT_HAS_RESV)
+#if !defined(NV_DRM_GEM_OBJECT_HAS_RESV)
     nv_dma_resv_init(&nv_gem->resv);
 #endif
 
@@ -144,12 +137,17 @@ void nv_drm_gem_object_init(struct nv_drm_device *nv_dev,
 #endif
 
     drm_gem_private_object_init(dev, &nv_gem->base, size);
+
+    /* Create mmap offset early for drm_gem_prime_mmap(), if possible. */
+    if (nv_gem->ops->create_mmap_offset) {
+        uint64_t offset;
+        nv_gem->ops->create_mmap_offset(nv_dev, nv_gem, &offset);
+    }
 }
 
 struct drm_gem_object *nv_drm_gem_prime_import(struct drm_device *dev,
                                                struct dma_buf *dma_buf)
 {
-#if defined(NV_DMA_BUF_OWNER_PRESENT)
     struct drm_gem_object *gem_dst;
     struct nv_drm_gem_object *nv_gem_src;
 
@@ -166,11 +164,13 @@ struct drm_gem_object *nv_drm_gem_prime_import(struct drm_device *dev,
              */
             gem_dst = nv_gem_src->ops->prime_dup(dev, nv_gem_src);
 
-            if (gem_dst)
-                return gem_dst;
+            if (gem_dst == NULL) {
+                return ERR_PTR(-ENOTSUPP);
+            }
+
+            return gem_dst;
         }
     }
-#endif /* NV_DMA_BUF_OWNER_PRESENT */
 
     return drm_gem_prime_import(dev, dma_buf);
 }
@@ -222,8 +222,7 @@ int nv_drm_gem_map_offset_ioctl(struct drm_device *dev,
     struct nv_drm_gem_object *nv_gem;
     int ret;
 
-    if ((nv_gem = nv_drm_gem_object_lookup(dev,
-                                           filep,
+    if ((nv_gem = nv_drm_gem_object_lookup(filep,
                                            params->handle)) == NULL) {
         NV_DRM_DEV_LOG_ERR(
             nv_dev,
@@ -232,6 +231,7 @@ int nv_drm_gem_map_offset_ioctl(struct drm_device *dev,
         return -EINVAL;
     }
 
+    /* mmap offset creation is idempotent, fetch it by creating it again. */
     if (nv_gem->ops->create_mmap_offset) {
         ret = nv_gem->ops->create_mmap_offset(nv_dev, nv_gem, &params->offset);
     } else {
@@ -247,7 +247,6 @@ int nv_drm_gem_map_offset_ioctl(struct drm_device *dev,
     return ret;
 }
 
-#if defined(NV_DRM_ATOMIC_MODESET_AVAILABLE)
 int nv_drm_mmap(struct file *file, struct vm_area_struct *vma)
 {
     struct drm_file *priv = file->private_data;
@@ -258,8 +257,8 @@ int nv_drm_mmap(struct file *file, struct vm_area_struct *vma)
     struct nv_drm_gem_object *nv_gem;
 
     drm_vma_offset_lock_lookup(dev->vma_offset_manager);
-    node = nv_drm_vma_offset_exact_lookup_locked(dev->vma_offset_manager,
-                                                 vma->vm_pgoff, vma_pages(vma));
+    node = drm_vma_offset_exact_lookup_locked(dev->vma_offset_manager,
+                                              vma->vm_pgoff, vma_pages(vma));
     if (likely(node)) {
         obj = container_of(node, struct drm_gem_object, vma_node);
         /*
@@ -285,7 +284,7 @@ int nv_drm_mmap(struct file *file, struct vm_area_struct *vma)
         goto done;
     }
 
-    if (!nv_drm_vma_node_is_allowed(node, file)) {
+    if (!drm_vma_node_is_allowed(node, file->private_data)) {
         ret = -EACCES;
         goto done;
     }
@@ -307,7 +306,6 @@ done:
 
     return ret;
 }
-#endif
 
 int nv_drm_gem_identify_object_ioctl(struct drm_device *dev,
                                      void *data, struct drm_file *filep)
@@ -319,26 +317,24 @@ int nv_drm_gem_identify_object_ioctl(struct drm_device *dev,
     struct nv_drm_gem_object *nv_gem = NULL;
 
     if (!drm_core_check_feature(dev, DRIVER_MODESET)) {
-        return -EINVAL;
+        return -EOPNOTSUPP;
     }
 
-    nv_dma_buf = nv_drm_gem_object_dma_buf_lookup(dev, filep, p->handle);
+    nv_dma_buf = nv_drm_gem_object_dma_buf_lookup(filep, p->handle);
     if (nv_dma_buf) {
         p->object_type = NV_GEM_OBJECT_DMABUF;
         nv_gem = &nv_dma_buf->base;
         goto done;
     }
 
-#if defined(NV_DRM_ATOMIC_MODESET_AVAILABLE)
-    nv_nvkms_memory = nv_drm_gem_object_nvkms_memory_lookup(dev, filep, p->handle);
+    nv_nvkms_memory = nv_drm_gem_object_nvkms_memory_lookup(filep, p->handle);
     if (nv_nvkms_memory) {
         p->object_type = NV_GEM_OBJECT_NVKMS;
         nv_gem = &nv_nvkms_memory->base;
         goto done;
     }
-#endif
 
-    nv_user_memory = nv_drm_gem_object_user_memory_lookup(dev, filep, p->handle);
+    nv_user_memory = nv_drm_gem_object_user_memory_lookup(filep, p->handle);
     if (nv_user_memory) {
         p->object_type = NV_GEM_OBJECT_USERMEMORY;
         nv_gem = &nv_user_memory->base;

kernel-open/nvidia-drm/nvidia-drm-gem.h

@@ -33,17 +33,12 @@
 #include <drm/drmP.h>
 #endif
 
-#if defined(NV_DRM_DRM_GEM_H_PRESENT)
 #include <drm/drm_gem.h>
-#endif
 
 #include "nvkms-kapi.h"
 #include "nv-mm.h"
 
-#if defined(NV_DRM_FENCE_AVAILABLE)
-#include "nvidia-dma-fence-helper.h"
 #include "nvidia-dma-resv-helper.h"
-#endif
 
 #include "linux/dma-buf.h"
 
@@ -73,7 +68,7 @@ struct nv_drm_gem_object {
 
     struct NvKmsKapiMemory *pMemory;
 
-#if defined(NV_DRM_FENCE_AVAILABLE) && !defined(NV_DRM_GEM_OBJECT_HAS_RESV)
+#if !defined(NV_DRM_GEM_OBJECT_HAS_RESV)
     nv_dma_resv_t  resv;
 #endif
 };
@@ -88,47 +83,14 @@ static inline struct nv_drm_gem_object *to_nv_gem_object(
     return NULL;
 }
 
-/*
- * drm_gem_object_{get/put}() added by commit
- * e6b62714e87c8811d5564b6a0738dcde63a51774 (2017-02-28) and
- * drm_gem_object_{reference/unreference}() removed by commit
- * 3e70fd160cf0b1945225eaa08dd2cb8544f21cb8 (2018-11-15).
- */
-
-static inline void
-nv_drm_gem_object_reference(struct nv_drm_gem_object *nv_gem)
-{
-#if defined(NV_DRM_GEM_OBJECT_GET_PRESENT)
-    drm_gem_object_get(&nv_gem->base);
-#else
-    drm_gem_object_reference(&nv_gem->base);
-#endif
-}
-
 static inline void
 nv_drm_gem_object_unreference_unlocked(struct nv_drm_gem_object *nv_gem)
 {
-#if defined(NV_DRM_GEM_OBJECT_GET_PRESENT)
-
 #if defined(NV_DRM_GEM_OBJECT_PUT_UNLOCK_PRESENT)
     drm_gem_object_put_unlocked(&nv_gem->base);
 #else
     drm_gem_object_put(&nv_gem->base);
 #endif
-
-#else
-    drm_gem_object_unreference_unlocked(&nv_gem->base);
-#endif
-}
-
-static inline void
-nv_drm_gem_object_unreference(struct nv_drm_gem_object *nv_gem)
-{
-#if defined(NV_DRM_GEM_OBJECT_GET_PRESENT)
-    drm_gem_object_put(&nv_gem->base);
-#else
-    drm_gem_object_unreference(&nv_gem->base);
-#endif
 }
 
 static inline int nv_drm_gem_handle_create_drop_reference(
@@ -169,17 +131,10 @@ done:
 void nv_drm_gem_free(struct drm_gem_object *gem);
 
 static inline struct nv_drm_gem_object *nv_drm_gem_object_lookup(
-    struct drm_device *dev,
     struct drm_file *filp,
     u32 handle)
 {
-#if (NV_DRM_GEM_OBJECT_LOOKUP_ARGUMENT_COUNT == 3)
-    return to_nv_gem_object(drm_gem_object_lookup(dev, filp, handle));
-#elif (NV_DRM_GEM_OBJECT_LOOKUP_ARGUMENT_COUNT == 2)
     return to_nv_gem_object(drm_gem_object_lookup(filp, handle));
-#else
-    #error "Unknown argument count of drm_gem_object_lookup()"
-#endif
 }
 
 static inline int nv_drm_gem_handle_create(struct drm_file *filp,
@@ -189,7 +144,6 @@ static inline int nv_drm_gem_handle_create(struct drm_file *filp,
     return drm_gem_handle_create(filp, &nv_gem->base, handle);
 }
 
-#if defined(NV_DRM_FENCE_AVAILABLE)
 static inline nv_dma_resv_t *nv_drm_gem_res_obj(struct nv_drm_gem_object *nv_gem)
 {
 #if defined(NV_DRM_GEM_OBJECT_HAS_RESV)
@@ -198,7 +152,6 @@ static inline nv_dma_resv_t *nv_drm_gem_res_obj(struct nv_drm_gem_object *nv_gem
     return nv_gem->base.dma_buf ? nv_gem->base.dma_buf->resv : &nv_gem->resv;
 #endif
 }
-#endif
 
 void nv_drm_gem_object_init(struct nv_drm_device *nv_dev,
                             struct nv_drm_gem_object *nv_gem,

kernel-open/nvidia-drm/nvidia-drm-helper.c

@@ -33,7 +33,7 @@
 
 #include "nvmisc.h"
 
-#if defined(NV_DRM_ATOMIC_MODESET_AVAILABLE)
+#if defined(NV_DRM_AVAILABLE)
 
 #if defined(NV_DRM_DRMP_H_PRESENT)
 #include <drm/drmP.h>
@@ -43,28 +43,7 @@
 #include <drm/drm_atomic_uapi.h>
 #endif
 
-/*
- * The inclusion of drm_framebuffer.h was removed from drm_crtc.h by commit
- * 720cf96d8fecde29b72e1101f8a567a0ce99594f ("drm: Drop drm_framebuffer.h from
- * drm_crtc.h") in linux-next, expected in v5.19-rc7.
- *
- * We only need drm_framebuffer.h for drm_framebuffer_put(), and it is always
- * present (v4.9+) when drm_framebuffer_{put,get}() is present (v4.12+), so it
- * is safe to unconditionally include it when drm_framebuffer_get() is present.
- */
-#if defined(NV_DRM_FRAMEBUFFER_GET_PRESENT)
 #include <drm/drm_framebuffer.h>
-#endif
-
-static void __nv_drm_framebuffer_put(struct drm_framebuffer *fb)
-{
-#if defined(NV_DRM_FRAMEBUFFER_GET_PRESENT)
-    drm_framebuffer_put(fb);
-#else
-    drm_framebuffer_unreference(fb);
-#endif
-
-}
 
 /*
  * drm_atomic_helper_disable_all() has been added by commit
@@ -150,7 +129,6 @@ int nv_drm_atomic_helper_disable_all(struct drm_device *dev,
             goto free;
     }
 
-#if defined(NV_DRM_ROTATION_AVAILABLE)
     nv_drm_for_each_plane(plane, dev) {
         plane_state = drm_atomic_get_plane_state(state, plane);
         if (IS_ERR(plane_state)) {
@@ -160,7 +138,6 @@ int nv_drm_atomic_helper_disable_all(struct drm_device *dev,
 
         plane_state->rotation = DRM_MODE_ROTATE_0;
     }
-#endif
 
     nv_drm_for_each_connector_in_state(state, conn, conn_state, i) {
         ret = drm_atomic_set_crtc_for_connector(conn_state, NULL);
@@ -190,29 +167,15 @@ free:
                 WARN_ON(plane->state->crtc);
 
                 if (plane->old_fb)
-                    __nv_drm_framebuffer_put(plane->old_fb);
+                    drm_framebuffer_put(plane->old_fb);
            }
            plane->old_fb = NULL;
        }
     }
 
-#if defined(NV_DRM_ATOMIC_STATE_REF_COUNTING_PRESENT)
     drm_atomic_state_put(state);
-#else
-    if (ret != 0) {
-        drm_atomic_state_free(state);
-    } else {
-        /*
-         * In case of success, drm_atomic_commit() takes care to cleanup and
-         * free @state.
-         *
-         * Comment placed above drm_atomic_commit() says: The caller must not
-         * free or in any other way access @state. If the function fails then
-         * the caller must clean up @state itself.
-         */
-    }
-#endif
+
     return ret;
 }
 
-#endif /* NV_DRM_ATOMIC_MODESET_AVAILABLE */
+#endif /* NV_DRM_AVAILABLE */

kernel-open/nvidia-drm/nvidia-drm-helper.h

@@ -31,62 +31,26 @@
 #include <drm/drmP.h>
 #endif
 
-#if defined(NV_DRM_DRM_DRV_H_PRESENT)
 #include <drm/drm_drv.h>
-#endif
 
-#if defined(NV_DRM_ALPHA_BLENDING_AVAILABLE) || defined(NV_DRM_ROTATION_AVAILABLE)
-/* For DRM_ROTATE_* , DRM_REFLECT_* */
+#if defined(NV_DRM_ALPHA_BLENDING_AVAILABLE)
 #include <drm/drm_blend.h>
 #endif
 
-#if defined(NV_DRM_ROTATION_AVAILABLE)
-/* For DRM_MODE_ROTATE_* and DRM_MODE_REFLECT_* */
+/*
+ * For DRM_MODE_ROTATE_*, DRM_MODE_REFLECT_*, struct drm_color_ctm_3x4, and
+ * struct drm_color_lut.
+ */
 #include <uapi/drm/drm_mode.h>
-#endif
-
-#if defined(NV_DRM_ROTATION_AVAILABLE)
-/*
- * 19-05-2017 c2c446ad29437bb92b157423c632286608ebd3ec has added
- * DRM_MODE_ROTATE_* and DRM_MODE_REFLECT_* to UAPI and removed
- * DRM_ROTATE_* and DRM_REFLECT_*
- */
-#if !defined(DRM_MODE_ROTATE_0)
-#define DRM_MODE_ROTATE_0       DRM_ROTATE_0
-#define DRM_MODE_ROTATE_90      DRM_ROTATE_90
-#define DRM_MODE_ROTATE_180     DRM_ROTATE_180
-#define DRM_MODE_ROTATE_270     DRM_ROTATE_270
-#define DRM_MODE_REFLECT_X      DRM_REFLECT_X
-#define DRM_MODE_REFLECT_Y      DRM_REFLECT_Y
-#define DRM_MODE_ROTATE_MASK    DRM_ROTATE_MASK
-#define DRM_MODE_REFLECT_MASK   DRM_REFLECT_MASK
-#endif
-
-#endif //NV_DRM_ROTATION_AVAILABLE
 
 /*
- * drm_dev_put() is added by commit 9a96f55034e41b4e002b767e9218d55f03bdff7d
- * (2017-09-26) and drm_dev_unref() is removed by
- * ba1d345401476a5f7fbad622607c5a1f95e59b31 (2018-11-15).
- *
- * drm_dev_unref() has been added and drm_dev_free() removed by commit -
- *
- *      2014-01-29: 099d1c290e2ebc3b798961a6c177c3aef5f0b789
+ * Commit 1e13c5644c44 ("drm/drm_mode_object: increase max objects to
+ * accommodate new color props") in Linux v6.8 increased the pre-object
+ * property limit to from 24 to 64.
  */
-static inline void nv_drm_dev_free(struct drm_device *dev)
-{
-#if defined(NV_DRM_DEV_PUT_PRESENT)
-    drm_dev_put(dev);
-#elif defined(NV_DRM_DEV_UNREF_PRESENT)
-    drm_dev_unref(dev);
-#else
-    drm_dev_free(dev);
-#endif
-}
+#define NV_DRM_USE_EXTENDED_PROPERTIES (DRM_OBJECT_MAX_PROPERTY >= 64)
 
-#if defined(NV_DRM_DRM_PRIME_H_PRESENT)
 #include <drm/drm_prime.h>
-#endif
 
 static inline struct sg_table*
 nv_drm_prime_pages_to_sg(struct drm_device *dev,
@@ -99,8 +63,6 @@ nv_drm_prime_pages_to_sg(struct drm_device *dev,
 #endif
 }
 
-#if defined(NV_DRM_ATOMIC_MODESET_AVAILABLE)
-
 /*
  * drm_for_each_connector(), drm_for_each_crtc(), drm_for_each_fb(),
  * drm_for_each_encoder and drm_for_each_plane() were added by kernel
@@ -154,18 +116,6 @@ nv_drm_prime_pages_to_sg(struct drm_device *dev,
     list_for_each_entry(crtc, &(dev)->mode_config.crtc_list, head)
 #endif
 
-#if defined(NV_DRM_CONNECTOR_LIST_ITER_PRESENT)
-#define nv_drm_for_each_connector(connector, conn_iter, dev) \
-        drm_for_each_connector_iter(connector, conn_iter)
-#elif defined(drm_for_each_connector)
-#define nv_drm_for_each_connector(connector, conn_iter, dev) \
-    drm_for_each_connector(connector, dev)
-#else
-#define nv_drm_for_each_connector(connector, conn_iter, dev) \
-    WARN_ON(!mutex_is_locked(&dev->mode_config.mutex));      \
-    list_for_each_entry(connector, &(dev)->mode_config.connector_list, head)
-#endif
-
 #if defined(drm_for_each_encoder)
 #define nv_drm_for_each_encoder(encoder, dev) \
     drm_for_each_encoder(encoder, dev)
@@ -336,54 +286,8 @@ int nv_drm_atomic_helper_disable_all(struct drm_device *dev,
     for_each_new_plane_in_state(__state, plane, new_plane_state, __i)
 #endif
 
-static inline struct drm_connector *
-nv_drm_connector_lookup(struct drm_device *dev, struct drm_file *filep,
-                        uint32_t id)
-{
-#if !defined(NV_DRM_CONNECTOR_LOOKUP_PRESENT)
-    return drm_connector_find(dev, id);
-#elif defined(NV_DRM_MODE_OBJECT_FIND_HAS_FILE_PRIV_ARG)
-    return drm_connector_lookup(dev, filep, id);
-#else
-    return drm_connector_lookup(dev, id);
-#endif
-}
-
-static inline void nv_drm_connector_put(struct drm_connector *connector)
-{
-#if defined(NV_DRM_CONNECTOR_PUT_PRESENT)
-    drm_connector_put(connector);
-#elif defined(NV_DRM_CONNECTOR_LOOKUP_PRESENT)
-    drm_connector_unreference(connector);
-#endif
-}
-
-static inline struct drm_crtc *
-nv_drm_crtc_find(struct drm_device *dev, struct drm_file *filep, uint32_t id)
-{
-#if defined(NV_DRM_MODE_OBJECT_FIND_HAS_FILE_PRIV_ARG)
-    return drm_crtc_find(dev, filep, id);
-#else
-    return drm_crtc_find(dev, id);
-#endif
-}
-
-static inline struct drm_encoder *nv_drm_encoder_find(struct drm_device *dev,
-    uint32_t id)
-{
-#if defined(NV_DRM_MODE_OBJECT_FIND_HAS_FILE_PRIV_ARG)
-    return drm_encoder_find(dev, NULL /* file_priv */, id);
-#else
-    return drm_encoder_find(dev, id);
-#endif
-}
-
-#if defined(NV_DRM_DRM_AUTH_H_PRESENT)
 #include <drm/drm_auth.h>
-#endif
-#if defined(NV_DRM_DRM_FILE_H_PRESENT)
 #include <drm/drm_file.h>
-#endif
 
 /*
  * drm_file_get_master() added by commit 56f0729a510f ("drm: protect drm_master
@@ -408,10 +312,6 @@ static inline struct drm_master *nv_drm_file_get_master(struct drm_file *filep)
  *     Ville Syrjälä <ville.syrjala@linux.intel.com>
  *
  * drm_connector_for_each_possible_encoder() is copied from
- * include/drm/drm_connector.h and modified to use nv_drm_encoder_find()
- * instead of drm_encoder_find().
- *
- * drm_connector_for_each_possible_encoder() is copied from
  *      include/drm/drm_connector.h @
  *      83aefbb887b59df0b3520965c3701e01deacfc52
  * which has the following copyright and license information:
@@ -437,9 +337,7 @@ static inline struct drm_master *nv_drm_file_get_master(struct drm_file *filep)
  * OF THIS SOFTWARE.
  */
 
-#if defined(NV_DRM_DRM_CONNECTOR_H_PRESENT)
 #include <drm/drm_connector.h>
-#endif
 
 /**
  * nv_drm_connector_for_each_possible_encoder - iterate connector's possible
@@ -458,8 +356,9 @@ static inline struct drm_master *nv_drm_file_get_master(struct drm_file *filep)
        for ((__i) = 0; (__i) < ARRAY_SIZE((connector)->encoder_ids) &&        \
                     (connector)->encoder_ids[(__i)] != 0; (__i)++)            \
                for_each_if((encoder) =                                        \
-                           nv_drm_encoder_find((connector)->dev,              \
-                                               (connector)->encoder_ids[(__i)]))
+                           drm_encoder_find((connector)->dev, NULL,           \
+                                            (connector)->encoder_ids[(__i)]))
+
 
 #define nv_drm_connector_for_each_possible_encoder(connector, encoder) \
     {                                                                  \
@@ -514,80 +413,14 @@ nv_drm_connector_update_edid_property(struct drm_connector *connector,
 #endif
 }
 
-#if defined(NV_DRM_CONNECTOR_LIST_ITER_PRESENT)
-#include <drm/drm_connector.h>
-
-static inline
-void nv_drm_connector_list_iter_begin(struct drm_device *dev,
-                                      struct drm_connector_list_iter *iter)
-{
-#if defined(NV_DRM_CONNECTOR_LIST_ITER_BEGIN_PRESENT)
-    drm_connector_list_iter_begin(dev, iter);
-#else
-    drm_connector_list_iter_get(dev, iter);
-#endif
-}
-
-static inline
-void nv_drm_connector_list_iter_end(struct drm_connector_list_iter *iter)
-{
-#if defined(NV_DRM_CONNECTOR_LIST_ITER_BEGIN_PRESENT)
-    drm_connector_list_iter_end(iter);
-#else
-    drm_connector_list_iter_put(iter);
-#endif
-}
-#endif
-
-/*
- * The drm_format_num_planes() function was added by commit d0d110e09629 drm:
- * Add drm_format_num_planes() utility function in v3.3 (2011-12-20). Prototype
- * was moved from drm_crtc.h to drm_fourcc.h by commit ae4df11a0f53 (drm: Move
- * format-related helpers to drm_fourcc.c) in v4.8 (2016-06-09).
- * drm_format_num_planes() has been removed by commit 05c452c115bf (drm: Remove
- * users of drm_format_num_planes) in v5.3 (2019-05-16).
- *
- * drm_format_info() is available only from v4.10 (2016-10-18), added by commit
- * 84770cc24f3a (drm: Centralize format information).
- */
-#include <drm/drm_crtc.h>
 #include <drm/drm_fourcc.h>
 
 static inline int nv_drm_format_num_planes(uint32_t format)
 {
-#if defined(NV_DRM_FORMAT_NUM_PLANES_PRESENT)
-    return drm_format_num_planes(format);
-#else
     const struct drm_format_info *info = drm_format_info(format);
     return info != NULL ? info->num_planes : 1;
-#endif
 }
 
-#if defined(NV_DRM_FORMAT_MODIFIERS_PRESENT)
-/*
- * DRM_FORMAT_MOD_LINEAR was also defined after the original modifier support
- * was added to the kernel, as a more explicit alias of DRM_FORMAT_MOD_NONE
- */
-#if !defined(DRM_FORMAT_MOD_VENDOR_NONE)
-#define DRM_FORMAT_MOD_VENDOR_NONE 0
-#endif
-
-#if !defined(DRM_FORMAT_MOD_LINEAR)
-#define DRM_FORMAT_MOD_LINEAR fourcc_mod_code(NONE, 0)
-#endif
-
-/*
- * DRM_FORMAT_MOD_INVALID was defined after the original modifier support was
- * added to the kernel, for use as a sentinel value.
- */
-#if !defined(DRM_FORMAT_RESERVED)
-#define DRM_FORMAT_RESERVED ((1ULL << 56) - 1)
-#endif
-
-#if !defined(DRM_FORMAT_MOD_INVALID)
-#define DRM_FORMAT_MOD_INVALID fourcc_mod_code(NONE, DRM_FORMAT_RESERVED)
-#endif
-
 /*
  * DRM_FORMAT_MOD_VENDOR_NVIDIA was previously called
  * DRM_FORMAT_MOD_VNEDOR_NV.
@@ -610,11 +443,9 @@ static inline int nv_drm_format_num_planes(uint32_t format)
                              (((c) & 0x7) << 23)))
 #endif
 
-#endif /* defined(NV_DRM_FORMAT_MODIFIERS_PRESENT) */
-
 /*
- * DRM_UNLOCKED was removed with linux-next commit 2798ffcc1d6a ("drm: Remove
- * locking for legacy ioctls and DRM_UNLOCKED"), but it was previously made
+ * DRM_UNLOCKED was removed with commit 2798ffcc1d6a ("drm: Remove locking for
+ * legacy ioctls and DRM_UNLOCKED") in v6.8, but it was previously made
  * implicit for all non-legacy DRM driver IOCTLs since Linux v4.10 commit
  * fa5386459f06 "drm: Used DRM_LEGACY for all legacy functions" (Linux v4.4
  * commit ea487835e887 "drm: Enforce unlocked ioctl operation for kms driver
@@ -626,76 +457,15 @@ static inline int nv_drm_format_num_planes(uint32_t format)
 #endif
 
 /*
- * drm_vma_offset_exact_lookup_locked() were added
- * by kernel commit 2225cfe46bcc which was Signed-off-by:
- *      Daniel Vetter <daniel.vetter@intel.com>
- *
- * drm_vma_offset_exact_lookup_locked() were copied from
- *      include/drm/drm_vma_manager.h @ 2225cfe46bcc
- * which has the following copyright and license information:
- *
- * Copyright (c) 2013 David Herrmann <dh.herrmann@gmail.com>
- *
- * Permission is hereby granted, free of charge, to any person obtaining a
- * copy of this software and associated documentation files (the "Software"),
- * to deal in the Software without restriction, including without limitation
- * the rights to use, copy, modify, merge, publish, distribute, sublicense,
- * and/or sell copies of the Software, and to permit persons to whom the
- * Software is furnished to do so, subject to the following conditions:
- *
- * The above copyright notice and this permission notice shall be included in
- * all copies or substantial portions of the Software.
- *
- * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
- * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
- * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
- * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
- * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
- * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
- * OTHER DEALINGS IN THE SOFTWARE.
+ * struct drm_color_ctm_3x4 was added by commit 6872a189be50 ("drm/amd/display:
+ * Add 3x4 CTM support for plane CTM") in v6.8. For backwards compatibility,
+ * define it when not present.
  */
-
-#include <drm/drm_vma_manager.h>
-
-/**
- * nv_drm_vma_offset_exact_lookup_locked() - Look up node by exact address
- * @mgr: Manager object
- * @start: Start address (page-based, not byte-based)
- * @pages: Size of object (page-based)
- *
- * Same as drm_vma_offset_lookup_locked() but does not allow any offset into the node.
- * It only returns the exact object with the given start address.
- *
- * RETURNS:
- * Node at exact start address @start.
- */
-static inline struct drm_vma_offset_node *
-nv_drm_vma_offset_exact_lookup_locked(struct drm_vma_offset_manager *mgr,
-                                      unsigned long start,
-                                      unsigned long pages)
-{
-#if defined(NV_DRM_VMA_OFFSET_EXACT_LOOKUP_LOCKED_PRESENT)
-    return drm_vma_offset_exact_lookup_locked(mgr, start, pages);
-#else
-    struct drm_vma_offset_node *node;
-
-    node = drm_vma_offset_lookup_locked(mgr, start, pages);
-    return (node && node->vm_node.start == start) ? node : NULL;
+#if !defined(NV_DRM_COLOR_CTM_3X4_PRESENT)
+struct drm_color_ctm_3x4 {
+    __u64 matrix[12];
+};
 #endif
-}
-
-static inline bool
-nv_drm_vma_node_is_allowed(struct drm_vma_offset_node *node,
-                           struct file *filp)
-{
-#if defined(NV_DRM_VMA_NODE_IS_ALLOWED_HAS_TAG_ARG)
-    return drm_vma_node_is_allowed(node, filp->private_data);
-#else
-    return drm_vma_node_is_allowed(node, filp);
-#endif
-}
-
-#endif /* defined(NV_DRM_ATOMIC_MODESET_AVAILABLE) */
 
 #endif /* defined(NV_DRM_AVAILABLE) */
 

kernel-open/nvidia-drm/nvidia-drm-ioctl.h

@@ -52,6 +52,7 @@
 #define DRM_NVIDIA_SEMSURF_FENCE_CREATE             0x15
 #define DRM_NVIDIA_SEMSURF_FENCE_WAIT               0x16
 #define DRM_NVIDIA_SEMSURF_FENCE_ATTACH             0x17
+#define DRM_NVIDIA_GET_DRM_FILE_UNIQUE_ID           0x18
 
 #define DRM_IOCTL_NVIDIA_GEM_IMPORT_NVKMS_MEMORY                           \
     DRM_IOWR((DRM_COMMAND_BASE + DRM_NVIDIA_GEM_IMPORT_NVKMS_MEMORY),      \
@@ -157,6 +158,11 @@
               DRM_NVIDIA_SEMSURF_FENCE_ATTACH),                         \
               struct drm_nvidia_semsurf_fence_attach_params)
 
+#define DRM_IOCTL_NVIDIA_GET_DRM_FILE_UNIQUE_ID                         \
+    DRM_IOWR((DRM_COMMAND_BASE +                                        \
+              DRM_NVIDIA_GET_DRM_FILE_UNIQUE_ID),                       \
+              struct drm_nvidia_get_drm_file_unique_id_params)
+
 struct drm_nvidia_gem_import_nvkms_memory_params {
     uint64_t mem_size;           /* IN */
 
@@ -176,6 +182,7 @@ struct drm_nvidia_gem_import_userspace_memory_params {
 
 struct drm_nvidia_get_dev_info_params {
     uint32_t gpu_id;             /* OUT */
+    uint32_t mig_device;         /* OUT */
     uint32_t primary_index;      /* OUT; the "card%d" value */
 
     uint32_t supports_alloc;     /* OUT */
@@ -385,4 +392,8 @@ struct drm_nvidia_semsurf_fence_attach_params {
     uint64_t wait_value;            /* IN Semaphore value to reach before signal */
 };
 
+struct drm_nvidia_get_drm_file_unique_id_params {
+    uint64_t id;                    /* OUT Unique ID of the DRM file */
+};
+
 #endif /* _UAPI_NVIDIA_DRM_IOCTL_H_ */

kernel-open/nvidia-drm/nvidia-drm-linux.c

@@ -34,10 +34,10 @@ MODULE_PARM_DESC(
     "Enable atomic kernel modesetting (1 = enable, 0 = disable (default))");
 module_param_named(modeset, nv_drm_modeset_module_param, bool, 0400);
 
-#if defined(NV_DRM_FBDEV_GENERIC_AVAILABLE)
+#if defined(NV_DRM_FBDEV_AVAILABLE)
 MODULE_PARM_DESC(
     fbdev,
-    "Create a framebuffer device (1 = enable, 0 = disable (default)) (EXPERIMENTAL)");
+    "Create a framebuffer device (1 = enable (default), 0 = disable)");
 module_param_named(fbdev, nv_drm_fbdev_module_param, bool, 0400);
 #endif
 

kernel-open/nvidia-drm/nvidia-drm-modeset.c

@@ -20,9 +20,9 @@
  * DEALINGS IN THE SOFTWARE.
  */
 
-#include "nvidia-drm-conftest.h" /* NV_DRM_ATOMIC_MODESET_AVAILABLE */
+#include "nvidia-drm-conftest.h" /* NV_DRM_AVAILABLE */
 
-#if defined(NV_DRM_ATOMIC_MODESET_AVAILABLE)
+#if defined(NV_DRM_AVAILABLE)
 
 #include "nvidia-drm-priv.h"
 #include "nvidia-drm-modeset.h"
@@ -34,14 +34,19 @@
 #include <drm/drmP.h>
 #endif
 
-#if defined(NV_DRM_DRM_VBLANK_H_PRESENT)
 #include <drm/drm_vblank.h>
-#endif
-
 #include <drm/drm_atomic.h>
 #include <drm/drm_atomic_helper.h>
 #include <drm/drm_crtc.h>
 
+#if defined(NV_LINUX_NVHOST_H_PRESENT) && defined(CONFIG_TEGRA_GRHOST)
+#include <linux/nvhost.h>
+#elif defined(NV_LINUX_HOST1X_NEXT_H_PRESENT)            
+#include <linux/host1x-next.h>
+#endif
+
+#include <linux/dma-fence.h>
+
 struct nv_drm_atomic_state {
     struct NvKmsKapiRequestedModeSetConfig config;
     struct drm_atomic_state base;
@@ -146,6 +151,163 @@ static int __nv_drm_put_back_post_fence_fd(
     return ret;
 }
 
+struct nv_drm_plane_fence_cb_data {
+    struct dma_fence_cb dma_fence_cb;
+    struct nv_drm_device *nv_dev;
+    NvU32 semaphore_index;
+};
+
+static void
+__nv_drm_plane_fence_cb(
+    struct dma_fence *fence,
+    struct dma_fence_cb *cb_data
+)
+{
+    struct nv_drm_plane_fence_cb_data *fence_data =
+        container_of(cb_data, typeof(*fence_data), dma_fence_cb);
+    struct nv_drm_device *nv_dev = fence_data->nv_dev;
+
+    dma_fence_put(fence);
+    nvKms->signalDisplaySemaphore(nv_dev->pDevice, fence_data->semaphore_index);
+    nv_drm_free(fence_data);
+}
+
+static int __nv_drm_convert_in_fences(
+    struct nv_drm_device *nv_dev,
+    struct drm_atomic_state *state,
+    struct drm_crtc *crtc,
+    struct drm_crtc_state *crtc_state)
+{
+    struct drm_plane *plane = NULL;
+    struct drm_plane_state *plane_state = NULL;
+    struct nv_drm_plane *nv_plane = NULL;
+    struct NvKmsKapiLayerRequestedConfig *plane_req_config = NULL;
+    struct NvKmsKapiHeadRequestedConfig *head_req_config =
+        &to_nv_crtc_state(crtc_state)->req_config;
+    struct nv_drm_plane_fence_cb_data *fence_data;
+    uint32_t semaphore_index;
+    uint32_t idx_count;
+    int ret, i;
+
+    if (!crtc_state->active) {
+        return 0;
+    }
+
+    nv_drm_for_each_new_plane_in_state(state, plane, plane_state, i) {
+        if ((plane->type == DRM_PLANE_TYPE_CURSOR) ||
+            (plane_state->crtc != crtc) ||
+            (plane_state->fence == NULL)) {
+            continue;
+        }
+
+        nv_plane = to_nv_plane(plane);
+        plane_req_config =
+            &head_req_config->layerRequestedConfig[nv_plane->layer_idx];
+
+        if (nv_dev->supportsSyncpts) {
+#if defined(NV_LINUX_NVHOST_H_PRESENT) && defined(CONFIG_TEGRA_GRHOST)
+#if defined(NV_NVHOST_DMA_FENCE_UNPACK_PRESENT)
+            int ret =
+                nvhost_dma_fence_unpack(
+                    plane_state->fence,
+                    &plane_req_config->config.syncParams.u.syncpt.preSyncptId,
+                    &plane_req_config->config.syncParams.u.syncpt.preSyncptValue);
+            if (ret == 0) {
+                plane_req_config->config.syncParams.preSyncptSpecified = true;
+                continue;
+            }
+#endif
+#elif defined(NV_LINUX_HOST1X_NEXT_H_PRESENT)
+            int ret =
+                host1x_fence_extract(
+                    plane_state->fence,
+                    &plane_req_config->config.syncParams.u.syncpt.preSyncptId,
+                    &plane_req_config->config.syncParams.u.syncpt.preSyncptValue);
+            if (ret == 0) {
+                plane_req_config->config.syncParams.preSyncptSpecified = true;
+                continue;
+            }
+#endif
+        }
+
+        /*
+         * Syncpt extraction failed, or syncpts are not supported.
+         * Use general DRM fence support with semaphores instead.
+         */
+        if (plane_req_config->config.syncParams.postSyncptRequested) {
+            // Can't mix Syncpts and semaphores in a given request.
+            return -EINVAL;
+        }
+
+        for (idx_count = 0; idx_count < nv_dev->display_semaphores.count; idx_count++) {
+            semaphore_index = nv_drm_next_display_semaphore(nv_dev);
+            if (nvKms->tryInitDisplaySemaphore(nv_dev->pDevice, semaphore_index)) {
+                break;
+            }
+        }
+
+        if (idx_count == nv_dev->display_semaphores.count) {
+            NV_DRM_DEV_LOG_ERR(
+                nv_dev,
+                "Failed to initialize semaphore for plane fence");
+            /*
+             * This should only happen if the semaphore pool was somehow
+             * exhausted. Waiting a bit and retrying may help in that case.
+             */
+            return -EAGAIN;
+        }
+
+        plane_req_config->config.syncParams.semaphoreSpecified = true;
+        plane_req_config->config.syncParams.u.semaphore.index = semaphore_index;
+
+        fence_data = nv_drm_calloc(1, sizeof(*fence_data));
+
+        if (!fence_data) {
+            NV_DRM_DEV_LOG_ERR(
+                nv_dev,
+                "Failed to allocate callback data for plane fence");
+            nvKms->cancelDisplaySemaphore(nv_dev->pDevice, semaphore_index);
+            return -ENOMEM;
+        }
+
+        fence_data->nv_dev = nv_dev;
+        fence_data->semaphore_index = semaphore_index;
+
+        ret = dma_fence_add_callback(plane_state->fence,
+                                     &fence_data->dma_fence_cb,
+                                     __nv_drm_plane_fence_cb);
+
+        switch (ret) {
+        case -ENOENT:
+            /* The fence is already signaled */
+            __nv_drm_plane_fence_cb(plane_state->fence,
+                                    &fence_data->dma_fence_cb);
+#if defined(fallthrough)
+            fallthrough;
+#else
+            /* Fallthrough */
+#endif
+        case 0:
+            /*
+             * The plane state's fence reference has either been consumed or
+             * belongs to the outstanding callback now.
+             */
+            plane_state->fence = NULL;
+            break;
+        default:
+            NV_DRM_DEV_LOG_ERR(
+                nv_dev,
+                "Failed plane fence callback registration");
+            /* Fence callback registration failed */
+            nvKms->cancelDisplaySemaphore(nv_dev->pDevice, semaphore_index);
+            nv_drm_free(fence_data);
+            return ret;
+        }
+    }
+
+    return 0;
+}
+
 static int __nv_drm_get_syncpt_data(
     struct nv_drm_device *nv_dev,
     struct drm_crtc *crtc,
@@ -245,6 +407,25 @@ nv_drm_atomic_apply_modeset_config(struct drm_device *dev,
         return -EINVAL;
     }
 
+    if (commit) {
+        /*
+         * This function does what is necessary to prepare the framebuffers
+         * attached to each new plane in the state for scan out, mostly by
+         * calling back into driver callbacks the NVIDIA driver does not
+         * provide. The end result is that all it does on the NVIDIA driver
+         * is populate the plane state's dma fence pointers with any implicit
+         * sync fences attached to the GEM objects associated with those planes
+         * in the new state, prefering explicit sync fences when appropriate.
+         * This must be done prior to converting the per-plane fences to
+         * semaphore waits below.
+         */
+        ret = drm_atomic_helper_prepare_planes(dev, state);
+
+        if (ret) {
+            return ret;
+        }
+    }
+
     memset(requested_config, 0, sizeof(*requested_config));
 
     /* Loop over affected crtcs and construct NvKmsKapiRequestedModeSetConfig */
@@ -258,11 +439,6 @@ nv_drm_atomic_apply_modeset_config(struct drm_device *dev,
                                commit ? crtc->state : crtc_state;
         struct nv_drm_crtc *nv_crtc = to_nv_crtc(crtc);
 
-        requested_config->headRequestedConfig[nv_crtc->head] =
-            to_nv_crtc_state(new_crtc_state)->req_config;
-
-        requested_config->headsMask |= 1 << nv_crtc->head;
-
         if (commit) {
             struct drm_crtc_state *old_crtc_state = crtc_state;
             struct nv_drm_crtc_state *nv_new_crtc_state =
@@ -282,7 +458,25 @@ nv_drm_atomic_apply_modeset_config(struct drm_device *dev,
 
                 nv_new_crtc_state->nv_flip = NULL;
             }
+
+            ret = __nv_drm_convert_in_fences(nv_dev,
+                                             state,
+                                             crtc,
+                                             new_crtc_state);
+
+            if (ret != 0) {
+                return ret;
+            }
         }
+
+        /*
+         * Do this deep copy after calling __nv_drm_convert_in_fences,
+         * which modifies the new CRTC state's req_config member
+         */
+        requested_config->headRequestedConfig[nv_crtc->head] =
+            to_nv_crtc_state(new_crtc_state)->req_config;
+
+        requested_config->headsMask |= 1 << nv_crtc->head;
     }
 
     if (commit && nvKms->systemInfo.bAllowWriteCombining) {
@@ -313,6 +507,10 @@ nv_drm_atomic_apply_modeset_config(struct drm_device *dev,
         }
     }
 
+    if (commit && nv_dev->requiresVrrSemaphores && reply_config.vrrFlip) {
+        nvKms->signalVrrSemaphore(nv_dev->pDevice, reply_config.vrrSemaphoreIndex);
+    }
+
     return 0;
 }
 
@@ -321,7 +519,6 @@ int nv_drm_atomic_check(struct drm_device *dev,
 {
     int ret = 0;
 
-#if defined(NV_DRM_COLOR_MGMT_AVAILABLE)
     struct drm_crtc *crtc;
     struct drm_crtc_state *crtc_state;
     int i;
@@ -337,7 +534,6 @@ int nv_drm_atomic_check(struct drm_device *dev,
             }
         }
     }
-#endif /* NV_DRM_COLOR_MGMT_AVAILABLE */
 
     if ((ret = drm_atomic_helper_check(dev, state)) != 0) {
         goto done;
@@ -464,9 +660,32 @@ int nv_drm_atomic_commit(struct drm_device *dev,
                     "Flip event timeout on head %u", nv_crtc->head);
             }
         }
-    }
 
-#if defined(NV_DRM_ATOMIC_HELPER_SWAP_STATE_HAS_STALL_ARG)
+        /*
+         * If the legacy LUT needs to be updated, ensure that the previous LUT
+         * update is complete first.
+         */
+        if (crtc_state->color_mgmt_changed) {
+            NvBool complete = nvKms->checkLutNotifier(nv_dev->pDevice,
+                                                      nv_crtc->head,
+                                                      !nonblock /* waitForCompletion */);
+
+            /* If checking the LUT notifier failed, assume no LUT notifier is set. */
+            if (!complete) {
+                if (nonblock) {
+                    return -EBUSY;
+                } else {
+                    /*
+                     * checkLutNotifier should wait on the notifier in this
+                     * case, so we should only get here if the wait timed out.
+                     */
+                    NV_DRM_DEV_LOG_ERR(
+                        nv_dev,
+                        "LUT notifier timeout on head %u", nv_crtc->head);
+                }
+            }
+        }
+    }
 
     /*
      * nv_drm_atomic_commit_internal()
@@ -478,18 +697,10 @@ int nv_drm_atomic_commit(struct drm_device *dev,
      * expected.
      */
 
-#if defined(NV_DRM_ATOMIC_HELPER_SWAP_STATE_RETURN_INT)
     ret = drm_atomic_helper_swap_state(state, false /* stall */);
     if (WARN_ON(ret != 0)) {
         return ret;
     }
-#else
-    drm_atomic_helper_swap_state(state, false /* stall */);
-#endif
-
-#else
-    drm_atomic_helper_swap_state(dev, state);
-#endif
 
     /*
      * nv_drm_atomic_commit_internal() must not return failure after
@@ -506,7 +717,6 @@ int nv_drm_atomic_commit(struct drm_device *dev,
 
         goto done;
     }
-    nv_dev->drmMasterChangedSinceLastAtomicCommit = NV_FALSE;
 
     nv_drm_for_each_crtc_in_state(state, crtc, crtc_state, i) {
         struct nv_drm_crtc *nv_crtc = to_nv_crtc(crtc);
@@ -587,20 +797,29 @@ int nv_drm_atomic_commit(struct drm_device *dev,
                 NV_DRM_DEV_LOG_ERR(
                     nv_dev,
                     "Flip event timeout on head %u", nv_crtc->head);
+                while (!list_empty(&nv_crtc->flip_list)) {
+                    __nv_drm_handle_flip_event(nv_crtc);
+                }
+            }
+
+            if (crtc_state->color_mgmt_changed) {
+                NvBool complete = nvKms->checkLutNotifier(nv_dev->pDevice,
+                                                          nv_crtc->head,
+                                                          true /* waitForCompletion */);
+                if (!complete) {
+                    NV_DRM_DEV_LOG_ERR(
+                        nv_dev,
+                        "LUT notifier timeout on head %u", nv_crtc->head);
+                }
             }
         }
     }
 
 done:
 
-#if defined(NV_DRM_ATOMIC_STATE_REF_COUNTING_PRESENT)
     /*
-     * If ref counting is present, state will be freed when the caller
-     * drops its reference after we return.
+     * State will be freed when the caller drops its reference after we return.
      */
-#else
-    drm_atomic_state_free(state);
-#endif
 
     return 0;
 }

kernel-open/nvidia-drm/nvidia-drm-modeset.h

@@ -25,7 +25,7 @@
 
 #include "nvidia-drm-conftest.h"
 
-#if defined(NV_DRM_ATOMIC_MODESET_AVAILABLE)
+#if defined(NV_DRM_AVAILABLE)
 
 #include "nvkms-kapi.h"
 
@@ -48,6 +48,6 @@ void nv_drm_handle_flip_occurred(struct nv_drm_device *nv_dev,
 
 int nv_drm_shut_down_all_crtcs(struct drm_device *dev);
 
-#endif /* NV_DRM_ATOMIC_MODESET_AVAILABLE */
+#endif /* NV_DRM_AVAILABLE */
 
 #endif /* __NVIDIA_DRM_MODESET_H__ */

kernel-open/nvidia-drm/nvidia-drm-os-interface.c

@@ -1,5 +1,5 @@
 /*
- * Copyright (c) 2015-2023, NVIDIA CORPORATION. All rights reserved.
+ * Copyright (c) 2015-2025, NVIDIA CORPORATION. All rights reserved.
  *
  * Permission is hereby granted, free of charge, to any person obtaining a
  * copy of this software and associated documentation files (the "Software"),
@@ -26,11 +26,8 @@
 
 #if defined(NV_DRM_AVAILABLE)
 
-#if defined(NV_LINUX_SYNC_FILE_H_PRESENT)
 #include <linux/file.h>
 #include <linux/sync_file.h>
-#endif
-
 #include <linux/vmalloc.h>
 #include <linux/sched.h>
 #include <linux/device.h>
@@ -42,7 +39,7 @@
 #endif
 
 bool nv_drm_modeset_module_param = false;
-bool nv_drm_fbdev_module_param = false;
+bool nv_drm_fbdev_module_param = true;
 
 void *nv_drm_calloc(size_t nmemb, size_t size)
 {
@@ -66,18 +63,6 @@ void nv_drm_free(void *ptr)
     kfree(ptr);
 }
 
-char *nv_drm_asprintf(const char *fmt, ...)
-{
-    va_list ap;
-    char *p;
-
-    va_start(ap, fmt);
-    p = kvasprintf(GFP_KERNEL, fmt, ap);
-    va_end(ap);
-
-    return p;
-}
-
 #if defined(NVCPU_X86) || defined(NVCPU_X86_64)
   #define WRITE_COMBINE_FLUSH()    asm volatile("sfence":::"memory")
 #elif defined(NVCPU_PPC64LE)
@@ -156,9 +141,15 @@ void nv_drm_unlock_user_pages(unsigned long  pages_count, struct page **pages)
 #define VM_USERMAP 0
 #endif
 
-void *nv_drm_vmap(struct page **pages, unsigned long pages_count)
+void *nv_drm_vmap(struct page **pages, unsigned long pages_count, bool cached)
 {
-    return vmap(pages, pages_count, VM_USERMAP, PAGE_KERNEL);
+    pgprot_t prot = PAGE_KERNEL;
+
+    if (!cached) {
+        prot = pgprot_noncached(PAGE_KERNEL);
+    }
+
+    return vmap(pages, pages_count, VM_USERMAP, prot);
 }
 
 void nv_drm_vunmap(void *address)
@@ -230,19 +221,8 @@ unsigned long nv_drm_timeout_from_ms(NvU64 relative_timeout_ms)
     return jiffies + msecs_to_jiffies(relative_timeout_ms);
 }
 
-bool nv_drm_del_timer_sync(nv_drm_timer *timer)
+int nv_drm_create_sync_file(struct dma_fence *fence)
 {
-    if (del_timer_sync(&timer->kernel_timer)) {
-        return true;
-    } else {
-        return false;
-    }
-}
-
-#if defined(NV_DRM_FENCE_AVAILABLE)
-int nv_drm_create_sync_file(nv_dma_fence_t *fence)
-{
-#if defined(NV_LINUX_SYNC_FILE_H_PRESENT)
     struct sync_file *sync;
     int fd = get_unused_fd_flags(O_CLOEXEC);
 
@@ -261,20 +241,12 @@ int nv_drm_create_sync_file(nv_dma_fence_t *fence)
     fd_install(fd, sync->file);
 
     return fd;
-#else /* defined(NV_LINUX_SYNC_FILE_H_PRESENT) */
-    return -EINVAL;
-#endif  /* defined(NV_LINUX_SYNC_FILE_H_PRESENT) */
 }
 
-nv_dma_fence_t *nv_drm_sync_file_get_fence(int fd)
+struct dma_fence *nv_drm_sync_file_get_fence(int fd)
 {
-#if defined(NV_SYNC_FILE_GET_FENCE_PRESENT)
     return sync_file_get_fence(fd);
-#else /* defined(NV_SYNC_FILE_GET_FENCE_PRESENT) */
-    return NULL;
-#endif  /* defined(NV_SYNC_FILE_GET_FENCE_PRESENT) */
 }
-#endif /* defined(NV_DRM_FENCE_AVAILABLE) */
 
 void nv_drm_yield(void)
 {

kernel-open/nvidia-drm/nvidia-drm-os-interface.h

@@ -1,5 +1,5 @@
 /*
- * Copyright (c) 2015, NVIDIA CORPORATION. All rights reserved.
+ * Copyright (c) 2015-2025, NVIDIA CORPORATION. All rights reserved.
  *
  * Permission is hereby granted, free of charge, to any person obtaining a
  * copy of this software and associated documentation files (the "Software"),
@@ -29,9 +29,7 @@
 
 #if defined(NV_DRM_AVAILABLE)
 
-#if defined(NV_DRM_FENCE_AVAILABLE)
-#include "nvidia-dma-fence-helper.h"
-#endif
+#include "linux/dma-fence.h"
 
 #if defined(NV_LINUX) || defined(NV_BSD)
 #include "nv-kthread-q.h"
@@ -58,15 +56,11 @@ typedef struct nv_timer nv_drm_timer;
 #error "Need to define kernel timer callback primitives for this OS"
 #endif
 
-#if defined(NV_DRM_FBDEV_GENERIC_SETUP_PRESENT) && defined(NV_DRM_APERTURE_REMOVE_CONFLICTING_PCI_FRAMEBUFFERS_PRESENT)
-#define NV_DRM_FBDEV_GENERIC_AVAILABLE
-#endif
-
 struct page;
 
 /* Set to true when the atomic modeset feature is enabled. */
 extern bool nv_drm_modeset_module_param;
-#if defined(NV_DRM_FBDEV_GENERIC_AVAILABLE)
+#if defined(NV_DRM_FBDEV_AVAILABLE)
 /* Set to true when the nvidia-drm driver should install a framebuffer device */
 extern bool nv_drm_fbdev_module_param;
 #endif
@@ -75,8 +69,6 @@ void *nv_drm_calloc(size_t nmemb, size_t size);
 
 void nv_drm_free(void *ptr);
 
-char *nv_drm_asprintf(const char *fmt, ...);
-
 void nv_drm_write_combine_flush(void);
 
 int nv_drm_lock_user_pages(unsigned long address,
@@ -84,7 +76,7 @@ int nv_drm_lock_user_pages(unsigned long address,
 
 void nv_drm_unlock_user_pages(unsigned long  pages_count, struct page **pages);
 
-void *nv_drm_vmap(struct page **pages, unsigned long pages_count);
+void *nv_drm_vmap(struct page **pages, unsigned long pages_count, bool cached);
 
 void nv_drm_vunmap(void *address);
 
@@ -105,17 +97,13 @@ void nv_drm_timer_setup(nv_drm_timer *timer,
 
 void nv_drm_mod_timer(nv_drm_timer *timer, unsigned long relative_timeout_ms);
 
-bool nv_drm_del_timer_sync(nv_drm_timer *timer);
-
 unsigned long nv_drm_timer_now(void);
 
 unsigned long nv_drm_timeout_from_ms(NvU64 relative_timeout_ms);
 
-#if defined(NV_DRM_FENCE_AVAILABLE)
-int nv_drm_create_sync_file(nv_dma_fence_t *fence);
+int nv_drm_create_sync_file(struct dma_fence *fence);
 
-nv_dma_fence_t *nv_drm_sync_file_get_fence(int fd);
-#endif /* defined(NV_DRM_FENCE_AVAILABLE) */
+struct dma_fence *nv_drm_sync_file_get_fence(int fd);
 
 void nv_drm_yield(void);
 

kernel-open/nvidia-drm/nvidia-drm-priv.h

@@ -31,13 +31,8 @@
 #include <drm/drmP.h>
 #endif
 
-#if defined(NV_DRM_DRM_DEVICE_H_PRESENT)
 #include <drm/drm_device.h>
-#endif
-
-#if defined(NV_DRM_DRM_GEM_H_PRESENT)
 #include <drm/drm_gem.h>
-#endif
 
 #include "nvidia-drm-os-interface.h"
 
@@ -85,14 +80,20 @@
     DRM_DEBUG_DRIVER("[GPU ID 0x%08x] " __fmt,     \
                      __dev->gpu_info.gpu_id, ##__VA_ARGS__)
 
+enum nv_drm_input_color_space {
+    NV_DRM_INPUT_COLOR_SPACE_NONE,
+    NV_DRM_INPUT_COLOR_SPACE_SCRGB_LINEAR,
+    NV_DRM_INPUT_COLOR_SPACE_BT2100_PQ
+};
+
 struct nv_drm_device {
     nv_gpu_info_t gpu_info;
+    MIGDeviceId gpu_mig_device;
 
     struct drm_device *dev;
 
     struct NvKmsKapiDevice *pDevice;
 
-#if defined(NV_DRM_ATOMIC_MODESET_AVAILABLE)
     /*
      * Lock to protect drm-subsystem and fields of this structure
      * from concurrent access.
@@ -122,9 +123,7 @@ struct nv_drm_device {
     NvU8 genericPageKind;
     NvU8 pageKindGeneration;
     NvU8 sectorLayout;
-#if defined(NV_DRM_FORMAT_MODIFIERS_PRESENT)
     NvU64 modifiers[6 /* block linear */ + 1 /* linear */ + 1 /* terminator */];
-#endif
 
     struct delayed_work hotplug_event_work;
     atomic_t enable_event_handling;
@@ -137,39 +136,65 @@ struct nv_drm_device {
      */
     wait_queue_head_t flip_event_wq;
 
-#endif
-
-#if defined(NV_DRM_FENCE_AVAILABLE)
     NvU64 semsurf_stride;
     NvU64 semsurf_max_submitted_offset;
-#endif
 
     NvBool hasVideoMemory;
 
     NvBool supportsSyncpts;
+    NvBool requiresVrrSemaphores;
     NvBool subOwnershipGranted;
     NvBool hasFramebufferConsole;
 
-    /**
-     * @drmMasterChangedSinceLastAtomicCommit:
-     *
-     * This flag is set in nv_drm_master_set and reset after a completed atomic
-     * commit. It is used to restore or recommit state that is lost by the
-     * NvKms modeset owner change, such as the CRTC color management
-     * properties.
-     */
-    NvBool drmMasterChangedSinceLastAtomicCommit;
-
     struct drm_property *nv_out_fence_property;
     struct drm_property *nv_input_colorspace_property;
 
+    struct {
+        NvU32 count;
+        NvU32 next_index;
+    } display_semaphores;
+
 #if defined(NV_DRM_HAS_HDR_OUTPUT_METADATA)
     struct drm_property *nv_hdr_output_metadata_property;
 #endif
 
+    struct drm_property *nv_plane_lms_ctm_property;
+    struct drm_property *nv_plane_lms_to_itp_ctm_property;
+    struct drm_property *nv_plane_itp_to_lms_ctm_property;
+    struct drm_property *nv_plane_blend_ctm_property;
+
+    struct drm_property *nv_plane_degamma_tf_property;
+    struct drm_property *nv_plane_degamma_lut_property;
+    struct drm_property *nv_plane_degamma_lut_size_property;
+    struct drm_property *nv_plane_degamma_multiplier_property;
+
+    struct drm_property *nv_plane_tmo_lut_property;
+    struct drm_property *nv_plane_tmo_lut_size_property;
+
+    struct drm_property *nv_crtc_regamma_tf_property;
+    struct drm_property *nv_crtc_regamma_lut_property;
+    struct drm_property *nv_crtc_regamma_lut_size_property;
+    struct drm_property *nv_crtc_regamma_divisor_property;
+
     struct nv_drm_device *next;
+
+    NvU64 vtFbBaseAddress;
+    NvU64 vtFbSize;
 };
 
+static inline NvU32 nv_drm_next_display_semaphore(
+    struct nv_drm_device *nv_dev)
+{
+    NvU32 current_index = nv_dev->display_semaphores.next_index++;
+
+    if (nv_dev->display_semaphores.next_index >=
+        nv_dev->display_semaphores.count) {
+        nv_dev->display_semaphores.next_index = 0;
+    }
+
+    return current_index;
+}
+
 static inline struct nv_drm_device *to_nv_device(
     struct drm_device *dev)
 {

kernel-open/nvidia-drm/nvidia-drm-sources.mk

@@ -30,77 +30,48 @@ NVIDIA_DRM_SOURCES += nvidia-drm/nvidia-drm-os-interface.c
 # Register the conftests needed by nvidia-drm.ko
 #
 
-NV_CONFTEST_GENERIC_COMPILE_TESTS += drm_available
-NV_CONFTEST_GENERIC_COMPILE_TESTS += drm_atomic_available
 NV_CONFTEST_GENERIC_COMPILE_TESTS += is_export_symbol_gpl_refcount_inc
 NV_CONFTEST_GENERIC_COMPILE_TESTS += is_export_symbol_gpl_refcount_dec_and_test
 NV_CONFTEST_GENERIC_COMPILE_TESTS += drm_alpha_blending_available
 NV_CONFTEST_GENERIC_COMPILE_TESTS += is_export_symbol_present_drm_gem_prime_fd_to_handle
 NV_CONFTEST_GENERIC_COMPILE_TESTS += is_export_symbol_present_drm_gem_prime_handle_to_fd
+NV_CONFTEST_GENERIC_COMPILE_TESTS += is_export_symbol_present_timer_delete_sync
+NV_CONFTEST_GENERIC_COMPILE_TESTS += is_export_symbol_gpl___vma_start_write
 
-NV_CONFTEST_FUNCTION_COMPILE_TESTS += drm_dev_unref
-NV_CONFTEST_FUNCTION_COMPILE_TESTS += drm_reinit_primary_mode_group
 NV_CONFTEST_FUNCTION_COMPILE_TESTS += get_user_pages_remote
 NV_CONFTEST_FUNCTION_COMPILE_TESTS += get_user_pages
 NV_CONFTEST_FUNCTION_COMPILE_TESTS += pin_user_pages_remote
 NV_CONFTEST_FUNCTION_COMPILE_TESTS += pin_user_pages
-NV_CONFTEST_FUNCTION_COMPILE_TESTS += drm_gem_object_lookup
-NV_CONFTEST_FUNCTION_COMPILE_TESTS += drm_atomic_state_ref_counting
 NV_CONFTEST_FUNCTION_COMPILE_TESTS += drm_driver_has_gem_prime_res_obj
-NV_CONFTEST_FUNCTION_COMPILE_TESTS += drm_atomic_helper_connector_dpms
 NV_CONFTEST_FUNCTION_COMPILE_TESTS += drm_connector_funcs_have_mode_in_name
 NV_CONFTEST_FUNCTION_COMPILE_TESTS += drm_connector_has_vrr_capable_property
 NV_CONFTEST_FUNCTION_COMPILE_TESTS += vmf_insert_pfn
-NV_CONFTEST_FUNCTION_COMPILE_TESTS += drm_framebuffer_get
-NV_CONFTEST_FUNCTION_COMPILE_TESTS += drm_gem_object_get
-NV_CONFTEST_FUNCTION_COMPILE_TESTS += drm_dev_put
-NV_CONFTEST_FUNCTION_COMPILE_TESTS += drm_format_num_planes
 NV_CONFTEST_FUNCTION_COMPILE_TESTS += drm_connector_for_each_possible_encoder
-NV_CONFTEST_FUNCTION_COMPILE_TESTS += drm_rotation_available
-NV_CONFTEST_FUNCTION_COMPILE_TESTS += drm_vma_offset_exact_lookup_locked
 NV_CONFTEST_FUNCTION_COMPILE_TESTS += drm_gem_object_put_unlocked
 NV_CONFTEST_FUNCTION_COMPILE_TESTS += nvhost_dma_fence_unpack
 NV_CONFTEST_FUNCTION_COMPILE_TESTS += list_is_first
-NV_CONFTEST_FUNCTION_COMPILE_TESTS += timer_setup
-NV_CONFTEST_FUNCTION_COMPILE_TESTS += dma_fence_set_error
-NV_CONFTEST_FUNCTION_COMPILE_TESTS += fence_set_error
-NV_CONFTEST_FUNCTION_COMPILE_TESTS += sync_file_get_fence
+NV_CONFTEST_FUNCTION_COMPILE_TESTS += drm_aperture_remove_conflicting_framebuffers
 NV_CONFTEST_FUNCTION_COMPILE_TESTS += drm_aperture_remove_conflicting_pci_framebuffers
+NV_CONFTEST_FUNCTION_COMPILE_TESTS += aperture_remove_conflicting_devices
+NV_CONFTEST_FUNCTION_COMPILE_TESTS += aperture_remove_conflicting_pci_devices
 NV_CONFTEST_FUNCTION_COMPILE_TESTS += drm_fbdev_generic_setup
+NV_CONFTEST_FUNCTION_COMPILE_TESTS += drm_fbdev_ttm_setup
+NV_CONFTEST_FUNCTION_COMPILE_TESTS += drm_client_setup
 NV_CONFTEST_FUNCTION_COMPILE_TESTS += drm_connector_attach_hdr_output_metadata_property
-NV_CONFTEST_FUNCTION_COMPILE_TESTS += drm_helper_crtc_enable_color_mgmt
-NV_CONFTEST_FUNCTION_COMPILE_TESTS += drm_crtc_enable_color_mgmt
+NV_CONFTEST_FUNCTION_COMPILE_TESTS += drm_plane_create_color_properties
 NV_CONFTEST_FUNCTION_COMPILE_TESTS += drm_atomic_helper_legacy_gamma_set
+NV_CONFTEST_FUNCTION_COMPILE_TESTS += vmf_insert_mixed
+NV_CONFTEST_FUNCTION_COMPILE_TESTS += drm_gem_prime_mmap
 
-NV_CONFTEST_TYPE_COMPILE_TESTS += drm_bus_present
-NV_CONFTEST_TYPE_COMPILE_TESTS += drm_bus_has_bus_type
-NV_CONFTEST_TYPE_COMPILE_TESTS += drm_bus_has_get_irq
-NV_CONFTEST_TYPE_COMPILE_TESTS += drm_bus_has_get_name
-NV_CONFTEST_TYPE_COMPILE_TESTS += drm_driver_has_device_list
 NV_CONFTEST_TYPE_COMPILE_TESTS += drm_driver_has_legacy_dev_list
-NV_CONFTEST_TYPE_COMPILE_TESTS += drm_driver_has_set_busid
-NV_CONFTEST_TYPE_COMPILE_TESTS += drm_crtc_state_has_connectors_changed
-NV_CONFTEST_TYPE_COMPILE_TESTS += drm_init_function_args
-NV_CONFTEST_TYPE_COMPILE_TESTS += drm_helper_mode_fill_fb_struct
-NV_CONFTEST_TYPE_COMPILE_TESTS += drm_master_drop_has_from_release_arg
-NV_CONFTEST_TYPE_COMPILE_TESTS += drm_driver_unload_has_int_return_type
-NV_CONFTEST_TYPE_COMPILE_TESTS += vm_fault_has_address
 NV_CONFTEST_TYPE_COMPILE_TESTS += vm_ops_fault_removed_vma_arg
-NV_CONFTEST_TYPE_COMPILE_TESTS += drm_atomic_helper_crtc_destroy_state_has_crtc_arg
-NV_CONFTEST_TYPE_COMPILE_TESTS += drm_atomic_helper_plane_destroy_state_has_plane_arg
-NV_CONFTEST_TYPE_COMPILE_TESTS += drm_mode_object_find_has_file_priv_arg
-NV_CONFTEST_TYPE_COMPILE_TESTS += dma_buf_owner
-NV_CONFTEST_TYPE_COMPILE_TESTS += drm_connector_list_iter
-NV_CONFTEST_TYPE_COMPILE_TESTS += drm_atomic_helper_swap_state_has_stall_arg
 NV_CONFTEST_TYPE_COMPILE_TESTS += drm_driver_prime_flag_present
 NV_CONFTEST_TYPE_COMPILE_TESTS += vm_fault_t
 NV_CONFTEST_TYPE_COMPILE_TESTS += drm_gem_object_has_resv
 NV_CONFTEST_TYPE_COMPILE_TESTS += drm_crtc_state_has_async_flip
 NV_CONFTEST_TYPE_COMPILE_TESTS += drm_crtc_state_has_pageflip_flags
 NV_CONFTEST_TYPE_COMPILE_TESTS += drm_crtc_state_has_vrr_enabled
-NV_CONFTEST_TYPE_COMPILE_TESTS += drm_format_modifiers_present
 NV_CONFTEST_TYPE_COMPILE_TESTS += mm_has_mmap_lock
-NV_CONFTEST_TYPE_COMPILE_TESTS += drm_vma_node_is_allowed_has_tag_arg
 NV_CONFTEST_TYPE_COMPILE_TESTS += drm_vma_offset_node_has_readonly
 NV_CONFTEST_TYPE_COMPILE_TESTS += drm_display_mode_has_vrefresh
 NV_CONFTEST_TYPE_COMPILE_TESTS += drm_driver_master_set_has_int_return_type
@@ -118,14 +89,20 @@ NV_CONFTEST_TYPE_COMPILE_TESTS += dma_resv_add_fence
 NV_CONFTEST_TYPE_COMPILE_TESTS += dma_resv_reserve_fences
 NV_CONFTEST_TYPE_COMPILE_TESTS += reservation_object_reserve_shared_has_num_fences_arg
 NV_CONFTEST_TYPE_COMPILE_TESTS += drm_connector_has_override_edid
-NV_CONFTEST_TYPE_COMPILE_TESTS += drm_master_has_leases
 NV_CONFTEST_TYPE_COMPILE_TESTS += drm_file_get_master
 NV_CONFTEST_TYPE_COMPILE_TESTS += drm_modeset_lock_all_end
-NV_CONFTEST_TYPE_COMPILE_TESTS += drm_connector_lookup
-NV_CONFTEST_TYPE_COMPILE_TESTS += drm_connector_put
 NV_CONFTEST_TYPE_COMPILE_TESTS += vm_area_struct_has_const_vm_flags
 NV_CONFTEST_TYPE_COMPILE_TESTS += drm_driver_has_dumb_destroy
 NV_CONFTEST_TYPE_COMPILE_TESTS += fence_ops_use_64bit_seqno
+NV_CONFTEST_TYPE_COMPILE_TESTS += drm_aperture_remove_conflicting_framebuffers_has_driver_arg
+NV_CONFTEST_TYPE_COMPILE_TESTS += drm_aperture_remove_conflicting_framebuffers_has_no_primary_arg
 NV_CONFTEST_TYPE_COMPILE_TESTS += drm_aperture_remove_conflicting_pci_framebuffers_has_driver_arg
 NV_CONFTEST_TYPE_COMPILE_TESTS += drm_mode_create_dp_colorspace_property_has_supported_colorspaces_arg
+NV_CONFTEST_TYPE_COMPILE_TESTS += drm_syncobj_features_present
 NV_CONFTEST_TYPE_COMPILE_TESTS += drm_unlocked_ioctl_flag_present
+NV_CONFTEST_TYPE_COMPILE_TESTS += drm_color_ctm_3x4_present
+NV_CONFTEST_TYPE_COMPILE_TESTS += drm_format_info_has_is_yuv
+NV_CONFTEST_TYPE_COMPILE_TESTS += drm_driver_has_gem_prime_mmap
+NV_CONFTEST_TYPE_COMPILE_TESTS += drm_output_poll_changed
+NV_CONFTEST_TYPE_COMPILE_TESTS += drm_driver_has_date
+NV_CONFTEST_TYPE_COMPILE_TESTS += drm_connector_helper_funcs_mode_valid_has_const_mode_arg

kernel-open/nvidia-drm/nvidia-drm-utils.c

@@ -20,18 +20,15 @@
  * DEALINGS IN THE SOFTWARE.
  */
 
-#include "nvidia-drm-conftest.h" /* NV_DRM_ATOMIC_MODESET_AVAILABLE */
+#include "nvidia-drm-conftest.h" /* NV_DRM_AVAILABLE */
 
-#if defined(NV_DRM_ATOMIC_MODESET_AVAILABLE)
+#if defined(NV_DRM_AVAILABLE)
 
 #if defined(NV_DRM_DRMP_H_PRESENT)
 #include <drm/drmP.h>
 #endif
 
-#if defined(NV_DRM_DRM_PLANE_H_PRESENT)
 #include <drm/drm_plane.h>
-#endif
-
 #include <drm/drm_modes.h>
 #include <uapi/drm/drm_fourcc.h>
 

kernel-open/nvidia-drm/nvidia-drm-utils.h

@@ -25,7 +25,7 @@
 
 #include "nvidia-drm-conftest.h"
 
-#if defined(NV_DRM_ATOMIC_MODESET_AVAILABLE)
+#if defined(NV_DRM_AVAILABLE)
 
 #include "nvkms-kapi.h"
 
@@ -49,6 +49,6 @@ void nvkms_display_mode_to_drm_mode(
 void drm_mode_to_nvkms_display_mode(const struct drm_display_mode *src,
                                     struct NvKmsKapiDisplayMode *dst);
 
-#endif /* NV_DRM_ATOMIC_MODESET_AVAILABLE */
+#endif /* NV_DRM_AVAILABLE */
 
 #endif /* __NVIDIA_DRM_UTILS_H__ */

kernel-open/nvidia-modeset/nv-kthread-q.c

@@ -1,5 +1,5 @@
 /*
- * SPDX-FileCopyrightText: Copyright (c) 2016 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+ * SPDX-FileCopyrightText: Copyright (c) 2016-2024 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
  * SPDX-License-Identifier: MIT
  *
  * Permission is hereby granted, free of charge, to any person obtaining a
@@ -29,12 +29,7 @@
 #include <linux/completion.h>
 #include <linux/module.h>
 #include <linux/mm.h>
-
-#if defined(NV_LINUX_BUG_H_PRESENT)
-    #include <linux/bug.h>
-#else
-    #include <asm/bug.h>
-#endif
+#include <linux/bug.h>
 
 // Today's implementation is a little simpler and more limited than the
 // API description allows for in nv-kthread-q.h. Details include:
@@ -176,7 +171,7 @@ static struct task_struct *thread_create_on_node(int (*threadfn)(void *data),
 {
 
     unsigned i, j;
-    const static unsigned attempts = 3;
+    static const unsigned attempts = 3;
     struct task_struct *thread[3];
 
     for (i = 0;; i++) {
@@ -201,7 +196,7 @@ static struct task_struct *thread_create_on_node(int (*threadfn)(void *data),
 
         // Ran out of attempts - return thread even if its stack may not be
         // allocated on the preferred node
-        if ((i == (attempts - 1)))
+        if (i == (attempts - 1))
             break;
 
         // Get the NUMA node where the first page of the stack is resident. If

kernel-open/nvidia-modeset/nvidia-modeset-linux.c

@@ -1,5 +1,5 @@
 /*
- * SPDX-FileCopyrightText: Copyright (c) 2015-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+ * SPDX-FileCopyrightText: Copyright (c) 2015-2025 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
  * SPDX-License-Identifier: MIT
  *
  * Permission is hereby granted, free of charge, to any person obtaining a
@@ -53,10 +53,15 @@
 #include "nv-procfs.h"
 #include "nv-kthread-q.h"
 #include "nv-time.h"
+#include "nv-timer.h"
 #include "nv-lock.h"
 #include "nv-chardev-numbers.h"
 
-#if !defined(CONFIG_RETPOLINE)
+/*
+ * Commit aefb2f2e619b ("x86/bugs: Rename CONFIG_RETPOLINE =>
+ * CONFIG_MITIGATION_RETPOLINE) in v6.8 renamed CONFIG_RETPOLINE.
+ */
+#if !defined(CONFIG_RETPOLINE) && !defined(CONFIG_MITIGATION_RETPOLINE)
 #include "nv-retpoline.h"
 #endif
 
@@ -73,15 +78,21 @@ module_param_named(disable_hdmi_frl, disable_hdmi_frl, bool, 0400);
 static bool disable_vrr_memclk_switch = false;
 module_param_named(disable_vrr_memclk_switch, disable_vrr_memclk_switch, bool, 0400);
 
-static bool hdmi_deepcolor = false;
+static bool hdmi_deepcolor = true;
 module_param_named(hdmi_deepcolor, hdmi_deepcolor, bool, 0400);
 
-static bool vblank_sem_control = false;
+static bool vblank_sem_control = true;
 module_param_named(vblank_sem_control, vblank_sem_control, bool, 0400);
 
 static bool opportunistic_display_sync = true;
 module_param_named(opportunistic_display_sync, opportunistic_display_sync, bool, 0400);
 
+static enum NvKmsDebugForceColorSpace debug_force_color_space = NVKMS_DEBUG_FORCE_COLOR_SPACE_NONE;
+module_param_named(debug_force_color_space, debug_force_color_space, uint, 0400);
+
+static bool enable_overlay_layers = true;
+module_param_named(enable_overlay_layers, enable_overlay_layers, bool, 0400);
+
 /* These parameters are used for fault injection tests.  Normally the defaults
  * should be used. */
 MODULE_PARM_DESC(fail_malloc, "Fail the Nth call to nvkms_alloc");
@@ -92,19 +103,50 @@ MODULE_PARM_DESC(malloc_verbose, "Report information about malloc calls on modul
 static bool malloc_verbose = false;
 module_param_named(malloc_verbose, malloc_verbose, bool, 0400);
 
+MODULE_PARM_DESC(conceal_vrr_caps, 
+                 "Conceal all display VRR capabilities");
+static bool conceal_vrr_caps = false;
+module_param_named(conceal_vrr_caps, conceal_vrr_caps, bool, 0400);
+
+/* Fail allocating the RM core channel for NVKMS using the i-th method (see
+ * FailAllocCoreChannelMethod). Failures not using the i-th method are ignored. */
+MODULE_PARM_DESC(fail_alloc_core_channel, "Control testing for hardware core channel allocation failure");
+static int fail_alloc_core_channel_method = -1;
+module_param_named(fail_alloc_core_channel, fail_alloc_core_channel_method, int, 0400);
+
 #if NVKMS_CONFIG_FILE_SUPPORTED
 /* This parameter is used to find the dpy override conf file */
 #define NVKMS_CONF_FILE_SPECIFIED (nvkms_conf != NULL)
 
 MODULE_PARM_DESC(config_file,
-                 "Path to the nvidia-modeset configuration file "
-                 "(default: disabled)");
+                 "Path to the nvidia-modeset configuration file (default: disabled)");
 static char *nvkms_conf = NULL;
 module_param_named(config_file, nvkms_conf, charp, 0400);
 #endif
 
 static atomic_t nvkms_alloc_called_count;
 
+NvBool nvkms_test_fail_alloc_core_channel(
+    enum FailAllocCoreChannelMethod method
+)
+{
+    if (method != fail_alloc_core_channel_method) {
+        // don't fail if it's not the currently specified method
+        return NV_FALSE;
+    } 
+
+    printk(KERN_INFO NVKMS_LOG_PREFIX 
+        "Failing core channel allocation using method %d", 
+        fail_alloc_core_channel_method);    
+
+    return NV_TRUE;
+}
+
+NvBool nvkms_conceal_vrr_caps(void)
+{
+    return conceal_vrr_caps;
+}
+
 NvBool nvkms_output_rounding_fix(void)
 {
     return output_rounding_fix;
@@ -135,11 +177,257 @@ NvBool nvkms_opportunistic_display_sync(void)
     return opportunistic_display_sync;
 }
 
+enum NvKmsDebugForceColorSpace nvkms_debug_force_color_space(void)
+{
+    if (debug_force_color_space >= NVKMS_DEBUG_FORCE_COLOR_SPACE_MAX) {
+        return NVKMS_DEBUG_FORCE_COLOR_SPACE_NONE;
+    }
+    return debug_force_color_space;
+}
+
+NvBool nvkms_enable_overlay_layers(void)
+{
+    return enable_overlay_layers;
+}
+
+NvBool nvkms_kernel_supports_syncpts(void)
+{
+/*
+ * Note this only checks that the kernel has the prerequisite
+ * support for syncpts; callers must also check that the hardware
+ * supports syncpts.
+ */
+#if (defined(CONFIG_TEGRA_GRHOST) || defined(NV_LINUX_HOST1X_NEXT_H_PRESENT))
+    return NV_TRUE;
+#else
+    return NV_FALSE;
+#endif
+}
+
 #define NVKMS_SYNCPT_STUBS_NEEDED
 
 /*************************************************************************
  * NVKMS interface for nvhost unit for sync point APIs.
  *************************************************************************/
+#if defined(NV_LINUX_NVHOST_H_PRESENT) && defined(CONFIG_TEGRA_GRHOST)
+
+#undef NVKMS_SYNCPT_STUBS_NEEDED
+
+#include <linux/nvhost.h>
+
+NvBool nvkms_syncpt_op(
+    enum NvKmsSyncPtOp op,
+    NvKmsSyncPtOpParams *params)
+{
+    struct platform_device *pdev = nvhost_get_default_device();
+
+    switch (op) {
+
+    case NVKMS_SYNCPT_OP_ALLOC:
+        params->alloc.id = nvhost_get_syncpt_client_managed(
+                                pdev, params->alloc.syncpt_name);
+        break;
+
+    case NVKMS_SYNCPT_OP_PUT:
+        nvhost_syncpt_put_ref_ext(pdev, params->put.id);
+        break;
+
+    case NVKMS_SYNCPT_OP_FD_TO_ID_AND_THRESH: {
+
+        struct nvhost_fence *fence;
+        NvU32 id, thresh;
+
+        fence = nvhost_fence_get(params->fd_to_id_and_thresh.fd);
+        if (fence == NULL) {
+            return NV_FALSE;
+        }
+
+        if (nvhost_fence_num_pts(fence) > 1) {
+            /*! Syncpoint fence fd contains more than one syncpoint */
+            nvhost_fence_put(fence);
+            return NV_FALSE;
+        }
+
+        if (nvhost_fence_get_pt(fence, 0, &id, &thresh) != 0) {
+            nvhost_fence_put(fence);
+            return NV_FALSE;
+        }
+
+        params->fd_to_id_and_thresh.id = id;
+        params->fd_to_id_and_thresh.thresh = thresh;
+
+        nvhost_fence_put(fence);
+
+        break;
+    }
+
+    case NVKMS_SYNCPT_OP_ID_AND_THRESH_TO_FD:
+        nvhost_syncpt_create_fence_single_ext(
+                pdev,
+                params->id_and_thresh_to_fd.id,
+                params->id_and_thresh_to_fd.thresh,
+                "nvkms-fence",
+                &params->id_and_thresh_to_fd.fd);
+        break;
+
+    case NVKMS_SYNCPT_OP_READ_MINVAL:
+        params->read_minval.minval =
+                nvhost_syncpt_read_minval(pdev, params->read_minval.id);
+        break;
+
+    }
+
+    return NV_TRUE;
+}
+
+#elif defined(NV_LINUX_HOST1X_NEXT_H_PRESENT) && defined(NV_LINUX_NVHOST_H_PRESENT)
+
+#include <linux/dma-fence.h>
+#include <linux/file.h>
+#include <linux/host1x-next.h>
+#include <linux/sync_file.h>
+
+/*
+ * If the host1x.h header is present, then we are using the upstream
+ * host1x driver and so make sure CONFIG_TEGRA_HOST1X is defined to pick
+ * up the correct prototypes/definitions in nvhost.h.
+ */
+#define CONFIG_TEGRA_HOST1X
+
+#include <linux/nvhost.h>
+
+#undef NVKMS_SYNCPT_STUBS_NEEDED
+
+NvBool nvkms_syncpt_op(
+    enum NvKmsSyncPtOp op,
+    NvKmsSyncPtOpParams *params)
+{
+    struct host1x_syncpt *host1x_sp;
+    struct platform_device *pdev;
+    struct host1x *host1x;
+
+    pdev = nvhost_get_default_device();
+    if (pdev == NULL) {
+        nvkms_log(NVKMS_LOG_LEVEL_ERROR, NVKMS_LOG_PREFIX,
+                  "Failed to get nvhost default pdev");
+         return NV_FALSE;
+    }
+
+    host1x = nvhost_get_host1x(pdev);
+    if (host1x == NULL) {
+        nvkms_log(NVKMS_LOG_LEVEL_ERROR, NVKMS_LOG_PREFIX,
+                  "Failed to get host1x");
+        return  NV_FALSE;
+    }
+
+    switch (op) {
+
+    case NVKMS_SYNCPT_OP_ALLOC:
+        host1x_sp = host1x_syncpt_alloc(host1x,
+                                        HOST1X_SYNCPT_CLIENT_MANAGED,
+                                        params->alloc.syncpt_name);
+        if (host1x_sp == NULL) {
+            return NV_FALSE;
+        }
+
+        params->alloc.id = host1x_syncpt_id(host1x_sp);
+        break;
+
+    case NVKMS_SYNCPT_OP_PUT:
+        host1x_sp = host1x_syncpt_get_by_id_noref(host1x, params->put.id);
+        if (host1x_sp == NULL) {
+            return NV_FALSE;
+        }
+
+        host1x_syncpt_put(host1x_sp);
+        break;
+
+    case NVKMS_SYNCPT_OP_FD_TO_ID_AND_THRESH: {
+
+        struct dma_fence *f;
+        NvU32 id, thresh;
+        int err;
+
+        f = sync_file_get_fence(params->fd_to_id_and_thresh.fd);
+        if (f == NULL) {
+            return NV_FALSE;
+        }
+
+        if (dma_fence_is_array(f)) {
+            struct dma_fence_array *array = to_dma_fence_array(f);
+
+            if (array->num_fences > 1) {
+                /* Syncpoint fence fd contains more than one syncpoint */
+                dma_fence_put(f);
+                return NV_FALSE;
+            }
+
+            f = array->fences[0];
+        }
+
+        err = host1x_fence_extract(f, &id, &thresh);
+        dma_fence_put(f);
+
+        if (err < 0) {
+            return NV_FALSE;
+        }
+
+        params->fd_to_id_and_thresh.id = id;
+        params->fd_to_id_and_thresh.thresh = thresh;
+
+        break;
+    }
+
+    case NVKMS_SYNCPT_OP_ID_AND_THRESH_TO_FD: {
+
+        struct sync_file *file;
+        struct dma_fence *f;
+        int fd;
+
+        host1x_sp = host1x_syncpt_get_by_id_noref(host1x,
+                                params->id_and_thresh_to_fd.id);
+        if (host1x_sp == NULL) {
+            return NV_FALSE;
+        }
+
+        f = host1x_fence_create(host1x_sp,
+                                params->id_and_thresh_to_fd.thresh, true);
+        if (IS_ERR(f)) {
+            return NV_FALSE;
+        }
+
+        fd = get_unused_fd_flags(O_CLOEXEC);
+        if (fd < 0) {
+            dma_fence_put(f);
+            return NV_FALSE;
+        }
+
+        file = sync_file_create(f);
+        dma_fence_put(f);
+
+        if (!file) {
+            return NV_FALSE;
+        }
+
+        fd_install(fd, file->file);
+
+        params->id_and_thresh_to_fd.fd = fd;
+        break;
+    }
+
+    case NVKMS_SYNCPT_OP_READ_MINVAL:
+        host1x_sp = host1x_syncpt_get_by_id_noref(host1x, params->read_minval.id);
+        if (host1x_sp == NULL) {
+            return NV_FALSE;
+        }
+
+        params->read_minval.minval = host1x_syncpt_read(host1x_sp);
+        break;
+    }
+
+    return NV_TRUE;
+}
+#endif
 
 #ifdef NVKMS_SYNCPT_STUBS_NEEDED
 /* Unsupported STUB for nvkms_syncpt APIs */
@@ -499,8 +787,9 @@ nvkms_event_queue_changed(nvkms_per_open_handle_t *pOpenKernel,
 
 static void nvkms_suspend(NvU32 gpuId)
 {
+    nvKmsKapiSuspendResume(NV_TRUE /* suspend */);
+
     if (gpuId == 0) {
-        nvKmsKapiSuspendResume(NV_TRUE /* suspend */);
         nvkms_write_lock_pm_lock();
     }
 
@@ -517,8 +806,9 @@ static void nvkms_resume(NvU32 gpuId)
 
     if (gpuId == 0) {
         nvkms_write_unlock_pm_lock();
-        nvKmsKapiSuspendResume(NV_FALSE /* suspend */);
     }
+
+    nvKmsKapiSuspendResume(NV_FALSE /* suspend */);
 }
 
 
@@ -678,7 +968,7 @@ static void nvkms_kthread_q_callback(void *arg)
      * pending timers and than waiting for workqueue callbacks.
      */
     if (timer->kernel_timer_created) {
-        del_timer_sync(&timer->kernel_timer);
+        nv_timer_delete_sync(&timer->kernel_timer);
     }
 
     /*
@@ -748,12 +1038,6 @@ inline static void nvkms_timer_callback_typed_data(struct timer_list *timer)
     _nvkms_timer_callback_internal(nvkms_timer);
 }
 
-inline static void nvkms_timer_callback_anon_data(unsigned long arg)
-{
-    struct nvkms_timer_t *nvkms_timer = (struct nvkms_timer_t *) arg;
-    _nvkms_timer_callback_internal(nvkms_timer);
-}
-
 static void
 nvkms_init_timer(struct nvkms_timer_t *timer, nvkms_timer_proc_t *proc,
                  void *dataPtr, NvU32 dataU32, NvBool isRefPtr, NvU64 usec)
@@ -786,13 +1070,7 @@ nvkms_init_timer(struct nvkms_timer_t *timer, nvkms_timer_proc_t *proc,
         timer->kernel_timer_created = NV_FALSE;
         nvkms_queue_work(&nvkms_kthread_q, &timer->nv_kthread_q_item);
     } else {
-#if defined(NV_TIMER_SETUP_PRESENT)
         timer_setup(&timer->kernel_timer, nvkms_timer_callback_typed_data, 0);
-#else
-        init_timer(&timer->kernel_timer);
-        timer->kernel_timer.function = nvkms_timer_callback_anon_data;
-        timer->kernel_timer.data = (unsigned long) timer;
-#endif
 
         timer->kernel_timer_created = NV_TRUE;
         mod_timer(&timer->kernel_timer, jiffies + NVKMS_USECS_TO_JIFFIES(usec));
@@ -990,6 +1268,11 @@ nvkms_register_backlight(NvU32 gpu_id, NvU32 display_id, void *drv_priv,
 
 #if defined(NV_ACPI_VIDEO_BACKLIGHT_USE_NATIVE)
     if (!acpi_video_backlight_use_native()) {
+#if defined(NV_ACPI_VIDEO_REGISTER_BACKLIGHT)
+        nvkms_log(NVKMS_LOG_LEVEL_INFO, NVKMS_LOG_PREFIX,
+                  "ACPI reported no NVIDIA native backlight available; attempting to use ACPI backlight.");
+        acpi_video_register_backlight();
+#endif
         return NULL;
     }
 #endif
@@ -1064,8 +1347,9 @@ static void nvkms_kapi_event_kthread_q_callback(void *arg)
     nvKmsKapiHandleEventQueueChange(device);
 }
 
-struct nvkms_per_open *nvkms_open_common(enum NvKmsClientType type,
+static struct nvkms_per_open *nvkms_open_common(enum NvKmsClientType type,
                                          struct NvKmsKapiDevice *device,
+                                         NvBool interruptible,
                                          int *status)
 {
     struct nvkms_per_open *popen = NULL;
@@ -1079,10 +1363,13 @@ struct nvkms_per_open *nvkms_open_common(enum NvKmsClientType type,
 
     popen->type = type;
 
-    *status = down_interruptible(&nvkms_lock);
-
-    if (*status != 0) {
-        goto failed;
+    if (interruptible) {
+        *status = down_interruptible(&nvkms_lock);
+        if (*status != 0) {
+            goto failed;
+        }
+    } else {
+        down(&nvkms_lock);
     }
 
     popen->data = nvKmsOpen(current->tgid, type, popen);
@@ -1116,7 +1403,7 @@ failed:
     return NULL;
 }
 
-void nvkms_close_pm_locked(struct nvkms_per_open *popen)
+static void nvkms_close_pm_locked(struct nvkms_per_open *popen)
 {
     /*
      * Don't use down_interruptible(): we need to free resources
@@ -1179,18 +1466,22 @@ static void nvkms_close_popen(struct nvkms_per_open *popen)
     }
 }
 
-int nvkms_ioctl_common
+static int nvkms_ioctl_common
 (
     struct nvkms_per_open *popen,
-    NvU32 cmd, NvU64 address, const size_t size
+    NvU32 cmd, NvU64 address, const size_t size,
+    NvBool interruptible
 )
 {
-    int status;
     NvBool ret;
 
-    status = down_interruptible(&nvkms_lock);
-    if (status != 0) {
-        return status;
+    if (interruptible) {
+        int status = down_interruptible(&nvkms_lock);
+        if (status != 0) {
+            return status;
+        }
+    } else {
+        down(&nvkms_lock);
     }
 
     if (popen->data != NULL) {
@@ -1217,7 +1508,10 @@ struct nvkms_per_open* nvkms_open_from_kapi
     struct nvkms_per_open *ret;
 
     nvkms_read_lock_pm_lock();
-    ret = nvkms_open_common(NVKMS_CLIENT_KERNEL_SPACE, device, &status);
+    ret = nvkms_open_common(NVKMS_CLIENT_KERNEL_SPACE,
+                            device,
+                            NV_FALSE /* interruptible */,
+                            &status);
     nvkms_read_unlock_pm_lock();
 
     return ret;
@@ -1228,6 +1522,28 @@ void nvkms_close_from_kapi(struct nvkms_per_open *popen)
     nvkms_close_pm_unlocked(popen);
 }
 
+NvBool nvkms_ioctl_from_kapi_try_pmlock
+(
+    struct nvkms_per_open *popen,
+    NvU32 cmd, void *params_address, const size_t param_size
+)
+{
+    NvBool ret;
+
+    // XXX PM lock must be allowed to fail, see bug 4432810.
+    if (nvkms_read_trylock_pm_lock()) {
+        return NV_FALSE;
+    }
+
+    ret = nvkms_ioctl_common(popen,
+                             cmd,
+                             (NvU64)(NvUPtr)params_address, param_size,
+                             NV_FALSE /* interruptible */) == 0;
+    nvkms_read_unlock_pm_lock();
+
+    return ret;
+}
+
 NvBool nvkms_ioctl_from_kapi
 (
     struct nvkms_per_open *popen,
@@ -1239,7 +1555,8 @@ NvBool nvkms_ioctl_from_kapi
     nvkms_read_lock_pm_lock();
     ret = nvkms_ioctl_common(popen,
                              cmd,
-                             (NvU64)(NvUPtr)params_address, param_size) == 0;
+                             (NvU64)(NvUPtr)params_address, param_size,
+                             NV_FALSE /* interruptible */) == 0;
     nvkms_read_unlock_pm_lock();
 
     return ret;
@@ -1408,10 +1725,10 @@ static size_t nvkms_config_file_open
     struct inode *file_inode;
     size_t file_size = 0;
     size_t read_size = 0;
-#if defined(NV_KERNEL_READ_HAS_POINTER_POS_ARG)
     loff_t pos = 0;
-#endif
 
+    *buff = NULL;
+    
     if (!nvkms_fs_mounted()) {
         printk(KERN_ERR NVKMS_LOG_PREFIX "ERROR: Filesystems not mounted\n");
         return 0;
@@ -1435,6 +1752,11 @@ static size_t nvkms_config_file_open
         goto done;
     }
 
+    // Do not alloc a 0 sized buffer
+    if (file_size == 0) {
+        goto done;
+    }
+
     *buff = nvkms_alloc(file_size, NV_FALSE);
     if (*buff == NULL) {
         printk(KERN_WARNING NVKMS_LOG_PREFIX "WARNING: Out of memory\n");
@@ -1446,14 +1768,8 @@ static size_t nvkms_config_file_open
      * kernel_read_file for kernels >= 4.6
      */
     while ((read_size < file_size) && (i++ < NVKMS_READ_FILE_MAX_LOOPS)) {
-#if defined(NV_KERNEL_READ_HAS_POINTER_POS_ARG)
         ssize_t ret = kernel_read(file, *buff + read_size,
                                   file_size - read_size, &pos);
-#else
-        ssize_t ret = kernel_read(file, read_size,
-                                  *buff + read_size,
-                                  file_size - read_size);
-#endif
         if (ret <= 0) {
             break;
         }
@@ -1518,6 +1834,48 @@ NvBool nvKmsKapiGetFunctionsTable
 }
 EXPORT_SYMBOL(nvKmsKapiGetFunctionsTable);
 
+NvU32 nvKmsKapiF16ToF32(NvU16 a)
+{
+    return nvKmsKapiF16ToF32Internal(a);
+}
+EXPORT_SYMBOL(nvKmsKapiF16ToF32);
+
+NvU16 nvKmsKapiF32ToF16(NvU32 a)
+{
+    return nvKmsKapiF32ToF16Internal(a);
+}
+EXPORT_SYMBOL(nvKmsKapiF32ToF16);
+
+NvU32 nvKmsKapiF32Mul(NvU32 a, NvU32 b)
+{
+    return nvKmsKapiF32MulInternal(a, b);
+}
+EXPORT_SYMBOL(nvKmsKapiF32Mul);
+
+NvU32 nvKmsKapiF32Div(NvU32 a, NvU32 b)
+{
+    return nvKmsKapiF32DivInternal(a, b);
+}
+EXPORT_SYMBOL(nvKmsKapiF32Div);
+
+NvU32 nvKmsKapiF32Add(NvU32 a, NvU32 b)
+{
+    return nvKmsKapiF32AddInternal(a, b);
+}
+EXPORT_SYMBOL(nvKmsKapiF32Add);
+
+NvU32 nvKmsKapiF32ToUI32RMinMag(NvU32 a, NvBool exact)
+{
+    return nvKmsKapiF32ToUI32RMinMagInternal(a, exact);
+}
+EXPORT_SYMBOL(nvKmsKapiF32ToUI32RMinMag);
+
+NvU32 nvKmsKapiUI32ToF32(NvU32 a)
+{
+    return nvKmsKapiUI32ToF32Internal(a);
+}
+EXPORT_SYMBOL(nvKmsKapiUI32ToF32);
+
 /*************************************************************************
  * File operation callback functions.
  *************************************************************************/
@@ -1532,7 +1890,10 @@ static int nvkms_open(struct inode *inode, struct file *filp)
     }
 
     filp->private_data =
-        nvkms_open_common(NVKMS_CLIENT_USER_SPACE, NULL, &status);
+        nvkms_open_common(NVKMS_CLIENT_USER_SPACE,
+                          NULL,
+                          NV_TRUE /* interruptible */,
+                          &status);
 
     nvkms_read_unlock_pm_lock();
 
@@ -1591,7 +1952,8 @@ static int nvkms_ioctl(struct inode *inode, struct file *filp,
     status = nvkms_ioctl_common(popen,
                                 params.cmd,
                                 params.address,
-                                params.size);
+                                params.size,
+                                NV_TRUE /* interruptible */);
 
     nvkms_read_unlock_pm_lock();
 
@@ -1788,7 +2150,11 @@ restart:
              * completion, and we wait for queue completion with
              * nv_kthread_q_stop below.
              */
+#if !defined(NV_BSD) && NV_IS_EXPORT_SYMBOL_PRESENT_timer_delete_sync
+            if (timer_delete_sync(&timer->kernel_timer) == 1) {
+#else
             if (del_timer_sync(&timer->kernel_timer) == 1) {
+#endif
                 /*  We've deactivated timer so we need to clean after it */
                 list_del(&timer->timers_list);
 

kernel-open/nvidia-modeset/nvidia-modeset-os-interface.h

@@ -67,6 +67,14 @@ enum NvKmsSyncPtOp {
     NVKMS_SYNCPT_OP_READ_MINVAL,
 };
 
+enum NvKmsDebugForceColorSpace {
+    NVKMS_DEBUG_FORCE_COLOR_SPACE_NONE,
+    NVKMS_DEBUG_FORCE_COLOR_SPACE_RGB,
+    NVKMS_DEBUG_FORCE_COLOR_SPACE_YUV444,
+    NVKMS_DEBUG_FORCE_COLOR_SPACE_YUV422,
+    NVKMS_DEBUG_FORCE_COLOR_SPACE_MAX,
+};
+
 typedef struct {
 
     struct {
@@ -96,12 +104,21 @@ typedef struct {
     } read_minval;
 } NvKmsSyncPtOpParams;
 
+enum FailAllocCoreChannelMethod {
+    FAIL_ALLOC_CORE_CHANNEL_RM_SETUP_CORE_CHANNEL = 0,
+    FAIL_ALLOC_CORE_CHANNEL_RESTORE_CONSOLE = 1,
+};
+
+NvBool nvkms_test_fail_alloc_core_channel(enum FailAllocCoreChannelMethod method);
+NvBool nvkms_conceal_vrr_caps(void);
 NvBool nvkms_output_rounding_fix(void);
 NvBool nvkms_disable_hdmi_frl(void);
 NvBool nvkms_disable_vrr_memclk_switch(void);
 NvBool nvkms_hdmi_deepcolor(void);
 NvBool nvkms_vblank_sem_control(void);
 NvBool nvkms_opportunistic_display_sync(void);
+enum NvKmsDebugForceColorSpace nvkms_debug_force_color_space(void);
+NvBool nvkms_enable_overlay_layers(void);
 
 void   nvkms_call_rm    (void *ops);
 void*  nvkms_alloc      (size_t size,
@@ -304,6 +321,11 @@ NvU32 nvkms_enumerate_gpus(nv_gpu_info_t *gpu_info);
 
 NvBool nvkms_allow_write_combining(void);
 
+/*!
+ * Check if OS supports syncpoints.
+ */
+NvBool nvkms_kernel_supports_syncpts(void);
+
 /*!
  * Checks whether the fd is associated with an nvidia character device.
  */
@@ -328,6 +350,16 @@ NvBool nvkms_ioctl_from_kapi
     NvU32 cmd, void *params_address, const size_t params_size
 );
 
+/*!
+ * Like nvkms_ioctl_from_kapi, but return NV_FALSE instead of waiting if the
+ * power management read lock cannot be acquired.
+ */
+NvBool nvkms_ioctl_from_kapi_try_pmlock
+(
+    struct nvkms_per_open *popen,
+    NvU32 cmd, void *params_address, const size_t params_size
+);
+
 /*!
  * APIs for locking.
  */

kernel-open/nvidia-modeset/nvidia-modeset.Kbuild

@@ -40,9 +40,6 @@ NV_KERNEL_MODULE_TARGETS += $(NVIDIA_MODESET_KO)
 NVIDIA_MODESET_BINARY_OBJECT := $(src)/nvidia-modeset/nv-modeset-kernel.o_binary
 NVIDIA_MODESET_BINARY_OBJECT_O := nvidia-modeset/nv-modeset-kernel.o
 
-quiet_cmd_symlink = SYMLINK $@
-cmd_symlink = ln -sf $< $@
-
 targets += $(NVIDIA_MODESET_BINARY_OBJECT_O)
 
 $(obj)/$(NVIDIA_MODESET_BINARY_OBJECT_O): $(NVIDIA_MODESET_BINARY_OBJECT) FORCE
@@ -55,7 +52,7 @@ nvidia-modeset-y += $(NVIDIA_MODESET_BINARY_OBJECT_O)
 # Define nvidia-modeset.ko-specific CFLAGS.
 #
 
-NVIDIA_MODESET_CFLAGS += -I$(src)/nvidia-modeset
+NVIDIA_MODESET_CFLAGS += -I$(src)/nvidia-modeset -I$(src)/common/inc
 NVIDIA_MODESET_CFLAGS += -UDEBUG -U_DEBUG -DNDEBUG -DNV_BUILD_MODULE_INSTANCES=0
 
 # Some Android kernels prohibit driver use of filesystem functions like
@@ -97,11 +94,10 @@ $(obj)/$(NVIDIA_MODESET_INTERFACE): $(addprefix $(obj)/,$(NVIDIA_MODESET_OBJECTS
 
 NV_OBJECTS_DEPEND_ON_CONFTEST += $(NVIDIA_MODESET_OBJECTS)
 
-NV_CONFTEST_TYPE_COMPILE_TESTS += timespec64
 NV_CONFTEST_TYPE_COMPILE_TESTS += proc_ops
 NV_CONFTEST_FUNCTION_COMPILE_TESTS += pde_data
-NV_CONFTEST_FUNCTION_COMPILE_TESTS += timer_setup
 NV_CONFTEST_FUNCTION_COMPILE_TESTS += list_is_first
-NV_CONFTEST_FUNCTION_COMPILE_TESTS += ktime_get_real_ts64
 NV_CONFTEST_FUNCTION_COMPILE_TESTS += ktime_get_raw_ts64
 NV_CONFTEST_FUNCTION_COMPILE_TESTS += acpi_video_backlight_use_native
+NV_CONFTEST_FUNCTION_COMPILE_TESTS += acpi_video_register_backlight
+NV_CONFTEST_SYMBOL_COMPILE_TESTS += is_export_symbol_present_timer_delete_sync

kernel-open/nvidia-modeset/nvkms.h

@@ -110,4 +110,18 @@ NvBool nvKmsSetBacklight(NvU32 display_id, void *drv_priv, NvU32 brightness);
 
 NvBool nvKmsOpenDevHasSubOwnerPermissionOrBetter(const struct NvKmsPerOpenDev *pOpenDev);
 
+NvU32 nvKmsKapiF16ToF32Internal(NvU16 a);
+
+NvU16 nvKmsKapiF32ToF16Internal(NvU32 a);
+
+NvU32 nvKmsKapiF32MulInternal(NvU32 a, NvU32 b);
+
+NvU32 nvKmsKapiF32DivInternal(NvU32 a, NvU32 b);
+
+NvU32 nvKmsKapiF32AddInternal(NvU32 a, NvU32 b);
+
+NvU32 nvKmsKapiF32ToUI32RMinMagInternal(NvU32 a, NvBool exact);
+
+NvU32 nvKmsKapiUI32ToF32Internal(NvU32 a);
+
 #endif /* __NV_KMS_H__ */

kernel-open/nvidia-peermem/nv-p2p.h

@@ -1,5 +1,5 @@
 /*
- * SPDX-FileCopyrightText: Copyright (c) 2011-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+ * SPDX-FileCopyrightText: Copyright (c) 2011-2025 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
  * SPDX-License-Identifier: MIT
  *
  * Permission is hereby granted, free of charge, to any person obtaining a
@@ -132,11 +132,16 @@ struct nvidia_p2p_page {
     } registers;
 } nvidia_p2p_page_t;
 
-#define NVIDIA_P2P_PAGE_TABLE_VERSION   0x00010002
+#define NVIDIA_P2P_PAGE_TABLE_VERSION   0x00020000
 
 #define NVIDIA_P2P_PAGE_TABLE_VERSION_COMPATIBLE(p) \
     NVIDIA_P2P_VERSION_COMPATIBLE(p, NVIDIA_P2P_PAGE_TABLE_VERSION)
 
+/*
+ * Page Table Flags
+ */
+#define NVIDIA_P2P_PAGE_TABLE_FLAGS_CPU_CACHEABLE 0x1
+
 typedef
 struct nvidia_p2p_page_table {
     uint32_t version;
@@ -144,6 +149,7 @@ struct nvidia_p2p_page_table {
     struct nvidia_p2p_page **pages;
     uint32_t entries;
     uint8_t *gpu_uuid;
+    uint32_t flags;
 } nvidia_p2p_page_table_t;
 
 /*
@@ -153,6 +159,9 @@ struct nvidia_p2p_page_table {
  *
  *   This API only supports pinned, GPU-resident memory, such as that provided
  *   by cudaMalloc().
+ *   This API does not support Coherent Driver-based Memory Management(CDMM) mode.
+ *   CDMM allows coherent GPU memory to be managed by the driver and not the OS.
+ *   This is done by the driver not onlining the memory as a NUMA node.
  *
  *   This API may sleep.
  *
@@ -189,13 +198,19 @@ int nvidia_p2p_get_pages( uint64_t p2p_token, uint32_t va_space,
         struct nvidia_p2p_page_table **page_table,
         void (*free_callback)(void *data), void *data);
 
+/*
+ * Flags to be used with persistent APIs
+ */
+#define NVIDIA_P2P_FLAGS_DEFAULT            0
+#define NVIDIA_P2P_FLAGS_FORCE_BAR1_MAPPING 1
+
 /*
  * @brief
  *   Pin and make the pages underlying a range of GPU virtual memory
  *   accessible to a third-party device. The pages will persist until
  *   explicitly freed by nvidia_p2p_put_pages_persistent().
  *
- *   Persistent GPU memory mappings are not supported on PowerPC,
+ *   Persistent GPU memory mappings are not supported on
  *   MIG-enabled devices and vGPU.
  *
  *   This API only supports pinned, GPU-resident memory, such as that provided
@@ -212,7 +227,11 @@ int nvidia_p2p_get_pages( uint64_t p2p_token, uint32_t va_space,
  * @param[out]    page_table
  *   A pointer to an array of structures with P2P PTEs.
  * @param[in]     flags
- *   Must be set to zero for now.
+ *   NVIDIA_P2P_FLAGS_DEFAULT:
+ *     Default value to be used if no specific behavior is expected.
+ *   NVIDIA_P2P_FLAGS_FORCE_BAR1_MAPPING:
+ *     Force BAR1 mappings on certain coherent platforms,
+ *     subject to capability and supported topology.
  *
  * @return
  *    0           upon successful completion.
@@ -443,35 +462,19 @@ typedef struct nvidia_p2p_rsync_reg_info {
 
 /*
  * @brief
- *   Gets rsync (GEN-ID) register information associated with the supported
- *   NPUs.
- *
- *   The caller would use the returned information {GPU device, NPU device,
- *   socket-id, cluster-id} to pick the optimal generation registers to issue
- *   RSYNC (NVLink HW flush).
- *
- *   The interface allocates structures to return the information, hence
- *   nvidia_p2p_put_rsync_registers() must be called to free the structures.
- *
- *   Note, cluster-id is hardcoded to zero as early system configurations would
- *   only support cluster mode i.e. all devices would share the same cluster-id
- *   (0). In the future, appropriate kernel support would be needed to query
- *   cluster-ids.
- *
- * @param[out]     reg_info
- *   A pointer to the rsync reg info structure.
+ *   This interface is no longer supported and will always return an error.  It
+ *   is left in place (for now) to allow third-party callers to build without
+ *   any errors.
  *
  * @Returns
- *   0 Upon successful completion. Otherwise, returns negative value.
+ *   -ENODEV
  */
 int nvidia_p2p_get_rsync_registers(nvidia_p2p_rsync_reg_info_t **reg_info);
 
 /*
  * @brief
- *   Frees the structures allocated by nvidia_p2p_get_rsync_registers().
- *
- * @param[in]     reg_info
- *   A pointer to the rsync reg info structure.
+ *   This interface is no longer supported.  It is left in place (for now) to
+ *   allow third-party callers to build without any errors.
  */
 void nvidia_p2p_put_rsync_registers(nvidia_p2p_rsync_reg_info_t *reg_info);
 

kernel-open/nvidia-peermem/nvidia-peermem.c

@@ -60,6 +60,13 @@ static int peerdirect_support = NV_MEM_PEERDIRECT_SUPPORT_DEFAULT;
 module_param(peerdirect_support, int, S_IRUGO);
 MODULE_PARM_DESC(peerdirect_support, "Set level of support for Peer-direct, 0 [default] or 1 [legacy, for example MLNX_OFED 4.9 LTS]");
 
+enum {
+        NV_MEM_PERSISTENT_API_SUPPORT_LEGACY = 0,
+        NV_MEM_PERSISTENT_API_SUPPORT_DEFAULT = 1,
+};
+static int persistent_api_support = NV_MEM_PERSISTENT_API_SUPPORT_DEFAULT;
+module_param(persistent_api_support, int, S_IRUGO);
+MODULE_PARM_DESC(persistent_api_support, "Set level of support for persistent APIs, 0 [legacy] or 1 [default]");
 
 #define peer_err(FMT, ARGS...) printk(KERN_ERR "nvidia-peermem" " %s:%d ERROR " FMT, __FUNCTION__, __LINE__, ## ARGS)
 #ifdef NV_MEM_DEBUG
@@ -235,6 +242,61 @@ err:
     return 0;
 }
 
+/* acquire return code: 1 mine, 0 - not mine */
+static int nv_mem_acquire_nc(unsigned long addr, size_t size, void *peer_mem_private_data,
+                             char *peer_mem_name, void **client_context)
+{
+
+    int ret = 0;
+    struct nv_mem_context *nv_mem_context;
+
+    nv_mem_context = kzalloc(sizeof *nv_mem_context, GFP_KERNEL);
+    if (!nv_mem_context)
+        /* Error case handled as not mine */
+        return 0;
+
+    nv_mem_context->pad1 = NV_MEM_CONTEXT_MAGIC;
+    nv_mem_context->page_virt_start = addr & GPU_PAGE_MASK;
+    nv_mem_context->page_virt_end   = (addr + size + GPU_PAGE_SIZE - 1) & GPU_PAGE_MASK;
+    nv_mem_context->mapped_size  = nv_mem_context->page_virt_end - nv_mem_context->page_virt_start;
+    nv_mem_context->pad2 = NV_MEM_CONTEXT_MAGIC;
+
+#ifdef NVIDIA_P2P_CAP_GET_PAGES_PERSISTENT_API
+    ret = nvidia_p2p_get_pages_persistent(nv_mem_context->page_virt_start,
+                                          nv_mem_context->mapped_size,
+                                          &nv_mem_context->page_table, 0);
+#else
+    ret = nvidia_p2p_get_pages(0, 0, nv_mem_context->page_virt_start, nv_mem_context->mapped_size,
+                               &nv_mem_context->page_table, NULL, NULL);
+#endif
+    if (ret < 0)
+        goto err;
+
+#ifdef NVIDIA_P2P_CAP_GET_PAGES_PERSISTENT_API
+    ret = nvidia_p2p_put_pages_persistent(nv_mem_context->page_virt_start,
+                                          nv_mem_context->page_table, 0);
+#else
+    ret = nvidia_p2p_put_pages(0, 0, nv_mem_context->page_virt_start,
+                               nv_mem_context->page_table);
+#endif
+    if (ret < 0) {
+        peer_err("nv_mem_acquire -- error %d while calling nvidia_p2p_put_pages()\n", ret);
+        goto err;
+    }
+
+    /* 1 means mine */
+    *client_context = nv_mem_context;
+    __module_get(THIS_MODULE);
+    return 1;
+
+err:
+    memset(nv_mem_context, 0, sizeof(*nv_mem_context));
+    kfree(nv_mem_context);
+
+    /* Error case handled as not mine */
+    return 0;
+}
+
 static int nv_dma_map(struct sg_table *sg_head, void *context,
                       struct device *dma_device, int dmasync,
                       int *nmap)
@@ -470,7 +532,7 @@ static int nv_mem_get_pages_nc(unsigned long addr,
 }
 
 static struct peer_memory_client nv_mem_client_nc = {
-    .acquire        = nv_mem_acquire,
+    .acquire        = nv_mem_acquire_nc,
     .get_pages      = nv_mem_get_pages_nc,
     .dma_map        = nv_dma_map,
     .dma_unmap      = nv_dma_unmap,
@@ -479,32 +541,8 @@ static struct peer_memory_client nv_mem_client_nc = {
     .release        = nv_mem_release,
 };
 
-#endif /* NV_MLNX_IB_PEER_MEM_SYMBOLS_PRESENT */
-
-static int nv_mem_param_conf_check(void)
+static int nv_mem_legacy_client_init(void)
 {
-    int rc = 0;
-    switch (peerdirect_support) {
-    case NV_MEM_PEERDIRECT_SUPPORT_DEFAULT:
-    case NV_MEM_PEERDIRECT_SUPPORT_LEGACY:
-        break;
-    default:
-        peer_err("invalid peerdirect_support param value %d\n", peerdirect_support);
-        rc = -EINVAL;
-        break;
-    }
-    return rc;
-}
-
-static int __init nv_mem_client_init(void)
-{
-    int rc;
-    rc = nv_mem_param_conf_check();
-    if (rc) {
-        return rc;
-    }
-
-#if defined (NV_MLNX_IB_PEER_MEM_SYMBOLS_PRESENT)
     // off by one, to leave space for the trailing '1' which is flagging
     // the new client type
     BUG_ON(strlen(DRV_NAME) > IB_PEER_MEMORY_NAME_MAX-1);
@@ -533,19 +571,96 @@ static int __init nv_mem_client_init(void)
                          &mem_invalidate_callback);
     if (!reg_handle) {
         peer_err("nv_mem_client_init -- error while registering traditional client\n");
-        rc = -EINVAL;
-        goto out;
+        return -EINVAL;
+    }
+    return 0;
+}
+
+static int nv_mem_nc_client_init(void)
+{
+    // The nc client enables support for persistent pages.
+    if (persistent_api_support == NV_MEM_PERSISTENT_API_SUPPORT_LEGACY)
+    {
+        //
+        // If legacy behavior is forced via module param,
+        // both legacy and persistent clients are registered and are named
+        // "nv_mem"(legacy) and "nv_mem_nc"(persistent).
+        //
+        strcpy(nv_mem_client_nc.name, DRV_NAME "_nc");
+    }
+    else
+    {
+        //
+        // With default persistent behavior, the client name shall be "nv_mem"
+        // so that libraries can use the persistent client under the same name.
+        //
+        strcpy(nv_mem_client_nc.name, DRV_NAME);
     }
 
-    // The nc client enables support for persistent pages.
-    strcpy(nv_mem_client_nc.name, DRV_NAME "_nc");
     strcpy(nv_mem_client_nc.version, DRV_VERSION);
     reg_handle_nc = ib_register_peer_memory_client(&nv_mem_client_nc, NULL);
     if (!reg_handle_nc) {
         peer_err("nv_mem_client_init -- error while registering nc client\n");
-        rc = -EINVAL;
-        goto out;
+        return -EINVAL;
     }
+    return 0;
+}
+
+#endif /* NV_MLNX_IB_PEER_MEM_SYMBOLS_PRESENT */
+
+static int nv_mem_param_peerdirect_conf_check(void)
+{
+    int rc = 0;
+    switch (peerdirect_support) {
+    case NV_MEM_PEERDIRECT_SUPPORT_DEFAULT:
+    case NV_MEM_PEERDIRECT_SUPPORT_LEGACY:
+        break;
+    default:
+        peer_err("invalid peerdirect_support param value %d\n", peerdirect_support);
+        rc = -EINVAL;
+        break;
+    }
+    return rc;
+}
+
+static int nv_mem_param_persistent_api_conf_check(void)
+{
+    int rc = 0;
+    switch (persistent_api_support) {
+    case NV_MEM_PERSISTENT_API_SUPPORT_DEFAULT:
+    case NV_MEM_PERSISTENT_API_SUPPORT_LEGACY:
+        break;
+    default:
+        peer_err("invalid persistent_api_support param value %d\n", persistent_api_support);
+        rc = -EINVAL;
+        break;
+    }
+    return rc;
+}
+
+static int __init nv_mem_client_init(void)
+{
+#if defined (NV_MLNX_IB_PEER_MEM_SYMBOLS_PRESENT)
+    int rc;
+    rc = nv_mem_param_peerdirect_conf_check();
+    if (rc) {
+        return rc;
+    }
+
+    rc = nv_mem_param_persistent_api_conf_check();
+    if (rc) {
+        return rc;
+    }
+
+    if (persistent_api_support == NV_MEM_PERSISTENT_API_SUPPORT_LEGACY) {
+        rc = nv_mem_legacy_client_init();
+        if (rc)
+            goto out;
+    }
+
+    rc = nv_mem_nc_client_init();
+    if (rc)
+        goto out;
 
 out:
     if (rc) {

kernel-open/nvidia-uvm/clc365.h

@@ -1,30 +1,25 @@
-/*******************************************************************************
-    Copyright (c) 2023 NVIDIA Corporation
-
-    Permission is hereby granted, free of charge, to any person obtaining a copy
-    of this software and associated documentation files (the "Software"), to
-    deal in the Software without restriction, including without limitation the
-    rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
-    sell copies of the Software, and to permit persons to whom the Software is
-    furnished to do so, subject to the following conditions:
-
-    The above copyright notice and this permission notice shall be
-    included in all copies or substantial portions of the Software.
-
-    THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
-    IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
-    FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
-    THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
-    LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
-    FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
-    DEALINGS IN THE SOFTWARE.
-
-*******************************************************************************/
-
-
-// AUTO GENERATED -- DO NOT EDIT - this file automatically generated by refhdr2class.pl
-// Command: ../../../bin/manuals/refhdr2class.pl clc365.h c365 ACCESS_COUNTER_NOTIFY_BUFFER --search_str=NV_ACCESS_COUNTER --input_file=nv_ref_dev_access_counter.h
-
+/*
+ * SPDX-FileCopyrightText: Copyright (c) 2021-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+ * SPDX-License-Identifier: MIT
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
+ * DEALINGS IN THE SOFTWARE.
+ */
 
 #ifndef _clc365_h_
 #define _clc365_h_

kernel-open/nvidia-uvm/clc369.h

@@ -1,30 +1,25 @@
-/*******************************************************************************
-    Copyright (c) 2023 NVIDIA Corporation
-
-    Permission is hereby granted, free of charge, to any person obtaining a copy
-    of this software and associated documentation files (the "Software"), to
-    deal in the Software without restriction, including without limitation the
-    rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
-    sell copies of the Software, and to permit persons to whom the Software is
-    furnished to do so, subject to the following conditions:
-
-    The above copyright notice and this permission notice shall be
-    included in all copies or substantial portions of the Software.
-
-    THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
-    IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
-    FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
-    THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
-    LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
-    FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
-    DEALINGS IN THE SOFTWARE.
-
-*******************************************************************************/
-
-
-// AUTO GENERATED -- DO NOT EDIT - this file automatically generated by refhdr2class.pl
-// Command: ../../../bin/manuals/refhdr2class.pl clc369.h c369 MMU_FAULT_BUFFER --search_str=NV_MMU_FAULT --input_file=nv_ref_dev_mmu_fault.h
-
+/*
+ * SPDX-FileCopyrightText: Copyright (c) 2021-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+ * SPDX-License-Identifier: MIT
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
+ * DEALINGS IN THE SOFTWARE.
+ */
 
 #ifndef _clc369_h_
 #define _clc369_h_

kernel-open/nvidia-uvm/clc36f.h

@@ -1,26 +1,25 @@
-/*******************************************************************************
-    Copyright (c) 2012-2015 NVIDIA Corporation
-
-    Permission is hereby granted, free of charge, to any person obtaining a copy
-    of this software and associated documentation files (the "Software"), to
-    deal in the Software without restriction, including without limitation the
-    rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
-    sell copies of the Software, and to permit persons to whom the Software is
-    furnished to do so, subject to the following conditions:
-
-    The above copyright notice and this permission notice shall be
-    included in all copies or substantial portions of the Software.
-
-    THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
-    IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
-    FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
-    THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
-    LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
-    FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
-    DEALINGS IN THE SOFTWARE.
-
-*******************************************************************************/
-
+/*
+ * SPDX-FileCopyrightText: Copyright (c) 2020-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+ * SPDX-License-Identifier: MIT
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
+ * DEALINGS IN THE SOFTWARE.
+ */
 
 #ifndef _clc36f_h_
 #define _clc36f_h_
@@ -257,7 +256,6 @@ typedef volatile struct Nvc36fControl_struct {
 #define NVC36F_CLEAR_FAULTED_TYPE                                        31:31
 #define NVC36F_CLEAR_FAULTED_TYPE_PBDMA_FAULTED                     0x00000000
 #define NVC36F_CLEAR_FAULTED_TYPE_ENG_FAULTED                       0x00000001
-#define NVC36F_QUADRO_VERIFY                                       (0x000000a0)
 
 
 /* GPFIFO entry format */

kernel-open/nvidia-uvm/clc46f.h

@@ -1,26 +1,25 @@
-/*******************************************************************************
-    Copyright (c) 2012-2015 NVIDIA Corporation
-
-    Permission is hereby granted, free of charge, to any person obtaining a copy
-    of this software and associated documentation files (the "Software"), to
-    deal in the Software without restriction, including without limitation the
-    rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
-    sell copies of the Software, and to permit persons to whom the Software is
-    furnished to do so, subject to the following conditions:
-
-    The above copyright notice and this permission notice shall be
-    included in all copies or substantial portions of the Software.
-
-    THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
-    IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
-    FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
-    THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
-    LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
-    FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
-    DEALINGS IN THE SOFTWARE.
-
-*******************************************************************************/
-
+/*
+ * SPDX-FileCopyrightText: Copyright (c) 2020-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+ * SPDX-License-Identifier: MIT
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
+ * DEALINGS IN THE SOFTWARE.
+ */
 
 #ifndef _clc46f_h_
 #define _clc46f_h_
@@ -259,7 +258,6 @@ typedef volatile struct Nvc46fControl_struct {
 #define NVC46F_CLEAR_FAULTED_TYPE                                        31:31
 #define NVC46F_CLEAR_FAULTED_TYPE_PBDMA_FAULTED                     0x00000000
 #define NVC46F_CLEAR_FAULTED_TYPE_ENG_FAULTED                       0x00000001
-#define NVC46F_QUADRO_VERIFY                                       (0x000000a0)
 
 
 /* GPFIFO entry format */

kernel-open/nvidia-uvm/clc56f.h

@@ -1,26 +1,25 @@
-/*******************************************************************************
-    Copyright (c) 2012-2015 NVIDIA Corporation
-
-    Permission is hereby granted, free of charge, to any person obtaining a copy
-    of this software and associated documentation files (the "Software"), to
-    deal in the Software without restriction, including without limitation the
-    rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
-    sell copies of the Software, and to permit persons to whom the Software is
-    furnished to do so, subject to the following conditions:
-
-    The above copyright notice and this permission notice shall be
-    included in all copies or substantial portions of the Software.
-
-    THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
-    IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
-    FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
-    THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
-    LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
-    FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
-    DEALINGS IN THE SOFTWARE.
-
-*******************************************************************************/
-
+/*
+ * SPDX-FileCopyrightText: Copyright (c) 2020-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+ * SPDX-License-Identifier: MIT
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
+ * DEALINGS IN THE SOFTWARE.
+ */
 
 #ifndef _clc56f_h_
 #define _clc56f_h_
@@ -261,7 +260,6 @@ typedef volatile struct Nvc56fControl_struct {
 #define NVC56F_CLEAR_FAULTED_TYPE                                        31:31
 #define NVC56F_CLEAR_FAULTED_TYPE_PBDMA_FAULTED                     0x00000000
 #define NVC56F_CLEAR_FAULTED_TYPE_ENG_FAULTED                       0x00000001
-#define NVC56F_QUADRO_VERIFY                                       (0x000000a0)
 
 
 /* GPFIFO entry format */

kernel-open/nvidia-uvm/clc5b5.h

@@ -1,19 +1,19 @@
 /*******************************************************************************
-    Copyright (c) 1993-2004 NVIDIA Corporation
+    Copyright (c) 2020, NVIDIA CORPORATION. All rights reserved.
 
-    Permission is hereby granted, free of charge, to any person obtaining a copy
-    of this software and associated documentation files (the "Software"), to
-    deal in the Software without restriction, including without limitation the
-    rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
-    sell copies of the Software, and to permit persons to whom the Software is
-    furnished to do so, subject to the following conditions:
+    Permission is hereby granted, free of charge, to any person obtaining a
+    copy of this software and associated documentation files (the "Software"),
+    to deal in the Software without restriction, including without limitation
+    the rights to use, copy, modify, merge, publish, distribute, sublicense,
+    and/or sell copies of the Software, and to permit persons to whom the
+    Software is furnished to do so, subject to the following conditions:
 
-    The above copyright notice and this permission notice shall be
-    included in all copies or substantial portions of the Software.
+    The above copyright notice and this permission notice shall be included in
+    all copies or substantial portions of the Software.
 
     THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
     IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
-    FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+    FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
     THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
     LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
     FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
@@ -21,8 +21,6 @@
 
 *******************************************************************************/
 
-
-
 #include "nvtypes.h"
 
 #ifndef _clc5b5_h_
@@ -34,64 +32,6 @@ extern "C" {
 
 #define TURING_DMA_COPY_A                                                            (0x0000C5B5)
 
-typedef volatile struct _clc5b5_tag0 {
-    NvV32 Reserved00[0x40];
-    NvV32 Nop;                                                                  // 0x00000100 - 0x00000103
-    NvV32 Reserved01[0xF];
-    NvV32 PmTrigger;                                                            // 0x00000140 - 0x00000143
-    NvV32 Reserved02[0x3F];
-    NvV32 SetSemaphoreA;                                                        // 0x00000240 - 0x00000243
-    NvV32 SetSemaphoreB;                                                        // 0x00000244 - 0x00000247
-    NvV32 SetSemaphorePayload;                                                  // 0x00000248 - 0x0000024B
-    NvV32 Reserved03[0x2];
-    NvV32 SetRenderEnableA;                                                     // 0x00000254 - 0x00000257
-    NvV32 SetRenderEnableB;                                                     // 0x00000258 - 0x0000025B
-    NvV32 SetRenderEnableC;                                                     // 0x0000025C - 0x0000025F
-    NvV32 SetSrcPhysMode;                                                       // 0x00000260 - 0x00000263
-    NvV32 SetDstPhysMode;                                                       // 0x00000264 - 0x00000267
-    NvV32 Reserved04[0x6];
-    NvV32 SetGlobalCounterUpper;                                                // 0x00000280 - 0x00000283
-    NvV32 SetGlobalCounterLower;                                                // 0x00000284 - 0x00000287
-    NvV32 SetPageoutStartPAUpper;                                               // 0x00000288 - 0x0000028B
-    NvV32 SetPageoutStartPALower;                                               // 0x0000028C - 0x0000028F
-    NvV32 Reserved05[0x1C];
-    NvV32 LaunchDma;                                                            // 0x00000300 - 0x00000303
-    NvV32 Reserved06[0x3F];
-    NvV32 OffsetInUpper;                                                        // 0x00000400 - 0x00000403
-    NvV32 OffsetInLower;                                                        // 0x00000404 - 0x00000407
-    NvV32 OffsetOutUpper;                                                       // 0x00000408 - 0x0000040B
-    NvV32 OffsetOutLower;                                                       // 0x0000040C - 0x0000040F
-    NvV32 PitchIn;                                                              // 0x00000410 - 0x00000413
-    NvV32 PitchOut;                                                             // 0x00000414 - 0x00000417
-    NvV32 LineLengthIn;                                                         // 0x00000418 - 0x0000041B
-    NvV32 LineCount;                                                            // 0x0000041C - 0x0000041F
-    NvV32 Reserved07[0xB8];
-    NvV32 SetRemapConstA;                                                       // 0x00000700 - 0x00000703
-    NvV32 SetRemapConstB;                                                       // 0x00000704 - 0x00000707
-    NvV32 SetRemapComponents;                                                   // 0x00000708 - 0x0000070B
-    NvV32 SetDstBlockSize;                                                      // 0x0000070C - 0x0000070F
-    NvV32 SetDstWidth;                                                          // 0x00000710 - 0x00000713
-    NvV32 SetDstHeight;                                                         // 0x00000714 - 0x00000717
-    NvV32 SetDstDepth;                                                          // 0x00000718 - 0x0000071B
-    NvV32 SetDstLayer;                                                          // 0x0000071C - 0x0000071F
-    NvV32 SetDstOrigin;                                                         // 0x00000720 - 0x00000723
-    NvV32 Reserved08[0x1];
-    NvV32 SetSrcBlockSize;                                                      // 0x00000728 - 0x0000072B
-    NvV32 SetSrcWidth;                                                          // 0x0000072C - 0x0000072F
-    NvV32 SetSrcHeight;                                                         // 0x00000730 - 0x00000733
-    NvV32 SetSrcDepth;                                                          // 0x00000734 - 0x00000737
-    NvV32 SetSrcLayer;                                                          // 0x00000738 - 0x0000073B
-    NvV32 SetSrcOrigin;                                                         // 0x0000073C - 0x0000073F
-    NvV32 Reserved09[0x1];
-    NvV32 SrcOriginX;                                                           // 0x00000744 - 0x00000747
-    NvV32 SrcOriginY;                                                           // 0x00000748 - 0x0000074B
-    NvV32 DstOriginX;                                                           // 0x0000074C - 0x0000074F
-    NvV32 DstOriginY;                                                           // 0x00000750 - 0x00000753
-    NvV32 Reserved10[0x270];
-    NvV32 PmTriggerEnd;                                                         // 0x00001114 - 0x00001117
-    NvV32 Reserved11[0x3BA];
-} turing_dma_copy_aControlPio;
-
 #define NVC5B5_NOP                                                              (0x00000100)
 #define NVC5B5_NOP_PARAMETER                                                    31:0
 #define NVC5B5_PM_TRIGGER                                                       (0x00000140)
@@ -125,14 +65,6 @@ typedef volatile struct _clc5b5_tag0 {
 #define NVC5B5_SET_DST_PHYS_MODE_TARGET_COHERENT_SYSMEM                         (0x00000001)
 #define NVC5B5_SET_DST_PHYS_MODE_TARGET_NONCOHERENT_SYSMEM                      (0x00000002)
 #define NVC5B5_SET_DST_PHYS_MODE_BASIC_KIND                                     5:2
-#define NVC5B5_SET_GLOBAL_COUNTER_UPPER                                         (0x00000280)
-#define NVC5B5_SET_GLOBAL_COUNTER_UPPER_V                                       31:0
-#define NVC5B5_SET_GLOBAL_COUNTER_LOWER                                         (0x00000284)
-#define NVC5B5_SET_GLOBAL_COUNTER_LOWER_V                                       31:0
-#define NVC5B5_SET_PAGEOUT_START_PAUPPER                                        (0x00000288)
-#define NVC5B5_SET_PAGEOUT_START_PAUPPER_V                                      4:0
-#define NVC5B5_SET_PAGEOUT_START_PALOWER                                        (0x0000028C)
-#define NVC5B5_SET_PAGEOUT_START_PALOWER_V                                      31:0
 #define NVC5B5_LAUNCH_DMA                                                       (0x00000300)
 #define NVC5B5_LAUNCH_DMA_DATA_TRANSFER_TYPE                                    1:0
 #define NVC5B5_LAUNCH_DMA_DATA_TRANSFER_TYPE_NONE                               (0x00000000)
@@ -199,8 +131,6 @@ typedef volatile struct _clc5b5_tag0 {
 #define NVC5B5_LAUNCH_DMA_VPRMODE                                               23:22
 #define NVC5B5_LAUNCH_DMA_VPRMODE_VPR_NONE                                      (0x00000000)
 #define NVC5B5_LAUNCH_DMA_VPRMODE_VPR_VID2VID                                   (0x00000001)
-#define NVC5B5_LAUNCH_DMA_VPRMODE_VPR_VID2SYS                                   (0x00000002)
-#define NVC5B5_LAUNCH_DMA_VPRMODE_VPR_SYS2VID                                   (0x00000003)
 #define NVC5B5_LAUNCH_DMA_RESERVED_START_OF_COPY                                24:24
 #define NVC5B5_LAUNCH_DMA_DISABLE_PLC                                           26:26
 #define NVC5B5_LAUNCH_DMA_DISABLE_PLC_FALSE                                     (0x00000000)

kernel-open/nvidia-uvm/clc6b5.h

@@ -1,19 +1,19 @@
 /*******************************************************************************
-    Copyright (c) 1993-2004 NVIDIA Corporation
+    Copyright (c) 2020, NVIDIA CORPORATION. All rights reserved.
 
-    Permission is hereby granted, free of charge, to any person obtaining a copy
-    of this software and associated documentation files (the "Software"), to
-    deal in the Software without restriction, including without limitation the
-    rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
-    sell copies of the Software, and to permit persons to whom the Software is
-    furnished to do so, subject to the following conditions:
+    Permission is hereby granted, free of charge, to any person obtaining a
+    copy of this software and associated documentation files (the "Software"),
+    to deal in the Software without restriction, including without limitation
+    the rights to use, copy, modify, merge, publish, distribute, sublicense,
+    and/or sell copies of the Software, and to permit persons to whom the
+    Software is furnished to do so, subject to the following conditions:
 
-    The above copyright notice and this permission notice shall be
-    included in all copies or substantial portions of the Software.
+    The above copyright notice and this permission notice shall be included in
+    all copies or substantial portions of the Software.
 
     THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
     IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
-    FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+    FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
     THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
     LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
     FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
@@ -21,8 +21,6 @@
 
 *******************************************************************************/
 
-
-
 #include "nvtypes.h"
 
 #ifndef _clc6b5_h_
@@ -34,64 +32,6 @@ extern "C" {
 
 #define AMPERE_DMA_COPY_A                                                            (0x0000C6B5)
 
-typedef volatile struct _clc6b5_tag0 {
-    NvV32 Reserved00[0x40];
-    NvV32 Nop;                                                                  // 0x00000100 - 0x00000103
-    NvV32 Reserved01[0xF];
-    NvV32 PmTrigger;                                                            // 0x00000140 - 0x00000143
-    NvV32 Reserved02[0x3F];
-    NvV32 SetSemaphoreA;                                                        // 0x00000240 - 0x00000243
-    NvV32 SetSemaphoreB;                                                        // 0x00000244 - 0x00000247
-    NvV32 SetSemaphorePayload;                                                  // 0x00000248 - 0x0000024B
-    NvV32 Reserved03[0x2];
-    NvV32 SetRenderEnableA;                                                     // 0x00000254 - 0x00000257
-    NvV32 SetRenderEnableB;                                                     // 0x00000258 - 0x0000025B
-    NvV32 SetRenderEnableC;                                                     // 0x0000025C - 0x0000025F
-    NvV32 SetSrcPhysMode;                                                       // 0x00000260 - 0x00000263
-    NvV32 SetDstPhysMode;                                                       // 0x00000264 - 0x00000267
-    NvV32 Reserved04[0x6];
-    NvV32 SetGlobalCounterUpper;                                                // 0x00000280 - 0x00000283
-    NvV32 SetGlobalCounterLower;                                                // 0x00000284 - 0x00000287
-    NvV32 SetPageoutStartPAUpper;                                               // 0x00000288 - 0x0000028B
-    NvV32 SetPageoutStartPALower;                                               // 0x0000028C - 0x0000028F
-    NvV32 Reserved05[0x1C];
-    NvV32 LaunchDma;                                                            // 0x00000300 - 0x00000303
-    NvV32 Reserved06[0x3F];
-    NvV32 OffsetInUpper;                                                        // 0x00000400 - 0x00000403
-    NvV32 OffsetInLower;                                                        // 0x00000404 - 0x00000407
-    NvV32 OffsetOutUpper;                                                       // 0x00000408 - 0x0000040B
-    NvV32 OffsetOutLower;                                                       // 0x0000040C - 0x0000040F
-    NvV32 PitchIn;                                                              // 0x00000410 - 0x00000413
-    NvV32 PitchOut;                                                             // 0x00000414 - 0x00000417
-    NvV32 LineLengthIn;                                                         // 0x00000418 - 0x0000041B
-    NvV32 LineCount;                                                            // 0x0000041C - 0x0000041F
-    NvV32 Reserved07[0xB8];
-    NvV32 SetRemapConstA;                                                       // 0x00000700 - 0x00000703
-    NvV32 SetRemapConstB;                                                       // 0x00000704 - 0x00000707
-    NvV32 SetRemapComponents;                                                   // 0x00000708 - 0x0000070B
-    NvV32 SetDstBlockSize;                                                      // 0x0000070C - 0x0000070F
-    NvV32 SetDstWidth;                                                          // 0x00000710 - 0x00000713
-    NvV32 SetDstHeight;                                                         // 0x00000714 - 0x00000717
-    NvV32 SetDstDepth;                                                          // 0x00000718 - 0x0000071B
-    NvV32 SetDstLayer;                                                          // 0x0000071C - 0x0000071F
-    NvV32 SetDstOrigin;                                                         // 0x00000720 - 0x00000723
-    NvV32 Reserved08[0x1];
-    NvV32 SetSrcBlockSize;                                                      // 0x00000728 - 0x0000072B
-    NvV32 SetSrcWidth;                                                          // 0x0000072C - 0x0000072F
-    NvV32 SetSrcHeight;                                                         // 0x00000730 - 0x00000733
-    NvV32 SetSrcDepth;                                                          // 0x00000734 - 0x00000737
-    NvV32 SetSrcLayer;                                                          // 0x00000738 - 0x0000073B
-    NvV32 SetSrcOrigin;                                                         // 0x0000073C - 0x0000073F
-    NvV32 Reserved09[0x1];
-    NvV32 SrcOriginX;                                                           // 0x00000744 - 0x00000747
-    NvV32 SrcOriginY;                                                           // 0x00000748 - 0x0000074B
-    NvV32 DstOriginX;                                                           // 0x0000074C - 0x0000074F
-    NvV32 DstOriginY;                                                           // 0x00000750 - 0x00000753
-    NvV32 Reserved10[0x270];
-    NvV32 PmTriggerEnd;                                                         // 0x00001114 - 0x00001117
-    NvV32 Reserved11[0x3BA];
-} ampere_dma_copy_aControlPio;
-
 #define NVC6B5_NOP                                                              (0x00000100)
 #define NVC6B5_NOP_PARAMETER                                                    31:0
 #define NVC6B5_PM_TRIGGER                                                       (0x00000140)
@@ -131,14 +71,6 @@ typedef volatile struct _clc6b5_tag0 {
 #define NVC6B5_SET_DST_PHYS_MODE_BASIC_KIND                                     5:2
 #define NVC6B5_SET_DST_PHYS_MODE_PEER_ID                                        8:6
 #define NVC6B5_SET_DST_PHYS_MODE_FLA                                            9:9
-#define NVC6B5_SET_GLOBAL_COUNTER_UPPER                                         (0x00000280)
-#define NVC6B5_SET_GLOBAL_COUNTER_UPPER_V                                       31:0
-#define NVC6B5_SET_GLOBAL_COUNTER_LOWER                                         (0x00000284)
-#define NVC6B5_SET_GLOBAL_COUNTER_LOWER_V                                       31:0
-#define NVC6B5_SET_PAGEOUT_START_PAUPPER                                        (0x00000288)
-#define NVC6B5_SET_PAGEOUT_START_PAUPPER_V                                      4:0
-#define NVC6B5_SET_PAGEOUT_START_PALOWER                                        (0x0000028C)
-#define NVC6B5_SET_PAGEOUT_START_PALOWER_V                                      31:0
 #define NVC6B5_LAUNCH_DMA                                                       (0x00000300)
 #define NVC6B5_LAUNCH_DMA_DATA_TRANSFER_TYPE                                    1:0
 #define NVC6B5_LAUNCH_DMA_DATA_TRANSFER_TYPE_NONE                               (0x00000000)
@@ -199,8 +131,6 @@ typedef volatile struct _clc6b5_tag0 {
 #define NVC6B5_LAUNCH_DMA_VPRMODE                                               23:22
 #define NVC6B5_LAUNCH_DMA_VPRMODE_VPR_NONE                                      (0x00000000)
 #define NVC6B5_LAUNCH_DMA_VPRMODE_VPR_VID2VID                                   (0x00000001)
-#define NVC6B5_LAUNCH_DMA_VPRMODE_VPR_VID2SYS                                   (0x00000002)
-#define NVC6B5_LAUNCH_DMA_VPRMODE_VPR_SYS2VID                                   (0x00000003)
 #define NVC6B5_LAUNCH_DMA_RESERVED_START_OF_COPY                                24:24
 #define NVC6B5_LAUNCH_DMA_DISABLE_PLC                                           26:26
 #define NVC6B5_LAUNCH_DMA_DISABLE_PLC_FALSE                                     (0x00000000)

kernel-open/nvidia-uvm/clc7b5.h

@@ -1,19 +1,19 @@
 /*******************************************************************************
-    Copyright (c) 1993-2004 NVIDIA Corporation
+    Copyright (c) 2020, NVIDIA CORPORATION. All rights reserved.
 
-    Permission is hereby granted, free of charge, to any person obtaining a copy
-    of this software and associated documentation files (the "Software"), to
-    deal in the Software without restriction, including without limitation the
-    rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
-    sell copies of the Software, and to permit persons to whom the Software is
-    furnished to do so, subject to the following conditions:
+    Permission is hereby granted, free of charge, to any person obtaining a
+    copy of this software and associated documentation files (the "Software"),
+    to deal in the Software without restriction, including without limitation
+    the rights to use, copy, modify, merge, publish, distribute, sublicense,
+    and/or sell copies of the Software, and to permit persons to whom the
+    Software is furnished to do so, subject to the following conditions:
 
-    The above copyright notice and this permission notice shall be
-    included in all copies or substantial portions of the Software.
+    The above copyright notice and this permission notice shall be included in
+    all copies or substantial portions of the Software.
 
     THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
     IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
-    FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+    FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
     THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
     LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
     FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
@@ -21,8 +21,6 @@
 
 *******************************************************************************/
 
-
-
 #include "nvtypes.h"
 
 #ifndef _clc7b5_h_
@@ -34,69 +32,6 @@ extern "C" {
 
 #define AMPERE_DMA_COPY_B                                                            (0x0000C7B5)
 
-typedef volatile struct _clc7b5_tag0 {
-    NvV32 Reserved00[0x40];
-    NvV32 Nop;                                                                  // 0x00000100 - 0x00000103
-    NvV32 Reserved01[0xF];
-    NvV32 PmTrigger;                                                            // 0x00000140 - 0x00000143
-    NvV32 Reserved02[0x36];
-    NvV32 SetMonitoredFenceType;                                                // 0x0000021C - 0x0000021F
-    NvV32 SetMonitoredFenceSignalAddrBaseUpper;                                 // 0x00000220 - 0x00000223
-    NvV32 SetMonitoredFenceSignalAddrBaseLower;                                 // 0x00000224 - 0x00000227
-    NvV32 Reserved03[0x6];
-    NvV32 SetSemaphoreA;                                                        // 0x00000240 - 0x00000243
-    NvV32 SetSemaphoreB;                                                        // 0x00000244 - 0x00000247
-    NvV32 SetSemaphorePayload;                                                  // 0x00000248 - 0x0000024B
-    NvV32 SetSemaphorePayloadUpper;                                             // 0x0000024C - 0x0000024F
-    NvV32 Reserved04[0x1];
-    NvV32 SetRenderEnableA;                                                     // 0x00000254 - 0x00000257
-    NvV32 SetRenderEnableB;                                                     // 0x00000258 - 0x0000025B
-    NvV32 SetRenderEnableC;                                                     // 0x0000025C - 0x0000025F
-    NvV32 SetSrcPhysMode;                                                       // 0x00000260 - 0x00000263
-    NvV32 SetDstPhysMode;                                                       // 0x00000264 - 0x00000267
-    NvV32 Reserved05[0x6];
-    NvV32 SetGlobalCounterUpper;                                                // 0x00000280 - 0x00000283
-    NvV32 SetGlobalCounterLower;                                                // 0x00000284 - 0x00000287
-    NvV32 SetPageoutStartPAUpper;                                               // 0x00000288 - 0x0000028B
-    NvV32 SetPageoutStartPALower;                                               // 0x0000028C - 0x0000028F
-    NvV32 Reserved06[0x1C];
-    NvV32 LaunchDma;                                                            // 0x00000300 - 0x00000303
-    NvV32 Reserved07[0x3F];
-    NvV32 OffsetInUpper;                                                        // 0x00000400 - 0x00000403
-    NvV32 OffsetInLower;                                                        // 0x00000404 - 0x00000407
-    NvV32 OffsetOutUpper;                                                       // 0x00000408 - 0x0000040B
-    NvV32 OffsetOutLower;                                                       // 0x0000040C - 0x0000040F
-    NvV32 PitchIn;                                                              // 0x00000410 - 0x00000413
-    NvV32 PitchOut;                                                             // 0x00000414 - 0x00000417
-    NvV32 LineLengthIn;                                                         // 0x00000418 - 0x0000041B
-    NvV32 LineCount;                                                            // 0x0000041C - 0x0000041F
-    NvV32 Reserved08[0xB8];
-    NvV32 SetRemapConstA;                                                       // 0x00000700 - 0x00000703
-    NvV32 SetRemapConstB;                                                       // 0x00000704 - 0x00000707
-    NvV32 SetRemapComponents;                                                   // 0x00000708 - 0x0000070B
-    NvV32 SetDstBlockSize;                                                      // 0x0000070C - 0x0000070F
-    NvV32 SetDstWidth;                                                          // 0x00000710 - 0x00000713
-    NvV32 SetDstHeight;                                                         // 0x00000714 - 0x00000717
-    NvV32 SetDstDepth;                                                          // 0x00000718 - 0x0000071B
-    NvV32 SetDstLayer;                                                          // 0x0000071C - 0x0000071F
-    NvV32 SetDstOrigin;                                                         // 0x00000720 - 0x00000723
-    NvV32 Reserved09[0x1];
-    NvV32 SetSrcBlockSize;                                                      // 0x00000728 - 0x0000072B
-    NvV32 SetSrcWidth;                                                          // 0x0000072C - 0x0000072F
-    NvV32 SetSrcHeight;                                                         // 0x00000730 - 0x00000733
-    NvV32 SetSrcDepth;                                                          // 0x00000734 - 0x00000737
-    NvV32 SetSrcLayer;                                                          // 0x00000738 - 0x0000073B
-    NvV32 SetSrcOrigin;                                                         // 0x0000073C - 0x0000073F
-    NvV32 Reserved10[0x1];
-    NvV32 SrcOriginX;                                                           // 0x00000744 - 0x00000747
-    NvV32 SrcOriginY;                                                           // 0x00000748 - 0x0000074B
-    NvV32 DstOriginX;                                                           // 0x0000074C - 0x0000074F
-    NvV32 DstOriginY;                                                           // 0x00000750 - 0x00000753
-    NvV32 Reserved11[0x270];
-    NvV32 PmTriggerEnd;                                                         // 0x00001114 - 0x00001117
-    NvV32 Reserved12[0x3BA];
-} ampere_dma_copy_bControlPio;
-
 #define NVC7B5_NOP                                                              (0x00000100)
 #define NVC7B5_NOP_PARAMETER                                                    31:0
 #define NVC7B5_PM_TRIGGER                                                       (0x00000140)
@@ -146,14 +81,6 @@ typedef volatile struct _clc7b5_tag0 {
 #define NVC7B5_SET_DST_PHYS_MODE_BASIC_KIND                                     5:2
 #define NVC7B5_SET_DST_PHYS_MODE_PEER_ID                                        8:6
 #define NVC7B5_SET_DST_PHYS_MODE_FLA                                            9:9
-#define NVC7B5_SET_GLOBAL_COUNTER_UPPER                                         (0x00000280)
-#define NVC7B5_SET_GLOBAL_COUNTER_UPPER_V                                       31:0
-#define NVC7B5_SET_GLOBAL_COUNTER_LOWER                                         (0x00000284)
-#define NVC7B5_SET_GLOBAL_COUNTER_LOWER_V                                       31:0
-#define NVC7B5_SET_PAGEOUT_START_PAUPPER                                        (0x00000288)
-#define NVC7B5_SET_PAGEOUT_START_PAUPPER_V                                      4:0
-#define NVC7B5_SET_PAGEOUT_START_PALOWER                                        (0x0000028C)
-#define NVC7B5_SET_PAGEOUT_START_PALOWER_V                                      31:0
 #define NVC7B5_LAUNCH_DMA                                                       (0x00000300)
 #define NVC7B5_LAUNCH_DMA_DATA_TRANSFER_TYPE                                    1:0
 #define NVC7B5_LAUNCH_DMA_DATA_TRANSFER_TYPE_NONE                               (0x00000000)
@@ -223,8 +150,6 @@ typedef volatile struct _clc7b5_tag0 {
 #define NVC7B5_LAUNCH_DMA_VPRMODE                                               23:22
 #define NVC7B5_LAUNCH_DMA_VPRMODE_VPR_NONE                                      (0x00000000)
 #define NVC7B5_LAUNCH_DMA_VPRMODE_VPR_VID2VID                                   (0x00000001)
-#define NVC7B5_LAUNCH_DMA_VPRMODE_VPR_VID2SYS                                   (0x00000002)
-#define NVC7B5_LAUNCH_DMA_VPRMODE_VPR_SYS2VID                                   (0x00000003)
 #define NVC7B5_LAUNCH_DMA_RESERVED_START_OF_COPY                                24:24
 #define NVC7B5_LAUNCH_DMA_DISABLE_PLC                                           26:26
 #define NVC7B5_LAUNCH_DMA_DISABLE_PLC_FALSE                                     (0x00000000)

kernel-open/nvidia-uvm/clc86f.h

@@ -1,51 +1,31 @@
-/*******************************************************************************
-    Copyright (c) 2012-2015 NVIDIA Corporation
-
-    Permission is hereby granted, free of charge, to any person obtaining a copy
-    of this software and associated documentation files (the "Software"), to
-    deal in the Software without restriction, including without limitation the
-    rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
-    sell copies of the Software, and to permit persons to whom the Software is
-    furnished to do so, subject to the following conditions:
-
-    The above copyright notice and this permission notice shall be
-    included in all copies or substantial portions of the Software.
-
-    THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
-    IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
-    FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
-    THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
-    LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
-    FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
-    DEALINGS IN THE SOFTWARE.
-
-*******************************************************************************/
-
-
-#ifndef _clc86f_h_
-#define _clc86f_h_
-
-#ifdef __cplusplus
-extern "C" {
-#endif
-
-#include "nvtypes.h"
-
-/* class HOPPER_CHANNEL_GPFIFO  */
 /*
- * Documentation for HOPPER_CHANNEL_GPFIFO can be found in dev_pbdma.ref,
- * chapter "User Control Registers". It is documented as device NV_UDMA.
- * The GPFIFO format itself is also documented in dev_pbdma.ref,
- * NV_PPBDMA_GP_ENTRY_*. The pushbuffer format is documented in dev_ram.ref,
- * chapter "FIFO DMA RAM", NV_FIFO_DMA_*.
+ * SPDX-FileCopyrightText: Copyright (c) 2003-2022 NVIDIA CORPORATION & AFFILIATES
+ * SPDX-License-Identifier: MIT
  *
- * Note there is no .mfs file for this class.
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
+ * DEALINGS IN THE SOFTWARE.
  */
+
+#ifndef __gh100_clc86f_h__
+#define __gh100_clc86f_h__
+
 #define  HOPPER_CHANNEL_GPFIFO_A                           (0x0000C86F)
 
-#define NVC86F_TYPEDEF                             HOPPER_CHANNELChannelGPFifoA
-
-/* dma flow control data structure */
 typedef volatile struct Nvc86fControl_struct {
  NvU32 Ignored00[0x010];        /*                                  0000-003f*/
  NvU32 Put;                     /* put offset, read/write           0040-0043*/
@@ -64,54 +44,7 @@ typedef volatile struct Nvc86fControl_struct {
  NvU32 Ignored05[0x5c];
 } Nvc86fControl, HopperAControlGPFifo;
 
-/* fields and values */
-#define NVC86F_NUMBER_OF_SUBCHANNELS                               (8)
 #define NVC86F_SET_OBJECT                                          (0x00000000)
-#define NVC86F_SET_OBJECT_NVCLASS                                         15:0
-#define NVC86F_SET_OBJECT_ENGINE                                         20:16
-#define NVC86F_SET_OBJECT_ENGINE_SW                                 0x0000001f
-#define NVC86F_ILLEGAL                                             (0x00000004)
-#define NVC86F_ILLEGAL_HANDLE                                             31:0
-#define NVC86F_NOP                                                 (0x00000008)
-#define NVC86F_NOP_HANDLE                                                 31:0
-#define NVC86F_SEMAPHOREA                                          (0x00000010)
-#define NVC86F_SEMAPHOREA_OFFSET_UPPER                                     7:0
-#define NVC86F_SEMAPHOREB                                          (0x00000014)
-#define NVC86F_SEMAPHOREB_OFFSET_LOWER                                    31:2
-#define NVC86F_SEMAPHOREC                                          (0x00000018)
-#define NVC86F_SEMAPHOREC_PAYLOAD                                         31:0
-#define NVC86F_SEMAPHORED                                          (0x0000001C)
-#define NVC86F_SEMAPHORED_OPERATION                                        4:0
-#define NVC86F_SEMAPHORED_OPERATION_ACQUIRE                         0x00000001
-#define NVC86F_SEMAPHORED_OPERATION_RELEASE                         0x00000002
-#define NVC86F_SEMAPHORED_OPERATION_ACQ_GEQ                         0x00000004
-#define NVC86F_SEMAPHORED_OPERATION_ACQ_AND                         0x00000008
-#define NVC86F_SEMAPHORED_OPERATION_REDUCTION                       0x00000010
-#define NVC86F_SEMAPHORED_ACQUIRE_SWITCH                                 12:12
-#define NVC86F_SEMAPHORED_ACQUIRE_SWITCH_DISABLED                   0x00000000
-#define NVC86F_SEMAPHORED_ACQUIRE_SWITCH_ENABLED                    0x00000001
-#define NVC86F_SEMAPHORED_RELEASE_WFI                                    20:20
-#define NVC86F_SEMAPHORED_RELEASE_WFI_EN                            0x00000000
-#define NVC86F_SEMAPHORED_RELEASE_WFI_DIS                           0x00000001
-#define NVC86F_SEMAPHORED_RELEASE_SIZE                                   24:24
-#define NVC86F_SEMAPHORED_RELEASE_SIZE_16BYTE                       0x00000000
-#define NVC86F_SEMAPHORED_RELEASE_SIZE_4BYTE                        0x00000001
-#define NVC86F_SEMAPHORED_REDUCTION                                      30:27
-#define NVC86F_SEMAPHORED_REDUCTION_MIN                             0x00000000
-#define NVC86F_SEMAPHORED_REDUCTION_MAX                             0x00000001
-#define NVC86F_SEMAPHORED_REDUCTION_XOR                             0x00000002
-#define NVC86F_SEMAPHORED_REDUCTION_AND                             0x00000003
-#define NVC86F_SEMAPHORED_REDUCTION_OR                              0x00000004
-#define NVC86F_SEMAPHORED_REDUCTION_ADD                             0x00000005
-#define NVC86F_SEMAPHORED_REDUCTION_INC                             0x00000006
-#define NVC86F_SEMAPHORED_REDUCTION_DEC                             0x00000007
-#define NVC86F_SEMAPHORED_FORMAT                                         31:31
-#define NVC86F_SEMAPHORED_FORMAT_SIGNED                             0x00000000
-#define NVC86F_SEMAPHORED_FORMAT_UNSIGNED                           0x00000001
-#define NVC86F_NON_STALL_INTERRUPT                                 (0x00000020)
-#define NVC86F_NON_STALL_INTERRUPT_HANDLE                                 31:0
-#define NVC86F_FB_FLUSH                                            (0x00000024) // Deprecated - use MEMBAR TYPE SYS_MEMBAR
-#define NVC86F_FB_FLUSH_HANDLE                                            31:0
 // NOTE - MEM_OP_A and MEM_OP_B have been replaced in gp100 with methods for
 // specifying the page address for a targeted TLB invalidate and the uTLB for
 // a targeted REPLAY_CANCEL for UVM.
@@ -206,67 +139,31 @@ typedef volatile struct Nvc86fControl_struct {
 #define NVC86F_MEM_OP_D_MMU_OPERATION_TYPE                               23:20
 #define NVC86F_MEM_OP_D_MMU_OPERATION_TYPE_RESERVED                 0x00000000
 #define NVC86F_MEM_OP_D_MMU_OPERATION_TYPE_VIDMEM_ACCESS_BIT_DUMP   0x00000001
-#define NVC86F_SET_REFERENCE                                       (0x00000050)
-#define NVC86F_SET_REFERENCE_COUNT                                        31:0
 #define NVC86F_SEM_ADDR_LO                                         (0x0000005c)
 #define NVC86F_SEM_ADDR_LO_OFFSET                                         31:2
 #define NVC86F_SEM_ADDR_HI                                         (0x00000060)
 #define NVC86F_SEM_ADDR_HI_OFFSET                                         24:0
 #define NVC86F_SEM_PAYLOAD_LO                                      (0x00000064)
-#define NVC86F_SEM_PAYLOAD_LO_PAYLOAD                                     31:0
 #define NVC86F_SEM_PAYLOAD_HI                                      (0x00000068)
-#define NVC86F_SEM_PAYLOAD_HI_PAYLOAD                                     31:0
 #define NVC86F_SEM_EXECUTE                                         (0x0000006c)
 #define NVC86F_SEM_EXECUTE_OPERATION                                       2:0
 #define NVC86F_SEM_EXECUTE_OPERATION_ACQUIRE                        0x00000000
 #define NVC86F_SEM_EXECUTE_OPERATION_RELEASE                        0x00000001
-#define NVC86F_SEM_EXECUTE_OPERATION_ACQ_STRICT_GEQ                 0x00000002
 #define NVC86F_SEM_EXECUTE_OPERATION_ACQ_CIRC_GEQ                   0x00000003
-#define NVC86F_SEM_EXECUTE_OPERATION_ACQ_AND                        0x00000004
-#define NVC86F_SEM_EXECUTE_OPERATION_ACQ_NOR                        0x00000005
-#define NVC86F_SEM_EXECUTE_OPERATION_REDUCTION                      0x00000006
 #define NVC86F_SEM_EXECUTE_ACQUIRE_SWITCH_TSG                            12:12
-#define NVC86F_SEM_EXECUTE_ACQUIRE_SWITCH_TSG_DIS                   0x00000000
 #define NVC86F_SEM_EXECUTE_ACQUIRE_SWITCH_TSG_EN                    0x00000001
 #define NVC86F_SEM_EXECUTE_RELEASE_WFI                                   20:20
 #define NVC86F_SEM_EXECUTE_RELEASE_WFI_DIS                          0x00000000
-#define NVC86F_SEM_EXECUTE_RELEASE_WFI_EN                           0x00000001
 #define NVC86F_SEM_EXECUTE_PAYLOAD_SIZE                                  24:24
 #define NVC86F_SEM_EXECUTE_PAYLOAD_SIZE_32BIT                       0x00000000
-#define NVC86F_SEM_EXECUTE_PAYLOAD_SIZE_64BIT                       0x00000001
 #define NVC86F_SEM_EXECUTE_RELEASE_TIMESTAMP                             25:25
 #define NVC86F_SEM_EXECUTE_RELEASE_TIMESTAMP_DIS                    0x00000000
 #define NVC86F_SEM_EXECUTE_RELEASE_TIMESTAMP_EN                     0x00000001
-#define NVC86F_SEM_EXECUTE_REDUCTION                                     30:27
-#define NVC86F_SEM_EXECUTE_REDUCTION_IMIN                           0x00000000
-#define NVC86F_SEM_EXECUTE_REDUCTION_IMAX                           0x00000001
-#define NVC86F_SEM_EXECUTE_REDUCTION_IXOR                           0x00000002
-#define NVC86F_SEM_EXECUTE_REDUCTION_IAND                           0x00000003
-#define NVC86F_SEM_EXECUTE_REDUCTION_IOR                            0x00000004
-#define NVC86F_SEM_EXECUTE_REDUCTION_IADD                           0x00000005
-#define NVC86F_SEM_EXECUTE_REDUCTION_INC                            0x00000006
-#define NVC86F_SEM_EXECUTE_REDUCTION_DEC                            0x00000007
-#define NVC86F_SEM_EXECUTE_REDUCTION_FORMAT                              31:31
-#define NVC86F_SEM_EXECUTE_REDUCTION_FORMAT_SIGNED                  0x00000000
-#define NVC86F_SEM_EXECUTE_REDUCTION_FORMAT_UNSIGNED                0x00000001
 #define NVC86F_WFI                                                 (0x00000078)
 #define NVC86F_WFI_SCOPE                                                   0:0
 #define NVC86F_WFI_SCOPE_CURRENT_SCG_TYPE                           0x00000000
 #define NVC86F_WFI_SCOPE_CURRENT_VEID                               0x00000000
 #define NVC86F_WFI_SCOPE_ALL                                        0x00000001
-#define NVC86F_YIELD                                               (0x00000080)
-#define NVC86F_YIELD_OP                                                    1:0
-#define NVC86F_YIELD_OP_NOP                                         0x00000000
-#define NVC86F_YIELD_OP_TSG                                         0x00000003
-#define NVC86F_CLEAR_FAULTED                                       (0x00000084)
-// Note: RM provides the HANDLE as an opaque value; the internal detail fields
-// are intentionally not exposed to the driver through these defines.
-#define NVC86F_CLEAR_FAULTED_HANDLE                                       30:0
-#define NVC86F_CLEAR_FAULTED_TYPE                                        31:31
-#define NVC86F_CLEAR_FAULTED_TYPE_PBDMA_FAULTED                     0x00000000
-#define NVC86F_CLEAR_FAULTED_TYPE_ENG_FAULTED                       0x00000001
-#define NVC86F_QUADRO_VERIFY                                       (0x000000a0)
-
 
 /* GPFIFO entry format */
 #define NVC86F_GP_ENTRY__SIZE                                          8
@@ -291,85 +188,4 @@ typedef volatile struct Nvc86fControl_struct {
 #define NVC86F_GP_ENTRY1_OPCODE_PB_CRC                        0x00000003
 #define NVC86F_GP_ENTRY1_OPCODE_SET_PB_SEGMENT_EXTENDED_BASE  0x00000004
 
-/* dma method formats */
-#define NVC86F_DMA_METHOD_ADDRESS_OLD                              12:2
-#define NVC86F_DMA_METHOD_ADDRESS                                  11:0
-#define NVC86F_DMA_SUBDEVICE_MASK                                  15:4
-#define NVC86F_DMA_METHOD_SUBCHANNEL                               15:13
-#define NVC86F_DMA_TERT_OP                                         17:16
-#define NVC86F_DMA_TERT_OP_GRP0_INC_METHOD                         (0x00000000)
-#define NVC86F_DMA_TERT_OP_GRP0_SET_SUB_DEV_MASK                   (0x00000001)
-#define NVC86F_DMA_TERT_OP_GRP0_STORE_SUB_DEV_MASK                 (0x00000002)
-#define NVC86F_DMA_TERT_OP_GRP0_USE_SUB_DEV_MASK                   (0x00000003)
-#define NVC86F_DMA_TERT_OP_GRP2_NON_INC_METHOD                     (0x00000000)
-#define NVC86F_DMA_METHOD_COUNT_OLD                                28:18
-#define NVC86F_DMA_METHOD_COUNT                                    28:16
-#define NVC86F_DMA_IMMD_DATA                                       28:16
-#define NVC86F_DMA_SEC_OP                                          31:29
-#define NVC86F_DMA_SEC_OP_GRP0_USE_TERT                            (0x00000000)
-#define NVC86F_DMA_SEC_OP_INC_METHOD                               (0x00000001)
-#define NVC86F_DMA_SEC_OP_GRP2_USE_TERT                            (0x00000002)
-#define NVC86F_DMA_SEC_OP_NON_INC_METHOD                           (0x00000003)
-#define NVC86F_DMA_SEC_OP_IMMD_DATA_METHOD                         (0x00000004)
-#define NVC86F_DMA_SEC_OP_ONE_INC                                  (0x00000005)
-#define NVC86F_DMA_SEC_OP_RESERVED6                                (0x00000006)
-#define NVC86F_DMA_SEC_OP_END_PB_SEGMENT                           (0x00000007)
-/* dma incrementing method format */
-#define NVC86F_DMA_INCR_ADDRESS                                    11:0
-#define NVC86F_DMA_INCR_SUBCHANNEL                                 15:13
-#define NVC86F_DMA_INCR_COUNT                                      28:16
-#define NVC86F_DMA_INCR_OPCODE                                     31:29
-#define NVC86F_DMA_INCR_OPCODE_VALUE                               (0x00000001)
-#define NVC86F_DMA_INCR_DATA                                       31:0
-/* dma non-incrementing method format */
-#define NVC86F_DMA_NONINCR_ADDRESS                                 11:0
-#define NVC86F_DMA_NONINCR_SUBCHANNEL                              15:13
-#define NVC86F_DMA_NONINCR_COUNT                                   28:16
-#define NVC86F_DMA_NONINCR_OPCODE                                  31:29
-#define NVC86F_DMA_NONINCR_OPCODE_VALUE                            (0x00000003)
-#define NVC86F_DMA_NONINCR_DATA                                    31:0
-/* dma increment-once method format */
-#define NVC86F_DMA_ONEINCR_ADDRESS                                 11:0
-#define NVC86F_DMA_ONEINCR_SUBCHANNEL                              15:13
-#define NVC86F_DMA_ONEINCR_COUNT                                   28:16
-#define NVC86F_DMA_ONEINCR_OPCODE                                  31:29
-#define NVC86F_DMA_ONEINCR_OPCODE_VALUE                            (0x00000005)
-#define NVC86F_DMA_ONEINCR_DATA                                    31:0
-/* dma no-operation format */
-#define NVC86F_DMA_NOP                                             (0x00000000)
-/* dma immediate-data format */
-#define NVC86F_DMA_IMMD_ADDRESS                                    11:0
-#define NVC86F_DMA_IMMD_SUBCHANNEL                                 15:13
-#define NVC86F_DMA_IMMD_DATA                                       28:16
-#define NVC86F_DMA_IMMD_OPCODE                                     31:29
-#define NVC86F_DMA_IMMD_OPCODE_VALUE                               (0x00000004)
-/* dma set sub-device mask format */
-#define NVC86F_DMA_SET_SUBDEVICE_MASK_VALUE                        15:4
-#define NVC86F_DMA_SET_SUBDEVICE_MASK_OPCODE                       31:16
-#define NVC86F_DMA_SET_SUBDEVICE_MASK_OPCODE_VALUE                 (0x00000001)
-/* dma store sub-device mask format */
-#define NVC86F_DMA_STORE_SUBDEVICE_MASK_VALUE                      15:4
-#define NVC86F_DMA_STORE_SUBDEVICE_MASK_OPCODE                     31:16
-#define NVC86F_DMA_STORE_SUBDEVICE_MASK_OPCODE_VALUE               (0x00000002)
-/* dma use sub-device mask format */
-#define NVC86F_DMA_USE_SUBDEVICE_MASK_OPCODE                       31:16
-#define NVC86F_DMA_USE_SUBDEVICE_MASK_OPCODE_VALUE                 (0x00000003)
-/* dma end-segment format */
-#define NVC86F_DMA_ENDSEG_OPCODE                                   31:29
-#define NVC86F_DMA_ENDSEG_OPCODE_VALUE                             (0x00000007)
-/* dma legacy incrementing/non-incrementing formats */
-#define NVC86F_DMA_ADDRESS                                         12:2
-#define NVC86F_DMA_SUBCH                                           15:13
-#define NVC86F_DMA_OPCODE3                                         17:16
-#define NVC86F_DMA_OPCODE3_NONE                                    (0x00000000)
-#define NVC86F_DMA_COUNT                                           28:18
-#define NVC86F_DMA_OPCODE                                          31:29
-#define NVC86F_DMA_OPCODE_METHOD                                   (0x00000000)
-#define NVC86F_DMA_OPCODE_NONINC_METHOD                            (0x00000002)
-#define NVC86F_DMA_DATA                                            31:0
-
-#ifdef __cplusplus
-};     /* extern "C" */
-#endif
-
-#endif /* _clc86f_h_ */
+#endif // __gh100_clc86f_h__

kernel-open/nvidia-uvm/clc8b5.h

@@ -1,160 +1,46 @@
-/*******************************************************************************
-    Copyright (c) 1993-2004 NVIDIA Corporation
+/*
+ * SPDX-FileCopyrightText: Copyright (c) 2003-2022 NVIDIA CORPORATION & AFFILIATES
+ * SPDX-License-Identifier: MIT
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
+ * DEALINGS IN THE SOFTWARE.
+ */
 
-    Permission is hereby granted, free of charge, to any person obtaining a copy
-    of this software and associated documentation files (the "Software"), to
-    deal in the Software without restriction, including without limitation the
-    rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
-    sell copies of the Software, and to permit persons to whom the Software is
-    furnished to do so, subject to the following conditions:
+#ifndef __gh100_clc8b5_h__
+#define __gh100_clc8b5_h__
 
-    The above copyright notice and this permission notice shall be
-    included in all copies or substantial portions of the Software.
-
-    THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
-    IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
-    FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
-    THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
-    LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
-    FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
-    DEALINGS IN THE SOFTWARE.
-
-*******************************************************************************/
-
-
-
-#include "nvtypes.h"
-
-#ifndef _clc8b5_h_
-#define _clc8b5_h_
-
-#ifdef __cplusplus
-extern "C" {
-#endif
-
-#define HOPPER_DMA_COPY_A                                                            (0x0000C8B5)
-
-typedef volatile struct _clc8b5_tag0 {
-    NvV32 Reserved00[0x40];
-    NvV32 Nop;                                                                  // 0x00000100 - 0x00000103
-    NvV32 Reserved01[0xF];
-    NvV32 PmTrigger;                                                            // 0x00000140 - 0x00000143
-    NvV32 Reserved02[0x36];
-    NvV32 SetMonitoredFenceType;                                                // 0x0000021C - 0x0000021F
-    NvV32 SetMonitoredFenceSignalAddrBaseUpper;                                 // 0x00000220 - 0x00000223
-    NvV32 SetMonitoredFenceSignalAddrBaseLower;                                 // 0x00000224 - 0x00000227
-    NvV32 Reserved03[0x6];
-    NvV32 SetSemaphoreA;                                                        // 0x00000240 - 0x00000243
-    NvV32 SetSemaphoreB;                                                        // 0x00000244 - 0x00000247
-    NvV32 SetSemaphorePayload;                                                  // 0x00000248 - 0x0000024B
-    NvV32 SetSemaphorePayloadUpper;                                             // 0x0000024C - 0x0000024F
-    NvV32 Reserved04[0x1];
-    NvV32 SetRenderEnableA;                                                     // 0x00000254 - 0x00000257
-    NvV32 SetRenderEnableB;                                                     // 0x00000258 - 0x0000025B
-    NvV32 SetRenderEnableC;                                                     // 0x0000025C - 0x0000025F
-    NvV32 SetSrcPhysMode;                                                       // 0x00000260 - 0x00000263
-    NvV32 SetDstPhysMode;                                                       // 0x00000264 - 0x00000267
-    NvV32 Reserved05[0x26];
-    NvV32 LaunchDma;                                                            // 0x00000300 - 0x00000303
-    NvV32 Reserved06[0x3F];
-    NvV32 OffsetInUpper;                                                        // 0x00000400 - 0x00000403
-    NvV32 OffsetInLower;                                                        // 0x00000404 - 0x00000407
-    NvV32 OffsetOutUpper;                                                       // 0x00000408 - 0x0000040B
-    NvV32 OffsetOutLower;                                                       // 0x0000040C - 0x0000040F
-    NvV32 PitchIn;                                                              // 0x00000410 - 0x00000413
-    NvV32 PitchOut;                                                             // 0x00000414 - 0x00000417
-    NvV32 LineLengthIn;                                                         // 0x00000418 - 0x0000041B
-    NvV32 LineCount;                                                            // 0x0000041C - 0x0000041F
-    NvV32 Reserved07[0x38];
-    NvV32 SetSecureCopyMode;                                                    // 0x00000500 - 0x00000503
-    NvV32 SetDecryptIv0;                                                        // 0x00000504 - 0x00000507
-    NvV32 SetDecryptIv1;                                                        // 0x00000508 - 0x0000050B
-    NvV32 SetDecryptIv2;                                                        // 0x0000050C - 0x0000050F
-    NvV32 Reserved_SetAESCounter;                                               // 0x00000510 - 0x00000513
-    NvV32 SetDecryptAuthTagCompareAddrUpper;                                    // 0x00000514 - 0x00000517
-    NvV32 SetDecryptAuthTagCompareAddrLower;                                    // 0x00000518 - 0x0000051B
-    NvV32 Reserved08[0x5];
-    NvV32 SetEncryptAuthTagAddrUpper;                                           // 0x00000530 - 0x00000533
-    NvV32 SetEncryptAuthTagAddrLower;                                           // 0x00000534 - 0x00000537
-    NvV32 SetEncryptIvAddrUpper;                                                // 0x00000538 - 0x0000053B
-    NvV32 SetEncryptIvAddrLower;                                                // 0x0000053C - 0x0000053F
-    NvV32 Reserved09[0x6F];
-    NvV32 SetMemoryScrubParameters;                                             // 0x000006FC - 0x000006FF
-    NvV32 SetRemapConstA;                                                       // 0x00000700 - 0x00000703
-    NvV32 SetRemapConstB;                                                       // 0x00000704 - 0x00000707
-    NvV32 SetRemapComponents;                                                   // 0x00000708 - 0x0000070B
-    NvV32 SetDstBlockSize;                                                      // 0x0000070C - 0x0000070F
-    NvV32 SetDstWidth;                                                          // 0x00000710 - 0x00000713
-    NvV32 SetDstHeight;                                                         // 0x00000714 - 0x00000717
-    NvV32 SetDstDepth;                                                          // 0x00000718 - 0x0000071B
-    NvV32 SetDstLayer;                                                          // 0x0000071C - 0x0000071F
-    NvV32 SetDstOrigin;                                                         // 0x00000720 - 0x00000723
-    NvV32 Reserved10[0x1];
-    NvV32 SetSrcBlockSize;                                                      // 0x00000728 - 0x0000072B
-    NvV32 SetSrcWidth;                                                          // 0x0000072C - 0x0000072F
-    NvV32 SetSrcHeight;                                                         // 0x00000730 - 0x00000733
-    NvV32 SetSrcDepth;                                                          // 0x00000734 - 0x00000737
-    NvV32 SetSrcLayer;                                                          // 0x00000738 - 0x0000073B
-    NvV32 SetSrcOrigin;                                                         // 0x0000073C - 0x0000073F
-    NvV32 Reserved11[0x1];
-    NvV32 SrcOriginX;                                                           // 0x00000744 - 0x00000747
-    NvV32 SrcOriginY;                                                           // 0x00000748 - 0x0000074B
-    NvV32 DstOriginX;                                                           // 0x0000074C - 0x0000074F
-    NvV32 DstOriginY;                                                           // 0x00000750 - 0x00000753
-    NvV32 Reserved12[0x270];
-    NvV32 PmTriggerEnd;                                                         // 0x00001114 - 0x00001117
-    NvV32 Reserved13[0x3BA];
-} hopper_dma_copy_aControlPio;
-
-#define NVC8B5_NOP                                                              (0x00000100)
-#define NVC8B5_NOP_PARAMETER                                                    31:0
-#define NVC8B5_PM_TRIGGER                                                       (0x00000140)
-#define NVC8B5_PM_TRIGGER_V                                                     31:0
-#define NVC8B5_SET_MONITORED_FENCE_TYPE                                         (0x0000021C)
-#define NVC8B5_SET_MONITORED_FENCE_TYPE_TYPE                                    0:0
-#define NVC8B5_SET_MONITORED_FENCE_TYPE_TYPE_MONITORED_FENCE                    (0x00000000)
-#define NVC8B5_SET_MONITORED_FENCE_TYPE_TYPE_MONITORED_FENCE_EXT                (0x00000001)
-#define NVC8B5_SET_MONITORED_FENCE_SIGNAL_ADDR_BASE_UPPER                       (0x00000220)
-#define NVC8B5_SET_MONITORED_FENCE_SIGNAL_ADDR_BASE_UPPER_UPPER                 24:0
-#define NVC8B5_SET_MONITORED_FENCE_SIGNAL_ADDR_BASE_LOWER                       (0x00000224)
-#define NVC8B5_SET_MONITORED_FENCE_SIGNAL_ADDR_BASE_LOWER_LOWER                 31:0
+#define HOPPER_DMA_COPY_A                                                       (0x0000C8B5)
 #define NVC8B5_SET_SEMAPHORE_A                                                  (0x00000240)
 #define NVC8B5_SET_SEMAPHORE_A_UPPER                                            24:0
 #define NVC8B5_SET_SEMAPHORE_B                                                  (0x00000244)
 #define NVC8B5_SET_SEMAPHORE_B_LOWER                                            31:0
 #define NVC8B5_SET_SEMAPHORE_PAYLOAD                                            (0x00000248)
-#define NVC8B5_SET_SEMAPHORE_PAYLOAD_PAYLOAD                                    31:0
-#define NVC8B5_SET_SEMAPHORE_PAYLOAD_UPPER                                      (0x0000024C)
-#define NVC8B5_SET_SEMAPHORE_PAYLOAD_UPPER_PAYLOAD                              31:0
-#define NVC8B5_SET_RENDER_ENABLE_A                                              (0x00000254)
-#define NVC8B5_SET_RENDER_ENABLE_A_UPPER                                        24:0
-#define NVC8B5_SET_RENDER_ENABLE_B                                              (0x00000258)
-#define NVC8B5_SET_RENDER_ENABLE_B_LOWER                                        31:0
-#define NVC8B5_SET_RENDER_ENABLE_C                                              (0x0000025C)
-#define NVC8B5_SET_RENDER_ENABLE_C_MODE                                         2:0
-#define NVC8B5_SET_RENDER_ENABLE_C_MODE_FALSE                                   (0x00000000)
-#define NVC8B5_SET_RENDER_ENABLE_C_MODE_TRUE                                    (0x00000001)
-#define NVC8B5_SET_RENDER_ENABLE_C_MODE_CONDITIONAL                             (0x00000002)
-#define NVC8B5_SET_RENDER_ENABLE_C_MODE_RENDER_IF_EQUAL                         (0x00000003)
-#define NVC8B5_SET_RENDER_ENABLE_C_MODE_RENDER_IF_NOT_EQUAL                     (0x00000004)
-#define NVC8B5_SET_SRC_PHYS_MODE                                                (0x00000260)
 #define NVC8B5_SET_SRC_PHYS_MODE_TARGET                                         1:0
 #define NVC8B5_SET_SRC_PHYS_MODE_TARGET_LOCAL_FB                                (0x00000000)
 #define NVC8B5_SET_SRC_PHYS_MODE_TARGET_COHERENT_SYSMEM                         (0x00000001)
-#define NVC8B5_SET_SRC_PHYS_MODE_TARGET_NONCOHERENT_SYSMEM                      (0x00000002)
 #define NVC8B5_SET_SRC_PHYS_MODE_TARGET_PEERMEM                                 (0x00000003)
-#define NVC8B5_SET_SRC_PHYS_MODE_BASIC_KIND                                     5:2
 #define NVC8B5_SET_SRC_PHYS_MODE_PEER_ID                                        8:6
 #define NVC8B5_SET_SRC_PHYS_MODE_FLA                                            9:9
 #define NVC8B5_SET_DST_PHYS_MODE                                                (0x00000264)
 #define NVC8B5_SET_DST_PHYS_MODE_TARGET                                         1:0
 #define NVC8B5_SET_DST_PHYS_MODE_TARGET_LOCAL_FB                                (0x00000000)
 #define NVC8B5_SET_DST_PHYS_MODE_TARGET_COHERENT_SYSMEM                         (0x00000001)
-#define NVC8B5_SET_DST_PHYS_MODE_TARGET_NONCOHERENT_SYSMEM                      (0x00000002)
 #define NVC8B5_SET_DST_PHYS_MODE_TARGET_PEERMEM                                 (0x00000003)
-#define NVC8B5_SET_DST_PHYS_MODE_BASIC_KIND                                     5:2
-#define NVC8B5_SET_DST_PHYS_MODE_PEER_ID                                        8:6
-#define NVC8B5_SET_DST_PHYS_MODE_FLA                                            9:9
 #define NVC8B5_LAUNCH_DMA                                                       (0x00000300)
 #define NVC8B5_LAUNCH_DMA_DATA_TRANSFER_TYPE                                    1:0
 #define NVC8B5_LAUNCH_DMA_DATA_TRANSFER_TYPE_NONE                               (0x00000000)
@@ -167,80 +53,41 @@ typedef volatile struct _clc8b5_tag0 {
 #define NVC8B5_LAUNCH_DMA_FLUSH_TYPE_SYS                                        (0x00000000)
 #define NVC8B5_LAUNCH_DMA_FLUSH_TYPE_GL                                         (0x00000001)
 #define NVC8B5_LAUNCH_DMA_SEMAPHORE_TYPE                                        4:3
-#define NVC8B5_LAUNCH_DMA_SEMAPHORE_TYPE_NONE                                   (0x00000000)
 #define NVC8B5_LAUNCH_DMA_SEMAPHORE_TYPE_RELEASE_SEMAPHORE_NO_TIMESTAMP         (0x00000001)
 #define NVC8B5_LAUNCH_DMA_SEMAPHORE_TYPE_RELEASE_SEMAPHORE_WITH_TIMESTAMP       (0x00000002)
 #define NVC8B5_LAUNCH_DMA_SEMAPHORE_TYPE_RELEASE_ONE_WORD_SEMAPHORE             (0x00000001)
-#define NVC8B5_LAUNCH_DMA_SEMAPHORE_TYPE_RELEASE_FOUR_WORD_SEMAPHORE            (0x00000002)
-#define NVC8B5_LAUNCH_DMA_SEMAPHORE_TYPE_RELEASE_CONDITIONAL_INTR_SEMAPHORE     (0x00000003)
-#define NVC8B5_LAUNCH_DMA_INTERRUPT_TYPE                                        6:5
-#define NVC8B5_LAUNCH_DMA_INTERRUPT_TYPE_NONE                                   (0x00000000)
-#define NVC8B5_LAUNCH_DMA_INTERRUPT_TYPE_BLOCKING                               (0x00000001)
-#define NVC8B5_LAUNCH_DMA_INTERRUPT_TYPE_NON_BLOCKING                           (0x00000002)
 #define NVC8B5_LAUNCH_DMA_SRC_MEMORY_LAYOUT                                     7:7
-#define NVC8B5_LAUNCH_DMA_SRC_MEMORY_LAYOUT_BLOCKLINEAR                         (0x00000000)
 #define NVC8B5_LAUNCH_DMA_SRC_MEMORY_LAYOUT_PITCH                               (0x00000001)
 #define NVC8B5_LAUNCH_DMA_DST_MEMORY_LAYOUT                                     8:8
-#define NVC8B5_LAUNCH_DMA_DST_MEMORY_LAYOUT_BLOCKLINEAR                         (0x00000000)
 #define NVC8B5_LAUNCH_DMA_DST_MEMORY_LAYOUT_PITCH                               (0x00000001)
 #define NVC8B5_LAUNCH_DMA_MULTI_LINE_ENABLE                                     9:9
 #define NVC8B5_LAUNCH_DMA_MULTI_LINE_ENABLE_FALSE                               (0x00000000)
-#define NVC8B5_LAUNCH_DMA_MULTI_LINE_ENABLE_TRUE                                (0x00000001)
 #define NVC8B5_LAUNCH_DMA_REMAP_ENABLE                                          10:10
 #define NVC8B5_LAUNCH_DMA_REMAP_ENABLE_FALSE                                    (0x00000000)
 #define NVC8B5_LAUNCH_DMA_REMAP_ENABLE_TRUE                                     (0x00000001)
-#define NVC8B5_LAUNCH_DMA_FORCE_RMWDISABLE                                      11:11
-#define NVC8B5_LAUNCH_DMA_FORCE_RMWDISABLE_FALSE                                (0x00000000)
-#define NVC8B5_LAUNCH_DMA_FORCE_RMWDISABLE_TRUE                                 (0x00000001)
 #define NVC8B5_LAUNCH_DMA_SRC_TYPE                                              12:12
-#define NVC8B5_LAUNCH_DMA_SRC_TYPE_VIRTUAL                                      (0x00000000)
 #define NVC8B5_LAUNCH_DMA_SRC_TYPE_PHYSICAL                                     (0x00000001)
 #define NVC8B5_LAUNCH_DMA_DST_TYPE                                              13:13
 #define NVC8B5_LAUNCH_DMA_DST_TYPE_VIRTUAL                                      (0x00000000)
 #define NVC8B5_LAUNCH_DMA_DST_TYPE_PHYSICAL                                     (0x00000001)
 #define NVC8B5_LAUNCH_DMA_SEMAPHORE_REDUCTION                                   17:14
-#define NVC8B5_LAUNCH_DMA_SEMAPHORE_REDUCTION_IMIN                              (0x00000000)
-#define NVC8B5_LAUNCH_DMA_SEMAPHORE_REDUCTION_IMAX                              (0x00000001)
-#define NVC8B5_LAUNCH_DMA_SEMAPHORE_REDUCTION_IXOR                              (0x00000002)
-#define NVC8B5_LAUNCH_DMA_SEMAPHORE_REDUCTION_IAND                              (0x00000003)
-#define NVC8B5_LAUNCH_DMA_SEMAPHORE_REDUCTION_IOR                               (0x00000004)
-#define NVC8B5_LAUNCH_DMA_SEMAPHORE_REDUCTION_IADD                              (0x00000005)
 #define NVC8B5_LAUNCH_DMA_SEMAPHORE_REDUCTION_INC                               (0x00000006)
-#define NVC8B5_LAUNCH_DMA_SEMAPHORE_REDUCTION_DEC                               (0x00000007)
-#define NVC8B5_LAUNCH_DMA_SEMAPHORE_REDUCTION_INVALIDA                          (0x00000008)
-#define NVC8B5_LAUNCH_DMA_SEMAPHORE_REDUCTION_INVALIDB                          (0x00000009)
-#define NVC8B5_LAUNCH_DMA_SEMAPHORE_REDUCTION_FADD                              (0x0000000A)
-#define NVC8B5_LAUNCH_DMA_SEMAPHORE_REDUCTION_FMIN                              (0x0000000B)
-#define NVC8B5_LAUNCH_DMA_SEMAPHORE_REDUCTION_FMAX                              (0x0000000C)
-#define NVC8B5_LAUNCH_DMA_SEMAPHORE_REDUCTION_INVALIDC                          (0x0000000D)
-#define NVC8B5_LAUNCH_DMA_SEMAPHORE_REDUCTION_INVALIDD                          (0x0000000E)
-#define NVC8B5_LAUNCH_DMA_SEMAPHORE_REDUCTION_INVALIDE                          (0x0000000F)
 #define NVC8B5_LAUNCH_DMA_SEMAPHORE_REDUCTION_SIGN                              18:18
-#define NVC8B5_LAUNCH_DMA_SEMAPHORE_REDUCTION_SIGN_SIGNED                       (0x00000000)
 #define NVC8B5_LAUNCH_DMA_SEMAPHORE_REDUCTION_SIGN_UNSIGNED                     (0x00000001)
 #define NVC8B5_LAUNCH_DMA_SEMAPHORE_REDUCTION_ENABLE                            19:19
-#define NVC8B5_LAUNCH_DMA_SEMAPHORE_REDUCTION_ENABLE_FALSE                      (0x00000000)
 #define NVC8B5_LAUNCH_DMA_SEMAPHORE_REDUCTION_ENABLE_TRUE                       (0x00000001)
 #define NVC8B5_LAUNCH_DMA_COPY_TYPE                                             21:20
 #define NVC8B5_LAUNCH_DMA_COPY_TYPE_PROT2PROT                                   (0x00000000)
 #define NVC8B5_LAUNCH_DMA_COPY_TYPE_DEFAULT                                     (0x00000000)
 #define NVC8B5_LAUNCH_DMA_COPY_TYPE_SECURE                                      (0x00000001)
 #define NVC8B5_LAUNCH_DMA_COPY_TYPE_NONPROT2NONPROT                             (0x00000002)
-#define NVC8B5_LAUNCH_DMA_COPY_TYPE_RESERVED                                    (0x00000003)
-#define NVC8B5_LAUNCH_DMA_VPRMODE                                               22:22
-#define NVC8B5_LAUNCH_DMA_VPRMODE_VPR_NONE                                      (0x00000000)
-#define NVC8B5_LAUNCH_DMA_VPRMODE_VPR_VID2VID                                   (0x00000001)
 #define NVC8B5_LAUNCH_DMA_MEMORY_SCRUB_ENABLE                                   23:23
 #define NVC8B5_LAUNCH_DMA_MEMORY_SCRUB_ENABLE_FALSE                             (0x00000000)
 #define NVC8B5_LAUNCH_DMA_MEMORY_SCRUB_ENABLE_TRUE                              (0x00000001)
-#define NVC8B5_LAUNCH_DMA_RESERVED_START_OF_COPY                                24:24
 #define NVC8B5_LAUNCH_DMA_DISABLE_PLC                                           26:26
-#define NVC8B5_LAUNCH_DMA_DISABLE_PLC_FALSE                                     (0x00000000)
 #define NVC8B5_LAUNCH_DMA_DISABLE_PLC_TRUE                                      (0x00000001)
 #define NVC8B5_LAUNCH_DMA_SEMAPHORE_PAYLOAD_SIZE                                27:27
 #define NVC8B5_LAUNCH_DMA_SEMAPHORE_PAYLOAD_SIZE_ONE_WORD                       (0x00000000)
-#define NVC8B5_LAUNCH_DMA_SEMAPHORE_PAYLOAD_SIZE_TWO_WORD                       (0x00000001)
-#define NVC8B5_LAUNCH_DMA_RESERVED_ERR_CODE                                     31:28
 #define NVC8B5_OFFSET_IN_UPPER                                                  (0x00000400)
 #define NVC8B5_OFFSET_IN_UPPER_UPPER                                            24:0
 #define NVC8B5_OFFSET_IN_LOWER                                                  (0x00000404)
@@ -249,41 +96,11 @@ typedef volatile struct _clc8b5_tag0 {
 #define NVC8B5_OFFSET_OUT_UPPER_UPPER                                           24:0
 #define NVC8B5_OFFSET_OUT_LOWER                                                 (0x0000040C)
 #define NVC8B5_OFFSET_OUT_LOWER_VALUE                                           31:0
-#define NVC8B5_PITCH_IN                                                         (0x00000410)
-#define NVC8B5_PITCH_IN_VALUE                                                   31:0
-#define NVC8B5_PITCH_OUT                                                        (0x00000414)
-#define NVC8B5_PITCH_OUT_VALUE                                                  31:0
 #define NVC8B5_LINE_LENGTH_IN                                                   (0x00000418)
-#define NVC8B5_LINE_LENGTH_IN_VALUE                                             31:0
-#define NVC8B5_LINE_COUNT                                                       (0x0000041C)
-#define NVC8B5_LINE_COUNT_VALUE                                                 31:0
 #define NVC8B5_SET_SECURE_COPY_MODE                                             (0x00000500)
 #define NVC8B5_SET_SECURE_COPY_MODE_MODE                                        0:0
 #define NVC8B5_SET_SECURE_COPY_MODE_MODE_ENCRYPT                                (0x00000000)
 #define NVC8B5_SET_SECURE_COPY_MODE_MODE_DECRYPT                                (0x00000001)
-#define NVC8B5_SET_SECURE_COPY_MODE_RESERVED_SRC_TARGET                         20:19
-#define NVC8B5_SET_SECURE_COPY_MODE_RESERVED_SRC_TARGET_LOCAL_FB                (0x00000000)
-#define NVC8B5_SET_SECURE_COPY_MODE_RESERVED_SRC_TARGET_COHERENT_SYSMEM         (0x00000001)
-#define NVC8B5_SET_SECURE_COPY_MODE_RESERVED_SRC_TARGET_NONCOHERENT_SYSMEM      (0x00000002)
-#define NVC8B5_SET_SECURE_COPY_MODE_RESERVED_SRC_TARGET_PEERMEM                 (0x00000003)
-#define NVC8B5_SET_SECURE_COPY_MODE_RESERVED_SRC_PEER_ID                        23:21
-#define NVC8B5_SET_SECURE_COPY_MODE_RESERVED_SRC_FLA                            24:24
-#define NVC8B5_SET_SECURE_COPY_MODE_RESERVED_DST_TARGET                         26:25
-#define NVC8B5_SET_SECURE_COPY_MODE_RESERVED_DST_TARGET_LOCAL_FB                (0x00000000)
-#define NVC8B5_SET_SECURE_COPY_MODE_RESERVED_DST_TARGET_COHERENT_SYSMEM         (0x00000001)
-#define NVC8B5_SET_SECURE_COPY_MODE_RESERVED_DST_TARGET_NONCOHERENT_SYSMEM      (0x00000002)
-#define NVC8B5_SET_SECURE_COPY_MODE_RESERVED_DST_TARGET_PEERMEM                 (0x00000003)
-#define NVC8B5_SET_SECURE_COPY_MODE_RESERVED_DST_PEER_ID                        29:27
-#define NVC8B5_SET_SECURE_COPY_MODE_RESERVED_DST_FLA                            30:30
-#define NVC8B5_SET_SECURE_COPY_MODE_RESERVED_END_OF_COPY                        31:31
-#define NVC8B5_SET_DECRYPT_IV0                                                  (0x00000504)
-#define NVC8B5_SET_DECRYPT_IV0_VALUE                                            31:0
-#define NVC8B5_SET_DECRYPT_IV1                                                  (0x00000508)
-#define NVC8B5_SET_DECRYPT_IV1_VALUE                                            31:0
-#define NVC8B5_SET_DECRYPT_IV2                                                  (0x0000050C)
-#define NVC8B5_SET_DECRYPT_IV2_VALUE                                            31:0
-#define NVC8B5_RESERVED_SET_AESCOUNTER                                          (0x00000510)
-#define NVC8B5_RESERVED_SET_AESCOUNTER_VALUE                                    31:0
 #define NVC8B5_SET_DECRYPT_AUTH_TAG_COMPARE_ADDR_UPPER                          (0x00000514)
 #define NVC8B5_SET_DECRYPT_AUTH_TAG_COMPARE_ADDR_UPPER_UPPER                    24:0
 #define NVC8B5_SET_DECRYPT_AUTH_TAG_COMPARE_ADDR_LOWER                          (0x00000518)
@@ -299,132 +116,18 @@ typedef volatile struct _clc8b5_tag0 {
 #define NVC8B5_SET_MEMORY_SCRUB_PARAMETERS                                      (0x000006FC)
 #define NVC8B5_SET_MEMORY_SCRUB_PARAMETERS_DISCARDABLE                          0:0
 #define NVC8B5_SET_MEMORY_SCRUB_PARAMETERS_DISCARDABLE_FALSE                    (0x00000000)
-#define NVC8B5_SET_MEMORY_SCRUB_PARAMETERS_DISCARDABLE_TRUE                     (0x00000001)
 #define NVC8B5_SET_REMAP_CONST_A                                                (0x00000700)
-#define NVC8B5_SET_REMAP_CONST_A_V                                              31:0
 #define NVC8B5_SET_REMAP_CONST_B                                                (0x00000704)
-#define NVC8B5_SET_REMAP_CONST_B_V                                              31:0
 #define NVC8B5_SET_REMAP_COMPONENTS                                             (0x00000708)
 #define NVC8B5_SET_REMAP_COMPONENTS_DST_X                                       2:0
-#define NVC8B5_SET_REMAP_COMPONENTS_DST_X_SRC_X                                 (0x00000000)
-#define NVC8B5_SET_REMAP_COMPONENTS_DST_X_SRC_Y                                 (0x00000001)
-#define NVC8B5_SET_REMAP_COMPONENTS_DST_X_SRC_Z                                 (0x00000002)
-#define NVC8B5_SET_REMAP_COMPONENTS_DST_X_SRC_W                                 (0x00000003)
 #define NVC8B5_SET_REMAP_COMPONENTS_DST_X_CONST_A                               (0x00000004)
 #define NVC8B5_SET_REMAP_COMPONENTS_DST_X_CONST_B                               (0x00000005)
-#define NVC8B5_SET_REMAP_COMPONENTS_DST_X_NO_WRITE                              (0x00000006)
 #define NVC8B5_SET_REMAP_COMPONENTS_DST_Y                                       6:4
-#define NVC8B5_SET_REMAP_COMPONENTS_DST_Y_SRC_X                                 (0x00000000)
-#define NVC8B5_SET_REMAP_COMPONENTS_DST_Y_SRC_Y                                 (0x00000001)
-#define NVC8B5_SET_REMAP_COMPONENTS_DST_Y_SRC_Z                                 (0x00000002)
-#define NVC8B5_SET_REMAP_COMPONENTS_DST_Y_SRC_W                                 (0x00000003)
-#define NVC8B5_SET_REMAP_COMPONENTS_DST_Y_CONST_A                               (0x00000004)
 #define NVC8B5_SET_REMAP_COMPONENTS_DST_Y_CONST_B                               (0x00000005)
-#define NVC8B5_SET_REMAP_COMPONENTS_DST_Y_NO_WRITE                              (0x00000006)
-#define NVC8B5_SET_REMAP_COMPONENTS_DST_Z                                       10:8
-#define NVC8B5_SET_REMAP_COMPONENTS_DST_Z_SRC_X                                 (0x00000000)
-#define NVC8B5_SET_REMAP_COMPONENTS_DST_Z_SRC_Y                                 (0x00000001)
-#define NVC8B5_SET_REMAP_COMPONENTS_DST_Z_SRC_Z                                 (0x00000002)
-#define NVC8B5_SET_REMAP_COMPONENTS_DST_Z_SRC_W                                 (0x00000003)
-#define NVC8B5_SET_REMAP_COMPONENTS_DST_Z_CONST_A                               (0x00000004)
-#define NVC8B5_SET_REMAP_COMPONENTS_DST_Z_CONST_B                               (0x00000005)
-#define NVC8B5_SET_REMAP_COMPONENTS_DST_Z_NO_WRITE                              (0x00000006)
-#define NVC8B5_SET_REMAP_COMPONENTS_DST_W                                       14:12
-#define NVC8B5_SET_REMAP_COMPONENTS_DST_W_SRC_X                                 (0x00000000)
-#define NVC8B5_SET_REMAP_COMPONENTS_DST_W_SRC_Y                                 (0x00000001)
-#define NVC8B5_SET_REMAP_COMPONENTS_DST_W_SRC_Z                                 (0x00000002)
-#define NVC8B5_SET_REMAP_COMPONENTS_DST_W_SRC_W                                 (0x00000003)
-#define NVC8B5_SET_REMAP_COMPONENTS_DST_W_CONST_A                               (0x00000004)
-#define NVC8B5_SET_REMAP_COMPONENTS_DST_W_CONST_B                               (0x00000005)
-#define NVC8B5_SET_REMAP_COMPONENTS_DST_W_NO_WRITE                              (0x00000006)
 #define NVC8B5_SET_REMAP_COMPONENTS_COMPONENT_SIZE                              17:16
 #define NVC8B5_SET_REMAP_COMPONENTS_COMPONENT_SIZE_ONE                          (0x00000000)
-#define NVC8B5_SET_REMAP_COMPONENTS_COMPONENT_SIZE_TWO                          (0x00000001)
-#define NVC8B5_SET_REMAP_COMPONENTS_COMPONENT_SIZE_THREE                        (0x00000002)
 #define NVC8B5_SET_REMAP_COMPONENTS_COMPONENT_SIZE_FOUR                         (0x00000003)
-#define NVC8B5_SET_REMAP_COMPONENTS_NUM_SRC_COMPONENTS                          21:20
-#define NVC8B5_SET_REMAP_COMPONENTS_NUM_SRC_COMPONENTS_ONE                      (0x00000000)
-#define NVC8B5_SET_REMAP_COMPONENTS_NUM_SRC_COMPONENTS_TWO                      (0x00000001)
-#define NVC8B5_SET_REMAP_COMPONENTS_NUM_SRC_COMPONENTS_THREE                    (0x00000002)
-#define NVC8B5_SET_REMAP_COMPONENTS_NUM_SRC_COMPONENTS_FOUR                     (0x00000003)
 #define NVC8B5_SET_REMAP_COMPONENTS_NUM_DST_COMPONENTS                          25:24
 #define NVC8B5_SET_REMAP_COMPONENTS_NUM_DST_COMPONENTS_ONE                      (0x00000000)
 #define NVC8B5_SET_REMAP_COMPONENTS_NUM_DST_COMPONENTS_TWO                      (0x00000001)
-#define NVC8B5_SET_REMAP_COMPONENTS_NUM_DST_COMPONENTS_THREE                    (0x00000002)
-#define NVC8B5_SET_REMAP_COMPONENTS_NUM_DST_COMPONENTS_FOUR                     (0x00000003)
-#define NVC8B5_SET_DST_BLOCK_SIZE                                               (0x0000070C)
-#define NVC8B5_SET_DST_BLOCK_SIZE_WIDTH                                         3:0
-#define NVC8B5_SET_DST_BLOCK_SIZE_WIDTH_ONE_GOB                                 (0x00000000)
-#define NVC8B5_SET_DST_BLOCK_SIZE_HEIGHT                                        7:4
-#define NVC8B5_SET_DST_BLOCK_SIZE_HEIGHT_ONE_GOB                                (0x00000000)
-#define NVC8B5_SET_DST_BLOCK_SIZE_HEIGHT_TWO_GOBS                               (0x00000001)
-#define NVC8B5_SET_DST_BLOCK_SIZE_HEIGHT_FOUR_GOBS                              (0x00000002)
-#define NVC8B5_SET_DST_BLOCK_SIZE_HEIGHT_EIGHT_GOBS                             (0x00000003)
-#define NVC8B5_SET_DST_BLOCK_SIZE_HEIGHT_SIXTEEN_GOBS                           (0x00000004)
-#define NVC8B5_SET_DST_BLOCK_SIZE_HEIGHT_THIRTYTWO_GOBS                         (0x00000005)
-#define NVC8B5_SET_DST_BLOCK_SIZE_DEPTH                                         11:8
-#define NVC8B5_SET_DST_BLOCK_SIZE_DEPTH_ONE_GOB                                 (0x00000000)
-#define NVC8B5_SET_DST_BLOCK_SIZE_DEPTH_TWO_GOBS                                (0x00000001)
-#define NVC8B5_SET_DST_BLOCK_SIZE_DEPTH_FOUR_GOBS                               (0x00000002)
-#define NVC8B5_SET_DST_BLOCK_SIZE_DEPTH_EIGHT_GOBS                              (0x00000003)
-#define NVC8B5_SET_DST_BLOCK_SIZE_DEPTH_SIXTEEN_GOBS                            (0x00000004)
-#define NVC8B5_SET_DST_BLOCK_SIZE_DEPTH_THIRTYTWO_GOBS                          (0x00000005)
-#define NVC8B5_SET_DST_BLOCK_SIZE_GOB_HEIGHT                                    15:12
-#define NVC8B5_SET_DST_BLOCK_SIZE_GOB_HEIGHT_GOB_HEIGHT_FERMI_8                 (0x00000001)
-#define NVC8B5_SET_DST_WIDTH                                                    (0x00000710)
-#define NVC8B5_SET_DST_WIDTH_V                                                  31:0
-#define NVC8B5_SET_DST_HEIGHT                                                   (0x00000714)
-#define NVC8B5_SET_DST_HEIGHT_V                                                 31:0
-#define NVC8B5_SET_DST_DEPTH                                                    (0x00000718)
-#define NVC8B5_SET_DST_DEPTH_V                                                  31:0
-#define NVC8B5_SET_DST_LAYER                                                    (0x0000071C)
-#define NVC8B5_SET_DST_LAYER_V                                                  31:0
-#define NVC8B5_SET_DST_ORIGIN                                                   (0x00000720)
-#define NVC8B5_SET_DST_ORIGIN_X                                                 15:0
-#define NVC8B5_SET_DST_ORIGIN_Y                                                 31:16
-#define NVC8B5_SET_SRC_BLOCK_SIZE                                               (0x00000728)
-#define NVC8B5_SET_SRC_BLOCK_SIZE_WIDTH                                         3:0
-#define NVC8B5_SET_SRC_BLOCK_SIZE_WIDTH_ONE_GOB                                 (0x00000000)
-#define NVC8B5_SET_SRC_BLOCK_SIZE_HEIGHT                                        7:4
-#define NVC8B5_SET_SRC_BLOCK_SIZE_HEIGHT_ONE_GOB                                (0x00000000)
-#define NVC8B5_SET_SRC_BLOCK_SIZE_HEIGHT_TWO_GOBS                               (0x00000001)
-#define NVC8B5_SET_SRC_BLOCK_SIZE_HEIGHT_FOUR_GOBS                              (0x00000002)
-#define NVC8B5_SET_SRC_BLOCK_SIZE_HEIGHT_EIGHT_GOBS                             (0x00000003)
-#define NVC8B5_SET_SRC_BLOCK_SIZE_HEIGHT_SIXTEEN_GOBS                           (0x00000004)
-#define NVC8B5_SET_SRC_BLOCK_SIZE_HEIGHT_THIRTYTWO_GOBS                         (0x00000005)
-#define NVC8B5_SET_SRC_BLOCK_SIZE_DEPTH                                         11:8
-#define NVC8B5_SET_SRC_BLOCK_SIZE_DEPTH_ONE_GOB                                 (0x00000000)
-#define NVC8B5_SET_SRC_BLOCK_SIZE_DEPTH_TWO_GOBS                                (0x00000001)
-#define NVC8B5_SET_SRC_BLOCK_SIZE_DEPTH_FOUR_GOBS                               (0x00000002)
-#define NVC8B5_SET_SRC_BLOCK_SIZE_DEPTH_EIGHT_GOBS                              (0x00000003)
-#define NVC8B5_SET_SRC_BLOCK_SIZE_DEPTH_SIXTEEN_GOBS                            (0x00000004)
-#define NVC8B5_SET_SRC_BLOCK_SIZE_DEPTH_THIRTYTWO_GOBS                          (0x00000005)
-#define NVC8B5_SET_SRC_BLOCK_SIZE_GOB_HEIGHT                                    15:12
-#define NVC8B5_SET_SRC_BLOCK_SIZE_GOB_HEIGHT_GOB_HEIGHT_FERMI_8                 (0x00000001)
-#define NVC8B5_SET_SRC_WIDTH                                                    (0x0000072C)
-#define NVC8B5_SET_SRC_WIDTH_V                                                  31:0
-#define NVC8B5_SET_SRC_HEIGHT                                                   (0x00000730)
-#define NVC8B5_SET_SRC_HEIGHT_V                                                 31:0
-#define NVC8B5_SET_SRC_DEPTH                                                    (0x00000734)
-#define NVC8B5_SET_SRC_DEPTH_V                                                  31:0
-#define NVC8B5_SET_SRC_LAYER                                                    (0x00000738)
-#define NVC8B5_SET_SRC_LAYER_V                                                  31:0
-#define NVC8B5_SET_SRC_ORIGIN                                                   (0x0000073C)
-#define NVC8B5_SET_SRC_ORIGIN_X                                                 15:0
-#define NVC8B5_SET_SRC_ORIGIN_Y                                                 31:16
-#define NVC8B5_SRC_ORIGIN_X                                                     (0x00000744)
-#define NVC8B5_SRC_ORIGIN_X_VALUE                                               31:0
-#define NVC8B5_SRC_ORIGIN_Y                                                     (0x00000748)
-#define NVC8B5_SRC_ORIGIN_Y_VALUE                                               31:0
-#define NVC8B5_DST_ORIGIN_X                                                     (0x0000074C)
-#define NVC8B5_DST_ORIGIN_X_VALUE                                               31:0
-#define NVC8B5_DST_ORIGIN_Y                                                     (0x00000750)
-#define NVC8B5_DST_ORIGIN_Y_VALUE                                               31:0
-#define NVC8B5_PM_TRIGGER_END                                                   (0x00001114)
-#define NVC8B5_PM_TRIGGER_END_V                                                 31:0
-
-#ifdef __cplusplus
-};     /* extern "C" */
-#endif
-#endif // _clc8b5_h
-
+#endif // __gh100_clc8b5_h__

kernel-open/nvidia-uvm/clc96f.h

@@ -0,0 +1,113 @@
+/*
+ * SPDX-FileCopyrightText: Copyright (c) 2003-2022 NVIDIA CORPORATION & AFFILIATES
+ * SPDX-License-Identifier: MIT
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
+ * DEALINGS IN THE SOFTWARE.
+ */
+
+#ifndef __gb100_clc96f_h__
+#define __gb100_clc96f_h__
+
+#define  BLACKWELL_CHANNEL_GPFIFO_A                           (0x0000C96F)
+
+typedef volatile struct Nvc96fControl_struct {
+ NvU32 Ignored00[0x23];        /*                                  0000-008b*/
+ NvU32 GPPut;                   /* GP FIFO put offset               008c-008f*/
+ NvU32 Ignored01[0x5c];
+} Nvc96fControl, BlackwellAControlGPFifo;
+
+#define NVC96F_SET_OBJECT                                          (0x00000000)
+#define NVC96F_MEM_OP_A                                            (0x00000028)
+#define NVC96F_MEM_OP_A_TLB_INVALIDATE_INVALIDATION_SIZE                   5:0  // Used to specify size of invalidate, used for invalidates which are not of the REPLAY_CANCEL_TARGETED type
+#define NVC96F_MEM_OP_A_TLB_INVALIDATE_INVAL_SCOPE                         7:6  // only relevant for invalidates with NVC96F_MEM_OP_C_TLB_INVALIDATE_REPLAY_NONE for invalidating  link TLB only, or non-link TLB only or all TLBs
+#define NVC96F_MEM_OP_A_TLB_INVALIDATE_INVAL_SCOPE_NON_LINK_TLBS             2
+#define NVC96F_MEM_OP_A_TLB_INVALIDATE_SYSMEMBAR                         11:11
+#define NVC96F_MEM_OP_A_TLB_INVALIDATE_SYSMEMBAR_EN                 0x00000001
+#define NVC96F_MEM_OP_A_TLB_INVALIDATE_SYSMEMBAR_DIS                0x00000000
+#define NVC96F_MEM_OP_A_TLB_INVALIDATE_TARGET_ADDR_LO                    31:12
+#define NVC96F_MEM_OP_B                                            (0x0000002c)
+#define NVC96F_MEM_OP_B_TLB_INVALIDATE_TARGET_ADDR_HI                     31:0
+#define NVC96F_MEM_OP_C                                            (0x00000030)
+#define NVC96F_MEM_OP_C_TLB_INVALIDATE_PDB                                 0:0
+#define NVC96F_MEM_OP_C_TLB_INVALIDATE_PDB_ONE                      0x00000000
+#define NVC96F_MEM_OP_C_TLB_INVALIDATE_PDB_ALL                      0x00000001  // Probably nonsensical for MMU_TLB_INVALIDATE_TARGETED
+#define NVC96F_MEM_OP_C_TLB_INVALIDATE_GPC                                 1:1
+#define NVC96F_MEM_OP_C_TLB_INVALIDATE_GPC_ENABLE                   0x00000000
+#define NVC96F_MEM_OP_C_TLB_INVALIDATE_GPC_DISABLE                  0x00000001
+#define NVC96F_MEM_OP_C_TLB_INVALIDATE_REPLAY                              4:2  // only relevant if GPC ENABLE
+#define NVC96F_MEM_OP_C_TLB_INVALIDATE_REPLAY_NONE                  0x00000000
+#define NVC96F_MEM_OP_C_TLB_INVALIDATE_ACK_TYPE                            6:5  // only relevant if GPC ENABLE
+#define NVC96F_MEM_OP_C_TLB_INVALIDATE_ACK_TYPE_NONE                0x00000000
+#define NVC96F_MEM_OP_C_TLB_INVALIDATE_ACK_TYPE_GLOBALLY            0x00000001
+#define NVC96F_MEM_OP_C_TLB_INVALIDATE_PAGE_TABLE_LEVEL                    9:7  // Invalidate affects this level and all below
+#define NVC96F_MEM_OP_C_TLB_INVALIDATE_PAGE_TABLE_LEVEL_ALL         0x00000000  // Invalidate tlb caches at all levels of the page table
+#define NVC96F_MEM_OP_C_TLB_INVALIDATE_PAGE_TABLE_LEVEL_PTE_ONLY    0x00000001
+#define NVC96F_MEM_OP_C_TLB_INVALIDATE_PAGE_TABLE_LEVEL_UP_TO_PDE4  0x00000006
+#define NVC96F_MEM_OP_C_TLB_INVALIDATE_PDB_APERTURE                          11:10  // only relevant if PDB_ONE
+#define NVC96F_MEM_OP_C_TLB_INVALIDATE_PDB_APERTURE_VID_MEM             0x00000000
+#define NVC96F_MEM_OP_C_TLB_INVALIDATE_PDB_APERTURE_SYS_MEM_COHERENT    0x00000002
+#define NVC96F_MEM_OP_C_TLB_INVALIDATE_PDB_ADDR_LO                       31:12  // only relevant if PDB_ONE
+
+#define NVC96F_MEM_OP_D                                            (0x00000034)
+#define NVC96F_MEM_OP_D_TLB_INVALIDATE_PDB_ADDR_HI                        26:0  // only relevant if PDB_ONE
+#define NVC96F_MEM_OP_D_OPERATION                                        31:27
+#define NVC96F_MEM_OP_D_OPERATION_MMU_TLB_INVALIDATE                0x00000009
+#define NVC96F_MEM_OP_D_OPERATION_MMU_TLB_INVALIDATE_TARGETED       0x0000000a
+#define NVC96F_MEM_OP_D_OPERATION_L2_FLUSH_DIRTY                    0x00000010
+#define NVC96F_MEM_OP_D_OPERATION_L2_SYSMEM_NCOH_INVALIDATE         0x00000011
+
+#define NVC96F_SEM_ADDR_LO                                         (0x0000005c)
+#define NVC96F_SEM_ADDR_LO_OFFSET                                         31:2
+#define NVC96F_SEM_ADDR_HI                                         (0x00000060)
+#define NVC96F_SEM_ADDR_HI_OFFSET                                         24:0
+#define NVC96F_SEM_PAYLOAD_LO                                      (0x00000064)
+#define NVC96F_SEM_PAYLOAD_HI                                      (0x00000068)
+#define NVC96F_SEM_EXECUTE                                         (0x0000006c)
+#define NVC96F_SEM_EXECUTE_OPERATION                                       2:0
+#define NVC96F_SEM_EXECUTE_OPERATION_ACQUIRE                        0x00000000
+#define NVC96F_SEM_EXECUTE_OPERATION_RELEASE                        0x00000001
+#define NVC96F_SEM_EXECUTE_RELEASE_WFI                                   20:20
+#define NVC96F_SEM_EXECUTE_RELEASE_WFI_DIS                          0x00000000
+#define NVC96F_SEM_EXECUTE_PAYLOAD_SIZE                                  24:24
+#define NVC96F_SEM_EXECUTE_PAYLOAD_SIZE_32BIT                       0x00000000
+
+/* GPFIFO entry format */
+#define NVC96F_GP_ENTRY__SIZE                                          8
+#define NVC96F_GP_ENTRY0_FETCH                                       0:0
+#define NVC96F_GP_ENTRY0_FETCH_UNCONDITIONAL                  0x00000000
+#define NVC96F_GP_ENTRY0_FETCH_CONDITIONAL                    0x00000001
+#define NVC96F_GP_ENTRY0_GET                                        31:2
+#define NVC96F_GP_ENTRY0_OPERAND                                    31:0
+#define NVC96F_GP_ENTRY0_PB_EXTENDED_BASE_OPERAND                   24:8
+#define NVC96F_GP_ENTRY1_GET_HI                                      7:0
+#define NVC96F_GP_ENTRY1_LEVEL                                       9:9
+#define NVC96F_GP_ENTRY1_LEVEL_MAIN                           0x00000000
+#define NVC96F_GP_ENTRY1_LEVEL_SUBROUTINE                     0x00000001
+#define NVC96F_GP_ENTRY1_LENGTH                                    30:10
+#define NVC96F_GP_ENTRY1_SYNC                                      31:31
+#define NVC96F_GP_ENTRY1_SYNC_PROCEED                         0x00000000
+#define NVC96F_GP_ENTRY1_SYNC_WAIT                            0x00000001
+#define NVC96F_GP_ENTRY1_OPCODE                                      7:0
+#define NVC96F_GP_ENTRY1_OPCODE_NOP                           0x00000000
+#define NVC96F_GP_ENTRY1_OPCODE_ILLEGAL                       0x00000001
+#define NVC96F_GP_ENTRY1_OPCODE_GP_CRC                        0x00000002
+#define NVC96F_GP_ENTRY1_OPCODE_PB_CRC                        0x00000003
+#define NVC96F_GP_ENTRY1_OPCODE_SET_PB_SEGMENT_EXTENDED_BASE  0x00000004
+
+#endif // __gb100_clc96f_h__