--- title: NVIDIA GPU integration source: https://docs.newrelic.com/docs/infrastructure/host-integrations/host-integrations-list/nvidia-gpu-integration --- Our NVIDIA GPU integration lets you monitor the status of your GPUs. This integration uses our infrastructure agent with the Flex integration, which lets us access NVIDIA's SMI utility. ![NVIDIA GPUs dashboard](https://docs.newrelic.com/images/infrastructure_screenshot-full_NVIDIA-GPU-dashboard.webp "NVIDIA GPUs dashboard") After you set up our NVIDIA GPU integration, we give you a dashboard for your GPU metrics. When you install, you'll get a pre-built dashboard containing crucial GPU metrics: - GPU utilization - ECC error counts - Active compute processes - Clock and performance states - Temperature and fan speed - Dynamic and static information about each supported device ## Install the infrastructure agent To capture data with New Relic, install our infrastructure agent. Our infrastructure agent collects and ingests data so you can keep track of your GPUs performance. You can install the infrastructure agent two different ways: - Our [guided install](https://one.newrelic.com/nr1-core?state=4f81feab-35f7-e97e-9903-52510f8542bd) is a CLI tool that inspects your system and installs the infrastructure agent alongside the application monitoring agent that best works for your system. To learn more about how our guided install works, check out our [Guided install overview](https://docs.newrelic.com/docs/infrastructure/host-integrations/installation/new-relic-guided-install-overview). - If you'd rather install our infrastructure agent manually, you can follow a tutorial for manual installation for [Linux](https://docs.newrelic.com/docs/infrastructure/install-infrastructure-agent/linux-installation/install-infrastructure-monitoring-agent-linux), [Windows](https://docs.newrelic.com/docs/infrastructure/install-infrastructure-agent/windows-installation/install-infrastructure-monitoring-agent-windows/). ## Configure Flex integration for NVIDIA GPUs Flex comes bundled with the New Relic infrastructure agent and it can be integrated with the [NVIDIA SMI](https://developer.nvidia.com/nvidia-management-library-nvml), a command line utility to monitor NVIDIA GPU devices. > #### ⚠️ IMPORTANT > > nvidia-smi ships pre-installed with NVIDIA GPU display drivers on Linux and Windows Server. Follow these steps to configure Flex: 1. Create a file named `nvidia-smi-gpu-monitoring.yml` in this path: ```shell sudo touch /etc/newrelic-infra/integrations.d/nvidia-smi-gpu-monitoring.yml ``` You may also download from the [git repository](https://github.com/newrelic/nri-flex/blob/master/examples/nvidia-smi-gpu-monitoring.yml). 2. Update the `nvidia-smi-gpu-monitoring.yml` file with the integration config: ```yml --- integrations: - name: nri-flex # interval: 30s config: name: NvidiaSMI variable_store: metrics: "name,driver_version,count,serial,pci.bus_id,pci.domain,pci.bus,\ pci.device_id,pci.sub_device_id,pcie.link.gen.current,pcie.link.gen.max,\ pcie.link.width.current,pcie.link.width.max,index,display_mode,display_active,\ persistence_mode,accounting.mode,accounting.buffer_size,driver_model.current,\ driver_model.pending,vbios_version,inforom.img,inforom.oem,inforom.ecc,inforom.pwr,\ gom.current,gom.pending,fan.speed,pstate,clocks_throttle_reasons.supported,\ clocks_throttle_reasons.gpu_idle,clocks_throttle_reasons.applications_clocks_setting,\ clocks_throttle_reasons.sw_power_cap,clocks_throttle_reasons.hw_slowdown,clocks_throttle_reasons.hw_thermal_slowdown,\ clocks_throttle_reasons.hw_power_brake_slowdown,clocks_throttle_reasons.sw_thermal_slowdown,\ clocks_throttle_reasons.sync_boost,memory.total,memory.used,memory.free,compute_mode,\ utilization.gpu,utilization.memory,encoder.stats.sessionCount,encoder.stats.averageFps,\ encoder.stats.averageLatency,ecc.mode.current,ecc.mode.pending,ecc.errors.corrected.volatile.device_memory,\ ecc.errors.corrected.volatile.dram,ecc.errors.corrected.volatile.register_file,ecc.errors.corrected.volatile.l1_cache,\ ecc.errors.corrected.volatile.l2_cache,ecc.errors.corrected.volatile.texture_memory,ecc.errors.corrected.volatile.cbu,\ ecc.errors.corrected.volatile.sram,ecc.errors.corrected.volatile.total,ecc.errors.corrected.aggregate.device_memory,\ ecc.errors.corrected.aggregate.dram,ecc.errors.corrected.aggregate.register_file,ecc.errors.corrected.aggregate.l1_cache,\ ecc.errors.corrected.aggregate.l2_cache,ecc.errors.corrected.aggregate.texture_memory,ecc.errors.corrected.aggregate.cbu,\ ecc.errors.corrected.aggregate.sram,ecc.errors.corrected.aggregate.total,ecc.errors.uncorrected.volatile.device_memory,\ ecc.errors.uncorrected.volatile.dram,ecc.errors.uncorrected.volatile.register_file,ecc.errors.uncorrected.volatile.l1_cache,\ ecc.errors.uncorrected.volatile.l2_cache,ecc.errors.uncorrected.volatile.texture_memory,ecc.errors.uncorrected.volatile.cbu,\ ecc.errors.uncorrected.volatile.sram,ecc.errors.uncorrected.volatile.total,ecc.errors.uncorrected.aggregate.device_memory,\ ecc.errors.uncorrected.aggregate.dram,ecc.errors.uncorrected.aggregate.register_file,ecc.errors.uncorrected.aggregate.l1_cache,\ ecc.errors.uncorrected.aggregate.l2_cache,ecc.errors.uncorrected.aggregate.texture_memory,ecc.errors.uncorrected.aggregate.cbu,\ ecc.errors.uncorrected.aggregate.sram,ecc.errors.uncorrected.aggregate.total,retired_pages.single_bit_ecc.count,\ retired_pages.double_bit.count,retired_pages.pending,temperature.gpu,temperature.memory,power.management,power.draw,\ power.limit,enforced.power.limit,power.default_limit,power.min_limit,power.max_limit,clocks.current.graphics,clocks.current.sm,\ clocks.current.memory,clocks.current.video,clocks.applications.graphics,clocks.applications.memory,\ clocks.default_applications.graphics,clocks.default_applications.memory,clocks.max.graphics,clocks.max.sm,clocks.max.memory,\ mig.mode.current,mig.mode.pending" apis: - name: NvidiaGpu commands: - run: nvidia-smi --query-gpu=${var:metrics} --format=csv # update this if you have an alternate path output: csv rename_keys: " ": "" "\\[MiB\\]": ".MiB" "\\[%\\]": ".percent" "\\[W\\]": ".watts" "\\[MHz\\]": ".MHz" value_parser: "clocks|power|fan|memory|temp|util|ecc|stats|gom|mig|count|pcie": '\d*\.?\d+' '.': '\[N\/A\]|N\/A|Not Active|Disabled|Enabled|Default' ``` ## Confirm GPU metrics are being ingested The Flex configuration will be automatically detected and executed by the infrastructure agent, there's no need to restart the agent. You can confirm metrics are being ingested by running this NRQL query: ````sql SELECT * FROM NvidiaGpuSample ``` ```` ## Monitor your application You can use our pre-built dashboard template to monitor your GPU metrics. Follow these steps: 1. Go to **[one.newrelic.com](https://one.newrelic.com/)** and click on **Dashboards**. 2. Click on the **Import dashboard** tab. 3. Copy the file content (`.json`) from the [NVIDIA GPU dashboard](https://raw.githubusercontent.com/newrelic/nri-flex/master/examples/nvidia-smi-gpu-monitoring-dashboard.json). 4. Select the target account where the dashboard needs to be imported. 5. Click on **Import dashboard** to confirm the action. Your `NVIDIA GPU Monitoring` dashboard is considered a custom dashboard and can be found in the **Dashboards** UI. For docs on using and editing dashboards, see [our dashboard docs](https://docs.newrelic.com/docs/query-your-data/explore-query-data/dashboards/introduction-dashboards). Here is a NRQL query to view all the telemetry available: ```sql SELECT * FROM NvidiaGpuSample ``` ## What's next? [#next] You can adapt the Flex configuration to include or exclude information available from the NVIDIA SMI utility. To learn more about building NRQL queries and generating dashboards, check out these docs: - [Introduction to the query builder](https://docs.newrelic.com/docs/query-your-data/explore-query-data/query-builder/introduction-query-builder) to create basic and advanced queries. - [Introduction to dashboards](https://docs.newrelic.com/docs/query-your-data/explore-query-data/dashboards/introduction-dashboards) to customize your dashboard and carry out different actions. - [Manage your dashboard](https://docs.newrelic.com/docs/query-your-data/explore-query-data/dashboards/manage-your-dashboard) to adjust your dashboards display mode, or to add more content to your dashboard.