--- title: On-host integrations: Standard configuration format source: https://docs.newrelic.com/docs/infrastructure/host-integrations/infrastructure-integrations-sdk/specifications/host-integrations-standard-configuration-format --- In December 2019, [Infrastructure agent version 1.8.0](https://docs.newrelic.com/docs/release-notes/infrastructure-release-notes/infrastructure-agent-release-notes) began supporting this new configuration format that makes use of a single configuration file (instead of two separate files), and provides other improvements. This document will explain how this newer format works. The older [legacy configuration format](https://docs.newrelic.com/docs/create-integrations/infrastructure-integrations-sdk/specifications/host-integrations-legacy-configuration-format) is also supported by current infrastructure agents. For an introduction to configuration, see [Config overview](https://docs.newrelic.com/docs/integrations/integrations-sdk/file-specifications/config-file-overview). ## Configuration structure [#configuration-basics] An on-host integration's configuration YAML must have an `integrations` top-level section containing a YAML array, where each entry represents an integration and its configuration. For each integration entry, only the `name` property is mandatory. The other properties are optional. Here's an example configuration featuring two integrations: our built-in [Docker integration](https://github.com/newrelic/nri-docker), which requires no configuration, and our [MySQL integration](https://docs.newrelic.com/docs/integrations/host-integrations/host-integrations-list/mysql-monitoring-integration): ```yaml integrations: # New Relic integration that does not require any configuration - name: nri-docker # New Relic integration that gets its configuration from the environment - name: nri-mysql env: PORT: 3306 USERNAME: newrelic PASSWORD: 123456789 # to hide this field, read the secrets management documentation # Any free-form integration executed via a user-provided command - name: my-own-integration exec: python /opt/integrations/my-script.py --host=127.0.0.1 ``` You can have as many configuration YAML files as you want and can group your integration instances. > #### 💡 TIP > > We recommend [linting](http://www.yamllint.com/) the YAML configuration files before using them to avoid formatting issues. Each configuration YAML file can also contain [`discovery`](https://docs.newrelic.com/docs/integrations/host-integrations/installation/container-auto-discovery) and [`variables`](https://docs.newrelic.com/docs/integrations/host-integrations/installation/secrets-management) top-level sections. > #### ⚠️ IMPORTANT > > This configuration format does not require an agent restart. When saved, changes are detected and implemented immediately. This means that saving intermediate configuration changes may cause the integration to stop working. ## List of configuration properties [#options-list] This is a list of the general properties used to configure an integration. For more details about using these properties, including example values, see the documentation following the table. | Config | Description | | ----------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | [`name`](#name) | Name of the integration. This is the only mandatory configuration property across all on-host integrations. If the `exec` field is not set it will also be the name of the integration executable. | | `cli_args` | Optional list of command line arguments when `name` is used to provide the integration executable. _Available since agent version **1.13.0**._ | | [`exec`](#exec) | Full path to the integration executable, plus arguments. It may be a single-line string or a string array. If left unspecified, the `exec` field defaults to the `name` field. | | [`env`](#env) | YAML map containing the environment variables to be passed to the integration, where `key` is the environment variable name, and `value` is the variable value. | | [`config`](#config) | Configuration that is written as an external file and the path that is passed to the integration with the `CONFIG_PATH` environment variable or the `${config.path}` variable placeholder. | | [`config_template_path`](#config_template_path) | Any external file whose path is passed to the integration with the `CONFIG_PATH` environment variable or the `${config.path}` variable placeholder. Its usage allows applying discovery and secrets binding to any external configuration. | | [`integration_user`](#integration_user) | Name of the user who runs the integration. | | [`interval`](#interval) | Time between consecutive executions of the integration. It must be a number followed by a time unit (`s`, `m` or `h`), without spaces. | | [`inventory_source`](#inventory_source) | Allows overriding the category and term of the inventory source. | | [`labels`](#labels) | Map with labels that decorate the data (metrics, events, inventory) reported by the integration. | | [`timeout`](#timeout) | A number followed by a time unit (`ms`, `s`, `m` or `h`). An integration that hasn't responded in this time period is killed and restarted. | | [`working_dir`](#working_dir) | Working directory for the integration binary. | | [`when`](#when) | Integration is only executed if the clause evaluates to true. Conditions are defined [below](#when). | The remainder of this document describes config properties grouped by their functionality: - [Select an integration to run](#select-integration) - [Pass configuration to the integration command](#pass-configuration) - [Configure how the agent executes integrations](#configure-agent-execution) ## Select an integration to run There are two properties to select which integration will run: `name` and `exec`. The only mandatory property across all on-host integrations is `name`. The remaining properties specified in this document are optional. Example: ```yaml integrations: - name: nri-docker - name: my-integration exec: /usr/local/bin/my-integration --metrics --inventory ``` **`name`** The mandatory `name` property can work in two ways: - **If the `exec` property is set**: The `name` property only provides an identifier for the integration instance. This identifier is used in log messages and to provide a default [inventory category/source](https://docs.newrelic.com/docs/infrastructure/infrastructure-ui-pages/infra-ui-pages/infrastructure-inventory-page-search-your-entire-infrastructure) in the form `integration/` (for example, `integration/nri-redis`). This inventory path can be overridden with the `inventory_source` configuration option. - **If the `exec` property is not set**: The agent looks for (and executes) an executable with the `name` value in any of the following folders: - Linux: - `/var/db/newrelic-infra/newrelic-integrations/bin` - `/var/db/newrelic-infra/newrelic-integrations` - `/var/db/newrelic-infra/custom-integrations/bin` - `/var/db/newrelic-infra/custom-integrations` - Windows - `C:\Program Files\New Relic\newrelic-infra\newrelic-integrations\bin` - `C:\Program Files\New Relic\newrelic-infra\newrelic-integrations` If there is no executable with this name in the above folders the agent logs an error and the integration is not executed. > #### ⚠️ IMPORTANT > > In Windows, do not append the `.exe` extension to the name. The agent does this for you (for example, `name: nri-mysql` would look for `nri-mysql.exe` in the above folders). **`exec`** The `exec` optional property specifies the path, command, and command-line arguments of the integration to execute. When none of the path folders or arguments have spaces, it can be written in a single-line string: ````yaml - name: my-integration exec: /usr/bin/python /opt/integrations/my-script.py --host=127.0.0.1 ``` If any of the paths or arguments have spaces that are part of a single element, you can use a YAML array notation. For Windows, be sure to escape backslashes with quotation marks like this: ```yaml - name: my-integration exec: - '"C:\Program Files\My Integration\integration.exe"' - --host - 127.0.0.1 - --port - 8080 ``` The default working directory is the root directory of the agent configuration. It can be overridden with the `working_dir` property. ```` **`cli_args`** The `cli_args` optional property specifies command line arguments that should be passed to the integration. It is useful when using `name` as it only provides the integration name identifier (not compatible with `exec`). ````yaml - name: my-integration cli_args: [ -interval 10s ] ``` Usual YAML multi-line list format can be used as well: ```yaml - name: my-integration cli_args: - -interval - 10s ``` ```` **`when`** The `when` property allows to execute the integration only when all the evaluated conditions are successful. Available conditions are: - `env_exists`: Environment variables exist and match value. - `file_exists`: Given file path exists. - `feature`: Provided feature-flag is enabled. Example: ```yaml integrations: - name: ssh-integration when: file_exists: /var/run/sshd.pid ``` ## Pass configuration to the integration executable Often integration executables need to receive a configuration to work properly (for example, hostname and port of the monitored system, user credentials, etc.). The infrastructure agent allows you to configure the integration commands in three ways (which you can combine): - Environment variables, using the [`env` property](#env). (recommended) - Command-line arguments, passed in the [`exec` property](#exec). - Configuration files, whose path needs to be passed through environment variables or command-line arguments (see the [config](#config)) property. Example: ```yaml integrations: - name: my-integration exec: /opt/path/bin/script --host 127.0.0.1 --port 8081 - name: nri-mysql env: STATUS_URL: http://10.0.0.3/server-status?auto REMOTE_MONITORING: true ``` **`env`** The `env` property allows you to set environment variables that are passed to the executable. It is a key-value map with the required variables. > #### ⚠️ IMPORTANT > > New Relic recommends passing `env` keys in capital letters, as per the example below, for compatibility with all infrastructure agent versions since 1.8.0. > If you are using agent version 1.20.0 or above you can use lower case letters as the agent will automatically [uppercase them](https://docs.newrelic.com/docs/release-notes/infrastructure-release-notes/infrastructure-agent-release-notes/new-relic-infrastructure-agent-1200#changed). Example: ````yaml integrations: - name: nri-postgresql env: DATABASE: postgres PORT: 6432 COLLECTION_LIST: '["postgres"]' COLLECT_DB_LOCK_METRICS: false VERBOSE: 1 ``` If you expect your integration to receive the configuration from the host's environment rather than specifying it explicitly in the configuration file, you need to set the required variables in the infrastructure agent [`passthrough_environment`](/docs/integrations/host-integrations/troubleshooting/pass-infrastructure-agent-parameters-host-integration) global configuration property ```` **`config`** This section describes various ways to pass configuration information to an integration. **Pass configuration file directly** Some integration commands may get their configuration from an external file. If your integration requires a configuration file, nothing prevents you from directly passing its path directly as a command-line argument or an environment variable. Here's an example using configuration of our [Flex integration](https://docs.newrelic.com/docs/integrations/host-integrations/host-integrations-list/flex-integration-tool-build-your-own-integration): ````yaml integrations: - name: nri-flex env: CONFIG_FILE: /etc/nri-flex/configs/http-service.yaml - name: other-integration exec: /opt/integration/integration -f /opt/integration/config.properties ``` The above example assumes that the `http-service.yaml` and `config.properties` files exist. We can see that the `nri-flex` integration is expecting the `http-service.yaml` complete path via the `CONFIG_FILE` environment variable and the `other-integration` expects the complete `config.properties` path after the `-f` command-line flag. In the above example, it's necessary for the integration installer/configurator that the configuration files exist in the provided path and that the agent and integrations have read permissions on them. **Pass configuration through `config` section** If you prefer to keep your configuration file with the rest of the integration configuration, you can use the `config` section in the integration entry, which can contain a valid YAML object or just a multi-line string: ```yaml integrations: - name: nri-flex env: CONFIG_FILE: ${config.path} config: name: csvFileExample apis: - name: csvFile file: /Users/hello/test.csv - name: other-integration exec: /opt/integration/integration -f ${config.path} config: | example.cfg.verbose=true example.cfg.logger=/var/logs/integration.log example.cfg.hostname=localhost example.cfg.port=9025 ``` In the above examples, every time the `nri-flex` integration is executed, the agent creates a temporary file with the following contents: ```yaml name: csvFileExample apis: - name: csvFile file: /Users/hello/test.csv ``` The above YAML is only a configuration example for the `nri-flex` integration. The agent ignores its contents; instead, it creates a temporary file and replaces the `${config.path}` variable placeholder with its path. When the integration completes execution the temporary file is removed. Also, the agent creates another temporary file before executing the `other-integration` integration: ``` example.cfg.verbose=true example.cfg.logger=/var/logs/integration.log example.cfg.hostid=localhost example.cfg.port=9025 ``` It replaces the `-f ${config.path}` command-line placeholder with the temporary path of the written file. By convention, if you do not place the `${config.path}` variable in any command-line argument or environment variable value, the agent passes the path of the configuration file via the `CONFIG_PATH` environment variable: ```yaml # assuming that nri-example command is prepared to receive the configuration # file via the CONFIG_PATH environment variable integrations: - name: nri-example config: name: csvFileExample apis: - name: csvFile file: /Users/hello/test.csv ``` **Pass secrets and discovery through `config` section** The main benefit for using a `config` section instead of hardcoding the full path of an external file is that you can insert `${variable}` placeholders to apply our [auto-discovery feature](/docs/integrations/host-integrations/installation/container-auto-discovery) and [secrets management](/docs/integrations/host-integrations/installation/secrets-management). Here's an example followed by some explanations: ```yaml variables: my_credentials: vault: http: url: http://my.vault.host/v1/newengine/data/secret headers: X-Vault-Token: my-vault-token discovery: docker: match: label.service: foo integrations: - name: foo-monitor exec: /opt/foo/bin/monitor --config=${config.path} config: | foo.host=${discovery.ip} foo.port=${discovery.port} foo.user=${my_credentials.user} foo.password=${my_credentials.password} ``` (For more details about the `variables` and `discovery` sections, please visit the [discovery](/docs/integrations/host-integrations/installation/container-auto-discovery) and [secrets management](/docs/integrations/host-integrations/installation/secrets-management) documentation). The above example relies on the following premises: * There is a [Vault](https://www.vaultproject.io/) service that allows retrieving a JSON object formed by the `user` and `password` fields. * There may be a variable number of Docker containers labeled with `service=foo`, which are accessible from the agent host via a discoverable public IP and Port. * The user has configured the `foo-monitor` integration to monitor all the `service=foo` labeled containers, which share a common user and password. Each instance of the `foo-monitor` integration requires executing the `/opt/foo/bin/monitor` executable, passing the text configuration inside the `config` section via the `--config=` command-line argument. As example of workflow, imagine that the Vault invocation returns the following JSON: ```json {"user":"monitorer","password":"5up3r53cr3t!"} ``` At the moment of executing the `foo-monitor` integration, there are three running containers labeled with `service=foo`: 1. ip: `10.0.0.3`, port: `8080` 2. ip: `10.0.0.3`, port: `8081` 3. ip: `10.0.0.3`, port: `8082` The agent then creates the following three temporary files, using the contents of the `config` property as a template, but replacing the `${placeholders}` by the acquired variables and discovered items (files' path is invented for the sake of simplicity): * First match (`/tmp/123_discovered`): ``` foo.host=10.0.0.3 foo.port=8080 foo.user=monitorer foo.password=5up3r53cr3t! ``` * Second match (`/tmp/456_discovered`): ``` foo.host=10.0.0.3 foo.port=8081 foo.user=monitorer foo.password=5up3r53cr3t! ``` * Third match (`/tmp/789_discovered`) ``` foo.host=10.0.0.3 foo.port=8082 foo.user=monitorer foo.password=5up3r53cr3t! ``` After the `config` variable placeholders have been replaced and the temporary files have been created, the `/opt/foo/bin/monitor` executable is executed three times (one per matched container), replacing the `${config.path}` command-line placeholder with the temporary file corresponding to each discovered configuration: * First match: `/opt/foo/bin/monitor --config=/tmp/123_discovered` * Second match: `/opt/foo/bin/monitor --config=/tmp/456_discovered` * Third match: `/opt/foo/bin/monitor --config=/tmp/789_discovered` To ensure security and to minimize the chance of leaking secrets to disk, the agent: * Creates the files owned by the agent user, for example, `root` or `nri-agent`, depending on the user you have configured to run the agent. * Sets read permissions only for the owner. * Removes the created files when the integration instance finishes its execution. ```` **`config_template_path`** If you want to use the secrets management and discovery in the configuration files that you're passing to the integration executable, but you prefer to keep them as an individual file, you can use the `config_template_path: ` option. It works exactly as in the `config` section: 1. The agent applies [secrets management](https://docs.newrelic.com/docs/integrations/host-integrations/installation/secrets-management) and [discovery](https://docs.newrelic.com/docs/integrations/host-integrations/installation/container-auto-discovery) to the file contents. 2. The agent creates different temporary files that are passed to the integration via the `${config.path}` placeholder (or the `CONFIG_PATH` environment variable). Example: ```yaml discovery: docker: match: name: /^redis/ integrations: - name: nri-flex env: CONFIG_FILE: ${config.path} config_template_path: /etc/flex-configs/redis.yml ``` In the above example, the `redis.yml` external file can contain container discovery variable placeholders, like `${discovery.ip}` or `${discovery.port}`. ## Configure how the agent executes your integrations The properties of this section modify the way the infrastructure agent executes and interacts with the integrations, or the way the agent decorates the integrations' data. **`integration_user`** The integration commands run as the same user as the agent (usually `root` or `nri-agent`). If due to permission restrictions an integration needs to run as another user, its name must be specified in the `integration_user` property. Example: ````yaml integrations: - name: dbus-inventory exec: python my-dbus-script.py integration_user: dbus ``` ```` **`interval`** The `interval` option sets the time between consecutive executions of an integration. The accepted format is an integer immediately followed by a time unit (`s` for seconds, `m` for minutes, `h` for hours). The default is `30s`, and the minimum accepted value is `15s`. Any value lower than `15s` is automatically set to `15s`. > #### ⚠️ IMPORTANT > > To ensure accurate reporting of metrics, especially rate metrics, set the interval to 5 minutes or less. Longer intervals may lead to inaccuracies or missing data. Example: ````yaml integrations: - name: nri-nginx env: STATUS_URL: http://127.0.0.1/status STATUS_MODULE: discover interval: 20s ``` ```` **`inventory_source`** Any inventory item must be catalogued under a `category/source` taxonomy. By default, each integration inventory is stored as `integration/` + `name` value (for example, `integration/nri-apache`, `integration/nri-mysql`). The `inventory_source` property allows you to override the default taxonomy of inventory data. Example: ````yaml integrations: - name: nri-nginx - name: nri-apache exec: - /var/db/newrelic-infra/newrelic-integrations/bin/nri-apache - --inventory inventory_source: config/apache ``` In the above example, the `nri-nginx` inventory, if any, would be visible in the New Relic UI under the `integration/nri-nginx` source. The `nri-apache` inventory would be visible under `config/apache`. ```` **`labels`** `labels` is a key-value map that allows extra metadata to be provided for the integration. The agent uses those labels to decorate the metrics, events, and inventory that it receives from a given integration instance. Example: ````yaml integrations: - name: nri-apache inventory_source: config/apache labels: env: production role: load_balancer ``` In the above example, the agent decorates all the metrics and events from the `nri-apache` instance with the following fields: * `label.env`: `production` * `label.role`: `load_balancer` Also, the following entries are added to the integration inventory: * `config/apache/labels/env`: `production` * `config/apache/labels/role`: `load_balancer` ```` **`timeout`** If an integration has not returned any metric (or a heartbeat message as described below) before the time specified in the `timeout` value, the agent kills the integration process and restarts it after the corresponding `interval`. The accepted format is an integer number immediately followed (without spaces) by a time unit (`ms` for milliseconds, `s` for seconds, `m` for minutes, `h` for hours). If a zero (or negative) `timeout` value is provided, the integration can run forever without being killed by a timeout expiration. For long-running integrations (integrations that keep running, periodically returning metrics/events/inventory), each time the integration submits a metrics/events/inventory payload, the timeout deadline is restarted. That means that long-running integrations must return a valid JSON payload in an interval that is lower than `timeout`. Returning an empty JSON (`{}`) is interpreted as a _heart-beat_ message that restarts the timeout, preventing the long-running integration from being killed, even if they don't have information to report. The default is `120s`, and the minimum accepted value is `100ms`. Any value lower than `100ms` is automatically set to `100ms`. Example: ````yaml integrations: - name: nri-jmx cli_args: JMX_HOST: jmx-host.localnet JMX_PORT: 7096 COLLECTION_FILES: "/etc/newrelic-infra/integrations.d/jvm-metrics.yml" timeout: 30s ``` ```` **`working_dir`** `working_dir` sets the working directory of the command. If empty or unspecified, the agent runs the command in the infrastructure agent's current directory. The default is the infrastructure agent root directory. Example: ````yaml integrations: - name: my-integration exec: /opt/integration/bin/integration working_dir: /opt/integration/scratch-zone ``` ```` ## Update older integration configuration [#update] In December 2019, the [Infrastructure agent version 1.8.0](https://docs.newrelic.com/docs/release-notes/infrastructure-release-notes/infrastructure-agent-release-notes) began using a different configuration format. The main difference between these formats is that the older configuration format uses two separate configuration files (a `INTEGRATION_NAME-definition.yml` file and a `INTEGRATION_NAME-config.yml` file) and the newer version uses a single configuration file. Here are some of the features added by the newer configuration functionality: - Flexible configuration via command-line arguments, environment variables, or external files. - Ability to group different integrations in the same file. - Hot reload: adding a new integration or changing its configuration does not require restarting the agent. - Timeouts: if an integration doesn't respond before a user-specified time, the integration process is killed and restarted. Not all on-host integrations come with the newer configuration format, but you can update the configuration to the new format for all on-host integrations to take advantage of the new features. The following YAML shows an example [Apache integration](https://docs.newrelic.com/docs/integrations/host-integrations/host-integrations-list/apache-monitoring-integration) configuration using the older configuration format. Note that this configuration will still work with newer agents, but we recommend updating your integrations to take full advantage of features. ```yaml integration_name: com.newrelic.apache instances: - name: apache-server-metrics command: metrics arguments: status_url: http://127.0.0.1/server-status?auto remote_monitoring: true labels: env: production role: load_balancer - name: apache-server-inventory command: inventory arguments: remote_monitoring: true labels: env: production role: load_balancer ``` To update an older integration configuration to the new format, use one of the following methods: ### Assisted method [#assisted-method] Using the [New Relic CLI](https://developer.newrelic.com/automate-workflows/get-started-new-relic-cli/), run the following command to automatically convert your old definition/configuration files to the new configuration format: ```shell newrelic agent config migrateV3toV4 -d /path/definitionFile -c /path/configFile -o /path/outputFile ``` Examples: **Redis** The path used below is the default location for Linux based integrations. You may need to adjust the path if you are using a custom location: ````shell newrelic agent config migrateV3toV4 \ -d /var/db/newrelic-infra/newrelic-integrations/redis-definition.yml \ -c /etc/newrelic-infra/integrations.d/redis-config.yml \ -o /etc/newrelic-infra/integrations.d/redis.yml ``` ```` **Microsoft SQL** The path used below is the default location for Windows based integrations. You may need to adjust the path if you are using a custom location: ````shell newrelic agent config migrateV3toV4 ^ -d 'C:\Program Files\New Relic\newrelic-infra\newrelic-integrations\mssql-definition.yml' ^ -c 'C:\Program Files\New Relic\newrelic-infra\integrations.d\mssql-config.yml' ^ -o 'C:\Program Files\New Relic\newrelic-infra\integrations.d\mssql.yml' ``` ```` ### Manual method [#manual-method] To convert the integration file manually: 1. Rename the `instances` top-level section to `integrations`. 2. Remove the `integration_name` top-level section, and add it to each integration entry. You are no longer required to keep a separate file for each integration type, and you can group your legacy integration entries in the same file as other integrations. Here's an example of the new version of the Apache integration configuration: ```yml integrations: - name: nri-apache env: METRICS: "true" STATUS_URL: http://127.0.0.1/server-status?auto REMOTE_MONITORING: true interval: 15s labels: env: production role: load_balancer - name: nri-apache env: INVENTORY: "true" STATUS_URL: http://127.0.0.1/server-status?auto REMOTE_MONITORING: true interval: 60s labels: env: production role: load_balancer inventory_source: config/apache ``` > #### ⚠️ IMPORTANT > > Please note that the older configuration format doesn't support hot reloading. Therefore, you need to restart the infrastructure agent to remove the old integrations configuration. Otherwise, the old instances will coexist with the new ones.