Manage your Auto-telemetry with Pixie data ingest

You have options for the type and the amount of data you ingest when you install Auto-telemetry with Pixie. During install, use Helm to reduce, restrict, or exclude data, by excluding specific namespaces or pods, by collecting data for only the nodes you want, or by redacting columns that contain sensitive information.

Exclude namespaces and pods

If you want to reduce the amount of Pixie data that New Relic ingests, you can exclude namespaces or pods by adding the following parameters to your Helm chart during installation. Note that the data still exists in Pixie:

• excludeNamespacesRegex - use to identify the namespaces that you want to exclude from sending observability data to New Relic. If empty, data for all namespaces is sent to New Relic. For example:

  bash

  카피

  $
  --set newrelic-pixie.excludeNamespacesRegex="examplenamespace-1|examplenamespace-2"

• excludePodsRegex - use to identify pods across all namespaces that you want to exclude from sending observability data to New Relic. If empty, data for all pods (except those in excluded namespaces) is sent to New Relic. For example:

  bash

  카피

  $
  --set newrelic-pixie.excludePodsRegex="examplepod-1|examplepod-2"

These two configuration options provide additional control over the Metric and Span data sent to New Relic from Pixie.

For example, if you wanted to configure the newrelic-pixie integration to exclude all namespaces except for px-sock-shop and kafka-demo, append the following config parameter to your Helm install or Helm upgrade command.

$
--set newrelic-pixie.excludeNamespacesRegex="default|kube-node-lease|kube-public|kube-system|newrelic|newrelic-custom-metrics|olm|px-operator"

Or, if you wanted to exclude pods running in a non-excluded namespace, you could add another configuration parameter to your Helm install or Helm upgrade. Instead of matching exact names, you can use a simple regex to match pod names related to load testing, just as an example.

$
--set newrelic-pixie.excludePodsRegex="load-test.*|loadgen.*"

If you're performing a brand new install, you'll need to append excludeNamespacesRegex and excludePodsRegex to the helm upgrade --install command provided by New Relic's guided install:

$
kubectl apply -f https://raw.githubusercontent.com/pixie-labs/pixie/main/k8s/operator/crd/base/px.dev_viziers.yaml && \
>
  kubectl apply -f https://raw.githubusercontent.com/pixie-labs/pixie/main/k8s/operator/helm/crds/olm_crd.yaml && \
>
  helm repo add newrelic https://helm-charts.newrelic.com && helm repo update && \
>
  kubectl create namespace newrelic ; helm upgrade --install newrelic-bundle newrelic/nri-bundle \
>
  --set global.licenseKey=NR_LICENSE_KEY \
>
  --set global.cluster=pixie-auto-telemetry \
>
  --namespace=newrelic \
>
  --set newrelic-infrastructure.privileged=true \
>
  --set ksm.enabled=true \
>
  --set prometheus.enabled=true \
>
  --set kubeEvents.enabled=true \
>
  --set logging.enabled=true \
>
  --set newrelic-pixie.enabled=true \
>
  --set newrelic-pixie.apiKey=PIXIE_API_KEY \
>
  --set pixie-chart.enabled=true \
>
  --set pixie-chart.deployKey=PIXIE_DEPLOY_KEY \
>
  --set pixie-chart.clusterName=pixie-auto-telemetry \
>
  --set newrelic-pixie.excludeNamespacesRegex="default|kube-node-lease|kube-public|kube-system|newrelic|newrelic-custom-metrics|olm|px-operator" \
>
  --set newrelic-pixie.excludePodsRegex="load-test.*|loadgen.*"

If you're just upgrading an existing install, then this is a much simpler approach:

$
helm upgrade newrelic-bundle newrelic/nri-bundle --reuse-values -n newrelic --set newrelic-pixie.excludeNamespacesRegex="default|kube-node-lease|kube-public|kube-system|newrelic|newrelic-custom-metrics|olm|px-operator"

Learn more about the available parameters for the newrelic-pixie Helm chart here.

Use Kubernetes features to collect selected data

When you deploy Auto-telemetry with Pixie, you're actually enabling the Auto-telemetry with Pixie Helm chart, as well as other New Relic components included in the New Relic infrastructure bundle. The Pixie Edge Module (PEM) pods are deployed to the cluster as a Kubernetes DaemonSet. This means that by default, a pod is scheduled to every cluster node and is responsible for collecting all of the observability metrics for that node.

In Kubernetes, you can assign pods to a specific subset of cluster nodes using nodeSelectors, taints/tolerations, and node affinity/anti-affinity. That way, you'll only collect metrics for the nodes you choose, instead of every node. This is helpful if you only want to deploy Auto-telemetry with Pixie to five of your ten cluster nodes, for example. Maybe the designated five nodes are responsible for high-priority workloads, or perhaps you're running both Linux and Windows nodes in your cluster and only want to deploy on the Linux nodes, since Windows nodes are not currently supported.

You can now assign pods to a subset of nodes by providing an additional option to the guided install command. This option passes an escaped JSON string to the Auto-telemetry with Pixie chart, which enables a nodeSelector that only schedules the PEM DaemonSet on nodes with the label pixie=allowed.

$
--set pixie-chart.patches.vizier-pem='\{\"spec\"\: \{\"template\"\: \{\"spec\"\: \{ \"nodeSelector\"\: \{\"pixie\"\: \"allowed\" \}\}\}\}\}'

If you're using a values file, common with Helm, then this is what that would look like in the nri-bundle values.yaml:

pixie-chart:
  enabled: true
  patches:
    vizier-pem: '{"spec": {"template": {"spec": { "nodeSelector": {"pixie": "allowed" }}}}}'

This approach gives you a multitude of configuration options; you just need to stick with the standard Kubernetes spec.

Redact columns that contain sensitive information

You can redact columns of potentially sensitive data, such as request and response bodies and headers. To do so, when you deploy Auto-telemetry with Pixie, use the following Helm command:

$
--set pixie-chart.dataAccess=Restricted

If you don't set dataAccess to Restricted, you'll continue to get full data access, which is the default state.

Currently, Full and Restricted are the only two options.

• Full: the default option. This is implied for all clusters that have already been deployed. In this method you can freely query all tables and columns and see all data.
• Restricted: When querying data, columns that may contain sensitive data, such as response/request bodies and request headers, are replaced with the REDACTED string. Other non-sensitive columns, such as error code, are still shown as normal. Note that restricted data access mode is not smart. It doesn't detect whether the row actually contains sensitive data. Instead, it identifies a type of content as potentially sensitive and hides it.