Monitorear Elasticsearch en Kubernetes con OpenTelemetry

Monitoree sus clústeres de Elasticsearch en Kubernetes desplegando el OpenTelemetry Collector con descubrimiento automático de pods. Esta integración utiliza el elasticsearchreceiver y el receivercreator para descubrir y monitorear automáticamente los pods de Elasticsearch sin configuración manual.

Para empezar, seleccione la distribución del recopilador que mejor se adapte a su entorno de Kubernetes:

You can choose between three collector options:

NRDOT: Distribución de New Relic de OpenTelemetry
OTel Collector Contrib: OpenTelemetry Collector estándar con componentes aportados por la comunidad
Prometheus Receiver: For environments already running a Prometheus Elasticsearch exporter

Opciones de instalación

Seleccione la distribución del recopilador que se ajuste a sus necesidades:

Antes de que empieces

Before deploying the NRDOT collector on Kubernetes, ensure you have:

Privilegios de acceso requeridos:

Tu New Relic
Acceso a su clúster de Kubernetes con kubectl
Privilegios de administrador de clúster de Elasticsearch con privilegio de clúster monitor o manage (consulte la documentación de privilegios de seguridad de Elasticsearch para obtener detalles)
Requisitos del sistema:
Versión 7.16 o superior de Elasticsearch - Esta integración requiere un clúster moderno de Elasticsearch
Clúster de Kubernetes - Un clúster de Kubernetes en ejecución donde Elasticsearch está desplegado
Conectividad de red - HTTPS saliente (puerto 443) al endpoint de ingesta OTLPde New Relic
Requisitos del pod de Elasticsearch:
Etiquetas de pod (Obligatorio) - Cada pod de Elasticsearch debe tener la etiqueta app: elasticsearch para que funcione el descubrimiento automático. Sin esta etiqueta, el colector no detectará ni monitoreará sus pods.
Importante
Cómo agregar etiquetas a los pods de Elasticsearch:
Si utiliza un StatefulSet o un Deployment para Elasticsearch, agregue la etiqueta en la plantilla del pod:
apiVersion: apps/v1 kind: StatefulSet metadata: name: elasticsearch spec: template: metadata: labels: app: elasticsearch # Required for auto-discovery spec: containers: - name: elasticsearch image: docker.elastic.co/elasticsearch/elasticsearch:8.x.x
Para los pods existentes sin etiquetas, actualice su StatefulSet/Deployment y reinicie los pods:
bash
```
$kubectl label pods -l <your-existing-selector> app=elasticsearch -n <namespace>
```
Puede verificar que las etiquetas estén configuradas correctamente:
bash
```
$kubectl get pods -n <namespace> --show-labels
```

Elige tu método de instalación

You can install the NRDOT Collector using Kubernetes manifests or Helm charts. Choose the method that best fits your workflow:

Manifest install:

More control over individual Kubernetes resources
Requires completing the base Kubernetes OpenTelemetry manifest installation first
Best for customized deployments
Helm install:
Simpler deployment with single command
Easier to manage and upgrade
Best for standard deployments
Proceed to the next step for detailed instructions for your chosen method.

Install and configure NRDOT Collector

Complete base Kubernetes OpenTelemetry manifest installation

Before configuring Elasticsearch monitoring, complete the base Kubernetes OpenTelemetry manifest installation.

Update collector image to NRDOT

En los archivos deployment.yaml y daemonset.yaml de su directorio local rendered, actualice la imagen a:

image: newrelic/nrdot-collector:latest

Configurar el monitoreo de Elasticsearch [#nrdot-configure-es]

Update the deployment-configmap.yaml for Elasticsearch monitoring.

You have two options:

Option 1: Replace all existing OpenTelemetry components (recommended for Elasticsearch-only monitoring)

Replace the entire deployment-configmap.yaml content with the Elasticsearch-specific configuration:

apiVersion: v1
kind: ConfigMap
metadata:
  name: nr-k8s-otel-collector-deployment-config
data:
  config.yaml: |
    extensions:
      health_check:
        endpoint: 0.0.0.0:13133
      k8s_observer:
        auth_type: serviceAccount
        observe_pods: true
        observe_nodes: true

    receivers:
      jaeger: null
      zipkin: null
      prometheus: null
      otlp: null
      receiver_creator/elasticsearch:
        watch_observers: [k8s_observer]
        receivers:
          elasticsearch:
            rule: type == "pod" && labels["app"] == "elasticsearch"
            config:
              endpoint: 'http://`endpoint`:9200'
              collection_interval: 15s
              metrics:
                elasticsearch.os.cpu.usage:
                  enabled: true
                elasticsearch.cluster.data_nodes:
                  enabled: true
                elasticsearch.cluster.health:
                  enabled: true
                elasticsearch.cluster.in_flight_fetch:
                  enabled: true
                elasticsearch.cluster.nodes:
                  enabled: true
                elasticsearch.cluster.pending_tasks:
                  enabled: true
                elasticsearch.cluster.shards:
                  enabled: true
                elasticsearch.cluster.state_update.time:
                  enabled: true
                elasticsearch.index.documents:
                  enabled: true
                elasticsearch.index.operations.merge.current:
                  enabled: true
                elasticsearch.index.operations.time:
                  enabled: true
                elasticsearch.node.cache.count:
                  enabled: true
                elasticsearch.node.cache.evictions:
                  enabled: true
                elasticsearch.node.cache.memory.usage:
                  enabled: true
                elasticsearch.node.shards.size:
                  enabled: true
                elasticsearch.node.cluster.io:
                  enabled: true
                elasticsearch.node.documents:
                  enabled: true
                elasticsearch.node.disk.io.read:
                  enabled: true
                elasticsearch.node.disk.io.write:
                  enabled: true
                elasticsearch.node.fs.disk.available:
                  enabled: true
                elasticsearch.node.fs.disk.total:
                  enabled: true
                elasticsearch.node.http.connections:
                  enabled: true
                elasticsearch.node.ingest.documents.current:
                  enabled: true
                elasticsearch.node.ingest.operations.failed:
                  enabled: true
                elasticsearch.node.open_files:
                  enabled: true
                elasticsearch.node.operations.completed:
                  enabled: true
                elasticsearch.node.operations.current:
                  enabled: true
                elasticsearch.node.operations.get.completed:
                  enabled: true
                elasticsearch.node.operations.time:
                  enabled: true
                elasticsearch.node.shards.reserved.size:
                  enabled: true
                elasticsearch.index.shards.size:
                  enabled: true
                elasticsearch.os.cpu.load_avg.1m:
                  enabled: true
                elasticsearch.os.cpu.load_avg.5m:
                  enabled: true
                elasticsearch.os.cpu.load_avg.15m:
                  enabled: true
                elasticsearch.os.memory:
                  enabled: true
                jvm.gc.collections.count:
                  enabled: true
                jvm.gc.collections.elapsed:
                  enabled: true
                jvm.memory.heap.max:
                  enabled: true
                jvm.memory.heap.used:
                  enabled: true
                jvm.memory.heap.utilization:
                  enabled: true
                jvm.threads.count:
                  enabled: true
                elasticsearch.index.segments.count:
                  enabled: true
                elasticsearch.index.operations.completed:
                  enabled: true
                elasticsearch.node.script.cache_evictions:
                  enabled: false
                elasticsearch.node.cluster.connections:
                  enabled: false
                elasticsearch.node.pipeline.ingest.documents.preprocessed:
                  enabled: false
                elasticsearch.node.thread_pool.tasks.queued:
                  enabled: false
                elasticsearch.cluster.published_states.full:
                  enabled: false
                jvm.memory.pool.max:
                  enabled: false
                elasticsearch.node.script.compilation_limit_triggered:
                  enabled: false
                elasticsearch.node.shards.data_set.size:
                  enabled: false
                elasticsearch.node.pipeline.ingest.documents.current:
                  enabled: false
                elasticsearch.cluster.state_update.count:
                  enabled: false
                elasticsearch.node.fs.disk.free:
                  enabled: false
                jvm.memory.nonheap.used:
                  enabled: false
                jvm.memory.pool.used:
                  enabled: false
                elasticsearch.node.translog.size:
                  enabled: false
                elasticsearch.node.thread_pool.threads:
                  enabled: false
                elasticsearch.cluster.state_queue:
                  enabled: false
                elasticsearch.node.translog.operations:
                  enabled: false
                elasticsearch.memory.indexing_pressure:
                  enabled: false
                elasticsearch.node.ingest.documents:
                  enabled: false
                jvm.classes.loaded:
                  enabled: false
                jvm.memory.heap.committed:
                  enabled: false
                elasticsearch.breaker.memory.limit:
                  enabled: false
                elasticsearch.indexing_pressure.memory.total.replica_rejections:
                  enabled: false
                elasticsearch.breaker.memory.estimated:
                  enabled: false
                elasticsearch.cluster.published_states.differences:
                  enabled: false
                jvm.memory.nonheap.committed:
                  enabled: false
                elasticsearch.node.translog.uncommitted.size:
                  enabled: false
                elasticsearch.node.script.compilations:
                  enabled: false
                elasticsearch.node.pipeline.ingest.operations.failed:
                  enabled: false
                elasticsearch.indexing_pressure.memory.limit:
                  enabled: false
                elasticsearch.breaker.tripped:
                  enabled: false
                elasticsearch.indexing_pressure.memory.total.primary_rejections:
                  enabled: false
                elasticsearch.node.thread_pool.tasks.finished:
                  enabled: false

    processors:
      memory_limiter:
        check_interval: 60s
        limit_mib: ${env:NEW_RELIC_MEMORY_LIMIT_MIB}
      cumulativetodelta: {}
      attributes/cluster_state_aggregate:
        include:
          match_type: strict
          metric_names:
            - elasticsearch.cluster.state_update.time
        actions:
          - key: type
            action: delete
          - key: state
            action: delete
      filter/critical_operations:
        metrics:
          datapoint:
            - 'attributes["operation"] == "query" or attributes["operation"] == "index" or attributes["operation"] == "get" or attributes["operation"] == "merge" or attributes["operation"] == nil'
      resource/cluster:
        attributes:
          - key: k8s.cluster.name
            value: "elasticsearch-cluster"
            action: insert
      resource/cluster_name_override:
        attributes:
          - key: elasticsearch.cluster.name
            value: "elasticsearch-cluster"
            action: upsert
      resourcedetection:
        detectors: [env, system]
        system:
          resource_attributes:
            host.name:
              enabled: true
            host.id:
              enabled: false
            os.type:
              enabled: true
      batch:
        timeout: 30s
        send_batch_size: 2048
        send_batch_max_size: 4096
      attributes/cardinality_reduction:
        actions:
          - key: process.pid
            action: delete
          - key: process.parent_pid
            action: delete
          - key: k8s.pod.uid
            action: delete
      transform/metadata_nullify:
        metric_statements:
          - context: metric
            statements:
              - set(description, "")
              - set(unit, "")

    exporters:
      otlp_http:
        endpoint: "${env:NEWRELIC_OTLP_ENDPOINT}"
        headers:
          api-key: "${env:NEWRELIC_LICENSE_KEY}"
        compression: gzip
        timeout: 30s
        retry_on_failure:
          enabled: true
          initial_interval: 5s
          max_interval: 30s
          max_elapsed_time: 300s

    service:
      extensions: [health_check, k8s_observer]
      pipelines:
        traces: null
        logs: null
        metrics: null
        metrics/elasticsearch:
          receivers: [receiver_creator/elasticsearch]
          processors: [memory_limiter, resourcedetection, resource/cluster, resource/cluster_name_override, attributes/cardinality_reduction, filter/critical_operations, attributes/cluster_state_aggregate, cumulativetodelta, transform/metadata_nullify, batch]
          exporters: [otlp_http]

Option 2: Merge with existing configuration (if monitoring multiple services)

Add the Elasticsearch receiver and processors to your existing configuration. Refer to the sections above and merge them with your current deployment-configmap.yaml.

Sugerencia

Customize for your environment:

Cambios requeridos:

Pod label rule - The rule labels["app"] == "elasticsearch" must match your pod labels
Cluster name - Replace elasticsearch-cluster in both resource/cluster and resource/cluster_name_override with your unique cluster name
Cambios opcionales:
Puerto - Actualice 9200 si Elasticsearch se ejecuta en un puerto diferente
Authentication - Add credentials if your Elasticsearch cluster is secured (see example below)

For secured Elasticsearch clusters:

receivers:
  receiver_creator/elasticsearch:
    watch_observers: [k8s_observer]
    receivers:
      elasticsearch:
        rule: type == "pod" && labels["app"] == "elasticsearch"
        config:
          endpoint: 'https://`endpoint`:9200'
          username: "your_elasticsearch_username"
          password: "your_elasticsearch_password"
          tls:
            insecure_skip_verify: false

Apply the updated manifests [#nrdot-apply-manifests]

Apply the updated manifests and restart the deployment:

bash

$kubectl apply -n newrelic -R -f rendered
$kubectl rollout restart deployment nr-k8s-otel-collector-deployment -n newrelic

Crear secreto de Kubernetes para credenciales

Crea un secreto de Kubernetes para almacenar tus credenciales de New Relic de forma segura:

Cree el espacio de nombres:
bash
```
$kubectl create namespace newrelic
```

Cree el secreto:

bash

$kubectl create secret generic newrelic-licenses \
>     --from-literal=NEWRELIC_LICENSE_KEY=YOUR_LICENSE_KEY_HERE \
>     --from-literal=NEWRELIC_OTLP_ENDPOINT=https://otlp.nr-data.net:4318 \
>     --from-literal=NEW_RELIC_MEMORY_LIMIT_MIB=100 \
>     -n newrelic

Actualice los valores:

Reemplace YOUR_LICENSE_KEY_HERE con su clave de licencia de New Relic real
Reemplace https://otlp.nr-data.net:4318 con el endpoint de su región (consulte la documentación del endpoint OTLP)

Replace 100 with your desired memory limit in MiB for the collector

Configurar el monitoreo de Elasticsearch

Create a values.yaml file for NRDOT Collector Elasticsearch monitoring:

mode: deployment

image:
  repository: newrelic/nrdot-collector
  tag: latest
  pullPolicy: IfNotPresent

resources:
  limits:
    cpu: 500m
    memory: 512Mi
  requests:
    cpu: 200m
    memory: 256Mi

ports: {}

extraEnvs:
  - name: NEWRELIC_LICENSE_KEY
    valueFrom:
      secretKeyRef:
        name: newrelic-licenses
        key: NEWRELIC_LICENSE_KEY
  - name: NEWRELIC_OTLP_ENDPOINT
    valueFrom:
      secretKeyRef:
        name: newrelic-licenses
        key: NEWRELIC_OTLP_ENDPOINT
  - name: NEW_RELIC_MEMORY_LIMIT_MIB
    valueFrom:
      secretKeyRef:
        name: newrelic-licenses
        key: NEW_RELIC_MEMORY_LIMIT_MIB
  - name: K8S_CLUSTER_NAME
    value: "elasticsearch-cluster"

clusterRole:
  create: true
  rules:
    - apiGroups: [""]
      resources: ["pods", "nodes", "nodes/stats", "nodes/proxy"]
      verbs: ["get", "list", "watch"]
    - apiGroups: ["apps"]
      resources: ["replicasets"]
      verbs: ["get", "list", "watch"]

config:
  extensions:
    health_check:
      endpoint: 0.0.0.0:13133
    k8s_observer:
      auth_type: serviceAccount
      observe_pods: true
      observe_nodes: true

  receivers:
    jaeger: null
    zipkin: null
    prometheus: null
    otlp: null
    receiver_creator/elasticsearch:
      watch_observers: [k8s_observer]
      receivers:
        elasticsearch:
          rule: type == "pod" && labels["app"] == "elasticsearch"
          config:
            endpoint: 'http://`endpoint`:9200'
            collection_interval: 15s
            metrics:
              elasticsearch.os.cpu.usage:
                enabled: true
              elasticsearch.cluster.data_nodes:
                enabled: true
              elasticsearch.cluster.health:
                enabled: true
              elasticsearch.cluster.in_flight_fetch:
                enabled: true
              elasticsearch.cluster.nodes:
                enabled: true
              elasticsearch.cluster.pending_tasks:
                enabled: true
              elasticsearch.cluster.shards:
                enabled: true
              elasticsearch.cluster.state_update.time:
                enabled: true
              elasticsearch.index.documents:
                enabled: true
              elasticsearch.index.operations.merge.current:
                enabled: true
              elasticsearch.index.operations.time:
                enabled: true
              elasticsearch.node.cache.count:
                enabled: true
              elasticsearch.node.cache.evictions:
                enabled: true
              elasticsearch.node.cache.memory.usage:
                enabled: true
              elasticsearch.node.shards.size:
                enabled: true
              elasticsearch.node.cluster.io:
                enabled: true
              elasticsearch.node.documents:
                enabled: true
              elasticsearch.node.disk.io.read:
                enabled: true
              elasticsearch.node.disk.io.write:
                enabled: true
              elasticsearch.node.fs.disk.available:
                enabled: true
              elasticsearch.node.fs.disk.total:
                enabled: true
              elasticsearch.node.http.connections:
                enabled: true
              elasticsearch.node.ingest.documents.current:
                enabled: true
              elasticsearch.node.ingest.operations.failed:
                enabled: true
              elasticsearch.node.open_files:
                enabled: true
              elasticsearch.node.operations.completed:
                enabled: true
              elasticsearch.node.operations.current:
                enabled: true
              elasticsearch.node.operations.get.completed:
                enabled: true
              elasticsearch.node.operations.time:
                enabled: true
              elasticsearch.node.shards.reserved.size:
                enabled: true
              elasticsearch.index.shards.size:
                enabled: true
              elasticsearch.os.cpu.load_avg.1m:
                enabled: true
              elasticsearch.os.cpu.load_avg.5m:
                enabled: true
              elasticsearch.os.cpu.load_avg.15m:
                enabled: true
              elasticsearch.os.memory:
                enabled: true
              jvm.gc.collections.count:
                enabled: true
              jvm.gc.collections.elapsed:
                enabled: true
              jvm.memory.heap.max:
                enabled: true
              jvm.memory.heap.used:
                enabled: true
              jvm.memory.heap.utilization:
                enabled: true
              jvm.threads.count:
                enabled: true
              elasticsearch.index.segments.count:
                enabled: true
              elasticsearch.index.operations.completed:
                enabled: true
              elasticsearch.node.script.cache_evictions:
                enabled: false
              elasticsearch.node.cluster.connections:
                enabled: false
              elasticsearch.node.pipeline.ingest.documents.preprocessed:
                enabled: false
              elasticsearch.node.thread_pool.tasks.queued:
                enabled: false
              elasticsearch.cluster.published_states.full:
                enabled: false
              jvm.memory.pool.max:
                enabled: false
              elasticsearch.node.script.compilation_limit_triggered:
                enabled: false
              elasticsearch.node.shards.data_set.size:
                enabled: false
              elasticsearch.node.pipeline.ingest.documents.current:
                enabled: false
              elasticsearch.cluster.state_update.count:
                enabled: false
              elasticsearch.node.fs.disk.free:
                enabled: false
              jvm.memory.nonheap.used:
                enabled: false
              jvm.memory.pool.used:
                enabled: false
              elasticsearch.node.translog.size:
                enabled: false
              elasticsearch.node.thread_pool.threads:
                enabled: false
              elasticsearch.cluster.state_queue:
                enabled: false
              elasticsearch.node.translog.operations:
                enabled: false
              elasticsearch.memory.indexing_pressure:
                enabled: false
              elasticsearch.node.ingest.documents:
                enabled: false
              jvm.classes.loaded:
                enabled: false
              jvm.memory.heap.committed:
                enabled: false
              elasticsearch.breaker.memory.limit:
                enabled: false
              elasticsearch.indexing_pressure.memory.total.replica_rejections:
                enabled: false
              elasticsearch.breaker.memory.estimated:
                enabled: false
              elasticsearch.cluster.published_states.differences:
                enabled: false
              jvm.memory.nonheap.committed:
                enabled: false
              elasticsearch.node.translog.uncommitted.size:
                enabled: false
              elasticsearch.node.script.compilations:
                enabled: false
              elasticsearch.node.pipeline.ingest.operations.failed:
                enabled: false
              elasticsearch.indexing_pressure.memory.limit:
                enabled: false
              elasticsearch.breaker.tripped:
                enabled: false
              elasticsearch.indexing_pressure.memory.total.primary_rejections:
                enabled: false
              elasticsearch.node.thread_pool.tasks.finished:
                enabled: false

  processors:
    memory_limiter:
      check_interval: 60s
      limit_mib: ${env:NEW_RELIC_MEMORY_LIMIT_MIB}
    cumulativetodelta: {}
    attributes/cluster_state_aggregate:
      include:
        match_type: strict
        metric_names:
          - elasticsearch.cluster.state_update.time
      actions:
        - key: type
          action: delete
        - key: state
          action: delete
    filter/critical_operations:
      metrics:
        datapoint:
          - 'attributes["operation"] == "query" or attributes["operation"] == "index" or attributes["operation"] == "get" or attributes["operation"] == "merge" or attributes["operation"] == nil'
    resource/cluster:
      attributes:
        - key: k8s.cluster.name
          value: "${env:K8S_CLUSTER_NAME}"
          action: insert
    resource/cluster_name_override:
      attributes:
        - key: elasticsearch.cluster.name
          value: "${env:K8S_CLUSTER_NAME}"
          action: upsert
    resourcedetection:
      detectors: [env, system]
      system:
        resource_attributes:
          host.name:
            enabled: true
          host.id:
            enabled: true
          os.type:
            enabled: true
    batch:
      timeout: 30s
      send_batch_size: 2048
      send_batch_max_size: 4096
    attributes/cardinality_reduction:
      actions:
        - key: process.pid
          action: delete
        - key: process.parent_pid
          action: delete
        - key: k8s.pod.uid
          action: delete
    transform/metadata_nullify:
      metric_statements:
        - context: metric
          statements:
            - set(description, "")
            - set(unit, "")

  exporters:
    otlp_http:
      endpoint: "${env:NEWRELIC_OTLP_ENDPOINT}"
      headers:
        api-key: "${env:NEWRELIC_LICENSE_KEY}"
      compression: gzip
      timeout: 30s
      retry_on_failure:
        enabled: true
        initial_interval: 5s
        max_interval: 30s
        max_elapsed_time: 300s

  service:
    extensions: [health_check, k8s_observer]
    pipelines:
      traces: null
      logs: null
      metrics: null
      metrics/elasticsearch:
        receivers: [receiver_creator/elasticsearch]
        processors: [memory_limiter, resourcedetection, resource/cluster, resource/cluster_name_override, attributes/cardinality_reduction, filter/critical_operations, attributes/cluster_state_aggregate, cumulativetodelta, transform/metadata_nullify, batch]
        exporters: [otlp_http]

Sugerencia

Customize for your environment:

Cambios requeridos:

Pod label rule - Update labels["app"] == "elasticsearch" to match your pod labels
Cluster name - Replace elasticsearch-cluster in extraEnvs.K8S_CLUSTER_NAME with your unique cluster name
Cambios opcionales:
Puerto - Actualice 9200 si Elasticsearch se ejecuta en un puerto diferente
Authentication - Add credentials if secured (see below)

For secured Elasticsearch clusters:

receivers:
  receiver_creator/elasticsearch:
    watch_observers: [k8s_observer]
    receivers:
      elasticsearch:
        rule: type == "pod" && labels["app"] == "elasticsearch"
        config:
          endpoint: 'https://`endpoint`:9200'
          username: "your_elasticsearch_username"
          password: "your_elasticsearch_password"
          tls:
            insecure_skip_verify: false

Instalar con Helm

Install the NRDOT Collector using Helm with your values.yaml configuration:

bash

$helm repo add open-telemetry https://open-telemetry.github.io/opentelemetry-helm-charts
$helm repo update
$helm upgrade --install elasticsearch-nrdot-collector open-telemetry/opentelemetry-collector \
>  --namespace newrelic \
>  --create-namespace \
>  -f values.yaml

Verifique la implementación y la recopilación de datos

Verify that the NRDOT collector is running and collecting Elasticsearch data:

Verifique que los pods del recopilador se estén ejecutando:
bash
```
$kubectl get pods -n newrelic --watch
```
For manifest install: You should see pods with names like nr-k8s-otel-collector-deployment-<hash> in a Running state.
For Helm install: You should see pods with names like elasticsearch-nrdot-collector-<hash> in a Running state.
Revise los logs del colector en busca de errores:
For manifest install:
bash
```
$kubectl logs -n newrelic -l app.kubernetes.io/name=nr-k8s-otel-collector -f
```
For Helm install:
bash
```
$kubectl logs -n newrelic -l app.kubernetes.io/name=opentelemetry-collector -f
```
Busque conexiones exitosas a los pods de Elasticsearch y New Relic. Si ve errores, consulte la guía de solución de problemas.

Ejecute una consulta NRQL en New Relic para confirmar que los datos están llegando (reemplace elasticsearch-cluster por el nombre de su clúster):

FROM Metric
SELECT *
WHERE metricName LIKE 'elasticsearch.%'
  AND instrumentation.provider = 'opentelemetry'
  AND k8s.cluster.name = 'elasticsearch-cluster'
SINCE 10 minutes ago

Antes de que empieces

Antes de desplegar el OTel Collector Contrib en Kubernetes, asegúrese de tener:

Privilegios de acceso requeridos:

Tu New Relic
Acceso a su clúster de Kubernetes con kubectl
Privilegios de administrador de clúster de Elasticsearch con privilegio de clúster monitor o manage (consulte la documentación de privilegios de seguridad de Elasticsearch para obtener detalles)
Requisitos del sistema:
Versión 7.16 o superior de Elasticsearch - Esta integración requiere un clúster moderno de Elasticsearch
Clúster de Kubernetes - Un clúster de Kubernetes en ejecución donde Elasticsearch está desplegado
Helm 3.0 o superior - Helm instalado en su sistema
Conectividad de red - HTTPS saliente (puerto 443) al endpoint de ingesta OTLPde New Relic
Requisitos del pod de Elasticsearch:
Etiquetas de pod (Obligatorio) - Cada pod de Elasticsearch debe tener la etiqueta app: elasticsearch para que funcione el descubrimiento automático. Sin esta etiqueta, el colector no detectará ni monitoreará sus pods.
Importante
Cómo agregar etiquetas a los pods de Elasticsearch:
Si utiliza un StatefulSet o un Deployment para Elasticsearch, agregue la etiqueta en la plantilla del pod:
apiVersion: apps/v1 kind: StatefulSet metadata: name: elasticsearch spec: template: metadata: labels: app: elasticsearch # Required for auto-discovery spec: containers: - name: elasticsearch image: docker.elastic.co/elasticsearch/elasticsearch:8.x.x
Para los pods existentes sin etiquetas, actualice su StatefulSet/Deployment y reinicie los pods:
bash
```
$kubectl label pods -l <your-existing-selector> app=elasticsearch -n <namespace>
```
Puede verificar que las etiquetas estén configuradas correctamente:
bash
```
$kubectl get pods -n <namespace> --show-labels
```

Crear secreto de Kubernetes para credenciales

Crea un secreto de Kubernetes para almacenar tus credenciales de New Relic de forma segura:

Cree el espacio de nombres:
bash
```
$kubectl create namespace newrelic
```

Cree el secreto:

bash

$kubectl create secret generic newrelic-licenses \
>  --from-literal=NEWRELIC_LICENSE_KEY=YOUR_LICENSE_KEY_HERE \
>  --from-literal=NEWRELIC_OTLP_ENDPOINT=https://otlp.nr-data.net:4318 \
>  --from-literal=NEW_RELIC_MEMORY_LIMIT_MIB=100 \
>  -n newrelic

Actualice los valores:

Reemplace YOUR_LICENSE_KEY_HERE con su clave de licencia de New Relic real
Reemplace https://otlp.nr-data.net:4318 con el endpoint de su región (consulte la documentación del endpoint OTLP)
Reemplace 100 con el límite de memoria deseado en MiB para el colector (predeterminado: 100 MiB). Ajuste según las necesidades de su entorno

Configurar el monitoreo de Elasticsearch

Cree un archivo values.yaml para configurar el OpenTelemetry Collector para el monitoreo de Elasticsearch:

Sugerencia

Personalice para su entorno: Actualice los siguientes valores en la configuración:

Cambios requeridos:

Regla de etiquetas de Pod - La regla labels["app"] == "elasticsearch" debe coincidir con las etiquetas de su Pod. Si sus pods de Elasticsearch usan etiquetas diferentes (p. ej., app: es-cluster), actualice la regla según corresponda:
```
rule: type == "pod" && labels["app"] == "es-cluster"
```
Nombre del clúster - Reemplace elasticsearch-cluster por un nombre único para identificar su clúster en New Relic. Este nombre se utilizará para crear e identificar sus entidades de Elasticsearch en la UI de New Relic. Elige un nombre que sea único en tu cuenta de New Relic (p. ej., prod-es-k8s, staging-elasticsearch)
Cambios opcionales:
Puerto - Actualice 9200 si Elasticsearch se ejecuta en un puerto diferente
Autenticación - Agregue credenciales si su clúster de Elasticsearch está protegido

mode: deployment

image:
  repository: otel/opentelemetry-collector-contrib
  pullPolicy: IfNotPresent

command:
  name: otelcol-contrib

resources:
  limits:
    cpu: 500m
    memory: 512Mi
  requests:
    cpu: 200m
    memory: 256Mi

extraEnvs:
  - name: NEWRELIC_LICENSE_KEY
    valueFrom:
      secretKeyRef:
        name: newrelic-licenses
        key: NEWRELIC_LICENSE_KEY
  - name: NEWRELIC_OTLP_ENDPOINT
    valueFrom:
      secretKeyRef:
        name: newrelic-licenses
        key: NEWRELIC_OTLP_ENDPOINT
  - name: NEW_RELIC_MEMORY_LIMIT_MIB
    valueFrom:
      secretKeyRef:
        name: newrelic-licenses
        key: NEW_RELIC_MEMORY_LIMIT_MIB
  - name: K8S_CLUSTER_NAME
    value: "elasticsearch-cluster"

clusterRole:
  create: true
  rules:
    - apiGroups: [""]
      resources: ["pods", "nodes", "nodes/stats", "nodes/proxy"]
      verbs: ["get", "list", "watch"]
    - apiGroups: ["apps"]
      resources: ["replicasets"]
      verbs: ["get", "list", "watch"]

config:
    extensions:
      health_check:
        endpoint: 0.0.0.0:13133
      k8s_observer:
        auth_type: serviceAccount
        observe_pods: true
        observe_nodes: true

    receivers:
      receiver_creator/elasticsearch:
        watch_observers: [k8s_observer]
        receivers:
          elasticsearch:
            rule: type == "pod" && labels["app"] == "elasticsearch"
            config:
              endpoint: 'http://`endpoint`:9200'
              collection_interval: 30s
              metrics:
                elasticsearch.os.cpu.usage:
                  enabled: true
                elasticsearch.cluster.data_nodes:
                  enabled: true
                elasticsearch.cluster.health:
                  enabled: true
                elasticsearch.cluster.in_flight_fetch:
                  enabled: true
                elasticsearch.cluster.nodes:
                  enabled: true
                elasticsearch.cluster.pending_tasks:
                  enabled: true
                elasticsearch.cluster.shards:
                  enabled: true
                elasticsearch.cluster.state_update.time:
                  enabled: true
                elasticsearch.index.documents:
                  enabled: true
                elasticsearch.index.operations.merge.current:
                  enabled: true
                elasticsearch.index.operations.time:
                  enabled: true
                elasticsearch.node.cache.count:
                  enabled: true
                elasticsearch.node.cache.evictions:
                  enabled: true
                elasticsearch.node.cache.memory.usage:
                  enabled: true
                elasticsearch.node.shards.size:
                  enabled: true
                elasticsearch.node.cluster.io:
                  enabled: true
                elasticsearch.node.documents:
                  enabled: true
                elasticsearch.node.disk.io.read:
                  enabled: true
                elasticsearch.node.disk.io.write:
                  enabled: true
                elasticsearch.node.fs.disk.available:
                  enabled: true
                elasticsearch.node.fs.disk.total:
                  enabled: true
                elasticsearch.node.http.connections:
                  enabled: true
                elasticsearch.node.ingest.documents.current:
                  enabled: true
                elasticsearch.node.ingest.operations.failed:
                  enabled: true
                elasticsearch.node.open_files:
                  enabled: true
                elasticsearch.node.operations.completed:
                  enabled: true
                elasticsearch.node.operations.current:
                  enabled: true
                elasticsearch.node.operations.get.completed:
                  enabled: true
                elasticsearch.node.operations.time:
                  enabled: true
                elasticsearch.node.shards.reserved.size:
                  enabled: true
                elasticsearch.index.shards.size:
                  enabled: true
                elasticsearch.os.cpu.load_avg.1m:
                  enabled: true
                elasticsearch.os.cpu.load_avg.5m:
                  enabled: true
                elasticsearch.os.cpu.load_avg.15m:
                  enabled: true
                elasticsearch.os.memory:
                  enabled: true
                jvm.gc.collections.count:
                  enabled: true
                jvm.gc.collections.elapsed:
                  enabled: true
                jvm.memory.heap.max:
                  enabled: true
                jvm.memory.heap.used:
                  enabled: true
                jvm.memory.heap.utilization:
                  enabled: true
                jvm.threads.count:
                  enabled: true
                elasticsearch.index.segments.count:
                  enabled: true
                elasticsearch.index.operations.completed:
                  enabled: true
                elasticsearch.node.script.cache_evictions:
                  enabled: false
                elasticsearch.node.cluster.connections:
                  enabled: false
                elasticsearch.node.pipeline.ingest.documents.preprocessed:
                  enabled: false
                elasticsearch.node.thread_pool.tasks.queued:
                  enabled: false
                elasticsearch.cluster.published_states.full:
                  enabled: false
                jvm.memory.pool.max:
                  enabled: false
                elasticsearch.node.script.compilation_limit_triggered:
                  enabled: false
                elasticsearch.node.shards.data_set.size:
                  enabled: false
                elasticsearch.node.pipeline.ingest.documents.current:
                  enabled: false
                elasticsearch.cluster.state_update.count:
                  enabled: false
                elasticsearch.node.fs.disk.free:
                  enabled: false
                jvm.memory.nonheap.used:
                  enabled: false
                jvm.memory.pool.used:
                  enabled: false
                elasticsearch.node.translog.size:
                  enabled: false
                elasticsearch.node.thread_pool.threads:
                  enabled: false
                elasticsearch.cluster.state_queue:
                  enabled: false
                elasticsearch.node.translog.operations:
                  enabled: false
                elasticsearch.memory.indexing_pressure:
                  enabled: false
                elasticsearch.node.ingest.documents:
                  enabled: false
                jvm.classes.loaded:
                  enabled: false
                jvm.memory.heap.committed:
                  enabled: false
                elasticsearch.breaker.memory.limit:
                  enabled: false
                elasticsearch.indexing_pressure.memory.total.replica_rejections:
                  enabled: false
                elasticsearch.breaker.memory.estimated:
                  enabled: false
                elasticsearch.cluster.published_states.differences:
                  enabled: false
                jvm.memory.nonheap.committed:
                  enabled: false
                elasticsearch.node.translog.uncommitted.size:
                  enabled: false
                elasticsearch.node.script.compilations:
                  enabled: false
                elasticsearch.node.pipeline.ingest.operations.failed:
                  enabled: false
                elasticsearch.indexing_pressure.memory.limit:
                  enabled: false
                elasticsearch.breaker.tripped:
                  enabled: false
                elasticsearch.indexing_pressure.memory.total.primary_rejections:
                  enabled: false
                elasticsearch.node.thread_pool.tasks.finished:
                  enabled: false

    processors:
      memory_limiter:
        check_interval: 60s
        limit_mib: ${env:NEW_RELIC_MEMORY_LIMIT_MIB}
      cumulativetodelta: {}
      resource/cluster:
        attributes:
          - key: k8s.cluster.name
            value: "${env:K8S_CLUSTER_NAME}"
            action: insert
      resource/cluster_name_override:
        attributes:
          - key: elasticsearch.cluster.name
            value: "${env:K8S_CLUSTER_NAME}"
            action: upsert
      resourcedetection:
        detectors: [env, system]
        system:
          resource_attributes:
            host.name:
              enabled: true
            host.id:
              enabled: true
            os.type:
              enabled: true
      batch:
        timeout: 10s
        send_batch_size: 1024
      attributes/cardinality_reduction:
        actions:
          - key: process.pid
            action: delete
          - key: process.parent_pid
            action: delete
          - key: k8s.pod.uid
            action: delete
      transform/metadata_nullify:
        metric_statements:
          - context: metric
            statements:
              - set(description, "")
              - set(unit, "")

    exporters:
      otlphttp:
        endpoint: "${env:NEWRELIC_OTLP_ENDPOINT}"
        headers:
          api-key: "${env:NEWRELIC_LICENSE_KEY}"

    service:
      extensions: [health_check, k8s_observer]
      pipelines:
        metrics/elasticsearch:
          receivers: [receiver_creator/elasticsearch]
          processors: [memory_limiter, resourcedetection, resource/cluster, resource/cluster_name_override, attributes/cardinality_reduction, cumulativetodelta, transform/metadata_nullify, batch]
          exporters: [otlphttp]

Sugerencia

Para clústeres de Elasticsearch protegidos: Si su clúster de Elasticsearch requiere autenticación, agregue las credenciales a la configuración del receptor:

receiver_creator/elasticsearch:
  watch_observers: [k8s_observer]
  receivers:
    elasticsearch:
      rule: type == "pod" && labels["app"] == "elasticsearch"
      config:
        endpoint: 'https://`endpoint`:9200'
        username: "your_elasticsearch_username"
        password: "your_elasticsearch_password"
        tls:
          insecure_skip_verify: false

Almacene las credenciales de forma segura utilizando secretos de Kubernetes en lugar de codificarlas directamente en el archivo de valores.

Instalar con Helm

Instale el OpenTelemetry Collector usando Helm con su configuración values.yaml:

bash

$helm repo add open-telemetry https://open-telemetry.github.io/opentelemetry-helm-charts
$helm repo update
$helm upgrade --install elasticsearch-otel-collector open-telemetry/opentelemetry-collector \
>  --namespace newrelic \
>  --create-namespace \
>  -f values.yaml

Verifique la implementación y la recopilación de datos

Verifique que el OpenTelemetry Collector se esté ejecutando y recopilando datos de Elasticsearch:

Verifique que los pods del recopilador se estén ejecutando:
bash
```
$kubectl get pods -n newrelic --watch
```
Debería ver pods con nombres como elasticsearch-otel-collector-<hash> en un estado Running.
Revise los logs del colector en busca de errores:
bash
```
$kubectl logs -n newrelic -l app.kubernetes.io/name=opentelemetry-collector -f
```
Busque conexiones exitosas a los pods de Elasticsearch y New Relic. Si ve errores, consulte la guía de solución de problemas.

Ejecute una consulta NRQL en New Relic para confirmar que los datos están llegando (reemplace elasticsearch-cluster por el nombre de su clúster):

FROM Metric
SELECT *
WHERE metricName LIKE 'elasticsearch.%'
  AND instrumentation.provider = 'opentelemetry'
  AND k8s.cluster.name = 'elasticsearch-cluster'
SINCE 10 minutes ago

Use this approach if you already have a Prometheus Elasticsearch exporter running in your Kubernetes cluster, or if you're migrating from a Prometheus-based monitoring stack.

Sugerencia

Recommended: If you don't already have a Prometheus exporter running, use the NRDOT Collector or OTel Collector Contrib tabs instead. They connect directly to the Elasticsearch API without needing an additional exporter component.

Deploy the Elasticsearch exporter

If you don't already have the exporter running, deploy it using Helm:

bash

$helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
$helm repo update
$
$helm install elasticsearch-exporter prometheus-community/prometheus-elasticsearch-exporter \
>  --namespace monitoring \
>  --create-namespace \
>  --set es.uri=http://elasticsearch.default.svc.cluster.local:9200

Replace elasticsearch.default.svc.cluster.local:9200 with your Elasticsearch service address.

Verify the exporter is running:

bash

$kubectl get pods -n monitoring -l app=prometheus-elasticsearch-exporter

Create the credentials Secret

bash

$kubectl create secret generic newrelic-credentials \
>  --namespace monitoring \
>  --from-literal=NEWRELIC_LICENSE_KEY=YOUR_NEWRELIC_LICENSE_KEY \
>  --from-literal=NEWRELIC_OTLP_ENDPOINT=https://otlp.nr-data.net:4318

Reemplace YOUR_NEWRELIC_LICENSE_KEY con su .

Sugerencia

For EU accounts, use NEWRELIC_OTLP_ENDPOINT=https://otlp.eu01.nr-data.net:4318

Create the collector ConfigMap

Create a ConfigMap with the collector configuration. This works with both NRDOT (newrelic/nrdot-collector) and OTel Collector Contrib (otel/opentelemetry-collector-contrib) container images. The configuration scrapes metrics from the Elasticsearch exporter and translates Prometheus metric names to OpenTelemetry-compatible names that power the New Relic Elasticsearch dashboard.

Save the following as otel-collector-prometheus-es.yaml and apply with kubectl apply -f otel-collector-prometheus-es.yaml:

apiVersion: v1
kind: ConfigMap
metadata:
  name: otel-collector-prometheus-es
  namespace: monitoring
data:
  config.yaml: |
    receivers:
      prometheus/elasticsearch:
        config:
          scrape_configs:
            - job_name: 'elasticsearch'
              scrape_interval: 15s
              static_configs:
                - targets: ['elasticsearch-exporter-prometheus-elasticsearch-exporter.monitoring.svc.cluster.local:9114']

    processors:
      memory_limiter:
        check_interval: 60s
        limit_mib: 100

      metricstransform/prom_to_otel:
        transforms:
          - include: ^elasticsearch_cluster_health_number_of_data_nodes$
            match_type: regexp
            action: update
            new_name: elasticsearch.cluster.data_nodes
          - include: ^elasticsearch_cluster_health_status$
            match_type: regexp
            action: update
            new_name: elasticsearch.cluster.health
            operations:
              - action: update_label
                label: color
                new_label: status
          - include: ^elasticsearch_cluster_health_number_of_in_flight_fetch$
            match_type: regexp
            action: update
            new_name: elasticsearch.cluster.in_flight_fetch
          - include: ^elasticsearch_cluster_health_number_of_nodes$
            match_type: regexp
            action: update
            new_name: elasticsearch.cluster.nodes
          - include: ^elasticsearch_cluster_health_number_of_pending_tasks$
            match_type: regexp
            action: update
            new_name: elasticsearch.cluster.pending_tasks
          - include: ^elasticsearch_cluster_health_active_shards$
            match_type: regexp
            action: update
            new_name: elasticsearch.cluster.shards
            operations:
              - action: add_label
                new_label: state
                new_value: active
          - include: ^elasticsearch_cluster_health_unassigned_shards$
            match_type: regexp
            action: update
            new_name: elasticsearch.cluster.shards
            operations:
              - action: add_label
                new_label: state
                new_value: unassigned
          - include: ^elasticsearch_indices_docs$
            match_type: regexp
            action: insert
            new_name: elasticsearch.node.documents
            operations:
              - action: add_label
                new_label: state
                new_value: active
          - include: ^elasticsearch_indices_docs$
            match_type: regexp
            action: insert
            new_name: elasticsearch.index.documents
            operations:
              - action: add_label
                new_label: state
                new_value: active
              - action: add_label
                new_label: aggregation
                new_value: primary_shards
              - action: add_label
                new_label: elasticsearch.index.name
                new_value: _all
          - include: ^elasticsearch_indices_fielddata_evictions$
            match_type: regexp
            action: update
            new_name: elasticsearch.node.cache.evictions
            operations:
              - action: add_label
                new_label: cache_name
                new_value: fielddata
          - include: ^elasticsearch_indices_query_cache_evictions$
            match_type: regexp
            action: update
            new_name: elasticsearch.node.cache.evictions
            operations:
              - action: add_label
                new_label: cache_name
                new_value: query
          - include: ^elasticsearch_transport_rx_size_bytes_total$
            match_type: regexp
            action: update
            new_name: elasticsearch.node.cluster.io
            operations:
              - action: add_label
                new_label: direction
                new_value: received
          - include: ^elasticsearch_transport_tx_size_bytes_total$
            match_type: regexp
            action: update
            new_name: elasticsearch.node.cluster.io
            operations:
              - action: add_label
                new_label: direction
                new_value: sent
          - include: ^elasticsearch_filesystem_data_available_bytes$
            match_type: regexp
            action: update
            new_name: elasticsearch.node.fs.disk.available
          - include: ^elasticsearch_filesystem_data_size_bytes$
            match_type: regexp
            action: update
            new_name: elasticsearch.node.fs.disk.total
          - include: ^elasticsearch_indices_indexing_index_total$
            match_type: regexp
            action: update
            new_name: elasticsearch.node.operations.completed
            operations:
              - action: add_label
                new_label: operation
                new_value: index
          - include: ^elasticsearch_indices_search_query_total$
            match_type: regexp
            action: update
            new_name: elasticsearch.node.operations.completed
            operations:
              - action: add_label
                new_label: operation
                new_value: query
          - include: ^elasticsearch_indices_get_total$
            match_type: regexp
            action: update
            new_name: elasticsearch.node.operations.completed
            operations:
              - action: add_label
                new_label: operation
                new_value: get
          - include: ^elasticsearch_indices_merges_total$
            match_type: regexp
            action: update
            new_name: elasticsearch.node.operations.completed
            operations:
              - action: add_label
                new_label: operation
                new_value: merge
          - include: ^elasticsearch_indices_indexing_index_time_seconds_total$
            match_type: regexp
            action: update
            new_name: elasticsearch.node.operations.time
            operations:
              - action: add_label
                new_label: operation
                new_value: index
              - action: experimental_scale_value
                experimental_scale: 1000
          - include: ^elasticsearch_indices_search_query_time_seconds$
            match_type: regexp
            action: update
            new_name: elasticsearch.node.operations.time
            operations:
              - action: add_label
                new_label: operation
                new_value: query
              - action: experimental_scale_value
                experimental_scale: 1000
          - include: ^elasticsearch_os_cpu_percent$
            match_type: regexp
            action: update
            new_name: elasticsearch.os.cpu.usage
          - include: ^elasticsearch_jvm_gc_collection_seconds_count$
            match_type: regexp
            action: update
            new_name: jvm.gc.collections.count
          - include: ^elasticsearch_jvm_memory_max_bytes$
            match_type: regexp
            action: update
            new_name: jvm.memory.heap.max
          - include: ^elasticsearch_jvm_memory_used_bytes$
            match_type: regexp
            action: update
            new_name: jvm.memory.heap.used
          - include: ^elasticsearch_thread_pool_threads_count$
            match_type: regexp
            action: update
            new_name: jvm.threads.count
            operations:
              - action: aggregate_labels
                label_set: [name, cluster]
                aggregation_type: sum
          - include: ^elasticsearch_indices_segments_count$
            match_type: regexp
            action: update
            new_name: elasticsearch.index.segments.count

      filter/allowed_metrics:
        metrics:
          include:
            match_type: strict
            metric_names:
              - elasticsearch.cluster.data_nodes
              - elasticsearch.cluster.health
              - elasticsearch.cluster.in_flight_fetch
              - elasticsearch.cluster.nodes
              - elasticsearch.cluster.pending_tasks
              - elasticsearch.cluster.shards
              - elasticsearch.index.documents
              - elasticsearch.index.segments.count
              - elasticsearch.node.cache.evictions
              - elasticsearch.node.cluster.io
              - elasticsearch.node.documents
              - elasticsearch.node.fs.disk.available
              - elasticsearch.node.fs.disk.total
              - elasticsearch.node.operations.completed
              - elasticsearch.node.operations.time
              - elasticsearch.os.cpu.usage
              - jvm.gc.collections.count
              - jvm.memory.heap.max
              - jvm.memory.heap.used
              - jvm.threads.count

      transform/rename_node_name:
        metric_statements:
          - context: datapoint
            statements:
              - set(attributes["elasticsearch.node.name"], attributes["name"]) where attributes["name"] != nil
              - delete_key(attributes, "name") where attributes["elasticsearch.node.name"] != nil

      groupbyattrs/node:
        keys:
          - elasticsearch.node.name
          - elasticsearch.cluster.name

      cumulativetodelta: {}

      resource/cluster_name_override:
        attributes:
          - key: elasticsearch.cluster.name
            value: "<elasticsearch-cluster-name>"
            action: upsert

      transform/scope_override:
        metric_statements:
          - context: scope
            statements:
              - set(name, "github.com/open-telemetry/opentelemetry-collector-contrib/receiver/elasticsearchreceiver")
              - set(version, "")

      resourcedetection:
        detectors: [env]
        env:
          resource_attributes:
            host.name:
              enabled: true

      batch:
        timeout: 30s
        send_batch_size: 2048
        send_batch_max_size: 4096

      attributes/cardinality_reduction:
        actions:
          - key: process.pid
            action: delete
          - key: process.parent_pid
            action: delete

      transform/metadata_nullify:
        metric_statements:
          - context: metric
            statements:
              - set(description, "")
              - set(unit, "")

    exporters:
      otlphttp:
        endpoint: ${env:NEWRELIC_OTLP_ENDPOINT}
        headers:
          api-key: ${env:NEWRELIC_LICENSE_KEY}
        compression: gzip
        timeout: 30s
        retry_on_failure:
          enabled: true
          initial_interval: 5s
          max_interval: 30s
          max_elapsed_time: 300s

    service:
      pipelines:
        metrics/elasticsearch:
          receivers: [prometheus/elasticsearch]
          processors:
            - memory_limiter
            - metricstransform/prom_to_otel
            - filter/allowed_metrics
            - transform/rename_node_name
            - groupbyattrs/node
            - transform/scope_override
            - resourcedetection
            - resource/cluster_name_override
            - attributes/cardinality_reduction
            - cumulativetodelta
            - transform/metadata_nullify
            - batch
          exporters: [otlphttp]

Replace the following values in the configuration:

<elasticsearch-cluster-name>: Your Elasticsearch cluster name
elasticsearch-exporter-prometheus-elasticsearch-exporter.monitoring.svc.cluster.local:9114: Your exporter's Kubernetes service address

Deploy the collector

Deploy the collector using either the NRDOT or OTel Collector Contrib image. Update the image field below based on your choice:

NRDOT: newrelic/nrdot-collector:latest

OTel Collector Contrib: otel/opentelemetry-collector-contrib:latest

Save the following as otel-collector-deployment.yaml and apply with kubectl apply -f otel-collector-deployment.yaml:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: otel-collector-elasticsearch
  namespace: monitoring
  labels:
    app: otel-collector-elasticsearch
spec:
  replicas: 1
  selector:
    matchLabels:
      app: otel-collector-elasticsearch
  template:
    metadata:
      labels:
        app: otel-collector-elasticsearch
    spec:
      containers:
        - name: otel-collector
          image: otel/opentelemetry-collector-contrib:latest
          args:
            - "--config=/etc/otel/config.yaml"
          env:
            - name: NEWRELIC_LICENSE_KEY
              valueFrom:
                secretKeyRef:
                  name: newrelic-credentials
                  key: NEWRELIC_LICENSE_KEY
            - name: NEWRELIC_OTLP_ENDPOINT
              valueFrom:
                secretKeyRef:
                  name: newrelic-credentials
                  key: NEWRELIC_OTLP_ENDPOINT
          volumeMounts:
            - name: config
              mountPath: /etc/otel
          resources:
            requests:
              memory: "128Mi"
              cpu: "100m"
            limits:
              memory: "256Mi"
              cpu: "500m"
      volumes:
        - name: config
          configMap:
            name: otel-collector-prometheus-es

Verify the deployment

Check the collector pod is running:

bash

$kubectl get pods -n monitoring -l app=otel-collector-elasticsearch

Check collector logs:

bash

$kubectl logs -n monitoring -l app=otel-collector-elasticsearch -f

Verify data in New Relic:

FROM Metric SELECT count(*)
WHERE metricName LIKE 'elasticsearch.%'
AND elasticsearch.cluster.name = '<elasticsearch-cluster-name>'
SINCE 10 minutes ago

Sugerencia

Correlacione APM con Elasticsearch: Para conectar su aplicación APM y el clúster de Elasticsearch, incluya el atributo de recurso es.cluster.name="your-cluster-name" en sus métricas de APM. Esto permite la visibilidad entre servicios y una solución de problemas más rápida dentro de New Relic.

Resolución de problemas

Si encuentra problemas durante la instalación o no ve datos en New Relic, consulte nuestra completa guía de solución de problemas para obtener soluciones paso a paso a los problemas comunes.

Para problemas específicos de Kubernetes, como el descubrimiento de pods, los permisos RBAC o la conectividad de red, consulte la sección de resolución de problemas de Kubernetes.

Te ofrecemos esta traducción automática para facilitar la lectura.

Monitorear Elasticsearch en Kubernetes con OpenTelemetry

Opciones de instalación

Antes de que empieces

Importante

Elige tu método de instalación

Install and configure NRDOT Collector

Instalación con manifiesto

Complete base Kubernetes OpenTelemetry manifest installation

Update collector image to NRDOT

Configurar el monitoreo de Elasticsearch [#nrdot-configure-es]

Sugerencia

Apply the updated manifests [#nrdot-apply-manifests]

Instalación con Helm

Crear secreto de Kubernetes para credenciales

Configurar el monitoreo de Elasticsearch

Sugerencia

Instalar con Helm

Verifique la implementación y la recopilación de datos

Antes de que empieces

Importante

Crear secreto de Kubernetes para credenciales

Configurar el monitoreo de Elasticsearch

Sugerencia

Sugerencia

Instalar con Helm

Verifique la implementación y la recopilación de datos

Sugerencia

Deploy the Elasticsearch exporter

Create the credentials Secret

Sugerencia

Create the collector ConfigMap

Full Prometheus receiver ConfigMap

Deploy the collector

Verify the deployment

Sugerencia

Resolución de problemas

Te ofrecemos esta traducción automática para facilitar la lectura.

Monitorear Elasticsearch en Kubernetes con OpenTelemetry

Opciones de instalación .css-21sua1{background:none;border:none;width:0;padding:0;}

Antes de que empieces

Importante

Elige tu método de instalación

Install and configure NRDOT Collector

Instalación con Helm

Verifique la implementación y la recopilación de datos

Antes de que empieces

Importante

Crear secreto de Kubernetes para credenciales

Configurar el monitoreo de Elasticsearch

Sugerencia

Sugerencia

Instalar con Helm

Verifique la implementación y la recopilación de datos

Sugerencia

Deploy the Elasticsearch exporter

Create the credentials Secret

Sugerencia

Create the collector ConfigMap

Full Prometheus receiver ConfigMap

Deploy the collector

Verify the deployment

Sugerencia

Resolución de problemas

Opciones de instalación