New Relic integrations with the Google Cloud Platform (GCP) include one that reports Google Cloud Pub/Sub data to New Relic. This document explains how to activate the GCP Pub/Sub integration and describes the data it reports.
Features
As part of the Google Cloud stream analytics solution, the Cloud Pub/Sub service ingests and delivers event streams for quick data processing and analysis. New Relic Pub/Sub integration collects publish, delivery, backlog, and retention metrics across topics, subscriptions, and snapshots.
Activate integration
To enable the integration, follow standard procedures to connect your GCP service to New Relic:
Polling frequency
New Relic integrations query your GCP services according to a polling interval that varies by integration. The polling frequency for Google Cloud Pub/Sub is 5 minutes. The resolution is 1 data point every minute.
Important
Cloud Pub/Sub supports 1-minute polling intervals for WIF integrations in limited preview. To request access, contact New Relic support.
Workload Identity Federation
Find and use data
After you enable the integration, your Pub/Sub resources appear as entities in the New Relic entity explorer. To see dashboards and manage services, go to one.newrelic.com > All capabilities > Infrastructure > GCP.
All Pub/Sub metrics available in GCP Cloud Monitoring are collected as dimensional metrics in the Metric event type. Additional metrics beyond this table are collected automatically. See Google's Pub/Sub metrics documentation for the complete list.
Entities
Metric data
Key metrics — Topic
Metric name | Unit | Description |
|---|---|---|
| Count | Number of publish requests to the topic, faceted by response code. |
| Microseconds | Publish request latency distribution at the topic. |
| Count | Number of messages published to the topic. |
| Bytes | Distribution of published message sizes at the topic. |
| Bytes | Byte cost of operations on the topic, used for quota accounting. |
| Count | Number of messages retained in the topic. |
| Count | Number of unacknowledged messages held by the topic, faceted by region. |
| Bytes | Total byte size of messages retained in the topic. |
| Seconds | Age of the oldest message retained in the topic. |
| Seconds | Age of the oldest unacknowledged message in the topic, faceted by region. |
| Percent | Publish-quota utilization ratio for the topic. |
| Count | Number of configuration changes applied to the topic, faceted by operation type and result. |
For the complete list of topic metrics, see Google's Pub/Sub metrics documentation.
Key metrics — Subscription
Metric name | Unit | Description |
|---|---|---|
| Count | Number of unacknowledged backlog messages in the subscription. |
| Bytes | Total byte size of unacknowledged messages in the subscription. |
| Seconds | Age of the oldest unacknowledged message in the subscription. |
| Count | Number of messages delivered to the subscription's push endpoint but not yet acknowledged. |
| Count | Number of messages acknowledged on the subscription. |
| Milliseconds | Acknowledgement latency distribution at the subscription. |
| Count | Number of pull requests issued against the subscription, faceted by response code. |
| Count | Number of acknowledge requests issued against the subscription, faceted by response code. |
| Count | Number of push attempts from the subscription to its endpoint, faceted by response code. |
| Microseconds | Push request latency distribution from the subscription to its endpoint, faceted by response code. |
| Count | Number of StreamingPull responses returned for the subscription, faceted by response code. |
| Count | Number of acknowledgement deadlines that expired before the subscriber acknowledged the message. |
| Count | Number of messages delivered from the subscription to its dead-letter topic. |
| Bytes | Byte cost of operations on the subscription, used for quota accounting. |
| Boolean | Health score indicating whether the subscription's delivery latency is within its target. |
For the complete list of subscription metrics, see Google's Pub/Sub metrics documentation.
Key metrics — Snapshot
Metric name | Unit | Description |
|---|---|---|
| Count | Number of messages held in the snapshot. |
| Bytes | Total byte size of messages held in the snapshot. |
| Seconds | Age of the oldest message in the snapshot. |
| Count | Number of messages held in the snapshot, faceted by region. |
| Bytes | Total byte size of messages held in the snapshot, faceted by region. |
| Seconds | Age of the oldest message in the snapshot, faceted by region. |
| Count | Number of configuration changes applied to the snapshot, faceted by operation type and result. |
For the complete list of snapshot metrics, see Google's Pub/Sub metrics documentation.
Service account or user account
Find and use data
After activating the integration and waiting a few minutes (based on the polling frequency), data will appear in the New Relic UI. To find and use your data, including links to your and alert settings, go to one.newrelic.com > All capabilities > Infrastructure > GCP > (select an integration).
Pub/Sub topics are named entities that represent feeds of messages, and subscriptions are named entities that represent message destinations on a particular topic.
To view metric data for your GCP Pub/Sub integration in New Relic, create NRQL queries for GcpPubSubTopicSample and GcpPubSubSubscriptionSample events and their related attributes.
GcpPubSubTopicSample
Query GcpPubSubTopicSample events in New Relic to view data for the following attributes:
Attribute | Description |
|---|---|
| Cost of operations, measured in bytes. This is used to measure utilization for quotas. |
| Cumulative count of configuration changes, grouped by operation type and result. |
| Distribution of publish message sizes (in bytes). |
| Number of acknowledged messages retained in a topic, broken down by Cloud region. |
| Number of unacknowledged messages in a topic, broken down by Cloud region. |
| Age (in seconds) of the oldest acknowledged message retained in a topic, broken down by Cloud region. |
| Age (in seconds) of the oldest unacknowledged message in a topic, broken down by Cloud region. |
| Total byte size of the acknowledged messages retained in a topic, broken down by Cloud region. |
| Cumulative count of publish message operations. |
| Cumulative count of publish requests. |
| Total byte size of the unacknowledged messages in a topic, broken down by Cloud region. |
GcpPubSubSubscriptionSample
Query GcpPubSubSubscriptionSample events in New Relic to view data for the following attributes:
Attribute | Description |
|---|---|
| Total byte size of the unacknowledged messages (a.k.a. backlog messages) in a subscription. |
| Cumulative cost of operations, measured in bytes. This is used to measure quota utilization. |
| Cumulative count of configuration changes for each subscription, grouped by operation type and result. |
| Cumulative count of ModifyAckDeadline message operations, grouped by result. |
| Cumulative count of ModifyAckDeadline requests, grouped by result. |
| Number of messages delivered to a subscription's push endpoint, but not yet acknowledged. |
| Number of acknowledged messages retained in a subscription. |
| Number of acknowledged messages retained in a subscription, broken down by Cloud region. |
| Number of unacknowledged messages in a subscription, broken down by Cloud region. |
| Number of unacknowledged messages (a.k.a. backlog messages) in a subscription. |
| Age (in seconds) of the oldest acknowledged message retained in a subscription. |
| Age (in seconds) of the oldest acknowledged message retained in a subscription, broken down by Cloud region. |
| Age (in seconds) of the oldest unacknowledged message (a.k.a. backlog message) in a subscription. |
| Age (in seconds) of the oldest unacknowledged message in a subscription, broken down by Cloud region. |
| Cumulative count of acknowledge message operations, grouped by result. |
| Cumulative count of acknowledge requests, grouped by result. |
| Cumulative count of pull message operations, grouped by result. |
| Cumulative count of pull requests, grouped by result. |
| Cumulative count of push attempts, grouped by result. Unlike pulls, the push server implementation does not batch user messages, so each request only contains one user message. The push server retries on errors, so a given user message can appear multiple times. |
| Distribution of push request latencies (in microseconds), grouped by result. |
| Total byte size of the acknowledged messages retained in a subscription. |
| Total byte size of the acknowledged messages retained in a subscription, broken down by Cloud region. |
| Cumulative count of StreamingPull acknowledge message operations, grouped by result. |
| Cumulative count of streaming pull requests with non-empty acknowledge ids, grouped by result. |
| Cumulative count of streaming pull message operations, grouped by result. |
| Cumulative count of StreamingPull ModifyAckDeadline operations, grouped by result. |
| Cumulative count of streaming pull requests with non-empty ModifyAckDeadline fields, grouped by result. |
| Cumulative count of streaming pull responses, grouped by result. |
| Total byte size of the unacknowledged messages in a subscription, broken down by Cloud region. |