Pipeline control overview

Pipeline Control is the data management component of New Relic Control. It helps you filter, transform, and manage telemetry data to reduce costs, improve signal quality, and maintain compliance.

Pipeline Control offers two processing layers that you can use independently or together:

Pipeline Control gateway

Gateway rules run in your infrastructure:

• Process data before it leaves your network (reduces egress costs)
• Organization-wide configuration (rules apply across all accounts in your organization)
• Full transformation, filtering, and sampling capabilities
• You manage the infrastructure (requires Kubernetes cluster deployment)
• Charged based on data volume received at gateway

The gateway is best when you need:

• To reduce network egress costs by filtering data before it leaves your infrastructure
• Richer data transformation capabilities for all MELT types (e.g. parse logs, normalize attributes, enrich with metadata)
• To drop or obfuscate sensitive data before it leaves your network for compliance requirements
• To process higher volumes of data, whereby gateway is cheaper than cloud rules

Learn more about the gateway in Gateway overview.

Pipeline Control cloud rules

Cloud rules run in New Relic's infrastructure after data reaches New Relic endpoints:

• Process data after it reaches New Relic (no infrastructure management required)
• Account-specific rules (each rule applies to a single New Relic account)
• NRQL-based drop rules for filtering data and attributes
• No infrastructure setup or management required
• Charged based on data dropped (7.5 GB per aCCU) plus billions of events scanned (2.237 BSE per aCCU)

Cloud rules are best when:

• You want simple drop rules without managing gateway infrastructure
• Your processing requirements are modest and don't justify a full-scale gateway deployment
• Lower volume filtering as a % of your total ingest where cloud rules pricing may be more cost-effective

Learn more: Create cloud rules

Using both together

You can also use gateway and cloud rules simultaneously:

• Telemetry sources (e.g. agents) can be configured differently to route through your gateway or not
• Defense-in-depth strategy ensures sensitive data is absolutely dropped
• Convenient to manage some rules via cloud API/UI while using gateway for primary processing

Comparison table

For detailed pricing information and cost calculations, see Pipeline Control costs.

Prerequisites

For cloud rules

• A New Relic account in a role with RBAC permissions to manage Pipeline Control cloud rules
• Understanding of NRQL to create filtering rules

For gateway

• A New Relic account with Organization Product Admin or Organization Manager permissions

• Familiarity with Kubernetes operations and management
• Kubernetes cluster admin access to deploy and manage resources within your cluster
• Permissions to create and manage private DNS hostnames within your network
• Permissions to create and manage DNS records
• Helm installed locally

Next step

Choose your approach based on your needs:

• Start with gateway: Set up and deploy Pipeline Control gateway
• Start with cloud rules: Create pipeline rules