Pipeline Control is the data management component of New Relic Control. It helps you filter, transform, and manage telemetry data to reduce costs, improve signal quality, and maintain compliance.
Pipeline Control offers two processing layers that you can use independently or together:
Pipeline Control gateway
Gateway rules run in your infrastructure:
- Process data before it leaves your network (reduces egress costs)
- Organization-wide configuration (rules apply across all accounts in your organization)
- Full transformation, filtering, and sampling capabilities
- You manage the infrastructure (requires Kubernetes cluster deployment)
- Charged based on data volume received at gateway
The gateway is best when you need:
- To reduce network egress costs by filtering data before it leaves your infrastructure
- Richer data transformation capabilities for all MELT types (e.g. parse logs, normalize attributes, enrich with metadata)
- To drop or obfuscate sensitive data before it leaves your network for compliance requirements
- To process higher volumes of data, whereby gateway is cheaper than cloud rules
Learn more about the gateway in Gateway overview.
Pipeline Control cloud rules
Cloud rules run in New Relic's infrastructure after data reaches New Relic endpoints:
- Process data after it reaches New Relic (no infrastructure management required)
- Account-specific rules (each rule applies to a single New Relic account)
- NRQL-based drop rules for filtering data and attributes
- No infrastructure setup or management required
- Charged based on data dropped (7.5 GB per aCCU) plus billions of events scanned (2.237 BSE per aCCU)
Cloud rules are best when:
- You want simple drop rules without managing gateway infrastructure
- Your processing requirements are modest and don't justify a full-scale gateway deployment
- Lower volume filtering as a % of your total ingest where cloud rules pricing may be more cost-effective
Learn more: Create cloud rules
Using both together
You can also use gateway and cloud rules simultaneously:
- Telemetry sources (e.g. agents) can be configured differently to route through your gateway or not
- Defense-in-depth strategy ensures sensitive data is absolutely dropped
- Convenient to manage some rules via cloud API/UI while using gateway for primary processing
Comparison table
| Feature | Gateway Rules | Cloud Rules |
|---|---|---|
| Processing location | Your infrastructure (Kubernetes) | New Relic cloud |
| When it processes | Before data leaves your network | After data reaches New Relic |
| Setup required | Yes (Kubernetes cluster, Helm chart) | No |
| Configuration method | YAML or UI | UI or NerdGraph API (NRQL-based) |
| Rule scope | Organization-wide | Account-specific |
| Capabilities | Transform, drop data/attributes, rate sample | Drop data/attributes |
| Supported data types | Metrics, Events, Logs, Traces | Metrics, Events, Logs, Traces |
| Best for | High-volume processing, transformations, egress cost reduction, compliance | Simple drop rules, low volume, no infrastructure management |
For detailed pricing information and cost calculations, see Pipeline Control costs.
Prerequisites
For cloud rules
- A New Relic account in a role with RBAC permissions to manage Pipeline Control cloud rules
- Understanding of NRQL to create filtering rules
For gateway
- A New Relic account with Organization Product Admin or Organization Manager permissions
important
The Organization Product Admin or Organization Manager user roles are required to:
- Install gateway
- Add clusters to a gateway
- Deploy a gateway configuration
- View gateway rules
- Familiarity with Kubernetes operations and management
- Kubernetes cluster admin access to deploy and manage resources within your cluster
- Permissions to create and manage private DNS hostnames within your network
- Permissions to create and manage DNS records
- Helm installed locally
Importante
- Pipeline Control rules only apply to data that arrives from the moment you create the rule, they don't delete data that's already been ingested.
- When creating rules, you are responsible for ensuring that the rules accurately identify and discard the data that meets the conditions that you have established. You are also responsible for monitoring the rule, as well as the data you disclose to New Relic.
Next step
Choose your approach based on your needs:
- Start with gateway: Set up and deploy Pipeline Control gateway
- Start with cloud rules: Create pipeline rules