Python and Machine Learning

Starting in the New Relic Python Agent v9.1.0, New Relic now enables monitoring for Machine Learning Models. These monitored ML models can be found in the APM in the Models section.

Setup

Tip

ML model metrics are available for Python agent version 9.1.0 and higher but are disabled by default. To change this configuration, check out our documentation.

ML settings can be found here.

To change the default ML harvest size of 100000 every 5 seconds, either set event_harvest_config.harvest_limits.ml_event_data in your newrelic.ini file to the desired value or set the environment variable NEW_RELIC_ML_INSIGHTS_EVENTS_MAX_SAMPLES_STORED to the desired value:

bash

$export NEW_RELIC_ML_INSIGHTS_EVENTS_MAX_SAMPLES_STORED=${desired_value}

Currently Instrumented Machine Learning Frameworks

ML Library	Version Available
scikit learn	9.1.0

Machine Learning APIs

Two new APIs exist to customize the ML instrumentation experience:

Record custom ML events: record_ml_event
Manually instrument an ML model: wrap_mlmodel

Ensure data privacy

Caution

You control what log data is sent to New Relic, so be sure to follow your organization's security guidelines to mask, obfuscate, or prevent sending personal identifiable information (PII), protected health information (PHI), or any other sensitive data.

You can also enable or disable raw inference values to be sent depending your desired privacy settings here.

Features and Functionality

Models can be seen in a separate Models category in the All Entities view:

Screen capture showing ML Models in APM All Entries view.

A summary of models in the Models view:

Screen capture showing ML Models in APM Models view.

Within the model summary, an overall model performance view can be seen as well as a Prediction Distribution.

Screen capture showing Label Propagation ML Model Summary view.