Skip to main content

Experiment Tracking

Using MLflow in Azure ML to track experiments

Overview​

Azure Machine Learning integrates seamlessly with MLflow to track experiments, runs, parameters, and metrics. During experimentation, MLflow can automatically log key metadata to the Azure ML workspace.

Azure Machine Learning tracks training jobs in what MLflow calls runs. Use runs to capture all the processing that your job performs.

Each run captures:

  • Parameters, metrics, and logs
  • Artifacts such as models and output files
  • Source code and environment details

You can visualize and compare runs in Azure ML Studio or the MLflow UI, making it easier to select the best-performing model.

Once a run produces a satisfactory result, the trained model can be registered directly from the MLflow run, linking experiment tracking with the Azure ML model registry for versioning and deployment.

Model and pipeline tracking example​

Description

Training and tracking an XGBoost classifier with MLflow

This notebook example, adapted for use in AI Platform, demonstrates how to train and track models locally using MLflow for experimentation and reproducibility.

This notebook focuses on:

  • Exploratory data analysis and preprocessing
  • Training an XGBoost classifier
  • Using MLflow to:
    • Track experiments (mlflow.set_experiment, start_run)
    • Automatically log model parameters and metrics (mlflow.xgboost.autolog)
    • Manually log additional metrics (accuracy, recall)
    • Explore logged runs and artifacts
    • Log and load models (including pipeline models with preprocessing)
    • Use model signatures

It also includes advanced MLflow usage like:

  • Logging a pipeline with a custom encoder
  • Logging with signatures
  • Comparing autologged vs. manually logged models

Instructions

  1. Go to the repository folder with the notebook example and associated files and folders:

  2. Copy the files and folders to your development environment in AI platform.

  3. Before running the xgboost_classification_mlflow.ipynb notebook to deploy the pipeline, ensure your environment is properly set up and the few configuration values (like workspace and compute names) are filled in.

  4. Follow the instructions in the notebook and run the code cells.

Further resources​

You can find documentation and examples for tracking experiments using MLflow in Azure ML in the following links:

Introductory documentation:

How to use MLflow for tracking experiments and runs in Azure Machine Learning workspaces: