1. 1. Introduction
    1. 1.1. Prerequisites
    2. 1.2. Create Azure Machine Learning workspace
    3. 1.3. Create an Azure Machine Learning compute instance
    4. 1.4. Install the .NET 3.1 SDK
    5. 1.5. Install .NET Interactive
    6. 1.6. Add .NET global tools directory to PATH
    7. 1.7. Create new .NET Jupyter Notebook
    8. 1.8. Conclusion
    9. 1.9. Sources

Use .NET Interactive to run .NET code in Jupyter Notebooks on an Azure Machine Learning compute instance

Introduction

In this post, I’ll go over how you can run .NET code in Jupyter Notebooks using .NET Interactive and Azure Machine Learning compute instances.

Jupyter Notebooks are an graphical interactive computing environment used in disciplines like data science and education. When it comes to prototyping or learning scenarios, they can help users see near real-time what their code is doing. The interactive computing protocol used by Jupyter Notebooks is extensible and as a result, you’re able to run a variety of languages inside of this environment via kernels. Kernels are processes that take the input from the user, execute the code, and return the output for display and further processing. Using .NET Interactive, a .NET Core global tool, that among its many features provides a kernel for Jupyter Notebooks to run .NET (C#/F#) and PowerShell code.

Although you can install and run Jupyter Notebooks on your local computer, services like Azure Machine Learning provide single VMs known as compute instances which have Jupyter Notebooks as well as many popular data science libraries and development tools like Docker preinstalled. Therefore, installing .NET Interactive on one of these VMs is a seamless experience. Additionally, Azure Machine Learning gives you the ability to share computing resources with teams of various sizes.

Let’s get started!

Prerequisites

Create Azure Machine Learning workspace

A workspace organizes all of you Azure Machine Learning resources and assets such as compute instances in a single place. In order to use Azure Machine Learning, you have to create a workspace.

There’s multiple ways to create a workspace. For this writeup, I use the Azure Portal.

Navigate to portal.azure.com

In the portal, select Create a resource.

Create new resource Azure Portal

From the resource list, select AI + Machine Learning > Machine Learning

Create Azure Machine Learning Resource

Fill in the form and select Review + create.

Create Azure Machine Learning workspace

Review your information before creating the workspace and select Create. Deployment takes a few minutes. As part of your workspace deployment, additional Azure resources are created such as Azure Container Registry, Azure Storage account, Azure Application Insights, and Azure KeyVault.

For more information on workspaces, see the Azure Machine Learning workspace documentation.

Create an Azure Machine Learning compute instance

As mentioned earlier, an Azure Machine Learning compute instance is a single VM that comes with a variety of development tools and libraries commonly used for data science and machine learning workflows preinstalled.

Navigate to Azure Machine Learning studio. The studio is a web interface for managing your Azure Machine Learning resources such as data, experiments, models, and compute.

In studio, select Create new > Compute instance to create a new compute instance.

Create new Azure Machine Learning compute instance

Provide a name for your instance and select Create. You can also customize the size of your VM and whether you want it to be GPU enabled. In this case, I just selected the preselected option.

Provision Azure Machine Learning compute instance

Once your compute instance is provisioned, select the Jupyter link to launch Jupyter Notebooks.

Launch Jupyter Notebooks

For more information on compute instances, see the Azure Machine Learning compute instance documentation.

Install the .NET 3.1 SDK

At the time of this writing, the image used by compute instances is Ubuntu 16.04. Therefore, we’ll be installing the Linux version of the .NET Core SDK.

Inside the Jupyter Notebooks, select New > Terminal to create a new terminal.

image

In the terminal, add the Microsoft package signing key to your list of trusted keys.

1
2
wget https://packages.microsoft.com/config/ubuntu/16.04/packages-microsoft-prod.deb -O packages-microsoft-prod.deb
sudo dpkg -i packages-microsoft-prod.deb

Install the .NET Core 3.1 SDK with the following command:

1
2
3
4
sudo apt-get update; \
sudo apt-get install -y apt-transport-https && \
sudo apt-get update && \
sudo apt-get install -y dotnet-sdk-3.1

Note that as of the time of this writing, .NET Interactive requires .NET Core 3.1.

You can also find these installation instructions in the .NET Core installation documentation.

Install .NET Interactive

Start off by checking which kernels are installed. In the terminal, enter the following command.

1
jupyter kernelspec list

The output should look something like the output below

1
2
3
4
Available kernels:
python3 /anaconda/envs/azureml_py36/share/jupyter/kernels/python3
ir /usr/local/share/jupyter/kernels/ir
python3-azureml /usr/local/share/jupyter/kernels/python3-azureml

Then, install the .NET Interactive global tool

1
dotnet tool install -g --add-source "https://dotnet.myget.org/F/dotnet-try/api/v3/index.json" Microsoft.dotnet-interactive

Use the .NET Interactive tool to install the .NET and powershell kernels.

1
dotnet interactive jupyter install

Finally, run the following command to confirm that the kernels have been installed.

1
jupyter kernelspec list

In the output you should see the C#, F#, and PowerShell kernels listed

1
2
3
4
5
6
7
Available kernels:
.net-csharp /home/azureuser/.local/share/jupyter/kernels/.net-csharp
.net-fsharp /home/azureuser/.local/share/jupyter/kernels/.net-fsharp
.net-powershell /home/azureuser/.local/share/jupyter/kernels/.net-powershell
python3 /anaconda/envs/azureml_py36/share/jupyter/kernels/python3
ir /usr/local/share/jupyter/kernels/ir
python3-azureml /usr/local/share/jupyter/kernels/python3-azureml

Add .NET global tools directory to PATH

The Jupyter Notebook server is managed by a systemd service. Although during installation .NET Interactive is added to your PATH environment variable, it’s not added to the environment file /etc/environment which is used by systemd. Therefore, in order to run .NET Interactive, you have to add the .NET global tools directory to that file.

The contents of the file should look similar to the one below

1
2
PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games"
AML_CloudName=AzureCloud

Open the /etc/environment using your preferred text editor (vi/nano) and replace the PATH with the following content:

1
PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/home/azureuser/.dotnet/tools"

Save your changes and restart the Jupyter Server. In the terminal, run the following command.

1
sudo service jupyter restart

This will close your terminal session since the notebook server has been restarted.

Close all Jupyter Notebook windows.

Create new .NET Jupyter Notebook

From Azure Machine Learning studio, launch Jupyter Notebooks again by selecting the Jupyter link.

Then, in Jupyter Notebooks select New and create a new notebook (C#/F#/PowerShell). In this case, I created an F# notebook.

Once the kernel is ready, enter code into the first cell and run it.

FSharp Jupyter Notebook running in AML compute instance

Congratulations! You should now be able to run .NET and PowerShell code inside of your Azure Machine Learning compute instance.

Conclusion

In this post, I showed how you can run .NET code in Jupyter Notebooks on an Azure Machine Learning compute instance with the help of .NET Interactive. This enables you to interactively prototype solutions remotely while still having control over your environment and dependencies. Now that you have .NET interactive setup, compute instances also give you the option of using JupyterLab. Feel free to tinker with the different environments and see which one works best for you. Happy coding!

Sources