Setting up Tensorflow for Unity’s ML-Agents

Setting up Tensorflow for use with Unity

These instructions are an extract from A Beginner’s Guide to Machine Learning with Unity.

Note: Unity’s ML-Agents with TensorFlow plugin is an experimental system in early beta (probably alpha … and what’s before alpha??) stages.  The processes in this document are likely to change.  If you notice something different in the setup please share with me and other students in the course via the Discussion Board for this lecture.

And keep in mind while you are setting up the Tensorflow/Unity environment on your machine:

What doesn’t kill you will only make you stronger!!

Getting the Starter Files

This following package is for use when you want to start a new ML-Agent package. Note however that there is also a version of these following files in the ml-agents download from GitHub used in the next section when setting up the Python environment.

1. Download the Starter Package from

2. Create a new Unity Project (you must be using Unity 2017.1 or later).

3. Drag and drop the package, downloaded in 1. into the Asset Folder.  Once unpacked you will have a directory structure as shown here.

4. Close Unity before Installing Python.

Install the Python API’s

First, download the Unity plugin master files from:  Put this entire folder on your computer where you will be storing the files on a permanent basis (this means that the Desktop is NOT a good location for them!)  We will come back to this folder later.

In order to train an agent within the framework, you will need to install Python 2 or 3, and the dependencies below.

  • Jupyter
  • Matplotlib
  • numpy
  • Pillow
  • docopt (Training)
  • TensorFlow (1.0+) (Training)

The installation is different depending if you are using Windows or Mac.  Instructions for both operating systems can be found below.

NOTE:  Software and plugin versions are going to update over time.  To attempt to give these documents some kind of longevity I’ve tried to show you where the version numbers of things may change and that you will have to get the best version to used based on what Tensorflow is using.

On the 10 Feburary, 2018 Tensorflow requires CUDA version 9.0 (not 9.1) and cuDNN v7.0.5 (Dec 5, 2017) for CUDA 9.0. – Please let me know if you notice this has changed when you are doing an install!! 😀


1. Download the NVIDIA CUDA Toolkit  from here:
"The NVIDIA® CUDA® Toolkit provides a development environment for creating high performance GPU-accelerated applications." - NVIDIA 
This image shows where to get 8.0 but you will need to get the one that is currently appropriate for Tensorflow.

CUDA Download Screen

2. Run the installer and select the Express option.

CUDA Installer

3. Visit to download the NVIDIA Deep Neural Network Library.  You will need to create a free account and login to access it.

4. Download the Library for Windows 10 for the CUDA release version you are using.

5. Unzip the downloaded file.

6. Copy the contents of the zip file, the bin, include and lib folders into the CUDA install folder.  By default the install folder will be c:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v8.0 [at least that’s where version 8 will be, other versions will be in a folder with their number on it]

You will find that the install directory already has these folders.  Just copy the new ones over the top by dragging and dropping from the unzipped folder into the install director.

7. Modify the environment variables and path settings by opening the environment variable dialog.  To get to the right place type “Control Panel” into the search bar.  Hit Enter when it comes up highlighted.
When the Control Panel opens, type “env” into the search.  Click on “Edit the system environment variables” when the link appears.

8. In the System Properties dialog that opens, click on the “Environment Variables” button.

9. To the “System Variables” add a new one with the settings
Variable name: CUDA_HOME
Variable value: C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v8.0


10. Locate the Path variable in the System Variables.  Select it and click on the Edit button.

11. Use the New Button to add the following two new paths:

C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v8.0\extras\CUPTI\libx64 and

C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v8.0\lib\x64


12.  Click the okay buttons on each dialog until you’ve closed it all down.

13.  Still with me?  Patience is a virtue. 🙂

14.  Now its time to install the python environment, Anaconda.  Visit and download the 64 Bit Python 3.6 version.

15. Run the installer and select all the default options.

16. Test if it has installed by opening a command prompt and typing int:
conda list

To get a command prompt type “cmd” in the search and the Enter key.

Conda Install Test

If you don’t get a list but rather a nasty message more like this:

You’ll need to run the following THREE commands by typing them in the command prompt.  Each may take a little while to run so wait until the prompt comes back up before typing the next in.  Type them one at a time and hit enter.  It’s probably even easier if you but and paste them.

cd %UserProfile%
powershell -command "& { (New-Object Net.WebClient).DownloadFile('', 'mc3.exe') }"
start /wait "" mc3.exe /InstallationType=JustMe /AddToPath=0 /RegisterPython=0 /NoRegistry=0 /S /D=%UserProfile%\anaconda3
conda install -y anaconda=5.0.1 conda-build _ipyw_jlab_nb_ext_conf

After this try running “conda list” again to check the install.  If you got this issue, it is a problem known to Anaconda who say it will be fixed in the next release.

To check that the environment variables for this are now setup, close the command window and open a new one.  Type “conda list” again.  If you get the error from before saying it’s not recognised you should add the following environment variables to your path, like we did in step 11.  Click OK until all these dialogs are closed again.

c:\Program Files\Anaconda3
c:\Program files\Anaconda3\Scripts

17.  Close the command prompt window and open it again.  Type in “conda list” and hit enter.  This will test that conda can now be found.  It might take it’s time but it should give you a list like the one shown in the figure above called “Conda Install Test”.

18.  We are now ready to install Tensorflow.  Open a command prompt and type in:
conda create -n tensorflow-gpu python=3.5.2

19  You have just created an environment called tensorflow-gpu.  To activate it type:
activate tensorflow-gpu

20. And finally we install tensorflow into this new environment with the command:
pip install tensorflow-gpu

21. Now test that python runs by typing in:

You’ll get a prompt with >>> on it which means you are inside the python environment.

At the python prompt type:

import tensorflow as tf

If you get an error like this:

Tensorflow has been updated to use a different version of CUDA.  In this case I was installing Tensorflow with CUDA v8.0 but it says it wants version 9.  If this happens to you then its back to the CUDA downloads to get the version that Tensorflow has been updated to use and run through the installation process again.  You will also need to get the cudaNN files for the same CUDA version and copy the bin, include and lib folders across as shown in step 6.


If tensorflow imports you won’t see any error messages.

Type COMMAND+Z and hit Enter to quit.

22. At this stage it probably seems like a lifetime ago that we downloaded the ml-agents folder from github.  It was the first thing we downloaded and put aside for later use.  I’ve put mine in:

We need to get into that python subfolder from the command line.  So, open the command prompt with administrator privileges.  To do this, search for the command prompt with “cmd” and then right click on it and choose “Run as Administrator”.

23. In the command prompt window change into the python folder.  Use the

cd foldername

where foldername is the location of your python directory inside the ml-agents.


24. To complete the install for ml-agents, On the command line run:
pip install .

This is where you may run into another versioning issue.  When I was installing I ended up with the latest version of Tensorflow which the ml-agent’s code wasn’t yet using.  So I got this error:

This tells us that ml-agents doesn’t use the version of Tensorflow that we are trying to use) but that it needs version 1.4.1.  So you need to install the correct version including the version number like this:

pip install tensorflow==1.4.1

[that’s a double equals]

However this will fail in this instance because there doesn’t seem to be such a thing as tensorflow 1.4.1.   There is an obvious error somewhere in all this….

I got around it by opening the file called requirements in the python folder of ml-agents with Notepad.

At the time of writing this, inside the file it says:

I changed where is says tensorflow==1.4.1 to tensorflow==1.4.0 and saved the file.

Then I made sure version 1.4.0 was installed with:

pip install tensorflow==1.4.0

then again tried to install the ml-agents specifics with:

pip install .

and it worked!!

25.  Is it working?  Type into the command prompt:

jupyter notebook

Hit enter and leave it.   The window is not running jupyter.  There will not be a command prompt.  Leave this window open.

Now go to your web-browser.  Navigate to:


If you get something that looks like this:

then you are good to go.  There is a little web server now running on your machine that will help us interact between Unity and Tensorflow.


The following are commands for installing Python to overcome any permission issue you might encounter.  If you are installing on a machine that is not your own seek assistance from a person with administration privileges.  The use of sudo gives the installer permission to do what it needs to without running into security issues that might otherwise stop a third-party app installing things onto your machine.  I don’t recommend you do this on a production machine.

  1. Open a terminal window.
  2. Type in:sudo easy-install pipIt should go something like this:

    Pip is the python package management system.

  3. Navigate to the python directory of your ml-agents folder in your terminal. e.g.
    cd /ml-agents-master/python
    Your location will depend on where you unzipped and placed the ml-agents GitHub download.
  4. In the terminal window type:
    sudo pip install .
    to start installing all the required packages.  If this installs everything then “lucky you”, but its more likely that you are going to get error here an there.  To resolve this there’s a bit of back and forth between installing and uninstalling things.So if you encountered errors, read on.  Otherwise you are done!
  5. You are aiming to run all the following to install the packages separately and in this order (this is the critical install list):
    sudo pip install jupyter
    sudo pip install matplotlib
    sudo pip install numpy
    sudo pip install pillow
    sudo pip install docopt
    sudo pip install tensorflow

    However the installation of one might cause a conflict with an old version of another.  If you run

    sudo pip install jupyter

    and get an error that looks like this:

    you will need to uninstall any previous versions of share with

    sudo pip uninstall share

    and then reinstall with

    sudo pip install share

    For me to install jupyter on my computer I had to run:

    sudo pip uninstall share
    sudo pip install share
    sudo pip install jupyter

    After this comes matplotlib, I got an error that an operation was not permitted on the package man.

    So I ended up performing these operations to get matplotlib installed.

    sudo pip uninstall man
    sudo pip install man
    sudo pip install matplotlib

    This process will continue as you make your way down the critical install list.

    If you get the message:

    You will need to run your command with -H, e.g.

    sudo -H pip uninstall numpy

    Make your way through the critical install list installing and uninstalling as required.  To help you I’ve added a transcript of my journey when I first installed TensorFlow on my Mac as a PDF attached to this lecture.  At worst, if you just type in everything I did in the same order, it should end up installed. 😀

    6.  To test it is installed correctly, open a terminal window and change to the python folder of the ml-agents master file Unity download that we did right at the beginning.

    From that folder run

    jupyter notebook

    This will start a web service for interaction between Unity and Tensorflow.  It may also open your web browser and show you the page belonging to your project.

    If it doesn’t, open a web browser and enter the URL:


    It will show you this:

    And if you can see that, you are ready to roll!