Python environments

Plenty of MagAO-X software is written in Python, so we need a consistent environment. We use the mamba distribution of the conda package manager (c.f. Anaconda) to manage it. (A mamba is a fast snake, and the mamba install command is orders of magnitude faster at solving environment installation requirements than plain old conda.)

Environment specifications are tracked in the same git repository as MagAO-X software, under the setup/ folder.

conda_env_base.yml stores just the top-level dependencies (i.e. not everything they depend on) without versions. This makes it easier to track what we really depend on in case we need to help conda along with failing dependency resolution.

The files named variations on conda_env_pinned_*.yml reflect the actual set of packages installed in the environment. This lets us recreate/update (nearly) identical environments on AOC/RTC/ICC. The suffix allows us to track different pinned packages for different instruction set architectures.

Adding a package

If there’s a package you need for your work on the instrument machines, it’s a good idea to make it part of the environment so a rebuild is sure to install it.

Installation

We use the mamba tool and require root access to install packages, so become root first:

you$ sudo -i
root$

Install as normal, preferring conda packages if possible.

root$ mamba install foopkg
# *only* if it's not found:
root$ pip install foopkg

Now when you do conda env export, you should see an entry for your new package. Example:

$ conda env export
name: base
channels:
  - conda-forge
  - defaults
dependencies:
[...]
  - foopkg=0.1
[...]

If it was installed via pip, the package will be in a sub-list under the pip: heading.

Updating the environment template

To add it to the template, first pull any changes:

cd /opt/MagAOX/source/MagAOX
git pull
# resolve any conflicts

Then open /opt/MagAOX/source/MagAOX/setup/conda_env_base.yml. You’ll see something like what conda env export output, but without the version numbers. Add your package name (but not version, unless you know what you’re doing) to the list, being careful to put it under the pip: heading if that’s how it was installed.

Updating the list of pinned packages

Now, you need to update the list of pinned packages and versions in /opt/MagAOX/source/MagAOX/setup/conda_env_pinned_$ARCH.yml (where $ARCH is your processor architecture) by exporting the current packages and versions from the system where you just installed a new package.

$ conda env export > /opt/MagAOX/source/MagAOX/setup/conda_env_pinned_$(uname -i).yml

This updates the versioned file in the MagAO-X source, and you can use git diff to see what has changed.

If you install on ARM this won’t update the pinned packages on Intel/AMD, and vice versa. To keep a pinned environment for the other processor architecture, you will need to install the package there and repeat the export. (Reproducing the same environment exactly is impossible on a different processor architecture for any nontrivial set of packages.)

Storing in version control

Follow these steps to commit the updates to these files to version control and push them to the central copy.

$ cd /opt/MagAOX/source/MagAOX/setup
$ git add conda_env_pinned_*.yml conda_env_base.yml
$ git commit -m "Added Python package foopkg to conda envs"
$ git push

Replicating across all the machines

If you just performed these steps on AOC, now you need to connect to ICC and RTC and run the following steps:

$ cd /opt/MagAOX/source/MagAOX/setup
$ git pull
$ conda env update -f /opt/MagAOX/source/MagAOX/setup/conda_env_pinned_$(uname -i).yml

(Or, if you made the change another one of the machines, just run the above steps on the two other ones.)

Updating `conda`

If there are updates to conda itself, it’ll probably tell you. You can run conda update -n base -c defaults conda to update it, but be sure to follow the steps beginning at Updating the pinned packages to record the upgrade.

Performing a fresh conda install/upgrade

Generally only something to do if things are totally messed up, there’s a new version of the Python interpreter itself, or both.

Move /opt/conda out of the way (i.e. mv /opt/conda /opt/conda.bak)
Edit /opt/MagAOX/source/MagAOX/setup/install_python.sh and change MINICONDA_VERSION="X-pyXX_X.Y.Z" appropriately, and commit/push to version control.
Run bash /opt/MagAOX/source/MagAOX/setup/install_python.sh to download and install the new conda to /opt/conda with appropriate permissions

At this point you should log out and back in to reset any environment variables that were set by the old conda.

If the Python version hasn’t increased

In most cases, the version of Python shipped with Miniconda hasn’t changed.

Run bash /opt/MagAOX/source/MagAOX/setup/configure_python.sh

If the Python version has changed

The default configure_python.sh would try to restore the pinned versions from conda_env_pinned_$ARCH.yml, but would fail because of the Python version mismatch. Instead, you need to create the environment from conda_env_base.yml and update conda_env_pinned_$ARCH.yml yourself

conda env update -f /opt/MagAOX/source/MagAOX/setup/conda_env_base.yml
conda env export > /opt/MagAOX/source/MagAOX/setup/conda_env_pinned_$(uname -i).yml

$ cd /opt/MagAOX/source/MagAOX/setup
$ git add conda_env_pinned_*.yml
$ git commit -m "Updated pinned packages for conda upgrade"
$ git push

You will also need to rerun some of the files in /opt/MagAOX/source/MagAOX/setup/steps that install Python packages into the environment. A (possibly incomplete) list:

cd /opt/MagAOX/source/MagAOX/setup/steps && \
bash install_purepyindi.sh && \
sudo /opt/conda/bin/pip install /opt/MagAOX/source/milk/src/ImageStreamIO/ && \
bash install_magpyx.sh && \
bash install_sup.sh

Replicate across all the machines

SSH to the other machines and:

Move /opt/conda out of the way (i.e. sudo mv /opt/conda /opt/conda.bak)
Update the MagAO-X source: cd /opt/MagAOX/source/MagAOX && git pull
Install Python via miniconda: bash /opt/MagAOX/source/MagAOX/setup/install_python.sh
Configure Python via conda environment files: bash /opt/MagAOX/source/MagAOX/setup/configure_python.sh

Ensure all our custom packages get installed:

cd /opt/MagAOX/source/MagAOX/setup/steps && \
bash install_purepyindi.sh && \
sudo /opt/conda/bin/pip install /opt/MagAOX/source/milk/src/ImageStreamIO/ && \
bash install_magpyx.sh && \
bash install_sup.sh