Welcome to the Comprehensive Adaptive Optics and Coronagraph Test Instrument (CACTI). Here we’ll show what you need to do for running CACTI remotely using the Virtual Machine, which operates very similarly to MagAO-X with some tweaks.
Sections TODO: Config, Python Scripts
A few things to know for running CACTI safely:
Do not power up the bench if there are thunderstorms. Ideally unplug the bench from the UPS in case of thunderstorms.
Do not power up the deformable mirror if humidity is above 15%.
Check the humidity reading about once every 30 minutes.
If over the course of several days the humidity is slowly rising check the following things:
The airflow from the hose to the DM box
The dessicant attached to the air flow pipes.
If restarting or shutting down with
xctrl, first zero the DM and then release it.
Remote operation with a virtual machine (VM)¶
Create a virtual machine to operate CACTI remotely. The VM handles the control GUIs and the cameras.
You will need an SSH key in ECDSA or ED25519 format, and a user account in group
exao0(a.k.a. TIC) with that key authorized
Connect to the VM by running
vagrant sshin the MagAOX folder. Your shell prompt will change to:
Verify connectivity to
exao0/TIC by connecting to it with ssh (entering
yesif prompted to verify the host key):
$ vagrant ssh Last login: Fri Jun 11 04:30:39 2021 from 10.0.2.2 [vagrant@centos7 ~]$ ssh tic The authenticity of host 'exao0.as.arizona.edu (220.127.116.11)' can't be established. ECDSA key fingerprint is SHA256:7a6jFVxlsQG9X5ZVPX3IHTxmT2c+qKEcStlxbZp4I4g. Are you sure you want to continue connecting (yes/no/[fingerprint])? yes Warning: Permanently added 'exao0.as.arizona.edu,18.104.22.168' (ECDSA) to the list of known hosts. Last login: Wed Jun 23 12:27:16 2021 from example.com exao0:~ you$ logout Connection to exao0.as.arizona.edu closed. [vagrant@centos7 ~]$
To get from the vagrant prompt back to your normal prompt, use
logout. Note that the virtual machine is still running in the background, and you can reconnect with
Navigate to the
vm/folder created within your
MagAOX/folder. You need to replace the MagAO-X
configfolders with CACTI’s
In terminal, go to the
$ cd vm/
Delete the original MagAOX config and calib folders:
$ rm -r calib config
Clone the testbed_calib and testbed_config repositories from GitHub into
$ git clone https://github.com/magao-x/testbed_calib calib $ git clone https://github.com/magao-x/testbed_config config
At this point, you should be ready to have regular virtual machine startup for CACTI.
These directions are not guaranteed to work unless your virtual machine setup is adjusted for CACTI.
In terminal, go to the MagAOX directory:
$ cd ~/your/path/to/MagAOX/
Start up the virtual machine:
$ vagrant up $ vagrant ssh
One you are in the virtual machine, bring up xctrl:
$ xctrl startup
From here, this should start up the following tmux sessions:
vm_tic_indi toc_tic_milkzmq toc_tic_jupyterlab
Your VM is now connected to the testbed control computer and you can go about your work.
These commands allow you to turn on various GUIs through the VM.
Power Control GUI. Allows you to turn on and off the cameras, lasers, DM, etc.
To operate, use:
$ pwrGUI &
To power on a device, slide the bar from left to right. Simiarly, to power off a device, slide the bar from right to left.
Real Time Image Viewer GUI. Allows you to view livestreams of the camera. A detailed
rtimv can be found in the Cameras
There is a little bit of preparation work to do before running
Power on the cameras you want to use in
rtimvcan see them. You can do this with:
$ milkzmqClient -p 9000 localhost <shmim-1> <shmim-2> ... &
<shmim>is a device (camera, DM channels). Load up all the cameras you want to use. For example,:$ milkzmqClient -p 9000 localhost camlgsfp camzwfs camtip &
will initialize the cameras
Tip: If you forgot the
&at the end of the command and the command line is hanging, you can press
ctrl + zto go back to the command line and then enter
milkzmqClientin the background.
milkzmqClientcan be a little slow at times. If the command line is hanging and you used
&, try hitting
enterto see if you get back to command line.
Here is an example of
milkzmqClientsuccessfully loading for
camlgsfp. Note that the command line entry that says
N: 2is automatic. The last 4 lines are proof that
camlgsfpis connected in
milkzmqClient.[vagrant@centos7 ~]$ milkzmqClient -p 9000 localhost camlgsfp &  6332 [vagrant@centos7 ~]$ N: 2 camlgsfp milkzmqClient: Beginning receive at tcp://localhost:9000 for camlgsfp milkzmqClient: Connected to camlgsfp [ MILK_SHM_DIR ] '/milk/shm' [ MILK_SHM_DIR ] '/milk/shm' [ MILK_SHM_DIR ] '/milk/shm'
Now you can run
rtimv. There’s two ways you can do this.
To see the camera GUI with the INDI connected display, use:
$ rtimv -c rtimv_<camera-name>.conf &
<camera-name>is the name of the camera. For example if using
$ rtimv -c rtimv_camlgsfp.conf &
.conffile for this
<camera-name>must exist for this to run. If it’s not present, contact Jared.
If you are not interested in the INDI connected display, use:
$ rtimv <camera-name> &
and you should get the
rtimvGUI with no notes on the sides.
Troubleshooting rtimv: Sometimes you do everything right and you are rewarded with a blank
rtimv window. Here are some steps to take for resetting the
milkzmqClientjobs. At the vm command line, enter
jobsand you will see all the jobs running with a number associated with it.
[vagrant@centos7 ~]$ jobs  Running pwrGUI & - Running milkzmqClient -p 9000 localhost camlgsfp & + Running rtimv -c rtimv_camlgsfp.conf &
To stop a job, enter
nis the number. In this example, you need to stop the
milkzmqClienton 2 and the
[vagrant@centos7 ~]$ kill %2 [vagrant@centos7 ~]$ milkzmqClient: Disconnected from camlgsfp - Done milkzmqClient -p 9000 localhost camlgsfp [vagrant@centos7 ~]$ jobs - Running pwrGUI & + Running rtimv -c rtimv_camlgsfp.conf & [vagrant@centos7 ~]$ kill %3 [vagrant@centos7 ~]$ jobs - Running pwrGUI & + Terminated rtimv -c rtimv_camlgsfp.conf [vagrant@centos7 ~]$ jobs + Running pwrGUI &
[vagrant@centos7 ~]$ milkzmqClient -p 9000 localhost camlgsfp &
[vagrant@centos7 ~]$ xctrl restart vm_tic_milkzmq Waiting for tmux session for vm_tic_milkzmq to exit... Waiting for tmux session for vm_tic_milkzmq to exit... Ended tmux session for vm_tic_milkzmq Session vm_tic_milkzmq does not exist Created tmux session for vm_tic_milkzmq Executed in vm_tic_milkzmq session: '/opt/MagAOX/bin/sshDigger -n vm_tic_milkzmq' [vagrant@centos7 ~]$ milkzmqClient: Connected to camlgsfp [ MILK_SHM_DIR ] '/milk/shm' [ MILK_SHM_DIR ] '/milk/shm' [ MILK_SHM_DIR ] '/milk/shm'
Here we can see at the last 4 lines that
camlgsfpis restarted in
rtimvlike in the previous directions. The GUI should be outputting properly now.
Region of Interest GUI for
rtimv. A detailed explanation for
roiGUI functions can be found
in the Cameras section.
To operate, use:
$ roiGUI <camera-name> &
<camera-name> is the camera you want to edit the ROI for
DM Control GUI. Controls the 1K DM. Apply flats, clear channels, release DM.
IMPORTANT: Before powering the DM in
pwrGUI and operating
dmCtrlGUI, you must verify the
1K DM humidity is below 15%. See Humidity Sensing for instructions on checking the humidity.
To operate, use:
$ dmCtrlGUI dmkilo &
This will open a GUI window.
Initialize the DM by clicking on the
initializeat the top right. Sometimes, the GUI starts pre-initialized.
To load a DM flat, choose which file you’d like from the top drop down menu.
set_flatto load the flat.
When you are done using the 1K DM, please click on
releasebefore powering it down in
Commands run on
To startup exao0, open a new terminal and ssh with your account into exao0. Always run it in xsup.
$ ssh exao0 $ su xsup
Startup and shutdown with
From here, you can start running the various processes with xctrl.
The Arduino humidity sensor has been moved from
exao0. The humidity
sensor is connected via USB to
/dev/ttyACM0 which can be monitored with
provided that you are in the
dialout user group on
If you are not in the
dialout group, get someone to do
sudo gpasswd -a USERNAME dialout
and log in again.
Open a separate terminal and log into
exao0 with your account (not
If you are starting from a fresh (re)boot:
$ screen /dev/ttyACM0
If the session already exists (i.e. was disconnected without killing it):
$ screen -rd
The screen should now show a bunch of environmental monitoring information that looks like this:
Humidity: 10.70 % Temperature: 22.80 *C 73.04 *F Heat index: 21.41 *C 70.55 *F
Please actively check the humidity levels every 30 minutes or so.
Do not operate the 1K DM if the humidity is above 15%!!
If somoene else is viewing the humidity monitor, even if they are “detached” from
you won’t be able to open it until they have killed their screen session (after reattaching if needed).
To kill (exit) the humidity monitor:
Ctrl + a, release, then “k”, then “y” to confirm.
This releases the device for other users.
Ctrl + a, release, then “d”.
This makes it easy to reattach with
Allows you to set exposure times, ROI, etc directly.
To start cursesINDI, enter it in the
exao0 terminal when in
For general use:
Enter the name of device and it will search for it.
Tip: Sometimes there are multiple versions of the device. Add “.” at the end of your device name to minimize scrolling.
Once at the list, curse over “target” in second to right hand column. Hit “e” for edit, enter a new number, and then “y” for yes.
To exit, hit
Ctrl + c.
The Eye Doctor for CACTI¶
Consult The Eye Doctor for general information.
Run eye doctor in the
exao0 terminal under
xsup. The general form of the command is:
$ dm_eye_doctor <portINDI> <dmModes> <camera-name> <psf_core_radius_pixels> <modes_to_optimize> <amplitude_search_range> --skip 1
For example, if you want to run
dm_eye_doctor for the 1K DM using the
camlgsfp camera and
correct the lower order modes, it would be:
$ dm_eye_doctor 7626 kiloModes camlgsfp 8 2...10 0.1 --skip 1
If you want to go on higher order modes, change the
$ dm_eye_doctor 7626 kiloModes camlgsfp 8 10...30 0.1 --skip 1
Once you have a DM flat that produces a nice PSF, you can save the flat with:
$ dm_eye_doctor_update_flat kilo
And it will save a new flat in the
dmCtrlGUI list at the very bottom with the date stamped on it.
To run the new flat, you need to update
dmCtrlGUI to zero the flat, select the new flat, and then
Running Python to control
There are a myriad of commands you can use to do things with
exao0, such as saving data and
running control loops.
Running Jupyter Notebook (Python)¶
To access Jupyter Notebook from
exao0, you need to ssh into
exao0 with another terminal:
$ ssh -L 9990:localhost:9999 exao0
Note: if your computer has a different access code for getting into
exao0, use that in
place of the
exao0 portion of the command above.
Once connected through ssh, you can navigate to
localhost:9990 on your internet browser. This
will open up the jupyter notebook directory page under
xsup. If a password is required, ask
someone who has access for it.
From here, you can create your own directories and jupyter notebooks to run your python code.
Saving Camera Images with
There’s two ways to save images from the cameras, either through
cursesINDI or using the
ImageStream function within the
magpyx python code. This section will cover how to use
magpyx. More information on
magpyx can be found here.
In jupyter, you need to import
$ from magpyx.utils import ImageStream
To declare a camera, set the name of the camera in
$ cam = ImageStream(<camera-name>)
From here, you can do multiple types of tasks with the camera. Commands for using
can be found in the source code. When
you collect data for the camera, it will use the settings for that camera that have been declared
If you want to get the immediate frame, you can run:
$ image = cam.grab_latest()
If you want to get a cube of images for
n number frames, you can do:
$ imagecube = cam.grab_many(n)
When you are done with the camera, please close it off:
Tips for running ``ImageStream``:
It’s generally better to leave the
ImageStreamon if you’re going to do multiple things instead of constantly opening and closing it.
If you make a change on the ROI, you will need to close and re-open it for it to work. Otherwise, a segfault and no one likes that.
If the camera isn’t collecting data, you can change the framerate in
cursesINDIand it will keep following it.