4D Air Temperature Anomaly - 1920 x 1080 Size¶
This tutorial illustrates using VCS and DV3D (two CDAT tools) to plot three dimensional air temperature anomaly data through time (the 4th dimension). This notebook differs from the "4D Air Temperature Anomaly" notebook in that it creates a video of the anomaly through time in a size that matches YouTube's preferred default size of 1920 pixels by 1080 pixels as opposed to the smaller 814 px by 606 px of the "4D Air Temperature Anomaly" notebook.
The most direct way to work with this Jupyter Notebook is to download the notebook by right clicking on the link below and chosing "Download linked file as", activating a CDAT + jupyter compatible environment, and running the notebook on its own or within a Jupyter Lab interface. If you'd like more explanation than this, see the Getting Started section below.
If you are unfamiliar with Jupyter Notebooks, they are files with an .ipynb extension that are made up of cells that can include executable code or regular text to explain what the code is doing. From Jupyter.org "The Jupyter Notebook is an open-source web application that allows you to create and share documents that contain live code, equations, visualizations and narrative text." Users can step through each cell in the notebook by putting their cursor in the cell they wish to run and either clicking on the "Run" button at the top of the page or pressing enter on the keyboard while holding down shift at the same time (shift-enter).
To download this Jupyter Notebook right click on the link and choose "Download Linked File As..." or "Save Link as...". Remember where you saved the downloaded file, which should have an .ipynb extension. (You'll need to launch the Jupyter notebook or JupyterLab instance from the location where you store the notebook file.)
Getting Started¶
This notebook uses Python 3.
If you only have Python 2 installed, the code should still work in Python 2, but if it doesn't, isn't this a good time to join the Python 3 crowd? The instructions below make it relatively painless to install a Python 3 environment.
For the code in this notebook or the stand-alone python script to work, you need to install a CDAT compatible environment. See the next cell in the notebook for details on how to create an appropriate environment and activate it. If you see three grey dots, that cell has been hidden. Just click on the dots to see the contents of the cell. (To hide the cell in JupyterLab, click on the vertical blue bar to the left of the cell.)
Conda Installation¶
We recommend using conda to install and manage environments. Conda itself can be installed either via Anaconda (the everything-but-the-kitchen-sink version) or Miniconda (the minimalist version).
I prefer Miniconda since it takes up less room on my computer and I don't mind installing the various packages I need when I need them. Miniconda (or Anaconda) will install Python on your computer. If you already have Python installed on your computer, see this helpful page. The choice of whether to install the Python 2.x or Python 3.x version of Miniconda will affect only your root environment. You can create both Python 2.x and Python 3.x environments with either version of Miniconda installed.
When installing Miniconda or Anaconda, let the installer add the conda installation of Python to your PATH environment variable. On a Mac or Linux machine, if asked "Do you wish the installer to initialize Anaconda3 by running conda init?" or something similar, we recommend saying "yes". For more details see this page.
Create CDAT Compatible Environment¶
Once you have a version of Miniconda (or Anaconda) installed, create a CDAT or Jupyter-VCDAT environment.
For CDAT:¶
- type the following at a command line prompt:
conda create -n cdat81 -c cdat/label/v81 -c conda-forge python=3.6 cdat- Once the cdat81 environment is installed, activate it by typing:
conda activate cdat81- Install JupyterLab within your cdat81 environment so you can run Jupyter notebooks:
conda install -c conda-forge jupyterlabFor additional details on creating a CDAT environment, see the following CDAT installation page.
For VCDAT and the jupyter-vcdat environment:¶
Follow these instructions these instructions for installing VCDAT 2.0 on your personal computer.
Once the jupyter-vcdat environment (and VCDAT) have been installed, activate the environment with:
conda activate jupyter-vcdatNote: if the "conda activate" command does not work, try:
source activate name_of_environmentStart JupyterLab¶
Once you have a cdat81 or jupyter-vcdat environment activated, navigate to a parent folder that contains this notebook, then type:
jupyter-labNote: you must launch JupyterLab from the highest level folder you want to be able to access. Jupyter can see folders below the directory from which it was launched, but it cannot see directories above its launch directory.
If you do not want to use the enhanced JupyterLab interface, you can run
jupyter-notebookto load only the notebook without the JupyterLab interface.
Once JupyterLab (or Jupyter Notebook) has started, you may be asked to pick a Kernel. Choose the generic Python 3 environment or cdat81 or jupyter-vcdat. If you chose the generic Python 3 environment that means Jupyter will use whichever Python environment started the JupyterLab (or Jupyter Notebook) session. Hence it is best to start the JupyterLab session from an environment that contains CDAT.
As a side note, VCDAT and VCS are two different CDAT modules though they have similar acronyms. VCDAT 2.0 helps you visualize, manipulate data and test out code within the JupyterLab interface. VCS stands for Visualization Control System which allows scientists to create customized plots and animations.
To Interact with the 3D image...¶
You will ultimately need to run the Python code in this notebook in a command line window, not within this notebook. One possible work flow is:
- Step through this notebook line by line to make sure everything works (e.g. you are running an environment with the needed packages, the code is working for you with the sample data, etc.).
- Once everything is working, either use the Python file we've provided above or save this notebook as a Python file by clicking on File (within the JupyterLab interface) > Export Notebook As... > Export Notebook to Executable Script.
- If you save/export the script from this notebook, you will need to edit the .py file to remove the "#" symbol in front of the x.interact() command at the bottom of the file. (If you download the python script directly, no editing is necessary.)
- Open a new command line window and activate your favorite CDAT environment (for testing this notebook, we used the jupyter-vcdat environment after installing VCDAT). Note: make sure you are using a "regular" version of the packages - NOT the mesalib version - since you will need to view the 3D image in a window that pops up once the Python script has been run and the mesalib version suppresses this pop up window.
- Navigate to where the Python script is stored and type:
at the prompt.python -i 4D_Air_Temp_Anomaly_1920x1080.py
A separate window should open where you can adjust the sliders and click on the "Configure" button to access more features of the plot. Click and drag on the center of the plot to change the angle at which you are viewing the data. The X, Y and Z slices are best viewed with the volume plot turned off. You will also likely need to turn off the volume plot to see the surface plot which shows an isosurface of the variable.
To exit the interactive mode, close the window and type control-D within your command line window to fully exit the interactive mode and to get your command prompt back. If control-D does not work, type control-Z.
An alternate work flow is to accomplish all the steps above in a command line window without using JupyterLab or the Jupyter Notebook.
Import the Necessary Python Modules¶
import vcs, cdms2, cdutil, genutil, sys
Get the Tutorial Data¶
Download many sample data files for all of our notebooks. For this exercise you need the ta_ncep_87-6-88-4.nc sample data file.
vcs.download_sample_data_files()
Open the sample data file¶
f=cdms2.open(vcs.sample_data+"/ta_ncep_87-6-88-4.nc")
Initialize the VCS canvas¶
In this example the VCS canvas is called x.
x = vcs.init()
The next line sets the canvas size to match YouTube's optimal setting of 1920 x 1080 px (16:9 aspect ratio). Comment it out if you want to use the smaller default canvas size of 814 x 606 pixels.
x.geometry(1920,1080)
Set DV3D settings¶
Choose various DV3D settings for the type of plot you want to make.
Get the data in a scalar format, as opposed to vector format (i.e. this data is scalar data, not vector data).
dv3d = vcs.get3d_scalar()
This plot will be a volume plot.
dv3d.ToggleVolumePlot = vcs.on
Set the camera position for the initial point of view.
dv3d.Camera={'Position': (-161, -171, 279), 'ViewUp': (.29, 0.67, 0.68), 'FocalPoint': (146.7, 8.5, -28.6)}
Choose variable to plot¶
Select the 3D air temperature (ta) variable from the open ta_ncep_87-6-88-4.nc file (f) and assign to the v0 variable. The Air Temperature is in degrees Kelvin (K). Outputting the shape of v0 tells use the dimensions of the matrix which is important for future steps. Here the data should have 11 time steps, 17 vertical levels above the ground, 73 latitude entries (data at every 2.5 degrees of latitude from -90 to 90 degrees), and 144 longitude entries (data at every 2.5 degrees of longidude from -180 to 180 or 0 to 360 depending on how you want to denote longitude).
v0 = f["ta"]
v0.shape
Calculate Average/Mean Temperature¶
Average the temperature of the 3D data, v0, over the x axis (latitude) and assign to the variable, va. We need the average in order to determine the temperature anomaly, i.e. how much each temperature value differs from the mean or average temperature of the dataset. Printing out the shape of the va matrix shows that it does not have data in the longitude dimension (i.e. the matrix dimensions are (11, 17, 73) not (11, 17, 73, 144)).
va = cdutil.averager( v0, axis='x' )
va.shape
Create Matricies Needed to Calculate the Temperature Anomaly¶
Now that we have the average we need to subtract it from each data value to get the temperature anomaly, but in order to do that, the original data (v0) and the average (va) need to be in matrcies of the same size. The grower method accomplishes that task and creates two matricies of the same size, one for the temperature values based on the original data (v01) and the other for the average temperature for the data set (va1).
v01,va1=genutil.grower(v0,va)
v01.shape
va1.shape
Calculate 3D Temperature Anomaly¶
With two matricies the same size, we can now subtract the average or mean temperature matrix, va1, from the original temperature data, v01, to get the temperature anomaly matrix, v, which will be plotted.
v = v01 - va1
v.shape
To create a nice-looking plot, we'll want the colors to cover the full range of the data so we get the minimum and maximum values using the .minmax method.
vcs.minmax(v)
The middle line below specifies the ScaleColormap to cover the min and max values of the data. If you'd like the program to create the plot based on the data without your specifying the temperature range for the ScaleColormap and ScaleTransferFunction methods, comment out the following three lines of code as we have done here.
#dv3d.VerticalScaling = 4.0
#dv3d.ScaleColormap = [-27.0, 24.0, 1]
#dv3d.ScaleTransferFunction = [-27.0, 24.0, 1]
Create the 3D plot¶
The next two lines of code are useful if you want to test that the plotting capability is working without creating a full sequence of .png files or the animation at the end. They are intended to be used when stepping through the notebook or python code one line at a time after the "#" is removed to turn them into lines of code instead of comments.
Note: when you run a line of code with x.plot( thing_to_plot, dv3d ), you can safely ignore the "can't open data file 'None'" warning.
#x.plot( v, dv3d )
#x.png("air_temp_anomaly_t_1.png")
The following lines of code create a .png for each time slice.
Due to a bug (as of 6/27/19), we need to loop through the first two time slices a second time so the sliders are not shown on the .pngs for these images, hence the additional [0,1] when specifying the range of the loop.
tc = v.getTime().asComponentTime()
for i in list(range(len(v.getTime())))+[0,1]: # due to a bug, we need to loop through i=0 and i=1 a second time
print("Time slice",i)
data = v[i]*1.
data.id = str(tc[i]) # assigns the time to the id attribute so it can be printed at the top of each .png
d = x.plot( data, dv3d )
x.png("anomaly_time_{:03d}".format(i))
d
Create the Animation¶
import glob
pngs = sorted(glob.glob("anomaly_time_*.png"))
x.ffmpeg("anomaly_anim_1920x1080.mp4", pngs, bitrate=1024, rate=None, options=None)
If you want to save a python script from this notebook to run outside the notebook (so you can interact with the 3D plot), save the script then delete the "#" in the line below to turn it from a comment to an active line of code.
# x.interact()
The CDAT software was developed by LLNL. This notebook was created by Holly Davis on June 27, 2019, updated on October 3, 2019, and is based on other CDAT notebooks created by Charles Doutriaux. This work was performed under the auspices of the U.S. Department of Energy by Lawrence Livermore National Laboratory under Contract DE-AC52-07NA27344.
If you have questions about this notebook, please email our CDAT Support address, cdat-support@llnl.gov.