E3SM DV3D ne30 Native Grid Test

This tutorial focuses on using DV3D, a CDAT tool, to create a 3D plot of temperature data on E3SM's ne30 native grid. When the code in this notebook is run as a separate Python script (with minor modifications), DV3D allows users to interact with the 3D plot that opens in a stand-alone window on their computer.

Download the Jupyter Notebook

Download the Python file

The CDAT software was developed by LLNL and this notebook was created and the example code was updated on December 17, 2019. This work was performed under the auspices of the U.S. Department of Energy by Lawrence Livermore National Laboratory under Contract DE-AC52-07NA27344.

To Interact with the 3D image...

You will ultimately need to run the Python code in this notebook in a command line window, not within this notebook. One possible work flow is:

  1. Step through this notebook line by line to make sure everything works (you are running an environment with the needed packages, the code is working for you with the sample data, etc.).
  2. Once everything is working, either use the Python file we've provided above or save this notebook as a Python file by clicking on File (within the JupyterLab interface) > Export Notebook As... > Export Notebook to Executable Script.
  3. If you save/export the script from this notebook, you will need to edit the .py file to remove the "#" symbol infront of the x.interact() command near the bottom of the file. (If you download the python script directly, no editing is necessary.)
  4. Open a new command line window and activate your favorite CDAT environment (for testing this notebook, I used a CDAT 8.1 mesalib environment, but you could also use a jupyter-vcdat environment after installing VCDAT. (Note: When you run the Python script to create the interactive window of the 3D data, make sure you are using a "regular" version of the conda environment - NOT a mesalib version - since you will need to see the window that opens as the Python script runs. (Mesalib environments suppress this window.))
  5. Navigate to where the Python script is stored and type the following at the prompt:
    python -i E3SM_ne30_Native_Grid_Test.py
    The generic interactive Python command is:
    python -i Name_of_Your_Notebook.py

A separate window should open where you can adjust the sliders and click on the "Configure" button to access more features of the plot. Click and drag on the center of the plot to change the angle at which you are viewing the data.

To exit the interactive mode, close the window and type control-D within your command line window to fully exit the interactive mode and to get your command prompt back. (If for some reason control-D does not work, try control-Z.)

An alternate work flow is to accomplish all the steps above in a command line window without using JupyterLab or the Jupyter Notebook.

In [1]:
import vcs, cdms2, sys

Get the tutorial data

This tutorial uses the E3SM model version 1, 1 degree (standard resolution (ne30)) CMIP6 data which is available on NERSC's HPSS storage system. See the V1 1 Deg CMIP6 Data on HPSS post on the E3SM.org website to download the data.

The data file for this tutorial is "20180129.DECKv1b_piControl.ne30_oEC.edison.cam.h0.0500-01.nc" from the CMIP6 standard resolution (ne30) piControl run.

The grid file used is "ne30np4_latlon.091226.nc". To download this grid file, go to https://portal.nersc.gov/project/acme/hdavis/ and click on the "ne30np4_latlon.091226.nc" file which should automatically download via your browser. To facilitate running this notebook, the CMIP6 data file can also be downloaded from this same location.

Make sure both the data file and the grid file are in the same directory as this notebook before continuing to run this notebook.

Open the CMIP6 data file

The variable f (for file) references the data file.

In [2]:
f=cdms2.open("20180129.DECKv1b_piControl.ne30_oEC.edison.cam.h0.0500-01.nc")

Initialize the VCS canvas

In this example the variable x is the VCS canvas.

In [3]:
x = vcs.init()

Set the DV3D settings

Choose various DV3D settings for the type of plot you want to make.

In [4]:
dv3d = vcs.get3d_scalar()
In [5]:
dv3d.ToggleVolumePlot = vcs.on
dv3d.ToggleSphericalProj = vcs.on
In [6]:
dv3d.Camera={'Position': (-161, -171, 279), 'ViewUp': (.29, 0.67, 0.68), 'FocalPoint': (146.7, 8.5, -28.6)}

Choose a variable to plot

The next line of code shows us all the data variables within the CMIP6 netCDF file.

In [7]:
print(sorted(f.variables.keys()))
['AEROD_v', 'ANRAIN', 'ANSNOW', 'AODABS', 'AODABSBC', 'AODBC', 'AODDUST', 'AODDUST1', 'AODDUST3', 'AODDUST4', 'AODMODE1', 'AODMODE2', 'AODMODE3', 'AODMODE4', 'AODNIR', 'AODPOM', 'AODSO4', 'AODSOA', 'AODSS', 'AODUV', 'AODVIS', 'AQRAIN', 'AQSNOW', 'AQ_DMS', 'AQ_H2O2', 'AQ_H2SO4', 'AQ_O3', 'AQ_SO2', 'AQ_SOAG', 'AREI', 'AREL', 'AWNC', 'AWNI', 'CCN3', 'CDNUMC', 'CLDHGH', 'CLDHGH_CAL', 'CLDHGH_CAL_ICE', 'CLDHGH_CAL_LIQ', 'CLDHGH_CAL_UN', 'CLDICE', 'CLDLIQ', 'CLDLOW', 'CLDLOW_CAL', 'CLDLOW_CAL_ICE', 'CLDLOW_CAL_LIQ', 'CLDLOW_CAL_UN', 'CLDMED', 'CLDMED_CAL', 'CLDMED_CAL_ICE', 'CLDMED_CAL_LIQ', 'CLDMED_CAL_UN', 'CLDTOT', 'CLDTOT_CAL', 'CLDTOT_CAL_ICE', 'CLDTOT_CAL_LIQ', 'CLDTOT_CAL_UN', 'CLDTOT_ISCCP', 'CLD_CAL', 'CLD_MISR', 'CLHMODIS', 'CLIMODIS', 'CLLMODIS', 'CLMMODIS', 'CLMODIS', 'CLOUD', 'CLOUDFRAC_CLUBB', 'CLTMODIS', 'CLWMODIS', 'CONCLD', 'DCQ', 'DF_DMS', 'DF_H2O2', 'DF_H2SO4', 'DF_O3', 'DF_SO2', 'DF_SOAG', 'DMS_SRF', 'DP_KCLDBASE', 'DP_MFUP_MAX', 'DP_WCLDBASE', 'DSTSFMBL', 'DTCOND', 'DTENDTH', 'DTENDTQ', 'EXTINCT', 'FICE', 'FISCCP1_COSP', 'FLDS', 'FLNS', 'FLNSC', 'FLNT', 'FLNTC', 'FLUT', 'FLUTC', 'FREQI', 'FREQL', 'FREQR', 'FREQS', 'FSDS', 'FSDSC', 'FSNS', 'FSNSC', 'FSNT', 'FSNTC', 'FSNTOA', 'FSNTOAC', 'FSUTOA', 'FSUTOAC', 'F_eff', 'H2O2_SRF', 'H2SO4_SRF', 'H2SO4_sfgaex1', 'ICEFRAC', 'ICIMR', 'ICWMR', 'IEFLX', 'IWC', 'IWPMODIS', 'LANDFRAC', 'LHFLX', 'LINOZ_DO3', 'LINOZ_DO3_PSC', 'LINOZ_O3CLIM', 'LINOZ_O3COL', 'LINOZ_SSO3', 'LINOZ_SZA', 'LND_MBL', 'LWCF', 'LWPMODIS', 'MEANCLDALB_ISCCP', 'MEANPTOP_ISCCP', 'MEANTAU_ISCCP', 'MEANTBCLR_ISCCP', 'MEANTB_ISCCP', 'Mass_bc', 'Mass_dst', 'Mass_mom', 'Mass_ncl', 'Mass_pom', 'Mass_so4', 'Mass_soa', 'NUMICE', 'NUMLIQ', 'NUMRAI', 'NUMSNO', 'O3', 'O3_SRF', 'OCNFRAC', 'OMEGA', 'OMEGA500', 'OMEGAT', 'P0', 'PBLH', 'PCTMODIS', 'PHIS', 'PRECC', 'PRECL', 'PRECSC', 'PRECSL', 'PS', 'PSL', 'Q', 'QFLX', 'QREFHT', 'QRL', 'QRS', 'RAINQM', 'RAM1', 'REFFCLIMODIS', 'REFFCLWMODIS', 'RELHUM', 'SFDMS', 'SFH2O2', 'SFH2SO4', 'SFO3', 'SFSO2', 'SFSOAG', 'SFbc_a1', 'SFbc_a3', 'SFbc_a4', 'SFdst_a1', 'SFdst_a3', 'SFmom_a1', 'SFmom_a2', 'SFmom_a3', 'SFmom_a4', 'SFncl_a1', 'SFncl_a2', 'SFncl_a3', 'SFnum_a1', 'SFnum_a2', 'SFnum_a3', 'SFnum_a4', 'SFpom_a1', 'SFpom_a3', 'SFpom_a4', 'SFso4_a1', 'SFso4_a2', 'SFso4_a3', 'SFsoa_a1', 'SFsoa_a2', 'SFsoa_a3', 'SHFLX', 'SH_KCLDBASE', 'SH_MFUP_MAX', 'SH_WCLDBASE', 'SNOWHICE', 'SNOWHLND', 'SNOWQM', 'SO2', 'SO2_CLXF', 'SO2_SRF', 'SOAG_CLXF', 'SOAG_SRF', 'SOAG_sfgaex1', 'SOLIN', 'SSAVIS', 'SSTSFMBL', 'SSTSFMBL_OM', 'SWCF', 'T', 'TAUGWX', 'TAUGWY', 'TAUILOGMODIS', 'TAUIMODIS', 'TAUTLOGMODIS', 'TAUTMODIS', 'TAUWLOGMODIS', 'TAUWMODIS', 'TAUX', 'TAUY', 'TGCLDCWP', 'TGCLDIWP', 'TGCLDLWP', 'TH7001000', 'TMQ', 'TREFHT', 'TROP_P', 'TROP_T', 'TS', 'TSMN', 'TSMX', 'TUH', 'TUQ', 'TVH', 'TVQ', 'U', 'U10', 'UU', 'V', 'VQ', 'VT', 'VU', 'VV', 'WD_H2O2', 'WD_H2SO4', 'WD_SO2', 'WSUB', 'Z3', 'aero_water', 'airFV', 'area', 'bc_a1DDF', 'bc_a1SFWET', 'bc_a1_SRF', 'bc_a1_sfgaex1', 'bc_a3DDF', 'bc_a3SFWET', 'bc_a3_SRF', 'bc_a4DDF', 'bc_a4SFWET', 'bc_a4_CLXF', 'bc_a4_SRF', 'bc_a4_sfgaex1', 'bc_c1DDF', 'bc_c1SFWET', 'bc_c3DDF', 'bc_c3SFWET', 'bc_c4DDF', 'bc_c4SFWET', 'ch4vmr', 'chla', 'co2vmr', 'cosp_ht_bnds', 'cosp_htmisr_bnds', 'cosp_prs_bnds', 'cosp_sr_bnds', 'cosp_tau_bnds', 'cosp_tau_modis_bnds', 'date', 'date_written', 'datesec', 'dst_a1DDF', 'dst_a1SF', 'dst_a1SFWET', 'dst_a1_SRF', 'dst_a3DDF', 'dst_a3SF', 'dst_a3SFWET', 'dst_a3_SRF', 'dst_c1DDF', 'dst_c1SFWET', 'dst_c3DDF', 'dst_c3SFWET', 'extinct_lw_bnd7', 'extinct_lw_inp', 'extinct_sw_inp', 'f11vmr', 'f12vmr', 'hstobie_linoz', 'hyai', 'hyam', 'hybi', 'hybm', 'lat', 'lon', 'mdt', 'mlip', 'mom_a1DDF', 'mom_a1SF', 'mom_a1SFWET', 'mom_a1_SRF', 'mom_a1_sfgaex1', 'mom_a2DDF', 'mom_a2SF', 'mom_a2SFWET', 'mom_a2_SRF', 'mom_a3DDF', 'mom_a3SFWET', 'mom_a3_SRF', 'mom_a4DDF', 'mom_a4SF', 'mom_a4SFWET', 'mom_a4_SRF', 'mom_a4_sfgaex1', 'mom_c1DDF', 'mom_c1SFWET', 'mom_c2DDF', 'mom_c2SFWET', 'mom_c3DDF', 'mom_c3SFWET', 'mom_c4DDF', 'mom_c4SFWET', 'mpoly', 'mprot', 'n2ovmr', 'nbdate', 'nbsec', 'ncl_a1DDF', 'ncl_a1SF', 'ncl_a1SFWET', 'ncl_a1_SRF', 'ncl_a2DDF', 'ncl_a2SF', 'ncl_a2SFWET', 'ncl_a2_SRF', 'ncl_a3DDF', 'ncl_a3SF', 'ncl_a3SFWET', 'ncl_a3_SRF', 'ncl_c1DDF', 'ncl_c1SFWET', 'ncl_c2DDF', 'ncl_c2SFWET', 'ncl_c3DDF', 'ncl_c3SFWET', 'ndbase', 'ndcur', 'nsbase', 'nscur', 'nsteph', 'num_a1DDF', 'num_a1SF', 'num_a1SFWET', 'num_a1_CLXF', 'num_a1_SRF', 'num_a1_sfgaex1', 'num_a2DDF', 'num_a2SFWET', 'num_a2_CLXF', 'num_a2_SRF', 'num_a3DDF', 'num_a3SF', 'num_a3SFWET', 'num_a3_SRF', 'num_a4DDF', 'num_a4SFWET', 'num_a4_CLXF', 'num_a4_SRF', 'num_a4_sfgaex1', 'num_c1DDF', 'num_c1SFWET', 'num_c2DDF', 'num_c2SFWET', 'num_c3DDF', 'num_c3SFWET', 'num_c4DDF', 'num_c4SFWET', 'pom_a1DDF', 'pom_a1SFWET', 'pom_a1_SRF', 'pom_a1_sfgaex1', 'pom_a3DDF', 'pom_a3SFWET', 'pom_a3_SRF', 'pom_a4DDF', 'pom_a4SFWET', 'pom_a4_CLXF', 'pom_a4_SRF', 'pom_a4_sfgaex1', 'pom_c1DDF', 'pom_c1SFWET', 'pom_c3DDF', 'pom_c3SFWET', 'pom_c4DDF', 'pom_c4SFWET', 'so4_a1DDF', 'so4_a1SFWET', 'so4_a1_CLXF', 'so4_a1_SRF', 'so4_a1_sfgaex1', 'so4_a2DDF', 'so4_a2SFWET', 'so4_a2_CLXF', 'so4_a2_SRF', 'so4_a2_sfgaex1', 'so4_a3DDF', 'so4_a3SFWET', 'so4_a3_SRF', 'so4_a3_sfgaex1', 'so4_c1DDF', 'so4_c1SFWET', 'so4_c2DDF', 'so4_c2SFWET', 'so4_c3DDF', 'so4_c3SFWET', 'soa_a1DDF', 'soa_a1SFWET', 'soa_a1_SRF', 'soa_a1_sfgaex1', 'soa_a2DDF', 'soa_a2SFWET', 'soa_a2_SRF', 'soa_a2_sfgaex1', 'soa_a3DDF', 'soa_a3SFWET', 'soa_a3_SRF', 'soa_a3_sfgaex1', 'soa_c1DDF', 'soa_c1SFWET', 'soa_c2DDF', 'soa_c2SFWET', 'soa_c3DDF', 'soa_c3SFWET', 'sol_tsi', 'time_bnds', 'time_written']

The variable we'll plot in this tutorial is temperature, T which is in degrees Kelvin. The next line assigns the temperature data (T) to the v variable.

In [8]:
v = f["T"]

To ensure the color scale of our plot covers the minimum and maximum values of the data, we'll first determine min and max values, then modify the colormap scale.

Both VCS and Genutil have minmax functions. Here we'll use VCS's minmax function, but if you'd like to use Genutil's function instead, comment out the first line of code and un-comment the second and third lines.

In [9]:
vcs.minmax(v)

#import genutil
#genutil.minmax(v)
Out[9]:
(184.34933471679688, 307.5133361816406)

Modify the ScaleColormap to cover the min and max values of the data.

In [10]:
dv3d.VerticalScaling = 1.5 
dv3d.ScaleColormap = [184.0, 308.0, 1] 
dv3d.ScaleTransferFunction =  [184.0, 308.0, 1]

Create the 3D plot

In [11]:
x.plot( v, dv3d, grid_file="ne30np4_latlon.091226.nc")
/Users/davis278/miniconda3/envs/cdat81-mesa/lib/python3.6/site-packages/vcs-8.1-py3.7.egg/vcs/Canvas.py:3823: UserWarning: Unrecognized vcs plot keyword: grid_file, assuming backend (vtk) keyword
 Init PCViewer, nInputPoints = 3499344, n_overview_points = 500000, n_subproc_points = 500000, nCollections = 6, overview skip index = 6, init_args = ['ne30np4_latlon.091226.nc', 'file:///Users/davis278/repos/sandbox/E3SM_DV3D_Native_Grid_Example_TEST_CASE_3/E3SM_DV3D_Native_Grid_Test/20180129.DECKv1b_piControl.ne30_oEC.edison.cam.h0.0500-01.nc', None, ['T'], [None, None, None, None, None], None, None, 'xyz', 1.5]
Setting current_subset_specs:  {}
initCamera: Camera => ((-161.0, -171.0, 279.0), (146.7, 8.5, -28.6), (0.2906693100067274, 0.6715463369120944, 0.6815694165674988)) 
Out[11]:

If you want to interact with the 3D plot, save a python script from this notebook to run outside the notebook, then edit the python script to un-comment out the following line. Use a non-mesalib conda environment when running the Python script so you can interact with the 3D image in the window that opens after running the x.plot( v, dv3d ) command.

In [12]:
# x.interact()

Save a 2D image of the 3D plot

When working inside a notebook, saving the above image to a separate .png file could be useful if you want to look at the image outside of the notebook. The line of code below will save the plot to a .png file in the same directory as this notebook.

If running the code in this notebook from the corresponding Python script, comment out or delete the following line of code since in the direct Python interface, saving to a .png file results in an all black .png file. If you want a 2D image of a particular view you've created while interacting with the 3D image, we recommend taking a screenshot.

In [13]:
x.png("native_T_sphere.png")

The CDAT software was developed by LLNL. This tutorial was written by Charles Doutriaux and Holly Davis. This work was performed under the auspices of the U.S. Department of Energy by Lawrence Livermore National Laboratory under Contract DE-AC52-07NA27344.

If you have questions about this notebook, please email our CDAT Support address, cdat-support@llnl.gov.