Module DSMACCanalysis

Julia script to analyse and display DSMACC model output.

Content

• Module ropa_tool to analyse chemical source and sink fluxes of DSMACC output
• Includes:
  • Module FluxAnalysis to determine chemical source and sink fluxes for each species
  • Module prepare_ropa to arrange DSMACC output from different scenarios in an array of data frames and analyse each reaction finding the educts and products with the corresponding stoichiometric indices
  • Module NCload to load species concentrations and reaction rates into a dictionary with data frames
  • Function test_file from module fhandle in repository auxdata to check existance of an nc file
  • Julia modules DataFrames and PyCall
  • Function input from Julia module Juno to read user input
• Module groupSPC to lump the NMVOC organic pool by certain properties
• Properties currently available:
  • Molecule's size (carbon and ether group skeleton)
  • O:C ratio
  • Chromophore class
• Includes:
  • Function rdinp from module fhandle in repository auxdata to read in data files
  • Julia modules DataFrames
  • Function input from Julia module Juno
• Module DSMACCplot to plot DSMACC model results from specifications of an input file
• Includes:
  • Module groupSPC to lump species according to their O:C ratio, their molecule's size, and their chromophore class
  • Module ropa_tool to analyse source and sink fluxes of each species
  • Module jlplot to compile plots of DSMACC model output in a pdf
  • Module plot_ropa to plot time-resolved chemical sink and source fluxes for a species from the ROPA analysis
  • Module make_plots from repository auxdata to generate python plots (with PyPlot)
  • Module NCload to load species concentrations and reaction rates into a dictionary with data frames
  • Module fhandle in repository auxdata for input file handling
  • Julia modules DataFrames, PyCall, and PyPlot
  • Function input from Julia module Juno

Installation

The script is written for and tested with the Julia Mac version 0.6.2. Further tests have been performed with the Anaconda 4.3.29 julia v0.6.1 version in an UNIX environment.
If not installed, install Julia. The script furthermore uses miniconda's python version for julia. Missing Julia and python packages will be installed automatically using Conda and pip.

The script will analyse DSMACC model output from the DSMACC-testing version. Create a git submodule in the AnalysisTools folder named DSMACCanalysis (or clone this repository into the AnalysisTools folder) for optimal use with DSMACC.

You will furthermore need the modules fhandle and make_plots from the auxdata repository. The script will automatically download the necessary modules from the auxdata repository and store it in the SRC/jl.mod/ folder. Alternatively, if you have further scripts that share files with auxdata, it might be worth cloning this repository to a directory of your liking and specify the directory paths in the preambles of the following modules:

• DSMACCplot (ll. 27 – 38)
• jl.mod/groupSPC (ll. 46 – 60)
• jl.mod/jlplot.jl (ll. 40 – 52)
• jl.mod/NCload (ll. 31 – 43)
• jl.mod/ropa/plot_ropa (ll. 33 – 44)

Either adjust the folder path in one of the if statements to the directory of the repository auxdata (3 times) as

directory/to/repo/jl.mod

or add a new case.

Usage of the script

The Julia ROPA tool

Function ropa in module ropa_tool (in the jl.mod/ropa folder) is used to analyse the sink and source fluxes for every species in a series of model runs of specified DSMACC model output netCDF files. It can be used as a standalone tool to return arrays with DataFrames for each scenario holding the source and sink fluxes for each species in the specified mechanisms and the species concentrations. Call the ropa tool with:

source, sink, specs = function ropa(scenarios=[]; specs=[], rates=[], cycles="reduce")

Where the argument scenarios is a string list with the species names (and folder positions). If you already retrieved the species concentrations and reaction rates as array of DataFrames for each scenario, you can omit the scenarios argument and specify the keyword arguments specs for the species concentrations and rates for the reaction rates instead. Furthermore, the ROPA tool calculates net fluxes of inorganic NOx and Ox recycling as those fluxes would otherwise overlay the true sinks and sources for important Ox and NOx species. If you don't want net fluxes, set the keyword argument cycles to "full". Otherwise, the following reactions will be shown as net main sources and sinks for O₃, O(¹D), O(³P), NO, and NO₂:

• O₃ ⟶ O(¹D)
• O(¹D) ⟶ O(³P)
• O₃ ⟶ O(³P)
• O(³P) ⟶ O₃
• NO₂ ⟶ NO + O(³P)
• NO + O₃ ⟶ NO₂

The output data is organised as arrays of type Any holding data frames of each scenario. The concs DataFrames hold columns for the concentrations at every time step of every species using the DSMACC species name as column header.

The sinks and sources DataFrames hold an n×2 matrix with the all the species names in the first column. The second column holds another n×2 matrix with the DSMACC reaction (from the labels of the rates DataFrames) in the first column and the chemical flux data in the second column. For each species all respective sink and source reaction fluxes are listed in the matrices in the second column of the outer matrix.

The Julia module groupSPC

The module is used by the plotting script DSMACCplot.jl (see below). It groups the pool of NMVOC species according to certain properties. The lumped concentrations are added to the DataFrames with the species concentrations in each scenario using the column headers given below. These column headers can be used in the input file with the plot specifications to plot the concentrations of the respective property.

The following properties exist:

Molecule's size

Molecules are lumped by size. The size is determined by the carbon number plus the number of ether groups in a molecule (making up the molecule's skeleton). 10 size classes exist, with class 1 to 9 holding that number of carbon + ether groups and class 10 holding 10 or more carbon + ether groups. The keyword for each class is sc#, where # is 1 to 9 for the first 9 classes and 0 for class 10.

O:C ratio

Molecules are grouped by O:C ratio into 4 classes with increasing O:C ratios and, hence, expected increased solubility. In the first class, pure gas phase species are expected, the last class exists of species, which are expected dominantly or exclusively in the aqueous phase. Classes 2 and 3 exist of semi-volatile species with increasing solubility.

• oc1: O:C ≤ 0.5
• oc2: 0.5 < O:C ≤ 1.0
• oc3: 1.0 < O:C ≤ 2.0
• oc4: 2.0 < O:C

Chromophore classes

Stable (non-radical) species are furthermore lumped by the chromophores they hold. Classes exist for species holding a single chromophore or chromophores of only 1 type. Another class exists for species with several chromophores of different type (polyfunctional chromophores) or no chromophores at all. Further special cases are inorganic species, organic radical species, and generic 'new' model species from test scenarios with a new photolysis protocol of the Master Chemical Mechanism (MCM).

The following classes exist:

• Ald: Mono-aldehydes
• Ket: Mono-ketones
• DiCar: Di- and polycarbonyls
• Kete: Ketenes
• Nitro: Mono-nitrocompounds
• DiNitro: Di- and poly-nitrocompounds
• Nit: Mono-alkyl nitrates
• DiNit: Di- and poly-alkyl nitrates
• PNit: Mono-alkyl pernitrates
• DiPNit: di- and poly-alkyl pernitrates
• PAN: peroxyacyl nitrates
• ROOH: Mono-alkyl hydroperoxides
• DiROOH: Di- and poly-alkyl hydroperoxides
• PAA: Organic peroxy acids
• SCI: Stabilised Criegee intermediates
• Poly: Compounds with polyfunctional chromophores
• NoChr: Stable organic compounds with no chromophores
• Rad: organic radical compounds (excluding SCI)
• Inorg: Inorganic compounds
• New: New generic model species

The Julia plotting script DSMACCplot.jl

Although the previous modules can be used as standalone versions, they are conveniently incorporated in the DSMACCplot.jl script, where their output is displayed graphically. Standalone versions of the above modules are only needed, if further data manipulation is desired. To display DSMACC model output, the following plot types exist in the current version:

• Line plots for species concentrations and reaction rates
• Stacked area plots for species concentrations (meant for lumped sum species) for up to 2 different scenarios using border lines with different line styles and different colour schemes for different scenarios
• Stacked area plots for time-resolved sink and source fluxes for a particular species in a chosen scenario

Run the script using

julia DSMACCplot.jl [[<path/>]<input file>] [<path/>[<output scenario name>]]

where the first script argument defines the name and directory of a text file with specifications of the desired plots. A default directory one level above this repository is define, if a folder path is obsolete. If you want plots in the current folder (main folder of the reop), us ./ as folder path. The second arguments defines the file name and folder path of the output file with the desired plots. By default, files will be saved in the DSMACC folder ./save/results assuming the repo is cloned to the AnalysisTools folder. Output files are in pdf format. The specification in the second script argument is optional and .pdf will be added to the file name, if missing in the script argument. If no second script argument is given, the file name will be the joined labels of all scenarios connected by an underscore (<scen 1>_<scen 2>_..._<scen n>).
If the input file name is missing in the first argument or the file doesn't exist, you will be asked for user input during the execution of the script.

The input file with plot specifications

The input file is divided in 4 sections, each addressed by a section caption. It is important to keep the order of the sections the following:

1. Scenarios section
2. Settings section (optional)
3. Plotting section
4. Comments section (optional)

Scenarios section

The first section is labelled with the keyword Scenarios:. On the first line after the caption, list all the netCDF files, you want to consider for plotting. If the files are saved in the folder ./save/results you can omit the folder path, otherwise folder paths are mandatory. List all files on one line. The following separators are allowed on horizontal lists on the same line throughout the input file:

• whitespace (spaces, tabs)
• commas (,)
• semicolons (;)

On the second line you can specify labels for each scenario, which are used in the figure legends. If this line is obsolete, file names are used as labels (without the file ending). Use those automatic labels to address, which scenarios you want to plot (see plotting section).

Settings section

The second section is optional and is introduced by the keyword Settings. It lists general specifications of the plots. Currently, options are available for:

• The MCM version used in the DSMACC model runs
• The time format
• Night-time shading in plots
• A lower and upper cut-off for displaying sink and source fluxes c
• Calculation of net fluxes for inorganic NOx/HOx recycling
• Saving additional single pdfs to folder FIG

By default, MCMv3.3.1 is assumed. You can specify, this or any other version using the keyword MCM: followed by the verion number, e.g. MCM: v3.3.1. This is important for internal processing as MCM names are translated to GECKO-A nomenclature for the analysis of the structure and the lumping by properties. While the script should work fine for the latest MCM version, there is no guarantee that the script will work correctly for older versions.

There are 2 options for the time format, either as model time in hours starting at 0 (default) or as Julian time (date/time of the year). Specify the format using the keyword time: followed by the keyword TIME for the default model time format or JTIME for the Julian time format, e.g.: time: TIME.

There is an option to shade night-time periods in the plots. By default, this option is shut off, if you want shading, use the keyword night: followed by specifications for the colour of the shade and a transparancy value between 0 (completely transparent) and 1 (completely opaque), e.g. night: black, 0.2.

Chemical fluxes smaller than a lower cut-off are combined in the plots. If fluxes larger than an upper cut-off exist, a second zoom plot will be created, where fluxes above the upper cut-off are omitted to allow the view of the smaller fluxes. Define the parameters using the keyword cut-off: followed by specifications for the lower and upper threshold, e.g. cut-off: 0.05, 0.7. If you don't want this feature, set the lower cut-off to 0.0 and the upper cut-off to 1.0. If you don't specify thresholds, standard values of 0.05 and 0.7 are used.

Furthermore, net fluxes of inorganic main NOx and Ox cycles can be formed during the ROPA analysis as described above. By default, this feature is set. To illustrate this in the input file use the keyword cycles: in the settings section followed by reduce. You may omit this line, if you want net fluxes. Use any other phrase after the keyword cycles: such as full to shut off this feature.

As single pdfs can be more convenient, if the plots are produced for presentations, single pdfs can be saved to a folder FIG, which will be created, if it doesn't exist. Otherwise, you will be asked, whether results can be overwritten or the script will terminate, so previous results can be saved first. Select between on and off of the Fig parameter, default is: Fig: off.

Plotting section

The actual plot specifications are given in the third section started by the caption Plotting:. Different plot types and plots can be specified. Plot types are introduced by a caption line, specifying for which scenarios plots are to be produced, what type of plot, and the units used in the plots. On the following lines, species or reactions can be specified, for which the data should be plotted. Different plot types can be specified in one file and must be separated by at least one blank line, while blank lines are prohibited within one plot type section.

The following plots are currently available and can be specified as follows:

Line plots

For species concentrations use the heading:

<scen1> ... <scen n>: specs/unit

For reaction rates use the heading:

<scen1> ... <scen n>: rates/unit

Use a horizontal list of scenarios, if you want to plot concentrations of the same species for more than one scenario and the respective keyword for concentrations or rates after a colon (:). Use a slash after the keyword and a keyword for the unit to define units (see below).

Specify the species or rates to be plotted using the DSMACC labels from the netCDF file for the species and rates. For every plot you want, start a new line and specify all species that go into the plot in a vertical list (see also example file plot.inp).

Stacked area plots of concentrations

To represent the total NMVOC mass differentiated by certain properties, stacked area plots exist. For this plot type, a transparent area plot with opaque boundary lines is used. This plot type can be defined for up to two different scenarios. Different line styles of the boundaries and different colour schemes will be used for the different scenarios. Specify the plot type by the heading

<scen1>[, <scen 2>]: stack/unit

Again, list data specification for different plots on different lines and the (lumped) species for each plots as a horizontal list on 1 line.

Time-resolved ROPA plots

You can plot time-resolved sink and source fluxes from the ROPA analysis in stacked area plots, where sink fluxes are negative and source fluxes are positive. Start this plot type by the head line

<scen1>, ... <scen n>: fluxes

followed by a list of species. The list of species can be on separate lines or in a horizontal list on a single lines or mixtures of both.

In any case, flux plots are plotted in separate plots for every species and every scenario no matter of the format in the input file with the plot specifications.

Unit specifications

Units can be specified after the plot keyword using a slash and the unit keyword. The below unit keywords/units are currently allowed. The standard unit is molecules cm^-3 for concentrations, cm³ molecule^-1 s^-1 for rates, and molecules cm^-3 s^-1 for fluxes. For standard units the keyword can be omitted in the plot specifications. Flux plots currently only exist in the default unit.

• mlc or cm-3 (default): molecules cm^-3 or cm³ molecule^-1 s^-1 or molecules cm^-3 s^-1
• ppm: ppm or ppm^1-n h^-1 (n … order of reaction)
• ppb: ppb or ppb^1-n h^-1
• ppt: ppt or ppt^1-n h^-1

Comments section

The last section is for comments only and is ignored by the actual plotting script. It is optional and can be started by the caption Comments:.

Version history

Version 0.6.2

• New features:
  • Optional night-time shading in plots
  • Options for model time (starting at 0) or Julian time (date and time of the year)
  • Treatment of older MCM versions (experimental)
• Updated MCM species database
• Bugfixes concerning the display of the order of magnitude of chemical fluxes
• New default paths of I/O files outside of repository
• Code improvements
• This version works with auxdata release v0.6.

Version 0.6.1

• Bugfix: Plotting previously omitted last line of species in the last plot type when Comments section is missing
• Bugfix: Correcting paths for module auxdata to be recognised on earth0

Version 0.6

• Julia ROPA tool for the time-resolved analysis of chemical sink and source fluxes for each species in a list of defined scenarios
• Julia module groupSPC to lump concentrations of NMVOC by the properties molecule's size, O:C ratio, and chromophore class
• Julia script DSMACCplot.jl to generate a pdf with line plots of species concentrations or reaction rates, stacked area plots (with boundary lines) of species concentrations (especially lumped concentrations from groupSPC), and time-resolved sink and source flux plots from the ROPA analysis
• Unit conversions from molecules cm^-3 / cm³ molecule^-1 s^-1 to ppm / ppm h^-1, ppb / ppb h^-1, and ppt / ppt h^-1 are available for line plots and stacked area plots of concentrations