VelmaSimRunner

Overview

VelmaSimRunner is a utility tool that helps automate running one or more VELMA simulations via VELMA's command line interface.
This guide explains its capabilities and how to use it.
(A separate quick-reference is available for experienced users.)

Background

The VELMA simulator is most often run via JVelma, an interactive GUI. However, a command line interface (VelmaSimulatorCmdLine) exists, and is useful when:

• You do not need to review or edit the simulation configuration .xml you intend to run.
• You do not need to view the runtime display charts that JVelma provides.
• You need the simulation to run as efficiently as possible: VelmaSimulatorCmdLine can usually run a VELMA simulation in a bit less time than JVelma, because VelmaSimulatorCmdLine does produce the afor-mentioned runtime display charts.
• You need to automate the process of running several VELMA simulations in sequence.
• You'd like to let a (perhaps fairly long) sequence of VELMA simulations run unattended, but later, you'd like know which runs succeeded and which failed without examining each and every simulation run's results directory's contents.

Regarding the final two points above: you can automate starting a sequence of VELMA simulations just by putting the complete, individual command lines into a .bat or .ps batch file (depending upon whether you are using Windows' Command Prompt or PowerShell.) However, as a developer, working with the VELMA .class files in a workspace, running VelmaSimulatorCmdLine can involve a fair amount of typing per command line.
Here is an PowerShell example command line that runs a VELMA scenario whose simulation configuration is named OR_BR_HJA_ws10_30m_userD_defW_2019-1-31a.xml:

PS C:\Users\kdjang\> java -Xmx4g -cp '.\Velma_GitHub\VelmaUtils\bin;.\Velma_GitHub\VelmaSimulator\bin' gov.epa.velmasimluator.VelmaSimulatorCmdLine F:\Users\kdjang\Velma_SiteData\OR_BlueRiver_Xml\OR_BR_HJA_ws10_30m_userD_defW_2019-1-31a.xml

Confronted with the need to run, say, 25 or more simulations (one after the other), each using the same basic pattern as the example above, but with some more or less memory ('-Xmx') allocated, and each with a unique .xml file name, most developers would copy the original example command line as many times as needed into a .bat or .ps file, then edit differences for each line in the file.
Doing exactly this sort of batch file creation caused VELMA developers to write VelmaSimRunner.
VelmaSimRunner provides you with 'shorthand' you can use to simplify batch files. It also provides a way to start and run a batch of VELMA simulations, tracking which runs succeed and fail in a .log file.

Platform Requirements

You'll need the following to run (and make use of) VelmaSimRunner.

• Windows 10 -- no other operating systems have been tested.
• Windows 10 PowerShell -- should be available on any Windows 10 system.
• Python 3.7 -- version 2 will not work, and no earlier versions of version 3 have been tested.
• Java 8 -- for running VELMA. Newer versions (e.g. Java 9) should be fine, but older versions (e.g. Java 7) will not do.
• Either a current, valid VELMA workspace with all the .class files built, or a current, valid JVelma.jar file.
  Note that the VelmaSimRunner.py utility tool is itself part of the fileset in a valid VELMA workspace, so you'll need that workspace even if you plan to invoke VelmaSimulatorCmdLine from a JVelma.jar file.

Tutorial: How VelmaSimRunner Works

You start VelmaSimRunner from a PowerShell window with the following command:

PS C:\Users\auser>py -3 C:\Users\asuer\Velma_Git\DeveloperUtils\VelmaSimRunner.py

The above command drops you into VelmaSimRunner's interpreter. ("C:\Users\auser\Velma_Git" is an example path to ) The interpreter has a set of commands that allow you to create a list of "CONFIGS". Each config item corresponds to one actual PowerShell command line that will start a VELMA simulation.

The interpretor's "ready" prompt is the '=' character.
Type the list command at the prompt and then press the Enter (i.e. Return) key:

= list
DEFAULTS:
    vm_name : ['java']
    vm_args : []
    vm_classpath : []
    tool_name : ['gov.epa.velmasimulator.VelmaSimulatorCmdLine']
    tool_before_args : []
    config_location : []
    config_item : []
    tool_after_args : []
    results_prefix : []
CONFIGS:
=

The list command lists the intepreter's current state.
The above output tells you what the current DEFAULTS are, and how many CONFIGS you have: (i.e. none, as yet).

To create a CONFIG item (i.e. a simulation command line to run) type the add command, followed by the config_item keyword , and follow that with the name of the VELMA simulation configuration .xml file you want to run:

= add config_item OR_BR_HJA_ws10_30m_userD_defW_2019-1-31a.xml

When you "add config_item somefile.xml", the interpreter builds a new config and adds it to its configs list:

= list
DEFAULTS:
    vm_name : ['java']
    vm_args : []
    vm_classpath : []
    tool_name : ['gov.epa.velmasimulator.VelmaSimulatorCmdLine']
    tool_before_args : []
    config_location : []
    config_item : []
    tool_after_args : []
    results_prefix : []
CONFIGS:
0 : ['java', 'gov.epa.velmasimulator.VelmaSimulatorCmdLine', 'OR_BR_HJA_ws10_30m_userD_defW_2019-1-31a.xml']
=

The interpreter builds the config entry by putting together all of the current DEFAULTS values in the order they're listed, before and after the config_item.
(Note: config_item is not a default -- you specified it in your add command -- but is listed as a placeholder amongst the defaults to remind you where it will occur in the config's sequence.)

The results of the list command above show that, apart from the config_item you added, only the vm_name and tool_name weren't blank, so your newly-added config entry consists of [vm_name, tool_name, config_item].

The run command runs each item in your CONFIGS list by passing each config item's list of values to the Windows PowerShell.
However, your single, current config might not run, because the command passed to PowerShell would literally be this:

java gov.epa.velmasimulator.VelmaSimulatorCmdLine OR_BR_HJA_ws10_30m_userD_defW_2019-1-31a.xml

If your PowerShell has a Java CLASSPATH set that covers all the necessary VELMA .jar and/or .class files, and the file OR_BR_HJA_ws10_30m_userD_defW_2019-1-31a.xml is located in the directory you are invoking VelmaSimRunner from, then the above command might run correctly.
However, a more common case for no CLASSPATH to be set and for the VELMA .xml file's location somewhere else. Here's an example of trying to run the above config without a proper CLASSPATH:

= run
Error: Could not find or load main class gov.epa.velmasimulator.VelmaSimulatorCmdLine

Whenever VelmaSimRunner trys to run one or more CONFIGS items, it logs the attempt in a file named VelmaSimRunner_Log.txt. The file is created in your Users Home directory, and new log messages are appended to it whenever you use VelmaSimRunner to run CONFIG items. You can delete the VelmaSimRunner_log.txt file if VelmaSimRunner is not currently running, and it will be re-created the next time it is needed. Here's what gets written to the log file for the (failed) run command we tried:

2019-02-27 09:03:03 INFO     Starting run: OR_BR_HJA_ws10_30m_userD_defW_2019-1-31a.xml
2019-02-27 09:03:03 ERROR    FAILED run: OR_BR_HJA_ws10_30m_userD_defW_2019-1-31a.xml
Traceback (most recent call last):
  File ".\DeveloperUtils\VelmaSimRunner.py", line 211, in run_simulation
    subprocess.run(cmd_args, shell=True, check=True)
  File "C:\Users\auser\AppData\Local\Programs\Python\Python37\lib\subprocess.py", line 487, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['java', 'gov.epa.velmasimulator.VelmaSimulatorCmdLine', OR_BR_HJA_ws10_30m_userD_defW_2019-1-31a.xml']' returned non-zero exit status 1.
2019-02-27 09:03:03 INFO     Elapsed Time=0.26074910163879395 seconds for OR_BR_HJA_ws10_30m_userD_defW_2019-1-31a.xml

Unfortunately, once you add a config_item to the CONFIGS list, you can't modify its sequence of values.
Fortunately, it's easy to delete a CONFIGS item and re-add a modified version:

= delete config_item 0
= list
DEFAULTS:
    vm_name : ['java']
    vm_args : []
    vm_classpath : []
    tool_name : ['gov.epa.velmasimulator.VelmaSimulatorCmdLine']
    tool_before_args : []
    config_location : []
    config_item : []
    tool_after_args : []
    results_prefix : []
CONFIGS:
=

The "delete config_item " command removes the nth item (zero-base) from the CONFIGS list.

To re-add the config we want to run, including classpath and .xml file location values, we need to set the appropriate DEFAULTS:

= set vm_classpath "C:\Users\auser\My Projects\Velma_Git\VelmaUtils\bin;C:\Users\auser\My Projects\Velma_Git\VelmaSimulator\bin"
= set config_location "C:\users\auser\My Data\Blue River Xml"
= list
DEFAULTS:
    vm_name : ['java']
    vm_args : []
    vm_classpath : ['C:\\Users\\auser\\My Projects\\Velma_Git\\VelmaUtils\\bin;C:\\Users\\auser\\My Projects\\Velma_Git\\VelmaSimulator\\bin']
    tool_name : ['gov.epa.velmasimulator.VelmaSimulatorCmdLine']
    tool_before_args : []
    config_location : ['C:\\users\\auser\\My Data\\Blue River Xml']
    config_item : []
    tool_after_args : []
    results_prefix : []
CONFIGS:

The "set keytext value" command sets the value of the specified DEFAULTS keytext to the specified value. Notice that we bounded the values we set with double-quote (") characters. This tells the interpreter to treat the text as a single value, even though the text includes whitespace.

Now, we re-add the config item with the same command we used previously.

= add config_item OR_BR_HJA_ws10_30m_userD_defW_2019-1-31a.xml
= list
DEFAULTS:
    vm_name : ['java']
    vm_args : []
    vm_classpath : ['C:\\Users\\auser\\My Projects\\Velma_Git\\VelmaUtils\\bin;C:\\Users\\auser\\My Projects\\Velma_Git\\VelmaSimulator\\bin']
    tool_name : ['gov.epa.velmasimulator.VelmaSimulatorCmdLine']
    tool_before_args : []
    config_location : ['C:\\users\\auser\\My Data\\Blue River Xml']
    config_item : []
    tool_after_args : []
    results_prefix : []
CONFIGS:
0 : ['java', 'C:\\Users\\auser\\My Projects\\Velma_Git\\VelmaUtils\\bin;C:\\Users\\auser\\My Projects\\Velma_Git\\VelmaSimulator\\bin', 'gov.epa.velmasimulator.VelmaSimulatorCmdLine', 'C:\\users\\auser\\My Data\\Blue River Xml\\OR_BR_HJA_ws10_30m_userD_defW_2019-1-31a.xml']
=

However, notice how different the list command results are from the first time we added it. The vm_classpath and config_location have been included in the config's collection of command line values.