batch

From autoplot.org

Jump to: navigation, search

Purpose: describe usage of the Run Batch Tool

Audience: scientists

Contents

  1. Introduction
  2. Other Use Cases
  3. Export Results File
  4. What's To-Be-Done
  5. See Also
  6. multi argument

1. Introduction

The Run Batch tool generates inputs to a script, and runs the script with each value of the inputs. For example, suppose you have a process you need to apply to each of the files of a two-spacecraft mission. It would be nice to be able to run a script for each day and each spacecraft of the mission, and this is what the "Run Batch" tool does. It lets you select a list of inputs for a parameter, and the script is run with each of those inputs as one long process. A second parameter can be set as well, and each combination is then run (for example, each day, each spacecraft).

To show an example suppose we have a pile of plot pngs and we want to find the one with a black background. (Of course, the pngwalk tool could be used for the task as well.) Here's a script that counts the number of black or white pixels in an image:

tr= getParam( 'timerange', '2008-01-01', 'timerange to load' )
wb= getParam( 'color', 'white', 'the color to count', ['white','black'] )
ds= getDataSet( 'http://autoplot.org/autoplot/autoplot/autoplot/data/pngwalk/product_$Y$m$d.png?channel=greyscale', tr )
if wb=='white': iwb=255 
else: iwb=0
r= where( ds.eq( iwb ) )
print r.length()

The batch tool can be used to used to run this on each image in a set of files. In Autoplot, select [menubar]->Tools->Run Batch..., and then in the GUI select the script "http://autoplot.org/data/script/examples/countColors.jy" and hit the green play button. On the left window, the parameters of the script are listed in the droplist. Select timerange, and then in the area below, enter each timerange. Typically this would be copy-and-pasted, and there is a context popup for generating lists. Right-click (or context click on a Mac), and press "Generate". "Jan 2008" and "$Y-$m-$d" will enter in the 31 intervals. Click the go button, and see that the console output contains the pixel counts.

The following script parameter types are supported:

Parameter Type Supported How
resourceURI yes File Chooser Dialog allows selection of files.
enumeration yes Checkbox list allows selection of values to use.
floats yes Dialog for creating linspace list (0.1,0.2,0.3,...,99.9)

2. Other Use Cases

  • rename weblogs by the last date in the file
  • reformatting files
  • run IDL script for each day, using adapter.

3. Export Results File

The results can be saved to a file, see File->"Export Results". Once the file name FILE is set, the results will be written to FILE.pending as the batch is run.

4. What's To-Be-Done

This is a long-overdue facility written to solve a problem one day, which turns out to be quite useful. As such, the feature set is still incomplete and needs to be flushed out. New features include:

  • more ways to generate inputs
  • methods for storing inputs, which can be generated by other software systems.
  • feedback during the run is poor, and should include green/red success fail status as jobs are completed, and the console output after the run is complete.
  • running with tuples of inputs.
  • list of results, containing canvas screenshot and stdout/strerr text for each run. This could have the green/red success icons.

Let us know ([1]) if you have ideas for how to make this more effective!

5. See Also

HTML Tutorial: http://autoplot.org/data/tutorials/20170223_runBatch/20170223_1231.html

6. multi argument

v2022a_2 will allow the first (left) parameter to be a list of parameters, so that you can just run through a list of values.

For example, using the script https://github.com/autoplot/dev/blob/master/rfe/sf/772/script.jy,

set the left box (param1) to "seed;red;green;blue" and then give it a list of seed, red, green, and blue values, split by a semicolon:

0;200;200;0
1;0;200;200
2;0;100;200
3;200;100;0
4;100;200;0
0;100;200;100
1;100;0;100
2;100;0;200


This will set seed, red, green, and blue parameters for each step.

Retrieved from "http://autoplot.org//batch"
Personal tools