Autoplot Application (20160723b)

The binary source is used to explore the data with a CDF file.

Autoplot attempts to allow plotting of data from many data sources, including remote CDF files and OpenDAP servers. Datasets are identified with URIs, and by the URI's extension, data are loaded into the internal data model. Data are then displayed by guessing the rendering method and axis ranges to provide a reasonable, useful view of the data. Data can then be additionally processed with other operations like smooth and FFT, and Jython scripting provides a capable programming environment.

This is the Autoplot2016a branch. This is an update to the production version containing bugfixes and minor new features. Java 7 is now required. Java 8 will be required sometime soon.

Single-jar version, autoplot.jar, can be run without network access and from the command line.

Experimental Mac Release is available.

Bugfixes (2016-07-23 r19239):

• DISPLAY_LOG_LEVEL property of Options included in syncToAll so we have more control over pngwalk output, to turn off message bubbles
• 1640: AutoplotDataServer d2s output drops metadata
• 1635: xaxis fails to unbind
• 1639: new bug shown by non-timeaxis-TSB script where TSB rebinding is not done. (Never released.)
• 1638: Jython getParam scraping doesn't support Chris' file with no indentation on closing paren.
• dataset( PyObject, Units ) was thoroughly implemented.
• 1632: copy over copy over SCALE_TYPE, TITLE, etc when writing das2streams.
• 1637: binary source fails to read large JADE files gt Integer.MAX_VALUE/8 bytes.
• 1634: AutoplotDataServer misformats NoDataInIntervalException
• 1629: recent change for setItem for datasets in Jython needed to account for sloppy codes that mishandled units.
• 1490: RichAscii delim was not set properly in one branch (unreleased bug).
• 1630: turn on flag that suppresses reset of TSB, which only affected some of the Hudson tests.
• 1566: rte_0610709130: load vap supports folder names with UTF-8 characters with local files and http.
• Juno Waves survey dataset showed a bug with the LanlNNRebin mode for spectrograms.
• copy plots down initializes DSF TSB properly. The bug would occasionally cause a fault in test100_042.
• ImageDataSource alpha channel was upside-down, relative to the other channels.
• CSV and ASCII table reader use skipLines consistently (instead of skip), and this is presented in completions.
• 1629: jython index bug, where two lists of indeces and one index are used.
• 1624: local .gz files are now uncompressed automatically. Create-parent-directories portion needed to be synchronized.
• in-lining weights calculation of histogram routine, after seeing use of WeightsDataSet doubles execution time.
• support compressed das2streams with zero-length compressed portion. This would hang before the fix.
• 1628: vap+inline:ripplesTimeSeries(100000) hangs in AutoHistogram.
• 1627: (rte_0554827090) reduce memory requirement by handling bundle dataset using just the last bundled dataset to keep track of fill.
• 1604: Tools Manager has overplot and plot below buttons.
• 1626: das2server dataset parameters are reset as id is changed.
• make block that creates /home/Dragon/autoplot_data/jython synchronized, to avoid errors seen during tests.
• das2stream adapter for rank 3 join of tables supports xTagWidth.
• 1625: writeHTML PngWalk crashes when --update is used. Thanks, Jon N.!
• 1569: check for zero in __add__ function, to support Jython sum operation.
• 1623: Jython index bug, where Jython list was misinterpretted as the single index for each dimension.
• 1621: wrong ordinal selected when table is sorted by ordinal.
• test005_014 corrected, as old site http://satdat.ngdc.noaa.gov/sem/goes/data/avg/$Y/A105$y$m.TXT appears to be gone.
• 1616: hooks in pngwalk to allow for scripting new apps. Add switch to disable automatic digitizer recording.
• rte_1585690286: code failed to check if null was found for units.
• 1541: Larry's Jython code where dom.dataSourceFilters would result in Concurrent exception

Older Bugfixes:

• 1618: botched bugfix breaks File->"Add Plot From..."->CDAWeb
• startDigitizer is a public method now, and startQC can be called multiple times without complaint.
• rte_0400646374: Index out of bounds when QC is running and pngwalk is empty. Thanks Ivar!
• rte_0370111585: SliceFilterEditorPanel.validateFilter assumed data was a QUBE.
• rte_0724070161: address inconsistent state associated with Time Series Browse observed by two people.
• corrections in digitizer (DataPointRecorder) when column sorts are used.
• corrections to pngwalk digitizer when digitizing plots without JSON metadata.
• catch annoying 403 error message that shows at LANL.
• 1582: tweak to the guessCadence logic shown by Rosie's data.
• 1594: HDF reader cannot find data in sweswi_data_12min_year1999.hdf. Now groups are supported.
• Rich ASCII code would run on any comment block containing {}, causing error message popup for some files.
• 1490: Improvements to Rich ASCII output following meeting with LANL folks.
• rte_1147829159: update the default pngwalk bookmarks, connect to github bookmarks file for easier maintenence.
• 1612: Mark's example showed where very negative Long for TT2000 was treated inconsistently and mucking up autoranging.
• updates to example scripts in svn.
• 1268: back out until this can be fixed carefully. (Bug is avoid download on the event thread in addPlot dialog)
• Recent change to Add Plot dialog would result in incomplete URIs and other strange errors.
• warning about using a timerange with a non-TSB getDataSet call removed. It wasn't effective.
• 0899: avoid extraneous load of data when createPngWalk is used to create single pngs.

Features:

• rfe509: horizontal and vertical slices support export data
• tearoffTabbedPane allows tab to be selected programmatically by title.
• rfe505: Jython setItem will correctly convert units as well as implicitly assign them.
• operations processor supports datums in histogram command.
• rfe504: install tool now checks the script for the label and description of the script.
• allow clients to declare new types of colorbars.
• UInt24DataSet added so that we can read pairs of two nybbles to get data from LWA telescope.
• rfe511: BinaryDataSource can now read data as array of nybbles.

Less recent features:

• NearestNeighborTcaFunction to support using calculated data for ticks. Thanks, Kristoff!
• PNGWalkTool provides more hooks for new Jython apps, including magnifier and auto-increment.
• planet.physics.uiowa.edu is the default das2server now.
• Refactor HTTPFileSystem to make the code a little easier to follow.
• PNGWalkTool provides a method for programmatically adding QC metadata.
• HDF reader supports structures.
• PngWalkTool digitizer now includes column identifying the image.
• PngWalkTool provides mechanism for extension, where scripts can get events and can plug in small GUI.

Known issues:

• see Source Forge

This completes the feature set for the new production branch. No major features will be added before branching this version. A production branch will be started in a couple of weeks, and the "Start Autoplot" link at the top of autoplot.org will run this version. Feedback on the function and quality of this release is greatly appreciated!

Data is input from:

• ASCII files (.dat,.txt)
• CSV files (.csv)
• Binary files (.bin)
• CDF files(.cdf)
• Excel spread sheets (.xls)
• das2Streams (.d2s or .das2Stream)
• ViRBO's TSDS server
• CDAWeb data server (vap+cdaweb:)
• das2servers (vap+das2server:)
• QStreams (.qds), QDataSet on a das2Stream
• Plasma Wave Group das2 Server
• NetCDF files (.nc,.ncml)
• HDF5 files (.h5,.hdf5)
• DODs servers (.dds)
• Cluster Exchange Format files (.cef)
• Fits Format Files
• Images (.jpg, .png, .gif)
• Wav audio files (.wav)

This is not complete, and limitations include:

• Quantities of rank 3 dimensionality (e.g. flux(Time,Energy,pitch) are plotted by slicing.
• Quantities of rank 4 dimensionality (e.g. flux(Time,Energy,azimuth,sector) are plotted by slicing twice.

autoplot.org is the wiki website, and contains more information than this launch page.