Autoplot Application (20140613b)

Butterworth filter verification.

Autoplot attempts to allow plotting of data from many data sources, including local files and OpenDAP servers. Datasets are identified with URIs, and by the URI's extension a module is used to load the data into the internal data model. Data is then displayed by guessing the rendering method and axis ranges to provide a reasonable, useful view of the data.

This is the Autoplot2014a branch. This is an update to the production version containing bugfixes and minor new features. Java 6 is now required.

Single-jar version, autoplot.jar, can be run without network access and from the command line. Note CDFs are only supported via the Java-based reader and Autoplot will attempt to use it when the C-based version is not available.

Bugfixes (2014-06-13):

• vapfile:http://server.gov/vapserver?data0=... works now
• more bugfixes to support v1.08 vap files (auto renderstyle)
• setStatus would often push extra event on to event queue.
• clean up for drag and drop URIs to make things work on Mac and Windows.
• 1210: http user agent was not indicated in web server logs.
• 1209: nextprev scan thread can spin in endless loop
• 1208: occasional hang that seems to be associated with TimSort bug
• 1205: aggregate all breaks with jyds script with timerange param
• File->print is performed off the event thread because this can be slow or even hang on Linux.
• 421: double load of Das2Source caused by misformatted URI
• "getting started" dialog had unused browseLocal branch that was broken
• bug in slicing would loose TCA metadata from das2server in d2s, loosing fill value.
• dataVisible and layoutVisible flags were not set properly new window, so these tabs were not properly initialized.
• isNotFile corrected to match "file:/home..." as a file in automatic completions-based editor.
• rte_0829456451: file:/home/... instead of file:///home/... would result in null pointer exception with completions-based GUI. Thanks Chris!
• bug 1195: Akebono MCA data script shows memory leaks. Thanks Takahiro!
• trim out indented blocks in removeSideEffects
• check for invalid URLs in Das2DataSource editor panel.
• aggregate one (just the address bar) no longer resets the range
• vap+orbit: URIs work with Automatic data source editor GUI.
• append( FDataSet, DDataSet ) would fail
• support $x in timeformat parameter of ascii parser.
• null pointer exception in logging because of misuse of logger.log( level, template, parms )
• fix occasional index-out-of-bounds seen when scripting by making defensive copy of application state.
• rte_1786254278: disable the pngwalk navigate menu when there is no sequence loaded. Thanks mariahelgacosentino!
• das2Stream "no data found" exception handled properly
• current das2server is set on the event thread after Chris showed where sloppy code would occasionally fail.
• newer Axis scan/step code could get stuck in a infinite loop when invalid bounds were returned (Cindy's data showed this)
• ascii table editor wasn't resizing with the GUI.
• EFW produced a CDF file with no depend 0, this should work instead of RTE.
• boolean properties failed in completions because of "isProp" instead of "getProp"
• set unit increment on filters list scroll bar on Filters panel.

Older Bugfixes:

• default vap version v1.07 was not set properly, and vap files v1.06 by default.
• doResetRanges no longer throws RuntimeError when component process string fails, allowing ECT file with missing channel to load.
• MouseModule messages don't clobber Autoplot messages like "plotElement_1 selected"
• autoRanging was always turned off when loading in vaps, making it more difficult to produce vap files dynamically. v1.08 introduced.
• 1184: reduce use of history.txt for repeat URIs motivated by filePollUpdating.
• 1183: clean up old code that was mis-managing transition from stack of series plots to spectrogram fails (Akebono MCA)
• addTab adds the component in a scroll pane, to protect the rest of the GUI from scripts. (Sebastian's digitizer was huge.)
• putProperty checks isImmutable flag and copies dataset if it is not mutable.
• 1182: CDF missing component labels resulted in editor failure
• Butterworth high-pass filter scale factor corrected.
• Survey places where locking is done without a try/finally block, which can hang the application.
• imax (index of the maximum value) was incorrectly coded to use MIN_VALUE (1e-308) instead of NEGATIVE_INFINITY
• Rob R pointed out RTE when ftp:// is entered and inspect button pressed.
• 440: any FileFilter needs to check for null from file.toString(). The DataPointRecorder didn't do this, Bill pointed out.
• don't print a warning when EnumerationUnits have the same ID. This happens with "default" and is harmless.
• 1180: use autoranging consistent with Series for Scatter
• Craig pointed out where makeAggregation was converting 20130402T00 to $Y$m$d-$H (T to -).
• rte 0948397190: runtime error in HDF5 reader when file is not found.
• 1129: experiments with rendering after we would still see occasional failures.
• lanlNNRebinner supports time-varying Y tags.
• FUNCT or FUNCTION in C-implementation of CDF reader
• improve feedback with errors from VOTable reader.
• FileStorageModel had an bug where getFiles would return multiple references to the same file, since 2014/3/14
• FileStorageModel was missing getField to make clean way to get the version ($v)

Features:

• export to xls allows append and more ordinal data
• experimental secret clean cache feature introduced.
• 351: save previous entries to jython getParam forms. Allow address bar trigger to contain parameter settings.
• add weights to output of rebinBundle.
• improve feedback while scanning local cache.
• add allow embed in vap file dialog. This works well enough.
• jython "show usages" action added
• jython script can be run from any Autoplot window, ask if switch context.
• move lookupUnits to Units class from SemanticOps, introduce lookupDatum to make standard spot for this logic
• das2streams allow the monitor taskSize to be set in a comment.
• put in filesystem completions when the method is "File" constructor
• rfe 361: give the user the option of resetting the main window when a script is run from a second window.
• rfe 337: Reset timerange when using URIs from bookmarks and recent history
• add code to get the PID to aid in debugging with JVisualVM
• keep track of reason why filesystem is offline.
• add option to disable scan (instead of step), which can cause problems with batch processing
• add monitor use to example codes in Jython editor.
• when zoomX is selected, mouse wheel will also only zoom in X.
• make it less likely to select a long, narrow box, which messes up ranges.
• rfe 363: improvements to completions in the editor.
• rfe 364: propogate timetags through findex and interpolate.
• support rank 2 DEPEND_1 in CDF export
• allow "file:" prefix in writeToPng
• add insert new plot above
• add break into stack of plots
• Ctrl-F is bound to Fix-Layout feature
• writeToPng allows "file:///tmp/png/foo.png" because it's easy to assume "file:" would be okay (Takahiro)
• add showYPosition(false) to the connector between two plots (Kristoff).
• 363: improvements to completions in Jython editor, including JOptionPane.s<C>
• About Autoplot dialog shows build server location.

Less recent features:

• add support for export to CDF use pure-java CDF library.
• "about autoplot" reports the Hudson/Jenkins URL where the code was built.
• 356: allow DELTA_PLUS and DELTA_MINUS in CDF to be empty string, so that master file can override incorrect value.
• 357: DataPointRecorderNew allows the columns to be pre-set with defaults and old files merged in.
• 358: cull repeat entries to history.txt to just first and last, filePollUpdates was adding an entry each cycle.
• export to CDF formats labels for rank 2 bundles
• Minor cleanup of html table reader
• completions suggest $Y$m$dT$H$M now
• FillPollUpdating works with http for single files
• In makePngWalk script, setting batchUri parameter in main method sets useBatchUri to true. This mode is in tests now.
• DataPointRecorderNew supports events lists, will replace DataPointRecorder
• rank 0 DDataSet.putValue checks rank because it's easy to forget the index
• SparseDataSetBuilder checks that setLength was called.
• Units.parse supports parsing 0x0000FF as 255, to support events lists.
• parseISO8601Range now throws ParseException, before this was a runtime exception
• introduce butterworth filter
• introduce indexOfBundledDataSet
• draw a box on the layout tab to hint at multiple selections, make the handle easier to grab
• EventQueueBlocker_1 renamed to EventQueueBlocker. FileStorageModelNew renamed to FileStorageModel.
• EnumerationUnits are always backed by a String context, was the class.
• enumerationUnit entries can be defined within the qstream as they occur.
• FileStorageModel has getFilesFor( String[] ref )
• FileStorageModel has getField("v") to retrieve version and other custom fields.

Known issues:

• see Source Forge

This completes the feature set for the new production branch. No major features will be added before branching this version. A production branch will be started in a couple of weeks, and the "Start Autoplot" link at the top of autoplot.org will run this version. Feedback on the function and quality of this release is greatly appreciated!

Data is input from:

• ASCII files (.dat,.txt)
• CSV files (.csv)
• Binary files (.bin)
• CDF files(.cdf)
• Excel spread sheets (.xls)
• das2Streams (.d2s or .das2Stream)
• ViRBO's TSDS server
• CDAWeb data server (vap+cdaweb:)
• das2servers (vap+das2server:)
• QStreams (.qds), QDataSet on a das2Stream
• Plasma Wave Group das2 Server
• NetCDF files (.nc,.ncml)
• HDF5 files (.h5,.hdf5)
• DODs servers (.dds)
• Cluster Exchange Format files (.cef)
• Fits Format Files
• Images (.jpg, .png, .gif)
• Wav audio files (.wav)

This is not complete, and limitations include:

• Quantities of rank 3 dimensionality (e.g. flux(Time,Energy,pitch) are plotted by slicing.
• Quantities of rank 4 dimensionality (e.g. flux(Time,Energy,azimuth,sector) are plotted by slicing twice.

autoplot.org is the wiki website, and contains more information than this launch page.