Autoplot Application (20160317a)
Fun with the pool over spring break.
Autoplot attempts to allow plotting of data from many data sources, including remote CDF files and OpenDAP servers.
Datasets are identified with URIs, and by the URI's extension, data are loaded into the internal data model. Data are then
displayed by guessing the rendering method and axis ranges to provide a reasonable, useful view of the data. Data can then
be additionally processed with other operations like smooth and FFT, and Jython scripting provides a capable programming environment.
This is the Autoplot2015a branch. This is an update to the production version containing bugfixes and minor new features.
Java 7 is now required. Java 8 will be required sometime soon.
Single-jar version, autoplot.jar, can be run without network access and from the command line.
Experimental Mac Release is available.
Bugfixes (2016-03-17 r18515):
- 1536: CDF Themis virtual variables slice1=i doesn't work. Thanks Cindy!
- Orbit plot type now has style tab editor.
- automatic completions-based GUI doesn't show "null" when label is not available for option. Show documentation.
- rte_1240063716: error: java.lang.String cannot be cast to org.virbo.dataset.QDataSet seen at Juno meeting. Thanks Rob!
- corrections to the AutoplotDataServer, where "NoDataInInterval" was not thrown properly in reduce.
- checkStatusLoop was not started in child applications, so the busy indicator used the old logic.
- bugfix where QStream had references to data that could not be resolved would prevent aggregation of JADE data.
- bugs when loading vap files with time series browse datasets that are not visible with a time axis.
- FilterChainPanel now limits the flashing of intermediate state by allowing 300ms to process before the color is changed.
- 1531: BufferDataSet append assumed compact data.
- swap needs to turn off TCAs. Experiment with fixLayout.
- 1530: support error bars in X, add control to turn off error bars with drawError=F. Thanks Larry!
- CDF where constraint works when data is rank 4 and filter data is rank 0.
- 1524: mashup handles when dataset loads fail. thumbnails are calculated more independently of one another.
- 1526: PDS/PPI didn't support quoted ISO8601 times
- annotation "point at" operation doesn't move the annotation, so "point at" and "move" are orthogonal operations.
- turn on timerange editor when vap+inline: is TimeSeriesBrowse
- events bar autoranging of one time datum should expand out to one hour duration
- the vertical slice "dump data set" now forces the .d2s extension so Autoplot can read this result.
- 1528: add an invisible plotElement when adding a hidden plot, so it can be turned on and provide overlays.
- vap+inline:3 now returns a rank 0 scalar.
- rte 1695465728: single-tab tear off on Mac would result in RTE.
- setDepend0Cadence would not properly set the units
- webstart on Linux/Mint creates a launcher that likes pass in %U when there is no file argument. -open xx (with any two characters) is now ignored.
- 1517: Sebastian's script hangs when calling DataPointRecorder.save
- 1515: remove extraneous user permissions dialogs caused when user@ added to it automatically.
- add PENDING_CHECK_REJECT so that the waitUntilIdle function properly detects that the application is not idle. Thanks Kenzie!
- improve feedback while reading CDF file data, MMS has large datasets that take non-trivial time to extract
- update the crosshair digitizer when we install RBGImageRenderer, so that R,G,B values are reported.
- treat http://user@... the same as http://... when credentials are needed either way. PNGWalkTool had a bug where user@ was added automatically, causing a second login dialog.
- openBrowser code assumed that the Java 6 system could be used, but it isn't supported on Linux (Mint).
- 1512: I/O Error reading CDF files on 32 bit machines. Fall back to alternate method for opening files.
- RGB indication on crosshair renderer corrected for images without DEPEND_1 tags.
- 1511: resetting the data set URI in a stack of plots would reset the timerange. Thanks, Bill, for pointing this out!
- relax timeout on read to 60 seconds, because sometimes the das2server would take more 10 seconds before sending out data.
- 1510: pngwalks of aggregations of sparse data, limitation is no longer needed, and we just don't look for gaps when this would be impractical.
- correction to PNGWalkTool thread sync bug, where check failed to see that thumbnail wasn't ready.
- keep track of separate control for HTTP read timeout, before there was just timeout for connection.
- pngwalk write to html corrected to make pngwalk more portable.
- 1506: allow subsequent repeat of lsd to reset the multiplier, fixing $Y$(m,delta=4)/$Y$m$d.dat
- grayPeaks didn't draw in StackedHistogram mode, because the lightGray.brighter() would result in white.
- undo operation was performed on event thread, which could easily lock the application
- CDAWeb result was missing CACHE_TAG, so result would be reloaded when it wasn't needed.
- CDAWeb uses web service to more accurately identify needed files.
- 1509: do sanity check to make sure new dialogs are not positioned outside of screen bounds.
- export to rich ascii will allocate name when dataset doesn't have name property.
- Screenshots tool would clip the pointer when it was partially outside of the GUI boundaries.
- 1437: export to rich ascii must preserve as much metadata as possible
- check for missing/problem with Rich PNG JSON, Where Dan's pngwalk contained bad metadata that showed in a demo at AGU.
- dashup tool gets quick-n-dirty rename variable code
- screenshots tool would clip the pointer when it was on the edge of window
- time unit t2000 works in das2stream again.
- clean up DataPointRecorder's use of threads, to avoid RTE.
- 1500: two autoplot script editors, pointing at same file, detect changes incorrectly
- 1498: pngwalk digitizer jumps ahead when delta=15 should have been used
- add typographicPoints unit, and correct meters to inches conversion, which was using inverse.
- bugfix: offline status wasn't respected in one code, causing hang.
- Netbeans is able to compile dasCore again, the project metadata was messed up.
- set the bounds properly when fontSize is used to enlarge font. Thanks Rich!
- add code to ensure that ephemeris data is monotonic and non-repeating.
- 1495: detect when maximized state should be used when restoring .vap files
- minor improvements to status bar messages
- 466: copy-n-paste plots. Feature now does error checking
- Updates to Rich ASCII parser to support Firebird files.
- "getParam (" wasn't parsed properly when generating GUI. Thanks Rich!
- Aggregation listings constrained to timerange, when timerange is in the URI.
- das2stream supports yFormat property
- annotations now visible in pngwalks and with the server.
- qds formatter also handles BIN_MINUS and BIN_PLUS
- corrections to autorange hints
- annotation bounds were not set properly for pngwalks and with the servlet.
- formatDataSet to CDF on Windows with Webstart failed because of move, so we fall back to copy.
- rte 1586184923: user exceptions were confusing because RTE dialog showed
- try/finally block used with web communication to get mime type.
- 1486: minor corrections to layout tab to improve usability
- corrections to Dashup tool caused by minus sign vs negation in Jython AST.
- "and" and "or" must be implemented as inline operators in Dashup tool
- introduce rank0 transform methods to support Jython rank 1 iterator returns rank 0 datasets instead of doubles, breaking Juno/Waves planning script.
- rfe 457: add hidden plot would intercept mouse events, making the connected plots difficult to work with.
- rfe 464: update the initial colors.txt template so that it is not confusing.
- Orbit mode allows tick label size to be controlled.
- Events list GUI is now two-column table. Thanks Chris for the suggestion!
- 479: consider if interpolateMod should allow range (e.g. -180 to 180).
- html table data source no longer requires a header row, first row will be interpretted as headers if no numbers are found.
- DataSetBuilder nextRecord method accepts array of doubles, as well as array of objects (datums)
- 1530: add control switch for error bars.
- support now-P1D/now+PT1H
- plug-in filters can now validate the string, allowing mousewheel on pipes string to be limited.
- BinaryDataSet supports "time21," fore where the ascii bytes of an ISO8601 time are found within binary PDS data.
- 476: control for data source filter's filter string is provided.
- linspace() supports rank 0 datasets.
- rank 4 random functions
- vap+inline: automatically supports timeSeriesBrowse
- events bar autoranging of one time datum should expand out to one hour duration
- add Units.nT.
- Autoplot servlet allows multiple colors to be specified for stack of traces.
- Data Mash up is now accessible from menus. Improvements to mashup including thumbnails.
- Autoplot servlet experiment with checking for exceptions in the plot and putting them in the response headers.
- 472: allow $x to be used in aggregation without time fields.
Less recent features:
- About screen includes a bullet explicitly identifying 32bit Java and provides a link to fix this.
- html version of pngwalk uses precalculated thumbnails.
- 470: DasProgressPanel shows the thread name to improve serviceability
- CDAWeb now shows availability.
- StackedHistogram gets style panel
- StackedHistogramRenderer name fixes
- CDF editor dialog indicates internal type to improve transparency and support creating new files.
- 468: unplot can use iso8601 times.
- Canvas size GUI supports "7 inch" or "30 centimeters" for dimensions, not just pixels.
- add LSpec rebin that uses bundle of buckshot (T,L,Z) for the input.
- 467: add accessory to allow override timerange when loading vap files.
- das2server items can now be a comma-delimited list of indeces.
- Single filesystems can be reset now, double-clicking in FileSystem manager will trigger reset.
- format to QDS supports BIN_PLUS and BIN_MINUS properies
- 443: autorange hint log=T supported, corrections to initial implementation of center and includeZero
- 466: copy-n-paste plots.
- script editor allows entry into data source editors with right-click→actions→"inspect URI"
- ISO8601 parser now supports time zone offsets.
- Clean-up and logging of OpenDap code so more predicatable failures can be studied.
- nc4 extension is recognized as NetCDF.
This completes the feature set for the new production branch. No major features will be added before branching this version.
A production branch will be started in a couple of weeks, and the "Start Autoplot" link at the top of
autoplot.org will run this version.
Feedback on the function and quality of this release is greatly appreciated!
Data is input from:
- ASCII files (.dat,.txt)
- CSV files (.csv)
- Binary files (.bin)
- CDF files(.cdf)
- Excel spread sheets (.xls)
- das2Streams (.d2s or .das2Stream)
- ViRBO's TSDS server
- CDAWeb data server (vap+cdaweb:)
- das2servers (vap+das2server:)
- QStreams (.qds), QDataSet on a das2Stream
- Plasma Wave Group das2 Server
- NetCDF files (.nc,.ncml)
- HDF5 files (.h5,.hdf5)
- DODs servers (.dds)
- Cluster Exchange Format files (.cef)
- Fits Format Files
- Images (.jpg, .png, .gif)
- Wav audio files (.wav)
This is not complete, and limitations include:
- Quantities of rank 3 dimensionality (e.g. flux(Time,Energy,pitch) are plotted by slicing.
- Quantities of rank 4 dimensionality (e.g. flux(Time,Energy,azimuth,sector) are plotted by slicing twice.
autoplot.org is the wiki website, and contains
more information than this launch page.