Autoplot Application (20170805a)
MMS FPI data causes problems with the default spectrogram interpolation, due to interleaved sweeps. This has been corrected by first interpolating in Y then X.
Autoplot attempts to allow plotting of data from many data sources, including remote CDF files and OpenDAP servers.
Datasets are identified with URIs, and by the URI's extension, data are loaded into the internal data model. Data are then
displayed by guessing the rendering method and axis ranges to provide a reasonable, useful view of the data. Data can then
be additionally processed with other operations like smooth and FFT, and Jython scripting provides a capable programming environment.
This is the Autoplot2017a branch. This is an update to the production version containing bugfixes and minor new features.
Java 7 is now required. Java 8 will be required sometime soon.
Single-jar version, autoplot.jar, can be run without network access and from the command line.
Mac DMG Release is available.
Bugfixes (2017-08-04 r21182):
- Don't draw lines at tick locations when "opposite axis" is used.
- 1871: many timerange text fields don't use history. Thanks, Dan!
- flattenRank2 always returns a rank 2 dataset, generating 0,1,2,... when DEPEND_0 or DEPEND_1 is missing.
- 1873: incorrect reset of slices filter. Thanks, Joe!
- 1869: restore function of pngwalk:http://jfaden.net/hudson/job/autoplot-test070/*/artifact/test070_008.png.
- Ivar's script caught where new code didn't handle fill properly because BUNDLE_1 hid it.
- convertUnitsTo now checks that a BUNDLE_0 is not present, which might have old UNITS within. Thanks, Ivar!
- 1868: where with append triggered a branch where an immutable dataset's internal buffer was accessed with write code. Thanks, Kristoff!
- 1867: AverageTableRebinner interpolate Y before interpolating X, to support MMS FPI interleaved sample modes.
- rank 4 support was never implemented for ListIndexDataSetIterator, so inline with where would not work.
- 1866: catch CDF recCount=0, which causes reader to enter infinite loop, triggered when [200:300] was used and a non-time-varying DELTA_PLUS was found.
- 1801: dataset iterator is inordinately slow, put in optimal loops for collapse of rank 4 data.
- rte_0164874230: index out of bounds on Slice operation editor panel. Thanks, Kris!
- bug where rank 2 DEPEND_3 wasn't sliced properly is corrected. Thanks, Arya!
- 1865: add a little hysteresis to the LanlNNRebinner, so it doesn't flop back and forth with axis changes.
- small round of findbugs, including one real bug in DigitalRenderer with xaxis units.
- rte_1708171669: space in CDAWeb filter would cause RTE.
- 1860: fill data within bundles of data is marked as fill in the metadata tab.
- fix the setUnits filter, where it was using an old method. Thanks, Kristoff!
- 1858: extend autorange to include all data when bundled data has different units.
- method "BoxSelected" renamed to "boxSelected" to conform to Java and Java Beans conventions, and Jython refactoring supports old scripts.
- 1661: format to CDF corrected on Windows by doing extra garbage collections. Thanks, Kristoff!
- 1836: copy dataset internally when the mutable dataset is no longer mutable. Thanks, Kristoff!
- rfe528: star imports are supported. Thanks, Kristoff!
- 1836: ds.putProperty( QDataSet.UNITS, None ) would cause RTE. Thanks, Kristoff!
- 1853: copy over code from other makeCanonical branch to handle case where NullPointerException occurred.
- 1854: clean up code which relied on file.toString to construct URLs for the NetCDF library, which would break on Windows. Thanks, Kristoff!
- rte020756902: old bug where getAllUnits() once returned array of strings now returns array of units showed with recent refactoring. Thanks George!
- rfe528: old references to org.virbo.autoplot.AutoplotUI were not supported in jumbojar, because there were no direct references to the routine.
- unintentionally committed testing code prevented PNGWalk creation on Windows machines, since May 03, 2017. Thanks, George!
- improvements to events renderer to better support nominal data.
- HAPI client considers data cadence when requesting data.
- rfe528: correct warning message import being corrected when correction wasn't necessary.
- 1850: when .jyds scripts reject because of error, the red squiggle was placed on the incorrect line.
- 1849: QC save button cannot be enabled if all records have been QC-marked. Thanks, Kris!
- 1766: $(Y,end)$m$(d,delta=10) now works.
- 1836: ArrayDataSet.copy used getBack to get the backing array to determine the class, causing runtime error with new mutability checks.
- 1840: display data made a little less confusing. Thanks, Ivar!
- correction of templates in das2streams. Thanks, Chris!
- das2stream reader allows yunits to be missing, just print a warning.
- rte_2052946987: index-out-of-bounds caused by failure to update firstIndex, lastIndex when data is reduced and all data is outside of the visible region. Thanks, Kris!
- 1839: append of two rank zero datasets should be a rank 1 dataset. Thanks, Kristoff!
- rte_1917581137: bug which shows with x2go and Ubuntu Mate. Thanks, Kris!
- digital renderer allows %E for exponential notation.
- colorbar resize control points allow resize of the vertical direction. Thanks, Ivar!
- rte_0153443069: CSV parser editor GUI would fail with NullPointerException because handling was delegated a sloppy code. Thanks, Mimai!
- drag-n-drop from Thunderbird email "save attachment as" dialog fixed.
- rte_2119934227: put a link in the runtime error popup for OutOfMemory errors.
- rte_1890761962: index out of bounds in "near real time" Time range editor.
- allow different credentials to be used on different paths on the same server.
- 1818: getTimeRange( 1990/12/01, 1month ) would fail the pngwalktool because 1990/13/01 was not normalized (via carry).
- 1833: catch bad path when "orbit:file:///" used where "file:///" should have been used. There was code to trap this exception, but the wrong exception type was thrown. Thanks, Ivar!
- 1698: proper support for bundles in qstreams.
- rte_0110509841: old events scheme check (time,deltaTime,color,str) was broken since last production release v2016a_15. Thanks, Chris!
- 1831: hint that something is wrong when URI is rejected in add plot dialog. Thanks, Ivar!
- 1829: check for null when adapting das2streams.
- file:///.../*.png used in PNG Walk tool would clip off the first two characters of each filename's label.
- rte_0424767770: new code didn't account for path where walkImageSequence would be null. Thanks, Ivar!
- DataPoint comparison of non-time values made no sense, causing some points to disappear.
- 1818: pngwalk of Omni DST vs KP skips each December.
- 0656: cancel press in load example script dialog is correctly handled.
- 0595: sftp is now recognized, fixing Jython listDirectory command.
- Data Point Recorder sorting is disabled while data is loaded from a file, for speed.
- 1825: Data Point Recorder drops points when x-axis is not a time axis.
- 1821,0656: load example script dialog cancel didn't work.
- 1697: A second path resulting in waiting threads when FileSystem simultaneously created by multiple threads. Thanks, Seth!
- 1824: -1e30 is treated as fill in DataPointRecorder, to support files from old Das2 applications. Thanks, Darrelle!
- in das2stream reader, buffer capacity now grows on data packet reads as well as header packets. Thanks, Chris!
- '$Y$m$(d,span=4)/P4D' counts off days consistently. Thanks, Masafumi!
- --enableResponseMonitor command line switch function restored, added to AutoplotServlet. Thanks, Seth!
- jnlp needed Codebase attribute to run without complaint.
- TimeParser ISO8601-deriving code would fail because it dropped the Z.
- DataPointRecorder support for old digitizing files is improved.
- 1823: PNGWalkTool QC needs, including show just QC records. Thanks, Kris and Craig!
- small round of findbugs cleanup.
- select time by orbit would pick previous orbit if it overlapped slightly.
- 1820: occasional failures of test033, double checking for valid ticks.
- 1723: Wind waves data then dbAboveBackground showed where VALID_MIN was improperly propagated. Thanks, Masafumi!
- 1559: trim getDataSet calls to the time range requested, when the time axis is not visible.
- debug printing for 1814 accidentally left in production release.
- Update RBSP orbits to Feb 2017.
- DasAxis foreground color is exposed in the property editor, under the controller node.
- 585: experimental TriScatRebinner added, to draw spectrograms from scatter data using Delauney triangulation.
- support new scheme, rank2WaveformRank2Offsets in hugeScatter and series renderers, for analysis with Ivar's new CDF files.
- rfe528: after years of being stuck with the "BoxSelected" mistake, this is finally corrected to "boxSelected"
- IDL/Matlab interface includes method for clearing used memory. Thanks, Masafumi!
- start SAMP now adds a tab to show SAMP hub status. Thanks, Pedro!
- runtime error reporting includes a flag for JavaWebstart use.
- savToHdf.pro included to encourage use of HDF5 and discourage use of IDLSav files.
- 528: giant refactoring where org.virbo.autoplot is moved to org.autoplot, org.virbo.dataset moved to org.das2.qds, etc. More about this here.
- 580: time parser supports "P1D/lastday" to refer to yesterday, where lastday refers to last day boundary; also lasthour and lastmonth.
- 597: export to wav can now write files containing timetag mapping, to support Larry's animations.
- IDL/Matlab bridge supports bundles of data (coming from HAPI servers, for example).
- IDL/Matlab bridge can now be used with native Python (SciPy), using Python JPype package.
- fftPower of rank 3 join detects when results can be combined to make simpler rank 2 result.
- reducex supports rank 3 join.
- 1840: display data dialog allows copy points to clipboard.
- decimate function added, which keeps every Nth point. Thanks, Ivar!
Less recent features:
- formatting events dataset with colors to qstream uses hex format for RGB color.
- das2Server credentials manager has add method for scripts to insert credentials.
- rfe528: Jemmy codes are moved into Autoplot app from VATesting, so that it's easier for any one to create tests.
- Introduce JythonRefactory which will support old names after path names move around in v2017a.
- add FileSystemUtil.unzipFile.
- DataPointRecorder turn off the sorted property as data is loaded, since it is too slow and presumably sorted already.
- display data mouse module keeps track of which renderer was selected, to facilitate comparisons.
- Add action to write PNGWalk as an animated gif.
- Add tools menu to PNGWalkTool, and action to write PNGWalk to PDF file.
- Add handling for yscans without yTags at all, and those that use yTagInterval as well. Thanks, Chris!
- 573: pngwalk tool now supports zoom on image.
- isEventsList now requires that each endTime be greater than or equal to startTime, and is used in guessRenderType, to support (X,Y)->Z where Z can be ordinal data.
- ASCII table reader is able to read in (X,Y)->Z where Z can be ordinal data.
- ASCII table reader where now has matches clause, for example where=sc.matches('.*A')
- Editor panel has switch to turn off arrow annotations for tab, and to allow tab-is-4-spaces instead of completions popup.
- extent filter (which returns min,max of a dataset) added to filters.
- see Source Forge
- Bugs often have a four-digit ID and can be reviewed with https://sourceforge.net/p/autoplot/bugs/<ID>/.
- Features typically have a three-digit ID and can be reviewed with https://sourceforge.net/p/autoplot/feature-requests/<ID>/.
This completes the feature set for the new production branch. No major features will be added before branching this version.
A production branch will be started in a couple of weeks, and the "Start Autoplot" link at the top of
autoplot.org will run this version.
Feedback on the function and quality of this release is greatly appreciated!
Data is input from:
- ASCII files (.dat,.txt)
- CSV files (.csv)
- Binary files (.bin)
- CDF files(.cdf)
- Excel spread sheets (.xls)
- das2Streams (.d2s or .das2Stream)
- ViRBO's TSDS server
- CDAWeb data server (vap+cdaweb:)
- das2servers (vap+das2server:)
- QStreams (.qds), QDataSet on a das2Stream
- Plasma Wave Group das2 Server
- NetCDF files (.nc,.ncml)
- HDF5 files (.h5,.hdf5)
- DODs servers (.dds)
- Cluster Exchange Format files (.cef)
- Fits Format Files
- Images (.jpg, .png, .gif)
- Wav audio files (.wav)
This is not complete, and limitations include:
- Quantities of rank 3 dimensionality (e.g. flux(Time,Energy,pitch) are plotted by slicing.
- Quantities of rank 4 dimensionality (e.g. flux(Time,Energy,azimuth,sector) are plotted by slicing twice.
autoplot.org is the wiki website, and contains
more information than this launch page.