Autoplot Application (20151028a)

The application timerange control appears on the axis tab now.

Autoplot attempts to allow plotting of data from many data sources, including local files and OpenDAP servers. Datasets are identified with URIs, and by the URI's extension a module is used to load the data into the internal data model. Data is then displayed by guessing the rendering method and axis ranges to provide a reasonable, useful view of the data.

This is the Autoplot2015a branch. This is an update to the production version containing bugfixes and minor new features. Java 7 is now required. Java 8 will be required sometime soon.

Single-jar version, autoplot.jar, can be run without netwok access and from the command line.

Bugfixes (2015-10-27 r17822):

• rte_2066281934: catch IndexOutOfBoundsException when trying to parse "...+...+...", now throws parse exception.
• rte_1417924229: transitional state produced NullPointerException
• 1478: slice and unbundle operations properly preserve units, eq and ne operators improve support for nominal data.
• PNG support corrects problems with alpha channel when reading and formatting PNG files.
• TimeParser $(subsec) had a bug with rounding.
• 1473: create pongwalk by orbit without rescale fails
• 1472: HTTPFileSystem fails to query for username after typo
• 1470: de-internationalize the three codes that were accidentally internationalized.
• 1468: Chris pointed out the setting for the default browser in the settings, which pointed out that Java handles this now, since Java6.
• 1465: check before closing an editor with changes.
• invokeScriptSoon dialog would leave progress monitor hanging if there's a exception thrown in the script..
• remove extra question mark when reporting data was exported.
• corrections to hrinterval and time format, correctly using the stop time and supporting end modifier.
• clean up address bar as vap is loaded.
• "2014-001 to 2015-001 UTC" is supported now.

Older Bugfixes:

• 1462: result of guessCadence mishandled in hugeScatter rendering mode
• 1463: null value in USER_PROPERTIES would result in NullPointerException
• 1440: vap+inline:ripp<C> works now.
• null value in USER_PROPERTIES would result in NullPointerException
• Export to CDF didn't properly support append keyword.
• HugeScatter wouldn't to paint the exception if that's why it didn't have a dataset
• vap+inline each line needs to be URL-decoded. Add monitor so downloadResourceAsTempFile can be used.
• if the first filter was an implicit unbundle, then it Filters GUI wouldn't update properly
• FFTPower GUI supports slide of 8 (7/8 overlap)
• use case where GCD didn't work to calculate the needed cadence for ephemeris ticks, fall back to difference
• CACHE_TAG was getting through the subset operator, causing strange caching problems.
• copyTo needs to reset the limit of the new buffer, I believe because of append.
• handle case where there is a CONTEXT_0 with Units.us2000 and DEPEND_0 with Units.seconds in fftFilter.
• 1458: valid function would return result with fill_value of its input for reference, but this could be zero, which is incorrect. SUGGEST_FILL property added.
• properly size the x-axis when running headless jobs with TimeSeriesBrowse-capable datasets.
• properly format the leap second with 23:59:60
• support for huge (>2GB) CDF files, switch to 20150828 CDF library version
• improvements to Huge Scatter mode allow 67 million points to be plotted interactively (somewhat).
• sloppy code in spinner progress bar be left behind when loading ephemeris.
• refactor ascii parser to make it more consistent.
• screenshots tool includes Windows as well as Frames when masking for privacy.
• 1453: AggregatingDataSource now uses the time constraint when identifying the delegate file for retrieving metadata, avoiding extraneous load.
• 1448: allow capitalized extentions like ".PNG" when explicit filenames are used.
• 1433: double check on timetags was too restrictive and throwing out too much data from RBSP waveform data, George pointed out.
• 2015 leap second was applied 64 seconds too early.
• Ephemeris ticks now use resolution when tick labels are sourced from TimeSeriesBrowse.
• PNGWalk "View in Autoplot" button will look at PNG metadata to extract the timerange.
• More zaxis properties are loaded in das2streams.
• 1165: create pngwalk uses user prefs for grid and other options.
• xls files would drop the last record.
• xls files would have times shifted by local time zone.
• null pointer when no color was selected in pick colors dialog on Linux.
• rank 3 data in QStream wasn't properly read in.
• Autoplot dialogs like the "additional operators" don't have goofy default question mark icons.
• output of contour hard to use, breaks in non-monotonic data. Thanks Sebastian!
• rte 2091983907: thread safety in pngwalk. Thanks usuario!
• rte_1604359977: limit the total number of filePollUpdates threads that can be started to 10000. Thanks, Bob!
• completions within a function call are corrected. plot( ripples<C> ) corrected.
• 1439: add "delete selected items" to edit menu of DataPointRecorder
• Sebastian demonstrated that QStream with DEPEND_0 having DEPEND_0 would cause failure.
• Sebastian demonstrated that embed data failed when the URI was ''.
• Kristoff demonstrated that embed data doesn't work when the file came via http.
• 1442-6: camelCase for the plotx command.
• 1442-7 symbolSize instead of symSize
• 1442-10: height and width boxes on mac
• resizeForCanvasSize didn't consider that the canvas may already be disconnected from the application
• 1433: Kristoff pointed out that when records contain fill or are not long enough for the interval, then the ytags would be left with a different length.
• bugfixes with rich ascii.
• 1088: clip off the extra stuff when we do completions in the middle of a parameter.
• tolerate mixed case DISPLAY_TYPE metadata in CDF files.
• 1434: metadata propagation through operators is inconsistent, inconvenient
• 1433: allow waveform cadence to change in fftPower.
• 1432: list of list to declare double arrays (and rank 2 datasets) in Jython.
• 1431: "Cannot create PyString from null" popup in editor.
• Ops.labels renamed to Ops.labelsDataset, to avoid name clashes.
• completions within command, like plotx(<C>) where we need to look up the documentation for plotx.
• more support for color argument in plotx
• plot and plotx command are finally one and the same.
• check for null in setSymbolConnector, setColor, setFillColor.
• cdfn and cdfj loggers combined to "cdf" since there's just one cdf source.
• bug where cdf editor had no visible variables because of bugs with CDF editor (Thanks Mark and Matt).
• bug where labels were array [N] instead of [1,N] corrected.
• ds[rank0ds] handled in Jython
• flattenRank2 handles rank 1 DEPEND_1.
• correction where FFT window was applied twice corrected.
• qds supports BIN_PLUS and BIN_MINUS properties, WEIGHTS.
• verify the qstream packet length is less than 1000000, to aid in debugging.
• warning icon persists, where it was cleared mistakenly before.
• scriptContext formatDataSet allows progress monitor.
• imports are allowed with server port again.
• updates to many of the example scripts.
• bugfixes in block from 2015-07-15 to 2015-07-26
• FFTPower would apply the window twice, affecting the data. Thanks, Chris!

Features:

• rfe456: jython editor detects external changes
• dashup can be enabled with environment variable "enableDashup=true" in config/system.properties.
• rfe453: axis tab has timerange editor for when the timerange isn't an axis.
• rfe454: support arbitrary rank data with HDF5 plugin.
• pngwalk allows &timerange= in URI to constrain the image timeranges.
• 451: keep track of running script so buffer doesn't swap
• 345: first go at annotations
• improvements to $(enum) in timeranges, providing code to read out the enumerated values.

Less recent features:

• 449: bitwise operators in QDataSets.
• export to HDF supports append
• trim operator has GUI now. Thanks Armond!
• ascii formatter now allows header=none and doDep=F, depend0Units=seconds+since+1970-01-01 to support import to Matlab and IDL.
• ascii parser depend0Units=seconds+since+1970-01-01
• add ctrl+0 to reset the font.
• 446: add binsWithin function, motivated by Ivar's script.
• ds[rand()*100]= 1 now works.
• plot command [xy]drawTickLabels=False added
• 445: speed enhancements for lanlNNRebinner, experiments with extent
• setWindowLocation command. commands for multiple windows in scripts.
• 441: allow non-time aggregations like data_$x.dat
• 444: data source editors may have list of reasons why the URI was rejected
• 443: autoRangingHints property allows more control over autoranging (https://sourceforge.net/p/autoplot/feature-requests/443/)
• axes have dialog supporting units change to convertible units
• 442: special support for MMS data website, which has uses login and cookies
• Add native memory available to the "About Autoplot" dialog.
• http://autoplot.org//developer.datasetToString clean up
• .pngwalk file has filepattern to support more pngwalks and subdirectories
• URI about:classpath added for debugging
• 441: allow $v and $x to trigger the split index.
• Introduce "flaky server" code to model slow servers so Autoplot can deal with them more effectively.
• 1448: explicit control of output filenames in pngwalk.
• where keyword support in HDF5
• HDF5 now has an editor panel.
• improvements to completions in Jython editor panel.
• plot named parameters have consistent capitalization and index named parameter introduced.
• embed data found in data sources that does not have a URI ("vap+internal:")
• opposite property for axes.
• support $x/$Y_$m for file storage models
• rfe 439: Need standard method for getting a second Autoplot app in a script.
• avoid filesystem completions with inline, cdaweb, or pdsppi.
• completions are working somewhat with vap+inline.
• within parameter added to CDF
• plotx command legendLabel named parameter added.
• plotx linestyle added.
• improvements to DashupTool.
• update the port reported when the server is on, to support automatic selection of port with --server=0.
• support Units.RGBColor for the units of the colorBy dataset.

Known issues:

• see Source Forge

This completes the feature set for the new production branch. No major features will be added before branching this version. A production branch will be started in a couple of weeks, and the "Start Autoplot" link at the top of autoplot.org will run this version. Feedback on the function and quality of this release is greatly appreciated!

Data is input from:

• ASCII files (.dat,.txt)
• CSV files (.csv)
• Binary files (.bin)
• CDF files(.cdf)
• Excel spread sheets (.xls)
• das2Streams (.d2s or .das2Stream)
• ViRBO's TSDS server
• CDAWeb data server (vap+cdaweb:)
• das2servers (vap+das2server:)
• QStreams (.qds), QDataSet on a das2Stream
• Plasma Wave Group das2 Server
• NetCDF files (.nc,.ncml)
• HDF5 files (.h5,.hdf5)
• DODs servers (.dds)
• Cluster Exchange Format files (.cef)
• Fits Format Files
• Images (.jpg, .png, .gif)
• Wav audio files (.wav)

This is not complete, and limitations include:

• Quantities of rank 3 dimensionality (e.g. flux(Time,Energy,pitch) are plotted by slicing.
• Quantities of rank 4 dimensionality (e.g. flux(Time,Energy,azimuth,sector) are plotted by slicing twice.

autoplot.org is the wiki website, and contains more information than this launch page.