Autoplot Application (20150131c)

Aggregation undid all the benefits of the new CDF library. Here's a blooper from adding special support for aggregation.

Autoplot attempts to allow plotting of data from many data sources, including local files and OpenDAP servers. Datasets are identified with URIs, and by the URI's extension a module is used to load the data into the internal data model. Data is then displayed by guessing the rendering method and axis ranges to provide a reasonable, useful view of the data.

This is the Autoplot2015a branch. This is an update to the production version containing bugfixes and minor new features. Java 6 is now required.

Single-jar version, autoplot.jar, can be run without network access and from the command line.

Bugfixes (2015-01-31 rev 16192):

• look for AUTOPLOT_DATA/config/logging.properties
• bug 1327: blur URI causes das2server to loose resolution
• correct misspelling/typo of addCapability after years without noticing. Yeah code completion!
• getDataSet menu option completion now correctly checks for var= to see if the user has already corrected the name.
• 1326: aggregations are no longer guarenteed monotonic. See ensureMonotonic to correct.
• allow vap+qds:/tmp/foo.dqt to work. Before it was using the extension.
• bugfix: support eventsListUri in vap file.
• 1326: new aggregation monotonic rule causes too many problems. Initial code to just sort the data.
• das2stream reader detects when a qstream is input.
• Events List tool handles exceptions properly.
• 1321: improvements to the IDL bridge, after trying to read 5M points of magnetometer data. Thanks, Kris!
• 1321: IDL bridge preserves TT2000 longs.
• 1322: aggregation supports NIO BufferDataSets that come out of the CDF reader. (Whoops!)
• 1323: "no valid y tags" message caused by fill in DELTA_PLUS when DELTA_PLUS wasn't really needed. Thanks, Seth!
• 1317: result of fftPower hints that log scale type should be used.
• 1315: keep track of das2server key so the user is prompted just once per session.
• turn on word wrap in runtime error submit form.
• Thin client support finally catches errors instead of leaving the progress wheel spinning.
• corrections to FiltersEditorPanel, such as allowing for float arguments and medianFilter.

Older Bugfixes:

• remove the dtd from internal filters.xml, which was preventing use on Seth's computer.
• 1312: tearing off tabs before initialization results in tiny tab
• 1311: new user data tab enabled after the initial data load resulted in RTE
• Copy data to local variables after seeing nullPointerException in test002
• update to new CDFJ library which fixes synchronization problems
• energy channels were reported as ranges in MAGEIS plots, because of BIN_PLUS and BIN_MINUS.
• the new addFilter dialog to match Tools->Filters menu
• rfe407: deleting a filter would also loose track of the input dataset.
• slices GUI was not tested thoroughly
• 1308: Demo 12 never completes when Data tab is torn off.
• remove confusing prompt to save and show new script or cancel, when a new script is referenced
• In PNGWalk, set window ancestor properly for "Unable to find any images" message.
• plotx(2,'vap+inline:ripples(20)') would always plot in the 0th position.
• support fill in non-ISTP HDF files, using _FillVal metadata.
• 1306: automatically set the width of any time columns in export to Excel.
• remove stderr noise from WriteIDLSav.
• 1305: das2server URIs would loose their parameters, because TSB.getURI was called before getDataSet
• 1304: Partial vap support introduced bug where linestyle setting would be reset for time series browsing.
• synchronize cdf reads because of occasional RTE shown in test500.
• das2server reader _res switch is not sent to the server.
• rte_2124254968: Das2Server editor panel allowed folders to be selected, causing RTE. Thanks Dan!
• dashed property was reset when browsing to the next day.
• Add pow and sqrt to list of filters. Thanks Allison!
• Fix CDAWeb demo in thin client
• In PNGWalk, NullPointerException when image with time axis was digitized, but the granule didn't have a corresponding timerange, (e.g. *.png)
• 1303: jyds scripts with SimpleServlet. Allow relative reference to scripts/data/, introduce whitelist.txt.
• correct progress feedback in demo 1
• 1302: indicate busy when Autoplot is launched with URI. There was a ~5 second period where no feedback was provided.
• 1301: redo mechanism that resolves the same file downloaded from two Autoplot instances, by allowing simultaneous loads and resolving the results. Thanks, Bill!
• 1300: strange export data behavior on Mac, where somehow discovery mechanism was getting the wrong type
• 1299: where trimProperties would drop the original properties
• fix the labels for Seth's data with Rank 2 DEPEND_1, where stack of lineplots were not given unique labels
• 1251: reuse of monitors
• bugfix: pickFont can return null.
• revert the message about varying slice locations.
• fix hang when thumbnailer code would get stuck in infinite loop (with small canvas).
• fix failure to update when canvas size is set explicitly.
• bugfix: rank 1, one-element DELTA_PLUS in vap+cdaweb:ds=C3_PP_CIS&id=T_p_par__C3_PP_CIS
• bug with rank 1 data and data gap for ephemeris ticks resolved. Thanks Joe!
• 1098: improve feedback as to where the two timeranges options are coming from in the query.
• 1295: get rid of Thread.sleep use by using new java.util.Timer aptimer.
• contoursRenderer accepts pre-contoured data, like the output of the contours filter
• CDFJ getPlottable was returning the LABL_PTR_1 variables, when it shouldn't be
• 214: correct for bold and italic when printing
• bugfix: filters recentComboBox didn't show the new value. I think it has always like this, but I just noticed this with the new editor.
• restore inspect (browse) behavior for .jy files.
• 1291: filters loose ranging too easily
• bugfix where &script=foo.jyds would be lost when entering the Jython datasource editor
• cleanup of hudson tests test017 and test021.
• rfe405: a number of the hudson tests showed problems in corner cases.
• bugfix in fits where height and width were exchanged.
• rfe405: Kristoff pointed out where new code ensuring monotonic from aggregations would clip almost all the data.
• improve handling and transparency when PDSPPI is down
• 1290: jython script editor left in read-only mode
• Correction to the CDAWeb plugin URL used, thanks Tami!
• aggregation works with ordinal data again.
• reform just returns the dataset when there are no indeces with just one element, thanks Bill!
• minor corrections to d2s parsing when each plane has a name
• the result of aggregations will now always be monotonically increasing.
• rte_0253344792: catch code fragment unintentionally evaluated with highlite in Jython editor. Thanks Kristoff!
• CDF where supports rank 1 where with rank 2 data.
• script name appears in stack traces now, to aid in debugging
• Improve support for ordinal data in CDF where keyword.
• get completions only returns completions where plottable=true, so scripts can once again verify contents of CDF files.
• legend plot symbol for dotdash linestyle was barely visible. Thanks Kristoff!
• das2server with stream defect could leave client in endless loop
• 1284: round of cleanup with PNGWalkTool
• 1288: setDepend0Cadence supports join-of-qubes datasets
• bugfix: restore ctrl+mouse wheel pan behavior.
• bugfix in aggregation completion, where the path is not part of the completion.

Features:

• add Jython formUri to provide a nicer way to construct URIs.
• 411: add hook to check for logging.properties to make it easier to debug webstart.
• .pngwalk files now have a codebase parameter so that they can be used out-of-context.
• add |getProperty('DEPEND_1') filter so I can easily have a look at timetags.
• BufferDataSet, which uses native-memory-backed-buffers, moved from BinaryDataSource to QDataSet
• allow aggregation reduce to be used when resolution is null, because trim will still have an effect.
• SimpleServlet will figure out if URI is a vap file, to support the thin client and to match the Autoplot interface.

Less recent features:

• reference cache is enabled for new users.
• 1313: switch to new version of OpenDAP library. Support grids.
• 1310: add jython sleep command which is often imported
• support Time and TimeRange qualifiers in Das2Streams
• completions servlet will support the thin client.
• das2server data source code supports 401 and conventional authentication
• add intrinsic=true to das2server data source.
• add leap second for July 1, 2015
• update RBSP orbits to 2015-01-13
• Start more Jemmy GUI testing. Thanks Kenzie!
• logging cleanup
• Code cleanup in Java CDF library
• Filters editor panel provides small GUIs to control each filter.
• Add smoothfit function to filters.
• don't include iconified window area in CreateScreenShot code
• Introduce Jython writeToSvg to support future servers.
• Update to the new version of CDFJ library
• put in label to show the canvas size in inches or centimeters.
• minor improvements to getParam insert in Jython editor.
• add branch for total(dsr2,1) where dsr2 is a rank 2 dataset, to get BMag from components of huge magnetometer dataset.
• experimental reluctantLegendIcons=true (autoplot_data/config/system.properties) hides icon when there is only one renderer
• URIs update with TSB, removing mistakes the old URI would cause.
• Export now supports aggregation
• New Pure-Java CDF library is in production, improving speed and memory use. Old library is removed.
• setDepend1Cadence added
• ImageDataSource keyword plotInfo=1 leverages JSON metadata to extract plots from images with lost data or vap files.
• Two PngWalkTools can be bound together to make comparisons of like filenames.
• PngWalkTool1 is finally renamed PngWalkTool
• SimpleServlet supports non-file resources like vap+inline, vap+jyds is disabled.
• PNGWalkTool supports digitizing, add jump to selected and white/black lines
• script parameter can now be relative path (e.g. --script=myscript.jy)
• addPlots(2,'below') in jython scripts
• any URI ending in .png .jpg or .gif that contains an asterisk is handled by the pngwalk tool
• add shift+mousewheel nudge behavior
• das2servers can return non-200 error codes.
• PNGWalkTool digitizer looks up plot coordinates when axis metadata is found.

Known issues:

• see Source Forge

This completes the feature set for the new production branch. No major features will be added before branching this version. A production branch will be started in a couple of weeks, and the "Start Autoplot" link at the top of autoplot.org will run this version. Feedback on the function and quality of this release is greatly appreciated!

Data is input from:

• ASCII files (.dat,.txt)
• CSV files (.csv)
• Binary files (.bin)
• CDF files(.cdf)
• Excel spread sheets (.xls)
• das2Streams (.d2s or .das2Stream)
• ViRBO's TSDS server
• CDAWeb data server (vap+cdaweb:)
• das2servers (vap+das2server:)
• QStreams (.qds), QDataSet on a das2Stream
• Plasma Wave Group das2 Server
• NetCDF files (.nc,.ncml)
• HDF5 files (.h5,.hdf5)
• DODs servers (.dds)
• Cluster Exchange Format files (.cef)
• Fits Format Files
• Images (.jpg, .png, .gif)
• Wav audio files (.wav)

This is not complete, and limitations include:

• Quantities of rank 3 dimensionality (e.g. flux(Time,Energy,pitch) are plotted by slicing.
• Quantities of rank 4 dimensionality (e.g. flux(Time,Energy,azimuth,sector) are plotted by slicing twice.

autoplot.org is the wiki website, and contains more information than this launch page.