developer.aggregation
From autoplot.org
Purpose: describe definitively how aggregation should be done.
Audience: developers and interested scientists.
Contents |
1. Introduction
Aggregation has been one of Autoplot's main features, making a pile of files into a useful service comparable to the Time Series Browse capabilities of the Das2Server and CDAWeb plugins. When the data returned by the aggregation are not a function of time, useful functionality is supported. For example, suppose you have a Jython datasource:
tr= getParam( 'timerange', '2014-01-09', 'timerange to load' ) ds1= getDataSet( 'http://cdaweb.gsfc.nasa.gov/istp_public/data/ace/sis/level_2_cdaweb/sis_k0/$Y/ac_k0_sis_$Y$m$d_v$v.cdf?H_lo', tr ) ds2= getDataSet( 'http://cdaweb.gsfc.nasa.gov/istp_public/data/ace/sis/level_2_cdaweb/sis_k0/$Y/ac_k0_sis_$Y$m$d_v$v.cdf?H_hi', tr ) result= link( ds1,ds2 )
this too is a TSB, and likewise
http://cdaweb.gsfc.nasa.gov/istp_public/data/ace/sis/level_2_cdaweb/sis_k0/$Y/ac_k0_sis_$Y$m$d_v$v.cdf?H_hi&DEPEND_0=H_lo
does reasonable things.
This is to enumerate the uses and expected behavior.
2. First Obvious case, when each granule is a time series
Each call returns dependent data that is a function of independent parameter time:
vap+dat:http://cdaweb.gsfc.nasa.gov/istp_public/data/voyager/voyager1/plasma/sedr/v1_sedr_1977.asc?time=field0&column=field7&timeFormat=$Y+$j+$H+$M+$S vap+dat:http://cdaweb.gsfc.nasa.gov/istp_public/data/voyager/voyager1/plasma/sedr/v1_sedr_1978.asc?time=field0&column=field7&timeFormat=$Y+$j+$H+$M+$S
Clearly, the two datasets are appended.
3. Data returns a non-time-series
vap+dat:http://cdaweb.gsfc.nasa.gov/istp_public/data/voyager/voyager1/plasma/sedr/v1_sedr_$Y.asc?column=field9&depend0=field8
X data is combined, Y data is combined.
4. Data returns an image
What got me thinking about this was I was surprised when aggregating image files, that the result was not a failure, but instead it started stacking the images side-by-side. That must be a side-effect of other code, and it's not an entirely undesirable result. However what I was hoping to add was logic that would add a dimension in this case, so if each granule returned
image[x;y;r,g,b]
then the aggregation would be
image[i;x;y;r,g,b]
and then I could slice on dim0 to quickly scan through the images and get slices of a pixel value over time.
See http://jfaden.net/~jbf/camE/2016/08/26/$Y$m$dT$H$M$x.jpg?channel=greyscale&timerange=2016-08-26
Another case is where each file contains one measurement.
5. Slicing and Aggregating should be inverse operations
Slicing adds the property CONTEXT_0, so aggregation should be able to form a DEPEND_0 by forming a dataset from the CONTEXT_0 properties.