PDS_VERSION_ID = PDS3 RECORD_TYPE = STREAM OBJECT = TEXT PUBLICATION_DATE = 2010-02-05 NOTE = "Description of contents of EXTRAS directory" END_OBJECT = TEXT END H. Metselaar 2010-02-05 1. Introduction 2. Contents of the EXTRAS directory 3. Explanation 1. Introduction ---------------- The EXTRAS directory contains additional files provided by data preparer for a better understanding of the data and data processing. 2. Contents of the EXTRAS directory ----------------------------------- EXTRAS | | TARGET_LIST.TXT A consolidated list of DCIXS targets merged | with the AMIE target list. | | ALL_HK.CSV A file in CSV format used by the pipeline to | convert the raw HK values into engineering | values. | | DIFF_SCET_UTC.PNG An image of a chart showing the difference | between SCET and UTC times. | | TIME_CORRELATION.XLS Smart-1 time correlation information spreadsheet. | 3. Explanation -------------- a. Calibration -------------- In principle the DCIXS data is uncalibrated. Because no calibrated data will be made available and the raw values are meaningless it was decided to convert the raw values to engineering values. This was done in the following way: a1. For all raw values with the exception of the temperature values the value of the parameter in engineering units is calculated from the equation: value = C + MX + NX^2 + PX^3 + RX^4 where X is the raw value, i.e. a polynomial of order four. However, because the coefficients N, P and R were always zero it is a simple linear transformation. The data pipeline uses an input file to provide the coefficients C and M. The file ALL_HK.CSV can be found in the EXTRAS directory. The format of the file is described hereunder. -------------------------------------------------------------------- | Field name | Description | (column) | -------------------------------------------------------------------- | Parameter | Parameter name | Byte_offset | Offset from start of archive pre-header | Parameter_bytes | Number of bytes used to create value | Signed | Indicates that value is signed (2's complement | | 'S' = signed, 'U' = unsigned | Min_Limit | Minimum expected value of parameter in | | Engineering Units | Max_Limit | Maximum expected value of parameter in | | Engineering Units | Units | Engineering Units e.g. Volts | Mode | Unit modes when test should be applied | | 0 = all modes. | C | Equation constant | M | Equation X coefficient | N | Equation X^2 coefficient | P | Equation X^3 coefficient | R | Equation X^4 coefficient --------------------------------------------------------------------- a2. For the thermistor calibration a lookup table was used to calculate the degrees in Celsius from the raw counts. The table can be found in the Data Handling ICD paragraph 3.3.1.1 DCIXS Thermistor Calibration. See the DOCUMENTS directory for the ICD. The snippet of C-code below illustrates how the temperature is calculated. The code was made available by Chris Howe. static int adc_tf[211] = { 8174, 8172, 8171, 8169, 8167, 8165, 8162, 8160, 8157, 8154, 8151, 8148, 8144, 8140, 8136, 8132, 8127, 8122, 8116, 8110, 8104, 8097, 8090, 8082, 8074, 8065, 8056, 8046, 8035, 8023, 8011, 7998, 7985, 7970, 7955, 7938, 7921, 7903, 7883, 7863, 7841, 7818, 7794, 7769, 7742, 7714, 7684, 7654, 7621, 7587, 7551, 7513, 7474, 7433, 7390, 7346, 7300, 7251, 7201, 7149, 7095, 7039, 6980, 6920, 6858, 6794, 6728, 6660, 6590, 6518, 6444, 6368, 6290, 6211, 6130, 6048, 5963, 5878, 5791, 5702, 5613, 5522, 5429, 5337, 5243, 5149, 5055, 4959, 4863, 4766, 4670, 4574, 4478, 4381, 4286, 4190, 4095, 4001, 3907, 3814, 3722, 3630, 3540, 3451, 3363, 3276, 3191, 3106, 3023, 2942, 2862, 2783, 2706, 2630, 2557, 2484, 2414, 2344, 2277, 2211, 2146, 2083, 2022, 1962, 1904, 1847, 1792, 1738, 1686, 1635, 1586, 1538, 1491, 1446, 1402, 1359, 1318, 1278, 1239, 1202, 1165, 1129, 1095, 1061, 1030, 998, 968, 938, 910, 883, 856, 830, 805, 781, 758, 735, 713, 692, 671, 652, 632, 614, 596, 578, 562, 545, 529, 514, 499, 485, 471, 458, 445, 432, 420, 408, 397, 385, 375, 364, 354, 345, 335, 326, 317, 308, 300, 292, 284, 277, 269, 262, 255, 248, 242, 236, 230, 224, 218, 212, 207, 201, 196, 191, 187, 182, 177, 173, 169, 164, 160 }; int calcTemperature(int adcValue) { int i, temperature; /* Do last array entry first */ if (adcValue <= adc_tf[210]) return 210-81; /* Halve the search time ... */ if (adcValue <= adc_tf[105]) i = 105; else i = 0; // run through array comparing adcValue against the array value for (temperature=i-81; adcValue < adc_tf[i]; i++) temperature++; return (temperature); } b. Detailed target list ----------------------- A lot of effort was put into creating a consolidated target list. The list was based upon the AMIE target list and was completed with DCIXS specific entries, especially for the cruise phase. As most of these targets have no meaning in PDS context, i.e. don't exist in the system so can't be put in the .LBL files, it was decided to put the target file in the EXTRAS directory. As input was used: - A list provided by K. Joy who kept track of the D-CIXS targets during the Smart-1 mission - Pointing information and command files as provided by D. Frew - The Cruise Operations Logbook (covering entries from 18/01/2004 - 14/08/2004) - Progress reports (no. 1 - 56) c. Information on S/C clock There were some issues with the S/C clock and time correlation: - It seems there were multiple resets, most of them early in the mission - One confirmed switch and one suspected switch from one physical clock to the other - Numerous bad correlation data points at different times during the mission due to unknown to me reasons In the EXTRAS directory the Smart-1 time correlation information spreadsheet, TIME_CORRELATION.XLS, can be found. It clearly shows the issues. The resulting chart also has been provided separately as DIFF_SCET_UTC.PNG. For the last 3/5 of the mission, starting at the end of June 2004 the correlation behaved better. For this period the the SCLK kernel provides correlation that is good to about 20 milliseconds. Basic geometry information was provided in the label files that accompany the data products. If the user of the data wants to do his/her own SPICE computations it is essential to use the SCLK clock with which the CK files were produced. If not you can end up with substantial timing errors. At the date of writing the SCLK to use is SMART1_070227_STEP.TSC. d. SPICE SPICE kernels are not included in the dataset. A consolidated SPICE dataset will be archived separately. e. Decompression of compressed Type 6 data During the mission a new science data format was proposed, the Compressed Low Count Spectrum. The data is identified with Packet Type 6. The compression used is RLE, Run Length Encoding, see Data Handling ICD (section 4) and EAICD (section 2.3.1) for details. The idea behind it was to maximize the science return. At first the idea was to, after decompression, regard the data as normal Low Count Spectrum. However, Type 2 data is binned differently compared to Type 6 data. Initially the idea was to re-bin the decompressed Type 6 data so it would exactly match the Type 2 data. After investiagtion it turned out that this is not feasible. Explanation Type 2 data takes the 4096 individual data values that come from the detectors and puts them into bins that are 16 wide. i.e. data values 0 to 15 are in the first bin, 16 to 31 in the 2nd... 4079 to 4095 in bin 256. For the compressed format the bins have different widths. The following table lists the spectral bin widths. Width Bin Numbers 8 0 to 96 12 97 to 144 16 145 to 176 20 177 to 200 24 201 to 224 32 225 to 244 48 245 to 254 56 255 The first 96 bins have width 8 so there are 2 of these bins for each one of the type 2 format (bin 1 is data 0 to 7, bin 2 data 8 to 15 ....). So for the first part of the spectrum you could take type 6 bin 1 and 2 add the counts together and put it in a type 2 bin 1, then sum 3 and 4 and put in bin 2 etc. The problem occurs when the type 6 data bin width gets >16. It is better to leave it up to the scientist working with the data to do re-binning if required at al. The following approach was agreed upon: - Decompress the Type 6 data - Don't do the re-binning, leave it "as-is" - Store the data as decompressed Type 6 data, i.e. a separate data product.