NEODAAS
Satellite Data File Formats
HRPT formats
- HRPT data
- HRPT data consists of minor frames of 11090 10-bit words. Each 10-bit word is
stored in two 8-bit bytes on disk. Each minor frame corresponds to one
AVHRR scan line covering a ground swath of
approximately 3000 km. The video data, for the NOAA series of satellites,
comprises 5 interleaved channels, each 2048 pixel wide. The SeaWiFS scanner
has 8 channels with 1285 pixels per line. As well as video data, each
minor frame contains sync, telemetry and
TIP data. A full format description is available in the
NOAA
Polar Orbiter Data Users Guide but we also have a
simplified format description.
- HRPT data buffered in blocks
- Some devices such as tapes find it easier to handle data which comes in
fixed-size blocks. Buffered HRPT data is exactly the same as HRPT data above but
with the addition of extra zero-filled bytes on the end of each scan line to pad
it out to 22 KBytes or 24 KBytes.
- HRPT data in packed Dundee format
- Dundee format is exactly the same as HRPT data above but packed to save
disc space. It stores 3 10-bit words in 4 8-bit bytes, leaving the last (top) 2
unused bits of the fourth byte as zero. Dundee format files are thus two-thirds
of the size of HRPT files without any loss of information. Each minor frame is
treated separately, and may be padded as described below in the tape formats
section.
- HRPT data in NOAA Level-1B format
- NOAA Level-1B format stores the image data packed and uses the space gained
to store better calibration data, geo-referencing data, solar zenith angles,
etc. A full format description is available in the
NOAA Polar Orbiter Data Users
Guide but we also have a simplified format
description. Note that the format has been changed for the
NOAA KLM series (NOAA-15 and
later) so check your software can handle the new format before ordering.
It also changed again on 2005/04/01 to format version 3, a notable change
being one of the calibration scaling factors.
To process HRPT data you can use:
SeaWiFS formats
- HRPT data in NASA SeaWiFS frame-formatter format
- SeaWiFS data at level-0 is straight from the
satellite but with the addition of a file header and some minor formatting. A
full
format
description is available.
- HRPT data in SeaWiFS HDF format
- SeaWiFS data at level-1a or level-2 comes in
HDF (Hierarchical Data Format). Level-1A packages calibration and navigation
information alongside the raw image data whereas level-2 converts the image data
into physical parameters such as chlorophyll and CZCS-pigment. A full format
description is available in the
SeaWiFS
Project Software Documentation.
To process SeaWiFS data you can use:
- swffx (Extract data from SeaWiFS
Frame Formatter files)
- swx (Extract data from SeaWiFS HDF
files)
- SeaDAS
- ENVI
MODIS formats
- Downlink data format
- The raw data stream from the satellite consists of CADUs of 1024 bytes each
containing multiplexed virtual channel VCDUs plus error correction data.
xeosdisp
is used for viewing this format in real-time during reception.
A MODIS packet format description is available.
- Level 0 format
- Time-ordered, error corrected VCDUs for a single virtual channel, MODIS in
this case. Image data can be extracted using eoslzx, please enquire for details.
- Level 1 format
- HDF-EOS format, an extension of HDF; can be read by eosx, ENVI etc.
To process MODIS data you can use:
CZCS formats
- Raw ingested data
- See CZCS format.
- Level 0 data
- Unpacked back to 12375-byte records.
- Level 1 data
- SeaDAS requires level 1 data. No further information on this format
is available.
PDUS formats
- Decommutated image files
- The received frames are decommutated into one file per channel, with padding
for missing/corrupt frames or lines. A full PDUS format
description is available.
To process PDUS data you can use:
Image formats
- Common image formats
- We can supply common formats such as GIF, JPEG, PNG, TIFF etc.
- Raw image files
- These files have no header or line separators; they are simply the
extracted image data as a stream of bytes. You must therefore make a note of the
image dimensions for further processing. Pixels in 10-bit data files are stored
in two bytes each; you should know whether you require bytes in little-endian
(PC) or big-endian (Sun) order.
- Calibrated raw files
- Sensor-calibrated data can be delivered in raw binary arrays of floating
point (real) numbers. These are typically 32-bits (4 bytes) per pixel in either
big- or little-endian byte order.
- TIP data
- The TIP (TIROS Information Processor) data extracted from the HRPT data as
520 8-bit words per line.
- CALIB data
- The CALIB data extracted from the first 103 words of the HRPT data as 103
10-bit (2 byte) words per line. See HRPT format above.
- PGM
- PGM files are raw for 8-bit data and ASCII for 10-bit.
SeaWiFS data calibrated to level-1b (sensor calibrated radiances) are available
in so-called PGM files but the values stored are in floating point format so the
file is not strictly PGM but easily readable nonethless.
To process image data you can use:
Tape formats
- HRPT (CCT/Exabyte)
- Data is stored as received (see above).
- HRPT (Cartridge)
- Each line is buffered to 22 kbytes by appending 174 words.
- Dundee (CCT/Exabyte)
- Each line is stored as 4 blocks of 3700 8-bit bytes. Packed Dundee format
is described above.
- Dundee (Cartridge)
- As for CCT/Exabyte but each line is buffered to 14848 8-bit bytes by
appending 48 8-bit bytes.
To process tape files you can use:
