PSM Data Formats

San Francisco State University Physics and Astronomy Department SFSU Pacific Oceanography Project Formats of Data on PSM Data Files ( data files |calibration data |sfsu private page )

Here is the data format for the various data files involving Pioneer Seamount data.

(a b bwc c d e f r )

"b" files: Time series for one "2-hour" time block, derived from the "m" files by combining the four channels - sometimes adding all four, sometimes using only one of the channels. Files from 2001 always consist of 1801 blocks of 4096 samples; files from 2002 span an exact 2-hour interval, but the number of blocks varies by one or two.

day0 (double): starting time (ms); the beginning of Jan 1 is at day0=86400000.
npt (long): number of points in the time series. Files from 2001 are always 1801 blocks of 4096 samples long, giving npt = 7xxxxxx. Files from 2002 vary slightly in length.
idat (signed integer): time series, values in the range -32768 to 32767.

Files with all four hydrophones added coherently are put in directories with the correct year (e.g., /2001/ or /2002/) in the path. Files with a single hydrophone have 10, 20, 30 0r 40 added to the year, giving for instance /2041/ for data from 2001 with only hydrophone 4 (of 4), and /2022/ for data from 2002 wity only hydrophone 2. Files which have been "deglitched for blue whales" have another 50 added to the year - giving, for example, /2091/ for data from 2001 with hydrophone 4 only, deglitched.

"bfit" files: results of carrying out a non-linear least- squares fit to events identified in "bwc" files. The format of these files is probably not final.

FORMAT

npt,exits,index,f0,alpha,phi0,res2,nchi2
npt (long): number of events
exits[npt] (long): exit (convergence) status from successive fits
index[npt] (long): index of correspoonding peak in "bwc" file
f0[npt] (double): central frequency (Hz)
alpha[npt] (double): -dfdt (Hz/sec)
phi0[npt] (double) phase of sine wave at center of track
res2[npt] (double): squared residuals
nchi2[npt] (double): normalized chi-squared, equal to the squared residuals, divided by the square root of the sum of the signal points times the sum of the fitted points; goes between 0 (good) and 2 (bad).

[The probable values of some of these numbers are as follows: npeaks: a few to 10,000; exits = 1101 for only second fit non-convergent; index 0 to 10,000; f0=16.00; appha = 0.045; phi0 from -4 pi to 4 pi; resw = 2x10¹⁰; nchi2 in [0,2].

"bwc" files: results of searching "c" files for blue-whale calls. The files searched are from year 2091 (or 2092). "m" files from 2001 are unpacked with only hydrophone number 4 to "b" files of year 2041. These "b" files are deglitched and rewritten as "b" files of year 2091. The deglitched "b" files are processed with the blue-whale 'B'-call matched filter to give the 'c' files. Then finally the program getbwb2.pro looks for peaks in the matched-filter output, with a dynamic noise-referenced threshold, and writes them to the 'bwc' files.

FORMAT

npeaks,cfile,iyear,cday,cdb,ccen,csig,cthrnsig,cthrfix
npeaks (long): number of peaks
cfile[npeaks] (string(50)): "c" file name, including path
iyear[npeaks] (int): year number (2001 -> 1, etc)
cday[npeaks] (double): starting times (ms)
cdb[npeaks] (float): peak amplitude, in dB re (???)
ccen[npeaks] (float): center of noise peak for 2-hr run
csig[npeaks] (float): sigma of noise peak for 2-hr run
cthnsig[npeaks] (float): threshold at "n sigmas"
cthfx[npeaks] (float): fixed threshold
day0[npeaks] (double): starting time of "c" file

[The probable values of some of these numbers are as follows: npeaks: a few to 10,000; cday: 2,073,600,000. for day 244; cdb: 3.0 (should be more like 90. when properly normalized); ccen: 1. (will go to 90.); csig: 0.3 (will go to perhaps 5.); cthnsig, cthfx - about equal to ccen + (several)*csig. LOCATION OF FILES Should be the same place as the "c" files (q.v.).

"c" files: output from blue-whale B-call detector. A quadrature matched filter for harmonics 1 and 3, linear chirp, is passed over the output from hydrophone 4, then downsampled to 10 Hz.

FORMAT

day0,dt,npt,lmsblk,datsq
day0 (double): starting time (ms)
dt (double): time interval between points (sec)
npt (long): number of points in the array
lblk (long): length of an averaging block
ccsq(npt) (float): time series of cross-correlation detector output

[The probable values of some of these numbers are as follows: dt = 100 ms; npt = 7200000/100 = 72000; lblk=100 LOCATION OF FILES

2001: /usr/data/bland/pioneer/yyyy/ddd/ryydddhh.mmc
2002: /usr/data/bland01/pioneer/yyyy/ddd/ryydddhh.mmc

"d" files: "detector" output for RAFOS sources

"f" files: Image files from which the jpegs are made.

sigdB(512,439): Spectral density in dB re (1 m Pa)²/Hz. As of Sept 6, 2002, the frequency response of the system has not been taken out.

"k" files: Starting times of blocks in "l" files.

nblk (long): number of blocks in the "l" file
dblkarr(nblk) (double): day value for the blocks (in msec)
tmstparr(nblk) (string): time stamps of blocks; the time stamps are 17 characters long: ddd:hh:mm:ss.sssS .

Ideally "l" files should have 21095 or 21096 blocks.

"r" files: mean-square signal values, with the channels combined in various ways, downsampled to 1 Hz. ("r" stands for "rms," but the values are mean-square values.)

FORMAT

day0,dt,npt,lmsblk,datsq
day0 (double): starting time in days
dt (double): time interval between points
npt (long): number of points in the array
lmsblk (long): length of an averaging block
datsq(npt,6) (float): time series of mean-square signal values
- series 0: hydrophone 0, squared, lmsblk values added together
- series 1,2,3: same thing for hydrophones 1, 2, and 3.
- series 4: hydrophones added coherently, then squared, then down sampled
- series 5: hydrophones added after squaring (this is the sum of series 0-3)

[The probable values of these numbers are as follows: lmsblk=1024; npt = 1801 x 4 = 7204, based on 1801 blocks of 4096 samples in a raw-data file; dt=1024/1000 sec, based on 1 kHz sampling.]

"s" files: spectral density data, from Fourier transforming the "b" files; Raw data is organized into 15-minute blocks, and 878 1.024-second chunks are fourier-transformed. The resulting power spectrum is combined with two time slices into a single 512-point array. This makes a 512x439- point array. These points are then corrected for the equalization frequency response and converted in to decibels re (1 microPascal)^2/Hz. The output is:

dstart (double): time for the first sample used.
dt (double): time interval (should be 2.096 sec)
nx (long): should be 512
ny (long): should be 439
pdensdb(nx,ny) (byte): the spectral density data