Process for adding a new station

How to add a CORIE station:

Each station is listed in the sites table in the telemetry database (on amb104, ambts01, amb105, and amb6400). The columns of the entry are:
stationid | site | latitude | longitude | x | y | refdepth | surface | bed | fullname | description | contact | type | status

site, latitude,longitude,x,y,description, type (fixed/buoy), and status (realtime/historical/offline) are the fields that are actually used. Separate sites table entries should be made for each instrument type at the station (e.g. red26 and red26adp get separate entries in the sites table).

Each station is stored in a separate table in the telemetry database, so a new station requires creating a new table. The table structure should match the table structure of equivalent instruments (ctd, adp, met, tide). The table name is a 5 character alphanumeric id combined with the instrument type (except for ctds, which just use the 5 character name).

The station must also be added to the depth reference table.
The master copy of the depth reference table is an ASCII file:
/home/workspace/ccalmr/data/verified/depth_reference_table.txt

The format of the file is:
[station name] [instrument id] [deployment start] [deployment end] [depth-frame]
abpoa CTD391 3636.5 current 410A

For the current deployment, the deployment end is listed as 'current'.

The depth reference table is loaded into the telemetry database using a perl script:
~pturner/psql/loaddepthref.pl ambts01
~pturner/psql/loaddepthref.pl amb105
~pturner/psql/loaddepthref.pl amb104

If the new station only uses standard instruments, then the rserialv2/rserialv2db code will handle the data for the new station. New instrument formats would require additional parsing code to be added to rserialv2. Depending on the format of the instrument data, a new line may also need to be added to config.txt (config.txt and rserialv2.c are in /usr/local/telemetry/ on ambts02 and base3).

These steps should get the new station data into the telemetry database. Additional steps are required to get the data to display on the web and to be compared to the models.

To add a display to the web (using the older CORIE interface):

• Create a directory under /usr/local/apache/htdocs/CORIE/network/
  using the 5 character station name
• copy and modify the index.html page from an existing station to refer to the new station
• modify the map section of the index.html to incorporate the new station
• regenerate the /Img/CORIE/c_telemetry_active.gif image to include the station
• upload a station image for the new station as:
  /usr/local/apache/htdocs/Img/CORIE/[station]_photo.jpg

Display of recent data is handled automatically by /usr/local/apache/cgi-bin/CORIE/oldlaststationdata
if the tables are set up.

Creation of the images is handled by
/usr/local/telemetry/cplots/corie/master_dboldplot.pl
on amb6400. Instrument type is determined from the instrument id in the depthreference table.

To add a station to be compared to the models:

Add the station id and station location to the stations.sta files. This determines the locations at which data is extracted from the model.

For ref,dev, and exp forecasts, the stations.sta file is an ascii file stored locally for each forecast (e.g. amb6400:/home/workspace/local0/forecasts/exp/reference/stations.sta ).

For NEFS forecasts, stations must be listed in the stations table of the geo database on amb102. This is read by makestations.pl to create the stations.sta file. The stations table has format:
Column | Type | Modifiers
------------+------------------------+-----------
name | character varying(128) | not null
type | character varying(32) |
descr | text |
location | geometry |
owner | character varying(32) |
forecastid | character varying(32) |
url | text |

forecastid can either be 'all', or can be a specific forecast name. The 'url' field is not currently used. Owner is either the person who requested the creation of the station for virtual stations or the agency that owns the physical station. The exact criteria for what stations are used by which forecast are specified in the makestations.pl script for each forecast.

For tampa, humboldt, monte, fraser, stations used to create the stations.sta file are listed in the sites table in the geo database on amb102.

How to add an external station

Scripts exist to read data from various sources (e.g.
/home/corie/psql/forcings/getexdata.pl on amb104 collects COOPS and USGS data). They can generally be found by looking in the crontabs for amb104 or amb105. The procedures for processing different external data sources are not standardized. Some of the processing scripts (e.g. getexdata.pl) determine which stations to process based on information in the sites table (in the forecast database on amb104 or amb105), while other processing scripts (e.g. /home/corie/psql/forcings/getusaceelev.pl) have which stations to process hard-coded in the script. Still other scripts (e.g. /home/corie/psql/buoy/getbuoy.pl) take the station name as an argument.

New stations have their data stored in tables by data type (in the forecasts database on amb104 and amb105). Temperature data is stored in the 'temperature' table, elevation data is stored in the 'elevation' table, etc. No external salinity data is currently incorporated, but if it were, it would be stored in a 'salinity' table.

The format of the variable specific tables is:
Column | Type | Modifiers
----------+-----------------------------+-----------
site | character varying(16) | not null
siteid | character varying(16) | not null
time | timestamp without time zone | not null
corieday | double precision | not null
[4 character variable name] | double precision |

This process applies to external stations used for model-data comparisons.

How to add a NANOOS station:

This is currently handled in an ad hoc manner. Each individual source is handled very differently, some (WaDoEcy) using the telemetry databases on amb104, amb105, and ambts01 , others (ORCA) using the exarchive database on amb104 and amb105. Image generation is also handled differently for different sources. While some new sources (additional ORCA buoys or WaDoEcy stations) could be handled relatively easily by an extension or modification of existing scripts, completely new sources would require the creation of new scripts.

Washington Department of Ecology stations bud01, sqx01 and wilbc are collected by systems equivalent to base3 and ambts02. Station specific scripts (e.g. ambts01:/home/corie/wdoe/get_sqx01.pl for sqx01) load the data into the telemetry databases on amb104, amb105, and ambts01. The stations are not currently listed in the sites table in the telemetry database, and are not currently used for model-data comparisons. Inclusion of the stations in the NANOOS web site is also a manual process.

ORCA station data is initially handled by the ORCA program, which provides partially processed seabird data files on its web site. A script (amb104:/home/corie/psql/getorca.pl) retrieves these files (using wget) and processes them into the exarchive database. All ORCA cast data is stored in two tables (orcacast and orcacastdata). Currently, only one ORCA station is provided to us, but additional ORCA stations (if provided in the same web location) should automatically get added to the same tables. The data files specify the station name, and this is included in the orcacast table.