This documentation and help text provides information about the archive of Finnish Meteorological Institute (FMI) products in the Geomagnetic Expert Service of the European Space Agency (ESA) space weather services. It gives guidance on how to access images and other data in and download those data from the archive. Descriptions of the archive and of the files in the archive are provided first, practical guide for access and download is at the end. The archive contains plots and data from the following products:
The archived data and products are for the service post validation purposes only. There is no human quality control, so the contents of this archive are not intended or suitable for scientific analyses.
For definite data, please contact the helpdesk at helpdesk.swe@ssa.esa.int.
The archive is subdivided into two branches, each containing data on different phenomena (aurorae and GIC). These data and the two branches are described in greater detail below.
The archive of Regional Auroral Forecast (RAF) products contains seven elements whose content and file types are described in Table 1 below. The files are stored to the archive once in every 5 min.
| File ID | Format | Description | Time stamp time | Examples |
|---|---|---|---|---|
| lamps | PNG | Indicators for NOAA X-ray alerts, global geomagnetic activity, probability to see auroras |  |
|
| oval_00 | PNG | Auroral oval observation (present) | The starting time for creating RAF products |  |
| oval_03 | PNG | Auroral oval forecast (3 hours ahead) | 3 h added to the to the time stamp of oval_00 | similar plot as shown in oval_00, but with time stamp ~3 hours ahead present time |
| oval_06 | PNG | Auroral oval forecast (6 hours ahead) | 6 h added to the to the time stamp of oval_00 | similar plot as shown in oval_00, but with time stamp ~6 hours ahead present time |
| oval_09 | PNG | Auroral oval forecast (9 hours ahead) | 9 h added to the to the time stamp of oval_00 | similar plot as shown in oval_00, but with time stamp ~9 hours ahead present time |
| oval_12 | PNG | Auroral oval forecast (12 hours ahead) | 12 h added to the to the time stamp of oval_00 | similar plot as shown in oval_00, but with time stamp ~12 hours ahead present time |
| table | text | Numeric data of dB/dt and data availability at selected locations, sunrise and sunset, moonrise and moonset. In HyperText Markup Language (HTML) format. | The contents of an example table.txt file is below this table |
An example table.txt file:
<p>
28.06.2021 17:11:13 UTC, rise and set times in local time<br />
<table border ="1">
<tr><td>station</td> <td>STA</td> <td>max_dB</td> <td>activity</td> <td>sunrise</td> <td>sunset</td> <td>moonrise</td> <td>moonset</td></tr>
<tr><td>Tromsø </td> <td>TRO</td> <td> 0.03</td> <td>0</td> <td>+</td> <td>+</td> <td>02:43</td> <td>05:27</td> </tr>
<tr><td>Kevo </td> <td>KEV</td> <td> 0.02</td> <td>0</td> <td>+</td> <td>+</td> <td>03:17</td> <td>05:49</td> </tr>
<tr><td>Muonio </td> <td>MUO</td> <td> 0.02</td> <td>0</td> <td>+</td> <td>+</td> <td>02:40</td> <td>06:51</td> </tr>
<tr><td>Oulujärvi </td> <td>OUJ</td> <td> 0.02</td> <td>0</td> <td>02:34</td> <td>23:53</td> <td>01:33</td> <td>07:28</td> </tr>
<tr><td>Hankasalmi </td> <td>HAN</td> <td> 0.02</td> <td>0</td> <td>03:23</td> <td>23:09</td> <td>01:12</td> <td>07:53</td> </tr>
<tr><td>Dombås </td> <td>DOB</td> <td> 0.02</td> <td>0</td> <td>03:36</td> <td>23:17</td> <td>01:21</td> <td>08:09</td> </tr>
<tr><td>Nurmijärvi</td> <td>NUR</td> <td> 0.02</td> <td>0</td> <td>03:55</td> <td>22:53</td> <td>01:05</td> <td>08:22</td> </tr>
<tr><td>Karmøya </td> <td>KAR</td> <td> 0.02</td> <td>0</td> <td>04:27</td> <td>22:57</td> <td>01:14</td> <td>08:47</td> </tr>
</table>
</p>
<p>
Previous full Moon: 24.06.2021 - Next new Moon: 10.07.2021
</p>
The archive of maps for power and pipeline operators contains seven elements described in Tables 2 and 3 below. The files are stored to the archive once in every 10 min.
| File ID | Format | Description | Time stamp time | Examples |
|---|---|---|---|---|
| sumgic_FI_NO_SSA | PNG | Sum of GIC in the Finnish and Norwegian power grids | The starting time for creating GIC plots |  |
| snapshot_B_SSA | PNG | Ground magnetic field at the peak time of the GIC sum within the latest 24 hours | time of the largest GIC sum in the power grids during the latest 24 hours |  |
| snapshot_dB_SSA | PNG | dB/dt map at the peak time of the GIC sum within the latest 24 hours | The time of the largest GIC sum in the power grids during the latest 24 hours |  |
| snapshot_E_SSA | PNG | Geoelectric field at the peak time of the GIC sum within the latest 24 hours | The time of the largest GIC sum in the power grids during the latest 24 hours |  |
| snapshot_GIC_SSA | PNG | GIC at power grid substations at the peak time of the GIC sum within the latest 24 hours | The time of the largest GIC sum in the power grids during the latest 24 hours |  |
| File ID | Format | Description | Time stamp time | Examples |
|---|---|---|---|---|
| gicman_SSA | PNG, text | Measured and modelled GIC in the Finnish natural gas pipeline in Mäntsälä | The starting time for creating GIC plots |  |
| Helsinki_psv_SSA | PNG | Pipe-to-soil voltage in the Finnish natural gas pipeline in Helsinki | The starting time for creating GIC plots |  |
All archived files except the HTML files include two pieces of time information:
These time tags and time stamps are described in greater detail in the two subsections below.
When the original image or text file is created, the file is assigned a timestamp by the file system. Since the observation time information is not easily accessible during the archival stage, the time tag used in the archive is based on the file creation time and date information and is shown as Coordinated Universal Time (UTC). This time tag therefore differs slightly from the time shown in an image (details explained below). For individual data files the time tag format used is YYYYMMDDThhmmssZ and for daily files YYYYMMDDTZ (for description see the table below). The format is compatible with the ISO-8601 standard for time representation.
| Substring | Description |
|---|---|
| YYYY | Year with four digits, e.g., 2020 |
| MM | Month with two digits, with prefix 0, e.g., 05 |
| DD | Month with two digits, with prefix 0, e.g., 11 |
| T | Indicates start of the time part of the time tag |
| hh | Hour (00-23) with two digits, with prefix 0, e.g., 06 |
| mm | Minute (00-59) with two digits, with prefix 0, e.g., 47 |
| ss | Second (00-59) with two digits, with prefix 0, e.g., 13 |
| Z | Zulu to indicate the date-time or date is given in UTC. |
The time these time stamps correspond to depends on the file content type. For instance the time shown in a Regional Aurora Forecast differs from the time shown in a GIC snapshot image. The time stamp time correspondences are described in column Time stamp time in Tables 1, 2 and 3 above. Note that near-real time data used for creating these images does not necessarily exist until the given time stamp.
The archive directories contain two types of files:
To keep the total number of files manageable, individual data files (text files or image snapshots) are not available separately. The tar file name indicates the content and file type of the files in the tar file. The tar file naming scheme is as follows:
<file content type>_<file format in uppercase>_<UTC date in ISO format>.tar
The file content types are listed in tables under the Section Archive structure above. The file content type is PNG or TXT. The date is the UTC date, meaning that all those individual files whose time tags fall within a UTC date are packed in the tar file of that date.
An example filename is below:
gicman_SSA_PNG_20200901Z.tar
All tar files include
The tar file contains the image (PNG) or text format files within the UTC day shown in the filename. The tar file may contain an incomplete set of files due to process anomalies or when the tar file corresponds to the current (yet incomplete) UTC date.
The tar file contains a single Error/quality/log file as the last file. This file contains a log of the error/quality anomaly checks performed during the archival process. The fields appended to the compact log file are
The keywords/tags are listed and their meanings explained in Table 5 below. Note: other types of checks (and resulting error keywords) can be envisioned; these are the ones currently being applied to the data files.
| Keyword | Meaning | File format used for |
|---|---|---|
| NC | No Checks. This means, that the contents of these files have not been checked. General and consistent checks are difficult to apply, since especially the internal formats of the text files are often unique and file content specific. | text |
| ND | None Detected. This means, that no errors have been detected with the types of checks that have been applied to the file. Other (but not applied) checks might reveal errors. | PNG |
| CFM | Cloud Forecast Missing. The image file is much larger, if the cloud forecast is in the image. If the file is smaller than a threshold, this keyword is set. | PNG (auroral ovals) |
| CPNGF | Corrupted PNG File. Tested with the ImageMagick tool identify. |
PNG |
The HTML files contain descriptive and help texts. Each subdirectory contains an index.html file which controls the view to the directory contents.
The tree-like directory structure is described in Figure 1 above. Typically the user accesses the archive via the root URL (other archive locations can of course be bookmarked) and then descends to the branch of interest using the hyperlinks. Access to the root or to the branches is through SSO authentication, hence a userid and password must be provided.
Command-line tools (such as curl and wget in unix-like environments) permit also scripting and automated or semi-automated retrieval processes within the limits imposed by the SSO-interface controlling access to the archiving system. Access to the root or to the branches is through SSO authentication, hence a userid and password must be provided.
A direct retrieval URL is compiled by combining
The tar file naming scheme is as follows:
<file content type>_<file format in uppercase>_<UTC date in ISO format>.tar
The file content types are listed in Tables 1, 2 and 3 above. The file format is either PNG or TXT.
As an example:
gicman_SSA_PNG_20200901Z.tar
The example file URL would hence become:
https://ssa.fmi.fi/image/realtime/SSA/archive/GIC/gicman_SSA/gicman_SSA_PNG_20200901Z.tar