YLA seamless GOES project

older version (11-Jan-2011)
19-Jan-2016, Aki Takeda.
1-Mar-2016, statistics added.
28-Mar-2016, link to More about XRS" page added.

1. GOES satellite (not a NASA satellite!)

Information mainly from the GOES website ( http://www.ngdc.noaa.gov/stp/satellite/goes/index.html)
Data center: NGDC(National Geophysical Data Center) under NCEI(National Centers for Environmental Informationl.

GOES : Geostationary Operational Environmental Satellites (No.1 -- 15)
SMS : Synchronous Meteorological Satellites (preceding mission of GOES)
SEM : Space Environment Monitor (instrument subsystem on board all GOES and SMS).

*** GOES coverage ***
    * SMS-1: 1974-07 to 1974-10
    * SMS-2:: 1975-02 to 1976-04
    * GOES-1 1976-01 to 1978-05
    * GOES-2 1977-08 to 1983-05
    * GOES-3 1978-07 to 1980-08
    * GOES-4 no data
    * GOES-5 1983-01 to 1987-02
    * GOES-6 1983-05 to 1994-11  ### XRS: ?-may-1983 to 30-may-1994
    * GOES-7 1987-03 to 1996-08  ### XRS: 6-mar-1987 to 13-aug-1996
    * GOES-8 1995-03 to 2003-06  ### XRS: 1-mar-1995 to 10-apr-2003  
    * GOES-9 1996-04 to 1998-08  ### XRS: 1-apr-1996 to 27-jul-1998
    * GOES-10 1998-07 to 2009-12 ### XRS: 1-jul-1998 to 30-nov-2009
    * GOES-11 2000-07 to present ### XRS: 1-jun-2006 to 9-feb-2008
    * GOES-12 2003-01 to 2010-09 ### XRS: 10-jan-2003 to 12-apr-2007 
    * GOES-13 2010-04 to present; EUV, XRS, SXI not operational. 
                                 ### launched in apr-2006.
    * GOES-14 2009-12 to present, XRS only, other systems in storage mode.
                                 ### XRS: 26-nov-2009 to 26-oct-2010 
    * GOES-15 2010-09 to present, XRS only, other systems in test mode.
                                 ### XRS: 27-oct-2010 to present 

### : added by AkT

2. The current GOES data archive

Two original data source at NOAA:

Data archive in SolarSoft database:

### problem ###
Both (1) and (2) sometimes have missing records, which often compensate with each other.

Two archives under $SSWDB

Comparison of coverage: SDAC vs YDB
GOES X-ray Flux Archive by Andreas Moller
NASA GOES IDL User Guide by Zarro and Tolbert
SDAC GOES FITS archive (1974-present)

GOES primary/secondary defined by SWPC (plus my findings from SolarMonior plots)
https://ngdc.noaa.gov/stp/satellite/goes/doc/GOES_XRS_readme.pdf

[Table 2.] 
Chronology of primary and secondary satellites for XRS measurements since 1986
------------------------------------------------------
Start Date  Time   Primary Secondary      Remarks
------------------------------------------------------
2015-06-09  00:00  GOES-15  GOES-13  (AkT from SolarMonitor plots)     
2015-05-21  00:00  GOES-14  GOES-13  (AkT from SolarMonitor plots)     
2015-01-24  00:00  GOES-15  GOES-13  (AkT from SolarMonitor plots)     
2012-11-19  16:31  GOES-15  None
2012-10-23  16:00  GOES-14  GOES-15
2011-09-01   0:00  GOES-15  GOES-14
2010-10-28   0:00  GOES-15  None
2010-09-01   0:00  GOES-14  GOES-15
2009-12-01   0:00  GOES-14  None     GOES-10 decommissioned
2008-02-10  16:30  GOES-10  None     GOES-11 XRS failure
2007-12-18   0:00  GOES-11  None     GOES-10 not tracked due to antenna problems
2007-12-05   0:00  GOES-11  GOES-10
2007-11-21   0:00  GOES-11  None     GOES-10 not tracked due to antenna problems
2007-04-12   0:00  GOES-11  GOES-10
2007-01-01   0:00  GOES-10  GOES-11
2006-06-28   0:00  GOES-12  GOES-11
2006-06-22  14:00  GOES-12  None
2003-05-15  15:00  GOES-12  GOES-10
2003-04-08   0:00  GOES-10  GOES-12
1998-07-27   0:00  GOES-8   GOES-10
1995-03-01   0:00  GOES-8   GOES-7
1994-12-11   0:00  GOES-7   GOES-8
1988-01-26   0:00  GOES-7   GOES-6
1986-01-01   0:00  GOES-6   GOES-5
------------------------------------------------------

3. YLA seamless GOES data archive

Specification of the product:

YLA seamless GOES data preparation steps:

  1. Creation of seamless data set.
  2. Adjustments/corrections of flux values.
  3. Creation of daily files in the ascii format.
  4. Prepare software to hadle new data base.
  5. Inject final products into SSWDB and SDAC and advertize.

Breakdown of seamless GOES data records.

Contributions from primary(P) and secondary(S) satellites.
year total rec. from P. rec. from S. S-chunks rec. no data no data
-chunks
satellite
(P/S)
daily plot
movies
1991 525600 516252 (98.2%) 7687 ( 1.5%) 170 1661 ( 0.3%) 40 7/6 lo & hi -bands
1992 527040 519137 (98.5%) 6347 ( 1.2%) 201 1556 ( 0.3%) 79 7/6 low & high
1993 525600 520301 (99.0%) 4255 ( 0.8%) 173 1044 ( 0.2%) 71 7/6 low & high
1994 525600 516658 (98.3%) 4166 ( 0.8%) 233 4776 ( 0.9%) 246 7/6 low & high
1995
1/1 --
3/1 --
525600
84960
440640
511420 (97.3%)
84575 (99.5%)
426845 (96.9%)
10603 ( 2.0%)
383 ( 0.5%)
10220 ( 2.3%)
244 3577 ( 0.7%)
2 ( 0.0%)
3575 ( 0.8%)
44 ---
7/8
8/7
low & high
1996
1/1 --
8/1 --
527040
306720
220320
518494 (98.4%)
302338 (98.6%)
214716 (98.1%)
8127 ( 1.5%)
4347 ( 1.4%)
3780 ( 1.7%)
237 419 ( 0.1%)
35 ( 0.0%)
384 ( 0.2%)
22 ---
8/7
8/9
low & high
1997 525600 507745 (96.6%) 16631 ( 3.2%) 235 1224 ( 0.2%) 76 8/9 low & high
1998
1/1 --
7/27 --
525600
298080
227520
518430 (98.6%)
294672 (98.9%)
223758 (98.3%)
6975 ( 1.3%)
3228 ( 1.1%)
3747 ( 1.6%)
248 195 ( 0.0%)
180 ( 0.1%)
15 ( 0.0%)
33 ---
8/9
8/10
low & high
1999 525600 518436 (98.6%) 7076 ( 1.3%) 292 88 ( 0.0%) 50 8/10 low & high
2000 527040 519885 (98.6%) 7010 ( 1.3%) 198 145 ( 0.0%) 36 8/10 low & high
2001 525600 517519 (98.4%) 7886 ( 1.5%) 190 195 ( 0.0%) 16 8/10 low & high
2002 525600 514599 (97.9%) 10998 ( 2.1%) 176 3 ( 0.0%) 2 8/10 low & high
2003
1/1 --
4/8 --
5/15 --
525600
139680
54180
331740
517102 (98.4%)
136772 (97.9%)
52497 (96.9%)
327833 (98.8%)
8404 ( 1.6%)
2908 ( 2.1%)
1674 ( 3.1%)
3822 ( 1.2%)
261 94 ( 0.0%)
0 ( 0.0%)
9 ( 0.0%)
85 ( 0.0%)
11 ---
8/10
10/12
12/10
low & high
2004 527040 519655 (98.6%) 7147 ( 1.4%) 164 238 ( 0.0%) 10 12/10 low & high
2005 525600 518532 (98.7%) 7018 ( 1.3%) 169 50 ( 0.0%) 6 12/10 low & high
2006
1/1 --
6/28 --
525600
256320
269280
510167 (97.1%)
249514 (97.3%)
260653 (96.8%)
15296 ( 2.9%)
6721 ( 2.6%)
8575 ( 3.1%)
268 137 ( 0.0%)
85 ( 0.0%)
52 ( 0.0%)
5 ---
12/10
12/11
low & high
2007
1/1 --
4/12 --
525600
145440
380160
513427 (97.7%)
137575 (94.6%)
375852 (98.9%)
11441 ( 2.2%)
7647 ( 5.3%)
3794 ( 1.0%)
217 732 ( 0.1%)
218 ( 0.1%)
514 ( 0.1%)
34 ---
10/11
11/10
low & high
2008
1/1 --
2/10 --
527040
58590
468450
518083 (98.3%)
58027 (99.0%)
460056 (98.2%)
---
---
---
--- 8957 ( 1.7%)
563 ( 1.0%)
8394 ( 1.8%)
203 ---
11/-
10/-
low & high
2009
1/1 --
12/10 --
525600
480960
44640
515661 (98.1%)
471745 (98.1%)
43916 (98.4%)
---
---
---
--- 9939 ( 1.9%)
9215 ( 1.9%)
724 ( 1.6%)
198 ---
10/-
14/-
low & high

Check new hybrid 1992 daily plots

Sample plots : 1 day flux    seamless goes performance
Problematic cases: 1 (error in original data)    2 (discrepant P & S int.)    3 (noise in origial data)

Questions to GOES specialists (Rodney@SWPC or Janet@NCEI )

Coverage of the GOES week files under $SSWDB/ydb (official) and $SSWDB/ydb_beta (YLA contribution) as of 11-Jan-2011.

1991 ydb1992 ydb1993 ydb 1994 ydb1995 ydb

1996 ydb1997 ydb1998 ydb 1999 ydb2000 ydb2001 ydb

2002 ydb2003 ydb2004 ydb 2005 ydb

2002 ydb_beta2003 ydb_beta 2004 ydb_beta2005 ydb_beta

2006 ydb2007 ydb2008 ydb 2009 ydb2010 ydb

2006 ydb_beta2007 ydb_beta 2008 ydb_beta2009 ydb_beta2010 ydb_beta

### Movie of GOES week data from 1991 - 2010.

checking connections between the successive GOES

The lowest observable flux (detection limit) --- derived from reviewing data