How to access the raw time-series dataset of GOES X-Ray Flux?

Question

I would like to access the time-series dataset of X-Ray Flux obtained by GOES. I expect that the flux units to be Watts / square-meter; this is because of the figure below (obtained from this Nature paper).

I first tried accessing the GOES-R dataset, which contains netcdf (aka .nc) files. I followed the steps in this post on stackoverflow but was unable to open any of the netcdf files.

I also tried accessing the flux dataset from NOAA SWPC; I clicked the "data" tab and click the "json" link, but this only contains data from the last week or so. I prefer to have the full dataset spanning a few decades. The same site hosts an FTP link; the text files present here have data fields for "year", "month", and "day", but not "hour", "minute", and "second".

I notice that most of the data products available are fluxes averaged over a time-interval; I prefer the raw time-series if possible.

I am hopeful that a representation of this dataset exists in the form of a text-file (such as this one provided by RHESSI). Regardless, I am not sure how to access this dataset. I wonder if the data I am seeking is in the netcdf files that I am unable to open, or if there is another way to do this - perhaps with astroquery or astropy?

TLDR: I am looking for the GOES dataset containing the time-series (containing year, month, day, hour, minute, and second) of flux (measured as Watts / square-meter). How can I access this raw data?

Answer 1

You can use SunPy, check the Retrieving and analyzing GOES X-Ray Sensor (XRS) data example. There it explains how to download data with a time range and if wanted selecting the GOES satellite number.

If you would like the raw data in csv format, then you can get it from the official archive for GOES data. There you can find full an average data in csv format up to March 4th, 2020 (it stops then because GOES 16 and later are part of GOES-R). The link looks like this:

https://satdat.ngdc.noaa.gov/sem/goes/data/full/YYYY/MM/goesXX/csv/gXX_xrs_2s_YYYYMMDD_YYYYMMDD.csv

Where XX is the satellite number. For example, this is the 2s cadence csv file from goes 15 for March 4th, 2021.

For GOES-R I don't believe there's a CSV archive, but SunPy can query and download it, as shown in the example on the gallery.

Once you donwload the file with SunPy, you can load, plot, manipulate, and convert that timeseries to any other format that astropy tables support (including csv).

This is an example on how to query the data and convert it into csv. This is only for one day, but you could extended to a different time range if wanted.

from sunpy import timeseries as ts
from sunpy.net import Fido
from sunpy.net import attrs as a

tstart = "2020-06-21 01:00"
tend = "2020-06-21 23:00"
result = Fido.search(a.Time(tstart, tend), a.Instrument("XRS"), a.goes.SatelliteNumber(16))
goes_16_files = Fido.fetch(result)
goes_16 = ts.TimeSeries(goes_16_files, concat=True)
goes_table = goes_16.to_table()
goes_table.write('goes_16_20200621.csv', format='csv')