CDAS Example Jupyter Notebook

This Jupyter notebook demonstrates using cdas related rountine and the cdasws Python package, for accessing the Coordinate Data Analysis System (CDAS) web services

References

Importing libraries 
from cdasws import CdasWs
import pandas as pd
import polars as pl
from rich import print
from great_tables import GT
Code 
cdas = CdasWs()

Get observatory groups with observatory ids (identifiers)

The following code demonstrates how to get the list of observatory groups with their respective observatory ids.

Code 
obs_groups = cdas.get_observatory_groups()
Convert to a DataFrame and display 
obs_groups_df = pl.DataFrame(obs_groups)
GT(obs_groups_df.with_columns(pl.col("ObservatoryId").list.join(", "))).tab_header(
    title="CDAS Observatory Groups",
).cols_label(
    ObservatoryId="Observatory Ids (list of identifiers for observatories that belong to this group)"
)
CDAS Observatory Groups
Name Observatory Ids (list of identifiers for observatories that belong to this group)
ACE AC, ACE, OMNI (1AU IP Data)
AIM AIM CIPS PMC SCI
AMPTE AMPTE-CCE
ARTEMIS THB, THC
Alouette Alouette-2
Apollo ALSEP
Arase (ERG) (null), Arase(ERG), ERG
Balloons PMC Turbo, bar_1A, bar_1B, bar_1C, bar_1D, bar_1G, bar_1H, bar_1I, bar_1J, bar_1K, bar_1M, bar_1N, bar_1O, bar_1Q, bar_1R, bar_1S, bar_1T, bar_1U, bar_1V, bar_2A, bar_2B, bar_2C, bar_2D, bar_2E, bar_2F, bar_2I, bar_2K, bar_2L, bar_2M, bar_2N, bar_2O, bar_2P, bar_2Q, bar_2T, bar_2W, bar_2X, bar_2Y, bar_3A, bar_3B, bar_3C, bar_3D, bar_3E, bar_3F, bar_3G, bar_4A, bar_4B, bar_4C, bar_4D, bar_4E, bar_4F, bar_4G, bar_4H, bar_5A, bar_6A, bar_7A
CNOFS CNOFS
CRRES CRRES Satellite
Cassini CASSINI, Cassini
Cluster C1, C2, C3, C4, CL, CT
DMSP DMSP, dmsp, dmsp-f13, dmsp-f16, dmsp-f17, dmsp-f18
DSCOVR DSCOVR
Dynamics Explorer DE, DE-1, DE1
Equator-S EQ
FAST FAST
Formosat FS-5, ROCSAT-1(FORMOSAT-1)
GOES G0, G10, G12, GOES-08, GOES-09, GOES-10, GOES-11, GOES-12, GOES-13, GOES-14, GOES-15, GOES-16, GOES-17, GOES10, GOES11, GOES12, GOES13, GOES14, GOES15, GOES16, GOES17, GOES18, GOES8, GOES9, GOES_10, GOES_11, GOES_12, GOES_6, GOES_7, GOES_8, GOES_9, g11
GOLD GOLD
GPS GPS
Genesis Genesis
Geosynchronous Investigations STPSat-6
Geotail GEOTAIL
Ground-Based Investigations CANOPUS, DARN, FMI, SESAME, THG_L2
Hawkeye HK
Helio Ephemeris BEPICOLOMBO, BORRELLY, COMETGS, COMETHMP, DAWN, EARTH, GALILEO, GIACOBINI, GIOTTO, HALEBOPP, HALLEY, HYAKUTAKE, JUNO, JUPITER, MARS, MERCURY, MSL, NEPTUNE, PHOBOS2, PLUTO, ROSETTA, SAKIGAKE, SATURN, SUISEI, URANUS, VENUS, WILD2
Helios HELIOS, HELIOS-1, HELIOS-2, HELIOS1, HELIOS2, Helios-1, Helios-2
IBEX IBEX
ICON ICON, ICON
IMAGE IM, IMAGE
IMP (All) I8, IMP-8, IMP7, IMP8, OMNI (1AU IP Data)
ISEE ISEE 1, ISEE-1, ISEE-3, ISEE1, ISEE2
ISIS ISIS-1, ISIS-2
ISS ISS
Interball INTERBALL-AURORAL, INTERBALL-GROUND, INTERBALL-TAIL
LANL L1, LANL-97A, LANL1989_046, LANL1990_095, LANL1991_080, LANL1994_084, Lanl-01A, Lanl-02A, Lanl-97A, Lanl1989_046, Lanl1990_095, Lanl1991_080, Lanl1994_084
MAVEN MAVEN
MESSENGER MESSENGER
MMS MMS1, MMS2, MMS3, MMS4, mms1, mms2, mms3, mms4
Mariner Mariner2
Mars Global Surveyor (MGS) MGS
Mars Science Laboratory (MSL) MSL
NOAA MetOp1, MetOp2, MetOp3, NOAA05, NOAA07, NOAA08, NOAA10, NOAA12, NOAA14, NOAA15, NOAA16, NOAA18, NOAA19
New Horizons NEW_HORIZONS, NH
OMNI (Combined 1AU IP Data; Magnetic and Solar Indices) OMNI (1AU IP Data)
POES/MetOp MetOp1, MetOp2, MetOp3, NOAA15, NOAA16, NOAA18, NOAA19
Parker Solar Probe (PSP) PSP, PSP_FLD
Pioneer PIONEER10, PIONEER11, Pioneer10, Pioneer11, Pioneer6, Pioneer7, PioneerVenus
Polar PO, POLAR, Polar, po
REACH reach
SAMPEX SAMPEX
SOHO SOHO
ST5 ST5
STEREO Ahead, Behind, STA, STB, STEREO, STEREOA, STEREOB, sta, stb
Smallsats/Cubesats CSSWE, Munin, SNOE, ac6a, ac6b, ela, elb
Solar Orbiter SOLO, solo
Sounding Rockets Dynamo-2, Endurance, RENU2, TOOWINDY
THEMIS THA, THB, THC, THD, THE, THEMIS, THG, THG_L1, THG_L2
TIMED TIMED
TSS-1R TSS-1R
TWINS TWINS-, TWINS1, TWINS2
Ulysses ULYSSES, Ulysses
Van Allen Probes (RBSP) RBSP, RBSP-A, RBSP-A-RBSPICE, RBSP-B, RBSP-B-RBSPICE, RBSPA, RBSPB, rbsp-a, rbsp-b, rbspa, rbspb
Voyager VG1, VG2, VGR-1, VGR-2, VOYAGER1, VOYAGER2, Voyager1, Voyager2
Wind OMNI (1AU IP Data), WI, WIND, Wi, Wind, wi

Get Intrument Types

The following code demonstrates how to get the available instrument types.

Code 
instr_types = cdas.get_instrument_types()
print(instr_types)
[
    {'Name': 'Activity Indices'},
    {'Name': 'Electric Fields (space)'},
    {'Name': 'Electron Precipitation Bremsstrahlung'},
    {'Name': 'Energetic Particle Detector'},
    {'Name': 'Engineering'},
    {'Name': 'Ephemeris/Attitude/Ancillary'},
    {'Name': 'Gamma and X-Rays'},
    {'Name': 'Ground-Based HF-Radars'},
    {'Name': 'Ground-Based Imagers'},
    {'Name': 'Ground-Based Magnetometers, Riometers, Sounders'},
    {'Name': 'Ground-Based VLF/ELF/ULF, Photometers'},
    {'Name': 'Housekeeping'},
    {'Name': 'Imaging and Remote Sensing (ITM/Earth)'},
    {'Name': 'Imaging and Remote Sensing (Magnetosphere/Earth)'},
    {'Name': 'Imaging and Remote Sensing (Sun)'},
    {'Name': 'Magnetic Fields (Balloon)'},
    {'Name': 'Magnetic Fields (space)'},
    {'Name': 'Particles (space)'},
    {'Name': 'Plasma and Solar Wind'},
    {'Name': 'Pressure gauge (space)'},
    {'Name': 'Radio and Plasma Waves (space)'},
    {'Name': 'Spacecraft Potential Control'},
    {'Name': 'UV Imaging Spectrograph (Space)'}
]

Get Datasets

The following code demontrates how to get a list of datasets.

Gets descriptions of the observatory groups (and associated instruments) from the server.

Code 
from beforerr.project import produce_or_load
Code 
obs_groups_and_instrs, file = produce_or_load(
    cdas.get_observatory_groups_and_instruments,
    filename="obs_groups_and_instrs",
    suffix="json",
)
Convert to a DataFrame and display it beautifully 
# Existing code to prepare the rows remains unchanged
rows = []
for observatory in obs_groups_and_instrs:
    for instrument_group in observatory["ObservatoryInstruments"]:
        for instrument in instrument_group["InstrumentDescription"]:
            rows.append(
                {
                    "Observatory": observatory["Name"],
                    "Instrument Group": instrument_group["Name"],
                    "Instrument Name": instrument["Name"],
                    "Short Description": instrument["ShortDescription"],
                    "Long Description": instrument["LongDescription"],
                }
            )

# Convert to DataFrame
obs_groups_instrs_df = pd.DataFrame(rows)

# Step 2: Adjust Column Multi-Index Creation
index_columns = ["Observatory", "Instrument Group", "Instrument Name"]
index = pd.MultiIndex.from_frame(obs_groups_instrs_df[index_columns])
obs_groups_instrs_df.drop(columns=index_columns).set_index(index)
Short Description Long Description
Observatory Instrument Group Instrument Name
ACE AC CRIS ACE Cosmic Ray Isotope Spectrometer ACE Cosmic Ray Isotope Spectrometer
DEF Definitive Data Definitive Data
EPAM ACE Electron Proton Alpha Monitor ACE Electron Proton Alpha Monitor
EPM ACE Electron, Proton, and Alpha Monitor ACE Electron, Proton, and Alpha Monitor
GIFWALK
... ... ... ... ...
Wind WIND directional-diff-h-flux-10min directional differential h fluxes 10min directional differential h fluxes 10min
directional-diff-he-flux-10min directional differential he fluxes 10min directional differential he fluxes 10min
Wi 3DP Three Dimensional Plasma Instrument Suite Three Dimensional Plasma Instrument Suite
Wind 3DP Three Dimensional Plasma Instrument Suite Three Dimensional Plasma Instrument Suite
wi SWE-Faraday Solar Wind Experiment Faraday Cup Solar Wind Experiment Faraday Cup

2075 rows × 2 columns

Code 
# Display it beautifully
GT(
    obs_groups_instrs_df.head(30),
    rowname_col="Instrument Group",
    groupname_col="Observatory",
).tab_header(
    title="CDAS Observatory Groups and Instruments",
)
CDAS Observatory Groups and Instruments
Instrument Name Short Description Long Description
ACE
AC CRIS ACE Cosmic Ray Isotope Spectrometer ACE Cosmic Ray Isotope Spectrometer
AC DEF Definitive Data Definitive Data
AC EPAM ACE Electron Proton Alpha Monitor ACE Electron Proton Alpha Monitor
AC EPM ACE Electron, Proton, and Alpha Monitor ACE Electron, Proton, and Alpha Monitor
AC GIFWALK
AC MAG ACE Magnetic Field Instrument ACE Magnetic Field Instrument
AC MFI ACE Magnetic Field Instrument ACE Magnetic Field Instrument
AC SEPICA The Solar Energetic Particle Ionic Charge Analyzer The Solar Energetic Particle Ionic Charge Analyzer
AC SIS ACE Solar Isotope Spectrometer ACE Solar Isotope Spectrometer
AC SWE ACE Solar Wind Electron Proton Alpha Monitor ACE Solar Wind Electron Proton Alpha Monitor
AC SWEP ACE Solar Wind Electron Proton Alpha Monitor ACE Solar Wind Electron Proton Alpha Monitor
AC SWICS Solar Wind Ion Composition Spectrometer Solar Wind Ion Composition Spectrometer
AC SWICS_1.1 Solar Wind Ion Composition Spectrometer 1.1 Solar Wind Ion Composition Spectrometer 1.1
AC SWICS_2.0 Solar Wind Ion Composition Spectrometer 2.0 Solar Wind Ion Composition Spectrometer 2.0
AC ULEIS Ultra Low Energy Isotope Spectrometer Ultra Low Energy Isotope Spectrometer
ACE SSC Satellite Situation Center Ephemeris Satellite Situation Center Ephemeris
OMNI (1AU IP Data) IMF and Plasma data IMF and Plasma data IMF and Plasma data
OMNI (1AU IP Data) IMF, Plasma, Indices, Energetic Proton Flux IMF, Plasma, Indices, Energetic Proton Flux IMF, Plasma, Indices, Energetic Proton Flux
OMNI (1AU IP Data) merged magnetic field and plasma data from cohoweb merged magnetic field and plasma data from cohoweb merged magnetic field and plasma data from cohoweb
AIM
AIM CIPS PMC SCI CIPS Cloud Imaging and Particle Size Cloud Imaging and Particle Size
AMPTE
AMPTE-CCE EPI Energetic Particles Energetic Particles
ARTEMIS
THB L1 L1 DATA L1 DATA
THB L2 L2 DATA L2 DATA
THB SSC SSC data SSC data
THB position sc positions sc positions
THC L1 L1 DATA L1 DATA
THC L2 L2 DATA L2 DATA
THC SSC SSC data SSC data
THC position sc positions sc positions
Alouette
Alouette-2 LIM Lima Lima
Code 
from space_analysis.utils.cdas import get_data
Code 
datasets = [
    "JUNO_HELIO1DAY_POSITION",
    "EARTH_HELIO1DAY_POSITION",
    "STA_HELIO1DAY_POSITION",
]
names = ["JUNO", "EARTH", "STA"]

timerange = ["2011-08-01", "2016-07-01"]
Code 
dfs = []
for dataset, name in zip(datasets, names):
    data = get_data(dataset, timerange)
    df = data.to_dataframe()
    df = df.rename(columns={c: f"{name}_{c}" for c in df.columns})
    dfs.append(df)

pd.concat(dfs, axis=1)
2024-06-04 13:27:40.742 | INFO     | space_analysis.utils.cdas:get_dataset_variables:25 - RAD_AU
2024-06-04 13:27:40.742 | INFO     | space_analysis.utils.cdas:get_dataset_variables:25 - SE_LAT
2024-06-04 13:27:40.743 | INFO     | space_analysis.utils.cdas:get_dataset_variables:25 - SE_LON
2024-06-04 13:27:40.743 | INFO     | space_analysis.utils.cdas:get_dataset_variables:25 - HG_LAT
2024-06-04 13:27:40.743 | INFO     | space_analysis.utils.cdas:get_dataset_variables:25 - HG_LON
2024-06-04 13:27:40.743 | INFO     | space_analysis.utils.cdas:get_dataset_variables:25 - HGI_LAT
2024-06-04 13:27:40.744 | INFO     | space_analysis.utils.cdas:get_dataset_variables:25 - HGI_LON
2024-06-04 13:27:41.389 | INFO     | space_analysis.utils.cdas:get_dataset_variables:25 - RAD_AU
2024-06-04 13:27:41.389 | INFO     | space_analysis.utils.cdas:get_dataset_variables:25 - SE_LAT
2024-06-04 13:27:41.390 | INFO     | space_analysis.utils.cdas:get_dataset_variables:25 - SE_LON
2024-06-04 13:27:41.390 | INFO     | space_analysis.utils.cdas:get_dataset_variables:25 - HG_LAT
2024-06-04 13:27:41.390 | INFO     | space_analysis.utils.cdas:get_dataset_variables:25 - HG_LON
2024-06-04 13:27:41.390 | INFO     | space_analysis.utils.cdas:get_dataset_variables:25 - HGI_LAT
2024-06-04 13:27:41.391 | INFO     | space_analysis.utils.cdas:get_dataset_variables:25 - HGI_LON
2024-06-04 13:27:42.070 | INFO     | space_analysis.utils.cdas:get_dataset_variables:25 - RAD_AU
2024-06-04 13:27:42.070 | INFO     | space_analysis.utils.cdas:get_dataset_variables:25 - SE_LAT
2024-06-04 13:27:42.071 | INFO     | space_analysis.utils.cdas:get_dataset_variables:25 - SE_LON
2024-06-04 13:27:42.071 | INFO     | space_analysis.utils.cdas:get_dataset_variables:25 - HG_LAT
2024-06-04 13:27:42.071 | INFO     | space_analysis.utils.cdas:get_dataset_variables:25 - HG_LON
2024-06-04 13:27:42.072 | INFO     | space_analysis.utils.cdas:get_dataset_variables:25 - HGI_LAT
2024-06-04 13:27:42.072 | INFO     | space_analysis.utils.cdas:get_dataset_variables:25 - HGI_LON
JUNO_RAD_AU JUNO_SE_LAT JUNO_SE_LON JUNO_HG_LAT JUNO_HG_LON JUNO_HGI_LAT JUNO_HGI_LON EARTH_RAD_AU EARTH_SE_LAT EARTH_SE_LON ... EARTH_HG_LON EARTH_HGI_LAT EARTH_HGI_LON STA_RAD_AU STA_SE_LAT STA_SE_LON STA_HG_LAT STA_HG_LON STA_HGI_LAT STA_HGI_LON
Epoch
2011-08-02 NaN NaN NaN NaN NaN NaN NaN 1.01 0.0 309.399994 ... 323.100006 5.8 233.199997 0.96 0.0 49.500000 3.2 63.599998 3.2 333.700012
2011-08-03 NaN NaN NaN NaN NaN NaN NaN 1.01 0.0 310.299988 ... 309.899994 5.9 234.199997 0.96 0.0 50.500000 3.1 50.500000 3.1 334.799988
2011-08-04 NaN NaN NaN NaN NaN NaN NaN 1.01 0.0 311.299988 ... 296.700012 6.0 235.199997 0.96 0.0 51.599998 2.9 37.400002 2.9 335.799988
2011-08-05 NaN NaN NaN NaN NaN NaN NaN 1.01 0.0 312.299988 ... 283.500000 6.0 236.100006 0.96 0.0 52.599998 2.8 24.200001 2.8 336.899994
2011-08-06 1.01 0.0 313.299988 6.1 270.299988 6.1 237.199997 1.01 0.0 313.200012 ... 270.299988 6.1 237.100006 0.96 0.0 53.700001 2.7 11.100000 2.7 337.899994
... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ...
2016-06-27 5.44 1.4 177.199997 -5.7 346.899994 -5.7 101.300003 1.02 0.0 275.799988 ... 85.300003 2.5 199.600006 0.97 -0.1 120.199997 -5.2 289.600006 -5.2 44.000000
2016-06-28 5.44 1.4 177.199997 -5.7 332.799988 -5.7 101.300003 1.02 0.0 276.700012 ... 72.000000 2.6 200.600006 0.97 -0.1 121.300003 -5.3 276.500000 -5.3 45.000000
2016-06-29 5.44 1.4 177.300003 -5.7 318.600006 -5.7 101.300003 1.02 0.0 277.700012 ... 58.799999 2.7 201.500000 0.97 -0.1 122.300003 -5.4 263.299988 -5.4 46.099998
2016-06-30 5.44 1.3 177.300003 -5.7 304.399994 -5.7 101.400002 1.02 0.0 278.600006 ... 45.599998 2.8 202.500000 0.97 -0.1 123.300003 -5.4 250.199997 -5.4 47.099998
2016-07-01 5.44 1.3 177.300003 -5.8 290.299988 -5.8 101.400002 1.02 0.0 279.600006 ... 32.299999 2.9 203.399994 0.97 -0.1 124.400002 -5.5 237.000000 -5.5 48.099998

1796 rows × 21 columns