Please enable JavaScript.

Coggle requires JavaScript to display documents.

Run_Workflow.sh, color code:, GFS_EnVar_Cycling.yaml, Driver_yaml, SBATCH…

- - - - Parse CLI (input_file, --test)
      - Load & parse experiment YAML (yamltools.parse_config)
        
        That is the GFS_EnVar_Cycling.yaml, now expconfig <-load yaml
      - Generate experiment ID (Util.get_expid / restart_expid)
      - Template substitution (yamltools.substitute_template_variables)
      - Read component switches (expconfig.COMPONENT_CONFIG['components'])
      - Log workflow type (Monitor_component.info_workflow_runtype)
      - Init parent pointers (parent_component, prior_parent_component)
      - Make workdir (Util.GetWorkDir, os.makedirs)
      - Grid prep (getGrid)
        
        Only run if comp_grid== True
        
        getGrid
        
        Input:
        
        config (dict): experiment configuration
        
        workdir (str): base directory for this component’s output
        
        Output:
        
        None
        
        Steps:
        
        Determine job count
        
        1 more item...
        
        Initialize monitoring
        
        2 more items...
        
        Prepare configuration
        
        1 more item...
        
        Select grid‐creation routine
        
        2 more items...
        
        First grid generation
        
        4 more items...
        
        Ensemble grid (if members enabled)
        
        2 more items...
        
        Log completion (regular run only)
        
        2 more items...
        
        Make‐the‐model‐grid: It checks if it’s already done, loads the right settings, picks the correct grid‐creation function for your chosen model, submits one (or two, for ensembles) batch jobs to build the grid, waits for them to finish, and then logs that “comp_grid” is complete
      - Begin time loop (while cycle < last_cycle)
        
        Need variables 'beg_cycle' ''last_cycle' 'step_cycle'
      - Compute cycle timestamps (beg_cycle, step_cycle)
        
        Need variables 'comp_obs', comp_ini, comp_forecast 'comp_varda 'comp_ensda', comp_recen, OBSERVATIONS_SET2'
      - Update component pointers (for job‑naming/dependency)
      - Create cycle job dirs (Util.Createjobdirs)
      - Observations ingestion (getObservations)
        
        getObservations
        
        Input:
        
        config (dict): experiment configuration (must include RAWOBSERVATIONS, SBATCH_CONFIG, debug_mode, wkflow_run_type, logname)
        
        cycle_date (datetime)
        
        obsworkdir (str): base directory for observation prep
        
        Output:
        
        None
        
        Steps:
        
        Generate date strings
        
        • cycle_date_YMDHM = cycle_date.strftime("%Y%m%d%H%M")
        
        • cycle_date_str2 = cycle_date.strftime("%Y%m%dT%H%M%S")
        
        Initialize monitoring
        
        • compname = 'comp_obs'
        
        • job_num = len(config.RAWOBSERVATIONS)
        
        • jobs_done = initialize_run(config, compname, job_num)
        
        • if jobs_done: return
        
        Read SLURM config
        
        • sbatchconfig = config.SBATCH_CONFIG
        
        Loop over each raw‐observation spec in config.RAWOBSERVATIONS
        
        • obsconf = config with obsconf['obs'] = deepcopy(current spec)
        
        • obstype = obsconf['obs']['obs ingest']['obs_type']
        
        • basedir['wdir'] = os.path.join(obsworkdir, 'obs_prep/'+obstype)
        
        • basedir['cycle'] = cycle_date_str2
        
        • os.makedirs(basedir['wdir'], exist_ok=True)
        
        Fetch observations
        
        • job1 = processobs.fetchObservations(obsconf, basedir)
        
        1 more item...
        
        • Util.createENV_and_runjob(basedir['wdir'], 'fetchObservations', job1)
        
        Convert observations
        
        • job2 = processobs.convertObservations(obsconf, basedir, rawfile=job1.output)
        
        1 more item...
        
        • Monitor_component.monitor_jobs(config, compname, 'obs_prep', 'convertObservations', job2, sbatchconfig['obs'], basedir['wdir'])
        
        • if not config['debug_mode']: Util.wait_single_run(basedir['wdir'])
        
        Store observations
        
        • job3 = processobs.storeObservations(obsconf, {'wdir': obsworkdir}, iodafiles=job2.output)
        
        1 more item...
        
        • yamltask = os.path.join(obsworkdir+'/obs_prep/'+obstype, "storeObservations.yaml")
        
        • runtimeconf = deepcopy(job3.RUNTIME_YAML); runtimeconf['ENV'] = job3.RUNTIME_ENV
        
        • yamltools.save_template(runtimeconf, yamltask)
        
        Final logging (regular run only)
        
        • logfile = os.path.join(Util.GetWorkDir(config), config['logname'])
        
        • logger = Util.Running_log(logfile, compname)
        
        • logger.info(f"Finish comp_obs @ {cycle_date_YMDHM}")
        
        getObservations takes each raw‐data source you’ve configured, fetches its files, converts them into the IODA schema, stores them into your central obs directory, and logs the completion.
      - Initial conditions (getInitialconditions)
        
        getInitialconditions
        
        Input:
        
        • config (dict): must include
        
        – ‘model’ (‘mpas’ or ‘HAFS’), optional ‘ENSDA’, ‘VARDA’, ‘members’
        
        – BACKGROUNDS, MODEL_CONF, SBATCH_CONFIG
        
        – debug_mode, wkflow_run_type, expid, workdir, logname
        
        • cyc_date (datetime)
        
        • workdir (str)
        
        Output:
        
        None
        
        Steps:
        
        Format cycle timestamps
        
        • cyc_date_YMDHM = cyc_date.strftime("%Y%m%d%H%M")
        
        • cyc_date_str2 = cyc_date.strftime("%Y%m%dT%H%M%S")
        
        • uses datetime.strftime
        
        Determine whether we only run EnKF (no variational step)
        
        • ensda_only = True if 'ENSDA' in config and 'VARDA' not in config
        
        Compute number of IC jobs required
        
        • If model=='mpas':
        
        1 more item...
        
        • If model=='HAFS':
        
        1 more item...
        
        • uses config['members']
        
        Initialize component monitoring and exit if already done
        
        • jobs_done = Monitor_component.initialize_run(config, 'comp_ini', job_num)
        
        • if jobs_done: return
        
        Merge background, model, and sbatch configurations
        
        • config.update(config.BACKGROUNDS[0])
        
        • config.update(config.MODEL_CONF)
        
        • config.update(config.SBATCH_CONFIG['ini'])
        
        • uses config.update
        
        Select IC creation routine based on model
        
        • creatIC = processics.mpas or processics.hafs
        
        1 more item...
        
        • module: processInitialconditions
        
        Determine ensemble member loop range
        
        • members = config.get('members',0)+1
        
        • beg_mem = 1 if ensda_only else 0
        
        • uses copy.deepcopy later
        
        Loop over each member to submit IC jobs
        
        • For jmember in range(beg_mem, members):
        
        6 more items...
        
        Wait for IC jobs to finish in regular runs
        
        • if not debug_mode and wkflow_run_type=='regular':
        
        2 more items...
        
        For HAFS, prepare LBCs and ocean component
        
        • LBCs: repeat loop in step 8 with usage='LBC', subdir 'atm_lbc[_ens]'
        
        2 more items...
        
        • Ocean prep:
        
        4 more items...
        
        Final logging for regular runs
        
        • logger = Util.Running_log(os.path.join(Util.GetWorkDir(config), config['logname']), 'comp_ini')
        
        • logger.info(f"Finish comp_ini @ {cyc_date_YMDHM}")
        
        builds model’s initial conditions for deterministic and ensemble
      - Forecast run (getForecast)
        
        getForecast
        
        Input:
        
        • config (dict): must include
        
        – ‘model’ (‘mpas’, ‘HAFS’, or ‘gfs’)
        
        – optional ‘ENSDA’/’VARDA’ flags and ‘members’ count
        
        – BACKGROUNDS, SBATCH_CONFIG, MODEL_CONF (for non-GFS)
        
        – debug_mode, wkflow_run_type, expid, workdir, logname
        
        • cycle_date (datetime): the current analysis time
        
        • workdir (str): base folder for forecast outputs
        
        Output:
        
        None
        
        Steps:
        
        Create two date labels
        
        • cycle_date_YMDHM = “YYYYmmddHHMM” (folder names)
        
        • cycle_date_str2 = “YYYYmmddTHHMMSS” (timestamp)
        
        Decide if we only run ensemble updates (EnKF)
        
        • ensda_only = True when config has ‘ENSDA’ but no ‘VARDA’
        
        Calculate how many forecast jobs to launch
        
        • job_num = config['members'] or config['members']+1, or 1 if no members
        
        Check if forecast step is already done
        
        • jobs_done = Monitor_component.initialize_run(config, 'comp_fcst', job_num)
        
        • If jobs_done True → exit
        
        Merge background and SLURM settings into config
        
        • config.update(config.BACKGROUNDS[0])
        
        • config.update(config.SBATCH_CONFIG['forecast'])
        
        • If model!='gfs', also config.update(config.MODEL_CONF)
        
        Pick the correct forecast routine
        
        • creatFCST = processForecast.mpas / .hafs / .gfs
        
        1 more item...
        
        Determine which member index to start with
        
        • beg_mem = 1 if ensda_only else 0
        
        • members = config.get('members',0) + 1
        
        Loop over each member to submit forecast jobs
        
        • For jmember in [beg_mem…members):
        
        5 more items...
        
        Wait for forecast jobs in regular runs
        
        • If not debug and wkflow_run_type=='regular':
        
        2 more items...
        
        Write a final log message (regular runs)
        
        • logpath = Util.GetWorkDir(config)/config['logname']
        
        • logger = Util.Running_log(logpath, 'comp_fcst')
        
        • logger.info("Finish comp_fcst @ "+cycle_date_YMDHM)
        
        submitting and monitoring the forecast jobs
      - Variational analysis (getAnalysis, “group1”/“group2”)
        
        getAnalysis
        
        Input:
        
        • config (dict): must include
        
        – ‘step_cycle’ (ISO timedelta string)
        
        – SBATCH_CONFIG, VARDA, MODEL_CONF
        
        – OBSERVATIONS (list), optional OBSERVATIONS_SET2 (list)
        
        – AN_VARIABLES, BGAN_VARIABLES_EXTRA, BACKGROUND_ERROR_LOCALIZATION_OBSSET2
        
        – debug_mode, wkflow_run_type, expid, workdir, logname
        
        • cycle_date (datetime): current analysis time
        
        • prior_cyc_workdir (str): folder from previous cycle
        
        • workdir (str): folder for this cycle’s analysis
        
        • obs_type (str): ‘group1’ or ‘group2’
        
        Output:
        
        None
        
        Steps:
        
        Compute previous cycle time
        
        • date_prior = yamltools.parse_datetime(cycle_date)
        
        1 more item...
        
        • sdate_prior = yamltools.jediformat(date_prior)
        
        • cyc_prior_YMDHM = format as “YYYYmmddHHMM”
        
        • uses: yamltools.parse_datetime, parse_timedelta, jediformat, datetime.strptime, strftime
        
        Format current cycle labels
        
        • cycle_date_YMDHM = cycle_date.strftime("YYYYmmddHHMM")
        
        • cycle_date_str2 = cycle_date.strftime("YYYYmmddTHHMMSS")
        
        • uses: datetime.strftime
        
        Initialize monitoring for one analysis job
        
        • job_num = 1; compname = 'comp_varda'
        
        • jobs_done = Monitor_component.initialize_run(config, compname, job_num)
        
        • if jobs_done True → return
        
        • uses: Monitor_component.initialize_run
        
        Build analysis working folder
        
        • varsubdir = f"analysis{obs_type}"
        
        • basedir['wdir']=os.path.join(workdir, var_subdir)
        
        • basedir['prior_cyc_wdir']=prior_cyc_workdir
        
        • basedir['cycle']=cycle_date_str2; basedir['prior_cycle']=cyc_prior_YMDHM
        
        • basedir['obs_type']=obs_type; os.makedirs(basedir['wdir'])
        
        Merge config for variational DA
        
        • varconfig = deepcopy(config)
        
        • varconfig.update(config['VARDA'])
        
        • varconfig.update(config.SBATCH_CONFIG['analysis'])
        
        • if model!='gfs': varconfig.update(config.MODEL_CONF)
        
        Fetch static support files
        
        • fixfiles = processana.getFixFiles(varconfig, basedir)
        
        1 more item...
        
        • ffiles = fixfiles.output
        
        • Util.createENV_and_runjob(basedir['wdir'], 'getGeometryFiles', fixfiles)
        
        • uses: processAnalysis.getFixFiles, Util.createENV_and_runjob
        
        Gather observations
        
        • obs_config = config.OBSERVATIONS or OBSERVATIONS_SET2
        
        • for each obs spec:
        
        3 more items...
        
        • uses: processAnalysis.getIODAObservations
        
        Get background fields
        
        • bgtmp = processana.getBackground(varconfig, basedir); bg = bgtmp.output
        
        • bgenstmp = processana.getBackgroundENS(varconfig, basedir); bgens = bgenstmp.output
        
        • uses: processAnalysis.getBackground, getBackgroundENS
        
        Run variational analysis
        
        • varDA = processana.variational
        
        1 more item...
        
        • job = varDA(varconfig, basedir, fix=ffiles, bg=bg, bgens=bgens, obs=obs)
        
        Write JEDI‐run YAML template
        
        • runtimeconf = deepcopy(job.RUNTIME_YAML['jeditmpl'])
        
        • yamltask = basedir['wdir']+'/jedi_var.yaml'
        
        • yamltools.save_template(runtimeconf, yamltask)
        
        • uses: yamltools.save_template
        
        Submit and monitor analysis job
        
        • Monitor_component.monitor_jobs(
        
        4 more items...
        
        • if regular run & not debug: Util.wait_single_run(basedir['wdir'])
        
        • uses: Monitor_component.monitor_jobs, Util.wait_single_run
        
        Final logging (regular run only)
        
        • logpath = Util.GetWorkDir(config)+'/'+config['logname']
        
        • logger = Util.Running_log(logpath, compname)
        
        • logger.info(f"Finish comp_varda @ {cycle_date_YMDHM}")
        
        sets up and runs a 3D‐Var analysis for the current cycle and a specified group of observations
      - Ensemble DA (getAnalysisENS, “group1”/“group2”)
      - Recenter analysis (getRecenter)
      - Loop increment (icycle += 1)
      - Mark completion (write ALLDONE file)
    - - External Files & Modules Involved
      - Experiment YAML passed as input_file
      - pyOUMAP.tools.yamltools (YAML parsing & date helpers)
        
        yamltools
      - pyOUMAP.tools.Monitor_component (logging)
        
        Monitor_component.py
        
        initialize_json
        
        Input:
        
        json_file (str), parent_component (str),
        
        child_component (str), jobnum (int)
        
        Output:
        
        bool: component_done
        
        Steps:
        
        If json_file missing, init data with parent→child empty list
        
        Else read existing data
        
        Ensure parent→child keys exist
        
        If entries exist, prune/update via check_all_jobs_in_component
        
        Write data back via write_json
        
        check_all_jobs_in_component
        
        Input:
        
        data (list of dict), jobnum (int)
        
        Output:
        
        (bool comp_status, updated data)
        
        Steps:
        
        Initialize comp_status=True, ijob=0
        
        Loop over each component dict:
        
        4 more items...
        
        If comp_status True but ijob<jobnum → comp_status=False
        
        Return (comp_status, data)
        
        write_json
        
        Input:
        
        json_file (str), data (object)
        
        Output:
        
        None
        
        Steps:
        
        Open json_file for write
        
        Dump data with indent=4
        
        Catch and print errors
        
        read_json
        
        Input:
        
        json_file (str)
        
        Output:
        
        dict or None
        
        Steps:
        
        Open json_file for read
        
        Load and return JSON
        
        Catch/print errors, return None
        
        check_job_log
        
        Input:
        
        json_file, parent_component, child_component, grandchild_component
        
        Output:
        
        str: status or "NEW"
        
        Steps:
        
        Read data via read_json
        
        Scan data[parent][child] for entry name==grandchild
        
        Return its status
        
        check_job_status
        
        Input:
        
        job_id (str)
        
        Output:
        
        str: "RUNNING"|"COMPLETED"|"UNKNOWN"|"ERROR"
        
        Steps:
        
        Run subprocess ['squeue','--job',job_id]
        
        If stdout contains job_id → "RUNNING"
        
        Else return "COMPLETED" or "UNKNOWN" on error
        
        add_job_to_json
        
        Input:
        
        json_file, parent_component, child_component,
        
        grandchild_component, job_id, workdir,
        
        script_name, output_flag, dependency_comp (list)
        
        Output:
        
        None
        
        Steps:
        
        Read data via read_json
        
        Determine status = "NEW" or "SUBMITTED"
        
        Build new_component dict
        
        If name exists, overwrite; else append
        
        Write data back
        
        monitor_or_resubmit
        
        Input:
        
        json_file, parent_component, child_component, grandchild_component
        
        Output:
        
        None
        
        Steps:
        
        Read data
        
        Find target entry; sleep 1 s
        
        Check current Slurm status
        
        If RUNNING → update status, continue
        
        If COMPLETED/UNKNOWN →
        
        2 more items...
        
        If FAILED & runtimes≤9 → resubmit via Util.Submit_batch_job, update job_id/status/runtimes
        
        Write updated data back
        
        info_workflow_runtype
        
        Input:
        
        config (dict)
        
        Output:
        
        None
        
        Steps:
        
        If wkflow_run_type=="regular":
        
        2 more items...
        
        If "efficient":
        
        1 more item...
        
        initialize_run
        
        Input:
        
        config (dict), child_component (str), job_num (int)
        
        Output:
        
        None or bool (jobs_done)
        
        Steps:
        
        If regular → set up logger via Util.Running_log
        
        If efficient → call initialize_json and return its bool
        
        obtain_json_file
        
        Input:
        
        config (dict)
        
        Output:
        
        str: full JSON file path
        
        Steps:
        
        Join GetWorkDir(config) and config['jsonname']
        
        update_log_json
        
        Input:
        
        config, child_component, job_id, job_script_name, workdir, grandchild_component, flagfile
        
        Output:
        
        None
        
        Steps:
        
        If regular → log INFO via Util.Running_log
        
        If efficient → build dependency_comp list
        
        1 more item...
        
        dependent_comp_status
        
        Input:
        
        data (dict), dependent_component (str with ""), config
        
        Output:
        
        bool: all-success
        
        Steps:
        
        Split dependent_component into prior_child, parent_key
        
        For each entry in data[parent_key][prior_child]:
        
        1 more item...
        
        Return True
        
        check_component_dependency
        
        Input:
        
        config, json_file, parent_component, child_component
        
        Output:
        
        bool or None
        
        Steps:
        
        Read data via read_json
        
        For each entry in data[parent][child]:
        
        1 more item...
        
        Return final bool
        
        monitor_jobs
        
        Input:
        
        config, child_component, grandchild_component,
        
        job_name, job (object), sbatchconfig, workdir, flagfile
        
        Output:
        
        None
        
        Steps:
        
        Derive parent_component, json_file
        
        If efficient and status=="NEW":
        
        4 more items...
        
        If regular → createENV_and_runjob
        
        If efficient → monitor_or_resubmit
        
        Monitoring Job Status
      - pyOUMAP.tools.Util (ID generation, working‐dir helpers)
        
        Util.py
        
        get_expid
        
        Input:
        
        1 more item...
        
        Output:
        
        1 more item...
        
        Steps:
        
        4 more items...
        
        Createjobdirs
        
        Input:
        
        3 more items...
        
        Output:
        
        1 more item...
        
        Steps:
        
        5 more items...
        
        GetWorkDir
        
        Input:
        
        1 more item...
        
        Output:
        
        1 more item...
        
        Steps:
        
        2 more items...
        
        Parse_jedi_date
        
        Input:
        
        3 more items...
        
        Output:
        
        1 more item...
        
        Steps:
        
        4 more items...
        
        Createjobenv
        
        Input:
        
        2 more items...
        
        Output:
        
        1 more item...
        
        Steps:
        
        2 more items...
        
        Create_login_job
        
        Input:
        
        1 more item...
        
        Output:
        
        1 more item...
        
        Steps:
        
        3 more items...
        
        Exec_login_job
        
        Input:
        
        1 more item...
        
        Output:
        
        1 more item...
        
        Steps:
        
        1 more item...
        
        Create_batch_job
        
        Input:
        
        2 more items...
        
        Output:
        
        1 more item...
        
        Steps:
        
        4 more items...
        
        Submit_batch_job
        
        Input:
        
        1 more item...
        
        Output:
        
        1 more item...
        
        Steps:
        
        4 more items...
        
        copyfile
        
        Input:
        
        1 more item...
        
        Output:
        
        1 more item...
        
        Steps:
        
        2 more items...
        
        movefile
        
        Input:
        
        1 more item...
        
        Output:
        
        1 more item...
        
        Steps:
        
        2 more items...
        
        wait_single_run
        
        Input:
        
        1 more item...
        
        Output:
        
        1 more item...
        
        Steps:
        
        2 more items...
        
        wait_ensemble_run
        
        Input:
        
        1 more item...
        
        Output:
        
        1 more item...
        
        Steps:
        
        3 more items...
        
        obsname_check
        
        Input:
        
        1 more item...
        
        Output:
        
        1 more item...
        
        Steps:
        
        2 more items...
        
        createENV_and_runjob
        
        Input:
        
        2 more items...
        
        Output:
        
        1 more item...
        
        Steps:
        
        4 more items...
        
        Running_log
        
        Input:
        
        1 more item...
        
        Output:
        
        1 more item...
        
        Steps:
        
        5 more items...
        
        Helper Functions related to Job submission
      - pyOUMAP.preObservations.getObservations
      - pyOUMAP.preGrid.getGrid
      - pyOUMAP.preInitialconditions.getInitialconditions
      - pyOUMAP.preForecast.getForecast
      - pyOUMAP.preAnalysis.getAnalysis
      - pyOUMAP.preAnalysisENS.getAnalysisENS
      - pyOUMAP.preRecenter.getRecenter
      - Task.py
        
        Task (class)
        
        Required Input:
        
        • config (dict)
        
        • workdir (dict)
        
        Optional Input:
        
        • **inputs (kwargs)
        
        Self Variables:
        
        • output
        
        • RUNTIME_YAML
        
        • RUNTIME_ENV
        
        • workdir
        
        • nodes
        
        • mpi_per_node
        
        • threads_per_task
        
        • walltime
        
        • triggers
        
        Methods:
        
        • init(config, workdir, **inputs)
        
        • setup(config, **inputs) # implemented by subclasses
        
        Initialize JEDI Tasks
- - - - eastward_wind
        
        northward_wind
        
        air_temperature
        
        surface_pressure
        
        specific_humidity
        
        cloud_liquid_water
        
        ozone_mass_mixing_ratio
        
        Claim ETKF Analysis Variables
  - - - u_component_of_native_D_grid_wind
      - v_component_of_native_D_grid_wind
      - eastward_wind
      - northward_wind
      - air_temperature
      - surface_pressure
      - specific_humidity
      - Claim Analysis Variables
  - - - u
      - v
      - eastward_wind
      - northward_wind
      - air_temperature
      - air_pressure_thickness
      - surface_pressure
      - specific_humidity
      - cloud_liquid_ice
      - cloud_liquid_water
      - rainwat
      - snowwat
      - graupel
      - ozone_mass_mixing_ratio
      - sfc_geopotential_height_times_grav
      - slmsk
      - sheleg
      - sea_surface_temperature
      - vtype
      - stype
      - vfrac
      - stc
      - smc
      - snwdph
      - surface_eastward_wind
      - surface_northward_wind
      - f10m
      - surface_temperature
      - zorl
      - Claim Bckground Variables
  - - - u
      - v
      - eastward_wind
      - northward_wind
      - air_temperature
      - air_pressure_thickness
      - surface_pressure
      - specific_humidity
      - cloud_liquid_ice
      - cloud_liquid_water
      - rainwat
      - snowwat
      - graupel
      - ozone_mass_mixing_ratio
      - sfc_geopotential_height_times_grav
      - slmsk
      - sheleg
      - sea_surface_temperature
      - vtype
      - stype
      - vfrac
      - stc
      - smc
      - snwdph
      - surface_eastward_wind
      - surface_northward_wind
      - f10m
      - surface_temperature
      - zorl
      - Claim Model Variables (usually same as bg)
  - - - npx: 97
      - npy: 97
      - npz: 127
      - ntiles: 6
      - layout: $(mpi_layout)
      - io_layout: [1,1]
      - _resol_name: c96
      - Model Grid Geometry High Res
  - - - SABER localization scheme in JEDI
  - - - npx
      - npy
      - npz
      - ntiles
      - layout: $(mpi_layout)
      - io_layout
      - _resol_name
      - Model Grid Geometry Low Res
  - - - observation localizations
      - localization method
      - lengthscale
  - - - do posterior observer: false
      - do test prints: false
      - save posterior mean: false
      - save posterior ensemble: false
      - save prior mean: false
      - save posterior mean increment: false
      - run as observer only: true
      - update obs config with geometry info: false
      - read HX from disk: false
      - ON/OFF JEDI observation/assimilation driver (old)
  - - - do posterior observer: false
      - do test prints: false
      - save posterior mean: true
      - save posterior ensemble: true
      - save prior mean: true
      - save posterior mean increment: true
      - save prior variance: true
      - save posterior variance: true
      - run as observer only: false
      - update obs config with geometry info: true
      - read HX from disk: true
      - ON/OFF JEDI assimilation driver
  - - - solver: $(SOLVER)
      - use linear observer: false
      - vertical localization:
        
        fraction of retained variance: .9
        
        lengthscale: 0.5
        
        lengthscale units: logp
      - inflation:
        
        rtps: 0.9
        
        rtpp: 0.0
        
        mult: 1.0
      - GETKF Setting
  - - - obs space:
      - name
      - obsdatain
      - obsdataout
      - io pool
      - _source
      - simulated variables
      - obs operator:
      - name
      - components
      - vertical coordinate
      - observation vertical coordinate
      - interpolation method
      - variables
      - linear obs operator:
      - name
      - components
      - variables
      - obs filters:
      - filter
      - assignments
      - name
      - value
      - type
      - function
      - options
      - variables
      - filter variables
      - minvalue
      - maxvalue
      - action
      - test variables
      - error parameter
      - error function
      - scale_factor_var
      - xvar
      - xvals
      - errors
      - threshold
      - absolute threshold
      - defer to post
      - where
      - variable
      - inflation variable
      - error_min
      - error_max
      - Air Craft Observations Configuration
  - - - obs space:
      - name
      - obsdatain
      - engine
      - type
      - obsgrouping
      - group variables
      - sort variable
      - sort order
      - obsdataout
      - allow overwrite
      - io pool
      - write multiple files
      - _source
      - simulated variables
      - obs operator:
      - name
      - vertical coordinate
      - observation vertical coordinate
      - interpolation method
      - obs filters:
      - filter
      - filter variables
      - minvalue
      - maxvalue
      - test variables
      - options
      - minimum_uv
      - action
      - defer to post
      - error parameter
      - error function
      - xvar
      - xvals
      - errors
      - where
      - variable
      - is_in
      - threshold
      - absolute threshold
      - Surface Pressure Observations Configuration
  - - - bg ingest:
        
        provider: 'noaa'
        
        bg_name: 'gep'
        
        bg_type: 'grib2'
        
        bg_interval: 3
        
        usage: 'IC'
        
        path: '/work2/noaa/uswrp-oum/yongming/MPAS_run_data/ensemble_BG'
      - grab background (GEFS) fields from disk
  - - - bg ingest:
        
        provider: 'noaa'
        
        bg_name: 'gfs'
        
        bg_type: 'grib2'
        
        bg_interval: 1
        
        usage: 'LBC'
        
        path: '/work2/noaa/wrfruc/yongming/ensemble_BG'
      - grab background (GFS) fields from disk
  - - - obs:
        
        account: uswrp-oum
        
        queue: orion
        
        partition: orion
        
        ntasks: 1
        
        omp_threads: 1
        
        taskspernode: 40
        
        walltime: '00:15:00'
      - grid
      - ini
      - ocn_prep
      - forecast
      - analysis
      - ens_analysis
      - recenter
      - bump
      - SLURM submission parameters
  - - - components: first cycle
        
        comp_grid: False # Not implemented yet ...
        
        comp_ini: False # Not implemented yet ...
        
        comp_obs: False # Set False if you have obs in IODA format available
        
        comp_fcst: True #was true, modified apr 7
        
        comp_varda: True #was true, modified apr 7
        
        comp_ensda: True
        
        comp_recen: False # Not implemented yet ...
      - dependency: depend on last cycle
        
        comp_grid:
        
        null
        
        comp_obs:
        
        null
        
        comp_ini:
        
        comp_grid@prior_parent_component
        
        comp_fcst:
        
        comp_ini@parent_component
        
        comp_recen@prior_parent_component
        
        comp_varda:
        
        comp_obs@parent_component
        
        comp_fcst@parent_component
        
        comp_ensda:
        
        comp_obs@parent_component
        
        comp_fcst@parent_component
        
        comp_recen:
        
        comp_varda@parent_component
        
        comp_ensda@parent_component
      - Pipeline Key Settings