Skip to content

obs_common_subset.f90 comments #1196

Open

Description

@hkershaw-brown

Some of the documentation is a bit misleading.
In the obs_common_subset.f90 header is written:

!>  The obs to be assimilated or evaluated can be selected by namelist
!>  at run time if they are different, and different experiment settings
!>  can be selected by other namelist values.

The ensemble size cannot differ between the files because it affects the structure of the files,
not just the contents (numbers, character strings, ...). There's a check of ensemble sizes,
which stops the program if they differ. It would be helpful to change the error message from

      write(msgstring2,*)'Different numbers of data copies found: ', &
                         num_copies1, ' vs ', num_copiesN

to

     write(msgstring2,*)'Different numbers of data copies found: ', &
                         num_copies1, ' vs ', num_copiesN, &
                         '.  Remove all ensemble copies from all files using obs_sequence_tool.'

or put the "Remove" line into msgstring1 and add msgstring1 to the error_handler call.


And this feature raises raises a question;

!>  If the next input obs don't match exactly (identical types, times, locations,
!>  and DART QC) they are discarded.

If the obs_seq files have the same obs, but in different orders, then some common obs won't be found
and the resulting obs_seq.common files will be misleading. It seems that the program should stop,
with instructions for how to fix the problem (if it's fixable).

Originally posted by @kdraeder in #808

Activity

  1. hkershaw-brown commented on Nov 2, 2026

    @hkershaw-brown
    CollaboratorAuthor

    A lot (all?) of these observation sequence programs maybe better done in pyDARTdiags.

    https://docs.dart.ucar.edu/en/latest/assimilation_code/programs/readme.html

    Creating Observation Sequence Files
    create_obs_sequence
    Creates a short andor synthetic observation sequence file using values read from standard input.
    
    create_fixed_network_seq
    Reads observation sequence file information from standard input and replicates it multiple times in a second observation sequence file, at user specified dates.
    
    obs_utils/create_obs_grid
    Create a set of observations located on a regular grid. Obs have no data values, but they are time ordered.
    
    obs_utils/obs_timejitter
    Randomly perturb the times of the observations in a (usually) set_def.out file. Writes the results to (usually) obs_seq.in.
    
    Querying Observation Sequence Files
    obs_utils/obs_info
    Summarize obs types, times, counts found in observation sequence file(s).
    
    obs_utils/obs_assim_count
    Prints out a quick table of obs types and counts, overall start and stop times, and metadata strings and counts. See obs_diag for more. There is an older version in the obs_assim_count directory.
    
    obs_seq_coverage
    Queries a set of observation sequence files to determine which observation locations report frequently enough to be useful for a verification study.
    
    obs_total_error
    Prints the total error in the mean and spread from an obs_seq file which has been through both perfect_model_obs and filter, so it has copies ‘truth’, ‘ensemble mean’, and ‘ensemble spread’. You can get more information by running the obs_diag program.
    
    Changing Observation Sequence Files
    obs_sequence_tool
    Subsets, combines, or alters observations from one or more observation sequence files and optionally writes them into a single output obs_seq file.
    
    obs_loop
    A template to read in observations from one obs_seq file and write them, optionally modified by user supplied code, to another obs_seq file.
    
    obs_utils/obs_sort
    Do a complete sort of an obs_seq file by location, observation type, then variance. An ancestor of obs_remove_dups.
    
    obs_utils/obs_remove_dups
    Removes duplicate observations from an obs_seq file, which involves a complete sort by time, location, observation type, then variance.
    
    obs_selection
    Extracts observations out of one or more obs_sequence files according to a list of observation types, times, and locations. The list is usually created by obs_seq_coverage, but can be an observation sequence file.
    
    obs_common_subset
    Select the subset of observations, which were successfully assimilated, from two or more assimilation cases (which used the same obs_seq.out file).
    
    obs_keep_a_few
    Creates an output observation sequence file that is shorter than the input obs_seq file.
    
    obs_seq_verify
    Reorders the observations from a forecast run of DART into a structure that is amenable for the evaluation of the forecast.
    
    obs_utils/obs_data_denial
    THIS IS NOT YET DONE! Help implement a data-denial experiment by randomly changing the error variance of N of each obs type in an observation sequence file to a huge value.
    
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions