ensemble manager: layout, layout_type, distribution_type

There are a couple of confusing (to me at least) parts of ensemble_manager, layout_type argument, distribution_type argument.

distribution_type you can not use:

DART/assimilation_code/modules/utilities/ensemble_manager_mod.f90

Lines 137 to 145 in b270ca6

    
           ! Distribution type controls pe layout of storage; Default is 1. 1 is only one implemented. 
        
           if(.not. present(distribution_type_in)) then 
        
              ens_handle%distribution_type = 1 
        
           else 
        
              ! Check for error: only type 1 implemented for now 
        
              if(distribution_type_in /= 1) call error_handler(E_ERR, 'init_ensemble_manager', & 
        
                 'only distribution_type 1 is implemented', source) 
        
              ens_handle%distribution_type = distribution_type_in 
        
           endif

You could have set a different layout (mapping of physical to logical tasks) when you initialize the ensemble handles. There is a note in ensemble manager that you should no do this.

DART/assimilation_code/modules/utilities/ensemble_manager_mod.f90

Lines 165 to 172 in b270ca6

 ! Optional layout_type argument to assign how my_pe is related to my_task_id 
 ! layout_type can be set individually for each ensemble handle. It is not advisable to do this 
 ! because get_obs_ens assumes that the layout is the same for each ensemble handle. 
 if(.not. present(layout_type) ) then 
    ens_handle%layout_type = layout ! namelist option 
 else 
    ens_handle%layout_type = layout_type 
 endif

Q. Why even have layout as an option, many parts of the code assume the layout is the same for ens, obs_ens, and qc_obs_ens? "Do not shoot your self in the foot with this foot gun we have provided"

Thoughts:

Distribution type has been unused for 20 years. I think it is time for it to go. If you did need this functionally is this where you would put it today?
map_pe_to_task is crazy annoying to have to call. Something should be taking care of that for me, e.g the mpi_utils or the ensemble manager or another layer.

I agree that map_pe_to_task should not be part of the ensemble manager everything else being equal. I do not want to be so fast in getting rid of the layout. I can think of reasons why it would be needed to avoid load balancing issues and we definitely confronted them before. As noted in the discussion, I don't think that we changed all the mappings, but maybe we did. However, I can think of good reasons why we might want different mappings. In particular, if we ever implemented observation groupings to either do correlated error or to allow increased parallelism by the methods covered in Anderson, Kershaw and Collins, we would need to stash groups of obs together for performance. If the capability is currently not fully supported, I would vote for keeping the ability in the ensemble_manager but temporarily disabling its use, rather than pulling it out. It doesn't seem to be very intrusive to me.

…

On Thu, Aug 13, 2026 at 1:12 PM Helen Kershaw ***@***.***> wrote: *hkershaw-brown* created an issue (NCAR/DART#1166) <#1166> There are a couple of confusing (to me at least) parts of ensemble_manager, layout_type argument, distribution_type argument. You could have set a different layout (mapping of physical to logical tasks) when you initialize the ensemble handles. There is a note in ensemble manager that you should no do this. DART/assimilation_code/modules/utilities/ensemble_manager_mod.f90 <https://github.com/NCAR/DART/blob/b270ca69bee4915141f0948043c64d5c9b263e5a/assimilation_code/modules/utilities/ensemble_manager_mod.f90#L165-L172> Lines 165 to 172 in b270ca6 <b270ca6> ! Optional layout_type argument to assign how my_pe is related to my_task_id ! layout_type can be set individually for each ensemble handle. It is not advisable to do this ! because get_obs_ens assumes that the layout is the same for each ensemble handle. if(.not. present(layout_type) ) then ens_handle%layout_type = layout ! namelist option else ens_handle%layout_type = layout_type endif Q. Why even have layout as an option, many parts of the code assume the layout is the same for ens, obs_ens, and qc_obs_ens? "Do not shoot your self in the foot with this foot gun we have provided" Thoughts: - Distribution type has been unused for 20 years. I think it is time for it to go. If you did need this functionally is this where you would put it today? - map_pe_to_task is crazy annoying to have to call. Something should be taking care of that for me, e.g the mpi_utils or the ensemble manager or another layer. — Reply to this email directly, view it on GitHub <#1166?email_source=notifications&email_token=ANDHUIWU4UVX6NTTR4E3YA35JYHKFA5CNFSL4Z3JMQ5C6L3HNF2C22DVMIXUS43TOVSS6NJRGQ2DSNRSHAYTPJTSMVQXG33OVJZXKYTTMNZGSYTFMSSWK5TFNZ2KYZTPN52GK4S7MNWGSY3L>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ANDHUIQ7XMRCUQ6LIYO2MW35JYHKFAVCNFSNUABFKJSXA33TNF2G64TZHMZDAOJRGAZTSOJVHNEXG43VMU5TKMJUGQ4TMMRYGE32C5QC> . Triage notifications, keep track of coding agent tasks and review pull requests on the go with GitHub Mobile for iOS <https://github.com/notifications/mobile/ios/ANDHUIT5SQOW3DMMJ3GWQW35JYHKFA5CNFSL4Z3JMQ5C6L3HNF2C22DVMIXUS43TOVSS6NJRGQ2DSNRSHAYTPJTSMVQXG33OVJZXKYTTMNZGSYTFMSSWK5TFNZ2KUZTPN52GK4S7NFXXG> and Android <https://github.com/notifications/mobile/android/ANDHUIRQL6LQDLHF3RGIFU35JYHKFA5CNFSL4Z3JMQ5C6L3HNF2C22DVMIXUS43TOVSS6NJRGQ2DSNRSHAYTPJTSMVQXG33OVJZXKYTTMNZGSYTFMSSWK5TFNZ2K4ZTPN52GK4S7MFXGI4TPNFSA>. Download it today! You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

@jlaucar

@jlaucar are you talking about the layout_type or the distribution_type?

I think it is good to get all the terminology and assumptions in the code listed before any changes, in fact before even speccing out changes.
e.g. I'm assuming you are not talking about different distributions of copies, only distributions of vars.

I believe the load balancing issues you're thinking of: all elements in a column ending up on the same processor ( unlucky number of tasks vs x*y grid cells) was "fixed" by using a different number of tasks. I may be missing other cases though.

Helen, Good idea on terminology since I'm not sure I understand the distinction between 'layout_type' and 'distribution_type'.

…

On Thu, Aug 13, 2026 at 2:26 PM Helen Kershaw ***@***.***> wrote: *hkershaw-brown* left a comment (NCAR/DART#1166) <#1166 (comment)> I think it is good to get all the terminology and assumptions in the code listed before any changes, in fact before even speccing out changes. e.g. I'm assuming you are not talking about different distributions of copies, only distributions of vars. I believe the load balancing issues you're thinking of: all elements in a column ending up on the same processor ( unlucky number of tasks vs x*y grid cells) was "fixed" by using a different number of tasks. I may be missing other cases though. — Reply to this email directly, view it on GitHub <#1166?email_source=notifications&email_token=ANDHUIUXTNTOTKVMBGPRXUL5JYP7JA5CNFSNUABFM5UWIORPF5TWS5BNNB2WEL2JONZXKZKDN5WW2ZLOOQXTKMRYGYYDAMRVGI42M4TFMFZW63VHNVSW45DJN5XKKZLWMVXHJLDGN5XXIZLSL5RWY2LDNM#issuecomment-5286002529>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ANDHUIVFDCG36T4E35277VL5JYP7JAVCNFSNUABFKJSXA33TNF2G64TZHMZDAOJRGAZTSOJVHNEXG43VMU5TKMJUGQ4TMMRYGE32C5QC> . Triage notifications, keep track of coding agent tasks and review pull requests on the go with GitHub Mobile for iOS <https://github.com/notifications/mobile/ios/ANDHUITRX4IT7WFBXBEKTML5JYP7JA5CNFSNUABFM5UWIORPF5TWS5BNNB2WEL2JONZXKZKDN5WW2ZLOOQXTKMRYGYYDAMRVGI42M4TFMFZW63VHNVSW45DJN5XKKZLWMVXHJKTGN5XXIZLSL5UW64Y> and Android <https://github.com/notifications/mobile/android/ANDHUIQQ4DCGESRKSWLC2UD5JYP7JA5CNFSNUABFM5UWIORPF5TWS5BNNB2WEL2JONZXKZKDN5WW2ZLOOQXTKMRYGYYDAMRVGI42M4TFMFZW63VHNVSW45DJN5XKKZLWMVXHJLTGN5XXIZLSL5QW4ZDSN5UWI>. Download it today! You are receiving this because you were mentioned.Message ID: ***@***.***>

Navigation Menu

Repository navigation

ensemble manager: layout, layout_type, distribution_type #1166

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

	! Distribution type controls pe layout of storage; Default is 1. 1 is only one implemented.
	if(.not. present(distribution_type_in)) then
	ens_handle%distribution_type = 1
	else
	! Check for error: only type 1 implemented for now
	if(distribution_type_in /= 1) call error_handler(E_ERR, 'init_ensemble_manager', &
	'only distribution_type 1 is implemented', source)
	ens_handle%distribution_type = distribution_type_in
	endif

ensemble manager: layout, layout_type, distribution_type #1166

Description

Activity

jlaucar commented on Aug 13, 2026

hkershaw-brown commented on Aug 13, 2026

hkershaw-brown commented on Aug 13, 2026

jlaucar commented on Aug 13, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions