[Documentation] Explain how to access the executorlib_worker_id for debugging #749

Activity

1. jan-janssen commented on Aug 7, 2025

   

   jan-janssen

   on Aug 7, 2025

   MemberAuthor

   More actions

   Example from @mgt16-LANL :

   import os
   from concurrent.futures import as_completed
   from executorlib import SingleNodeExecutor
    
   def gpu_init_simple(executorlib_worker_id):
       import os
       cuda_inds = os.environ.get('CUDA_VISIBLE_DEVICES','1').split(',')
       sel_ind = int(executorlib_worker_id) % len(cuda_inds)
       cuda_ind = cuda_inds[sel_ind]
       print('Device is cuda init:' + cuda_ind)
       print('executorlib worker_id init: ', executorlib_worker_id)
       return {}
    
   def run_parallel(ind, executorlib_worker_id):
       print('Ind: ', ind, 'executorlib worker_id: ', executorlib_worker_id)
    
   with SingleNodeExecutor(max_workers=4, init_function=gpu_init_simple,
                           block_allocation=True) as exe:
       futs = []
       print('Starting')
       for ind in range(8):
           fut = exe.submit(run_parallel, ind)
           futs.append(fut)
       print('Submitted')
       count = 0
       for done in as_completed(futs):
           print('Done: ',count)
           count += 1

   👍React with 👍1Reacted by Michael G. Taylor
2. jan-janssen

   added

   documentationImprovements or additions to documentationImprovements or additions to documentation

   on Sep 9, 2025
3. jan-janssen commented on Feb 3, 2026

   

   jan-janssen

   on Feb 3, 2026

   MemberAuthor

   More actions

   This example nicely highlights how to oversubscribe GPUs, this should be mentioned in the FAQ.