Repository navigation
Output hidden states and attention scores for each transformer layer #21
Description
Existing TransformerBlockList cannot output the hidden states and attention scores for each transformer layer. Sometimes we want to get the hiddens and attention scores to conduct analysis and feed them into the next modules.
Activity
[-]Output hidden states and attention scores for transformer[/-][+]Output hidden states and attention scores for each transformer layer[/+]Achazwl commentedon Aug 30, 2022
Support backward of middle hidden states by PR #40 .