You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
[FeatureRequest]bmt.OpTransformerBlockList **DO NOT** support multiple return values of transformer block's forward propogation #91
1. Currently bmt.OpTransformerBlockList can only handle the hidden states returned by transformer block.
Recent released flash_atten implemented transformer block returns hidden_states as well as residual in order to fuse Dropout -> Add -> LN. Additionally, the above two will be passed to the next block as input;
Activity
eggiter commentedon May 5, 2023
Close this issue since feature was supported by #92