ChoppedTransferCompound no_buffer drops small transfers despite the TT-v1 equivalence claim #807

Summary

In AIHWKit v1.1.0, the ChoppedTransferCompound.no_buffer docstring says this setting is identical to Tiki-taka v1 when gamma=0. With two devices, zero chopper probabilities, sequential one-hot transfer, one read per event, and otherwise matching update and transfer settings, I expected the same fast-to-slow transfer as TransferCompound. A controlled 1×1 test instead shows that the first slow-device update is lost.

The discrepancy is an extra deterministic quantization before the slow device's pulsed updater. In the CUDA transfer kernel, no_buffer initializes omega to zero for each transfer, but the kernel still applies trunc(omega) and discards any remainder. The resulting integer n_steps is then passed to writeMatrix. By comparison, TransferCompound::readAndUpdate passes the analog read directly to the pulsed updater. The CPU chopped path uses the same truncation, so this is not limited to the CUDA implementation.

This may be intentional behavior for a stateless version of BufferedTransferCompound: the class documentation explicitly relates the zero-chopper configuration to BufferedTransferCompound. If so, the unqualified TT-v1 equivalence statement in no_buffer is misleading for finite device granularity and the TransferCompound implementation of TT-v1.

Minimal reproduction

This CPU example uses AIHWKit v1.1.0. It copies the shared TT-v1 controls into the chopped compound, sets gamma=0 and both chopper probabilities to zero, and uses a perfect transfer read and NONE_WITH_DEVICE write to isolate transfer arithmetic. The outer update receives zero input and error; both fast weights therefore remain at 0.04. The continuous write setting is diagnostic only: it avoids conflating the extra buffer truncation with the slow device's stochastic pulse sampling.

from copy import deepcopy
import torch
from aihwkit.simulator.configs import build_config
from aihwkit.simulator.configs.devices import SoftBoundsReferenceDevice
from aihwkit.simulator.parameters.enums import PulseType
from aihwkit.simulator.tiles.analog import AnalogTile

def device():
    return SoftBoundsReferenceDevice(dw_min=0.001, w_min=-0.8, w_max=0.8)

legacy = build_config("ttv1", device=device(), construction_seed=123)
chopped = build_config("ttv2", device=device(), construction_seed=123)
chopped.forward = deepcopy(legacy.forward)
chopped.backward = deepcopy(legacy.backward)
chopped.update = deepcopy(legacy.update)
for key in (
    "gamma", "gamma_vec", "transfer_every", "transfer_every_vec",
    "no_self_transfer", "units_in_mbatch", "n_reads_per_transfer",
    "transfer_columns", "with_reset_prob", "random_selection",
    "fast_lr", "transfer_lr", "transfer_lr_vec",
    "scale_transfer_lr", "scale_fast_lr", "transfer_forward",
    "transfer_update",
):
    setattr(chopped.device, key, deepcopy(getattr(legacy.device, key)))
chopped.device.no_buffer = True
chopped.device.in_chop_prob = 0.0
chopped.device.out_chop_prob = 0.0
chopped.device.auto_scale = False
chopped.device.auto_granularity = 0.0
chopped.device.buffer_granularity = 1.0
chopped.device.step = 1.0

def trace(config):
    for subdevice in config.device.unit_cell_devices:
        subdevice.dw_min_dtod = 0.0
        subdevice.dw_min_std = 0.0
        subdevice.w_max_dtod = 0.0
        subdevice.w_min_dtod = 0.0
    config.device.transfer_forward.is_perfect = True
    config.device.transfer_update.pulse_type = PulseType.NONE_WITH_DEVICE
    tile = AnalogTile(1, 1, config, bias=False)
    tile.set_weights(torch.zeros((1, 1)))
    hidden = tile.get_hidden_parameters()
    hidden["hidden_weights_0"].fill_(0.04)
    tile.set_hidden_parameters(hidden)
    tile.set_learning_rate(0.02)
    states = []
    for _ in range(2):
        tile.update(torch.zeros((1, 1)), torch.zeros((1, 1)))
        h = tile.get_hidden_parameters()
        states.append((float(h["hidden_weights_0"].item()),
                       float(h["hidden_weights_1"].item())))
    return states

print("TransferCompound", trace(legacy))
print("ChoppedTransferCompound(no_buffer=True)", trace(chopped))

Observed output, rounded as (fast, slow):

TransferCompound                  [(0.04, 0.0008), (0.04, 0.0016)]
ChoppedTransferCompound/no_buffer [(0.04, 0),      (0.04, 0)]

For the first transfer, the chopped kernel's dimensionless buffer value is 0.04 × 0.02 / (1 × 0.001) = 0.8. Since trunc(0.8) = 0, the slow device receives no update, while TransferCompound passes the read through and produces a 0.0008 slow update in this deterministic test. Because no_buffer discards the remainder, the second identical transfer is also zero. The dead-zone threshold for |A| is 0.05 at learning rate 0.02 and grows to 0.5 and 5 when that rate is reduced to 0.002 and 0.0002.

Training impact

In a separate 80-epoch CIFAR-10 CNN run with seed 0 and otherwise matched TT-v1 settings, TransferCompound reached 48.35% best accuracy and 46.414% mean accuracy over the last ten epochs. The original ChoppedTransferCompound(no_buffer=True) reached 43.27% and 42.322%. An isolated direct-transfer CUDA change that bypasses only the extra buffer-step conversion reached 47.04% and 45.949% while retaining the original STOCHASTIC_COMPRESSED slow-device update. These are single-seed convergence observations, not evidence of identical stochastic sample paths.

Possible resolutions

1. Clarify the contract if the current behavior is intentional. Describe no_buffer=True as disabling persistent digital-buffer state while retaining integer buffer-step quantization. Qualify or remove the unconditional TT-v1 equivalence claim, and explain that equivalence to the TransferCompound implementation requires additional conditions or a different transfer path.
2. Provide an explicit direct TT-v1 transfer mode if equivalence is intended. For the supported two-device, zero-chopper, gamma=0 case, pass the signed fast-device read directly to the existing slow-device pulsed updater, with the same transfer learning-rate scaling and pulse settings as TransferCompound. Implement corresponding CPU and CUDA paths. A new opt-in mode would preserve behavior for users who rely on today's stateless integer-step transfer; changing no_buffer unconditionally may be a compatibility break.
3. Add step-level regression tests. Under identical initial hidden weights and perfect transfer I/O, compare visible, fast, and slow weights after steps 1, 2, and several later steps at learning rates 0.02, 0.002, and 0.0002. Test the direct path with the normal pulsed updater as well. Exact random trajectories across different backends need not match for the deterministic transfer check.

May it be clarified whether no_buffer=True is intended to mean stateless buffered transfer or direct TransferCompound-style TT-v1 transfer?