Description
The paper introduces Deep Delta Learning (DDL), a new neural network architecture that extends traditional deep residual connections by replacing the fixed identity shortcut with a learnable, data-dependent geometric transformation called the Delta Operator. By parameterizing this operator with a direction vector and a gating scalar, the model can smoothly interpolate between identity mapping, projection, and reflection behaviors, giving the network fine-grained control over how information is erased and written at each layer. This enables it to represent more complex, non-monotonic dynamics than standard residual networks while retaining stable training, effectively broadening the expressive power of deep models without losing the benefits of residual learning.
