<div class="csl-bib-body">
<div class="csl-entry">Zihan, W. S., Delrocq, A., Gerstner, W., & Bellec, G. (2026, July 7). <i>Can local learning match self-supervised backpropagation?</i> [Poster Presentation]. Forty-Third International Conference on Machine Learning (ICML 2026), Seoul, Korea (the Republic of). http://hdl.handle.net/20.500.12708/232177</div>
</div>
-
dc.identifier.uri
http://hdl.handle.net/20.500.12708/232177
-
dc.description.abstract
While end-to-end self-supervised learning with backpropagation (global BP-SSL) has become central for training modern AI systems, theories of local self-supervised learning (local-SSL) have struggled to build functional representations in deep neural networks. To establish a link between global and local rules, we first develop a theory for deep linear networks: We identify conditions for local-SSL algorithms (like Forward-forward or CLAPP) to implement exactly the same weight update as a global BP-SSL. Starting from the theoretical insights, we then develop novel variants of local-SSL algorithms to approximate global BP-SSL in deep non-linear convolutional neural networks. Variants that improve the similarity between updates of local-SSL with those of global BP-SSL also show better performance on image datasets (CIFAR-10, STL-10, Tiny ImageNet, and ImageNet). The best local-SSL rule with the CLAPP loss function matches the performance of a comparable global BP-SSL with InfoNCE or CPC-like loss functions, and improves upon state-of-the-art for layer-wise SSL on these benchmarks.
en
dc.language.iso
en
-
dc.subject
Machine Learning
en
dc.subject
AI
en
dc.subject
Computational Neuroscience
en
dc.title
Can local learning match self-supervised backpropagation?