Article Dans Une Revue Transactions on Machine Learning Research Journal Année : 2024

Deep Neural Networks and Brain Alignment: Brain Encoding and Decoding (Survey)

Résumé

Can artificial intelligence unlock the secrets of the human brain? How do the inner mechanisms of deep learning models relate to our neural circuits? Is it possible to enhance AI by tapping into the power of brain recordings? These captivating questions lie at the heart of an emerging field at the intersection of neuroscience and artificial intelligence. Our survey dives into this exciting domain, focusing on human brain recording studies and cutting-edge cognitive neuroscience datasets that capture brain activity during natural language processing, visual perception, and auditory experiences. We explore two fundamental approaches: encoding models, which attempt to generate brain activity patterns from sensory inputs; and decoding models, which aim to reconstruct our thoughts and perceptions from neural signals. These techniques not only promise breakthroughs in neurological diagnostics and brain-computer interfaces but also offer a window into the very nature of cognition. In this survey, we first discuss popular representations of language, vision, and speech stimuli, and present a summary of neuroscience datasets. We then review how the recent advances in deep learning transformed this field, by investigating the popular deep learning based encoding and decoding architectures, noting their benefits and limitations across different 1 Published in Transactions on Machine Learning Research (12/2024) sensory modalities. From text to images, speech to videos, we investigate how these models capture the brain's response to our complex, multimodal world. While our primary focus is on human studies, we also highlight the crucial role of animal models in advancing our understanding of neural mechanisms. Throughout, we mention the ethical implications of these powerful technologies, addressing concerns about privacy and cognitive liberty. We conclude with a summary and discussion of future trends in this rapidly evolving field. Given the large amount of recently published work in the computational cognitive neuroscience (CCN) community, we believe that this survey provides an invaluable entry point for deep neural network (DNN) researchers looking to diversify into CCN research, inviting them to join in unraveling the ultimate puzzle: the human brain.
Fichier principal
Vignette du fichier
Oota2024_TMLR_review__HAL-v2.pdf (6.91 Mo) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04906035 , version 1 (22-01-2025)

Licence

Identifiants

  • HAL Id : hal-04906035 , version 1

Citer

Subba Reddy Oota, Zijiao Chen, Manish Gupta, Bapi Raju Surampudi, Gaël Jobard, et al.. Deep Neural Networks and Brain Alignment: Brain Encoding and Decoding (Survey). Transactions on Machine Learning Research Journal, 2024. ⟨hal-04906035⟩
0 Consultations
0 Téléchargements

Partager

More