Traditional efforts to decode mental content have primarily focused on identifying discrete words or concepts associated with objects or actions in a person’s mind. However, the new ‘thought translation’ method using artificial intelligence goes beyond that by producing coherent sentences that accurately describe entire scenes or dynamic events. Using functional magnetic resonance imaging (FMRI), researchers scanned the brain activity of participants as they watched thousands of short video clips showing different objects, actions, and interactions. This brain data was then applied with semantic features derived from a deep language model in an iterative process involving ‘masked language models’ (MLMs), such as the ‘ROBERTA – Large’ model, as reported by the medical site. This technique also showed promising results in reading memories, where participants remembering video clips during FMRI scanning exhibited brain patterns that the system successfully translated into descriptive text. While individual accuracy varied, some reached nearly 40% in correctly identifying recalled video clips among many possibilities, highlighting the method’s capability to decode internal visual memories. Therefore, researchers emphasize the need for strict consent protocols and caution that this technology currently decodes semantic interpretations rather than private thoughts or raw mental content. The ‘thought translation’ technique represents a transformative step in neuroscience’s quest to uncover how the brain encodes and processes complex experiences. By converting brain signals into meaningful and organized text, this innovation could revolutionize communication among non-verbal individuals and shed light on the neural basis of human consciousness.
