Neuroscout: a web-based platform for flexible re-analysis of naturalistic fMRI datasets

Presented During:

Poster No:


Submission Type:

Abstract Submission 


Alejandro de la Vega1, Ross Blair2, Christopher Markiewicz2, Roberta Rocca1, Michael Hanke3, Tal Yarkoni1


1University of Texas at Austin, Austin, TX, 2Stanford University, Stanford, CA, 3Institute of Neuroscience and Medicine (INM-7: Brain and Behaviour), Research Center Jülich, Jülich, Germany

First Author:

Alejandro de la Vega  
University of Texas at Austin
Austin, TX


Ross Blair  
Stanford University
Stanford, CA
Christopher Markiewicz  
Stanford University
Stanford, CA
Roberta Rocca  
University of Texas at Austin
Austin, TX
Michael Hanke  
Institute of Neuroscience and Medicine (INM-7: Brain and Behaviour), Research Center Jülich
Jülich, Germany
Tal Yarkoni, PhD  
University of Texas at Austin
Austin, TX


fMRI studies using complex naturalistic stimulation, such as movies or audio narratives, hold great promise to reveal the neural activity underlying dynamic perception. However, this potential is limited by the resource-intensive nature of fMRI analysis, and the difficulty of annotating events in rich, multi-modal stimuli. Consequently, only a small fraction of viable hypotheses are ever tested, even as the number of public datasets increases. Here we present Neuroscout, a platform that harnesses automated feature extraction tools and a web-based analysis builder to enable researchers to flexibly define and test novel statistical models in public fMRI datasets.


The Neuroscout server currently indexes a nine publicly available naturalistic fMRI datasets, which is regularly expanded through a semi-automated pipeline. This pipeline leverages state-of-the-art feature extraction algorithms (standardized in our library, pliers; McNamara et al, 2017) to extract hundreds of neural predictors from multi-modal experimental stimuli.
Using NeuroScout's analysis builder web application (available at:, users can explore predictors and flexibly define novel fMRI analyses models (Fig 1a). Interactive quality-control reports enable users to inspect their models and iteratively refine their design. Analyses are assigned a unique ID, and a self-contained execution bundle is generated (Fig 1d).
Analysis execution is achieved with no configuration as fMRI data is automatically retrieved using DataLad (Yaroslav et al., 2018). Containerized model-fitting pipelines minimize software dependencies (Fig 1e), ensuring portability and reproducibility across execution environments-- including HPCs. Results are made available as interactive publication-like reports, and statistical images are automatically made available via the NeuroVault repository (Fig 1f).
As a proof-of-concept, we fit conventional fMRI models to four automatically extracted predictors in seven movie-watching datasets indexed by Neuroscout (Life: Nasatase et al., 2017; Raiders: Haxby et at., 2011; Merlin: Zadbood et al., 2017; Sherlock: Chen et al., 2017;
HBN: Alexander et al., 2017; Forrest: Hanke et al., 2016; SM: Baldasanno et al., 2018).
Supporting Image: OHBM_Diagram_2019.png
   ·Figure 1. Neuroscout schematic overview.


Automatically extracted predictors generally exhibited associations that converged with those obtained using conventional experimental approaches (Figure 2). For example, speech was associated with robust activity in language processing regions such as STS and IFG, while scenes detected to contain "buildings" were associated with activity in the PPA. As an example of a result consistent with the existing literature that would be difficult to manually annotate, we observed consistent activity in the FFA for the first presentation of an individual face across an entire movie stimulus. Finally, we also observed consistent results for novel phenomena that would be less likely to be studied without automation; for example, changes in shots (visual scenes) were consistently associated with increased activity in visual, parietal and retrosplenial cortices.
Supporting Image: all_fig2.png
   ·Figure 2. fMRI results using automatically extracted predictors


Neuroscout reduces the burden of reanalyzing public naturalistic fMRI data by providing a flexible web application and largely automating the analysis process. Additionally, using multimodal feature extraction tools, we allow researchers to rapidly identify stimulus features relevant to their theoretical hypotheses. The consistency of the results obtained when applying these tools to naturalistic datasets--even when the results depart from experiment-based approaches--also highlights the importance of studying and understanding perceptual mechanisms not only under highly controlled experimental conditions, but also under ecologically valid naturalistic conditions involving complex scene statistics.

Modeling and Analysis Methods:

Activation (eg. BOLD task-fMRI)

Neuroinformatics and Data Sharing:

Databasing and Data Sharing 2
Informatics Other 1


Machine Learning

1|2Indicates the priority used for review

My abstract is being submitted as a Software Demonstration.


Please indicate below if your study was a "resting state" or "task-activation” study.


Healthy subjects only or patients (note that patient studies may also involve healthy subjects):

Healthy subjects

Are you Internal Review Board (IRB) certified? Please note: Failure to have IRB, if applicable will lead to automatic rejection of abstract.


Was any human subjects research approved by the relevant Institutional Review Board or ethics panel? NOTE: Any human subjects studies without IRB approval will be automatically rejected.

Not applicable

Was any animal research approved by the relevant IACUC or other animal research panel? NOTE: Any animal studies without IACUC approval will be automatically rejected.

Not applicable

Please indicate which methods were used in your research:

Functional MRI

Which processing packages did you use for your study?

Other, Please list  -   nistats

Provide references using author date format

Nastase, S. A. (2017). Attention selectively reshapes the geometry of distributed semantic representation. Cerebral Cortex.
McNamara, Q. (2017). Developing a comprehensive framework for multimodal feature extraction. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining.
Haxby J.V. (2011). A common, high-dimensional model of the representational space in human ventral temporal cortex. Neuron.
Zadbood, A. (2017). How we transmit memories to other brains: constructing shared neural representations via communication. Cerebral Cortex, 27(10), 4988-5000.
Halchenko Y.O. (2018). datalad/datalad 0.11.0 (Version 0.11.0). Zenodo.
Chen, J. (2017) Shared memories reveal shared structure in neural activity across individuals. Nature Neuroscience, 20, 115–125
Hanke, M. (2016). A studyforrest extension, simultaneous fMRI and eye gaze recordings during prolonged natural stimulation. Scienific Data, 3, 160092.
Baldassano, C. (2018). Event Schemas during Narrative Perception. Journal of Neuroscience.
Alexander, L. (2017). An open resource for transdiagnostic research in pediatric mental health and learning disorders. Sci Data, 4, 170181