Generative adversarial networks for reconstructing natural images from brain activity
release_g6qpcbg6ubhlbgkur27qzkdtzq
by
Katja Seeliger, U. Güçlü, L. Ambrogioni, Y. Güçlütürk, M. A. J. van Gerven
2017
Abstract
We explore a method for reconstructing visual stimuli from brain activity. Using large databases of natural images we trained a deep convolutional generative adversarial network capable of generating gray scale photos, similar to stimuli presented during two functional magnetic resonance imaging experiments. Using a linear model we learned to predict the generative model's latent space from measured brain activity. The objective was to create an image similar to the presented stimulus image through the previously trained generator. Using this approach we were able to reconstruct structural and some semantic features of a proportion of the natural images sets. A behavioral test showed that subjects were capable of identifying a reconstruction of the original stimulus in 67.2% and 66.4% of the cases in a pairwise comparison for the two natural image datasets respectively. Our approach does not require end-to-end training of a large generative model on limited neuroimaging data. Rapid advances in generative modeling promise further improvements in reconstruction performance.
In application/xml+jats
format
Archived Files and Locations
application/pdf 3.4 MB
file_hxdhc5okubarvkfpyqd7lezaiy
|
web.archive.org (webarchive) www.biorxiv.org (web) |
application/pdf 7.3 MB
file_h6bxt2jydfachljqjtgycfem3y
|
web.archive.org (webarchive) www.biorxiv.org (web) |
application/pdf 1.8 MB
file_yoi5ldprordxxj2pumcf3fdwda
|
repository.ubn.ru.nl (web) web.archive.org (webarchive) |
post
Stage
unknown
Date 2017-12-08
10.1101/226688
access all versions, variants, and formats of this works (eg, pre-prints)
Crossref Metadata (via API)
Worldcat
wikidata.org
CORE.ac.uk
Semantic Scholar
Google Scholar