Generative adversarial networks for reconstructing natural images from brain activity release_g6qpcbg6ubhlbgkur27qzkdtzq

by Katja Seeliger, U. Güçlü, L. Ambrogioni, Y. Güçlütürk, M. A. J. van Gerven

Released as a post by Cold Spring Harbor Laboratory.

2017  

Abstract

We explore a method for reconstructing visual stimuli from brain activity. Using large databases of natural images we trained a deep convolutional generative adversarial network capable of generating gray scale photos, similar to stimuli presented during two functional magnetic resonance imaging experiments. Using a linear model we learned to predict the generative model's latent space from measured brain activity. The objective was to create an image similar to the presented stimulus image through the previously trained generator. Using this approach we were able to reconstruct structural and some semantic features of a proportion of the natural images sets. A behavioral test showed that subjects were capable of identifying a reconstruction of the original stimulus in 67.2% and 66.4% of the cases in a pairwise comparison for the two natural image datasets respectively. Our approach does not require end-to-end training of a large generative model on limited neuroimaging data. Rapid advances in generative modeling promise further improvements in reconstruction performance.
In application/xml+jats format

Archived Files and Locations

application/pdf  3.4 MB
file_hxdhc5okubarvkfpyqd7lezaiy
web.archive.org (webarchive)
www.biorxiv.org (web)
application/pdf  7.3 MB
file_h6bxt2jydfachljqjtgycfem3y
web.archive.org (webarchive)
www.biorxiv.org (web)
application/pdf  1.8 MB
file_yoi5ldprordxxj2pumcf3fdwda
repository.ubn.ru.nl (web)
web.archive.org (webarchive)
Read Archived PDF
Preserved and Accessible
Type  post
Stage   unknown
Date   2017-12-08
Work Entity
access all versions, variants, and formats of this works (eg, pre-prints)
Catalog Record
Revision: e062d516-45aa-4cf4-8c02-fcf34ce9e886
API URL: JSON