Vertical Machine Unlearning: Selectively Removing Sensitive Information From Latent Feature Space
release_a7ilytvcizdjrav5bigyc5g37u
by
Tao Guo, Song Guo, Jiewei Zhang, Wenchao Xu, Junxiao Wang
2022
Abstract
Recently, the enactment of privacy regulations has promoted the rise of
machine unlearning paradigm. Most existing studies mainly focus on removing
unwanted data samples from a learnt model. Yet we argue that they remove
overmuch information of data samples from latent feature space, which is far
beyond the sensitive feature scope that genuinely needs to be unlearned. In
this paper, we investigate a vertical unlearning mode, aiming at removing only
sensitive information from latent feature space. First, we introduce intuitive
and formal definitions for this unlearning and show its orthogonal relationship
with existing horizontal unlearning. Secondly, given the fact of lacking
general solutions to vertical unlearning, we introduce a ground-breaking
solution based on representation detachment, where the task-related information
is encouraged to retain while the sensitive information is progressively
forgotten. Thirdly, observing that some computation results during
representation detachment are hard to obtain in practice, we propose an
approximation with an upper bound to estimate it, with rigorous theoretical
analysis. We validate our method by spanning several datasets and models with
prevailing performance. We envision this work as a necessity for future machine
unlearning system and an essential component of the latest privacy-related
legislation.
In text/plain
format
Archived Content
There are no accessible files associated with this release. You could check other releases for this work for an accessible version.
Know of a fulltext copy of on the public web? Submit a URL and we will archive it
2202.13295v1
access all versions, variants, and formats of this works (eg, pre-prints)