Approximate resilience, monotonicity, and the complexity of agnostic learning release_mj3c5ji3hffizfsdgzur5pz74a

by Dana Dachman-Soled and Vitaly Feldman and Li-Yang Tan and Andrew Wan and Karl Wimmer

Released as a article .

2014  

Abstract

A function f is d-resilient if all its Fourier coefficients of degree at most d are zero, i.e., f is uncorrelated with all low-degree parities. We study the notion of approximate resilience of Boolean functions, where we say that f is α-approximately d-resilient if f is α-close to a [-1,1]-valued d-resilient function in ℓ_1 distance. We show that approximate resilience essentially characterizes the complexity of agnostic learning of a concept class C over the uniform distribution. Roughly speaking, if all functions in a class C are far from being d-resilient then C can be learned agnostically in time n^O(d) and conversely, if C contains a function close to being d-resilient then agnostic learning of C in the statistical query (SQ) framework of Kearns has complexity of at least n^Ω(d). This characterization is based on the duality between ℓ_1 approximation by degree-d polynomials and approximate d-resilience that we establish. In particular, it implies that ℓ_1 approximation by low-degree polynomials, known to be sufficient for agnostic learning over product distributions, is in fact necessary. Focusing on monotone Boolean functions, we exhibit the existence of near-optimal α-approximately Ω(α√(n))-resilient monotone functions for all α>0. Prior to our work, it was conceivable even that every monotone function is Ω(1)-far from any 1-resilient function. Furthermore, we construct simple, explicit monotone functions based on Tribes and CycleRun that are close to highly resilient functions. Our constructions are based on a fairly general resilience analysis and amplification. These structural results, together with the characterization, imply nearly optimal lower bounds for agnostic learning of monotone juntas.
In text/plain format

Archived Files and Locations

application/pdf  324.2 kB
file_gbk4xja6b5alpkev54z64rguyu
arxiv.org (repository)
web.archive.org (webarchive)
Read Archived PDF
Preserved and Accessible
Type  article
Stage   submitted
Date   2014-05-21
Version   v1
Language   en ?
arXiv  1405.5268v1
Work Entity
access all versions, variants, and formats of this works (eg, pre-prints)
Catalog Record
Revision: 38c8f40c-e87d-46a9-90c4-1576d062b622
API URL: JSON