1
$\begingroup$

Let $\{\phi_k\}_{k=1}^K$ be a family of functions mapping from an interval $[a, b]$ to $[-1, 1]$. That is, $\phi_k \colon[ a,b] \to [-1, 1]$ are nondecreasing maps on some finite interval $[a, b] \subset \mathbb{R}$.

We can define the Rademacher complexity of these functions as $$ R(\phi_1, \dots, \phi_K) := \mathbb{E}_{\sigma_1, \dots, \sigma_K} \bigg[\sup_{x \in [a, b]} \Big|\sum_{k=1}^K \sigma_k~\phi_k(x) \Big|\bigg]. $$

Quesiton: What is a tight bound on $R(\phi_1, \dots, \phi_K)$?

$\endgroup$
0

1 Answer 1

1
$\begingroup$

The collection of dunctions maximizing $R$ is $$ \phi_i(x)=\begin{cases} -1,& x<i/n;\\ 1.& x\geq i/n \end{cases} $$ (on the interval $[0,1]$).

To see this, observe first that each $\phi_i$ can be replaced with a piecewise constant function. Next, if, say, $\phi_1$ attains values $a_j$ on $[t_j,t_{j+1})$ with $0=t_1<t_2<\dots<t_N(>1)$, then $R(\phi_1,\dots,\phi_n)$ is a convex function of the $a_i$ (it is a sum of maxima of absolute values of linear functions), so its maximum on the simplex $-1\leq a_1\leq \dots\leq a_N\leq 1$ is attained at one of the vertixes, where $a_1=\dots=a_i=-1$ and $a_{i+1}=\dots=a_N=1$. So in fact each $\phi_i$ is a function of the form $\phi_i=-1+2\cdot \mathbb I_{x\geq x_i}$ for some $x_i$. Without loss of generality, we may assume that $x_1<\dots<x_n$, arriving at the above example.

It remains to compute the value of $R(\phi_1,\dots,\phi_n)$. I am not sure, however, that this can be found in a closed form...

$\endgroup$
0

Not the answer you're looking for? Browse other questions tagged or ask your own question.