0
$\begingroup$

I am reading a paper in which the author uses projected gradient descent to produce iterates of the form:

$$\pi_{t + 1} = \underset{\pi \in \Pi}{\arg \max}\left\{ \left \langle \pi, Q \right \rangle - \frac{1}{2 \eta} \left \lVert \pi - \pi_t \right \rVert_2^2 \right\}$$

where $\Pi$ is a set of finite-dimensional vectors. The author then claims for all $\pi' \in \Pi$ it holds that

$$\left \langle \pi' - \pi_{t + 1}, \eta Q - \pi_{t + 1} + \pi_t \right \rangle \leq 0$$

By optimality of $\pi_{t + 1}$ I can deduce

$$\left \langle \pi_{t + 1}, Q \right \rangle - \frac{1}{2 \eta} \left \lVert \pi_{t + 1} - \pi_t \right \rVert_2^2 \geq \left \langle \pi', Q \right \rangle - \frac{1}{2 \eta} \left \lVert \pi' - \pi_t \right \rVert_2^2$$

and therefore

$$\left \langle \pi' - \pi_{t + 1}, \eta Q \right \rangle \leq \frac{1}{2} \left \lVert \pi' - \pi_t \right \rVert_2^2 - \frac{1}{2} \left \lVert \pi_{t + 1} - \pi_t \right \rVert_2^2$$

but it remains for me to show

$$\frac{1}{2} \left \lVert \pi' - \pi_t \right \rVert_2^2 - \frac{1}{2} \left \lVert \pi_{t + 1} - \pi_t \right \rVert_2^2 \leq \left\langle \pi' - \pi_{t + 1}, \pi_{t + 1} - \pi_t \right\rangle$$

Any help would be appreciated!

EDIT: Note the update can also be written

$$\pi_{t + 1} = \mathtt{Proj}_{\Pi} \Big \{ \pi_t + \eta Q \Big \}$$

$\endgroup$
6
  • 1
    $\begingroup$ The inequality you’re asking for help proving is false if $\pi’ = \pi_t$ $\endgroup$ Commented Jul 7 at 1:05
  • $\begingroup$ Hm indeed it is. I don't think that's enough to disprove the author's claim, though. I am still looking to prove $$\left \langle \pi' - \pi_{t + 1}, \eta Q - \pi_{t + 1} + \pi_t \right \rangle \leq 0$$ $\endgroup$
    – msantama
    Commented Jul 7 at 2:36
  • $\begingroup$ Isn’t that statement also false if $\pi’ = \eta Q + \pi_t$, unless $\pi_{t+1} = \eta Q + \pi_t$, in which case the inequality is trivially true as it is always equal to $0$? $\endgroup$ Commented Jul 7 at 10:20
  • $\begingroup$ If we can choose $\pi' = \eta Q + \pi_t$ then $\eta Q + \pi_t \in \Pi$ and necessarily $\pi_{t + 1} = \eta Q + \pi_t$. $\endgroup$
    – msantama
    Commented Jul 7 at 17:21
  • $\begingroup$ I'm fairly convinced this must be wrong unless you have some further conditions on $\Pi$. Consider the $1D$ case for $\Pi = [-1, 1]$ with $Q = \eta = 1$ and set $\pi_0 = 0$. Then $\pi_1 = \frac{1}{2}$. Then the inequality becomes $\frac{1}{2}(\pi' - \frac{1}{2}) \leq 0$, which is false for e.g. $\pi' = 1$. $\endgroup$ Commented Jul 10 at 16:48

1 Answer 1

0
$\begingroup$

There's a further condition that $\Pi$ is convex, which is key. It is then a direct application of Theorem 2.1 here.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .