Questions tagged [convex]
A convex set includes all points lying between any two points from the set. A convex function on such a set is a function lying below any straight line connecting two points from its graph. Convex optimization is concerned with searching for the minimum of such a function.
168
questions
2
votes
1
answer
31
views
Do convergence rates for (convex) gradient descent apply when domain is (convex) subset of reals?
I have a convex multi-variate optimization problem where each variable lies on the domain $[x, \infty)$ for some positive number $x$. I know the problem has a unique finite solution in the domain, ...
- 123
1
vote
0
answers
24
views
Convex predictive mean of Gaussian Process
In Gaussian process (GP) regression, predictive mean is
$$ K(X^*,X)[K(X,X)+\sigma^2I]^{-1} \textbf{y}$$
Is there a method to ensure that the predictive mean is convex with respect to the test input $X^...
- 11
2
votes
1
answer
36
views
Expectation under convex order by multiplying
I am trying to understand if the following statement is true, or the conditions under it is satisfied. Let $M,N$ and $X>0$ be random variables. If the following inequality holds for any concave non-...
- 157
2
votes
1
answer
48
views
Expectation under convex order
I am trying to understand if the following statement is true. Let $M,N$ and $X$ be random variables. If the following inequality holds for any concave non-decreasing function $u$
\begin{equation}
\...
- 157
0
votes
0
answers
41
views
Understanding GAN Proof
I was reading the original GAN paper, and in the proof of Proposition 2, it is states that $U(p_g, D)$ is convex in $p_g$. I'm not sure how this is implied to be convex. This comment said that it was ...
0
votes
0
answers
20
views
Intercept term of logistic regression in ADMM algorithm
On page 66, the authors of article of ADMM says that the algorithm can be modified to obtain the intercept term easily in the sparse logistic regression model. Can someone explain this easy ...
- 313
2
votes
0
answers
73
views
Do discontinuous functions have subgradients also?
Typically, the subgradient is defined for convex functions. And convex functions are continuous.
However, DeepMind's VQ-VAE paper defines a model with a discontinuous vector quantization (VQ) layer, ...
- 1,337
3
votes
1
answer
179
views
Scenario where minimizing 0-1 loss is different than minimizing hinge loss
Suppose we're using linear predictors. I'm trying to conceptually understand how minimizing hinge loss and 0-1 loss aren't necessarily the same. For instance I was told that one can choose a set of ...
- 121
1
vote
0
answers
30
views
Convexitiy of multi-class hinge loss
The empirical risk of a multi-class hinge-loss is given by
$$L(\Theta,(x,y) = \max_{j \neq y} \Big[1+ \sum_{i=1}^{d} x_i(\Theta_{ij} - \Theta_{iy}) \Big]_{+} $$
where $x \in \mathbb{R}^{d}$ is a ...
- 265
1
vote
1
answer
77
views
Perturbation for more stable convex optimization
I am thinking of adding some perturbation to my convex optimization problem. The idea is straight forward like below chart. Supposed you are solving $\text{argmax} f(x) $, we want to find an $x$ that'...
- 258
0
votes
0
answers
10
views
how to solve for wasserstein duality easily in a special case when 2-Wasserstein inequality constraint is binding
I was going through this nice paper ” A Simple and General Duality Proof for
Wasserstein Distributionally Robust Optimization”, and one quick qu on applying Theorem 1 to my poject:
What if in my ...
- 21
0
votes
0
answers
34
views
Relationship between Ratio of expectation squared vs ratio of squared expection
I have these pair of numbers
$ (a, b) = (\frac{4}{9}, \frac{1}{9}) $ and $(c, d) = (\frac{1}{2}, \frac{1}{6}) $.
Note that - (a, b) are pair of numbers which represent $((E(e_1))^2, (E(e_2))^2) $ and (...
- 143
3
votes
1
answer
320
views
Is there an exponential family such that its natural parameter mapping is non-invertible or has non-convex range?
On the Wikipedia article for exponential families the density of a distribution on a measure space $(X, \xi)$ from an exponential family is written as $$f_{\theta} \colon X \to \mathbb{R}_{\ge 0}, \...
- 137
1
vote
0
answers
54
views
Find the minimum of a concave function [closed]
I have proved that my function with two variable is concave. I am looking for the minimum of the function. Since the function is continuous over a convex set the minimum should occur on the border of ...
- 11
0
votes
0
answers
149
views
Do contour plots over first two principal components reveal local convexity/concavity?
When I plot a contour plot of a variable over two principal components I can see what appears to be hills and valleys. But I also know I am only looking at the contours over a projection.
Here is a ...
- 9,401