Questions tagged [regularization]
Regularization, in mathematics and statistics and particularly in the fields of machine learning and inverse problems, refers to a process of introducing additional information in order to solve an ill-posed problem or to prevent overfitting. (Def: http://en.wikipedia.org/wiki/Regularization_(mathematics))
369
questions
-2
votes
0
answers
49
views
An example of infinite divergent series giving rational fraction of Pi. [closed]
Can an example of divergent integer sequence along some regularization method be found where the generalized sum is $c π^k $, with c, k rational (or rational complex number of the form p + qi, where p ...
- 15
1
vote
0
answers
15
views
Divergent Tail Sums of Approximations of Non-trace Class Compact Operators
I'm working on approximations of compact operators that are not trace class, and I'm looking for ways to provide meaningful approximation error estimates for truncated eigenfunction expansions. I ...
user1328882
0
votes
1
answer
32
views
convex optimization with L1 and L2 regularization
Can we solve the problem with the form $$min_{x}||Ax - b||^2_2+\lambda||x||_1+\mu||x^2-y||^2_2$$ where y is a given vector.
This problem is a combination of least-squares regression with L1 and L2 ...
1
vote
1
answer
42
views
Solve the Soft SVM Dual Problem with L1 Regularization
I'm considering a support vector regression model with a prediction
$$ \hat{y}(\mathbf{x}_\star)=\boldsymbol{\theta}^{\top} \boldsymbol{\phi}(\mathbf{x}_\star)$$
where $\boldsymbol{\theta}$ are the ...
- 239
1
vote
0
answers
49
views
Make a non-smooth function smooth
I am dealing with a piecewise affine function $f$ defined as follows: $f(x)=0$ if $x<1$, $f(x)=1-x$ if $x\in [1,2]$ and $f(x)=-1$ otherwise.
I want to make it smooth. I looked at sigmoid functions ...
- 109
2
votes
0
answers
99
views
Generalization error bound for Empirical Risk minimizer on Gaussian noisy data
I have datapoints that are sampled from a distribution $\mathbb{D}$. Each datapoint is a tuple $(t,y)$ of a time $t \in [0,T]$ that is sampled uniformly and a value $y(t) \sim u(t) + \mathcal{N}(0, \...
- 559
0
votes
0
answers
30
views
Is the function $\lVert A(A^TA+\lambda I)^{-1}A^Ty \rVert_2$ decreasing in $\lambda$?
Let $A$ be some matrix over $\mathbb R$.
Is the function $f(\lambda)=\lVert A(A^TA+\lambda I)^{-1}A^Ty \rVert_2$ decreasing for $\lambda > 0$? Here $y$ is an arbitrary real vector of the correct ...
- 929
0
votes
1
answer
44
views
Regularization of Interest Rate Inverse Problem failing
I'm going through this Inverse Problems textbook: https://www.rose-hulman.edu/~bryan/invprobs/inversefin1.pdf . In one exercise, I have to perform a regularization on an inverse problem, but I get ...
- 3
0
votes
1
answer
32
views
Can one use asymptotic behaviour to deduce the induced distribution from a function?
As an example, imagine I have $e^{i\sqrt{x^2+y^2}\cdot k}$ defined for $x$ on the reals. Then, in a distributional sense, I want to understand
$$\int e^{i\sqrt{x^2+y^2}\cdot k} dx$$
My naive approach ...
- 165
0
votes
0
answers
14
views
How this variational derivative is calculated?
In this paper https://arxiv.org/pdf/1907.09605.pdf \
let $\Omega \subset \mathbb{R}^n$ with $n \geq 1$ be a bounded Lipschitz domain with boundary $\partial \Omega$, $f: \Omega \rightarrow \mathbb{R}$ ...
- 1
1
vote
1
answer
55
views
Making a symmetric matrix positive (semi-)definite by adding a diagonal matrix
Say I have a matrix $A \in R^{p \times p}$ which is symmetric and with non-negative diagonal entries (i.e. $a_{ii} \geq 0 \forall i\in \{1, \ldots, p\}$). However, $A$ is not positive (semi-)definite.
...
- 25
1
vote
0
answers
60
views
Strong & weak solution of $xy' + y = 0$ [Friedlander ex. 2.3]
The ODE $xy' + y = 0$ has no strong solution over $\mathbb R$ but has solution $y(x) = \begin{cases} c_1/x & x<0 \\ c_2/x & x>0 \end{cases}$ over $\mathbb R^*$, which may equivalently be ...
- 677
0
votes
0
answers
42
views
Regularisation for a differential equation
When stuying a 1D differential equation, I tried to solve the problem by finding the Green's function and then solving with the inverse Fourier transform.
To make the problem well-behaved, I used a ...
- 1,238
1
vote
0
answers
50
views
Is there a Closed-Form Solution for L2 Regularization Raised to a Power?
Recently, I came across a modified L2 regularization term as stated in the equation below, where $\gamma$ is a positive number.
$$
\lambda'(w^Tw)^\gamma
$$
I'm curious if a closed-form solution ...
- 11
1
vote
1
answer
45
views
Is there any method that can optimize the problem whose regularizer is kurtosis term?
I recently worked on an optimization problem, whose regularizer $g(x)$ is kurtosis. The overall optimization formula is as follows.
$$\begin{align} \arg \min_x \frac12 \Vert Ax-b\Vert_2^2 + \lambda g(...
- 35