Questions tagged [neural-networks]
Questions related to mathematical aspects of neural networks including approximation properties, optimization, expressive power, algorithmic aspects etc.
13
questions
3
votes
0
answers
201
views
The curse of dimensionality of the Kolmogorov–Arnold neural network
The Kolmogorov–Arnold neural networks (KAN), Ziming Liu et al., KAN: Kolmogorov–Arnold Networks is inspired by the Kolmogorov–Arnold representation theorem (KA theorem). Though it is not proved in the ...
- 2,229
4
votes
1
answer
357
views
Difference between deep neural networks and expectation maximization algorithm
Having had a short encounter with deep neural networks, it seems to boil down to the task of determining the values of a vast amount of parameters.
The expectation maximization algorithm, of which I ...
- 12.8k
1
vote
0
answers
79
views
Approximation of continuous function by multilayer Relu neural network
For continuous/holder function $f$ defined on a compact set K， a fix $L$ and $m_1,m_2,\dots,m_L$, can we find a multilayer Relu fully connected network g with depth $L$ and each $i$-th layer has width ...
- 781
2
votes
0
answers
81
views
Neural network quantization and arithmetic on affine functions
I'm trying to understand the basics of quantization in Neural networks. Quantization tries to convert a neural network that uses floating point arithmetic to one that uses a lower precision integer ...
- 121
17
votes
3
answers
938
views
Neural networks over gadgets other than $\mathbb{R}$
Recently, I learned that neural networks (NN) can be defined over fields other than $\mathbb{R}$: for example, Khrennikov and Tirozzi wrote a paper in 1999 (!) on $p$-adic neural networks, or neural ...
- 1,056
4
votes
1
answer
283
views
Using a poset or directed graph as input for a neural network
I'm not sure if this is the right community to post this in but I would appreciate any help. As the title states, I'm trying to train a neural network using some unconventional input. I'm wondering if ...
11
votes
1
answer
798
views
Abstract mathematical concepts/tools appeared in machine learning research
I am interested in knowing about abstract mathematical concepts, tools or methods that have come up in theoretical machine learning. By "abstract" I mean something that is not immediately related to ...
10
votes
1
answer
413
views
How sensitive are Neural Networks to weight change?
Let's consider the space of feedforward neural networks with a given structure: $L$ layers, $m$ neurons per layer, ReLu activation, input dimension $d$, output dimension $k$.
Which means I'm ...
- 889
7
votes
1
answer
913
views
Universal approximation theorem for whole $\mathbb{R}^d$
The well-known universal approximation theorem states that neural network with one hidden layer can approximate any continuous function on every compact subset of $\mathbb{R}^d$.
My question is ...
- 361
29
votes
1
answer
4k
views
Is there any paper which summarizes the mathematical foundation of deep learning?
Is there any paper which summarizes the mathematical foundation of deep learning?
Now, I am studying about the mathematical background of deep learning.
However, unfortunately I cannot know to what ...
- 361
23
votes
2
answers
2k
views
Structures of the space of neural networks
A neural network can be considered as a function
$$\mathbf{R}^m\to\mathbf{R}^n\quad
\text{by}\quad x\mapsto w_N\sigma(h_{N-1}+w_{N-1}\sigma(\dotso h_2+w_2\sigma(h_1+w_1 x)\dotso)),$$
where the $w_i$ ...
- 333
94
votes
14
answers
14k
views
Deep learning / Deep neural nets for mathematician
I am interested in finding out the math ideas behind the technologies that are under the umbrella of "Deep Learning" or "Deep neural nets".
Most of the papers/books that are often quoted in papers/...
30
votes
6
answers
11k
views
Mathematics for machine learning
I would like to know what mathematics topics are the most important to learn before actually studying the theory on neural networks.
I ask that because I will start to learn about neural networks and ...
- 419