All Questions
Tagged with neural-network rnn
226
questions
0
votes
0
answers
14
views
Semantics Building In LSTM-Based Models - How does a LSTM is able to extract and represent long data using just one value (long-memory)
How does a LSTM is able to extract and represent long sequences with data while using just one value (long-memory / LM) to maintain all this information?
If multiple value were used, it could be ...
0
votes
0
answers
11
views
Deep neural network is plateauing on a regression task
I'm training a deep neural network on temporal graph data. Currently, I'm trying to get a feel for how large / complex of a model I should aim for, so I'm trying to overfit to my smallest dataset. ...
0
votes
1
answer
23
views
Connecting Flatten layer to Dense layer
I'm struggling with my neural network.
In short, I need to recreate a model from anywhere on the internet, I've found a model that combines BiLSTM, LSTM and GRU. However, based on the error I got when ...
0
votes
0
answers
8
views
Has someone designed a neural network which can select its own activation functions and/or have multiple activation functions in one model?
I'm wonder if there are any papers or implementations where a neural network has multiple activation functions in a single model (and layer), and preferably also where such activation functions ...
- 111
1
vote
1
answer
63
views
Why my validation loss and accuracy decays over epochs?
Im trying to build 2 simple networks with cleaned dataset for tweets sentiment classification(0/1):
one with all dense layers(binary bag of words)
another with RNN layer(embedding layer).
But it both ...
- 35
1
vote
0
answers
44
views
LSTM for classification
I am new to neural networks and I want to use LSTM to classify the on/off state of devices based on power values. In my training dataset, I have power values, device one (0,1), and device 2 (0,1). 0 ...
- 11
0
votes
1
answer
39
views
Confusion regarding what constitutes a feature in a LSTM?
I have a Time Series problem, where I am trying to predict a single output at time $t$, $y_t$, given the $2$ previous time steps; $X_{t-2}, X_{t-1}$.
Let's just look at one observation for simplicity.
...
- 139
5
votes
2
answers
348
views
Modeling uncertainty from known physics
I have an equation given by:
$$
\frac{\mathrm{d} s}{\mathrm{d} t}=4a−2s+\lambda(s)
$$
where, $a$ is an input constant and $\lambda$ is a non-linear term that depends on $s$.
I know that the true ...
user149991
0
votes
1
answer
38
views
Why is a RNN inherently better for Time series than normal NN?
Similar to this question but I would like further clarification.
I understand that in abstract, RNNs can process inputs recursively and feed some state of memory through the recursion to have a sense ...
- 3
1
vote
1
answer
138
views
How is RNN decoder output calculated?
I was trying to read RNN Encoder Decoder paper.
RNN (plain RNN i.e. non encoder-decoder RNN)
It starts with giving equation for RNN:
hidden state in RNN is given as:
... equation (1)
where f is a ...
- 199
0
votes
0
answers
38
views
Input size vs hidden state in RNNs
Im using PyTorch to implement RNNs on univariate time series data. This is the documentation for the RNN class: link
I think I'm understanding the math behind an RNN cell. But I have an specific ...
- 101
1
vote
1
answer
30
views
Should I annotate additional information besides the categories I already need in a text?
I have a dataset with bank transfer reasons. They vary a lot because humans wrote them.
From the reasons that are linked to invoice payments I need to extract several things:
invoice number(s)
IBAN
...
- 175
0
votes
1
answer
26
views
RNN basic doubt
Suppose if I have 2 sentences:
"My name is Alex"
"Alex is my name"
If I am using a RNN, After processing both the sentences, Will the final output vector be the same?
Because RNN ...
- 109
0
votes
0
answers
19
views
Multilayer/deep recurrent layer
I might be missing something, but I'm completely unable to find any reference about this topic.
In the literature, there are many references about RNN, GRU, LSTM, STAR and many other architecture that ...
anon
0
votes
1
answer
19
views
How to deal with different amounts of data every day?
I am doing a time series prediction task. There are different amounts of news headlines every day, and the goal is a binary prediction task to predict next day's stock movement.
The amount of ...
- 165