I'm trying to set a different activation function for each hidden unit in a layer. Is this possible in Keras with 'Concatenate'?
$\begingroup$
$\endgroup$
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
6
If I get the point, you can use a similar code like the following:
from keras.layers import merge, Convolution2D, MaxPooling2D, Input
input = Input(shape=(256, 256, 3))
seq1 = Dense(1, activation = 'relu')(input)
seq2 = Dense(1, activation = 'sigmoid')(input)
seq3 = Dense(1, activation = 'tanh')(input)
acum = merge([seq1, seq2, seq3], mode='concat', concat_axis=1)
Depending on your task, specify concat_axis.
answered Dec 23, 2017 at 18:46
-
$\begingroup$ how should I set "concat_axis"? I want to have all units in one layer. I read the documentation, but I didn't get much from it. $\endgroup$– P.JosephCommented Dec 23, 2017 at 21:04
-
$\begingroup$ suppose you have [[1, 2], [3, 4]] and [[98, 99], [100, 101]] if your axis is 0 the rows get concatenated, if 1, the columns get concatenated. $\endgroup$ Commented Dec 23, 2017 at 21:09
-
$\begingroup$ ok, but what does the default value that is -1 mean in this case? $\endgroup$– P.JosephCommented Dec 24, 2017 at 9:36
-
$\begingroup$ I don't have access to its documentation right now, but if you are familiar with numpy.reshap, whenever you specify -1 it does a default operation, which in this case it unrolls the array to reshape that as a vector if you set it as the first input if I remember. $\endgroup$ Commented Dec 24, 2017 at 12:09
-
$\begingroup$ well, the code works fine but I'm still not sure that it is doing what I want $\endgroup$– P.JosephCommented Dec 25, 2017 at 22:30