5
$\begingroup$

Scikit-learn lists these as the implemented activation functions for it's multi-layer perceptron classifier:

‘identity’, no-op activation, useful to implement linear bottleneck, returns f(x) = x
‘logistic’, the logistic sigmoid function, returns f(x) = 1 / (1 + exp(-x)).
‘tanh’, the hyperbolic tan function, returns f(x) = tanh(x).
‘relu’, the rectified linear unit function, returns f(x) = max(0, x)

Does anyone know if it is possible to implement a custom activation function? If not, can someone point me to a library where this is possible?

Improve this question
$\endgroup$
4
  • 2
    $\begingroup$ sklearn wasn't made specifically for this task, so it is missing features like this. I recommend pyTorch instead; it's the latest and greatest in the field, and it's simple. Here's a relevant discussion. Welcome to the site and good luck. $\endgroup$
    – Emre
    Commented Apr 27, 2017 at 22:53
  • $\begingroup$ Thanks so much Emre! I just got pyTorch up and running and am fiddling with the forward function right now $\endgroup$
    – pennydreams
    Commented Apr 28, 2017 at 17:47
  • $\begingroup$ @pennydreams did you ever get to completing the function and how did you implement it? I am trying something similar and could use some pointers! $\endgroup$
    – dsforlife84
    Commented Nov 6, 2017 at 13:40
  • $\begingroup$ @dsforlife84 As Emre stated, it doesn't seem to be possible to implement a custom activation function in scikit-learn's MPLClassifier. I'd look at tensorflow or pytorch for implementing neural nets with custom activation functions. $\endgroup$
    – pennydreams
    Commented Nov 7, 2017 at 14:42

1 Answer 1

Reset to default
2
$\begingroup$

Although @Emre's comment that sklearn wasn't specifically made for the task is correct, nothing stops you from introducing another activation function by implementing it in sklearn/neural_network/_base.py. You should be able to find the file in your installation.

To this end, you would define your single-parameter function and add it in the ACTVATIONS list. Refer to the existing implementations for some guidance how to do it.

Improve this answer
$\endgroup$
1
  • 1
    $\begingroup$ Looks like we'd also need to implement its derivative $\endgroup$
    – Benitok
    Commented Feb 24, 2021 at 4:30

Not the answer you're looking for? Browse other questions tagged or ask your own question.