I'm doing logistic regression in C, but I have problems with calculating the gradient for the y-intercept.
Here's the code: - sigmoid_j()
and log_odds_pred()
works perfectly. I'll explain the problem in more detail in the specific line - at the function definition of gradient_i()
#include <stdio.h>
#include <stdlib.h>
#include <math.h>
char m = 2, n = 2;
double sigmoid_j(double); //Gives probability prediction for a training example
double log_odds_pred(double *theta, double *b, float x[2][2], int ex_number); //Gives log-odds prediction for a training example
void gradient_i(double *theta, double *b, float x[2][2], char y[], double *dj_db); //Calculates gradient of a parameter for feature n
int main(){
//Please ignore why I reset m, n. I'll fix this later
m = 2;
n = 2;
float x[2][2] = {{16, 23}, {0.3,1}};
char y[] = {0,1};
//Perfect parameters - these are just the ones that my
double theta_init[] = {0.018,0.034};
double b_init = -0.025;
//TEST - GRADIENT
double dj_db = 0;
gradient_i(theta_init,&b_init,x,y,&dj_db);
printf("Gradient of b: %lf\n",dj_db);
return 0;
}
void gradient_i(double *theta, double *b, float x[2][2], char y[], double *dj_db){
double grad = 0;
double y_hat_j; //prediction/hypothesis
for(int j=0;j<m;j++){
y_hat_j = sigmoid_j(log_odds_pred(theta,b,x,j));
//printf("h_%d, %lf\n", j, y_hat_j);
//and here's the problem. If I don't use printf, I'm getting 0.1176
//for the gradient of b (output of the function), which is wrong. If I do print this line,
//I'll get 0.86, which is the right result. I'm guessing it updates
//y_hat_j, but I don't understand why?
grad += (y_hat_j - y[j]);
}
grad /= m;
*dj_db = grad;
}
double sigmoid_j(double z_j){
return 1/(1+exp(-z_j));
}
double log_odds_pred(double *theta, double *b, float x[2][2], int ex_number){
double result;
for(int i=0; i<n;i++){
result += (*(theta+i)) * (x[i][ex_number]);
}
result += *b;
return result;
}
Without a print statement in each loop, the prediction is not getting updated.
return
in a function or something similar. Try compiling your program with Address Sanitizer (-fsanitize=address
for GCC).printf
. You're not initializingresult
inlog_odds_pred
-Wuninitialized
/-Wmaybe-uninitialized
(both enabled by-Wextra
) depend on the optimization level in use.