Neural Network training using PSO in R

Question

I need to train a neural network using PSO algorithm in R enviroment. I already know all the R packages about neural networks ( neuralnet, AMORE, etc. ), but no one of these includes PSO training ( only backpropagation ).

Ideas? Thanks for the help.

Answer 1

I used PSO to optimise the hyperparameters of a XGBoost model in R in the past. See this link for the article

Code Example below:

# Set parameter settings for search algorithm
max_iter <- 10 # maximum number of iterations
pop_size <- 10 # population size

# Create custom function for assessing solutions
eval_function_XGBoost_Linear <- function(x, data, train_settings) {
  
  x1 <- x[1]; x2 <- x[2]; x3 <- x[3]; x4 <- x[4]
  
suppressWarnings(
  # Create dataframe with proportion of each solid component
  XGBoost_Linear_model <- caret::train(Strength ~., 
                                      data = data,
                                      method = "xgbLinear",
                                      trControl = train_settings,
                                      verbose = FALSE, 
                                      silent = 1,
                                      tuneGrid = expand.grid(
                                                            nrounds = round(x1),
                                                            eta = 10^x2, 
                                                            alpha = 10^x3,
                                                            lambda = 10^x4
                                                            ) 
                                      )
)

    return(XGBoost_Linear_model$results$RMSE) # minimize RMSE

}

# Define minimum and maximum values for each input
nrounds_min_max <- c(10,10^3)
eta_min_max <- c(-5,3)
alpha_min_max <- c(-3,1)
lambda_min_max <- c(-3,1)

set.seed(1)
n_cores <- detectCores()-1

# Run search algorithm
PSO_model_XGBoost_Linear <- pso::psoptim(
  par = rep(NA, 4),
  fn = eval_function_XGBoost_Linear, 
  lower = c(nrounds_min_max[1], eta_min_max[1], alpha_min_max[1], lambda_min_max[1]),
  upper = c(nrounds_min_max[2], eta_min_max[2], alpha_min_max[2], lambda_min_max[2]), 
  control = list(
                trace = 1, 
                maxit = max_iter, 
                REPORT = 1, 
                trace.stats = T,
                s = pop_size, 
                maxit.stagnate = round(0.75*max_iter), 
                vectorize = T,
                type = "SPSO2011"
                ),
  data = training_set,
  train_settings = train_control
  )

Hope this helps!