What Causes PHP Forks to Consolidate on a Single CPU Core in FreeBSD 13.3?

Question

I'm using a PHP 8.x script to process a series of images, performing various conversions and resizing tasks. To make the most of the server's multiple cores, I employ the pcntl_fork() function to create child processes that can simultaneously handle different images. This means instead of processing images sequentially, each image can be processed concurrently on separate cores.

For instance, if I have 10 images to process and each takes 3 seconds individually, without parallel processing, it would take a total of 30 seconds. However, with parallel processing, all 10 images can finish processing simultaneously in just 3 seconds.

This approach has been effective until we updated to FreeBSD 13.3. After the update, the forked processes no longer distribute across different cores; instead, they all run on a single core. Consequently, if I have 10 forked processes running, each is constrained to using only 10% of a single core, resulting in a 10-fold increase in processing time.

We've conducted tests with FreeBSD versions ranging from 9.x up to 13.2-RELEASE-p11 and found that the issue doesn't occur. Additionally, when using a 13.2 userland and temporarily booting the 13.3 kernel, the problem still doesn't manifest. However, when both the userland and kernel are updated to 13.3, the problem consistently occurs.

Further tests with a fresh installation of FreeBSD 14.0 on a separate system confirm that the issue persists there as well.

We've also ruled out PHP version as a factor, as testing across versions 8.0 to 8.3 yields the same results.

Do you have any insights into what might be causing this issue, or suggestions for resolving it?

Edit: Adding MRE code, as suggested in the comments:

<?php
$fullLocalImagePath = "test.jpg";
$forkedProcessIds = array();
for($i = 0; $i < 5; $i++) {
  $pid = pcntl_fork();// Create a child process
  if($pid == -1) {
    exit(-1); // Could not fork the process
  } else if($pid) {
    $forkedProcessIds[] = $pid; // Keep track of the child IDs
  } else {
    $finalImagePath = "test_result_".$i.".jpg";

    // Load the image using Imagick
    $imagick = new Imagick();
    $imagick->readImage($fullLocalImagePath);
    echo " → Finished reading image $i into Imagick.\n";
    $imagick->setImageCompressionQuality(88);
    $imagick->resizeImage(4800, 4800, imagick::FILTER_LANCZOS, .9, false);
    $imagick->writeImage($finalImagePath);
    echo " → → Finished resizing and saving image $i into Imagick.\n";
    $imagick->clear();

    exit(0); // Exit the child process after processing the image
  }
}

// Wait for the forked processes to finish
while ($childPID = pcntl_waitpid(0, $status)) {
  if ($childPID == -1) {
    // No child processes left to wait for
    break;
  }

  echo " → → → Child process $childPID has finished.\n";

  // Handle the exit status based on the child process PID
  if (in_array($childPID, $forkedProcessIds)) {
    // Remove the child process ID from the tracking array
    $forkedProcessIds = array_diff($forkedProcessIds, array($childPID));
  }
}
?>

Answer 1

The problem, and solution, have now been found. It is a bug in the LLVM code.

From GitHub issues:

It's a bug in the atfork() handler on Unix systems + logic in reinitializing the child process. The current library incorrectly sets the child process' affinity to compact, which roughly translates to "pin consecutive threads to consecutive cores", even when the user hasn't set KMP_AFFINITY to anything. So every child process was pinned to the first core instead of the entire system.