Set pipe capacity in Linux

Question

Is there a way to set the pipe capacity of pipes defined in a Bash (or other shell) script? Take e.g.

cmd1 | cmd2

In recent Linuxes the pipe capacity is set to 64KB by default. I know I can control the amount of data "buffered" between the two processes in two ways:

1. Using buffer(1): e.g. cmd1 | buffer | cmd2
2. Using fcntl(2) with the F_SETPIPE_SZ flag from inside cmd1 or cmd2

Each solution has downsides: buffer can only be used to increase the buffer; also writes over the default pipe capacity will still require waking up the downstream command. fcntl, as far as I know, can only be called from inside cmd1 or cmd2.

My question is: is there a way, when the shell creates the pipe, to specify in the shell how much capacity the pipe should have?

Answer 1

writes over the default pipe capacity will still require waking up the downstream command

If your goal is not to wake up the downstream command too often, did you try using the -p option to buffer? It should cause buffer to hold writes until the buffer is filled to certain percentage. You might need the -s option as well to write large chunks.

Update: D'oh, the pipes between the commands still limit things. Maybe try using the following adapter program:

#define _GNU_SOURCE
#include <stdlib.h>
#include <unistd.h>
#include <fcntl.h>
int main(int argc, char** argv)
{
  fcntl(atoi(argv[1]), F_SETPIPE_SZ, atoi(argv[2]));
  execvp(argv[3],argv+3);
  while (1);
}

as in:

adapter 1 (BIGSIZE) cmd1 | cmd2

or even:

adapter 1 (BIGSIZE) cmd1 | adapter 1 (BIGSIZE) buffer [args] | cmd2

if cmd1 still makes small writes.

Answer 2

Based on DepressedDaniel and Stéphane Chazelas suggestions I settled on the closest thing to a oneliner I could find:

function hugepipe {
  perl -MFcntl -e 'fcntl(STDOUT, 1031, 1048576) or die $!; exec { $ARGV[0] } @ARGV or die $!' "$@"
}

This allows to do:

hugepipe <command> | <command>

and the pipe between the two commands is going to have the capacity specified via the fcntl in the perl script.