13

In a linux application I'm using pipes to pass information between threads.

The idea behind using pipes is that I can wait for multiple pipes at once using poll(2). That works well in practice, and my threads are sleeping most of the time. They only wake up if there is something to do.

In user-space the pipes look just like two file-handles. Now I wonder how much resources such a pipes use on the OS side.

Btw: In my application I only send single bytes every now and then. Think about my pipes as simple message queues that allow me to wake-up receiving threads, tell them to send some status-data or to terminate.

Improve this question
2
  • 6
    Posix specifies the pipe interface, not the implementation, which is where the "weight" comes from.
    – anon
    Commented Mar 28, 2010 at 15:27
  • How many channels? How many messages per second? Hundreds? Millions? Make a simple prototype using several technologies (anything from IPC message queues to pipes to UNIX sockets etc.) and compare.
    – Tadeusz A. Kadłubowski
    Commented Mar 28, 2010 at 15:34

3 Answers 3

Reset to default
8

No, I would not consider pipes "lightweight", but that doesn't necessarily mean they're the wrong answer for your application either.

Sending a byte over a pipe is going to require a minimum of 3 system calls (write,poll,read). Using an in-memory queue and pthread operations (mutex_lock, cond_signal) involves much less overhead. Open file descriptors definitely do consume kernel resources; that's why processes are typically limited to 256 open files by default (not that the limit can't be expanded where appropriate).

Still, the pipe/poll solution for inter-thread communication does have advantages too: particularly if you need to wait for input from a combination of sources (network + other threads).

Improve this answer
4
  • 1
    Why would it take 3 system calls? a blind write() call will block if the pipe's buffers are full. if each thread only needs to listen to its own pipe, you can drop the poll() as well, simply waiting for read() to succeed
    – Hasturkun
    Commented Mar 28, 2010 at 15:17
  • 2
    OP specifically said that he's using poll() to wait for multiple pipes from one consumer thread.
    – David Gelhar
    Commented Mar 28, 2010 at 15:19
  • I have to read and write to one file (/dev/ttySxx serial) and be able to respond to commands from another thread. Blocking is out of question. If I for example have a 10 megabyte write on the serial I can't wait until it is finished if the application tells me that it wants to abort the transfer. Therefore two non-blocking files that I poll for: One for the serial port and one to receive commands.
    – Nils Pipenbrinck
    Commented Mar 28, 2010 at 15:28
  • 1
    It's worth noting that if you want poll to wait for other things then you pretty much have to use pipes/eventfd since you cannot mix POSIX synchronization primitives with anything else (even with sockets). And if you spawn a new thread to deal with network event and convert these to POSIX mutexes and the like then you probably already thrown their performance advantage out of the window.
    – jpc
    Commented Nov 23, 2012 at 17:53
5

As you are using Linux you can investigate and compare pipe performance with eventfd's. They are technically faster and lighter weight but you'll be very lucky to actually see the gains in practice.

http://www.kernel.org/doc/man-pages/online/pages/man2/eventfd.2.html

Improve this answer
1
  • WOW! Thanks for pointing this out. I'm using pipes for this kind of things, and I'm sure I can replace at least half of them using eventfd's. Most of the time all I really want is to signal a state to a thread that is blocked in poll. If eventfd is more lightweight I'll use it.
    – Nils Pipenbrinck
    Commented Mar 21, 2011 at 21:35
1

Measure and you'll know. Full processes with pipes are sufficiently lightweight for lots of applications. Other applications require something lighter weight, like OS threads (pthreads being the popular choice for many Unix apps), or superlightweight, like a user-level threads package that never goes into kernel mode except to handle I/O. While the only way to know for sure is to measure, pipes are probably good enough for up to a few tens of threads, whereas you probably want user-level threads once you get to a few tens of thousands of threads. Exactly where the boundaries should be drawn using today's codes, I don't know. If I wanted to know, I would measure :-)

Improve this answer
1
  • I guess I'm confused. I think Linux's NPTL implementation is kernel-level. Does glibc provide a user-level implementation on top of that?
    – Wei Hu
    Commented Mar 29, 2010 at 2:59

Not the answer you're looking for? Browse other questions tagged or ask your own question.