User-space pipe implementation

Help & Support
1

A couple months ago I wrote a shard that works like IO.pipe but lives entirely in user-space. That is, it does not involve syscalls at all. The shard is called pipe:

Two reasons this is noteworthy:

  • IO.pipe returns file descriptors, so all I/O between them must pass all the way through the kernel
  • POSIX systems set limits on how many file descriptors you can have open at one time (ulimit -n) and IO.pipe consumes two of them, so you could end up running out of file descriptors in your app if you use IO.pipe regularly

Yesterday I wrote a benchmark that shows a realistic approach to using pipes that I use pretty regularly: send a large stream of data into the writer end of a pipe and use the reader end of it as the HTTP::Request#body. This way, you’re not loading an unbounded quantity of data into RAM and serializing it into a string that is also in RAM before passing it to the HTTP::Request.

The results of that benchmark show an order of magnitude difference in performance at the CPU even while performing HTTP and JSON serialization. On a cheap DigitalOcean node:

Pipe::Reader: 2.65s (2.51s user, 132ms system)
IO::FileDescriptor: 20.7s (10.3s user, 10.5s system)

On my laptop (Apple M4):

Pipe::Reader: 306ms (299ms user, 7.89ms system)
IO::FileDescriptor: 4.23s (1.24s user, 2.99s system)

Notice that it’s not just faster in “user” CPU, but it’s spending almost no time on “system” CPU (syscalls inside the kernel). This is because we’re not using syscalls at all! That system time is due to the I/O involved in sending the data over HTTP and not in sending the data over the pipe.

4 Likes
2

I have a use-case scenario for it. In my PoC for LXC container management, I am currently providing shell/terminal access through Process.exec at github.com/admiracloud/admira-containers/blob/develop/src/library/admiractl.cr#L53, killing my own program process and returning the shell process directly.

def enter(name : String)  
  return Process.exec("lxc-attach", args: [name], shell: false)  
end

For CLI usage this is fine, but for the real-time API in which my orchestrator should “proxy” the terminal and not die, I will have to pipe it, something like:

def enter(name : String)
  stdin_read, stdin_write = IO.pipe
  stdout_read, stdout_write = IO.pipe
  stderr_read, stderr_write = IO.pipe

  Process.run("lxc-attach", [name],
    input: stdin_read,
    output: stdout_write,
    error: stderr_write
  ) do |process|
    # ...
  end  
end

And with your shard this would be clearly more efficient. :high_voltage:

3

I almost forgot: congratulations on this shard. High quality work