One hotly requested feature for my single header C/C++ process spawning and management library subprocess.h was the ability to read from the standard output or error of a spawned process while it was still executing. In this PR I’ve added support for this, but it requires some change of behaviour in your code if you want to use it.

C FILE’s and Asynchronous Reading⌗

I use a really clever hack to give my users a standard C FILE handle with which they can read the standard output and error, and write to the standard input of any processes they spawn. This is immensely powerful because it means anyone using my API can use an already familiar paradigm to interact with the processes they spawn. Users could fprintf to the standard input of a spawned process, or call fgets to read from the standard output or error. While I love this feature - it causes a complication when trying to support asynchronous reading.

For the normal process spawning on Windows I was using CreatePipe to map a HANDLE on the parent process to HANDLE that’ll be used for the standard pipes on the spawned process. A CreatePipe is also known as an anonymous pipe in Windows lingo - and it turns out you cannot sync from this pipe until the spawned process has complete. But there is another mechanism called a named pipe using the CreateNamedPipe call that can help us here.

Named pipes support an additional flag on creation FILE_FLAG_OVERLAPPED that lets you get at the data from the pipe before the other end of the pipe has been closed. I thought ‘Great! I’ll use this et voila! Everything will work!’. It didn’t.

The problem is that if you use the overlapped creation flags to a named pipe you cannot then do the clever hack anymore to get a C FILE that reads from this. It’ll just hang forever.

So instead I’ve introduced some new helpers methods subprocess_read_stdout and subprocess_read_stderr that use the correct APIs on each operating system to allow for asynchronous reading from a pipe.

Note that while the main problems for this feature have been with Windows support and the example below is Windows specific - the feature supports all three major desktop platforms. Worry not!

Example⌗

As an example of how we can use this I’ve wrote up the following little example that works on Windows (no reason why it couldn’t work on other operating systems, I just used Windows for the test):

int main() {
  const char *const commandLine[] = {"ping", "-t", "www.bbc.net.uk", 0};
  struct subprocess_s process;
  unsigned bytes_read;

  if (0 != subprocess_create(
    commandLine,
    subprocess_option_inherit_environment | subprocess_option_enable_async,
    &process)) {
    fprintf(stderr, "subprocess_create failed!");
  }

  do {
    char buffer[1024] = {0};
    bytes_read = subprocess_read_stdout(&process, buffer, sizeof(buffer));
    printf("Read %u bytes - '%s'\n", bytes_read, buffer);
  } while (bytes_read != 0);
}

This will loop infinitely reading from the ping command with the -t option specified. It’ll read the output from ping as it is flushed and spit out what it read and the number of bytes.

Read 42 bytes - '
Pinging www.bbc.net.uk [212.58.237.253] '
Read 24 bytes - 'with 32 bytes of data:
'
Read 27 bytes - 'Reply from 212.58.237.253: '
Read 27 bytes - 'bytes=32 time=22ms TTL=53
'
Read 27 bytes - 'Reply from 212.58.237.253: '
Read 27 bytes - 'bytes=32 time=22ms TTL=53
'

A few things:

  • Note the new subprocess_create option subprocess_option_enable_async. This is needed to setup the pipe for being able to read from it asynchronously. If you use this option you should refrain from using subprocess_stdout or subprocess_stderr to read from the spawned process - they will not work.
  • Note the new function subprocess_read_stdout to read from the standard output of the spawned process. It’ll return non-zero while there is any data still to be read. When the child process terminates then it’ll return 0 signalling the end.

One additional thing to bear in mind is that the subprocess_read_stdout and subprocess_read_stderr functions are blocking. If you want to read from both of these simultaneously then it is recommended that you either:

  • Spawn a thread for one or both to read from it. It is thread safe to read from both pipes of a spawned process at the same time.
  • Or use the subprocess_option_combined_stdout_stderr option if you just want the full contents of both pipes both don’t care to differentiate from either pipes output.

I hope you find this new functionality useful.

An additional gentle reminder that you can stay tuned for future updates by using the RSS feed.