Chrizog

Three Ways For C++ Thread Synchronization in C++11 and C++14

March 20, 2022

Three Ways For C++ Thread Synchronization in C++11 and C++14

In this article I will give you an overview about different ways of synchronizing C++ threads. Wile C++ is a powerful language and you can get almost anything done with it, it's daunting for beginners to capture all the possible options provided by the language. For that reason I provide an overview and comparison of different C++ synchronization mechanisms in one place.

With threading in C++ there are mainly two tasks for the developer:

  • Protecting shared data from concurrent access
  • Synchronizing concurrent operations

Protecting shared data is achieved by mutual exclusion. By avoiding that multiple threads access the same data at the same time race conditions are prevent. Two read operations are permitted. But two write operations or one write and one read operation lead to race conditions. For using locks and mutexes I can recommend this article.

This article is about the second issue: Synchronizing concurrent operations in C++ which means coordinating the order of events between multiple threads. It happens that one thread must wait on another thread to complete a task before it can continue its own task. For example one thread indexes data in the background and the second thread needs to use the indexed data. The first thread is maybe not ready yet with indexing and the second thread shall wait on the result of the first thread.

Many of the synchronization problems can be broken down to the producer-consumer pattern: A consumer thread is waiting for a condition to be true. The producer thread is setting the condition. How can you achieve the waiting on a condition in C++? And how can you pass the results of tasks between different threads? I will explain three different methods:

  1. Periodically checking the condition (the naive and worst approach)
  2. Condition variables
  3. Futures and promises

1. Periodically checking the condition - A naive approach

Pros:

  • Easy to understand

Cons:

  • High resource consumption (CPU usage up to 100%)
  • Unnecessary wakeups of processes
  • Bad design

Imagine you are waiting for the parcel delivery at your home for your long awaited packet (a brand new PlayStation 5). In order to not miss the delivery, there are several options. One option, which is also the worst idea, is to sit at your front window all day and starr at the entrance. As soon as you see the delivery car, you run to the door and open it. The drawbacks of this approach are obvious: Your eyes will get tired since you are starring out of the window all day. You also cannot do anything else in the meantime but only wait for the packet. A lot of wasted time.

You can put this example in relation to C++ threads. The waiting thread (= consumer) is checking the condition to be true continuously in a while loop. The producing thread is setting the condition as soon as it is ready. The condition can be a simple boolean flag protected by a mutex. The condition could be also checking if a queue is empty. The queue is protected by a mutex. See the following code for an example:

At (1) in main we are starting two threads: the producing and the consuming thread.

At (2) a new Packet is put into the queue periodically. For the write operation to the queue, the mutex m is locked using a std::lock_guard. The sleep of 1 second simulates a long running task (e.g. the parcel delivery must drive to your house).

In (3) the mutex is periodically locked and it is tested if there is a packet in the queue. If there is a packet in the queue it is taken out of the queue and processed.

#include <chrono>
#include <iostream>
#include <mutex>
#include <queue>
#include <ratio>
#include <thread>

struct Packet {
  int length;
  int width;
  int height;
};

std::mutex m;
std::queue<Packet> packet_queue;

void parcel_delivery() {
  // (2): The producer thread: Periodically put a new packet into a queue
  int counter = 0;
  while (true) {
    // Perform some long running task
    std::this_thread::sleep_for(std::chrono::seconds(1));
    std::cout << "Work is done and packet is ready!" << std::endl;
    Packet new_packet{counter, counter + 1, counter + 2};
    {
      std::lock_guard<std::mutex> l{m};
      packet_queue.push(new_packet);
    }
    counter++;
  }
}

void recipient() {
  while (true) {
    {
      // (3): Lock the mutex and test if a packet was received
      std::lock_guard<std::mutex> l{m};
      if (!packet_queue.empty()) {
        auto new_packet = packet_queue.front();
        packet_queue.pop();
        std::cout << "Received new packet with dimension: " << new_packet.length
                  << "; " << new_packet.width << "; " << new_packet.height
                  << std::endl;
      }
    }
  }
}

int main(int argc, const char **argv) {
  // (1): Start two threads: Producer and consumer
  std::thread producing_thread(parcel_delivery);
  std::thread consuming_thread(recipient);
  producing_thread.join();
  consuming_thread.join();
  return 0;
}

When running the above code, the CPU usage will likely jump to 100% for that process due to the while-loop in the consuming-thread. It is reading the value of packet_queue.empty() all time. You might think: "Hmm, I could simply put a sleep in the recipient's while loop - problem solved!"

void recipient() {
  while (true) {
    {
      // 3: Lock the mutex and test if a packet was received
      std::lock_guard<std::mutex> l{m};
      if (!packet_queue.empty()) {
        auto new_packet = packet_queue.front();
        packet_queue.pop();
        std::cout << "Received new packet with dimension: " << new_packet.length
                  << "; " << new_packet.width << "; " << new_packet.height
                  << std::endl;
      }
    }
    // 4: Introduce a sleep for 100 milliseconds
    std::this_thread::sleep_for(std::chrono::milliseconds(100));
  }
}

With the sleep at (4) you will reduce the CPU usage as the consuming thread will go into sleep state. But you will introduce an undesired delay and unnecessary wakeups. In the worst case, you need to wait 100 milliseconds to react to the newly arrived packet.

You can decrease the sleep time, but the question will be always: How fast is fast enough and what delay is acceptable?

This naive approach is usually considered as bad design.

2. Using a condition variable

Pros:

  • No CPU resource waste
  • Condition variables can be used multiple times (in comparison to futures which are for one-shot events)

Cons:

  • Take care of spurious and lost wakeups

Typical problem scenario:

  • Having a network thread that receives data regularly and notifies the main worker thread
  • Watching file changes periodically in a background thread

Following the previous story of waiting for the packet, wouldn't it be a lot better if you'd just get notified by the door bell by the postman? That would make a lot more sense. While waiting for the door bell to ring, you could even clean the house in the meantime.

With C++ threads this can be achieved using a condition variable. A condition variable has a wait-function, that you can use to block the consuming thread. Using the wait-function your thread is consuming no CPU resources.

In the producing thread you call a notify function on the condition variable. This will cause the consuming thread to wake up and continue after the wait call.

At (1) in main we are starting two threads again: a producing and a consuming thread.

At (2) in the consuming thread, you must use a std::unique_lock and acquire the lock on the mutex m. Condition variables only work with std::unique_lock. Afterwards you call the wait function and pass the lock and a predicate.

A predicate can be a function that returns a boolean. The predicate shall return true when the thread should not wait anymore, i.e. when the event happens. For that purpose we use a lambda expression that returns true if there is a Packet in the queue: []() { return !packet_queue.empty(); }.

The wait function releases the mutex and suspends the thread. There is no active waiting (meaning actively reading the variable as it is done in the naive approach) and therefore no CPU resources are wasted. When a wake-up happens, the lock is automatically reacquired. You then take the element out of the queue and release the lock. Since we take out the element by a copy, the std::cout which represents the processing of the packet does not need to be inside the critical section protected by the mutex.

There are existing variants wait_for and wait_until that allow you to have a timeout on the wait operation as well. Read more at cppreference.

At (3) the mutex is locked on the producer thread. Here a lock_guard is enough, since you only need to protect the critical section for pushing a new packet to the packet_queue. Afterwards call notify_one() in order to wakeup the waiting thread. There is also the possibility that multiple threads wait on the condition variable. Then you can use notify_all() instead of notify_one().

#include <chrono>
#include <condition_variable>
#include <iostream>
#include <mutex>
#include <queue>
#include <thread>

struct Packet {
  int length;
  int width;
  int height;
};

std::condition_variable cond_var;
std::mutex m;

std::queue<Packet> packet_queue;

void parcel_delivery() {
  int counter = 0;
  while (true) {
    // Perform some long running task
    std::this_thread::sleep_for(std::chrono::seconds(1));
    std::cout << "Work is done and data is ready!" << std::endl;
    Packet new_packet{counter, counter + 1, counter + 2};
    {
      // 3: Lock the mutex before writing to the queue and notify the waiting
      // thread
      std::lock_guard<std::mutex> l{m};
      packet_queue.push(new_packet);
    }
    cond_var.notify_one();
  }
}

void recipient() {
  while (true) {
    // 2: Use a unique_lock and check the condition with a predicate
    std::unique_lock<std::mutex> lock{m};
    cond_var.wait(lock, []() { return !packet_queue.empty(); });

    auto new_packet = packet_queue.front();
    packet_queue.pop();
    lock.unlock();
    std::cout << "Received new packet with dimension: " << new_packet.length
              << "; " << new_packet.width << "; " << new_packet.height
              << std::endl;
  }
}

int main(int argc, const char **argv) {
  // 1: Start two threads: Producer and consumer
  std::thread producing_thread(parcel_delivery);
  std::thread consuming_thread(recipient);
  producing_thread.join();
  consuming_thread.join();
  return 0;
}

Lost Wakeups

One issue that comes with condition variables are lost wakeups. A lost wakeup happens when the notify call executes before the wait call. In that case the condition variable would wait forever or until the next notify call. The originally intended wakeup was lost.

For that reason you should always use the wait function overload with the predicate that I used in the example above. There is also a wait function without the predicate that is not recommended to use.

With the predicate the wakeup is not lost even if the notify call executes before the wait call. Because of the condition that is evaluated in the predicate !packet_queue.empty() the wait function returns immediately and the waiting thread continues its work.

Spurious Wakeups

It can also happen that the waiting thread wakes up from waiting on the condition variable for no reason. It wakes up even if no notification was intended.

That's another reason to always use wait function overload with the predicate. If the thread wakes up, it checks if the predicate returns true. Since it was a spurious wakeup the condition is not satisfied and the thread is suspended again and waits. Without the predicate the waiting thread would continue its work before notify_one was called. With the predicate this is prevented.

3. Futures and Promises

Pros:

  • Easily return results from threads
  • No spurious or lost wakeups happen
  • Do multithreading without using std::thread

Cons:

  • Only one-shot events are possible

A very nice and simple way to synchronize concurrent tasks is std::future<T>. A future represents a value that may be computed in the future, e.g. it could be the result of a computation that returns an integer (std::future<int>). This can be the case when you want to execute a long running computation in a separate thread and return the result from the thread back to the main thread. Or it can be simply an event that you want to wait for where no result is returned from the thread (std::future<void>).

The major difference compared to condition variables is that futures can be only used once. With the condition variable the parcel delivery thread could produce packets multiple times and notify the waiting thread. With an std::future you could only deliver a single packet.

Additionally with std::future you may not even need to use std::thread which keeps the complexity of your code low and understandable.

Future with std::async

There are different ways to get a std::future. The easiest way is to use std::async. With std::async you handover a task (a Callable) and a future is returned. On the future you can call the get() function to retrieve the result that is returned from the thread. The get() call blocks the waiting thread until the task from std::async is finished.

Without an std::future and std::async you would normally need a mutex and modify shared data in order to return a result from the thread. With std::async and std::future this is not needed at all and very easy. Compare the following code to the condition variable example. It's more compact and more understandable:

#include <chrono>
#include <future>
#include <iostream>

struct Packet {
  int length;
  int width;
  int height;
};

Packet parcel_delivery() {
  Packet packet{1, 2, 3};
  std::this_thread::sleep_for(std::chrono::seconds(2));
  return packet;
}

int main(int argc, const char **argv) {
  std::future<Packet> future = std::async(std::launch::async, parcel_delivery);

  std::cout << "Wait for delivery" << std::endl;

  auto new_packet = future.get();
  std::cout << "Received new packet with dimension: " << new_packet.length
            << "; " << new_packet.width << "; " << new_packet.height
            << std::endl;
  return 0;
}

If you only want to wait for the result you can also use future.wait() instead of future.get(). Wait does not consume the result of the future in contrast to future.get().

Future and Promise pairs

Sometimes you cannot use std::async when you cannot set the future's value by a return-statement from the task running in a separate thread. This can be the case when you want to set the value two or more futures from one thread. In that case you can use a pair or multiple pairs of std::promise and std::future.

std::promise is the other side of the coin of an std::future. Whereas std::future behaves like a receiver, std::promise can be seen as the sender.

For using promise-future pairs, you first create a promise and use the get_future() function to retrieve the future. On the future you use get() or wait(). On the promise you use set_value(). Have a look on the example:

#include <chrono>
#include <future>
#include <iostream>
#include <thread>

struct Packet {
  int length;
  int width;
  int height;
};

void parcel_delivery(std::promise<Packet> &&promise) {
  Packet packet{1, 2, 3};
  std::this_thread::sleep_for(std::chrono::seconds(2));
  promise.set_value(packet);
}

int main(int argc, const char **argv) {

  std::promise<Packet> packet_promise;
  std::future<Packet> packet_future = packet_promise.get_future();
  std::thread delivery_thread(parcel_delivery, std::move(packet_promise));

  std::cout << "Wait for delivery" << std::endl;

  auto new_packet = packet_future.get();
  std::cout << "Received new packet with dimension: " << new_packet.length
            << "; " << new_packet.width << "; " << new_packet.height
            << std::endl;

  delivery_thread.join();

  return 0;
}

Note the difference to std::async before: Instead of returning the value from the task, now you pass a promise to the std::thread. Inside the thread you use set_value(..) to resolve the future.

When you start the thread you move the promise to the thread. Promises cannot be copied.

C++ Thread Synchronization Summary

We had a look on three different ways of C++ thread synchronization:

  1. The naive approach that better should not be used
  2. Condition variables, that are ideal for recurring events in combination with worker threads
  3. Futures with either std::async or std::promise (for one-shot events)

For simplicity reasons you can strive for std::async whenever it's possible. It's the simplest approach and least likely to introduce bugs. You don't need to use mutexes and locks with std::async.

If std::async cannot be used, use a condition variable as a synchronization mechanism. With condition variables make sure to use a predicate to test the condition to be protected against lost and spurious wakeups.

You may also like...