A common misconception about Python is that the GIL prevents any form of asynchronous code to be executed. For clarifying this, we’ll take a step back and look at what the GIL is and what asynchronous code means.
The global interpreter lock (GIL) 🔗
In CPython, the Python GIL is a mutex (short for mutual exclusion) that prevents two threads to access the same python resource at the same time. It’s been added because CPython’s memory management is not thread safe. I can hear you saying, “let’s fix that and remove it!”. It’s actually harder than it looks for a few reasons. The first one is that many standard and third party libraries relies on the GIL being there and would need to be fully reengineered. The second one is that removing the GIL while keeping the code simple and easily maintainable is challenging. Finally, getting rid of the GIL while preserving single threaded programs performance is also very hard.
The good news is that the GIL mostly impacts applications that actually spend a lot of time inside the GIL, interpreting CPython bytecode. In most applications, the blocking operations (e.g. I/O bound operations) or long running operations (e.g. image processing) actually happen outside the GIL and can run concurrently.
Synchronous vs asynchronous 🔗
OK, so now that we have a better view on what the GIL is, let’s take a look at what asynchronous code mean as opposed to synchronous code.
Let’s make it clear: asynchronous code does not mean multi-threaded. Said differently, single-threaded code can be asynchronous.
Synchronous code waits for a task to be finished before starting another one. Asynchronous code can start a new task while a task in progress hasn’t finished.
Single thread
# Synchronous
# B starts when A is finished
thread 1: [<--- A --->][<--- B --->]
# Asynchronous
# B starts when A is still in progress
thread 1: [<--- A][<--- B][A --->][B --->]
# Note that the example above make asynchronous code look slower
# than synchronous code but identifying which one is faster or
# slower actually depends on your specific use case.
Multi threads
# Synchronous
# B starts when A is finished
thread 1: [<--- A --->]
thread 2: [<--- B --->]
# Asynchronous
# B starts when A is still in progress
thread 1: [<--- A --->]
thread 2: [<--- B --->]
Hopefully, we now have a clearer view on the fact that asynchronous code can still run on single-threaded programs.
Concurrent vs parallel 🔗
Within asynchronous programming, we can disinguish concurrent and parallel processing.
Concurrent processing means that two or more tasks can be in progress at the same time. Given two tasks A and B, this means that B doesn’t need A to be finished to start. However, this means that when B is executing – given the lock – A can’t be executing and vice versa.
Parallel processing means that two or more tasks can be executed simultaneously. B doesn’t need A to release the lock before being able to execute.
While parallelism can take full benefit of hardware capabilities (e.g. multi processors, multi cores etc.) it also brings in memory and resources access challenges. Concurrency is often easier to manage and can still bring benefits, especially for I/O blocking tasks such as database access tasks.
Multi-threading vs multi-processing 🔗
Since we started discussing the challenges brought by the GIL, we’ve only talked about multi-threading.
Threads share the same memory space and resources and are then limited by the GIL. On the plus side, threads are lightweight and are faster to create than processes. During I/O blocking tasks, the CPU cores can context switch and progress through other operations but cannot run them in parallel.
Processes each get their own isolated memory space, resources and Python interpreter. This means that spawning processes is more costly than creating threads but it does take you away from the limitations of the GIL to fully leverage multiple processors or multiple cores on a given machine.
In Python, multi-processing is then recommended for CPU bound tasks while multi-threading is preferred for I/O bound tasks.
Conclusion 🔗
In this post, we’ve described the basics of asynchronous code in general and asynchronous code in Python. Now that we have a clearer picture (even at the high level) of the GIL limitations, we’ll have a look at the different Python libraries for delivering asynchronous code. But let’s take a break, rehydrate a little bit and look at that in a future post.