The Error
If you have ever tried to speed up a Python script on Windows using the multiprocessing module, you might have hit a frustrating roadblock. Instead of faster code, you get a wall of text ending in a RuntimeError.
RuntimeError: An attempt has been made to start a new process before the current process has finished its bootstrapping phase.
This probably means that you are not using fork to start your child processes and you have forgotten to use the proper idiom in the main module:
if __name__ == '__main__':
freeze_support()
...
Why This Happens
This crash stems from a fundamental difference in how operating systems handle new processes. Linux and macOS typically use fork(). This creates a child process that is an identical clone of the parent, inheriting everything already in memory.
Windows lacks fork(). Instead, it uses a method called spawn. When you start a child process on Windows, Python launches a completely fresh interpreter. To get your functions and variables into this new instance, Python has to re-import your script from the top.
Here is where the logic loops. If your code to start a subprocess is sitting at the "top level" of your script, the child process will execute that same line when it imports the file. This triggers a second child process. That child then triggers a third. This recursive chain reaction continues until your CPU usage spikes and Python kills the script to prevent a total system freeze.
The Solution: The Main Guard
The fix is a simple structural change. You must wrap any code that kicks off a new process inside an if __name__ == '__main__': block. This conditional check ensures the code only runs when you execute the script directly, not when a child process imports it.
Incorrect Code (Triggers the Loop)
import multiprocessing
import time
def worker():
print("Worker started")
time.sleep(2)
# This line runs immediately upon import, causing recursion on Windows
p = multiprocessing.Process(target=worker)
p.start()
p.join()
Correct Code (The Fix)
import multiprocessing
import time
def worker():
print("Worker started")
time.sleep(2)
if __name__ == '__main__':
# This block is protected and only runs in the primary process
p = multiprocessing.Process(target=worker)
p.start()
p.join()
print("Worker finished")
Handling Pools and Queues
The same logic applies to multiprocessing.Pool. If you define a pool of 4 workers outside the main guard, Windows will try to spawn 4 new processes, each of which will try to spawn 4 more. Your script will crash almost instantly.
import multiprocessing
def square(n):
return n * n
if __name__ == '__main__':
# Explicitly defining process count is often safer
with multiprocessing.Pool(processes=4) as pool:
results = pool.map(square, [1, 2, 3, 4, 5])
print(f"Results: {results}")
How to Verify the Fix
Once you apply the guard, test your script using these three methods:
- **Terminal Execution:** Run `python your_script.py` in PowerShell. It should run to completion without throwing the bootstrapping error.
- **Monitor Task Manager:** Open Task Manager (Ctrl+Shift+Esc) while the script runs. If you set `processes=4`, you should see exactly five `python.exe` instances: one parent and four workers.
- **Check for Ghost Processes:** Ensure all Python processes disappear once the script finishes. If they linger, you may have a `join()` or synchronization issue.
Best Practices for Windows Users
I recommend following these habits to keep your parallel code stable:
- **Always use the guard:** Make `if __name__ == '__main__':` a default habit for every script. It prevents accidental side effects if you ever decide to import your functions into another project.
- **Keep the top level clean:** Only define constants, classes, and functions at the top of your file. Any "action" like printing or starting a pool belongs inside a function or the main guard.
- **Add freeze_support():** If you plan to turn your script into a standalone `.exe` using PyInstaller, call `multiprocessing.freeze_support()` as the very first line inside your main guard.