I use a ThresPoolExecutor
object to executes four thread with a function that receives several arguments that are used but are not changes. When I read my program's log file I can see that each new thread is executed only after the previous one is done (my machine has enough CPUs). I of course want them to run in parallel since they are completely independent of each other.
My code is:
with ThreadPoolExecutor(max_workers=4) as executor:
rows += executor.submit(concurrent_comparison, df, model, transformer, n, lock, set(s[0]), str(0)).result()
rows += executor.submit(concurrent_comparison, df, model, transformer, n, lock, set(s[1]), str(1)).result()
rows += executor.submit(concurrent_comparison, df, model, transformer, n, lock, set(s[2]), str(2)).result()
rows += executor.submit(concurrent_comparison, df, model, transformer, n, lock, set(s[3]), str(3)).result()