Groups 218 of 99+ julia-users › Waiting on multiple subprocesses to finish? 7 posts by 4 authors Elliot Saba 7 17 14 I was reading the docs, and it seems to me that it's saying I can use tasks to run multiple subprocesses at once. E.g., if I have some long-running subprocesses such as `sleep 10`, I should be able to wrap each in a Task and use the inherent wait command that running each subprocess would entail to switch to another task and kick off another subprocess. Is this correct? If it is, can someone provide me a quick example? I can't seem to get this to work, but I've never used Tasks before so that's hardly surprising. ; -E Tim Holy 7 17 14 Check out the implementation of multi-jl:pmap the part in the sync and async blocks , it's a great example. --Tim On Thursday, July 17, 2014 04:23:48 PM Elliot Saba wrote: I was reading the docs http: julia.readthedocs.org en latest manual control-flow tasks-and-event s , and it seems to me that it's saying I can use tasks to run multiple Kevin Squire 7 17 14 Note that the tasks all run in the same thread, so if any of the tasks or the main process have a long running computation which does not yield, the other processes won't run. I believe that `sleep 10` falls in this category, although I'm not 100 certain. Cheers, Kevin Elliot Saba 7 17 14 I'm trying to run shell commands in parallel. Shouldn't waiting for a process to finish just call wait which then invokes the scheduler? Or did I misread the above linked paragraph completely wrong? Specifically, I'm talking about the sections that say: The basic function for waiting for an event is wait. Several objects implement wait; for example, given a Process object, wait will wait for it to exit. In the next paragraph, it says: When a task calls wait on a Condition, the task is marked as non-runnable, added to the condition’s queue, and switches to the scheduler. The scheduler will then pick another task to run, or block waiting for external events. I guess I thought that block waiting for external events would only happen when there were no other tasks to run, but perhaps what's happening is that switching to other tasks only happens when I'm waiting on a condition constructed within Julia? -E Elliot Saba 7 17 14 Thanks, I'll take a look, although I have the gut feeling that these functions are targeted toward multiple processes. -E Jameson 7 17 14 you can also call `spawn` or `open` rather than `run` to run a process asynchronously in the current task it returns a process handle rather than a return value : function run cmds::AbstractCmd, args... ps spawn cmds, spawn_opts_inherit args... ... success ps ? nothing : pipeline_error ps end Elliot Saba 7 17 14 Great! That answers it, Jameson. Thanks!