option to set processor affinity in addprocs #10645
Closed
amitmurthy opened this Issue on Mar 26, 2015 · 7 comments
Projects
None yet
Labels
parallel
Milestone
No milestone
Assignees
No one assigned
4 participants
@amitmurthy
@ViralBShah
@jakebolewski
@ihnorton
Notifications

You’re not receiving notifications from this thread.
@amitmurthy
The Julia Language member
amitmurthy commented on Mar 26, 2015

The ability to lock julia workers to specific cores results in substantial gains for certain workloads. See #10427 (comment) for some test results. addprocs should provide a means for the same.
@ViralBShah
The Julia Language member
ViralBShah commented on Mar 27, 2015

I don't fully recollect, but there was some issue with openblas and affinity - we disable it by default. Do see the openblas build options, and the earlier discussion - I will try to find the relevant links.
@amitmurthy
The Julia Language member
amitmurthy commented on Mar 27, 2015

With threaded openblas it may not make much sense. But in a parallel run, openblas threads are set to 1.
@jakebolewski
The Julia Language member
jakebolewski commented on Mar 27, 2015

Wouldn't this be best handled by the cluster manager. SGE, Slurm, MPI, etc. all have options to do this.
@amitmurthy
The Julia Language member
amitmurthy commented on Mar 27, 2015
What about folks running on largish machines like julia.mit.edu? I think
the built in managers, LocalManager and SSHManager should support it.
…
@ViralBShah
The Julia Language member
ViralBShah commented on Mar 28, 2015

That is probably the most common use case.
@amitmurthy
The Julia Language member
amitmurthy commented on Mar 28, 2015

Another example - https://groups.google.com/d/msg/julia-dev/BYVCyUlNR8c/_9XGgjolNNIJ - a 192 core system.
@ihnorton ihnorton added the parallel label on Mar 28, 2015
@amitmurthy amitmurthy referenced this issue on Apr 3, 2015
Closed
support basic core affinities in LocalManager and SSHManager #10733
@amitmurthy
The Julia Language member
amitmurthy commented on Apr 30, 2015

Have added this feature to ClusterManagers.jl - JuliaParallel/ClusterManagers.jl@9afe252
@amitmurthy amitmurthy closed this on Apr 30, 2015