Groups   188 of 99+      julia-users › How to launch a Julia cluster under Torque scheduler 5 posts by 2 authors      David Parks    Jun 3   I've been reading various discussions on here about launching a Julia cluster.   It's hard to tell how current they are, and the documentation is a bit lacking on how to launch clusters under environments such as Torque. I'm using Julia 0.4.5  e.g. no cookies   The primary approach that is documented is for the master Julia process to `addprocs` and ssh to the workers in the cluster.  But this seem to conflict with the torque model, where torque launches a script under each assigned host. When those scripts exit the worker process are considered finished  as far as I understand .  So it would make most sense to let torque launch the Julia workers, and have those workers connect back to a designated master.  Is that straight forward to do with 0.4.5?   I see --worker, and I thought --bind-to would allow me to specify the master host, but I don't see a way to tell the master that it should listen for workers. This is about where the documentation ends.       Erik Schnetter    Jun 3   David  One way to go would be to use the MPI package, and to use mpirun to start the Julia processes. In this way, all processes are started automatically, and you do not need to run addprocs. The example  06-cman-transport-jl  in the MPI package shows how to use this setup.  -erik  --  Erik Schnetter  schn... gmail.com  http:  www.perimeterinstitute.ca personal eschnetter      David Parks    Jun 12   Nice idea Erik, I appreciate it! Though if I'm not wrong this locks me into only using MPI as the only transport mechanism and I want to use julias remotecall and other built in functionality rather than just MPI constructs.  I see that the latest documentation for julia shows a --bind-to option that looks like it allows you to launch remote processes and have them connect back to the master rather than the other way around. The documentation was a little unspecific, so I wasn't completely sure this is how it works, but that would be perfect if it did.   If anyone has any information on this new functionality I'd love to hear more details about it.  David        Erik Schnetter    Jun 14     You can use the MPI package while still using TCP  i.e. ssh  as transport mechanism. See the `test_cman_tcp-jl` test case.  -erik  --  Erik Schnetter  schn... gmail.com  http:  www.perimeterinstitute.ca personal eschnetter      David Parks    Jun 20   I meant that the MPI package doesn't create standard Julia workers, and hence doesn't allow you to using things like `remotecall ... ` and other julia-specific parallel constructs. By using the MPI package you are restricted to using the constructs supplied in the MPI package  or so it appears to me .  I ended up playing with the JuliaManagers package, which deploys to a cluster, but the PBSManager there was lacking some features I needed and I ended up writing a new ClusterManager for Torque that supports deploying workers using qsub.