Groups


121 of 99+  



julia-users ›
Using @spawnat to define var on particular process
2 posts by 2 authors  



Alex Williams 	

May 17


Hi all,

I'm trying to figure out basic communication patterns for Julia's parallel computing framework. I'd like to write up a little tutorial when I'm done, since the documentation on this (in my opinion) is a bit sparse.

The basic question is why this doesn't define `x` on worker #2:

@spawnat 2 x=1



Meanwhile this seems to work:



@spawnat 2 eval(:(x=1))



And this seems to work if you want to define `x` on all procs:




@everywhere x=1



To be clear, by "working" I mean that running fetch(@spawnat 2 x) will return 2 only in the second two cases. The first case throws an error (`x` not defined on worker 2) .

I tried using macroexpand to see the difference. I don't have a good sense of what things like `:copyast` and `:localize` mean. If someone could help me parse this, I'd greatly appreciate it.

julia> macroexpand(:(@spawnat 2 x=1))

:(Base.spawnat(2,$(Expr(:localize, :(()->begin  # expr.jl, line 113:

           ()->begin  # multi.jl, line 1358:

                   x = 1

               end

       end)))))


julia> macroexpand(:(@spawnat 2 eval(:(x=1))))

:(Base.spawnat(2,$(Expr(:localize, :(()->begin  # expr.jl, line 113:

           ()->begin  # multi.jl, line 1358:

                   eval($(Expr(:copyast, :($(QuoteNode(:(x = 1)))))))

               end

       end), :eval))))


Apologies if there is a good explanation of this elsewhere on the web. You can just point me there if so.

-- Alex

 


Isaiah 	

May 19


On Mon, May 16, 2016 at 7:48 PM, Alex Williams <alex.h...@gmail.com> wrote:

The basic question is why this doesn't define `x` on worker #2:
@spawnat 2 x=1

Because it is wrapped in a closure; so `x` is local to the closure when the code is executed on the other process. To see the difference, try `@spawnat 2 global x = 1` then `@spawnat 2 names(Main)` 
Meanwhile this seems to work:

@spawnat 2 eval(:(x=1))

`eval` operates in global scope by default (you can optionally specify a module as the first argument).
 
And this seems to work if you want to define `x` on all procs:


@everywhere x=1

This calls `eval`, see above :) 

I tried using macroexpand to see the difference. I don't have a good sense of what things like `:copyast` and `:localize` mean. If someone could help me parse this, I'd greatly appreciate it.

- `copyast` makes a copy of a given AST, handling GC roots, etc. (see definition as `jl_copy_ast` in "src/ast.c").
- `Expr(:localize, expr)` encloses the expression in a let block containing an entry for each variable referenced in the expression (see the definition of `localize_vars` in "base/expr.jl").

Apologies if there is a good explanation of this elsewhere on the web. You can just point me there if so.

Chris Rackauckas has done some nice tutorials on parallelization recently, which might touch on some of this (check juliabloggers.com).

If you have any improvement suggestions for the Base documentation, those would be great (PRs welcome).

 
-- Alex