Groups


10 of 99+  



julia-users ›
copy a local variable to all parallel workers
7 posts by 5 authors  



Gabor 	

10/5/15


I would like to copy a local variable to all parallel workers.

I thought that the following code would do the job:
local_var=123
@everywhere var=local_var


But I get the following error:

ERROR: On worker 2:
UndefVarError: local_var not defined
 in eval at sysimg.jl:14
 in anonymous at multi.jl:1350
 in anonymous at multi.jl:892
 in run_work_thunk at multi.jl:645
 [inlined code] from multi.jl:892
 in anonymous at task.jl:63
 in remotecall_fetch at multi.jl:731

...and 2 other exceptions.

 in sync_end at task.jl:413
 in anonymous at multi.jl:1361


I ask for your help.
What is the simplest solution for this simple problem?
 


Jameson 	

10/5/15


despite your naming convention, it looks like "local_var" was a global. try wrapping it in a let block:

let local_var=123
@everywhere var=$local_var
end
- show quoted text -
 


Seth 	

10/5/15


As a related aside, what's the significance of the "$" in front of "local_var"?
- show quoted text -
 


Gabor 	

10/5/15


Thank you very much for your help!
Even better, your solution also works within functions.
I am with Seth with his question. 
Could you explain the significance of $ in front of "local_var" ?
- show quoted text -
 


Andre Bieler 	

10/31/15


I have a similar question about getting data to all workers. Consider the following code:

'''
addprocs(2)
using pfuncs

#=
@everywhere function showData()
  @show(data)
end
=#

function pstep()
  @sync begin
    for p in workers()
        @async begin
            remotecall_fetch(p, showData)
        end
    end
  end
end

@everywhere data = myid()
pstep()
'''

If I uncomment the part where the showData() function is defined, it works as expected. If I put the definition of showData() into the file pfuncs.jl and import it as in the example above, it does not work.

From the manual I figure

"using DummyModule causes the module to be loaded on all processes; however, the module is brought into scope only on the one executing the statement."

addresses the problem, but does not help me solve it.
I also tried the let block posted by Jameson.

How do I get the same behaviour as with the @everywhere function definition in the commented block but being able to use modules?

For completeness here is the pfuncs.jl module:

'''
module pfuncs
  export showData

function showData()
  @show(data)
end

end
'''



 


Jonathan Malmaud 	

10/31/15


The method 'pfunc.showData' isn't able to see the variable `data`, which exists as a global variable in the worker's 'Main' module, not it's 'pfuncs' module. 
- show quoted text -
 


Andre Bieler 	

11/1/15


Thanks for the clarification.

How do I make it available to pfunc.showData then?
(without having to explicitly send it as an argument in the function call)

The goal would be to have a large chunk of data having locally available to the workers
which then can independently work on it and only send the computed results.
(I cannot do shared arrays as the actual data will not be bit types)

Thanks