Groups


16 of 99+  



julia-users ›
SharedArray / parallel question
6 posts by 3 authors  



Christopher Alexander 	

Jan 29


Hello all, I have a question about the proper usage of SharedArray / @parallel.  I am trying to use it in a particular part of my code, but I am not getting the expected results (instead I am getting an array of zeros each time).

Here are the two functions that are involved:
function partial_rollback!(lattice::TreeLattice, asset::DiscretizedAsset, t::Float64)
  from = asset.common.time

  if QuantJulia.Math.is_close(from, t)
    return
  end

  iFrom = findfirst(lattice.tg.times .>= from)
  iTo = findfirst(lattice.tg.times .>= t)

  @simd for i = iFrom-1:-1:iTo
    newVals = step_back(lattice, i, asset.common.values)
    @inbounds asset.common.time = lattice.tg.times[i]
    asset.common.values = sdata(newVals)
    
    if i != iTo
      adjust_values!(asset)
    end
  end

  return asset
end

function step_back(lattice::TreeLattice, i::Int, vals::Vector{Float64})

  newVals = SharedArray(Float64, get_size(lattice.impl, i))
  @parallel for j = 1:length(newVals)
    val = 0.0
    for l = 1:lattice.n
      val += probability(lattice.impl, i, j, l) * vals[descendant(lattice.impl, i, j, l)]
    end
    val *= discount(lattice.impl, i, j)
    newVals[j] = val
  end
  retArray = sdata(newVals)
  
  return retArray
end

Is that to much complexity in the parallel loop?  Right now the max # of times I've seen over this loop is in the 9000+ range, so that's why I thought it would be better than pmap.
Any suggestions?

Thanks!

Chris
 


Christopher Alexander 	

Jan 29


I tried using @sync @parallel and ended up getting a segmentation fault, so I'm not really sure how to parallelize this loop.  This is inside of a larger module, so I'm not sure if something special has to be done (e.g. putting @everywhere somewhere).  I've searched this forum and the documentation, which is where I got the idea to add @sync.  I'd appreciate any input, as I'm kind of stuck.  The larger code in which this parallelization should take place is here:
https://github.com/pazzo83/QuantJulia.jl/blob/master/src/methods/lattice.jl

Thanks!

Chris
- show quoted text -
 


Lutfullah Tomak 	

Jan 30


There is this issue on on github https://github.com/JuliaLang/julia/issues/14764 . I am no expert about parallel computing but may be related.
Regards

 


Christopher Alexander 	

Jan 30


I can confirm that I do not see the seg fault issue in 0.5 (the latest master), but I am getting fundamentally different results when using the @sync @parallel construction.  In essence, @sync @parallel is causing arrays of different values (compared to using a non-parallelized construction) to be built, which is causing an issue further along in my program.  It is also much slower, so I am wondering if I my syntax is incorrect.

Any input would be appreciated.  You can see what is supposed to be generated by loading this script (https://github.com/pazzo83/QuantJulia.jl/blob/master/swaption_test_code.jl) and calling main().

Thanks!


On Saturday, January 30, 2016 at 4:48:51 AM UTC-5, Lutfullah Tomak wrote:
There is this issue on on github https://github.com/JuliaLang/julia/issues/14764 . I am no expert about parallel computing but may be related.
Regards

 


Christopher Alexander 	

Jan 31


Is this something that would really only be solved by proper multi-threading?
- show quoted text -
 


ele...@gmail.com 	

Jan 31


Are you absolutely sure that, in the part you are trying to parallelize, no computation depends on the results of a previous computation.  If those two computations are moved from serial to parallel then there is no guarantee that the first is complete before the second happens on a different process.   Subtle situations include using the values from cells in an array (other than the one you are about to update).  Those other cells may or may not be updated if the update code is in another process.
- show quoted text -