RemoteRef leaks memory on remote worker #13626
Closed
Allardvm opened this Issue on Oct 15, 2015 · 2 comments
Projects
None yet
Labels
bug
parallel
Milestone
No milestone
Assignees
No one assigned
3 participants
@Allardvm
@amitmurthy
@tkelman
Notifications

You’re not receiving notifications from this thread.
@Allardvm
Allardvm commented on Oct 15, 2015

Using the constructor RemoteRef(f::Function, pid) to construct a reference to a Channel on pid (following the docs) seems to leak memory. The below code causes the remote worker to hold on to a large amount of garbage (>1 GB) that I have so far not been able to free.

addprocs(1)

@everywhere function foobar(c)
    put!(c, zeros(2^27))
    return nothing
end

c = RemoteRef(() -> Channel{Array{Float64,1}}(1), 2)
remotecall_fetch(2, foobar, c)
c = 0

@everywhere gc()
@everywhere gc()
@everywhere gc()

Calling remotecall_fetch(2, whos) before running the above code results in:

    From worker 2:                            Base  23543 KB     Module : Base
    From worker 2:                            Core   3039 KB     Module : Core
    From worker 2:                            Main  26349 KB     Module : Main

Calling it afterwards results in:

    From worker 2:                            Base 1072491 KB     Module : Base
    From worker 2:                            Core   3040 KB     Module : Core
    From worker 2:                            Main 1075246 KB     Module : Main
    From worker 2:                          foobar   1227 bytes  Function : footer

My versioninfo():

Julia Version 0.4.0
Commit 0ff703b* (2015-10-08 06:20 UTC)
Platform Info:
  System: Darwin (x86_64-apple-darwin13.4.0)
  CPU: Intel(R) Core(TM) i7-4850HQ CPU @ 2.30GHz
  WORD_SIZE: 64
  BLAS: libopenblas (USE64BITINT DYNAMIC_ARCH NO_AFFINITY Haswell)
  LAPACK: libopenblas64_
  LIBM: libopenlibm
  LLVM: libLLVM-3.3

@tkelman tkelman added the parallel label on Oct 15, 2015
@amitmurthy
The Julia Language member
amitmurthy commented on Oct 15, 2015

I think I found the issue.

Looks like https://github.com/JuliaLang/julia/blob/ac36fecbf6c42da47fdf5823729ecf2f7d087ed1/base/multi.jl#L612 is not being called when the RemoteRef whose exact type is RemoteRef{Channel{Array{Float64,1}}} is being deserialized.

Any idea on how to fix this? The deserialize method needs to be called for any type of RemoteRef
@amitmurthy amitmurthy added the bug label on Oct 15, 2015
@amitmurthy
The Julia Language member
amitmurthy commented on Oct 15, 2015

I think a definition of deserialize{T<:RemoteRef}(s::SerializationState, t::Type{T}) should work. Will test and submit a PR.
@amitmurthy amitmurthy referenced this issue on Oct 15, 2015
Merged
fix leak in RemoteRef deserialization #13630
@amitmurthy amitmurthy closed this in #13630 on Oct 15, 2015