Groups 158 of 99+ julia-users › VirtualArrays.jl 8 posts by 5 authors Samuel Massinon Jan 15 A way to concatenate arrays without copying values. https://github.com/invenia/VirtualArrays.jl Yichao Yu Jan 15 - show quoted text - FYI, don't call eval in functions Cedric St-Jean Jan 16 That's cool, a lazy vcat! I wrote a similar class that's a lazy `fill`, and it worked really well for my purposes. I wonder what the endgame will be for these datastructures. 0.5 will bring us "lazy slicing". Should everything be lazy by default? Does it scale? - show quoted text - Erik Schnetter Jan 16 There's probably a threshold: Below, it's cheaper to copy, and above, it's cheaper to re-use the existing arrays. LLVM goes to great length to optimize string operations in this respect, with their own string class ("Twine"). Obviously what works for strings can be extended to work with any kind of array. I'd argue that, once you go to parallel processing, keeping independent chunks of data instead of having everything in a single array is a clear performance benefit. So: Does it scale? Yes. Is it efficient? That depends on where you place the threshold. My guess is that it's somewhere near the L1 cache size. -erik - show quoted text - -- Erik Schnetter http://www.perimeterinstitute.ca/personal/eschnetter/ Cedric St-Jean Jan 16 Thanks for the Twine pointer, it was interesting. I wasn't thinking so much about CPU efficiency, but more so scaling code. There's a potential N² problem with lazy data structures, because binary operation should ideally be defined for all combinations of them to reap full benefits. Another issue is when/how to convert to Array. What should eg. `det(::VirtualArray)` look like? It should probably convert to a full Array first for best performance. Should it cache the Array in case another operation needs it? There seems to be an upper-bound on how many such lazy abstractions we can stack for any particular problem without making it too hard to reason about performance. OTOH, Haskell seems to manage with laziness, kind-of? But they don't have mutability, and that simplifies the problem. Cédric - show quoted text - Eric Davies Jan 18 On Friday, 15 January 2016 17:14:26 UTC-6, Yichao Yu wrote: FYI, don't call eval in functions What's wrong with calling eval in functions when you're evaluating expressions without side-effects? Yichao Yu Jan 18 - show quoted text - Orders of magnitude slower Not necessary Breaks type inference Eric Davies Jan 18 It's gone now :) https://github.com/invenia/VirtualArrays.jl/pull/2#event-518201512 - show quoted text -