Groups 137 of 99+ julia-users › A Very Simple Benchmark for Brutal-force Loops in Several Languages: revised, Julia is fast! 35 posts by 15 authors Zhong Pan Jul 11 Hi, Sorry I have to post a revision so quickly. Just after I posted my previous benchmark, I found the message on the top advising against using global variables when running performance test. I realized I did exactly that, so I went back to redo my test wrapped in a function. And that did the magic! The difference is a ~18x speedup! I didn't know how to recall my previous post, so I have to post a revised one (attached PDF) here. Since there is no surprise finding about Julia anymore, the benchmark results are not that interesting now; but I have to make sure the information I posted is not misleading, so here you go. Cheers, -Zhong Attachments (1) Simple_Benchmark_For_Brutal-force_Loops_Revised.pdf 73 KB View Download Mosè Giordano Jul 11 Hi Zhong, you may want to check out Julia 0.5, on my box your benchmark is ~13% faster with Julia 0.5: 3.367658 seconds with Julia 0.4.6 and 2.898068 seconds with Julia 0.5. Bye, Mosè - show quoted text - Andreas Lobinger Jul 11 2 small things: * a more recent Matlab should already be faster, especially in this loop thing * random generators' runtime -depending on the complexity they spend- really makes a difference. Zhong Pan Jul 11 Thanks! I will check out 0.5. It's amazing it can even be faster since it's already a bit faster than C++. :-) -Zhong - show quoted text - Zhong Pan Jul 11 Hi Andreas, Thanks for the comments. * If someone has a more recent Matlab it'll be interesting to try. The license is so expensive and I don't have access to newer version now. * Yes you are right, I also realized that I don't know how much the random number generator implementation difference would contribute. One thing to try is to leave out the random number generations. I tried it and here's the result: Python 166.44 sec (107.4x, was 64.3x), Julia 2.56 sec (1.7x, was 0.8x), VC++ 1.55 sec (1.0x as reference), C#.NET 3.49 sec (2.3x, was 1.1x), Java 10.14 sec (6.5x, was 3.0x), and Matlab 7.75 sec (5.0x, was 3.3x). Therefore, it seems VC++ improved the most by removing random number generations, and other languages just all look relatively 1.5 to 2.2 times more slower. Julia is still the fastest aside from VC++, and C#.NET is still not far behind. Cheers, -Zhong - show quoted text - David Barton Jul 11 For reference, with Matlab 2016a: 4.97 sec; Julia 0.4.6: 2.76 sec; Python 3.5.1: 166.76 sec. Note that there is a mistake in your Matlab code - zeros(n) returns an n by n matrix of zeros (hence running out of memory). Instead you want zeros(1, n) to get a vector. David - show quoted text - Alexander Ranaldi Jul 11 I am not sure calling the rand function in a loop is fair; you're benchmarking that function, not the mathematics. Perhaps allocate a vector of random numbers and index them rather than calling rand repeatedly. - show quoted text - Zhong Pan Jul 11 David, Thanks for the test results and the correction - now I recall how it's done in Matlab. Haven't been using it for a while. :-) -Zhong - show quoted text - Zhong Pan Jul 11 Alexander, You are right. I have posted some results in a previous reply with the calls to random number generator left out. Relatively speaking, C++ execution time is reduced more significantly than others, making it the fastest in that test. -Zhong - show quoted text - Zhong Pan Jul 11 Thanks to all the helpful comments. Just for the sake of tidiness, I attached rev 3 of the (now even simpler) benchmark result PDF. Changes are: * Removed random number generation as it was pointed out that random generators are relatively complex, and difference in implementation can lead to unfairly large impact on the test results. * Fixed the bug in Matlab code that had caused memory overflow. Now codes in all languages run the same number of loops for same size arrays. * I found if I wrap the Python code in a function, too, the performance is also improved, although just slightly and far less dramatic comparing to what happened to me in Julia. I think I got what I am looking for in this simple test. Look forward to using Julia a lot and enjoying the experience. -Zhong Attachments (1) Simple_Benchmark_For_Brutal-force_Loops_rev3.pdf 65 KB View Download Sisyphuss Jul 11 It's surprising to see Python so slow and Matlab so fast. Python: maybe it will get much faster when using comprehension? Matlab: does the JIT compiler translate the loops to C? - show quoted text - Andreas Lobinger Jul 11 On Monday, July 11, 2016 at 6:38:13 PM UTC+2, Sisyphuss wrote: It's surprising to see Python so slow and Matlab so fast. Matlab: does the JIT compiler translate the loops to C? Not to C, rather directly to machine code. LLVM seems to be in use here, also. Chris Rackauckas Jul 11 You should add @inbounds and try adding @fastmath to the Julia code. Maybe @simd, though the compiler should be doing that automatically. Make sure Julia is compiling with -O3. I wouldn't be surprised if this gets nearly to C++. If you want to put the random number generation back in, you can improve Julia's time by using ChunkedArrays.jl which allows you to take the random numbers in chunks instead of one at a time in a loop. That should get rid of most of the RNG time in Julia, but the code won't be any more bloated. I think this is one thing to show off in Julia: Julia types (/object) don't have speed penalties, so you can use them to make "fast versions" of just about anything. [Master branch is for Julia v0.5, last tagged version if for Julia v0.4.x. It allows for the random buffer to replenish using a parallel process, but that part had to change in the update] Other notes: MATLAB really improved their JIT in 2015b, but as you can see, it cannot come close to Julia. The main reason is that, although it can do quite a bit because it ensures type stability, it has type stability only because everything in MATLAB is by default a complex number. So even in the simplest example (where its JIT works correctly. You'll notice that in many cases it won't work, and since it doesn't cache well it may recompile all the time, and ... it works okay but there's a reason I switched to Julia) you'd expect Julia to be at least twice as fast. You should test how whether the performance regresses on Julia v0.5 with LLVM 3.7 - show quoted text - Chris Rackauckas Jul 11 Also: it's "brute force", not "brutal-force". The connotation is quite different :). - show quoted text - Zhong Pan Jul 12 Chris, Thanks for the comments! I tried @inbounds and @fastmath, which reduced Julia execution time from 2.55 sec to 2.46 sec. @simd wouldn't be appropriate here as the point is to test brute-force loops that in reality are employed in calculations that cannot be vectorized. (And yes, I agree I should have used "brute-force". :-)) -Zhong - show quoted text - Ferran Mazzanti Jul 12 ...but then is Julia *really* fast ? I guess that depends on what you compare with. I've been doing lots of numerics for the last 20+ years and for that I use fortran. This is *damn* fast, even more if you use something like the Intel compiler. Fortran many times performs better than C, and the benchmarks says that Julia is 1.6 times slower than C whichm froma performance point of view, is not a minor factor. So should I understand that Julia is about1.6-1.8 times slower than fortran? Then depending on what you call fast, Julia is fast :) Ferran Mazzanti Jul 12 btw, I use Julia also, and I really like it very much :) Andreas Lobinger Jul 12 just my two cents... On Tuesday, July 12, 2016 at 5:28:24 AM UTC+2, Chris Rackauckas wrote: MATLAB really improved their JIT in 2015b, but as you can see, it cannot come close to Julia. The main reason is that, although it can do quite a bit because it ensures type stability, it has type stability only because everything in MATLAB is by default a complex number. This is not correct, a variable is by default double. And extended to be complex if needed. But that's not the point. So even in the simplest example (where its JIT works correctly. You'll notice that in many cases it won't work, and since it doesn't cache well it may recompile all the time, and ... it works okay but there's a reason I switched to Julia) you'd expect Julia to be at least twice as fast. I think i have written this before: Julia enables you to write fast code, however it's not automatically faster at a constant factor compared to something. It's not like: matlab does it wrong, julia does it correct and therefore is magically faster. Some people on this mailing list (including myself) already had experiences with julia code running surprisingly slow or with a heavy memory impact (row-first vs. column-first arrays and temporary arrays and similar). Chris Rackauckas Jul 12 On Tuesday, July 12, 2016 at 5:57:27 AM UTC-7, Andreas Lobinger wrote: just my two cents... On Tuesday, July 12, 2016 at 5:28:24 AM UTC+2, Chris Rackauckas wrote: MATLAB really improved their JIT in 2015b, but as you can see, it cannot come close to Julia. The main reason is that, although it can do quite a bit because it ensures type stability, it has type stability only because everything in MATLAB is by default a complex number. This is not correct, a variable is by default double. And extended to be complex if needed. But that's not the point. It was originally all complex, but now everything is sort of double but can switch to complex at anytime. So all the standard numbers are not type-stable doubles in MATLAB which is one reason why you wouldn't ever expect it to be as efficient. It's a tradeoff between ease of use and performance where Julia took the performance choice. So even in the simplest example (where its JIT works correctly. You'll notice that in many cases it won't work, and since it doesn't cache well it may recompile all the time, and ... it works okay but there's a reason I switched to Julia) you'd expect Julia to be at least twice as fast. I think i have written this before: Julia enables you to write fast code, however it's not automatically faster at a constant factor compared to something. It's not like: matlab does it wrong, julia does it correct and therefore is magically faster. Some people on this mailing list (including myself) already had experiences with julia code running surprisingly slow or with a heavy memory impact (row-first vs. column-first arrays and temporary arrays and similar). It's not magically faster. It's obvious why it's faster: type-stability plus the ability to turn off all the extra "dynamic language helpers" like bounds checking. If you restrict yourself to very basic examples you will always see this is the case like the one the OP chose you will always see a speedup over MATLAB for these two reasons. "Surprisingly slow" code usually comes from type-instability, a slow library/package function, or bad programming (which sometimes can be easy to do). Type-instability is the biggest one: it can easily lead to orders of magnitude slower code. Choosing to iterate along a row in a column-major language is just a bad idea (and thus will slow down the MATLAB code for the same reason). Temporary arrays usually come from vectorized calls which aren't fused, which usually happens in MATLAB as well and is going to be largely fixed very soon, but is always fixable by re-writing the code as a loop. It's true that, as you say, if you ignore these facts and the other performance rules, your code will only be as fast as (or as slow as) other dynamic languages, and probably even be slower than something like MATLAB/Python. But I don't see how that applies to the case I mentioned: "the simplest example". In this case, you have type-stability, looped correctly through columns, etc., and your code will get close to C performance, or you read the manual wrong (or you found a bug and should open an issue). MATLAB's JIT can in some limited cases do almost as well. However, it's obvious why it's only a subset of cases: type-stability. f1(x) = sqrt(x) f2(x) = x^(-1) Those are 2 cases just off the top of my head where the equivalent MATLAB code using the JIT cannot be as fast as the Julia code simply because it cannot guarantee type stability in MATLAB. So if you have anything like this hanging around in a function, it will not do well. This isn't magic, it's a tradeoff between performance and ease of use (though you can make a new type in Julia called EasyFloat which just holds a float, and then make dispatches for each math function which are type-unstable in the same way as MATLAB. It wouldn't even be hard to do, and you'd no longer get InexactError() or DomainError() message, but it would be as slow as MATLAB). [Another case which MATLAB's JIT won't do well is the equivalent of f3(x) = sin.(cos.(log.(x))) due to temporary variables. Julia's will handle this well after @stevengj's PR (but you could always handle just by writing out the broadcast or the loop).] Zhong Pan Jul 12 Ferran, I can totally understand Fortran is still the king of fast and efficient numerical calculation in terms of computation time and memory usage. As to "fast", I am comparing Julia main to high-level languages like Python, R, and Matlab which offers similar level of fast prototyping. Since we are on this topic, I'd like to expand a little. Different users have different needs. As someone trained to be an engineer who can write some code, my desired go-to programming language should have the following: (1) Fast prototyping: I can get a one-time calculation done or try an idea in very short time - most probably I won't ever repeat it so I don't want to spend much time optimizing it. This requires the language to be simple, expressive, yet flexible and powerful. (2) Production ready: However, if something I tried is shown to be useful, I can go back and spend some time to improve it such that the code is good for production. "Improve" mainly means making it fast enough, efficient enough, reliable enough, and user friendly enough. Note the "enough" in every criteria - I won't try my best to make it as fast, efficient, reliable, and user friendly as possible, but only enough to satisfy the use case. (3) Decent speed for both vectorized calculations and brute-force loops in its native form: Some algos are easier to write and understand in vectorized form, others are easier in loops. I'd like to write an algo in the most natural way without frequently stopping to try fitting a square peg in a round hole. Also, even for fast prototyping, the data can be large, and I don't want to wait for hours or even days to see the result. (4) Vast collection of free, high-quality libraries/modules: I think everyone wants this. :-) (5) Open source, most libraries preferably in the same language: Though most of the time I trust other's library, once in a while I may bump into a problem that can only be solved if I can peek under the hood and even tweak a thing or two. It's not just about source code - I want to be able to run my application and trace into the library for debugging, as many subtle bugs can only be revealed when you watch things running. (6) Easy to collaborate: This requires the language to be simple to pick up - someone can learn enough to start using it in a couple days while learning more advanced features. Code written in it should be easy to read, and most of the time there shouldn't bee too many (obscure) ways to do the same thing. Also it needs to be free: it's hard enough to persuade someone else to learn a new programming language, not to mention persuading anyone to pay hundreds of dollars for using it. (7) Can scale up to a bigger project with many contributors: The language has enough protections so: (a) Compiler can check for mistakes as much as possible (with some help from the programmer), as run-time errors are annoying and costly; (b) Things can be modularized and collaborators won't easily step on each other's toes (style guide is nice but reinforcement is sometimes necessary). (8) Cross-platform: Windows and Linux support at the minimum. (9) Talk to other main-stream languages easily: There will always be a time when one really needs to use a library in C/C++/Fortran/Python/R or even Java/.NET that one desires to use. The wishlist is long and demanding. :-) For many of the above reasons I was a huge fan of Python. However, Python is way too slow at loops (Cython is not convenient as I often need to access objects and other libraries inside the loops); many errors can only show up in run-time (imagine running an algo for hours and then stop due to one type mismatch - ); and it's scarily easy to change class definition (Programmer A and B can both decide to add a member variable "n" to the object of the same class at run time! And sometimes this is done without the programmer even knowing it, as he/she may think the member variable already exists.); also a lot of powerful Python libraries are written in C/C++, so though they are open source they are effectively "black box" to me unless I am willing to invest a lot of time to dig into the gear box. So far I see Julia solving several major pain points in Python for me. I'd love to learn about it and use it everyday, and possibly persuade some friends to pick it up, too. :-) My main worry is would Julia is still a very young and somewhat specialized language with limited user base. I hope it will grow to surpass the critical mass for it to become main stream. Thanks, -Zhong On Tuesday, July 12, 2016 at 2:45:33 AM UTC-5, Ferran Mazzanti wrote: ...but then is Julia *really* fast ? I guess that depends on what you compare with. I've been doing lots of numerics for the last 20+ years and for that I use fortran. This is *damn* fast, even more if you use something like the Intel compiler. Fortran many times performs better than C, and the benchmarks says that Julia is 1.6 times slower than C whichm froma performance point of view, is not a minor factor. So should I understand that Julia is about1.6-1.8 times slower than fortran? Then depending on what you call fast, Julia is fast :) Jeffrey Sarnoff Jul 13 Zong, we would be pleased to increment the tally of Julia users with your use. - show quoted text - Ferran Mazzanti Jul 13 Dear Zhong, I understand your points very well... as I said I also love and use Julia. But at this point I understood that the *only* thing that was discussed was speed. And in that matter, a factor 1.6 is still a big difference. In my case I'm willing to use it instead of fortran right now, although 1 week calculation can turn into 2 weeks, which makes a difference. But for me, the speed competition is still won by other languages. Which are much worse in other fields, I agree :) Best, Ferran. dexto...@gmail.com Jul 25 Just for the sake of comprehensiveness, I ran your Python benchmark through the Numba JIT library (which uses the same underlying LLVM infrastructure that Julia does) and on my computer the Python code is faster than Julia by 68%. Vanilla CPython is terrible for this kind of simple explicit loop code, but Numba and other JIT libraries largely solve that issue with minimal effort as long as the code is simple enough. That by no means solves all of Python's issues in the context of numerical programming and I'm sure the Julia benchmark could be improved as others have already mentioned, but benchmarking Python this way isn't necessarily representative of how a performance-conscious programmer would reasonably approach a problem of this kind. dnm Jul 25 Interesting. Did you use the updated Julia code? Have you done any comparisons between reading and writing Numba JIT classes and Julia types in tight loops? On Monday, July 25, 2016 at 10:41:48 AM UTC-4, dexto...@gmail.com wrote: Just for the sake of comprehensiveness, I ran your Python benchmark through the Numba JIT library (which uses the same underlying LLVM infrastructure that Julia does) and on my computer the Python code is faster than Julia by 68%. Vanilla CPython is terrible for this kind of simple explicit loop code, but Numba and other JIT libraries largely solve that issue with minimal effort as long as the code is simple enough. That by no means solves all of Python's issues in the context of numerical programming and I'm sure the Julia benchmark could be improved as others have already mentioned, but benchmarking Python this way isn't necessarily representative of how a performance-conscious programmer would reasonably approach a problem of this kind. dexto...@gmail.com Jul 25 I haven't done any systematic benchmarking since Numba introduced the ability to JIT compile entire classes. In my experience, very well written Julia code is usually equivalent or better (in cases when @simd is helpful) compared to Numba JIT'd code. The Python code is sometimes easier to write, since Numba takes care of everything, but it's a double edged sword - if you run into a case where Numba doesn't work well or at all, you're just out of luck. In my personal view, the availability of Numba and other libraries just means Python vs Julia performance comparisons aren't particularly relevant, you should pick the language you prefer, not because you think it's faster, whereas if absolute performance is the only metric, you have to resort to Fortran/C++ anyway. I'm writing most of my code in Julia because I prefer the type system and multiple dispatch to Python's OOP, but I don't think it's meaningfully faster as long as you're willing to use appropriate libraries. The one outlier is the fact that GPU programming is much easier in Python at the moment. Hopefully that will change soon, some of the progress in ArrayFire.jl and CUDA libraries is very promising. - show quoted text - Zhong Pan Jul 25 Agree that while raw speed is important, in most situations it wouldn't be the most important reason to choose one programming language over another. I came from the angle of an engineer in a small company. For myself, the main attraction of Julia was the easiness to achieve decent speed without making much explicit effort: that means what feels more natural vectorized will be vectorized, while what feels more natural in a loop will be in a loop; that means I don't need to resort to another language or a library only for improving speed; and that means apart from sticking to a couple good habits, I can use objects, functions etc. the same way inside a loop vs. outside. None of these is critical by itself, but they add up to an uninterrupted flow of thoughts while writing code to explore, try, fail, and retry, for many iterations. During this "less careful" prototyping, 1-2x slow down is fine, but with Julia I know I won't sit there for tens of minutes waiting for a result while debating myself whether I should rewrite it in C++ or rehaul the code with Cython etc.; instead I can rest assured that as long as my algo and coding have no mistakes or major flaws, the speed is close to what I will get even if I make several times more effort to rewrite it in C++. Another big deal for me is the resulted removal of the barrier between prototype and production code. For production I can review and improve my code carefully, but rewriting it in a less expressive language is too much. I was a huge fan of Python (heck I even persuaded my previous boss, a VP, to pick up Python - though I don't know if he really had time to finish it. :-)). However, the slow raw speed and the over-freedom to change class definition anywhere always gave me the itch to find something better. My brother at JPL who worked on Python projects also complained about having to think really hard to vectorize almost everything and then couldn't easily understand what he was doing a few months later because the code was too unnatural for the problem; the indentation was also a big headache as collaborators use different editors with different tab definitions. So I'm really happy to have found Julia, which gave me the same joy as coding in Python and removed the main itches. -Zhong Tim Holy Jul 25 Re: [julia-users] Re: A Very Simple Benchmark for Brutal-force Loops in Several Languages: revised, Julia is fast! Given the apparent interest in the topic and the decisions that people seem to be making, it seems worth pointing out that folks are still using apples-to- oranges comparisons on this benchmark. There are at least two important differences: - in the other languages, `linspace` allocates a vector, but in Julia it (currently) creates a compact object from which values are computed on-the-fly. That computation involves a division, and division is slow. - The languages like C aren't doing bounds-checking. You might imagine adding `@inbounds` to the Julia version. But `@inbounds` has no impact on LinSpace objects in julia 0.4. In julia 0.5, it does work like you'd expect (thanks, Blake Johnson). Combining these observations, we can `collect` the values into a `Vector` and then use `@inbounds`. For me the version below is nearly twice as fast as the original: function benchmark() nsamples = 1000000 x = collect(linspace(0, 5, nsamples)) y = zeros(nsamples) # attempt to trigger JIT to compile all functions needed in the loops # before profiling a = cos(0.0); a = 1.5*2.5; a = 1.5+2.5; println("\nBrutal-force loops, 100 times:") @time begin for m = 1:100 @inbounds for n = 1:nsamples y[n] = cos(2*x[n]+5); end end end end benchmark(); Best, --Tim - show quoted text - dexto...@gmail.com Jul 26 Re: [julia-users] Re: A Very Simple Benchmark for Brutal-force Loops in Several Languages: revised, Julia is fast! Since I contributed the Numba JIT timing earlier in the thread, it seems only fair to note that the modified Julia version with the properly preallocated data is now 17% faster than the Numba version on my computer. Overall, this seems to support my thesis that good Julia code is on par or slightly faster than the Python code using the JIT libraries (Numba, Hope, Parakeet, etc.) with the added benefit of not requiring external libraries. I've also seen some cases where the difference is even more pronounced in the favor of Julia because of the ability to use @simd to prod the LLVM vectorizer along and the ability to exert precise control over memory allocations, but this is highly use case specific. - show quoted text - hustf Jul 31 Re: [julia-users] Re: A Very Simple Benchmark for Brutal-force Loops in Several Languages: revised, Julia is fast! It is nice to have a little check on speed from time to time. I still use VBA for easy cooperation with less programming savvy colleguaes. Julia 1.17s. VBA (excel alt + f11): 12 s. This is a bit unfair to neolithic man Joel Spolsky since no optimization was performed: Sub benchmark() nsamples = 1000000 Dim y() As Double ReDim y(1 To nsamples) x = y For i = 1 To nsamples x(i) = (i - 1) * 5 / (nsamples - 1) Next Debug.Print ("\nBrutal-force loops, 100 times:") sngtime = Timer For m = 1 To 100 For n = 1 To nsamples y(n) = Cos(2 * x(n) + 5) Next Next Debug.Print Timer - sngtime End Sub Eric Forgy Aug 1 Re: [julia-users] Re: A Very Simple Benchmark for Brutal-force Loops in Several Languages: revised, Julia is fast! I mentioned to Prof. Edelman (only half jokingly) at an event in Singapore, that we should add Excel/VBA to the list of benchmarks. If I'm in a corporate setting and trying to sell Julia for some internal project, the person making the call has probably never heard of any of the languages in the Julia benchmark, but they have heard of Excel/VBA, so, as silly as it may seem, I actually think it could go a long way for Julia evangelists to see more comparisons to Excel/VBA. - show quoted text - Sisyphuss Aug 1 Re: [julia-users] Re: A Very Simple Benchmark for Brutal-force Loops in Several Languages: revised, Julia is fast! Julia is not as "mature" as VBA, which prevents "analysts" of large firms adopting it. In addition, they will be happier to continue using global variables. - show quoted text - Zhong Pan Aug 2 Re: [julia-users] Re: A Very Simple Benchmark for Brutal-force Loops in Several Languages: revised, Julia is fast! Eric, hustf, I think making Julia attractive to Excel/VBA users will be quite valuable. Excel still rules in business world for simple to moderately complex data analysis. Strangely, even engineers love it - there is still a large group of hardware/mechanical engineers who are not productive in a general purpose programming language, but they love Excel for its simplicity, programmability, and visual appeal (who doesn't like cells with dizzying colors and fancy fonts :-)). And I know some engineers who can program really well actually wrote quite sophisticated VBA macros inside Excel so the resulted sheet can be used intuitively by a non-programmer (e.g. a manufacturing staff or a field support) while saving the trouble of developing an independent GUI which will take an extra head count. While Julia will certainly win the speed contest, I don't think Julia should or could replace Excel/VBA. Maybe a more practical and gentle approach is to make Julia conveniently available from inside Excel/VBA, similar to what RExcel does? When the complexity or computational load run out of VBA's capability, the Exceler can trust Julia to get the job done, fast. -Zhong - show quoted text - Stefan Karpinski Aug 3 Re: [julia-users] Re: A Very Simple Benchmark for Brutal-force Loops in Several Languages: revised, Julia is fast! Julia Computing offers a product (JuliaInXL) which does exactly this. - show quoted text - hustf Aug 3 Re: [julia-users] Re: A Very Simple Benchmark for Brutal-force Loops in Several Languages: revised, Julia is fast! Zhong, people chop wood and they play the piano, too. Fill in Julia or Excel where you like. Sisyphuss, VBA show a 4% speed GAIN through making nsamples and y() public variables. Try it out! Zhong Pan Aug 4 Re: [julia-users] Re: A Very Simple Benchmark for Brutal-force Loops in Several Languages: revised, Julia is fast! Thanks, Stefan, it's good to know this already exists! Seeing it mentioned side-by-side with an Bloomberg API indicates that this feature request probably came from the financial industry. -Zhong - show quoted text -