Groups   115 of 99+      julia-users › Code runs 500 times slower under 0.4.0 20 posts by 12 authors      Kris De Meyer    10 22 15   Are there any general style guidelines for moving code from 0.3.11 to 0.4.0? Running the unit and functionality tests for a module that I developed under 0.3.11 in 0.4, I experience a 500 times slowdown of blocks of code that I time with  time.   Can't even imagine where I have to start looking, and find it flabbergasting that perfectly valid julia code under 0.3.11  not generating a single warning  can show such a performance degradation under 0.4.0.  Anyone seen anything similar? Is there some fundamental difference in how code is JIT-compiled under 0.4.0?  Thanks,  Kris        Stefan Karpinski    10 22 15   Are there any deprecation warnings? Deprecated calls are pretty slow.      Gunnar Farnebäck    10 22 15   If you don't have deprecation warnings I would suspect some change in 0.4 has introduced type instabilities. If you are using typed concatenations you could be hit by https:  github.com JuliaLang julia issues 13254.      Kristoffer Carlsson    10 22 15   Just run ProfileView and see what is taking time. Then post the relevant code here and it can likely be fixed.     Stefan Karpinski    10 22 15  Re:  julia-users  Re: Code runs 500 times slower under 0.4.0  You can try using  code_warntype to see if there are type instabilities.      Kris De Meyer    10 23 15  Re:  julia-users  Re: Code runs 500 times slower under 0.4.0  Thanks for the tips. There are no depreciation warnings in my own code, but the Sundials package which I rely on has quite a few. Although the tests which are really slow don't print out depreciation warnings  the printed warnings occur earlier , I suppose depreciation checking can still be slowing things down, even if no warnings are printed anymore? I guess I'll just have to wait to see if Sundials can catch up  I see it's failing to pass tests under 0.4.0 . This is very annoying as I wanted to move up from 0.3.11 because of problems in the PyCall PyPlot Conda packages that I don't seem to find a solution for in 0.3.11 but have been told are resolved in 0.4.0.  Please accept the following not as unbridled criticism but as a way to improve working procedures. I've been developing Julia code since release v0.3.7. Unfortunately this isn't the first time that I lose many hours having to figure out why something that works in one release stops working in a subsequent one. To be honest, it's putting me off Julia and it must have similar effects on other potential users too. To me this points to the need for better procedures and guidelines in the way the language progresses and how the  official  packages catch up. I worked a couple of years for the MathWorks. Breaking backwards compatibility was generally not allowed, and the coordinated testing and release procedures made that nearly impossible. For Julia to be taken seriously by people outside the academic community, it would do well to start looking at how similar procedures can be adopted into an open-source development model. It's one thing to write good code and develop the  ideal  language, it's another thing altogether to release workable software to an outside community.   On a slightly different note, in 2 or 3 release cycles, Matlab will have caught up on any performance gains Julia may have introduced  by using the same LLVM compiler procedures Julia uses  and then the only thing Julia will have going for it is that it's free. But my cost to my employers is such that if I lose as little as 3 days a year on compatibility issues, they would be better off paying for a Matlab license...  Best.  Kris      Stefan Karpinski    10 23 15  Re:  julia-users  Re: Code runs 500 times slower under 0.4.0  On Fri, Oct 23, 2015 at 7:18 AM, Kris De Meyer  kr... aniku.co.uk  wrote: Thanks for the tips. There are no depreciation warnings in my own code, but the Sundials package which I rely on has quite a few. Although the tests which are really slow don't print out depreciation warnings  the printed warnings occur earlier , I suppose depreciation checking can still be slowing things down, even if no warnings are printed anymore? I guess I'll just have to wait to see if Sundials can catch up  I see it's failing to pass tests under 0.4.0 . This is very annoying as I wanted to move up from 0.3.11 because of problems in the PyCall PyPlot Conda packages that I don't seem to find a solution for in 0.3.11 but have been told are resolved in 0.4.0.  Sorry for the annoyance, the situation is quite frustrating. Sundials should get updated soon.   Please accept the following not as unbridled criticism but as a way to improve working procedures. I've been developing Julia code since release v0.3.7. Unfortunately this isn't the first time that I lose many hours having to figure out why something that works in one release stops working in a subsequent one. To be honest, it's putting me off Julia and it must have similar effects on other potential users too. To me this points to the need for better procedures and guidelines in the way the language progresses and how the  official  packages catch up. I worked a couple of years for the MathWorks. Breaking backwards compatibility was generally not allowed, and the coordinated testing and release procedures made that nearly impossible. For Julia to be taken seriously by people outside the academic community, it would do well to start looking at how similar procedures can be adopted into an open-source development model. It's one thing to write good code and develop the  ideal  language, it's another thing altogether to release workable software to an outside community.   This is a real issue. Julia is not yet at the point where prohibiting backwards incompatible changes is realistic. But we're getting there – in another year or so, after 1.0 is release, that will change. This is also one of the reasons we founded Julia Computing – one of our first products is a supported Julia distribution, including a set of packages that will be updated in sync. We will support these versions for years, including making sure supported packages continue to work and get bug fixes.  On a slightly different note, in 2 or 3 release cycles, Matlab will have caught up on any performance gains Julia may have introduced  by using the same LLVM compiler procedures Julia uses  and then the only thing Julia will have going for it is that it's free. But my cost to my employers is such that if I lose as little as 3 days a year on compatibility issues, they would be better off paying for a Matlab license...  Matlab has made good progress with their JIT performance recently – probably in no small part due to Julia's existence. I suspect, however, that most of the low hanging performance fruit has already been picked: going from slow to decently fast is relatively easy compared to going from decently fast to really fast. So I don't think you can necessarily extrapolate that rate of performance improvement into the future. Yes, Matlab uses LLVM which Julia also uses, but that doesn't mean it can do the same things – the language designs are very different.  The notion of a closing gap between Matlab and Julia also supposes that Julia is standing still, which is far from true. Julia is currently gaining language-level multithreading support  as compared to having a few kernels that are written in Clanguage to use threads . Soon we'll have the ability to statically compile Julia programs without any dependency on LLVM – you will get a standalone binary that only depends on a fairly small Julia runtime library  i.e. no JIT . There's a lot of work being done on distributed parallel computing as well – the current state of affairs is somewhat usable but not nearly what we've envisioned for it. And of course we've only begun to touch the kinds of fancy performance optimizations that can be done – compared to, say, the kinds of stuff that's done by JavaScript JITs. So even if Matlab catches up to where we are now, by they time that happens, Julia will be way off in the distance, speeding into the future :-      Tony Kelman    10 23 15  Re:  julia-users  Re: Code runs 500 times slower under 0.4.0     I've been developing Julia code since release v0.3.7. Unfortunately this isn't the first time that I lose many hours having to figure out why something that works in one release stops working in a subsequent one.  This surprises me. 0.4.0 is the only release that was intended to be breaking in any meaningful way since 0.3.0. Every point release announcement has said  We would like to get feedback if someone has a working program that breaks after this upgrade.  I meant that seriously, if an unintended breaking change was made in any of the 0.3.x series and an issue was reported about it, we would find a way to fix it  for example, there was a problem in the embedding API that led us to put out 0.3.5 on an accelerated schedule to fix it . The same holds for 0.4.x relative to 0.4.0.     Stefan Karpinski    10 23 15  Re:  julia-users  Re: Code runs 500 times slower under 0.4.0  Yes, my experience with upgrading large code bases to new point releases is that no changes are required. Kris, if your experience with point releases has been otherwise, it would be good to hear about specific issues since those releases are strictly bug fix releases. Perhaps you are talking about the last major upgrade from 0.2.x. to 0.3.x?      Patrick O'Leary    10 23 15  Re:  julia-users  Re: Code runs 500 times slower under 0.4.0  On Friday, October 23, 2015 at 6:18:43 AM UTC-5, Kris De Meyer wrote: ...and then the only thing Julia will have going for it is that it's free. But my cost to my employers is such that if I lose as little as 3 days a year on compatibility issues, they would be better off paying for a Matlab license...   It's not quite that simple--if I have a problem with something in MATLAB, I file a bug report. The issue may  or may not  be fixed in 3-6 months, with the next release. If I'm lucky I catch problems in prereleases--but that's also time I have to spend to install the prerelease and test the fragile parts of the process. This isn't hypothetical; we've skipped entire releases which have variously broken code generation.  Full disclosure, we haven't seen such problems with MATLAB proper, but deployment with Coder is essential for what we do.   But if something broke in Julia, there's a chance I can fix it. Free software matters.  Patrick     Jonathan Malmaud    10 23 15  Re:  julia-users  Re: Code runs 500 times slower under 0.4.0  If the only thing that keeps you on Julia is the JIT for loops where everything has straightforward types like Array Float64 , there are already tons of choices: PyPy, Numba, Cython, Luajit, and V8 will all JIT that kind of code decently.   But Julia has that kind of speed while also having a rich type system for generic programming, macros, and  other well-thought-out semantics due to being designed by scratch. I doubt Matlab is very likely to add those features going for, both for backwards-compatibility reasons and since it doesn't like a priority for Mathworks  which isn't to impinge on the technical ability of Mathworks engineers at all - of course they could if they wanted to .      Andreas Lobinger    10 23 15  Re:  julia-users  Re: Code runs 500 times slower under 0.4.0   On Friday, October 23, 2015 at 3:06:13 PM UTC+2, Patrick O'Leary wrote: On Friday, October 23, 2015 at 6:18:43 AM UTC-5, Kris De Meyer wrote: ...and then the only thing Julia will have going for it is that it's free. But my cost to my employers is such that if I lose as little as 3 days a year on compatibility issues, they would be better off paying for a Matlab license...    But if something broke in Julia, there's a chance I can fix it. Free software matters.  Well, you can. But the average julia USER is not in the same position. Fixing something in packages needs some background in development of the package and fixing something in julia itself needs even more background.  In any case, i prefer Open Source because there's even the possibility to look inside and see how the magic is happening.  But the argument, 'you can help yourself' is not equal to 'you are able to help yourself'.       Patrick O'Leary    10 23 15  Re:  julia-users  Re: Code runs 500 times slower under 0.4.0    I didn't intend for the word  chance  to be read as anything other than  nonzero probability.       Kris De Meyer    10 23 15  Re:  julia-users  Re: Code runs 500 times slower under 0.4.0  Thanks for all the information.   The problem is probably not with the language releases itself  I take your word for it that point releases don't introduce incompatibilities  but with the package releases. Sometimes they lag behind  the language release  as I'm currently experiencing with Sundials  and sometimes they may well pass individual tests but then fail on real-world package dependencies which aren't noticed because they are not covered by tests. To give an example of the latter, a few weeks ago, when I started to test a major code re-write of my own, I got a set of  convert  errors. After digging around fruitlessly in my own code for several hours  the obscurity of the message didn't help , I finally discovered it was an error triggered by loading one package before another  forgot now which one it was, I think Colors-jl was involved . A third problem is that by depending on Python, some packages introduce all of the nightmares  and then some  of the multiverse of Python releases, especially on Linux  which is the cause of the error I'm experiencing with a 0.3.11 Julia IJulia Conda installation on Linux at the moment . A fourth problem is that, despite the assertion that I've seen that for-loops over Arrays should run at the same speed as vectorized code, I've found on several occasions that for loops are still quite a bit slower. It may be that I'm not writing it in the most optimal way but as far as I know I am following the documentation and guidelines that exist.  I know that all of these problems are solvable, but that's not the point I am trying to make. All of these issues take time that keep me away from writing the software that I am hired to write in a limited number of days, and my employers may not be interested in the reasons for why I can't get it to work on time. Please also note that when in the midst of a crisis and up against deadlines, I  and probably other users in similar situations  may not have time to file bug reports, and not even have the time to note down how I fixed or circumvented a certain problem.  I understand that my criticism may sound misplaced because I know that Julia is in early development, but as a  real  user of Julia  not involved in the development itself, but attempting to write real production code, rather than using it as an academic exercise , I'd rather let you know of my experience of using it.   Thanks! And keep up the good work. Free software matters indeed - though so does paid-for.      Andrew    10 23 15  Re:  julia-users  Re: Code runs 500 times slower under 0.4.0  I don't think performance is the only thing Julia has going for it over Matlab. Julia has multiple dispatch, a sophisticated type system, macros, and functions can modify arrays and other mutable objects. I'm unaware of any plan for Matlab to add these things--they would be major changes and possibly very confusing for long-time users.   On Friday, October 23, 2015 at 7:18:43 AM UTC-4, Kris De Meyer wrote: On a slightly different note, in 2 or 3 release cycles, Matlab will have caught up on any performance gains Julia may have introduced  by using the same LLVM compiler procedures Julia uses  and then the only thing Julia will have going for it is that it's free. But my cost to my employers is such that if I lose as little as 3 days a year on compatibility issues, they would be better off paying for a Matlab license...  Best.  Kris         On Thursday, October 22, 2015 at 10:58:30 PM UTC+1, Stefan Karpinski wrote: You can try using  code_warntype to see if there are type instabilities.  On Thu, Oct 22, 2015 at 5:50 PM, Gunnar Farnebäck  gun... lysator.liu.se  wrote: If you don't have deprecation warnings I would suspect some change in 0.4 has introduced type instabilities. If you are using typed concatenations you could be hit by https:  github.com JuliaLang julia issues 13254.   Den torsdag 22 oktober 2015 kl. 23:03:00 UTC+2 skrev Kris De Meyer: Are there any general style guidelines for moving code from 0.3.11 to 0.4.0? Running the unit and functionality tests for a module that I developed under 0.3.11 in 0.4, I experience a 500 times slowdown of blocks of code that I time with  time.   Can't even imagine where I have to start looking, and find it flabbergasting that perfectly valid julia code under 0.3.11  not generating a single warning  can show such a performance degradation under 0.4.0.  Anyone seen anything similar? Is there some fundamental difference in how code is JIT-compiled under 0.4.0?  Thanks,  Kris         Miguel Bazdresch    10 23 15  Re:  julia-users  Re: Code runs 500 times slower under 0.4.0  These are my thoughts exactly. I used Matlab because it was a convenient, easy way to get results, but the warts in the design are real. Julia, besides being convenient and easy, is a pleasure to program in.  -- mb      Tamas Papp    10 24 15  Re:  julia-users  Re: Code runs 500 times slower under 0.4.0  On Fri, Oct 23 2015, Kris De Meyer  kr... aniku.co.uk  wrote:     I know that all of these problems are solvable, but that's not the point I    am trying to make. All of these issues take time that keep me away from    writing the software that I am hired to write in a limited number of days,    and my employers may not be interested in the reasons for why I can't get    it to work on time. Please also note that when in the midst of a crisis and    up against deadlines, I  and probably other users in similar situations     may not have time to file bug reports, and not even have the time to note    down how I fixed or circumvented a certain problem.   If your project is time critical and you absolutely cannot deal with any  breakage, just make a snapshot of everything as is and don't upgrade  anything until you are sure you have time to deal with bugs. If you need  to upgrade  eg because a bug fix just came out which you need , use  version control carefully so you can roll back to the previous state, or  backport the fix if necessary.   Some languages have developed special tools for this --- eg for R, check  out Packrat: https:  rstudio.github.io packrat  I am not aware of  anything similar for Julia, so you may have to do that manually.   If you cannot implement any of the solutions above for some reason, you  may just have to accept that languages environments under development  and critical projects just don't mix.   Also, there is little point in telling people you don't have time to  file bug reports or make your workarounds known when using an open  source project. Open source projects thrive on user contributions  bug  reports count too , and that's pretty much the only way to make things  move forward.   Best,   Tamas      Stefan Karpinski    10 24 15  Re:  julia-users  Re: Code runs 500 times slower under 0.4.0  ReproZip is a pretty awesome language-agnostic tool for easy reproducibility:  https:  vida-nyu.github.io reprozip       Sisyphuss    10 25 15  Re:  julia-users  Re: Code runs 500 times slower under 0.4.0  It's interesting. It's like a virtual machine, but maybe smaller than it.      Stefan Karpinski    10 25 15   Not exactly. It uses ptrace to follow all of the dependencies that a sequence of programs uses and then creates an archive of those files in a way that will allow the program to run again in the future. Running on other OSes is accomplished with a VM, iirc.