Groups 217 of 99+ julia-users › OpenCV.jl: A package for computer vision in Julia 35 posts by 4 authors Max Suster 12/6/14 Hi all, A few months ago I set out to learn Julia in an attempt to find an alternative to MATLAB for developing computer vision applications. Given the interest (1,2,3,4) and wide application of OpenCV for fast real-time computer vision applications, I set myself to put together a simple interface for OpenCV in Julia. Coding in Julia and developing the interface between C++ and Julia has been a lot of fun! OpenCV.jl aims to provide an interface for OpenCV computer vision applications (C++) directly in Julia. It relies primarily on Keno´s amazing Cxx.jl, the Julia C++ foreign function interface (FFI). You can find all the information on my package at https://github.com/maxruby/OpenCV.jl. You can download and run the package as follows: Pkg.clone("git://github.com/maxruby/OpenCV.jl.git") using OpenCV For MacOSX, OpenCV.jl comes with pre-compiled shared libraries, so it is extremely easy to run. For Windows and Linux, you will need to first compile the OpenCV libraries, but this is well documented and links to the instructions for doing so are included in the README.md file. The package currently supports most of OpenCV´s C++ API; however, at this point I have created custom wrappings for core, imgproc, videoio and highgui modules so that these are easy to use for anyone. The package also demonstrates/contains preliminary interface with the Qt GUI framework (see imread() and imwrite() functions) thin-wrappers for C++ objects such as std::vectors, std::strings conversion from Julia arrays to C++ std::vector conversion of Julia images (Images.jl) to Mat (OpenCV) - though this has much room for improvement (i.e., color handling) Please let me know if there are any features you would like to see added and I will try my best to integrate them. In the meantime, I will continue to integrate more advanced algorithms for computer vision and eventually extend the documentation as needed. Cheers, Max Simon Danisch 12/6/14 How very exciting!!!!!!!!!!!! And its even over the C++ interface =) I just recently checked out, how complicated it would be to do this, as I really want to have the GPU accelerated features from OpenCV... Would be so cool to fuse this natively with my OpenGL library, to view and filter images completely on the GPU, enabling great real-time performance! Did you manage to do the conversions without copying? - show quoted text - Simon Danisch 12/6/14 Personal note: I needed to do a lot of interactive 2D and 3D visualizations with results from OpenCV and it was all just very cumbersome... This was actually one of the primers for me to start searching for a better language, in which you could do the 2D/3D visualizations, without performance penalty and with a high degree of interactivity. Am Samstag, 6. Dezember 2014 11:44:45 UTC+1 schrieb Max Suster: - show quoted text - Max Suster 12/6/14 Glad to hear interest in this package :) I have indeed started to work on getting the CUDA features into OpenCV.jl (this was reorganized/relabelled from gpu to CUDA). My understanding is that OpenCV CUDA algorithms can use only a single GPU, and to utilize multiple GPUs, its necessary to distribute the work between several GPUs manually. I am experienced and not sure how to do this now with the Julia interface, but if you do know, I would be happy to collaborate on this. My main goal is to use OpenCV for real-time tracking applications (e.g., principal skeleton tracking), and using GPU (with up to 30x the speed for acquisition) would be invaluable. I have tested OpenCV with both boost C++ (multithreading) and GPU-accelerated approaches, and it seems to me that the GPU approach is most promising. One challenge however is that I found it very messy to compile OpenCV 3.0 with CUDA on OSX 10.9.5 and it seems to me that a number of people have reported bugs with the v3.0 build itself (at least on OSX). The second issue (as I am sure you know) is that for the GPU features to be worthwhile, you need a decent NVIDIA card and my GTX-Force 330M with a Computing Capability (CC) of 1.2 is not exactly amazing Hopefully this will change soon with a new Mac :) Since OpenCV is such a large API and it is used widely for so many applications, it will nice to hear from those interested here what features are worth expanding and which maybe less so. Max - show quoted text - Tim Holy 12/6/14 This looks really cool. Nice work! As you know well, it looks like most or all of what's in your README is covered with native Julia solutions (Images, VideoIO, and ImageView). But clearly the fun is going to come via the "advanced applications," for which there is very little publicly available in julia. I'm really looking forward to playing with this! Best, --Tim On Saturday, December 06, 2014 02:44:45 AM Max Suster wrote: > Hi all, > > A few months ago I set out to learn Julia in an attempt to find an > alternative to MATLAB for developing computer vision applications. > Given the interest (1 > jyfzxPt8Gk/SuwKtjTd9j4J> ,2 > 1V5zSNJY3Q/DRUT0dR2qhQJ> ,3 > /iUPqo8drYek/pUeHECk91AQJ> ,4 > /6QunG66MfNs/C63pDfI-EMAJ> ) and wide application of OpenCV for fast > real-time computer vision applications, I set myself to put together a > simple interface for OpenCV in Julia. Coding in Julia and developing the > interface between C++ and Julia has been a lot of fun! > > OpenCV.jl aims to provide an interface for OpenCV > computer vision applications (C++) directly in Julia > . It relies primarily on > Keno´s amazing Cxx.jl , the Julia C++ > foreign function interface (FFI). You can find all the information on my > package at https://github.com/maxruby/OpenCV.jl. > > You can download and run the package as follows: > > Pkg.clone("git://github.com/maxruby/OpenCV.jl.git")using OpenCV > > > For MacOSX, OpenCV.jl comes with pre-compiled shared libraries, so it is > extremely easy to run. For Windows and Linux, you will need to first > compile the OpenCV libraries, but this is well documented and links to the > instructions for doing so are included in the README.md file. > > The package currently supports most of OpenCV´s C++ API; however, at this > point I have created custom wrappings for core, imgproc, videoio and > highgui modules so that these are easy to use for anyone. > > The package also demonstrates/contains > > - preliminary interface with the Qt GUI framework (see imread() and > imwrite() functions) > - thin-wrappers for C++ objects such as std::vectors, std::strings > - conversion from Julia arrays to C++ std::vector > - conversion of Julia images (Images.jl) to Mat (OpenCV) - though this - show quoted text - Simon Danisch 12/6/14 Oh this is odd, why do they name the NVIDIA restricted version GPU-accelerated Computer Vision, and the hardware vendor independent version OpenCL-accelerated Computer Vision? I personally would first go for the OpenCL module, as its supported by more GPU's (I don't even have an NVIDIA card)... - show quoted text - Max Suster 12/6/14 Hi Tim, Thanks for your feedback. By no means, OpenCV.jl is meant to be a replacement of your outstanding contributions to Images.jl and Kevin´s wonderful VideoIO.jl package. As you may know, I spent some time recently contributing to VideoIO´s options API - so I realize that these packages are highly valuable :) Having said that, I think you will see that already the README.md file describes the following which are not currently available at all in Julia (or to the same extent via C++): Highly efficient pixel scanning and manipulation using pointers in C++ (smooth interface between Julia and C++) Conversion between images in Images.jl and OpenCV Mat format (to take advantage of hundreds of algorithms readily available in C++) Convenient live image processing with adjustable trackbars (e.g., thresholding, and dozens of image processing filters possible without much effort), including options for Qt GUI support Object tracking (which is not currently available for Julia) is already demonstrated in test/cxx/demos.jl There is clearly much work to be done on this package to get it to a mature state so that it supports great computer vision applications with Julia (especially for real-time tracking). I am looking forward to suggestions for further development. Cheers, Max - show quoted text - Max Suster 12/6/14 Translate message to English Simon, Straight from the OpenCV 3.0 documentation site: While in the future we hope to validate (and enable) the OCL module in all OpenCL capable devices, we currently develop and test on GPU devices only. This includes both discrete GPUs (NVidia, AMD), as well as integrated chips (AMD APU and Intel HD devices). Performance of any particular algorithm will depend on the particular platform characteristics and capabilities. However, currently, accuracy and mathematical correctness has been verified to be identical to that of the pure CPU implementation on all tested GPU devices and platforms (both Windows and Linux). Does this mean that they have not tested on OSX or is this more likely due to hardware support? Someone has encountered problems running OpenCL on OSX, but perhaps this is an isolated case (OpenCL example failed on OSX). According to the OpenCV 3.0 documentation Many kernels use 256 as its workgroup size if possible, so the max work group size of the device must larger than 256. All GPU devices we are aware of indeed support 256 workitems in a workgroup, however non GPU devices may not. This will be improved in the future. However, there is a nice and useful link here for how to support OpenCV-OpenCL on OSX so it maybe worth trying: https://fogofcode.wordpress.com/2014/02/23/running-opencv-on-osx-with-opencl-support/ Apparently, building OpenCV with OpenCL requires AMDBLAS and AMDFFT. I guess you are using Linux or Windows, right? clAmdFft and clAmdBlas are only built for Windows and Linux so I will have to build them first from source in OSX. Max - show quoted text - Tim Holy 12/6/14 Oh, I'm well aware of your great contributions to VideoIO. And OpenCV looks like an even bigger one. However, I think quite a bit of "live processing" is already supported via the `map/MapInfo` framework. For example, below is a complete, high-performance GUI for interactively setting a threshold. Because the whole MapInfo framework is lazy, it can be used with both static images and video. --Tim using Base.Cartesian using Images, Tk, ImageView, Color import Base: map! export threshold_slider # Threshold pixels using a slider type ThresholdSlider{To,A<:AbstractArray,From,M<:Images.MapInfo} <: Images.MapInfo{To} value::A slider_min::From slider_max::From mapi0::M slider end function ThresholdSlider(value::AbstractArray, mapi0, s, extrma=extrema(value)) smin, smax = promote(extrma[1], extrma[2]) ThresholdSlider{Uint32,typeof(value),typeof(smin),typeof(mapi0)}(value, smin, smax, mapi0, s) end function threshold_slider(img, value, extrma=extrema(value); mapi0 = mapinfo(RGB24, img)) w = Toplevel() f = Frame(w) pack(f, expand=true, fill="both") l = Label(f, "Threshold: $(extrma[1])") pack(l, side="top") s = Slider(f, extrma[1], extrma[2]) ts = ThresholdSlider(value, mapi0, s, extrma) imgc, imsl = view(img, scalei=ts) pack(s, side="top", expand=true, fill="both", anchor="w") bind(s, "command", path -> (set_value(l, @sprintf("%4.2e", get_value(s))); ImageView.redraw(imgc, imsl))) end function map!{T,N}(mapi::ThresholdSlider{Uint32}, dest::Array{Uint32,N}, src::AbstractImageDirect{T,N}) value = sub(mapi.value, data(src).indexes[1:N]) map_thresh!(mapi, dest, src, value, get_value(mapi.slider)) end @ngenerate N typeof(dest) function map_thresh!{T,N} (mapi::ThresholdSlider{Uint32}, dest::Array{Uint32,N}, src::AbstractImageDirect{T,N}, value, thresh) size(dest) == size(src) || throw(DimensionMismatch()) mapi0 = mapi.mapi0 @nloops N i dest begin c = map(mapi0, @nref N src i) x = @nref N value i (@nref N dest i) = x > thresh? convert(RGB24, c) : RGB24(0) end dest end On Saturday, December 06, 2014 04:34:55 AM Max Suster wrote: > Hi Tim, > > Thanks for your feedback. By no means, OpenCV.jl is meant to be a > replacement of your outstanding contributions to Images.jl and Kevin´s > wonderful VideoIO.jl package. As you may know, I spent some time recently > contributing to VideoIO´s options API - so I realize that these packages > are highly valuable :) > > Having said that, I think you will see that already the README.md file > describes the following which are not currently available at all in Julia > (or to the same extent via C++): > > - Highly efficient pixel scanning and manipulation using pointers in C++ > (smooth interface between Julia and C++) > - Conversion between images in Images.jl and OpenCV Mat format (to take > advantage of hundreds of algorithms readily available in C++) > - Convenient live image processing with adjustable trackbars (e.g., > thresholding, and dozens of image processing filters possible without > much effort), including options for Qt GUI support > - Object tracking (which is not currently available for Julia) is - show quoted text - Tim Holy 12/6/14 But I should say again: the "advanced" stuff looks great, and definitely there is no julia analog that I know of. The tracking stuff in particular will be very nice to have. --Tim On Saturday, December 06, 2014 04:34:55 AM Max Suster wrote: > Hi Tim, > > Thanks for your feedback. By no means, OpenCV.jl is meant to be a > replacement of your outstanding contributions to Images.jl and Kevin´s > wonderful VideoIO.jl package. As you may know, I spent some time recently > contributing to VideoIO´s options API - so I realize that these packages > are highly valuable :) > > Having said that, I think you will see that already the README.md file > describes the following which are not currently available at all in Julia > (or to the same extent via C++): > > - Highly efficient pixel scanning and manipulation using pointers in C++ > (smooth interface between Julia and C++) > - Conversion between images in Images.jl and OpenCV Mat format (to take > advantage of hundreds of algorithms readily available in C++) > - Convenient live image processing with adjustable trackbars (e.g., > thresholding, and dozens of image processing filters possible without > much effort), including options for Qt GUI support > - Object tracking (which is not currently available for Julia) is > already demonstrated in test/cxx/demos.jl > > There is clearly much work to be done on this package to get it to a mature > state so that it supports great computer vision applications with Julia > (especially for real-time tracking). I am looking forward to suggestions > for further development. > > Cheers, > Max > > On Saturday, December 6, 2014 1:19:24 PM UTC+1, Tim Holy wrote: > > This looks really cool. Nice work! > > > > As you know well, it looks like most or all of what's in your README is > > covered with native Julia solutions (Images, VideoIO, and ImageView). But > > clearly the fun is going to come via the "advanced applications," for > > which > > there is very little publicly available in julia. > > > > I'm really looking forward to playing with this! > > > > Best, > > --Tim > > > > On Saturday, December 06, 2014 02:44:45 AM Max Suster wrote: > > > Hi all, > > > > > > A few months ago I set out to learn Julia in an attempt to find an > > > alternative to MATLAB for developing computer vision applications. > > > Given the interest (1 > > > < > > > > https://groups.google.com/forum/#!searchin/julia-users/OpenCV/julia-users/ > > P > > > > > jyfzxPt8Gk/SuwKtjTd9j4J> ,2 > > > < > > > > https://groups.google.com/forum/#!searchin/julia-users/OpenCV/julia-users/ > > 8 > > > > > 1V5zSNJY3Q/DRUT0dR2qhQJ> ,3 > > > < > > > > https://groups.google.com/forum/%23!searchin/julia-users/OpenCV/julia-user > > s > > > > > /iUPqo8drYek/pUeHECk91AQJ> ,4 > > > < > > > > https://groups.google.com/forum/%23!searchin/julia-users/OpenCV/julia-user > > s > > - show quoted text - Tim Holy 12/6/14 Re: [julia-users] Re: OpenCV.jl: A package for computer vision in Julia I don't know if it will help at all in the context of OpenCV, but here's a test script that demonstrates farming out work to multiple GPUs: https://github.com/JuliaGPU/CUDArt.jl/blob/6409b57f7c80ed2459fd46c0e86ab8de681fd9bc/test/test.jl#L185-L222 --Tim - show quoted text - > >> Given the interest (1 > >> >> s/PjyfzxPt8Gk/SuwKtjTd9j4J> ,2 > >> >> s/81V5zSNJY3Q/DRUT0dR2qhQJ> ,3 > >> >> ers/iUPqo8drYek/pUeHECk91AQJ> ,4 > >> >> ers/6QunG66MfNs/C63pDfI-EMAJ> ) and wide application of OpenCV for fast > >> real-time computer vision applications, I set myself to put together a > >> simple interface for OpenCV in Julia. Coding in Julia and developing > >> the interface between C++ and Julia has been a lot of fun! > >> > >> OpenCV.jl aims to provide an interface for OpenCV > >> computer vision applications (C++) directly in Julia > >> . It relies primarily on > >> Keno´s amazing Cxx.jl , the Julia C++ > >> foreign function interface (FFI). You can find all the information on my > >> package at https://github.com/maxruby/OpenCV.jl. > >> > >> You can download and run the package as follows: > >> > >> Pkg.clone("git://github.com/maxruby/OpenCV.jl.git")using OpenCV > >> > >> > >> For MacOSX, OpenCV.jl comes with pre-compiled shared libraries, so it is > >> extremely easy to run. For Windows and Linux, you will need to first > >> compile the OpenCV libraries, but this is well documented and links to > >> the > >> instructions for doing so are included in the README.md file. > >> > >> The package currently supports most of OpenCV´s C++ API; however, at this > >> point I have created custom wrappings for core, imgproc, videoio and > >> highgui modules so that these are easy to use for anyone. > >> > >> The package also demonstrates/contains > >> > >> - preliminary interface with the Qt GUI framework (see imread() and > >> imwrite() functions) > >> - thin-wrappers for C++ objects such as std::vectors, std::strings > >> - conversion from Julia arrays to C++ std::vector > >> - conversion of Julia images (Images.jl) to Mat (OpenCV) - though - show quoted text - Max Suster 12/6/14 Re: [julia-users] Re: OpenCV.jl: A package for computer vision in Julia Thanks a lot for pointing out the script. I will definitely have a close look at the CUDA implementation. Regarding the live GUI trackbar with Images, I did not know that this was implemented anywhere (i.e., did not see it in the documentation). Personally, for my own needs, I also think there are advantages to having a single unified package that allows both the basic image array handling, GUI, and real-time tracking. . . and in the spirit of Christmas, the more the merrier . . . Cheers, Max - show quoted text - Tim Holy 12/6/14 Re: [julia-users] Re: OpenCV.jl: A package for computer vision in Julia It's not in ImageView, it's just something I use privately. Given package load times, there's some tension between "having everything" and "making things easy to implement." So Images & ImageView largely focus on ingredients, and need users to put them together to make the soup. But I suppose we could have an ImageViewExtras package or something. Anyway, I fully agree that having more options is great! --Tim - show quoted text - Max Suster 12/6/14 http://developer.amd.com/community/blog/2014/10/15/opencv-3-0-transparent-api-opencl-acceleration/ The link above makes it sounds very promising that enabling OpenCL in OpenCV.jl will be rather feasible (assuming no surprises along the way...). I was first looking frantically for the module ("ocl") in my OpenCV 3.0 folder because the (outdated) web API documentation said that there should be such a folder. However, this is not true - it turns out that from v3.0, the “ocl” namespace and folder are gone, and so is the “oclMat.” To transfer data one has to use UMat. I will try to get this implemented soon. Max Max Suster 12/6/14 Simon, It turns out that using OpenCV.jl with OpenCL support is actually remarkably easy thanks to the transparent T-API of OpenCV 3.0. All that is needed is to use cv::UMat in place of cv::Mat (and an extra header file in OpenCV.jl). A comparison of a simple RGB to gray conversion with CPU vs. GPU Mat suggest at least 10x improvement (even with a mediocre GTX-Force 330M 512MB, CC 1.2): Declare the Mat and UMat (1000x1000 RGB) source and initialize target images julia> srcMat = Mat(1000, 1000, CV_8UC3, cvScalar(0, 255, 0)); julia> srcUMat = UMat(1000, 1000, CV_8UC3, cvScalar(0, 255, 0)); julia> dstMat = Mat() julia> dstUMat = UMat() CPU julia> @time(cvtColor(srcMat, dstMat, COLOR_BGR2GRAY)) elapsed time: 0.00164426 seconds (80 bytes allocated) elapsed time: 0.001682741 seconds (80 bytes allocated) elapsed time: 0.002086002 seconds (80 bytes allocated) elapsed time: 0.001711169 seconds (80 bytes allocated) elapsed time: 0.001686306 seconds (80 bytes allocated) elapsed time: 0.001655958 seconds (80 bytes allocated) elapsed time: 0.001707453 seconds (80 bytes allocated) GPU julia> @time(cvtColor(srcUMat, dstUMat, COLOR_BGR2GRAY)) elapsed time: 0.000149589 seconds (80 bytes allocated) elapsed time: 0.000149705 seconds (80 bytes allocated) elapsed time: 0.000149575 seconds (80 bytes allocated) elapsed time: 0.000151404 seconds (80 bytes allocated) elapsed time: 0.00017067 seconds (80 bytes allocated) elapsed time: 0.000181821 seconds (80 bytes allocated) - show quoted text - Stefan Karpinski 12/6/14 That's really cool. I'm looking forward to playing around with this. - show quoted text - Simon Danisch 12/8/14 That's really great! I will have to investigate, how to convert between OpenGL datatypes and UMat. It would be incredible, if we can convert between julia, opencl, opengl and opencv datatypes without a big overhead. I'm pretty sure, that no other language has this solved nicely! ;) Am Samstag, 6. Dezember 2014 11:44:45 UTC+1 schrieb Max Suster: - show quoted text - Max Suster 12/8/14 http://stackoverflow.com/questions/9097756/converting-data-from-glreadpixels-to-opencvmat Seem that the link above shows a solution to this. Max - show quoted text - Simon Danisch 12/8/14 Re: [julia-users] Re: OpenCV.jl: A package for computer vision in Julia That's the trivial solution with quite a few copies and transfers involved. But if the Umat is used with OpenCL, the image data should reside on the GPU already, which means you don't need to download it from the gpu and upload it back again for displaying the data. But that means, it must be possible to create a UMat from an OpenCL buffer, as it must be created first via OpenGL (silly restriction for the OpenCL OpenGL interoperability). If that works, one can manipulate the Umat via OpenCL and the OpenGL buffer will be the same memory and can be displayed without any copies ;) - show quoted text - Simon Danisch 12/8/14 Re: [julia-users] Re: OpenCV.jl: A package for computer vision in Julia This would be a related (unanswered ) question: http://answers.opencv.org/question/38848/opencv-displaying-umat-efficiently/ As I don't see any related constructors for the Umat, the easiest starting point would be to ask in the forum. This will probably get faster results, than diving into the code ourselves. - show quoted text - Simon Danisch 12/8/14 If you're interested here are some more links: https://software.intel.com/en-us/articles/opencl-and-opengl-interoperability-tutorial Valentine's and mine prototype for OpenGL OpenCL interoperability in Julia: https://github.com/vchuravy/qjulia_gpu Am Samstag, 6. Dezember 2014 11:44:45 UTC+1 schrieb Max Suster: - show quoted text - Max Suster 12/8/14 Thanks for the feedback. I realize that the copying needs to be skipped if possible . . . I have been playing a bit with the OpenCL UMat and it will need indeed some tweeking because UMat is not always advantageous. While there 10x gain with cvtColor and other functions such as GasussianBlur are actually a little slower. I will have closer look at this tonight. Max - show quoted text - Simon Danisch 12/8/14 Re: [julia-users] Re: OpenCV.jl: A package for computer vision in Julia That's interesting, gaussian blur should definitely be faster on the gpu! Maybe this thread helps? http://answers.opencv.org/question/34127/opencl-in-opencv-300/ It seems like things are a little complicated, as it isn't really clear if the data is currently in VRAM or RAM... - show quoted text - Max Suster 12/8/14 Re: [julia-users] Re: OpenCV.jl: A package for computer vision in Julia I know this is a Julia coding forum, but if you have a chance, can you compare the two examples below? Also, what OS are you using? Before testing a lot of Julia wrapped C++ API with OpenCV/OpenCL/OpenGL, it would be good to know what we can expect at best. . . CPU int main(int argc, char** argv) { Mat img, gray; img = imread("your RGB image", -1); imshow("original", img); cvtColor(img, gray, COLOR_BGR2GRAY); // 50 ms GaussianBlur(gray, gray, Size(7, 7), 1.5); // 2 ms double t = (double)getTickCount(); Canny(gray, gray, 0, 50); // 6 ms t = ((double)getTickCount() - t)/getTickFrequency(); cout << "Times passed in seconds: " << t << endl; imshow("edges", gray); waitKey(); return 0; } GPU static UMat img, gray; int main(int argc, char** argv) { Mat src; src = imread("your RGB image", -1); src.copyTo(img); // copy from CPU to GPU buffer imshow("original", src); double t = (double)getTickCount(); cvtColor(img, gray, COLOR_BGR2GRAY); // 3 ms t = ((double)getTickCount() - t)/getTickFrequency(); cout << "Times passed in seconds: " << t << endl; GaussianBlur(gray, gray, Size(7, 7), 1.5); // 13 ms Canny(gray, gray, 0, 50); // 10 ms imshow("edges", gray); waitKey(); return 0; } Tim Holy 12/8/14 Re: [julia-users] Re: OpenCV.jl: A package for computer vision in Julia I wonder if the bigger problem might be that your numbers for the grayscale conversion (which were very promising) might be misleading. Are you sure the calculation is "done" (and the results are available to the CPU) by the time it finishes? If we assume a best-case scenario of 6GB/s of data transfer to the GPU, then transferring 3MB to the GPU and 1MB back takes about 0.7ms. That's many times longer than what you reported for that calculation. Or did you not include transfer time in your results? --Tim - show quoted text - > >>> Given the interest (1 > >>> >>> rs/PjyfzxPt8Gk/SuwKtjTd9j4J> ,2 > >>> >>> rs/81V5zSNJY3Q/DRUT0dR2qhQJ> ,3 > >>> >>> sers/iUPqo8drYek/pUeHECk91AQJ> ,4 > >>> >>> sers/6QunG66MfNs/C63pDfI-EMAJ> ) and wide application of OpenCV for fast > >>> real-time computer vision applications, I set myself to put together a > >>> simple interface for OpenCV in Julia. Coding in Julia and developing > >>> the interface between C++ and Julia has been a lot of fun! > >>> > >>> OpenCV.jl aims to provide an interface for OpenCV > >>> computer vision applications (C++) directly in Julia > >>> . It relies primarily > >>> on Keno´s amazing Cxx.jl , the Julia > >>> C++ foreign function interface (FFI). You can find all the information > >>> on > >>> my package at https://github.com/maxruby/OpenCV.jl. > >>> > >>> You can download and run the package as follows: > >>> > >>> Pkg.clone("git://github.com/maxruby/OpenCV.jl.git")using OpenCV > >>> > >>> > >>> For MacOSX, OpenCV.jl comes with pre-compiled shared libraries, so it is > >>> extremely easy to run. For Windows and Linux, you will need to first > >>> compile the OpenCV libraries, but this is well documented and links to > >>> the > >>> instructions for doing so are included in the README.md file. > >>> > >>> The package currently supports most of OpenCV´s C++ API; however, at > >>> this point I have created custom wrappings for core, imgproc, videoio > >>> and highgui modules so that these are easy to use for anyone. > >>> > >>> The package also demonstrates/contains > >>> > >>> - preliminary interface with the Qt GUI framework (see imread() and > >>> imwrite() functions) > >>> - thin-wrappers for C++ objects such as std::vectors, std::strings > >>> - conversion from Julia arrays to C++ std::vector > >>> - conversion of Julia images (Images.jl) to Mat (OpenCV) - though - show quoted text - Max Suster 12/8/14 Re: [julia-users] Re: OpenCV.jl: A package for computer vision in Julia Its an exact comparison side-by-side of the same code, and actually already tested by others in the OpenCV forum. The Mat/UMat image is available for display with imshow -- the step "imshow("edges", gray);" in both cases -- which is how the test was set up. The main point is the time it takes to complete the entire process and the fact that cvtColor with OpenCL can generate an image for viewing much more quickly. I never intended to be misleading, and I am sorry that you interpret it this way. Max On Monday, December 8, 2014 6:22:39 PM UTC+1, Tim Holy wrote: I wonder if the bigger problem might be that your numbers for the grayscale conversion (which were very promising) might be misleading. Are you sure the calculation is "done" (and the results are available to the CPU) by the time it finishes? If we assume a best-case scenario of 6GB/s of data transfer to the GPU, then transferring 3MB to the GPU and 1MB back takes about 0.7ms. That's many times longer than what you reported for that calculation. Or did you not include transfer time in your results? --Tim On Monday, December 08, 2014 05:50:32 PM Simon Danisch wrote: > That's interesting, gaussian blur should definitely be faster on the gpu! > Maybe this thread helps? > http://answers.opencv.org/question/34127/opencl-in-opencv-300/ > It seems like things are a little complicated, as it isn't really clear if > the data is currently in VRAM or RAM... > > 2014-12-08 17:39 GMT+01:00 Max Suster : > > Thanks for the feedback. I realize that the copying needs to be skipped > > if possible . . . > > I have been playing a bit with the OpenCL UMat and it will need indeed > > some tweeking because UMat is not always advantageous. > > While there 10x gain with cvtColor and other functions such as > > GasussianBlur are actually a little slower. > > > > I will have closer look at this tonight. > > > > Max > > > > On Monday, December 8, 2014 4:15:28 PM UTC+1, Simon Danisch wrote: > >> If you're interested here are some more links: > >> https://software.intel.com/en-us/articles/opencl-and-opengl-> >> interoperability-tutorial > >> Valentine's and mine prototype for OpenGL OpenCL interoperability in > >> Julia: > >> https://github.com/vchuravy/qjulia_gpu > >> > >> Am Samstag, 6. Dezember 2014 11:44:45 UTC+1 schrieb Max Suster: > >>> Hi all, > >>> > >>> A few months ago I set out to learn Julia in an attempt to find an > >>> alternative to MATLAB for developing computer vision applications. > >>> Given the interest (1 > >>> >>> rs/PjyfzxPt8Gk/SuwKtjTd9j4J> ,2 > >>> >>> rs/81V5zSNJY3Q/DRUT0dR2qhQJ> ,3 > >>> >>> sers/iUPqo8drYek/pUeHECk91AQJ> ,4 > >>> >>> sers/6QunG66MfNs/C63pDfI-EMAJ> ) and wide application of OpenCV for fast > >>> real-time computer vision applications, I set myself to put together a > >>> simple interface for OpenCV in Julia. Coding in Julia and developing > >>> the interface between C++ and Julia has been a lot of fun! > >>> > >>> OpenCV.jl aims to provide an interface for OpenCV > >>> computer vision applications (C++) directly in Julia > >>> . It relies primarily > >>> on Keno´s amazing Cxx.jl , the Julia > >>> C++ foreign function interface (FFI). You can find all the information > >>> on > >>> my package at https://github.com/maxruby/OpenCV.jl. > >>> > >>> You can download and run the package as follows: > >>> > >>> Pkg.clone("git://github.com/maxruby/OpenCV.jl.git")using OpenCV > >>> > >>> > >>> For MacOSX, OpenCV.jl comes with pre-compiled shared libraries, so it is > >>> extremely easy to run. For Windows and Linux, you will need to first > >>> compile the OpenCV libraries, but this is well documented and links to > >>> the > >>> instructions for doing so are included in the README.md file. > >>> > >>> The package currently supports most of OpenCV´s C++ API; however, at > >>> this point I have created custom wrappings for core, imgproc, videoio > >>> and highgui modules so that these are easy to use for anyone. > >>> > >>> The package also demonstrates/contains > >>> > >>> - preliminary interface with the Qt GUI framework (see imread() and > >>> imwrite() functions) > >>> - thin-wrappers for C++ objects such as std::vectors, std::strings > >>> - conversion from Julia arrays to C++ std::vector > >>> - conversion of Julia images (Images.jl) to Mat (OpenCV) - though > >>> this has much room for improvement (i.e., color handling) > >>> > >>> Please let me know if there are any features you would like to see added > >>> and I will try my best to integrate them. In the meantime, I will > >>> continue > >>> to integrate more advanced algorithms for computer vision and eventually > >>> extend the documentation as needed. > >>> > >>> Cheers, > >>> Max Simon Danisch 12/8/14 Re: [julia-users] Re: OpenCV.jl: A package for computer vision in Julia Under the premise, that everything can be done on the GPU up to a certain point, transfer times shouldn't be included in the benchmark, as it wouldn't be necessary to have the data available to the CPU. For all simple image transformation, OpenCL should be strictly faster. ( obviously not if you have a Intel Xeon E5-2699 and a weak graphic card) Could it be that OpenCL automatically accelerates even normal Mats with the GPU? This might be the reason why the initial cvtColor is so slow on the CPU, as it includes transferring the data to the CPU. Other reason could be, that OpenCV does transfers and copies even with UMat. I'm not sure what's going on here, but I might try to build OpenCV with OpenCL tomorrow and see for myself. Have you tried to explicitly turn OpenCL on and off with OPENCV_OPENCL_RUNTIME? - show quoted text - Tim Holy 12/8/14 Re: [julia-users] Re: OpenCV.jl: A package for computer vision in Julia Now that you posted the code (along with timing markers), it's much clearer. Your timings are just for the computation, and don't include transfer time. My "misleading" statement was not intended as an accusation :-), merely trying to help explain the gaussian blur result: if you included transfer time in one but not the other, that might explain it. But from your posted code, it's clear that wasn't the problem. --Tim - show quoted text - > > > 2014-12-08 17:39 GMT+01:00 Max Suster >: - show quoted text - Max Suster 12/8/14 Re: [julia-users] Re: OpenCV.jl: A package for computer vision in Julia Thanks Tim (I did not mean to be defensive). I totally agree that it is best to get to the bottom of the issue so that we can get it work! The point I was trying to make more generally is that there does seem to be quite a bit of variability in the gains from OpenCL-acceleration at least with OpenCV. It would be nice to figure out how and which bits can truly be accelerated most efficiently and which may not be worth spending a lot more energy on. . . Max - show quoted text - Simon Danisch 12/8/14 Re: [julia-users] Re: OpenCV.jl: A package for computer vision in Julia Strictly faster is probably a little bit exaggerated, but simple image manipulations fit the GPU very well. This should be valid for all algorithms which can be massively parallelized and don't use scatter writes or reads. So if you have decent video card, the cpu should have a hard time to match the performance. I'm not sure about the transfers, as OpenCV might actually do transfers even in the UMat case... It's not that obvious how they manage their memory. - show quoted text - Simon Danisch 12/8/14 Re: [julia-users] Re: OpenCV.jl: A package for computer vision in Julia To give it some more underpinning, I searched for a benchmark. I only found one for cuda so far from 2012: http://www.timzaman.com/?p=2256 Results in a table: https://docs.google.com/spreadsheet/pub?key=0AkfBSyx6TpqMdDFnclY5UjlyNmVsSnhGV0hscnJQcVE&output=html Resulting graph: I think the scale is speedup from using the gpu, so the cpu is always one. - show quoted text - Tim Holy 12/8/14 Re: [julia-users] Re: OpenCV.jl: A package for computer vision in Julia Certainly it's possible that the quality of the kernels varies. The blur case is presumably harder to write a good kernel for and more subject to the chosen schedule, because it has to access memory non-locally. The grayscale conversion is pointwise and so is fairly trivial. But I would have expected the blur kernel to be highly optimized; I agree it's a puzzling result. But that table that Simon linked suggests it should be faster, so I wonder if there's something fishy going on. --Tim - show quoted text - Max Suster 12/8/14 Re: [julia-users] Re: OpenCV.jl: A package for computer vision in Julia Thanks! That is very useful. Why is cvColor so slow in the Intel tests? I notice they used a 4000x4000, CV_8UC4 (Uint8, 4 channels) for the GaussianBlur test! - show quoted text - Simon Danisch 12/8/14 Re: [julia-users] Re: OpenCV.jl: A package for computer vision in Julia That's a good point! Are you using CV_8UC3? Video cards like everything that has 32 bits, so if you use RGB (3*8=24) depending on how OpenCV optimizes things, it could mean quite a slow down for the UMat case. - show quoted text - Max Suster 12/8/14 Re: [julia-users] Re: OpenCV.jl: A package for computer vision in Julia Yes, I am using CV_8UC3 for my tests (and dim 512x512 or 1000x1000). I think the most likely explanation for my results is that while OpenCV comes with OpenCL enabled in the makefile -D WITH_OPENCL=ON the flags for including OpenCL AMD FFT and AMD BLAS libraries are OFF by default: -D WITH_OPENCLAMDFFT=OFF -D WITH_OPENCLAMDBLAS=OFF I suppose that the consequence is likely that the gains with OpenCL are inconsistent at best on OSX. Unfortunately, these libraries are not available for OSX right off the shelf, and it seems that I need to build them first. http://developer.amd.com/tools-and-sdks/opencl-zone/amd-accelerated-parallel-processing-math-libraries/ They are readily available for Windows and Linux. I found a post in which someone managed to get them working with a hack. I need to this investigate further. . .