Groups   167 of 99+      julia-users › Code review request: interacting with another program via stdin, stdout, stderr. 5 posts by 2 authors      Miguel Bazdresch    Jan 8   Hello,  I'd be grateful if you could take a look at some code and suggest improvements.  I'm trying to interact with a long-lived process  gnuplot . This process reads commands from its STDIN; after each command is executed, it produces output on either its STDOUT or STDERR. It's impossible to predict ahead of time which of the two streams will be used.  To simplify my tests, I wrote an  echo server  in C, which reads a character from its STDIN and outputs it again over either STDOUT or STDERR. The code is here: https:  gist.github.com mbaz 1e242694a9c4f1eca576  Then I wrote a julia program that reads a character from its own STDIN, sends it to the Clanguage echo server, and then tries to read the server's STDOUT and STDERR until it finds a character. The code is here: https:  gist.github.com mbaz bb7e2cbaaecc031b1d88  This code works, but I don't know if it is the best approach. Two specific questions I have:    Is my `popen3  ` function necessary? This closed issue seems to suggest it's not, but I can't figure out how to accomplish what I need in a more simple manner: https:  github.com JuliaLang julia issues 11824    Are global variables required to share data with asynchronous tasks? Since global variables are slow, this approach may produce undesired code slowdowns.  Thanks,  -- mb     Tomas Lycken    Jan 11   Interesting question! If you find a good approach to do this, wrapping it in a package is certainly interesting  have you checked that there aren’t any packages that handle this already? .  I can’t help much with the threading stuff, but regarding your global variable it should be possible to work around the slowness of that. I don’t know how, or when, you re-bind readnow, but there are two ways to fix it depending on the specifics:  You never rebind readnow, just mutate it. Mark it const, and it’s fast.  const means impossible to rebind, not impossible to mutate  You rebind readnow, and Condition   is mutable: Mark it const and mutate it instead of rebinding. You rebind readnow and Condition   is immutable: Wrap the Condition in a mutable type, which you assign to const readnow instead, and then rebind the field in this mutable type. Something like this: type ConditionWrapper     c::Condition end  const readnow   ConditionWrapper Condition       where you update: readnow.c   Condition args...  This all assumes that Condition is an immutable concrete type, and you just want to switch it out for an instance with other field values. If you actually need to change the type of readnow, all bets are off and this trick won’t work.     T       Miguel Bazdresch    Jan 11  Re:  julia-users  Re: Code review request: interacting with another program via stdin, stdout, stderr.  Tomas,  Thanks for your feedback! I couldn't find a package that handles this already. If this technique proves to work reliably in Gaston, I'll be happy to put it in a package.  Regarding `readnow`, I never rebind it. It doesn't hold any actual value; it is just a `Condition  `, which can be used to notify asynchronous tasks. I followed your advice and made it a `const`.  The other global variables actually hold data, which changes according to the values read from the different streams. Inspired by the wrapper idea, I wrapped them in a type so that it becomes clear that they are always strings; hopefully this will help with performance.  The latest version of the code is at https:  gist.github.com mbaz bb7e2cbaaecc031b1d88  -- mb      Tomas Lycken    Jan 12  Re:  julia-users  Re: Code review request: interacting with another program via stdin, stdout, stderr.  The other global variables actually hold data, which changes according to the values read from the different streams. Inspired by the wrapper idea, I wrapped them in a type so that it becomes clear that they are always strings; hopefully this will help with performance.  You won’t see any performance difference from just wrapping them. However, since you can now rebind the fields of the wrappers  which have strictly specified types  instead of rebinding the variables themselves, all those variables can now be marked const. This should give you some additional speed gain.  Another thing to consider, is that you probably want to be able to do this with several commands in parallel.  Consider, for example, running multiple echo programs and passing stuff between them  For that, you would need to wrap all of this in a function anyway. I would consider the following API:  function popen3 f::Function, cmd::Cmd        setup all streams, like your current popen3 function does        also, create all state handling for the different streams        finally, pass suitable arguments to f to let the user provide the actual action     f pin.in, out.out, err.out    for example, maybe something else makes more sense end The caller can then use a do block to consume this:  popen3 `. inout`  do pin, pout, perr       use the streams here end Since it’s all now wrapped in a function, “globals” won’t be a performance problem anymore. As a bonus, you can work with several, parallel, invocations without conflict between them. You might still see some slowness due to passing anonymous functions, but it’s a  probably negligible compared to the IO cost, and b  it’s actively being worked on and will eventually be fast.     T       Miguel Bazdresch    Jan 20  Re:  julia-users  Re: Code review request: interacting with another program via stdin, stdout, stderr.  Tom,  Interesting thoughts; I'll consider putting a package together. For the time being, I've implemented these ideas in Gaston, and after some testing it looks like the implementation is solid.  Thanks,  -- mb