{"id":340,"date":"2017-03-27T18:12:30","date_gmt":"2017-03-27T18:12:30","guid":{"rendered":"http:\/\/perfectionatic.org\/?p=340"},"modified":"2017-03-29T08:35:21","modified_gmt":"2017-03-29T08:35:21","slug":"using-pipes-while-running-external-programs-in-julia","status":"publish","type":"post","link":"https:\/\/perfectionatic.org\/?p=340","title":{"rendered":"Using pipes while running external programs in Julia"},"content":{"rendered":"<p>Recently I was using Julia to run <code>ffprobe<\/code> to get the length of a video file. The trouble was the <code>ffprobe<\/code> was dumping its output to <code>stderr<\/code> and I wanted to take that output and run it through <code>grep<\/code>. From a <code>bash<\/code> shell one would typically run:<\/p>\n<pre lang=\"bash\">\nffprobe somefile.mkv 2>&1 |grep Duration\n<\/pre>\n<p>This would result in  an output like<\/p>\n<pre lang=\"bash\">\n Duration: 00:04:44.94, start: 0.000000, bitrate: 128 kb\/s\n<\/pre>\n<p>This works because we used <code>2&gt;&amp;1<\/code> to redirect <code>stderr<\/code> to <code>stdout<\/code> which would in be piped to <code>grep<\/code>.<\/p>\n<p>If you were try to run this in Julia<\/p>\n<pre lang=\"julia\">\njulia> run(`ffprobe somefile.mkv 2>&1 |grep Duration`)\n<\/pre>\n<p>you will get errors. Julia does not like pipes <code>|<\/code> inside the backticks command (for very sensible <a href=\"http:\/\/julialang.org\/blog\/2013\/04\/put-this-in-your-pipe\">reasons<\/a>). Instead you should be using Julia&#8217;s <a href=\"http:\/\/docs.julialang.org\/en\/stable\/manual\/running-external-programs\/#pipelines\"><code>pipeline<\/code><\/a> command. Also the redirection <code>2&gt;&amp;1<\/code> will not work. So instead, the best thing to use is and instance of <code>Pipe<\/code>. This was not in the manual. I stumbled upon it in an <a href=\"https:\/\/github.com\/JuliaLang\/julia\/issues\/14437\">issue discussion on GitHub<\/a>. So a good why to do what I am after is to run.<\/p>\n<pre lang=\"julia\">\njulia> p=Pipe()\nPipe(uninit => uninit, 0 bytes waiting)\n\njulia> run(pipeline(`ffprobe -i  somefile.mkv`,stderr=p))\n<\/pre>\n<p>This would create a pipe object <code>p<\/code> that is then used to capture <code>stderr<\/code> after the execution of the command. Next we need to close the input end of the pipe.<\/p>\n<pre lang=\"julia\">\njulia> close(p.in)\n<\/pre>\n<p>Finally we can use the pipe with <code>grep<\/code> to filter the output.<\/p>\n<pre lang=\"julia\">\njulia> readstring(pipeline(p,`grep Duration`))\n\"  Duration: 00:04:44.94, start: 0.000000, bitrate: 128 kb\/s\\n\"\n<\/pre>\n<p>We can then do a little regex magic to get the duration we are after.<\/p>\n<pre lang=\"julia\">\njulia> matchall(r\"(\\d{2}:\\d{2}:\\d{2}.\\d{2})\",ans)[1]\n\"00:04:44.94\"\n<\/pre>\n","protected":false},"excerpt":{"rendered":"<p>Recently I was using Julia to run ffprobe to get the length of a video file. The trouble was the ffprobe was dumping its output to stderr and I wanted to take that output and run it through grep. From a bash shell one would typically run: ffprobe somefile.mkv 2>&#038;1 |grep Duration This would result [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[91,74,73,71],"tags":[92],"class_list":["post-340","post","type-post","status-publish","format-standard","hentry","category-ffmpeg","category-julia","category-julialang","category-programming","tag-julia"],"_links":{"self":[{"href":"https:\/\/perfectionatic.org\/index.php?rest_route=\/wp\/v2\/posts\/340","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/perfectionatic.org\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/perfectionatic.org\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/perfectionatic.org\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/perfectionatic.org\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=340"}],"version-history":[{"count":19,"href":"https:\/\/perfectionatic.org\/index.php?rest_route=\/wp\/v2\/posts\/340\/revisions"}],"predecessor-version":[{"id":357,"href":"https:\/\/perfectionatic.org\/index.php?rest_route=\/wp\/v2\/posts\/340\/revisions\/357"}],"wp:attachment":[{"href":"https:\/\/perfectionatic.org\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=340"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/perfectionatic.org\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=340"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/perfectionatic.org\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=340"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}