Re: Shell commands with output to string

[Zelphir Kaltstahl]

Hello Josselin and all!

On 2/23/22 15:01, Josselin Poiret wrote:
> Hello,
>
>
> post@thomasdanckaert.be  writes:
>
> > Hi,
> >
> > to throw in an example: I once used a function like the one below to
> > handle stdout and stderr from external commands (from
> > https://github.com/tdanckaert/jobview/blob/master/jobtools.scm#L38  ).
> > Probably far from perfect (my first and only scheme project...), but
> > hopefully it gives you an idea.
> Just chiming in to say that [1] isn't fixed yet, so you may run into
> issues if you try to redirect out and err to the same port.  In Guix, we
> use the following workaround for now ([2]):
> --8<---------------cut here---------------start------------->8---
>    (match-let (((input . output) (pipe)))
>      ;; Hack to work around Guile bug 52835
>      (define dup-output (duplicate-port output "w"))
>      ;; Void pipe, but holds the pid for close-pipe.
>      (define dummy-pipe
>        (with-input-from-file "/dev/null"
>          (lambda ()
>            (with-output-to-port output
>              (lambda ()
>                (with-error-to-port dup-output
>                  (lambda ()
>                    (apply open-pipe* (cons "" command)))))))))
>      (close-port output)
>      (close-port dup-output)
>      (handler input)
>      (close-port input)
>      (close-pipe dummy-pipe))
> --8<---------------cut here---------------end--------------->8---
>
> [1]https://debbugs.gnu.org/cgi/bugreport.cgi?bug=52835
> [2]https://git.savannah.gnu.org/cgit/guix.git/tree/gnu/installer/utils.scm?id=c0bc08d82c73e464a419f213d5ae5545bc67e2bf#n87
>
> Best,

I have questions regarding this workaround:

Can you explain how and why this works? I have tried to make sense of it and 
here are my notes so far (reference: 
https://notabug.org/ZelphirKaltstahl/guile-examples/src/2dead9f7bb9b40fc26eb490a93e1dc7abca7252c/shell/system-asterisk-stdout-to-stderr-redirection-bug.scm):

~~~~
(match-let (((input . output) (pipe)))
  ;; Hack to work around Guile bug 52835 -- How does
  ;; duplicating the port help? From the docs: "Returns a
  ;; new port which is opened on a duplicate of the file
  ;; descriptor underlying port, with mode string modes as
  ;; for open-file. The two ports will share a file position
  ;; and file status flags. [...]"
  (define dup-output (duplicate-port output "w"))
  ;; Void pipe, but holds the pid for close-pipe.
  (define dummy-pipe
    ;; Set current-input-port to /dev/null. -- What will be
    ;; read from there? Nothing?
    (with-input-from-file "/dev/null"
      (lambda ()
        ;; Set the current-output-port to the one created
        ;; above using (pipe).
        (with-output-to-port output
          (lambda ()
            ;; Set the error port to the duplicated output
            ;; port. This might be the redirection of stderr
            ;; to stdout.
            (with-error-to-port dup-output
              (lambda ()
                ;; Run open-file*, but why is there an empty
                ;; string prepended to command? Perhaps to
                ;; allow using either a list or a string as
                ;; a command?
                (apply open-pipe* (cons "" command)))))))))

  (close-port output)
  (close-port dup-output)
  (handler input)
  (close-port input)
  (close-pipe dummy-pipe))
~~~~

My other question is: Do I still need this workaround, if I use the following, 
to run commands? And if so, why? In which cases would my code not do the right 
thing? (reference: 
https://notabug.org/ZelphirKaltstahl/guile-examples/src/2dead9f7bb9b40fc26eb490a93e1dc7abca7252c/shell/example-03-using-popen-get-out-and-error.scm):

~~~~
(import (ice-9 popen)
        (ice-9 textual-ports)
        (ice-9 exceptions)
        (ice-9 receive)
        (ice-9 match))


;; Removed comments to shorting this example. For more
;; explanation see the first example.
(define run-command
  (λ (cmd)
    "Runs CMD as an external process, with an input port
from which the process' stdout may be read."
    (match-let ([(err-read . err-write) (pipe)]
                [stderr (current-error-port)])
      (with-error-to-port err-write
        (λ ()
          (let* (;; Run the actual command. If an error
                 ;; happens, it should write to the
                 ;; err-write port. Output of the command
                 ;; should be written to an output port,
                 ;; which corresponds to the input-port,
                 ;; which is returned by open-input-pipe.
                 [in-port (open-input-pipe cmd)]
                 ;; Read in block mode.
                 [_ignored (setvbuf in-port 'block)]
                 ;; Get command output and error output.
                 [command-output (get-string-all in-port)]
                 ;; Get the exit code of the command.
                 [exit-code (close-pipe in-port)])
            ;; Close the port, to which the child process
            ;; was to write errors, as the child process has
            ;; finished (either successfully or
            ;; unsuccessfully, but definitely finished).
            (close-port err-write)
            (let (;; Get the error message, if there is any.
                  [error-message (get-string-all err-read)])
              (values exit-code
                      command-output
                      error-message))))))))


(receive (exit-code command-output error-message)
    (let ([command "echo 'bong' 1>&2"])
      (run-command command))
  (display (simple-format #f "exit code: ~a\n" exit-code))
  (unless (string-null? command-output)
    (display (simple-format #f "command-output: \n~a" command-output)))
  (unless (string-null? error-message)
    (display (simple-format #f "error-message: \n~a" error-message))))


(receive (exit-code command-output error-message)
    (let ([command "ls -al"])
      (run-command command))
  (display (simple-format #f "exit code: ~a\n" exit-code))
  (unless (string-null? command-output)
    (display (simple-format #f "command-output: \n~a" command-output)))
  (unless (string-null? error-message)
    (display (simple-format #f "error-message: \n~a" error-message))))


;; Both, output and error:
(receive (exit-code command-output error-message)
    (let ([command "ls -al 2>&1 && echo 'bong' 1>&2"])
      (run-command command))
  (display (simple-format #f "exit code: ~a\n" exit-code))
  (unless (string-null? command-output)
    (display (simple-format #f "command-output: \n~a" command-output)))
  (unless (string-null? error-message)
    (display (simple-format #f "error-message: \n~a" error-message))))
~~~~

Best regards,
Zelphir

--
repositories:https://notabug.org/ZelphirKaltstahl