In Python 2.4b1 I am trying to use the subprocess module and 
running into problems when args is a string and executable is 
specified.

For example:

>>> p = subprocess.Popen(
  executable = "xpaget",
  args = "xpaget ds9 mode",
  shell = True,
  stdout = subprocess.PIPE,
)
results in this mysterious error msg:
>>> sh: illegal option -- c
and the data in p.stdout is a boatload of help text
that strongly suggests xpaget never got the "ds9 mode" command.

Piping stdin and stderr make no difference, of course,
but omitting the stdout makes one strange difference:
I don't see the error message, just the boatload of help text.


Removing the executable argument makes it work as expected:
>>> p = subprocess.Popen(
  args = "xpaget ds9 mode",
  shell = True,
  stdout = subprocess.PIPE,
)
>>> p.stdout.read()
'pointer\n'

And the executable argument works fine if I specify the arguments as 
a list and don't use shell=True:
p = subprocess.Popen(
  executable = "xpaget",
  args = ["xpaget", "ds9", "mode"],
  stdout = subprocess.PIPE,
)
>>> p.stdout.read()
'pointer\n'

xpa and ds9 are free from <http://hea-www.harvard.edu/RD/ds9/>
but I hope think they are not needed to debug this.

I saw this problem using a unix installation of Python 2.4b1 on MacOS 
X 10.3.5.

Logged In: YES 
user_id=431773

I looked at the code and the problem is here:

        def _execute_child(self, args, executable, preexec_fn, close_fds,
                           cwd, env, universal_newlines,
                           startupinfo, creationflags, shell,
                           p2cread, p2cwrite,
                           c2pread, c2pwrite,
                           errread, errwrite):
            """Execute program (POSIX version)"""

            if isinstance(args, types.StringTypes):
                args = [args]

            if shell:
                args = ["/bin/sh", "-c"] + args

            if executable == None:
                executable = args[0]

You can se that if shell is true (as it must be if one specifies args as a string) 
and executable is supplied, one ends p with a mess, i.e. the executable ends 
up trying to execute  ["/bin/sh", "-c", args]

Fortunately cwd offers a hackaround. Still, I hope this can be either fixed or 
else can produce a warning.

While on that subject, should specifying args as a string without specifying 
shell=True should produce a warning?

Thanks for subprocess. It's a wonderful module. I am really looking forward 
to using it (once a few kinks get worked out).

Logged In: YES 
user_id=344921

Russell: What did you expect? Shells doesn't support that
you specify an alternative executable, only the raw system
calls do. 

Currently, if you use shell=True, the executable argument
can be used to specify an alternative shell, if you are not
happy with the default /bin/sh. I think this is useful. 

I must admit that this is not very well documented, though. 

Logged In: YES 
user_id=344921

reowen: 

>While on that subject, should specifying args as a string
without
>specifying shell=True should produce a warning?

No. This is documented:

"A string will be treated as a sequence with the string as
the only item (the program to execute)."

>Thanks for subprocess. It's a wonderful module.

Thanks :)

Logged In: YES 
user_id=431773

What did I expect? Well, obviously I expected it to "just work", but I don't 
have much idea what's going on "under the hood" and have always been 
puzzled by the need in os.popen... to specify the executable twice. And I 
have even less idea what's going on under the hood on Windows!

I was trying to use it in my code to work around bug 1057048 which applies 
to Windows. I figured if an executable with a space in its path name 
wouldn't get through as part of the command string, it might get through 
when specified as a separate 
argument.

So...yes if possible I think it would be very good to document what happens 
if executable and 
shell=True are both specified at the same time. Especially since it may be 
radically different on Windows and unix.

It'd also be nice to say in the module header that list2cmdline is part of this 
module, i.e. "If args is a sequence, it will beconverted to a string using the 
list2cmdline method." could be changed to: "If args is a sequence, it will be 
converted to a string using list2cmdline (below). I had the impression it was 
a standard module and failed miserably finding any documentation about it. 
Probably seems silly, since you know where to look, but I bet I'll not be the 
only one.

Logged In: YES 
user_id=344921

I've documented how the executable argument works when
shell=True. Committed in revision 1.2 of libsubprocess.tex.