17

I started down this rabbit hole as a means to familiarize myself with how one would go about creating a setup script in python. The choice of python was simply rooted in my familiarity with it while I am sure there would be better alternatives than python for this task.

The goal of this script was to install ROS onto the machine running the script and also setup the catkin environment. Directions can be found here and here, respectively.

The script as it currently sits is as follows:

subprocess.call(["sudo", "sh", "-c", "'echo \"deb http://packages.ros.org/ros/ubuntu $(lsb_release -sc) main\" > /etc/apt/sources.list.d/ros-latest.list'"])
subprocess.call(["sudo", "apt-key", "adv", "--keyserver", "hkp://ha.pool.sks-keyserver.net:80", "--recv-key", "0xB01FA116"])
subprocess.call(["sudo", "apt-get", "update"])
subprocess.call(["sudo", "apt-get", "install", "ros-kinetic-desktop-full", "-y"])
subprocess.call(["sudo", "rosdep", "init"])
subprocess.call(["rosdep", "update"])
subprocess.call(["echo", '"source /opt/ros/kinetic/setup.bash"', ">>", "~/.bashrc", "source", "~/.bashrc"])
subprocess.call(["sudo", "apt-get", "install", "python-rosinstall", "-y"])
mkdir_p(os.path.expanduser('~') + "/catkin_ws/src")
subprocess.call(["(cd "+ os.path.expanduser('~') + "/catkin_ws/src)"])
subprocess.call(["(cd "+ os.path.expanduser('~') + "/catkin_ws && catkin_make)"])
subprocess.call(["(cd "+ os.path.expanduser('~') + "/catkin_ws && source devel/setup.bash"])

When the script is currently runs it errors out with the error:

Traceback (most recent call last):
  File "setup.py", line 46, in <module>
    subprocess.call(["(cd "+ os.path.expanduser('~') + "/catkin_ws/src)"])
  File "/usr/lib/python2.7/subprocess.py", line 523, in call
    return Popen(*popenargs, **kwargs).wait()
  File "/usr/lib/python2.7/subprocess.py", line 711, in __init__
    errread, errwrite)
  File "/usr/lib/python2.7/subprocess.py", line 1343, in _execute_child
    raise child_exception
OSError: [Errno 2] No such file or directory

I have verified that the command does correctly work when manually executed from a terminal window, and as such I believe this is a fundamental misunderstanding about how this script and its scope are handled within the OS. The part that is causing me a lot of confusion is why it complains that it is unable to locate the provided directory, while I have verified that this directory exists. When the command is rather printed from python and pasted into a terminal window no errors are encountered.

2
  • Python has its own os.chdir() Commented Jul 22, 2016 at 6:09
  • 1
    If you're using Python 3, just pass cwd argument to call
    – intsco
    Commented Oct 30, 2019 at 12:43

4 Answers 4

24

By default subprocess.call doesn't use a shell to run our commands you so can't shell commands like cd.

To use a shell to run your commands use shell=True as parameter. In that case it is recommended to pass your commands as a single string rather than as a list. And as it's run by a shell you can use ~/ in your path, too:

subprocess.call("(cd ~/catkin_ws/src && catkin_make)", shell=True)
8
  • 1
    Thank you! I was under the impression that subprocess.call used a shell, and was unaware that it had to be explicitly stated. The above command worked exactly as intended
    – beeedy
    Commented Jul 22, 2016 at 0:04
  • 1
    Why not use os.chdir()? Commented Jul 22, 2016 at 6:08
  • 3
    How about subprocess.call(['catkin_make'], cwd=os.path.expanduser('~/catkin_ws/src'))? Commented Jul 22, 2016 at 6:44
  • shell=True will call default shell, which is dash. If a script that OP contains bashisms it may break. I've added edit to my answer, alternative solution would be to explicitly call specific shell. Especially useful if someone is dealing with csh script Commented Jul 22, 2016 at 8:53
  • 1
    The best solution is Matt Nordhoff suggestion . Using shell=True even with fixed commands opens security vulnerabilities (e.g. shellshock could be triggered on a vulnerable system). The rule of thumb: if you can avoid using shell=True you should avoid it. The cwd parameter is there exactly to do the kind of call the OP wants.
    – Bakuriu
    Commented Jul 22, 2016 at 10:24
6

subprocess.call() expects a list, with first item obviously being a legitimate shell command. Compare this for instance:

>>> subprocess.call(['echo hello'])
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.7/subprocess.py", line 523, in call
    return Popen(*popenargs, **kwargs).wait()
  File "/usr/lib/python2.7/subprocess.py", line 711, in __init__
    errread, errwrite)
  File "/usr/lib/python2.7/subprocess.py", line 1343, in _execute_child
    raise child_exception
OSError: [Errno 2] No such file or directory
>>> subprocess.call(['echo', 'hello'])
hello
0

In your case , subprocess.call(["(cd "+ os.path.expanduser('~') + "/catkin_ws/src)"]) will expect to find binary that looks like so(note backslash designating space charater):

 cd\ /home/user/catkin_ws/src

That is treated as one single name that is to be expected to live somewhere on your system. What you really would wanna do is:

 subprocess.call(["cd", os.path.expanduser('~') + "/catkin_ws/src"])

Note that I have removed parenthesis around the comma, as there is no reason to use subshell.

EDIT:

But it has been already mentioned by progo in the comments that using cd in this case is redundant. Florian's answer also properly mentions that subprocess.call() doesn't use shell. You could approach that in two ways. One , you could use subprocess.call("command string",shell=True)

The other way , is to call specific shell explicitly. This is especially useful if you want to run a script that requires specific shell. Thus you could do:

subprocess.call(['bash' , os.path.expanduser('~')  + "/catkin_ws/src"  ) ] )
3
  • 1
    call() doesn't expect a legitimate shell command; it expects to find a path to an actual executable. And calling a standalone cd doesn't achieve anything: the CWD is a process specific variable that ceases to exist once the process exits.
    – mike3996
    Commented Jul 22, 2016 at 8:06
  • @progo good point , I was so focused on editing OP's command that I didn't even notice that cd won't do anything here. . . . But as for "legitimate" , it still is appropriate phrasing I believe - if I give subprocess.call() something it cannot find , like ['ls -l'] , it won't be legitimate Commented Jul 22, 2016 at 8:20
  • @progo made a small edit, please review Commented Jul 22, 2016 at 8:42
4

Use os.chdir() instead.

Apart from the issues, mentioned in the existing answers, I wouldn't prefer using shell=True, nor subprocess.call() here to change directory.

Python has its own way of changing directory in os.chdir() (don't forget to import os). ~ ("home") can be defined in several ways, a.o. os.environ["HOME"].

Reasons to prefer that over shell=True can be read a.o. here

0

Note that using os.chdir() can cause unintended side-effects, e.g. if you are using multithreading. subprocess methods all provide a cwd keyword argument that will run the requested subprocess in that directory, without affecting other parts of your python process.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .