Parsing Config Files Safely in Python

Introduction¶

Config parsing has two hazards: an unsafe loader that executes code, and code that assumes the file's contents are the right shape. Both are avoidable.

The formats¶

YAML — always safe_load¶

import yaml

with open("config.yaml") as f:
    config = yaml.safe_load(f)     # NOT yaml.load

yaml.load (without a safe loader) can instantiate arbitrary Python objects — !!python/object/apply:os.system ["rm -rf /"] in a config file is remote code execution. safe_load only produces dicts, lists, strings, numbers, bools, None. There is no reason to use the unsafe loader for config.

TOML — stdlib since 3.11¶

import tomllib      # read-only, stdlib

with open("config.toml", "rb") as f:      # note: binary mode
    config = tomllib.load(f)

Writing TOML needs a third-party lib (tomli-w, tomlkit). TOML is a good choice for human-edited config — unambiguous types, comments, no significant whitespace.

JSON — stdlib, no comments¶

import json
config = json.loads(path.read_text())

Fine for machine-generated config; painful for humans (no comments, trailing commas rejected).

INI — configparser, everything is a string¶

import configparser
cp = configparser.ConfigParser()
cp.read("config.ini")
port = cp.getint("server", "port")        # explicit conversion; cp["server"]["port"] is str

Never trust the file's types¶

A config file gives you strings, and whatever structure the author typed. port might be "8080", 8080, missing, or null. Validate into a typed model:

With pydantic-settings (recommended)¶

from pydantic import Field
from pydantic_settings import BaseSettings, SettingsConfigDict

class Settings(BaseSettings):
    model_config = SettingsConfigDict(
        env_prefix="MYAPP_",
        env_nested_delimiter="__",
    )
    host: str = "0.0.0.0"
    port: int = Field(default=8080, ge=1, le=65535)
    debug: bool = False
    database_url: str
    allowed_origins: list[str] = []

settings = Settings()      # raises a clear ValidationError listing every problem

Pydantic coerces "8080" → 8080, "true" → True, validates ranges, and reports all errors at once with paths. It also layers env vars over defaults automatically.

Layering: env over file over defaults¶

Precedence, low to high: hardcoded defaults → config file → environment variables → explicit CLI flags. A small merge:

def load_config(path: Path | None) -> Settings:
    file_data = {}
    if path and path.exists():
        file_data = yaml.safe_load(path.read_text()) or {}
    # pydantic-settings reads env vars itself; pass file data as the base
    return Settings(**file_data)

Env vars win so deployments can override without editing files (and secrets stay out of the file).

Plain-stdlib validation with a schema¶

import jsonschema      # pip install jsonschema

SCHEMA = {
    "type": "object",
    "required": ["database_url"],
    "properties": {
        "port": {"type": "integer", "minimum": 1, "maximum": 65535},
        "database_url": {"type": "string"},
    },
    "additionalProperties": False,
}
jsonschema.validate(config, SCHEMA)

Verification and troubleshooting¶

# Fail fast at startup, with a readable message
try:
    settings = Settings()
except ValidationError as e:
    print(e)          # lists field, error, and input value
    sys.exit(1)

• yaml.load() missing Loader argument — newer PyYAML deprecated the unsafe default. Use yaml.safe_load.
• TypeError: a bytes-like object is required with tomllib — open in binary mode ("rb").
• Config value is a string when you expected a number — INI/env always give strings; validate/coerce. Pydantic does this; configparser needs getint/getboolean.
• additionalProperties / extra rejected a key — a typo in the config, or a new option you forgot to add to the model. Pydantic's model_config with extra="forbid" catches typos; extra="ignore" is more forgiving.
• Env var not overriding — wrong prefix/case, or nested key needs the delimiter (MYAPP_DATABASE__HOST). Settings().model_dump() shows the resolved values.
• Secrets in the config file — move them to env vars or a secrets manager; keep the file for non-sensitive structure. SecretStr in pydantic prevents accidental logging.
• Different behaviour in prod — env vars present there override the file silently. Log the resolved config (redacting secrets) at startup.

Related tools and reading¶

• On-site: YAML validator, JSON formatter, TOML / JSON tools.
• Related posts: subprocess without shell injection, Type hints that actually pay off.