# Exceptions & context managers > Design custom exceptions, chain and annotate errors, use try/except/else/finally correctly, and write context managers that always clean up. Python roadmap · Core language · 17 min read · https://levelcode.org/roadmaps/python/exceptions-and-context-managers Beginner code catches `ValueError` from `int()`. Production code has to decide which errors a module raises, how callers tell them apart, how to keep the original cause when wrapping an error, and how to guarantee that files, locks and connections are released whatever goes wrong. This lesson covers designing exceptions, the full `try` statement, exception chaining and groups, and context managers, the `with` statement's machinery for reliable cleanup. ## The exception hierarchy Exceptions are classes arranged in a hierarchy, and `except SomeError` catches that class **and its subclasses**: ```text BaseException ├── KeyboardInterrupt, SystemExit (don't catch these in normal code) └── Exception ├── ValueError │ └── UnicodeDecodeError, json.JSONDecodeError ├── LookupError │ ├── KeyError │ └── IndexError ├── OSError │ ├── FileNotFoundError │ ├── PermissionError │ └── TimeoutError └── ... ``` So `except LookupError` catches both `KeyError` and `IndexError`, and `except OSError` catches every file and network error. Catch the narrowest class that covers what you can actually handle. `except Exception` catches nearly everything. It's legitimate in exactly one kind of place: the **top level** of a program or request handler, where you log the error and keep the server alive. Anywhere else it hides bugs. ## Designing your own exceptions A module or library should raise exceptions that mean something in its own terms. Define a base class for the package and specific subclasses: ```python class PaymentError(Exception): """Base class for errors raised by the payments module.""" class CardDeclined(PaymentError): def __init__(self, reason: str) -> None: super().__init__(f"card declined: {reason}") self.reason = reason class ProviderUnavailable(PaymentError): """The payment provider could not be reached; safe to retry later.""" ``` Callers can then handle precisely what they care about: ```python try: charge(order) except CardDeclined as e: show_message(f"Your card was declined ({e.reason}).") except ProviderUnavailable: schedule_retry(order) except PaymentError: alert_on_call("unexpected payment failure") raise ``` Guidelines: - **Raise early, with a clear message** that includes the offending value: `ValueError(f"quantity must be positive, got {qty}")`. - **Don't invent exceptions the standard library already has.** Bad arguments are `ValueError` or `TypeError`; missing keys are `KeyError`. - **Attach data as attributes** (`e.reason`), so callers don't parse message strings. - A bare `raise` inside `except` re-raises the current exception with its original traceback, after you've logged or cleaned up. ## try, except, else, finally The full statement has four parts: ```python def read_config(path: str) -> dict: try: f = open(path, encoding="utf-8") except FileNotFoundError: return {} # expected: first run, no config yet else: with f: return json.load(f) # runs only if open() succeeded finally: print(f"config lookup for {path} finished") # always runs ``` - `except` runs if the `try` block raised a matching exception. - `else` runs if the `try` block **didn't** raise. Putting follow-up code here, rather than inside `try`, means a `JSONDecodeError` from `json.load` isn't accidentally caught by an `except` meant for `open`. - `finally` runs **no matter what**: success, exception or `return`. It's for cleanup, though as you'll see, a context manager is usually a better way to clean up. ## Chaining: keep the original cause When you translate a low-level error into your own, keep the original with `raise ... from`: ```python import httpx def fetch_rates() -> dict[str, float]: try: response = httpx.get("https://rates.example.com/latest", timeout=5) response.raise_for_status() except httpx.TransportError as e: raise ProviderUnavailable("exchange rate service unreachable") from e return response.json()["rates"] ``` The traceback then shows both, joined by "The above exception was the direct cause of the following exception", so whoever debugs it sees the real network error behind your friendly one. Use `raise NewError(...) from None` only when the original is genuinely irrelevant noise. Python 3.11 added `add_note`, which attaches context without changing the exception type: ```python for line_no, line in enumerate(lines, start=1): try: records.append(parse(line)) except ValueError as e: e.add_note(f"while parsing line {line_no} of {path}") raise ``` The note appears at the end of the traceback, which is often exactly the missing clue. ## Exception groups Sometimes several things fail at once, such as concurrent tasks or validating many fields. Python 3.11 added `ExceptionGroup` to raise them together, and `except*` to handle each kind: ```python def validate(user: dict) -> None: errors: list[Exception] = [] if "@" not in user.get("email", ""): errors.append(ValueError("invalid email")) if user.get("age", 0) < 0: errors.append(ValueError("age must not be negative")) if "name" not in user: errors.append(KeyError("name")) if errors: raise ExceptionGroup("invalid user", errors) try: validate({"email": "nope", "age": -1}) except* ValueError as group: for e in group.exceptions: print("value problem:", e) except* KeyError as group: print("missing fields:", [str(e) for e in group.exceptions]) ``` ```text value problem: invalid email value problem: age must not be negative missing fields: ["'name'"] ``` Unlike `except`, every matching `except*` clause runs, each receiving the subgroup of exceptions it matches. You'll meet exception groups mostly through `asyncio.TaskGroup` in the async lesson. ## Context managers: cleanup you can't forget Every resource that must be released, such as files, locks, database connections and temporary directories, has the same shape: acquire, use, release, even if the use fails. The `with` statement packages that: ```python with open("data.txt", encoding="utf-8") as f: process(f) # f is closed here, whether process() returned normally or raised ``` An object works with `with` if it has two methods. `__enter__` runs at the start, and its return value is bound by `as`. `__exit__` runs at the end, receiving details of any exception: ```python import time class Timer: def __enter__(self) -> "Timer": self.start = time.perf_counter() return self def __exit__(self, exc_type, exc, tb) -> None: self.elapsed = time.perf_counter() - self.start print(f"took {self.elapsed:.3f}s") # returning None (falsy) lets any exception propagate with Timer() as t: sum(range(10_000_000)) ``` If `__exit__` returns `True`, the exception is suppressed. Do that only deliberately. ### Writing one with contextlib For most context managers, a generator is shorter. `@contextmanager` turns the code before `yield` into `__enter__` and the code after into `__exit__`: ```python from collections.abc import Iterator from contextlib import contextmanager import os @contextmanager def working_directory(path: str) -> Iterator[None]: previous = os.getcwd() os.chdir(path) try: yield finally: os.chdir(previous) # runs even if the with-block raised with working_directory("/tmp"): print(os.getcwd()) # /tmp print(os.getcwd()) # back where you were ``` The `try`/`finally` around `yield` is essential: without it, an exception inside the `with` block would skip the cleanup. `contextlib` has ready-made ones too: ```python from contextlib import ExitStack, suppress with suppress(FileNotFoundError): # "delete if it exists" os.remove("cache.tmp") with ExitStack() as stack: # a variable number of resources files = [stack.enter_context(open(p, encoding="utf-8")) for p in paths] merge(files) # all closed on exit, in reverse order ``` Other standard context managers you'll use: `threading.Lock` (`with lock:`), `tempfile.TemporaryDirectory()`, database transactions in most drivers (`with conn.transaction():`), and `unittest.mock.patch` in tests. > [!TIP] > If you find yourself writing `try: ... finally: something.close()`, look for a context manager instead. If none exists, write one: it puts acquire and release next to each other, in one tested place, instead of in every caller. ## Practice Write `atomic_write(path)`, a context manager for safely replacing a file: 1. Inside the `with` block, the caller writes to a file object as usual. 2. The data is actually written to a temporary file in the same directory. 3. If the block finishes without an exception, the temporary file replaces the target in one step (`os.replace`), so readers never see a half-written file. 4. If the block raises, the temporary file is deleted, the original file is untouched, and the exception propagates. Then use it to save JSON, and test that an exception halfway through leaves the old file intact. **Solution** ```python import json import os import tempfile from collections.abc import Iterator from contextlib import contextmanager from typing import TextIO @contextmanager def atomic_write(path: str, encoding: str = "utf-8") -> Iterator[TextIO]: directory = os.path.dirname(os.path.abspath(path)) fd, tmp_path = tempfile.mkstemp(dir=directory, prefix=".tmp-") try: with os.fdopen(fd, "w", encoding=encoding) as tmp: yield tmp tmp.flush() os.fsync(tmp.fileno()) # make sure the bytes reached the disk os.replace(tmp_path, path) # atomic on the same filesystem except BaseException: os.unlink(tmp_path) raise # Normal use with atomic_write("settings.json") as f: json.dump({"theme": "dark"}, f) # A failure halfway leaves the old file intact try: with atomic_write("settings.json") as f: f.write('{"theme": ') raise RuntimeError("crashed mid-write") except RuntimeError: pass with open("settings.json", encoding="utf-8") as f: print(json.load(f)) # {'theme': 'dark'} ``` The temporary file is created in the **same directory** as the target because `os.replace` is only atomic within one filesystem; a temp file in `/tmp` might live on a different one. Catching `BaseException` (not just `Exception`) means even Ctrl+C mid-write cleans up the temporary file, and the bare `raise` re-raises the original exception unchanged. The guarantee comes from `os.replace`: at every instant, `settings.json` is either the complete old version or the complete new one. ## Key takeaways - `except` catches a class and its subclasses: catch the narrowest class you can handle, and `Exception` only at top-level boundaries. - Give a module its own exception hierarchy with data in attributes, and re-raise with bare `raise` after logging. - Use `else` for code that should run only when `try` succeeded, and `finally` for unconditional cleanup. - Preserve causes with `raise ... from e`, add context with `add_note`, and handle concurrent failures with `ExceptionGroup` and `except*`. - Context managers pair acquire and release; write them with `@contextmanager` and a `try`/`finally` around `yield`. ## Quiz 1. Which clause catches both KeyError and IndexError? A. except ValueError B. except LookupError C. except OSError D. except TypeError **Answer: B.** KeyError and IndexError are both subclasses of LookupError, and except matches subclasses. 2. Why put follow-up code in the else block of a try statement rather than inside try? A. It runs faster B. else always runs, even after an exception C. Exceptions raised by the follow-up code won't be accidentally caught by except clauses meant for the try block D. Python requires it for file operations **Answer: C.** Keeping the try block minimal means each except clause only handles errors from the operation it was written for. 3. What does raise ProviderUnavailable("...") from e achieve? A. It suppresses the original exception completely B. It retries the operation C. It converts e into a warning D. It raises the new exception while recording e as its direct cause, shown in the traceback **Answer: D.** Chaining keeps the original error visible for debugging while giving callers an exception in your module's own terms. 4. In a @contextmanager function, why wrap yield in try/finally? A. So the cleanup code runs even if the with-block raises an exception B. So the yielded value can't be None C. So the exception is always suppressed D. It's required syntax for generators **Answer: A.** If the with-block raises, the exception is thrown into the generator at the yield. Without finally, the cleanup code after yield would be skipped.