Python File Handling: Reading and Writing Files Safely

Quick Answer

Use open() inside a with statement so the file closes automatically even if an error occurs. Mode r reads, w writes and truncates the file immediately, a appends, and x creates but fails if the file exists. Always pass encoding='utf-8' explicitly, because the default varies by platform and causes text to break on someone else's machine. To update a file safely, write to a temporary file and replace the original.

Always Use with

You can open and close a file manually, but there is a reason nobody recommends it.

# Fragile: if anything raises in between, close() never runs
f = open('data.txt', 'r')
content = f.read()
f.close()

# Correct: closes automatically, even on an exception
with open('data.txt', 'r', encoding='utf-8') as f:
    content = f.read()

The with statement guarantees the file is closed when the block ends, whether it finishes normally or raises. That matters more than tidiness: on Windows an open handle prevents the file being renamed or deleted, and unflushed writes can be lost entirely if the program exits unexpectedly.

Three ways to read, each suited to a different situation:

with open('data.txt', encoding='utf-8') as f:
    everything = f.read()          # whole file as one string

with open('data.txt', encoding='utf-8') as f:
    lines = f.readlines()          # list of lines, newlines included

with open('data.txt', encoding='utf-8') as f:
    for line in f:                 # one line at a time — memory friendly
        process(line.rstrip('\n'))

The loop is the one to default to. read() loads the entire file into memory, which is fine for a config file and disastrous for a two-gigabyte log.

The Modes, and the One That Destroys Data

The mode argument decides what happens the moment the file opens — before you write anything.

• r — read. Raises FileNotFoundError if missing. The default.
• w — write. Truncates the file to empty immediately, or creates it.
• a — append. Writes go to the end; creates if missing.
• x — exclusive create. Fails if the file already exists.
• r+ — read and write, without truncating.

The dangerous one is w. Opening an existing file in write mode empties it at that instant. If your program then crashes, or you were only inspecting the file, the contents are already gone.

# If this line runs, important.txt is now empty. No write required.
f = open('important.txt', 'w')

So when you mean to add to a file, use a. When you must not overwrite something that exists, use x and let it fail loudly rather than checking first — checking then opening leaves a gap where another process could create the file in between.

Add b for binary — rb, wb — for images, PDFs and anything that is not text. Reading binary data in text mode will raise a decoding error or silently corrupt it.

Always Specify the Encoding

This is the bug that only appears on someone else's computer.

If you omit encoding, Python uses a platform-dependent default. On most Linux and macOS systems that is UTF-8. On many Windows installations it has historically been a regional codepage such as cp1252. So a file written on one machine and read on another can produce mojibake or an outright crash.

# Fine on your machine, breaks on a colleague's
with open('names.txt') as f: ...

# Explicit and portable
with open('names.txt', encoding='utf-8') as f: ...

The symptom is usually a UnicodeDecodeError mentioning a byte position, or names in Hindi, Bengali or Tamil appearing as question marks and boxes.

Two practical extras. When writing CSV files, pass newline='' as well, or the csv module produces a blank line between every row on Windows:

with open('out.csv', 'w', encoding='utf-8', newline='') as f:
    writer = csv.writer(f)

And if you genuinely must read a file with unknown or broken encoding, errors='replace' substitutes an unreadable character instead of raising — useful for salvaging a log, never for data you intend to keep.

Updating a File Without Losing It

A common task: read a file, change something, save it back. The obvious approach has a real failure mode.

# Risky: if anything fails after opening 'w', the original is gone
with open('data.txt', encoding='utf-8') as f:
    lines = f.readlines()

with open('data.txt', 'w', encoding='utf-8') as f:   # truncated here
    f.writelines(transform(lines))                    # crash here = data lost

The file is emptied before the new content is written. A crash, a full disk or a power cut in between leaves you with nothing.

The safe pattern is to write a temporary file and replace the original only once it is complete:

import os

with open('data.txt', encoding='utf-8') as f:
    lines = f.readlines()

tmp = 'data.txt.tmp'
with open(tmp, 'w', encoding='utf-8') as f:
    f.writelines(transform(lines))

os.replace(tmp, 'data.txt')     # atomic on the same filesystem

os.replace is atomic, so the file is either the old version or the new one and never a half-written mixture. This is the same technique databases and editors use, and it is worth making a habit for anything you would be upset to lose.

Paths and Structured Data

Use pathlib rather than string concatenation. Joining paths with + and slashes breaks across platforms.

from pathlib import Path

p = Path('data') / 'reports' / 'may.txt'   # correct separator everywhere
print(p.exists(), p.suffix, p.stem)

text = p.read_text(encoding='utf-8')       # shortcut for small files
p.write_text(text.upper(), encoding='utf-8')

Path.mkdir(parents=True, exist_ok=True) creates a directory tree without raising if it is already there — the usual first line before writing output.

For JSON, use the json module rather than parsing by hand.

import json

with open('config.json', encoding='utf-8') as f:
    config = json.load(f)          # load reads a file object

with open('config.json', 'w', encoding='utf-8') as f:
    json.dump(config, f, indent=2, ensure_ascii=False)

Note load and dump work with files, while loads and dumps work with strings — the trailing s means string, and mixing them up is a frequent small error. Passing ensure_ascii=False keeps non-English characters readable instead of escaping them.

For CSV use the csv module, which handles quoting and embedded commas that a naive split(',') gets wrong.

Frequently Asked Questions