Reading and Writing Files
The fs module is where Node.js stops being browser JavaScript. A web page cannot open a file on your disk; a Node program can read, write, rename, delete and inspect anything the operating system will let its user touch. That power is the reason Node is used for build tools, log processors, data-cleaning scripts and file uploads, not just servers.
Every operation in fs comes in three flavours, and choosing between them is most of what this lesson is about. There is a callback form (fs.readFile), a promise form (fs.promises.readFile, also importable as node:fs/promises), and a synchronous form ending in Sync (fs.readFileSync). They do exactly the same work; they differ only in how they hand the answer back and in whether they block the event loop while doing it.
The callback form is the original, and it follows a convention you will see everywhere in Node: the callback's first argument is the error, and the data comes second. This is called an error-first callback. It exists because in asynchronous code you cannot simply throw — by the time the file read finishes, the try/catch block that started it has long since exited. The error therefore has to be delivered as data, and Node's convention is to put it first so that it is hard to ignore.
The 'utf8' argument matters more than it looks. Leave it out and you do not get a string — you get a Buffer, Node's representation of raw bytes. Logging a Buffer prints something like <Buffer 48 65 6c 6c 6f> rather than the text you expected. That is not a bug; it is Node refusing to guess the character encoding of arbitrary bytes. Pass 'utf8' for text files, and omit it deliberately when you are handling images, PDFs or anything else that is not text.
Note also that writeFile replaces the entire file rather than adding to it. If you meant to add a line to a log, writeFile will silently destroy everything already there and appendFile is what you wanted. This is a quietly expensive mistake, because nothing errors — the data is simply gone.
const fs = require('fs');
// --- Asynchronous (non-blocking) ---
// Reading a file
fs.readFile('message.txt', 'utf8', (err, data) => {
if (err) {
console.error('Error reading file:', err.message);
return;
}
console.log('File contents:', data);
});
// Writing a file (creates or overwrites)
fs.writeFile('output.txt', 'Hello, Node.js!', (err) => {
if (err) throw err;
console.log('File written successfully');
});
// Appending to a file
fs.appendFile('log.txt', 'New log entry\n', (err) => {
if (err) throw err;
console.log('Data appended');
});
// --- Synchronous (blocking) ---
const data = fs.readFileSync('message.txt', 'utf8');
console.log(data); fs.readFile(path, 'utf8', cb)— read a file; without'utf8'you get a Bufferfs.writeFile(path, data, cb)— create or overwrite the whole filefs.appendFile(path, data, cb)— add to the end, creating the file if needed- Error-first callbacks — the first argument is always the error, the data comes second
fs.readFileSync()— same job, but nothing else in your program runs until it finishes
writeFileoverwrites without warning. When you want to add a line to a log or a CSV, the method isappendFile. Getting this wrong destroys the existing contents and produces no error at all.
Blocking vs Non-Blocking: When Sync Is Actually Fine
"Never use the Sync methods" is advice you will read constantly, and it is nearly right but not quite. The accurate version is: never use them where they block someone else. Understanding when that is true stops you from writing awkward asynchronous code for startup tasks where it buys nothing.
Recall that all your JavaScript runs on one thread. fs.readFileSync tells that thread to stop and wait for the disk. On a server handling requests, this is a catastrophe in slow motion: while your handler waits for a 40 MB file, every other user's request sits untouched in the queue. Ten users hitting that route at once do not get ten parallel reads — they get ten sequential ones, each waiting for the previous. A page that felt fine when you tested it alone falls apart under a class of students clicking at the same time.
But there is a case where blocking costs nothing: startup. When your program is loading a configuration file or reading a certificate before app.listen() is ever called, there are no other requests to block. Nobody is waiting. Writing that as a promise chain adds noise for no benefit, and using readFileSync makes the loading order obvious and lets you crash cleanly if the file is missing. This is why you will see readFileSync at the top of well-written production code and it is not a mistake.
The same logic applies to command-line scripts and build tools. A script that renames two hundred files and then exits has no concurrent users; if the sync version is simpler to read, use it. The rule is about who is waiting, not about the method name.
One last variation worth knowing: even the asynchronous fs methods do not truly run in parallel on the disk. Node performs them on an internal thread pool, and by default that pool is small. Firing off two hundred simultaneous file reads will not be two hundred times faster than doing them one at a time. For bulk file work, process in batches rather than mapping over ten thousand filenames and awaiting them all at once.
// FINE — startup, before the server accepts anything
const fs = require('node:fs');
const config = JSON.parse(fs.readFileSync('./config.json', 'utf8'));
// If this throws, the app should not start anyway.
app.listen(config.port);
// HARMFUL — inside a request handler
app.get('/report', (req, res) => {
const data = fs.readFileSync('./big-report.csv', 'utf8'); // freezes everyone
res.type('text/csv').send(data);
});
// CORRECT — the same route, non-blocking
const fsp = require('node:fs/promises');
app.get('/report', async (req, res, next) => {
try {
const data = await fsp.readFile('./big-report.csv', 'utf8');
res.type('text/csv').send(data);
} catch (err) {
next(err);
}
}); - Sync is fine: reading config at startup, one-off scripts, build tools, tests
- Sync is harmful: anything inside a route handler, middleware, or an event that fires per request
- The question is never "is this file small" but "is anyone else waiting on this thread"
- Async
fswork runs on a limited internal thread pool — thousands of parallel reads will not all run at once - For bulk file jobs, process in batches instead of awaiting one enormous array of promises
- A useful smell test during code review: search the project for
Sync(. Every hit that is not in a startup file, a script or a test deserves a second look.
fs.promises and Async/Await
The promise-based version of the same API is what you should reach for in new code. Import it as require('node:fs/promises'), or as require('node:fs').promises, and every method returns a promise instead of taking a callback. Combined with async/await, file code stops looking like a staircase of nested callbacks and starts reading top to bottom like ordinary instructions.
The gain is not only cosmetic. With callbacks, each step needs its own error check, and forgetting one means a failure vanishes silently. With await inside a try/catch, a failure at any step jumps straight to the catch, in exactly the way you would expect from synchronous code. That is the real argument for promises: errors behave the way your instincts say they should.
The example below is a small file-processing script of the kind you actually write — read a file, transform it, save the result, make sure an output folder exists, then report on what was produced. Read it as a sequence: nothing nests, and one catch at the bottom covers every step.
Notice how existence is checked. There is no fs.exists worth using; the promise API gives you fs.access, which resolves if the file can be reached and rejects if it cannot, so you wrap it in its own small try/catch. And note the wider point: checking whether a file exists and then reading it is two operations with a gap in between, during which the file can disappear. In most cases the better pattern is simply to attempt the read and handle the ENOENT error, which has no such gap.
fs.mkdir with { recursive: true } deserves a mention of its own. It creates any missing parent folders in one call and, importantly, does not complain if the folder is already there. Without that option, creating a directory that already exists rejects with EEXIST, and you end up writing a pointless check around it.
const fs = require('fs').promises;
const path = require('path');
async function processFiles() {
try {
// Read a file
const data = await fs.readFile('input.txt', 'utf8');
console.log('Read:', data);
// Write to a file
await fs.writeFile('output.txt', data.toUpperCase());
console.log('Written successfully');
// Check if file exists
try {
await fs.access('config.json');
console.log('Config file exists');
} catch {
console.log('Config file not found');
}
// Create a directory
await fs.mkdir('logs', { recursive: true });
// List files in a directory
const files = await fs.readdir('.');
console.log('Files:', files);
// Get file info
const stats = await fs.stat('output.txt');
console.log('Size:', stats.size, 'bytes');
} catch (err) {
console.error('Error:', err.message);
}
}
processFiles(); node:fs/promises— the promise API; use this in new codemkdir(dir, { recursive: true })— create nested folders, and do not fail if they existreaddir(dir)— list names inside a folder; add{ withFileTypes: true }to tell files from foldersstat(path)— size, timestamps, andisDirectory()unlink(path)— delete a file (the name comes from the underlying system call, not from links)rename(from, to)— rename or move, but only within the same disk volume
- Prefer attempting an operation and catching its error over checking first and then acting. Between the check and the action, another process can create, delete or lock the file, and the check will have told you a comfortable lie.
Streams: Reading Files Too Big for Memory
readFile loads the whole file into memory before giving you anything. For a 5 KB configuration file that is exactly right. For a 2 GB log file it is a way to crash your server, because Node will try to allocate two gigabytes of memory to hold a string you are going to read line by line and throw away.
The alternative is a stream: instead of one enormous value, the file arrives in chunks, and you handle each chunk as it comes. Memory use stays flat no matter how large the file is, because you only ever hold one chunk at a time. This is the same idea as watching a video online rather than downloading the whole film before pressing play.
The example below counts error lines in a log file. Written with readFile, its memory use grows with the file. Written with createReadStream and Node's built-in readline module, it uses about the same memory for a 2 KB file and a 2 GB one. On a server, that difference decides whether a large upload takes the whole application down.
Streams are also how you should send large files to a client. Reading a 500 MB video into memory and then calling res.send holds all of it in RAM for every simultaneous viewer. Piping a read stream into the response instead lets Node push bytes out as it reads them, and pipeline handles the awkward parts — closing both ends properly, and surfacing an error from either side instead of leaving a half-written file and a hanging request.
You do not need streams on day one, and it is fine to use readFile for the small files in this course. What you do need is the instinct to ask "how large can this file get?" before choosing. Anything a user can upload has no natural limit, and that is exactly where the question matters.
const fs = require('node:fs');
const readline = require('node:readline');
const { pipeline } = require('node:stream/promises');
// Count error lines in a log of ANY size — memory stays flat
async function countErrors(path) {
const stream = fs.createReadStream(path, { encoding: 'utf8' });
const lines = readline.createInterface({ input: stream, crlfDelay: Infinity });
let errors = 0;
for await (const line of lines) {
if (line.includes('ERROR')) errors++;
}
return errors;
}
// Send a large file to the client without buffering it in memory
app.get('/download/:name', async (req, res, next) => {
const filePath = path.join(FILES_DIR, path.basename(req.params.name));
try {
await pipeline(fs.createReadStream(filePath), res);
} catch (err) {
next(err);
}
});
// Copy while transforming, again without loading the whole file
await pipeline(
fs.createReadStream('input.csv'),
someTransformStream,
fs.createWriteStream('output.csv')
); createReadStream/createWriteStream— process a file in chunksreadlineover a read stream — the standard way to handle a file line by linepipelinefromnode:stream/promises— connects streams and reports errors from any of them- Use
readFilewhen the size is small and known; use streams when it is user-supplied or unbounded - Streaming a response keeps memory flat even with many simultaneous downloads
- Do not connect streams with a bare
.pipe()chain in production code. If the source fails halfway,.pipe()leaves the destination open and the error unhandled.pipelineexists specifically to clean up both ends and give you one place to catch the failure.
File Errors You Will Actually Hit
File errors in Node arrive with a short uppercase code property, and that code is far more useful than the message when you are writing error handling. Comparing err.code === 'ENOENT' is reliable; matching on the text of the message is not, because messages change between versions and vary by platform.
ENOENT — "error: no entry" — means the path does not exist. In practice it almost always means the path is not what you think it is, rather than that the file is genuinely missing. Relative paths in Node resolve against the folder the process was started from, not the folder the source file lives in. Run node src/app.js from the project root and a readFile('data.json') inside src/app.js looks for the file in the project root. The fix is the next lesson's subject: build paths from __dirname.
EACCES means the operating system refused on permission grounds. Common on Linux servers when your application runs as a low-privileged user and tries to write into a folder owned by root. The correct fix is to give the application a folder it owns, not to run the application as root.
EISDIR means you asked to read something that turned out to be a folder, and ENOTDIR is the reverse. EEXIST comes from creating something that already exists — usually mkdir without { recursive: true }. EMFILE means too many files are open at once, which is what happens when a loop opens files and never closes them, or when you fire off tens of thousands of reads simultaneously.
There is one more error that is not really a file error but shows up here constantly: SyntaxError: Unexpected token ... in JSON from JSON.parse. The file was read successfully; its contents are simply not valid JSON. When you write configuration loaders, catch parse failures separately from read failures, because "config.json is missing" and "config.json has a trailing comma on line 12" need very different messages to whoever has to fix it.
const fsp = require('node:fs/promises');
async function loadConfig(path) {
let raw;
try {
raw = await fsp.readFile(path, 'utf8');
} catch (err) {
if (err.code === 'ENOENT') {
throw new Error(`Config file not found at ${path}`);
}
if (err.code === 'EACCES') {
throw new Error(`No permission to read ${path}`);
}
throw err; // something unexpected — do not swallow it
}
try {
return JSON.parse(raw);
} catch {
throw new Error(`${path} exists but is not valid JSON`);
}
}
// Why ENOENT usually means "wrong folder", not "missing file"
console.log(process.cwd()); // where you RAN node from
console.log(__dirname); // where THIS FILE lives
// A relative path is resolved against the first, which is rarely what you meant. ENOENT— path does not exist; suspect the working directory before the fileEACCES— the operating system denied permissionEISDIR/ENOTDIR— you treated a folder as a file, or the other way roundEEXIST— creating something already there;mkdirwants{ recursive: true }EMFILE— too many open files, usually an unbounded loop of reads- Check
err.code, never the message text
- Never write
catch {}with an empty body around a file operation. A swallowedENOENTturns "the upload folder is missing" into "uploads mysteriously vanish", and you will spend an afternoon on a problem the error message had already solved.
