Lesson 14 of 20

Working with Databases (MongoDB)

MongoDB and Mongoose ODM

MongoDB stores documents rather than rows. A document is essentially a JSON object, documents live in collections rather than tables, and there is no fixed set of columns — two documents in the same collection may have entirely different fields. That flexibility is what people mean by "schemaless", and it is genuinely helpful early on, while you are still discovering what a record needs to contain.

It is also the trap. With no structure enforced, nothing stops one part of your code writing price: 199 while another writes price: '199', and six months later half your prices cannot be compared with the other half. Schemaless does not mean you may skip deciding on a shape. It means the database will not decide it for you, and will not complain when you are inconsistent.

That gap is exactly what Mongoose fills. It is an ODM — an object–document mapper — and it lets you declare a schema in your Node code: which fields exist, what type each one is, which are required, what the defaults are. Mongoose then validates and casts every write against that schema, so the discipline the database declines to enforce is enforced in your application instead.

A schema is compiled into a model, and the model is what you actually use: User.create(), User.find(), User.findById(). Every one of those returns a promise, so this entire lesson is async/await. One convention to know straight away: you name the model in the singular — 'User' — and Mongoose creates a collection called users, lowercased and pluralised. That catches people out the first time they open the database and cannot find their data.

For development you can install MongoDB locally or use a hosted free tier; the hosted route is often easier and has the side benefit of forcing you to keep the connection string in an environment variable from day one. Connect once when the process starts, never inside a route handler. A connection per request is slow, exhausts the server's connection limit under load, and is among the more common mistakes in a first database-backed project.

Example
// npm install mongoose

const mongoose = require('mongoose');

// Connect ONCE at startup — never inside a route handler
mongoose.connect(process.env.DATABASE_URL)   // never hard-code credentials
  .then(() => console.log('Connected to MongoDB'))
  .catch(err => {
    console.error('Connection failed:', err.message);
    process.exit(1);                         // do not serve traffic without a database
  });

// Define a Schema
const userSchema = new mongoose.Schema({
  name: { type: String, required: true, trim: true },
  email: {
    type: String,
    required: true,
    unique: true,
    lowercase: true
  },
  age: { type: Number, min: 0 },
  role: {
    type: String,
    enum: ['user', 'admin'],
    default: 'user'
  },
  createdAt: { type: Date, default: Date.now }
});

// Create a Model
const User = mongoose.model('User', userSchema);

module.exports = User;
  • Document — a JSON-like record; collection — a group of them, in place of a table
  • The database enforces no shape; Mongoose enforces one, in your code
  • Schema — which fields exist, their types, what is required, what defaults apply
  • mongoose.model('User', schema) — the model you actually call methods on
  • A model named 'User' maps to a collection called users
  • Connect once at startup, from process.env.DATABASE_URL, never per request
  • Every model method returns a promise — this whole lesson is async/await
Notes
  • Never write a connection string into your source code. It contains a username and a password, and a repository is a very public place — automated scanners find committed database credentials within minutes of a push, and a free-tier cluster with an open credential is found and abused quickly. Put it in .env, add .env to .gitignore, and commit a .env.example with the key and no value.

CRUD Operations with Mongoose

Every model method returns a promise, so the whole file is async/await wrapped in try/catch. Nine times out of ten the methods you need are create, find, findById, findOne, findByIdAndUpdate and findByIdAndDelete, and the rest of Mongoose can wait until you have a reason to look for it.

find() with no argument returns everything, which is fine with twenty documents and a catastrophe with two hundred thousand. Pair it with a filter, a sort and a limit, always. Note also the difference in what "nothing" looks like: find returns an empty array when nothing matches — a 200 with [] — while findById returns null, and that null is your 404.

findByIdAndUpdate has two options you almost always want, and neither is on by default. { new: true } returns the document as it is after the change rather than before it. { runValidators: true } makes your schema rules apply to updates as well as to creates — without it, a schema that requires an email will happily let an update set that email to an empty string.

Two errors deserve deliberate handling because they arrive constantly. A duplicate value on a unique field comes back as an error whose code is 11000; that is a 409, or a 400 with a clear message, and certainly not a 500. And an id that is not a valid ObjectId — a request for /api/users/abc — throws a CastError before the query even runs, which should be a 400 for exactly the same reason.

Finally, notice the shape every handler shares: await, check for null, choose a status code. Forgetting the await is the classic mistake here, and it is nasty because nothing crashes — res.json(promise) serialises a promise as {}, so your API returns empty objects and you spend an hour blaming the database.

Example
const express = require('express');
const User = require('./models/User');
const router = express.Router();

// CREATE — Add a new user
router.post('/', async (req, res) => {
  try {
    const user = await User.create(req.body);
    res.status(201).json(user);
  } catch (err) {
    if (err.code === 11000) {
      return res.status(400).json({ error: 'Email already exists' });
    }
    res.status(400).json({ error: err.message });
  }
});

// READ — Get all users (with filtering)
router.get('/', async (req, res) => {
  try {
    const { role, sort = '-createdAt' } = req.query;
    const filter = role ? { role } : {};
    const users = await User.find(filter).sort(sort).limit(50);
    res.json(users);
  } catch (err) {
    res.status(500).json({ error: err.message });
  }
});

// READ — Get a single user
router.get('/:id', async (req, res) => {
  try {
    // An id like 'abc' throws a CastError BEFORE the query ever runs
    const user = await User.findById(req.params.id);
    if (!user) return res.status(404).json({ error: 'User not found' });
    res.json(user);
  } catch (err) {
    if (err.name === 'CastError') {
      return res.status(400).json({ error: 'Invalid id format' });
    }
    res.status(500).json({ error: 'Server error' });
  }
});

// UPDATE — Modify a user
router.put('/:id', async (req, res) => {
  try {
    const user = await User.findByIdAndUpdate(
      req.params.id,
      req.body,
      { new: true, runValidators: true }
    );
    if (!user) return res.status(404).json({ error: 'User not found' });
    res.json(user);
  } catch (err) {
    res.status(400).json({ error: err.message });
  }
});

// DELETE — Remove a user
router.delete('/:id', async (req, res) => {
  try {
    const user = await User.findByIdAndDelete(req.params.id);
    if (!user) return res.status(404).json({ error: 'User not found' });
    res.status(204).send();
  } catch (err) {
    res.status(500).json({ error: err.message });
  }
});

module.exports = router;
  • Model.create(doc) — insert, with the schema's validation applied
  • Model.find(filter) — an array; empty means 200 and [], not 404
  • Model.findById(id) — one document or null; that null is your 404
  • findByIdAndUpdate(id, data, { new: true, runValidators: true }) — both options matter
  • err.code === 11000 — duplicate key on a unique field; answer 409, not 500
  • err.name === 'CastError' — a malformed id; answer 400, not 500
  • A forgotten await sends {} without crashing — suspect it first when data looks empty
Notes
  • { new: true } is the most-searched Mongoose option, and the reason is that the default genuinely surprises people: without it, an update returns the document as it was before the change. Your API then answers with the old values, the frontend displays them, and everyone concludes the update silently failed when in fact it worked perfectly.

Designing the Schema

A schema is a design document that happens to be executable. Twenty minutes spent on it before you write any routes saves considerably more time later, because every route in the project inherits the decisions you make here.

The everyday options are worth knowing by heart. type and required do the obvious; required: [true, 'Name is required'] lets you supply the message a user will eventually see. default fills a field that was not sent, enum restricts a string to a fixed set, trim and lowercase normalise input on the way in, and { timestamps: true } maintains createdAt and updatedAt for you, which you will want on nearly every model.

unique: true deserves its own paragraph because it is not what it looks like. It is not a validator — it asks MongoDB to build a unique index. The check therefore happens in the database, the failure arrives as an error with code 11000 rather than as a Mongoose validation error, and, the part that catches everybody, if duplicates already exist in the collection then the index quietly fails to build and the constraint simply is not there.

select: false is the option most first projects are missing. A password hash should never leave the server by accident, and declaring the field with select: false excludes it from every query unless you explicitly ask for it. That turns an accidental leak into something you would have to opt into, which is the right way round for anything sensitive.

Finally, embedding versus referencing. Data that is always read together — an address inside a user — can be embedded directly. Anything with a life of its own, or that many other documents point at, should be referenced with an ObjectId and a ref, then loaded with populate when you need it. The rule of thumb: embed what you always want, reference what you sometimes want, and never embed a list that can grow without bound, because a document has a maximum size and a comment list will find it.

Example
const mongoose = require('mongoose');

const userSchema = new mongoose.Schema({
  name: {
    type: String,
    required: [true, 'Name is required'],
    trim: true,
    maxlength: 80
  },
  email: {
    type: String,
    required: true,
    unique: true,      // an INDEX, not a validator — failures arrive as code 11000
    lowercase: true,
    trim: true
  },
  password: {
    type: String,
    required: true,
    minlength: 8,
    select: false      // never returned unless a query asks for it explicitly
  },
  role: { type: String, enum: ['user', 'admin'], default: 'user' }
}, { timestamps: true });   // createdAt and updatedAt, maintained for you

// Referencing: a book has an identity of its own, so it points at its owner
const bookSchema = new mongoose.Schema({
  title: { type: String, required: true, trim: true },
  owner: {
    type: mongoose.Schema.Types.ObjectId,
    ref: 'User',
    required: true,
    index: true        // you will filter by this constantly
  }
}, { timestamps: true });

const User = mongoose.model('User', userSchema);
const Book = mongoose.model('Book', bookSchema);

// Pulling the reference in when you actually need it
const books = await Book.find({ owner: userId }).populate('owner', 'name email');

// Asking for the hidden field on purpose, in the one place that needs it
const account = await User.findOne({ email }).select('+password');
  • required, default, enum, maxlength, trim, lowercase — declared once, applied everywhere
  • { timestamps: true } — createdAt and updatedAt maintained automatically
  • unique: true builds an index; it is not a validator, and existing duplicates block it
  • select: false on a password hash makes leaking it something you must opt into
  • ObjectId + ref + populate for anything with an independent life
  • Embed what you always read together; never embed a list that grows without limit
  • index: true on the fields you filter or sort by most often
Notes
  • The unique behaviour catches nearly everybody once. You add unique: true to a collection that already holds two identical emails, the index cannot be built, MongoDB reports it somewhere you were not watching, and your application carries on cheerfully accepting duplicates. If a uniqueness rule appears not to work, confirm the index actually exists before you start doubting your own code.

NoSQL Injection and Query Safety

"NoSQL databases cannot be injected" is one of the more expensive myths in web development. MongoDB queries are objects, and if you build a query out of a request body, the client can send an object where you expected a string — and objects are precisely how MongoDB expresses its operators.

The classic example is a login. User.findOne({ email: req.body.email, password: req.body.password }) looks harmless. Now have the client post {"email": "admin@site.com", "password": {"$ne": null}} as JSON. The query becomes "this email, with any password that is not null", which matches, and the attacker is logged in as the administrator without knowing anything. Nothing was escaped wrongly; the query did exactly what it was asked to do.

The fix costs two lines. Force the type — String(req.body.email) — so an object can never arrive where an operator would be interpreted, and validate the input before you query, which you should be doing regardless. A schema library that rejects a non-string email closes this hole at the same moment it closes ten ordinary bugs.

The other query mistake is subtler and does just as much damage. Task.findById(req.params.id) returns the task regardless of who is asking, so anybody who edits the id in the URL reads somebody else's data. This is broken access control, and it is by far the most common serious flaw in student projects. Put the owner into the query itself — Task.findOne({ _id: req.params.id, owner: req.user.id }) — and another person's task is simply not found.

Two more habits. Never spread req.query into a filter object, because that hands the client the ability to query anything by any field, including ones you never meant to expose. And if a value that a user controls must be passed into a query at all, reject keys that begin with $ rather than hoping nobody tries.

Example
// VULNERABLE — a JSON body can send an object where you expected a string
const user = await User.findOne({
  email: req.body.email,
  password: req.body.password
});
// POST {"email":"admin@site.com","password":{"$ne":null}}
//   -> { email: 'admin@site.com', password: { $ne: null } }
//   -> matches, and the attacker is now the administrator

// SAFE — force the types, and never compare the password in the query
const email = String(req.body.email || '').toLowerCase().trim();
const password = String(req.body.password || '');

const account = await User.findOne({ email }).select('+password');
if (!account || !(await bcrypt.compare(password, account.password))) {
  return res.status(401).json({ error: 'Invalid email or password' });
}

// BROKEN ACCESS CONTROL — returns the task to whoever asks for it
const task = await Task.findById(req.params.id);

// CORRECT — ownership is part of the query, so someone else's task is "not found"
const mine = await Task.findOne({ _id: req.params.id, owner: req.user.id });
if (!mine) return res.status(404).json({ error: 'Task not found' });

// NEVER let the client shape the filter
// const tasks = await Task.find({ ...req.query });   // any field, any operator
const filter = { owner: req.user.id };                // build it yourself
if (req.query.status === 'done') filter.completed = true;
  • MongoDB queries are objects, so a JSON body can inject an operator
  • { "$ne": null } in a password field is a working login bypass
  • Force types — String(req.body.email) — and validate before querying
  • Put the owner in the query, not in an if afterwards
  • Never spread req.query or req.body into a filter
  • Answer 404 rather than 403 for someone else's record — do not confirm it exists
Notes
  • Editing the id in a URL to see somebody else's data is the first thing anybody tries, and it takes no skill whatsoever. Every query for a user-owned resource should carry the owner inside the filter rather than checking it afterwards — an ownership check written as a separate condition is one refactor away from being dropped, and nothing will fail loudly when it is.

Errors, Indexes and Habits That Scale

Three Mongoose errors account for most of what your API will meet, and handling them once in the global error handler is far better than repeating the same three if statements in every route. A ValidationError becomes a 400 with the offending fields listed, a CastError becomes a 400, and code 11000 becomes a 409. Everything else is a 500 with the detail logged rather than sent.

.lean() is the cheapest performance improvement available to you. A Mongoose document is a rich object with change tracking and helper methods attached, and building thousands of them costs real time and memory. When you are only going to turn the result into JSON and send it, .lean() returns plain objects instead and is noticeably faster. Use it for any read you do not intend to modify.

Ask for what you need. .select('title author') fetches two fields instead of whole documents, and paginating list endpoints with limit and skip keeps a single request from loading a collection into memory. skip becomes slower on very large collections, which is why big systems paginate by a cursor value, but for a project it is entirely reasonable.

An index is what stops a query from reading every document in a collection. Any field you regularly filter or sort by — owner, email, createdAt — deserves one. Without indexes everything is fast with fifty documents and unusably slow with fifty thousand, which is exactly the kind of problem that only appears after you have shipped and can no longer reproduce it locally.

Finally, never put a query inside a loop. Fetching a list of books and then looking up each author one at a time turns one request into fifty-one, and the pattern is very easy to write without noticing. Use populate, or collect the ids and fetch the whole set with a single find({ _id: { $in: ids } }), then match them up in JavaScript.

Example
// Map database errors once, in the global error handler (lesson 17)
app.use((err, req, res, next) => {
  if (err.name === 'ValidationError') {
    return res.status(400).json({
      error: 'Validation failed',
      details: Object.values(err.errors).map(e => ({ field: e.path, message: e.message }))
    });
  }
  if (err.name === 'CastError') return res.status(400).json({ error: 'Invalid id format' });
  if (err.code === 11000)       return res.status(409).json({ error: 'That value already exists' });

  console.error(err.stack);
  res.status(500).json({ error: 'Internal Server Error' });
});

// A read you will not modify: fewer fields, paginated, plain objects
const books = await Book.find({ owner: req.user.id })
  .select('title author createdAt')
  .sort('-createdAt')
  .skip((page - 1) * limit)
  .limit(limit)
  .lean();

// N+1 — one request quietly becomes fifty-one
// for (const book of books) {
//   book.owner = await User.findById(book.owner);
// }

// One query instead
const owners = await User.find({ _id: { $in: books.map(b => b.owner) } })
  .select('name')
  .lean();
  • Map ValidationError, CastError and code 11000 once, in the error handler
  • .lean() for reads you will not modify — plain objects, much less work
  • .select() the fields you need rather than whole documents
  • Always paginate a list endpoint; limit and skip are fine at project scale
  • Index anything you filter or sort by — owner, email, createdAt
  • Never query inside a loop; use populate or one $in query
Notes
  • Performance problems in a database-backed API almost never come from Node being slow. They come from missing indexes and from queries hidden inside loops, and both are invisible at the scale you develop at. If an endpoint gets slower as your data grows, check those two things before you consider anything else.
Ask AI