Streaming Files

Streaming moves a file in chunks instead of loading the whole thing into memory at once. The bytes flow straight from storage to the client as they arrive. It's drinking through a straw instead of swallowing the whole bottle. You take it a sip at a time, no matter how big the bottle is.

Streaming isn't an S3 feature. It's a general idea that shows up all over: HTTP request and response bodies, reading a file off disk, piping between processes, and network sockets. S3 just happens to hand you a stream.

Why stream

• A 2 GB video loaded into memory might crash your server; streamed, it barely registers
• Users see the first bytes of a video or PDF immediately, not after a full download
• Ten concurrent downloads cost the same memory as one

How to use it

import { GetObjectCommand } from "@aws-sdk/client-s3";

app.get("/videos/:id", async (req, res) => {
  const object = await s3.send(new GetObjectCommand({
    Bucket: "my-app-uploads",
    Key: `videos/${req.params.id}.mp4`,
  }));

  res.setHeader("Content-Type", "video/mp4");

  // Body is a readable stream; pipe it straight to the HTTP response.
  // Chunks flow through to the client and are never all held in memory at once.
  object.Body.pipe(res);
});

When to stream (and when not to)

Streaming should be your default: stream unless you have a concrete reason not to. If you find yourself reading a whole file into a variable before sending it, that's the signal to stream instead. The exceptions are narrow:

• The file is tiny and you need it all in memory anyway (parsing a small JSON or CSV, transforming the whole thing).
• You need random access, jumping around the file rather than reading start to finish.
• A library or API you're handing the data to only accepts a full buffer.

Exercise