Part 6 – Mastering File System Operations

Node.js provides powerful file system capabilities through the fs module. This guide covers everything from basic file operations to advanced streaming.

1. FS Module Overview

Synchronous vs Asynchronous

• Async: Non-blocking (preferred)
• Sync: Blocking (use cautiously)
• Promise-based: fs.promises API

Common Use Cases

• Reading/writing files
• Directory operations
• File metadata
• Watching for changes

2. Basic File Operations

Reading Files

// Async callback version
fs.readFile('file.txt', 'utf8', (err, data) => {
    if (err) throw err;
    console.log(data);
});

// Promise version (modern)
async function readFile() {
    try {
        const data = await fs.promises.readFile('file.txt', 'utf8');
        console.log(data);
    } catch (err) {
        console.error(err);
    }
}

Writing Files

// Write new file
await fs.promises.writeFile('newfile.txt', 'Hello World!');

// Append to file
await fs.promises.appendFile('log.txt', 'New log entry\n');

3. Working with Directories

// Create directory
fs.mkdir('new-folder', { recursive: true }, (err) => {
    if (err) throw err;
});

// Read directory contents
const files = await fs.promises.readdir('./');
console.log(files);

// Remove directory
await fs.promises.rmdir('empty-folder');

4. File Metadata and Stats

const stats = await fs.promises.stat('file.txt');
console.log(`
    Is file: ${stats.isFile()}
    Is directory: ${stats.isDirectory()}
    Size: ${stats.size} bytes
    Created: ${stats.birthtime}
    Modified: ${stats.mtime}
`);

5. Streaming Large Files

Read Stream

const readStream = fs.createReadStream('large-file.mp4');

readStream.on('data', (chunk) => {
    console.log(`Received ${chunk.length} bytes`);
});

readStream.on('end', () => {
    console.log('Finished reading');
});

Write Stream

const writeStream = fs.createWriteStream('output.mp4');

readStream.pipe(writeStream);

writeStream.on('finish', () => {
    console.log('Finished writing');
});

6. Watching for File Changes

// Watch single file
fs.watch('config.json', (eventType, filename) => {
    console.log(`${filename} changed (${eventType})`);
});

// Watch directory recursively
fs.watch('src', { recursive: true }, (event, filename) => {
    console.log(`Change detected in ${filename}`);
});

7. Practical File System Examples

Example 1: Log File Rotator

async function rotateLogs() {
    const logDir = './logs';
    const files = await fs.promises.readdir(logDir);

    for (const file of files) {
        if (file.endsWith('.log')) {
            const stats = await fs.promises.stat(`${logDir}/${file}`);
            if (stats.size > 1024 * 1024) { // 1MB
                const newName = `${file}.${Date.now()}.bak`;
                await fs.promises.rename(
                    `${logDir}/${file}`,
                    `${logDir}/${newName}`
                );
                await fs.promises.writeFile(`${logDir}/${file}`, '');
            }
        }
    }
}

Example 2: Directory Backup

const { pipeline } = require('stream');
const zlib = require('zlib');

async function backupDirectory(src, dest) {
    const archiveName = `${path.basename(src)}-${new Date().toISOString()}.tgz`;
    const output = fs.createWriteStream(`${dest}/${archiveName}`);
    const tarStream = tar.create({ gzip: false }, [
        { cwd: src, read: false, file: '.' }
    ]);

    await new Promise((resolve, reject) => {
        pipeline(
            tarStream,
            zlib.createGzip(),
            output,
            (err) => {
                if (err) reject(err);
                else resolve();
            }
        );
    });
}

8. File System Best Practices

• Always handle errors – File operations can fail for many reasons
• Use streams for large files – Avoid memory issues
• Validate paths – Prevent directory traversal attacks
• Consider file locks – For concurrent access scenarios
• Clean up temp files – Use tools like tmp-promise

Next: Introduction to Express.js →

FS Module Cheat Sheet

• fs.readFile/fs.writeFile – Whole file operations
• fs.createReadStream/createWriteStream – Streaming
• fs.stat – File metadata
• fs.watch – File change notifications
• fs.promises – Promise-based API