NFSDrive

A distributed network file system. Files live on storage servers spread across machines, a naming server keeps track of which server holds what, and clients work against one shared namespace without knowing or caring where a file physically sits.

Every export is mounted into a single path tree. A client reads /data/reports/q3.csv the same way whether that file is on one server or one of fifty, and it keeps working when the server holding it goes down.

For architecture, the wire protocol, and setup, see DEVDOC.md.

Features

One namespace across many servers

• Each storage server exports a local directory at a mount point, and everything under it appears in the global tree at that point.
• Clients address files by global path only. Nothing in a client command names a host, a port, or a directory on disk.
• ls lists the whole namespace, or any subtree of it, across every server.
• Adding a storage server while the system is running makes its files available immediately. No restart, no configuration change anywhere else.
• The naming server can be restarted underneath a running cluster. Storage servers notice and re-register themselves, and the namespace rebuilds within a few seconds without anyone touching them.

File operations

• create, mkdir, delete, rmdir, copy, read, write, put, echo and stat.
• create builds any missing parent directories, so making a file several levels deep is one command.
• copy handles files and whole directory trees, including copying between two different storage servers, and copying into an existing directory places the source inside it rather than replacing it.
• read prints to stdout or saves to a local file; write appends, put replaces. Both take a local file or stdin.

Files of any size

• Transfers are streamed in chunks, so a file is never held in memory in one piece and there is no size beyond which a transfer stops working.
• A large write is acknowledged as soon as the data is safely on disk, with the final commit finishing in the background. The client is told which of the two happened rather than being left to guess.
• Writes are staged and swapped into place atomically. A transfer that dies half way leaves the previous contents intact rather than a truncated file.

Concurrent access

• Many clients can read the same file at once.
• One writer at a time per file. A second writer is told the file is busy immediately instead of being left to wait.
• Locking is per file, so a slow write to one file does not block anything on another.

Redundancy and failover

• A storage server can run as a replica of another. It mirrors the primary's namespace and contents.
• A replica joining a primary that already holds files is seeded with them, rather than starting empty and staying behind.
• Every change to a primary is mirrored to its replicas, including file contents written directly by clients.
• When a primary stops answering, reads are served from a live replica and the replica is promoted so writes keep working. When the primary comes back it reclaims its namespace.
• A write is never accepted by a stand-in replica that has not been promoted, because the primary would come back not knowing about it.

Operational visibility

• servers shows every storage server: endpoint, mount point, online state, whether it is a primary or a replica, and how many paths it holds.
• Every component logs to stderr and optionally to a file, with one line per event and a configurable level.
• Errors are named, not numbered at the user: "directory not empty" rather than a bare code, with the code alongside for scripts.

Built for scripting

• Any command can be run directly from the shell, not just from the interactive prompt: nfs-client read /data/notes.txt.
• Exit status is zero on success and non-zero on failure, so commands compose in a script without output parsing.
• --quiet suppresses progress notes so stdout carries only the result.

Components

Component	What it does
Naming server	Holds the namespace, routes requests, tracks which storage servers are alive, drives replication and failover. Never carries file data.
Storage server	Holds the files. Serves reads and writes directly to clients. Runs as a primary or, with one flag, as a replica of another.
Client	Interactive shell and one-shot command runner.

A request, end to end

Reading /data/reports/q3.csv:

1. The client asks the naming server who owns the path.
2. The naming server walks its path tree, finds the storage server that mounted /data, checks it is alive, and returns its address. If it is down and a replica is up, it returns the replica instead.
3. The client connects to that storage server directly and asks for the file.
4. The storage server takes a shared lock on the path and streams the contents back in chunks.

File data goes straight between the client and the storage server. The naming server is only ever consulted for metadata, so it does not become the bottleneck as the number of clients grows.

Getting started

make

Then, in three terminals:

bin/nfs-naming-server
bin/nfs-storage-server --root ~/nfs-data --port 8801 --mount /data
bin/nfs-client

nfs> create /data/notes.txt
nfs> echo /data/notes.txt hello there
nfs> read /data/notes.txt
hello there
nfs> ls /data

Full setup, including replicas and multi-machine deployment, is in DEVDOC.md.

Tech stack

C11 on POSIX, with pthreads and BSD sockets. No external dependencies: the build needs a C compiler and make, and nothing else.