[Day 6] A Connection Came In — and a New fd Appeared

Yesterday in Day 5, while digging into why rm doesn't immediately reclaim disk space, I wrote this line:

"Files, pipes, sockets — they're all fds."

But that was just words. The only thing I actually touched in Day 5 was a file. Do sockets really show up as fds? I wanted to see it with my own eyes.

And while doing that, I ran into something I didn't expect — one connection came into the server, and instead of one fd, two fds appeared. Why?

That's today's story.

---

1. Setup: Spinning Up a Server with nc

nc (netcat) is the Linux "Swiss Army knife of networking." A one-liner that creates a socket. If cat streams a file's contents to the terminal, nc streams contents across the network. Same philosophy, different medium.

Here's the important thing to get straight:

nc is a program that creates sockets. nc itself is not a socket.

To spell out the relationship:

• nc is a program

• When it runs, it becomes a process

• That process creates a socket

Fire up the server in terminal 1:

$ nc -l 9999

-l = listen. It waits on port 9999 for someone to connect. The cursor just blinks. That's correct.

Open terminal 2 for observation. Save the PID to a variable:

\( NC_PID=\)(pgrep -x nc)
\( echo \)NC_PID
3211711

(pgrep -x nc finds processes whose name is exactly nc. Without -x, you'd also catch things like rpcbind, sync, or containerd — anything with "nc" in the name.)

---

2. Peeking at the fds While in Listening State

Just like in Day 5 where I peeked at the tail process via /proc/PID/fd/, I did the same for nc:

\( ls -l /proc/\)NC_PID/fd/
total 0
lrwx------ 1 frog frog 64 Apr 21 15:16 0 -> /dev/pts/1
lrwx------ 1 frog frog 64 Apr 21 15:16 1 -> /dev/pts/1
lrwx------ 1 frog frog 64 Apr 21 15:16 2 -> /dev/pts/1
lrwx------ 1 frog frog 64 Apr 21 15:16 3 -> 'socket:[12134645]'

First observation right here:

• fd 0, 1, 2 → /dev/pts/1 (a file path)

• fd 3 → 'socket:[12134645]' (a socket)

Files and sockets coexist inside the same fd namespace. The terminal device (/dev/pts/1) is a "file" as far as Linux is concerned. This is the famous Linux philosophy:

"Everything is a file."

What I'd only said in words in Day 5, I was now seeing with my eyes.

---

3. The Socket's Identity Looks a Little Weird

That 'socket:[12134645]' next to fd 3 looks unfamiliar. If this had been a file, I'd see something like /home/user/big2.txt — a path. But sockets don't have paths. Why?

Makes sense when you think about it. A file has an actual location somewhere on disk, so it can be expressed as a path. But sockets don't exist on disk. They live only in memory, as kernel data structures.

So how do you address something that doesn't exist on disk? Linux gives sockets an inode number too. Just like files.

As I saw in Day 5, a file's real identity isn't the "name" but the "inode." Sockets follow the same rule. Since they have no name, they're identified by socket:[inode_number] instead of a path.

12134645 is this socket's social security number — a globally unique ID within the kernel.

---

4. Cross-Checking the Same Socket with Three Tools

"Is this really the same socket across tools?" I wanted to verify.

Tool 2: ss -ltnp (listening TCP sockets)

$ ss -ltnp | grep 9999
LISTEN 0 1 0.0.0.0:9999 0.0.0.0:* users:(("nc",pid=3211711,fd=3))

Reading the rightmost column:

• "nc" → program name

• pid=3211711 → PID of the process

• fd=3 → the fd number that points to this socket inside that process

Tool 3: lsof -i :9999 (everything related to port 9999)

$ sudo lsof -i :9999
COMMAND     PID USER FD   TYPE   DEVICE SIZE/OFF NODE NAME
nc      3211711 frog 3u   IPv4 12134645      0t0  TCP *:9999 (LISTEN)

The DEVICE column shows 12134645 — the exact same inode number I saw in /proc.

Putting the three views side by side:

Tool	Perspective	Representation
/proc/PID/fd/	process	fd 3 → socket:[12134645]
ss -ltnp	network	0.0.0.0:9999 ... fd=3
lsof -i :9999	unified	PID 3211711, FD 3, DEVICE 12134645

Same socket. Different windows. From any angle, 12134645 shows up — it's the socket's SSN.

Worth distinguishing two concepts that are easy to confuse:

• fd is the number this process uses to refer to the socket. Different per process.

• inode is the socket's globally unique ID across the whole kernel. Same in every view.

fd is a nickname, inode is a social security number. That analogy stuck with me.

---

5. Now the Real Question: What Happens When a Connection Comes In?

With the server still running, I opened terminal 3 and connected:

$ nc localhost 9999

Then back in terminal 2, I checked the server's fds again:

\( ls -l /proc/\)NC_PID/fd/
total 0
lrwx------ 1 frog frog 64 Apr 21 15:16 0 -> /dev/pts/1
lrwx------ 1 frog frog 64 Apr 21 15:16 1 -> /dev/pts/1
lrwx------ 1 frog frog 64 Apr 21 15:16 2 -> /dev/pts/1
lrwx------ 1 frog frog 64 Apr 21 15:16 3 -> 'socket:[12134645]'
lrwx------ 1 frog frog 64 Apr 21 16:13 4 -> 'socket:[12134646]'

An extra fd appeared. fd 3 is still there, and fd 4 is brand new. Different inode too (12134645 → 12134646).

Which raises the question:

If a connection came in, why not just use fd 3? Why create fd 4?

---

6. Listening Socket ≠ Connected Socket

Here's the answer. They're both called "sockets," but their roles are completely different.

• fd 3 (listening socket): "The reception desk at port 9999, waiting for new connections."

• fd 4 (connected socket): "The actual conversation channel with the client that just connected."

Picture a restaurant:

• fd 3 is the host at the door — the one who greets guests.

• fd 4 is the table that guest gets seated at — where the actual orders and service happen.

If the host abandoned their post to serve one table, who'd greet the next guest? Nobody. So the host (listening socket) has to stay at the door, and each new guest gets their own new table (connected socket).

Which means:

• fd 3 stays in listening state (ready to accept more connections)

• fd 4 is newly created to handle communication with terminal 3

This is the core pattern of a TCP server. It's how a web server serves many clients at once: the listening socket stays alive, and every incoming connection spawns a new fd (= a new connected socket).

---

7. The Two Ends of a Connection Have Different inodes

Here's another thing that surprised me. I checked the client-side nc in terminal 3:

$ ls -l /proc/3217244/fd/
0 -> /dev/pts/3
1 -> /dev/pts/3
2 -> /dev/pts/3
3 -> 'socket:[12159488]'

Server side: socket:[12134646]. Client side: socket:[12159488]. Different inodes.

Wait — if two processes open the same file, they share the same inode. Why don't the two ends of a socket?

A file is one piece of data on disk that multiple processes share by reading it. But a connected socket has its own send/receive buffers on each side. "Stuff to send" buffer and "stuff to receive" buffer on my side, and another pair on their side. These two pairs are wired together over the network. That's all a TCP connection is, at the kernel level.

ss -tnp | grep 9999 makes this even clearer:

ESTAB 0 0 127.0.0.1:9999  127.0.0.1:39010 users:(("nc",pid=3211711,fd=4))
ESTAB 0 0 127.0.0.1:39010 127.0.0.1:9999  users:(("nc",pid=3217244,fd=3))

Two rows, not one. The two ends of a single TCP connection are registered as two separate socket objects in the kernel.

That's why the inodes differ. Same "pipe," two ends, but each end is a separate managed resource.

---

8. Connecting Back to Day 5

Applying Day 5's reference count rule here gets interesting:

inode reference count = (number of names) + (number of open fds)

Sockets have no "name" (no directory entry), so their reference count is determined entirely by the number of open fds.

• If fd 3 (listening) closes → port 9999 is released. No new connections possible.

• If fd 4 (connected) closes → the connection to terminal 3 is terminated.

What if the server forgets to close(fd)? That's an fd leak. Same mechanism as the 500MB file story from Day 5. Eventually you hit Too many open files.

So when I said "files, pipes, sockets are all fds," the point isn't just that they look alike. It's that the kernel manages them by the same rules. Opening increments a reference count. Closing decrements it. When it hits zero, the resource is reclaimed. Whether the resource is a file or a socket.

---

9. Why This Matters in Practice

If you've ever debugged a web server seeing Accept queue overflow, understanding this structure is what makes the error legible.

• The listening socket's queue (fd 3) is full — ss -ltnp's Send-Q column shows that queue's capacity (on my system, 4096 or 128 depending on the process).

• Every new connection spawns a connected socket (fd 4, 5, 6, ...). When this exceeds ulimit -n, accept() starts failing.

• This is also why network monitoring separates ss -tnp (connected) from ss -ltnp (listening) — they're completely different sockets.

Coming from a backend background and pivoting to infra, once this structure becomes visible, things like "why is the server slow," "why is the connection pool exhausted," and "why am I getting Too many open files" all start to link into one picture.

---

Quick Reference

Command	What it does
nc -l PORT	Start a listening server on that port
nc HOST PORT	Connect to HOST on that port
pgrep -x PROG	Find PID of process with exact name match
ss -ltnp	Show listening TCP sockets
ss -tnp	Show connected TCP sockets (includes ESTAB)
lsof -i :PORT	Show all processes/fds related to that port
ls -l /proc/PID/fd/	Show every fd a process is holding

Core mental model:

Listening socket (fd 3) = the host at the door. Stays alive.
Connected socket (fd 4, 5, 6, ...) = a new table for each guest.

The two ends of a connected socket have different inodes.
→ Because each side maintains its own send/receive buffers.

fd vs. inode:

fd    = the number this process uses (nickname)
inode = the globally unique ID in the kernel (social security number)

---