Ray Shell

Most of us who use Linux will be using terminal for their everyday programming. We type ls and files are shown. We type cat /etc/passwd | awk -F: '{print $1 " -> " $7}', we see users mapped with their default shell, we just spam Tab for command autocompletes. But how? How are files read and processed? and How does it even autocomplete?

As a person who loves looking under the hood, I took some time to learn and understand about how shell works. And what better way for a developer to understand things, We build it. So that's exactly what I did. CodeCrafters.io has this wonderful interactive shell building exercise and it even had tests to check if we implemented the features as intended.

What do you mean I can't use .split(" ")?

We all know how echo command works. According to the echo's man page description,

Echo the STRING(s) to standard output.

echo hello world 
# hello world

Looks pretty simple enough build a parser by splitting strings by spaces, right? What about something like this

echo "spaces    will    work" spaces    wont    work    now 
# spaces    will    work spaces wont work

See the problem now? That's when I realized splitting by spaces will no longer work and we have to build a lexer for parsing.

They don't just run?

One of big moment was realizing that a shell doesn't actually "run" programs, it gives birth to them. To run an external command like grep or wc, the shell can't just jump to that code or it would lose its own state.

• fork(): The shell creates a literal clone of itself (child process).
• exec(): The child process replaces its own memory space with the new program's code.
• wait(): The parent (shell) pauses until the child finishes, collecting its exit code.

Where is the binary, is it here?

We just type ls, but where is it's binary and how does shell know about it? That's were $PATH environment variable comes in. We have to implement a manual lookup system that reads the $PATH environment variable, splits it into directories (delimited by colons), and iterate all the binaries and check for executable permissions and all these happens in milliseconds. Autocomplete is implemented the same way but with a lot more iterations.

Reinventing the wheel?

Did I reinvent the wheel? Absolutely. Did I learn things that I have never learnt before? Yes, a lot. I ventured into the deep grounds where keystrokes interacts with software, it was one hell of a ride. The contract between the kernel and the software will never seize to amaze me and that's why I will keep building and practice these dark arts.