Below are references for my talk on the history of the shell. January 10th, 2024 at SLUG Base presentation. Included for further reading if you want to go further. For the purposes of this post, a Shell in a modern context is both a Command Line Interface and also a scripting language. The latter aspect is not extensively covered here but any modern shell is a proper programming language. In fact, they are Turing Complete which boils down to the statement: Anything that can be computed by any Turing Machine, of which all Turing complete languages reduce down to, can be also be computed by any other Turing complete language. Anything that can be computed will be computed. ... eventually in Javascript. (Apologies to Murphy) You could, for example, write a flight simulator in the Bourne shell. I wouldn't recommend it, though. About the term 'shell': You may have heard other things besides CLIs referred to as shells. such as the Gnome shell and the Windows shell. The term probably dates back to when OS internal services were first referred as kernels, like the kernel of a nut. The 'shell' was that user land layer that wraps kernel services like process and filesystem resources.So, the term is overloaded and here we are referring to shells that are also command line interpreters. Application programs may also have CLIs. Frequently they occur in programming languages like Python, Ruby and Javascript (Also BeanShell for Java written by a St. Louisan) These are also referred to as the REPL, or the Read-Eval-Print-Loop. This term probably goes back to the first Lisp interpreters. (1964 on the PDP-1 for Lisp) https://en.wikipedia.org/wiki/Read%E2%80%93eval%E2%80%93print_loop A Unix/Linux/Windows shell can also be referred to as a REPL. The Read function collects strings from the terminal (or a file if running a script) It then parses it to some internal format (usually an Abstract Syntax Tree or AST) The AST is then fed to the eval function which walks this tree and executes commands and functions encountered therein. (Note: The 'eval' command in Bash and other shells combines these 2 functions 'read' and 'eval' into one statement. E.g. $ eval "echo hello" hello Finally, the output is processed by the printer (the 'print' function) and maybe further formatted. (E.g. internal integers are converted to their string counterparts) And then we loop again to start the whole process again. Primary source. Coders at work (2009), by Peter Seibel. https://www.amazon.com/Coders-Work-Reflections-Craft-Programming/dp/1430219483 Features interviews with Ken Thompson, Donald Knuth and Fran Allen (First female winner of the ACM Turing Award) The book is also available in audio but while good the voices of the interviewees are not their own. Different narraters are chosen. Comparisons of shells (Wikipedia.org) https://en.wikipedia.org/wiki/Comparison_of_command_shells This was a late entry (thanks, Stan) but was useful. Covers from Thompson shell through PowerShell. Early history. Timeline of operating systems https://en.wikipedia.org/wiki/Timeline_of_operating_systems Project Whirlwind (1949 - 1959) A collaboration between the U.S. Navy and M.I.T. which had done a lot of work for the War department in WWII. The idea was to build a super computer and begun in 1944 to design a aircraft simulator to train Navy pilots. The machine eventually supported 50 KIPs. Contrast to the 10 Trillion neural IPS in the Apple A17. The project evolved into ATC (Air Traffic Control) and air defense. Within the Defense Dept, Whirlwind evolved into S.A.G.E., a giant project to track incoming Russian nuke equipped bombers via the D.E.W. line. The Whirlwind machine at MIT is said to pioneered the use of magnetic core main memory. The machine did not have a keyboard input but did feature one or more CRT displays for output. One of the engineers on the project was Ken Olson who later founded Digital Equipment Corporation. More on D.E.C. later. Later, another MIT professor, Fernando Corvabo who worked on the Whirlwind project would contribute to both timesharing and the MULTICS project. [ Note how much the influence of New England geography was involved in much of computer evolution. ] A video presentation on Project Whirlwind for the MIT + The Computer History Museum: https://www.youtube.com/watch?v=ou53_Z99wi8 Resident Monitors https://en.wikipedia.org/wiki/Resident_monitor Probably the first example of interfacing a TeleType hard copy terminal to an operating system to command functions. IPL Or, Initial Program Load. Before there was bootstrapping. Bootstrapping refers to pulling oneself up by their own boot straps. Usually, this requires a permanent ROM thatprobably chains into a boot loader off some disk drive that really launchs the actual OS. In early mainframes, after a cold start, a menu or just a simple prompt asked for the sysop to load the operating system There is a scene in Wargames where the character played by Mathew Broderick is asked if he wants to play a game. The movie essentially reverts back to this screen on the big screen in the underground SAC HQ. https://en.wikipedia.org/wiki/Booting Go to the heading for IBM S/360 and later mainframes. The IMP Internet Message Processor on the Honeywell 515. https://en.wikipedia.org/wiki/Interface_Message_Processor The actual first one at UCLA: https://www.youtube.com/watch?v=yU9oMOcRsuE BBC mini-documentrary on IMP0: https://www.youtube.com/watch?v=C1y3TA-DF_4 One of the coders interviewed in the book "Coders at work" above was one of 3 programmers on this dsystem. The story of hard copy patching of paper tape was very entertaining. Early timesharing systems CTSS (or the Compatible Time Sharing System) (M.I.T. 1961 ..) Christopher Strachey wrote a paper on sharing time across multiple user programs delivered in 1959. John McCarthy wrote a memo while at MIT proposing such a system. Fernando Corvabo worked on this project. It was called the Experimental Time Sharing System. Eventually, this system was called the Compatible Time Sharing System,or CTSS. It ran on a new IBM 7090 mainframe the university just acquired. Some main storage was dedicated to the system executive. The ETSS had only enough memory for 3 users plus the system executive. One of the first applications written for CTSS was a Braille translator that could output to a Braille embosser printer. One of the first versions of Elizaa a AI powered psycologist program ran on CTSS. CTSS used passwords for user logins as concieved by Corvabo. RUNCOM Or Run Command (1964) was implemented on CTSS and allowed for executing commands with argument substitution (a major step, IMHO). This was a kind of scripting language. RUNcOM was later enhanced in the MULTICS system. CTSS also sported the qed text editor a forerunner of the ed text editor. Each user had their own subdirectory and files could be symlinked. Dennis Richie, in 1977, wrote the Unix could be seen as a modern version of CTSS. https://en.wikipedia.org/wiki/Compatible_Time-Sharing_System Interview with Fernando Corvabo on how time sharing worked in CTSS. https://www.youtube.com/watch?v=Q07PhW5sCEk The Hercules Mainframe emulator http://www.hercules-390.org/ This allows one to run IBM software for mainframes as far back as the S/370 such as C.I.C.S. From the installation page above, apparently you start it up (after configuration) as a daemon and then connect a TN3270 terminal emulator to that IP address at port 3270 (Slick!) You might be able to also connect a plain old vanilla telnet program to that ip:port combo (again depending on configuration) Here is further help on getting a TN3270 client for various platforms: https://en.wikipedia.org/wiki/3270_emulator (Keep in mind that whenever you see 'TNXXX the TN refers to telnet. Thus, TN3270 is the 3270 terminal protocol over Telnet) (The 3270 protocol is block oriented not character or line oriented like we are used to.) JCL or Job Control Language This fulfills our requirement for a shell to be a scripting language. However, it is not interactive, which is our other requirement. [ In my (albeit limited) experience, actual SysOps actually wrote the JCL cards that were prepended to my batch programming jobs. I remember that I filled out a form and rubber-banded it around the entire deck. I imagined, but can not be certain, that once they punched the JOB card #1, they just grabbed either a COBOL or FORTRAN compile and run deck from some JCL library. ] From the horse's mouth: https://www.ibm.com/docs/en/zos-basic-skills?topic=collection-basic-jcl-concepts 3 types of JCL statements - JOB statement : Identifies the job and accounting info - EXEC statements : What actions to perform - DD statements : Data Definition of data sets to be used. This YouTube channel is the realy comprehensive Mainframe topics channel: https://www.youtube.com/channel/UCR1ajTWGiUtiAv8X-hpBY7w There are a lot of long form vids explaining JCL practical use cases. If you wan to play around with the Hercules Mainframe emulator above, you will have to know how to create and submit JCL, which is still in use today. C.I.C.S. https://en.wikipedia.org/wiki/CICS A multithreaded terminal server for IBM terminals and various "open source" applications. A lot of OLTP applications were developed for CICS, like airline reservations, .etc. C.I.C.S on the cloud https://www.ibm.com/docs/en/cics-tx/10.1.0?topic=overview-what-is-cics-tx-cloud You can deploy CICS services via Docker containers. It is also possible to access CICS via web browser interface. Command line interfaces https://en.wikipedia.org/wiki/Command-line_interface#History One thing I forgot was that CLIs like command.com on both D.R.s CP/M and MS-DOS were originally modeled on DECs RSX-11 operating system shell. (1972) The Realtime System Executive ran on PDP 11s. (PDP 11/40) https://en.wikipedia.org/wiki/RSX-11#bodyContent The CLI for RSX-11 was DCL (Gigital Command Language) https://en.wikipedia.org/wiki/DIGITAL_Command_Language A cross-DEC OS command line parser and executive. It uses the venerable 'verb [object1, ... objectn] and options/flags. style syntax we have come to love so well. It lacked while or any other looping requiring script writers to fall back to if/then/else with goto lable statements. Dartnouth Timesharing System Influenced by John McCarthy at M.I.T., Thomas Kurtz, a math professor decided that his college needed this new-fangled time sharing thing. You might remember him as one of the 2 co-founders (with John Kemeny) of B.A.S.I.C. or the Beginner's All purpose Symbolic Instruction Code language. Steve Garland was one of the grad students who were actively recruited to implement the TSS. this link has many nice pix from the time frame. https://www.dartmouth.edu/library/rauner/exhibits/sharing-the-computer.html The DTSS used 2 G.E. computers one for the programs and another to handle all the Teletype terminals. GECOS General Electric Comprehensive Operating System (1962-1964) [For information on the GECOS field in /etc/passwd see: https://www.quora.com/What-is-Gecos-in-Linux] A true first Second Generation OS featuring multiprogramming or multitasking and dynamic allocation of memory (as opposed to IBM arch which only supported fixed sized memory partitions) This allowd it do both Timesharing and batch processing. [ First generation OS only supported batch processing] Designed for the 36 bit 600 series mainframes made by GE. Later, MULTICS supported running GECOS jobs (both timesharing and batch jobs) in an emulator. MULTICS Professor Corvabo headed up the MIT portion that worked on the Operating System itself. Written in PL/1; probably the first OS to be written entirely in a high level programming language. The standard way to access files on Multics was to use memory mapping. Had full hierarchical directory structure and pathnames. Since based on GE 645 arch, characters were 6 bits. Programs were written in PL/1 and dynamically linked to the current running program: The user's shell. So, a program was actually a function in the current process. Command names: in the Multics shell - cwd : Change working directory - cd : create dir - print_wd : Print working dir; short name: pwd - list: list contents of dir; short name: ls - quit: Quit and log out Probably the simplest and most consise video on using Multics: https://www.youtube.com/watch?v=bjG4m77PQm8 The v6 shell or the Thompson shell (1971) This ink is from the re-implementation of V6 Unix called: xv6 and can be compiled and run using Qemu. https://github.com/mit-pdos/xv6-public.git If you clone this, you will see the sh.c file in the root folder. They have abandoned the x86 port and are only concentrating on the RISC V ISA. I am trying to compile it on its own, but gcc on the MacOS barfs on some redeclarations. But maybe you can read the sh.c code and get a feel for how the slightly simpler shell was implemented. The v6 shell was not scriptable but introduced features that live on to this day. - Redirection via '<' and '>' operators * No way to redirect stderr; would have to wait for PWB shell - pipelines with '|' operator between commands (courtesy of Doug McElroy) - ':' was a comment and allowed for goto to be a seperate command ```sh : lable here ... work goto here ``` - Globbing was done with an external command. - '&' operator for background jobs - No variables and no environment variables. Would need to wait for v7 Unix and the Bourne shell. PWB Programmers' Workbench PWB shell (1975) (also known as the Mashey shell) The Programers' workbench was mostly a collection of new programs and included changes to the shell written by John Mashey. https://en.wikipedia.org/wiki/John_Mashey After Bell Labs, Mashey worked at MIPS on the RISC processor there and then moved to SGI. The sh(1) man page for the Mashey shell: https://www.in-ulm.de/~mascheck/bourne/PWB/ Single character variable names. Some are reserved: - $S : User's home dir - $P : the anscestor of $PATH https://en.wikipedia.org/wiki/PWB_shell If you were forced to use this shell, I think it would still feel like a glove. The Bourne Shell (1977 https://en.wikipedia.org/wiki/Bourne_shell Probably the first generation of what we would call a modern shell today. Steve Bourne arrived at Bell Labs in 1975. Was a big fan of ALGOL68 which explains the arcane syntax of Bourne (and children) shells. 'sh',or the Bourne shell, was first released with V7 Unix. Another innovation was the use of the 'Environment' to pass variables to child processes without the need for even more positional parameters. Fun fact about the Bourne shell: Because Steve Bourne was more familiar with ALGOL68 then C, he relied on a lot of preprocessor macros to make his style of coding better suited to him. This inspired the Obsfucated C coding contest. The Bourne shell did not use lex and yacc but was instead a hand coded parser written in C. Contrast to Bash, which the last time I checked used bison. (Not sure about flex) Steve Bourne at 2015 conference. Poor quality audio) https://www.youtube.com/watch?v=7tQ2ftt3LO8 [ The ALGOL 68c compiler written by Steve Bourne while at Cambridge University in the U.K: https://en.wikipedia.org/wiki/ALGOL_68C ] Fun facts re: The C programming language (of which sh was written in) On Bell Labs PDP 11s, address 0 always had the number 0 in it. This leds to many a bug in which indirect pointers through a null pointer always returned 0. This assumption caused portability problems when they went to the VAX 11s. Also, the construct ' *p++ ' compiles down to a single instruction on PDP-11s. I imagine many buffer overruns were due to this bias to platform specific code. The Bourne shell spawned many offsprings the Almquist shell (ash) and it's Debian flavor (No dash) the former is on many BSD varients and dash is present on many Linux distros including MacOS.. The C Shell or csh (1979) Written by Bill Joy and included in 2BSD Unix distribution. More like C than ALGOL68. https://en.wikipedia.org/wiki/C_shell The tcsh shell (circa 1983) The 't' in the name is an homage to the Tenex OS shell. https://en.wikipedia.org/wiki/TENEX_(operating_system) Tenex was developed by BBN of IMP fame for the PDP-10 Mainframe from DEC. Fostered the innovation of virtual memory in OSes. It later inspired TOPS-10 from Digital. Tcsh was essentially the C shell with improvements and new features Main web site: http://www.tcsh.org/ Repository: https://github.com/tcsh-org/tcsh On the MacOS /bin/ csh is linked to /bin/tcsh Although I cannot verify this, this might also be the case on RHEL and maybe Fedora. On Debian derived Linux distros like Ubuntu they are seperate packages. License BSD v2 Features: - Programmable command line completions - Command line editing - The history command - The '!' directive to execute the previous command. - No functions but expansions of alias to allow for positional parameter insertion. * No other shell has this feature * Other shells do allow for user defined functions to get over this limitation. Example of parameter substitution in tcsh aliases: ```tcsh alias my_cd='cd \!* && ls' After invoking my_cd ~/my/dir, the next command ls is executed. I wish this functionality existed in Bash. - Job control Bash Zsh Fish or the Friendly interactive shell Powershell (2006) Maybe the first shell that deviates from the idea that everything piped from one program to another must be a simple text/character stream. FreeBSD shells For sysadmins: TCSH For system scripts: ash Users can install and use any Unix compatible shell like KSH and BASH. Modern Apple Macintosh systems now default to the ZSH due to GPL license differences over Bash 4 and beyond. But can be replaced by Bash with the 'chsh' command. See 'man chsh' Nushell https://www.nushell.sh Busybox (and alternatives) Favorites for embedded Linux systems Busybox is a single executable with the shell (ash) and several builtin commands like ls, editors and much more. Busybox can be configured at compile time to add or remove functionality. Busybox is a lightweight userland system and is often used in Docker containers. Source: https://github.com/mirror/busybox Mirror for the BusyBox distribution. YMMV. [www.busybox.net seems to have disappeared] Toybox Toybox is a lighter alternative to Busybox and generally includes fewer applets. Toybox is the shell in Android. http://landley.net/toybox/ BusyBox is GPLv2 and Toybox is BSD. RC Shell for Plan 9 (1989) Similar to the Bourne shell. https://pdos.csail.mit.edu/6.828/2007/readings/rc-shell.pdf You should check out rc. Nicer syntax and better uses of lists for aggregate data structures. Uses curly braces instead of traditional do/done and if/then/else/fi etc. https://en.wikipedia.org/wiki/Rc Finally: In the Beginning was the Command Line Neil Stephenson https://web.archive.org/web/20180218045352/http://www.cryptonomicon.com/beginning.html