Introduction to the Command Line

My hope for this post is to lower the barrier to entry for using the shell for people who are otherwise overwhelmed by not knowing where to begin. I will outline a series of commands that will allow you to become comfortable with the basics of the command line on unix/linux/Mac and learn enough to be able to explore and learn more about it on your own (think of this as your launchpad).

By the end of this, you will know:

    • The difference between a command line, a shell, and a terminal, and the variety of shells
    • Arrows, especially the up arrow
    • When spaces do and do not matter
    • How to see which directory you're in, list contents of directories, and move to other directories (pwd/ls/cd)
    • Arguments and parameters
    • Some basic commands and how to run them: du, grep, sort
    • Pipes and grep, sort, uniq
    • echo and printf
    • globbing (*/star/asterisk)
  • Tab completion for directories/files/parameters
  • Man pages, pagers, the "more" pager, and how to use the "less" pager
  • How to read man pages to find out more about bash (also -h/--help/help/info and their differences)
  • Variables: listing, creating, and deleting
  • Where the shell finds commands: "type" and "which"; The PATH variable: reading it and modifying it
  • Running multiple commands in a row using semicolon (";")
  • Putting it all together: "git log | grep current -C 2 | grep latitude"

I use quotes often in this post, because I am familiar with the pain of trying to read programming-related tutorials/readmes/etc that don't have quotes. I hope that the quotes improve readability.

Knowing how to use the command line allows you to automate repetitive tasks, and gives you superpowers by allowing you to attach many small pieces (commands) together and create powerful pipelines. For reference, alternative keywords for referring to the command line are terminal (Mac/Linux) and console (Windows) (although Windows console is too different from Mac and Linux for this tutorial to help you with Window's built-in console, but you can get Linux/Mac-like command lines on Windows using programs like cygwin or git bash). A terminal is also a general term, because there are multiple interpreters (such as sh (shell), bash, zsh, tcsh, fish...) that can read your input commands and execute them. Note that this tutorial uses "bash" because that's the default in Mac and Linux; while not the default, they come with sh (shell) and other shell flavors can be installed. "Command line" is a generic term for referring to programs that allow you to type text commands, such as shell (sh), bash, and console. Bash is short for "bourne-again shell", where "shell" is the original "shell" that bash is based on. "Shell" refers to the fact that, when computers were massive expensive machines such that each person couldn't have their own computer, multiple people could share the same computer or log in to a computer remotely (either the computer could be in the same room as the user or be in another room/building/city/state) by using a "shell" like an egg shell that doesn't actually have the computer on it but allows you to type input and see the output on a screen. A "terminal" is a word similar to "shell" in that it is just a screen that allows you to send commands and receive outputs from a remote machine. The Windows "PuTTY" software is similar to the old-style terminals, because (at least to my knowledge) it doesn't let you start a shell on your local machine - it can only be used to connect to other machines (such as linux servers or raspberry pis).

Before we get started, here's a hint at the power of the command line: imagine you are a developer who writes code, and use git or another source code management tool. You remember seeing some code in the past that you're trying to find again, but based on the context, you remember that some keyword was on one line and another keyword was on a line nearby. Imagine you're trying to find this code snippet, but it has been deleted from the codebase so you can't search the current code for it:


currentLocation: {
    latitude: action.payload.latitude,
    longitude: action.payload.longitude,

This command will help you find this snippet in the code history: "git log | grep current -C 2 | grep latitude". I explain this command in complete detail later near the end of this post.

Now let's get started with the command line. The first thing to do is type "pwd" and then hit the "enter"/"return" key: "pwd" is short for "print working directory", and "enter"/"return" runs/executes the command that you just typed. pwd tells you where you are right now. In the command line, you are always somewhere in some directory. In a couple paragraphs, I'll show you how to change where you are. By default, when you first open the terminal, you should be in your home directory, which is "/Users/YOURUSERNAME" on Mac or something like "/home/YOURUSERNAME" on linux OSes. Knowing where you are makes it easier to know what you're looking at and decide where to go.

Now that you executed the "pwd" command, hit the up arrow on the keyboard (the same up arrow you would use to go up one line in a word document). The "pwd" command that you just executed should appear again. You can use the left arrow key to move the cursor to the left, the right arrow to go right, and you can move up and down through previous commands using the up and down arrows. This allows you to run the same command multiple times in a row, or edit the previous command and run the edited command.

"ls": "list directory contents". It prints the contents of a directory. Try it out. Without specifying an argument, it lists the files in your current working directory (you just found out what your working directory is by typing "pwd"). An argument is something that comes after a command to give it more information about how to run that command. So far, we have learned the commands "pwd" and "ls" - whatever the first thing you type in your command line before the first space is the command, and anything that comes after that are arguments. Ubuntu has very friendly default output for "ls": blue indicates directories, green indicates files that have the executable bit set (from your user's perspective), pink indicates sockets, and everything else (text files, image files, etc..) is white. On Mac, everything is white by default. To get color output on mac, try typing "ls -G" ("-G" is an argument - arguments are explained in more detail later) (also note that it doesn't matter how many spaces come before "ls", or how many spaces come between between "ls" and "-G", or how many spaces come after "-G", but that "- G" with a space between - and G is different from "-G"). Now that it listed files in the current directory, try running another "ls" command but with an argument by choosing one of the directories that "ls" just listed. For example, on Mac, there should be a "Desktop" directory that was just listed when you typed "ls". Try typing "ls Desktop", where "Desktop" is the argument: that will list the files on your desktop. "ls" can accept any number of arguments. Find two folders in your current directory, for example "Desktop" and "Downloads", and then type the command "ls Desktop Downloads": it will print the contents of each directory in turn.

Now let's move to a different directory. "cd" is the command we use to move around and stands for "change directory". Typing "cd" without an argument (i.e. by itself) gets you back to your home directory, "cd SOMEDIRECTORY" takes you into SOMEDIRECTORY, "cd "SOMEDIRECTORY/ANOTHERDIRECTORY" takes you two down two directories, and "cd -" takes you back to the directory you were in most recently, regardless of where you are right now. After typing "ls", look in the output list for a directory, and then type "cd SOMEDIRECTORY" where "SOMEDIRECTORY" refers to the directory you just chose. For example, on Mac, there's a "Desktop" folder in the home directory by default, so type "cd Desktop". We should now be on the desktop: type "pwd" to confirm that you are now on the desktop, and then type "ls" again to list the files that are on the desktop. To move up one directory, type "cd ..". ".." refers to the directory above where you are right now (we were just in your home directory and are now in the directory you chose to cd to). "." refers to your current directory. We can move up two directories at once by typing "cd ../.." (you can add any additional number of "/.." at the end to keep going up), and should now be in "/Users" on mac or "/home" on linux. You could also have moved into a directory that is contained within the directory above where you currently are by typing "cd ../SOMEDIRECTORY". If you are on mac, then your home directory should also have a "Music" directory, so while you're on the desktop you should be able to type "cd ../Music" to move to your Music directory. You can also type "ls .." to list the contents of the directory that contains the directory you are currently in.

Now let's learn about arguments and parameters. As mentioned before, an argument is something that comes after a command that lets the command know how to run differently. Some commands can only take one argument, but some commands can take many arguments, depending on the type of the command. In the case of "cd Desktop", "Desktop" is the argument so that "cd" knows to change the directory to the desktop. Without an argument, "cd" changes to the home directory, and "ls" lists the contents of the current directory. As a simple example, let's learn "du". "du" is called "disk usage" and lists the sizes of files. Without any arguments (if you try it, the output will be a huge list of lines), it recursively lists the size of every file contained within the current directory. With a directory as an argument, it recursively lists the size of every file contained within that directory (e.g. "du Desktop"). With a file as an argument, it just prints the size of that file (e.g. "du somefile.txt"). By default, it prints sizes in "system blocks" (whatever that means) and prints recursively. To get a more meaningful output, let's use the "-h" (human-readable: print in kilobytes/megabytes/gigabytes instead of system blocks) and "-s" (summary: don't list files recursively, just print the size for the file or directory specified) arguments on a directory. (If you're not in your home directory, type "cd" to get back to the home directory.) Try "du -h -s Desktop". If your desktop has lots of small files or some big files, that command could take a while (hopefully only a few seconds at the most). "-h" and "-s" are single-letter arguments. Some argument names are longer (usually named arguments start with "--", but it's not consistent, for example the "find" command has a "-name" argument), and some short arguments have a long-argument equivalent (for example, grep has a "--perl-regexp" argument, but "-P" is a short version that has the same effect). Often, single-letter arguments can be strung together (not all commands support that). For example, in the command above, we could type "du -hs Desktop" instead - it has the same meaning. Also, sometimes the order of arguments matter, and sometimes it doesn't. For "du", another equivalent would be "du Desktop -hs". "du" doesn't care what order those arguments come in. Some commands care about the order of arguments, which you can usually learn about by using either the "-h" or "--help" arguments, the man page (short for manual), the info page, or the help page (I'll explain man/help later in this post). One example is the "echo" command described later:

mica-pro:~ mica$ echo -n no newline at the end
no newline at the endmica-pro:~ mica$ echo this has a newline at the end because -n only works when it comes before the echo text
this has a newline at the end because -n only works when it comes before the echo text
mica-pro:~ mica$

Now for the difference between an argument and a parameter. My understanding of the difference between an argument and a parameter is based on experience rather than on cold hard definitions. An argument is an actual input to a command. In "du -s", "-s" is an argument, because "du" is using "-s" to decide how to run. However, the "sort" command has a "-t char" or "--field-separator=char" option, where "--field-separator" or "-t" is a parameter while "char" is the argument. If "-t"/"--field-separator" didn't take a "char" argument, then they themselves would be the argument to the "sort" command. Some parameters don't take any arguments, in which case they are actually arguments, and some parameters take one argument (e.g. "ls --color=auto" on linux), some parameters take a specific number of arguments (e.g. "codetool --mergefiles file1 file2" [that's a made-up command]), and some parameters can take many arguments (e.g. "dostuff --inputs file1 file2 file3 ..."). Sometimes (but not always), a parameter that takes a single argument can be written with an equal sign, such as "--color=auto" or "--color auto" (but in this case, that doesn't work, because linux ls requires that --color uses an equal sign and no space).

Now let's learn about pipes and grep. The "|" vertical bar symbol means pipe: it takes the standard output of one command and passes it as the standard input to the next command. "grep" is probably the most common command to use with a pipe, which lets you search for text. For example, from your home directory (remember to type "cd" to get back to your home directory) type "ls | grep Desk". Remember that by itself, "ls" lists all the contents of the current directory, e.g. "Desktop Documents ...". "ls | grep Des" says "look at the output of 'ls' and search for matches containing 'Des'". The result should be only "Desktop", unless you have additional files that contains "Des" in their names such as "MyDesires.txt" or "DestroySomething.txt". An important thing to note is spaces - pipes don't care about spaces. "ls|grep Des" is equivalent to "ls | grep Des". Also note that a series of commands with pipes is called a "pipeline", and that you may see that word used in documentation. Back to grep: grep can be used without a pipe, for example, to search for the word "cheese" inside of a file called "recipes.txt", type the command "grep cheese recipes.txt". grep can also be used to search in all files recursively using "-r", for example, to search for the word "cheese" in all files contained recursively in a directory called "recipes", type "grep -r cheese recipes". Back to pipes: pipes allow nearly infinite possibilities. Common commands for use with pipes are "wc" (word count), "sort", "uniq" (short for "unique"). Someday, you may even write your own commands/programs that can read from standard input. Pipes can be chained together many times. For example, if you have a file "words.txt" with many words, one word on each line (some repeating, and not sorted), to find all words that contain the letter "e" and the number of times each matching word occurs, you could use this command: "grep e words.txt | sort | uniq -c" (the "-c" argument for uniq says to count the number of occurrences of each word, but uniq can only count things if the repeats are on consecutive lines, which necessitates the use of "sort").

Next comes "echo" and "printf" (the "f" in printf means "format"). "echo" is a pretty simple, straightforward command. It prints any arguments that are given to it. try running "echo stuff" - it will print "stuff" on the next line. By default, "echo" always prints a newline at the end, but that can be suppressed using "-n". If you print the same thing using printf, you will get the next command prompt on the same line as the word "stuff" because it doesn't print a newline at the end. Here is the example:

mica-pro:~ mica$ echo stuff
stuff
mica-pro:~ mica$ echo -n stuff
stuffmica-pro:~ mica$ printf stuff
stuffmica-pro:~ mica$

By default, echo ignores escape sequences like "\n" (newline), while printf doesn't, but echo can be made to heed escape sequences using the -e argument ("escape sequences"), and "\n" can be added to the end of printf for it to match the print-newline-at-the-end-of-output behavior of echo:

mica-pro:~ mica$ echo "hello\nnew line"
hello\nnew line
mica-pro:~ mica$ echo -e "hello\nnew line"
hello
new line
mica-pro:~ mica$ printf "hello\nnew line"
hello
new linemica-pro:~ mica$ printf "hello\nnew line\n"
hello
new line
mica-pro:~ mica$

Now let's learn about star/asterisk, the "*" symbol. In the command line, this is called a glob or wildcard. Some languages have support for globs, such as python's "glob" module in the standard library (once you understand globs, you can search google for "python glob" to see how to use it). Let's go back to the home directory by running "cd". Now run "ls". There should be, at least, "Desktop" and "Documents" (or Downloads). If you type "echo D*", it should print "Desktop Documents Downloads" and anything else it matched. It should also match both "Desktop" and "Documents" if you type "echo *s*", because Desktop will match "*" to "De", "s" to "s", and "*" to "ktop", and Documents will match "*" to "Document", "s" to "s", and "*" to "" (nothing) (asterisk can match with nothing / empty string), etc. If you type "echo *", the output looks very similar as the output of running "ls". However, remember we typed "ls Desktop" earlier and saw the contents of the desktop. Now let's understand the behavior of globs a bit more deeply. I'll explain this command: "ls D*s". On my computer, from my home directory, "echo D*s" prints "Documents Downloads". The bash expands the globs before passing the output to the "echo" command. When the "echo" command ran, it didn't see the asterisk - it saw "Documents Downloads". So it behaves as if you just typed "echo Documents Downloads". The behavior matters more when getting into the intricacies of bash. Now let's try typing "ls D*": the command auto-expands to "ls Documents Downloads", so "ls" receives the arguments "Documents Downloads" (and ls never saw the asterisk). One important thing to note here is that glob is not memory-efficient: if you try to glob to match a huge number of files, your command line or whatever command you're passing the globbed arguments to might not be capable of handling such a large amount of input/output. In cases like that, other methods should be used. For another example, from your home directory, type "echo *": the output should look similar similar to the output of "ls".

Let's learn about tab completion. First, let's move back to the home directory by typing "cd" without any arguments. Now type "ls" to see the directories that are currently available. Depending on your OS, there may be both "Desktop" and "Documents" directories. Now type "ls D", and then hit tab a couple times - it should print, at least, "Desktop" and "Documents". It is listing all the things that could come after "ls D". If you add an "e" so that it becomes "ls De" and then hit tab, if there are no other directories that start with "De", then it should automatically complete to "ls Desktop". Many commands have autocomplete - sometimes it's dumb autocompletion, and sometimes it's smart autocompletion. For example, if you try to autocomplete using the "cd" command, it will smartly only let you autocomplete to directories, not to files - if you have a directory called "Desktop" and a file called "Delicatessen.txt", and you type "cd De" and hit tab, it will autocomplete to "Desktop" and ignore "Delicatessen.txt", because "cd" is only for changing directories, so it doesn't make sense to "cd" to a file. Some commands even have autocomplete for their arguments. For example, for the "sort" command mentioned previously, there is a "--human-numeric-sort" argument. Linux flavors can usually autocomplete that argument by typing "sort --hum" and then typing tab. The linux-oriented versions of some commands, including sort, can be installed on Mac using brew (the package is called coreutils). It can also autocomplete the same way we saw earlier with less - type "sort --h" and then hit tab twice, and it should show you at least two possible completions, "--help" and "--human-numeric-sort". Let's go back to our "du" example from before to show off the power of "sort --human-numeric-sort". Inside your home directory, type "du -hs D*" (remember that "D*" is a glob as mentioned before). Hopefully "D*" will match at least 3 things (Desktop, Documents, Downloads) so we can see some interesting output, but also hopefully your Desktop/Documents/Downloads directories are relatively small, otherwise it could take a while for the command to run and could make your computer's fan spin as it calculates the recursive sizes of the directories. Also, hopefully the output doesn't coincidentally happen to be sorted, so we can compare the results using the sort command. Here is an example:

du -hs *
12K	a.txt
1.0M	b.txt
4.0K	c.txt

Notice that the output is sorted alphabetically by filename, not by the size of the files. The way that these particular files is listed in order of size is: middle size (12K), largest (1M), smallest (4K). Now let's try "du -hs * | sort --human-numeric-sort", which is smart enough to be able to tell the difference between file sizes, including the difference between Kilobytes/Megabytes/Gigabytes:

du -hs * | sort --human-numeric-sort
4.0K	c.txt
12K	a.txt
1.0M	b.txt

Voila! The output is sorted by ascending size! Note, however, that "sort" sorts by whatever appears on the left-most side of the line, so if "du" had decided to print in the format of "filename size" instead of "size filename", that would not have worked.

The next step is to learn how to learn more. There are a few methods, but the man pages (manuals) are the first and most comprehensive source of more information. In order to learn how to use man pages, we have to learn about pagers, especially "less" and "more". A pager is a command-line tool that allows you to view something that is too big to fit on one command-line screen. Some commands, such as "man" and "git log", automatically open in a pager (usually the "less" pager by default), but output can almost always be pushed to a pager. "more" was the original pager, but was very limited in its capabilities, so a new pager called "less" (a play on the word "more") was introduced. "less" introduced a whole suite of functionality that made it much easier to view anything on the command line screen.

Let's start by typing "man bash": this will open the man page for bash using the "less" pager. The first thing to notice (besides the screen filling with text) is the colon (":") in the bottom left of the screen. That is where your cursor is, but if you type characters, "less" grabs what you type and interprets it as commands rather than showing what you type on the screen. First let's get out of less by typing "q". As mentioned before, you can click the up arrow to get your "man bash" command again - run it again and let's explore the other commands.

j - forward one line (you can hold it down to go continuously). the "return"/"enter" key has the same behavior as j.
k - back/up one line
f - forward/down one page
b - backward/up one page
h - get help for "less" commands
q - quit/exit out of less
/ - search (type something after, and then hit enter to search) (if you previously ran a search but want to edit it, you can use up and down arrows to look at your previous searches, just as you can up and down arrow through your shell commands outside of "less"). note that "less" search supports ERE Extended Regular Expressions (a simple version of regular expression compared to some of the flavors of regular expressions that are around like perl regex)
n - search for next occurrence
N - search for previous occurrence
g - go to the first line (top) of whatever you are viewing in "less"
G - go to the last line (bottom) of whatever you are viewing in "less"
d - forward/down half a page
u - backward/up half a page
s - "save" - opens a prompt that says "log file:" asking for a filename to save the currently-viewed document to (all of it, not just the current visible part) to a file. this is especially useful when you piped the output of a long-running command directly to "less" and realized that you want to save the output, such as a sql command. this command may not work in man pages.

I'll also explain how to use the other kind of "less" commands that start with "-". All these commands could be specified on the command line (at least when you get to specify "less" - if you do "man bash" then you didn't get to specify the "less" arguments, but you can specify them while "less" is open), or while inside/using "less". Try typing "-M", which then says "Long prompt (press RETURN)" at the bottom of the screen, and then press return as it suggested. Now you can see line information at the bottom of the screen, something like "lines 1-54/4757 0%": 1-54 means your screen is showing lines 1-54 of the less, 4757 says the total lines of the thing you are "less"ing, and 0% says how far you are in the page. You can jump to a specific line or specific percentage of the page. Type "50g" to go to line 50, or type "50%" to go to approximately the 50% mark of the document.

Additional "less" commands I use:
"-S" chop long lines (i.e. if the page is too big to fit on the line, then just let it go off of the page to the right side - you can then scroll to the right by using the right arrow and to the left using the left arrow)
"-i" ignore case when searching
"-#" horizontal shift - opens a prompt that says "Horizontal shift:" - it is expecting a number, which corresponds to the number of characters to scroll left/right when hitting the left/right keys. Sometimes the default horizontal shift is so high that when you scroll to the right, the text you're trying to read flies past when you scroll right by a single click of the right arrow key.
"-N" display line numbers

These types of commands can usually be un-done by running them again. For example, type "-M" and "return" to see the long prompt, then type "-M" and "return" again to make the prompt go away.

Now let's learn about other ways to get help on the command line and shell builtins. Before typing it, what would you expect if you run the command "man cd"? We would expect to see the man page for "cd". However, "cd" is a shell builtin command. Typing "man cd" opens a very generic man page that provides a very long list of commands, but doesn't say anything about how to use them. Because it's a shell builtin, help can be seen for it using "help cd". Some commands have a "-h" argument that shows help, some have a "--help" argument that shows help, some have both, and some have neither. Sometimes none of the methods work and you have to google to find out how to use the command properly, but you can usually start out with man/help/-h/--help to find out how to use the command. However, also note that some documentation can be spotty - some commands are extremely difficult to figure out what they do without a google search, even when you have access to the full man pages on your computer! Another command that I never got used to is "info" - it's similar to "man", but uses emacs or an emacs-like interface, which is a behemoth that has entire websites dedicated to it (what is it? a shell? a text editor? a development environment? a TODO tool?.. all of the above, and more! It's practically a whole operating system, and there have been flame wars between emacs users and vim users since at least as early as the 1980s).

Note that most/many(/all?) of the shell builtin commands like "cd" are described in the "man bash" page. The "man bash" page is huge - you can't hope to read and understand all of it within a short period of time. You can find the descriptions of the builtin commands by searching the "man bash" page for "SHELL BUILTIN COMMANDS". Since "SHELL BUILTIN COMMANDS" is referenced multiple times in the "man bash" page before the actual section, I usually get there faster by executing the command "67%" (just by typing it while the man page is open) and then using "f" to go forward a couple pages. The commands are listed in alphabetical order. While "help cd" will get you documentation on the specific command, the documentation in the "man bash" page is usually a bit different (although the functionality may be the same), so that provides two sources of information to look up a single command.

Now let's learn about variables and see the "more" pager in action. Make sure you start with a blank line on your command line. Type "$" (which indicates a variable), and type the tab key twice. The shell will warn you: "Display all 198 possibilities? (y or n)" (anyways, I hope you have enough environment variables to trigger more - if not, try resizing the terminal window to make it small enough so that it can trigger "more"). When you hit the "y" key, the "more" pager will display the matches. "more" is similar to less but is very primitive. It can only go forwards ("enter" to go forward one line, or space bar to go forward a whole page), and when you get to the end of whatever you're viewing using "more", it automatically terminates the "more" command (in "less", when you went to the end of the page, you were still inside "less" and could scroll back up). The things you're seeing on the screen are all the variables in the current shell. To see the value of a variable, you can type "echo $SOMEVARIABLE". For example, run "echo $BASH". "$BASH" holds the path to the executable of the shell that is currently running. That is the executable file that reads in your commands from the command line, processes them, and runs the commands you tell it to run. The method I just described only lists variables names (and not their values), but some additional information, including functions in the current shell, can be seen by using the "env" command. You can either run "env" by itself or pipe it to "less" using "env | less".

You can create new variables quite simply. There are ways to create arrays (and possibly also other types of variables), but this is the most simple way to create a simple variable: "somevariable=asdf" on a line by itself. then run "echo $somevariable". It should print "asdf" on the next line. There are more intricacies to bash variables that deserves its own tutorial/post, and before you try to use them in scripts, you need to learn about the export/source/set/unset commands. You could use variables like this: first create a variable "logfile=log.txt", then you can use the "$logfile" variable in all subsequent commands that take a filename: "echo logging to file $logfile".

Sometimes we also want to be able to delete a variable or function that we created, especially if we accidentally clobbered (used the same name as something else) something else, e.g. created a function called "cd". The command to delete a function called "cd" is "unset cd".

Now that we know about variables, it's time to talk about the PATH variable, also known as the "search path", and where to find commands. When you type a command such as "cd", the bash has a behavior that goes something like this: first it checks if it's a function in the current environment, then it checks if it's a built-in command, then it looks for the command in each directory in the PATH. Check your path by running "echo $PATH". The result is a list of directories separated by colons (":"). It checks each directory in order, so if you have the same command/executable in multiple directories on your path, it will run the one that is in contained in the directory that comes earlier on the path. Once you run a command, the location of the command is saved in your shell so when you type the command again, it doesn't have to search for it. Sometimes that produces undesirable behavior when you make modifications to your path, because you want it to use the command in a directory at the front of the path, but it saved the old destination. First, to check where the shell is looking for a specific command, use the commands "type" and "which". The benefit of "type" is that, in addition to searching the path, it can also search functions and aliases. Both "type" and "which" accept a "-a" argument to list all matches instead of only the first match, and can also search for multiple things at once. When installing a new operating system, I often look for all the pre-installed python-related executables using "type -a python python3 pip pip3". Back to running the wrong command after changing the path: the bash builtin command called "hash" is made to "Remember or display program locations." A current terminal session can be made to forget the memorized location of a command (and re-search for it the next time you call it) by running "hash -d COMMAND_NAME" ("-d" is for delete/forget).

Back to the PATH. It's important to be careful with the path, because if you destroy the path, then the current shell session can be broken. It can usually be fixed, but if the terminal you're using has limited functionality (imagine that you're logged into a remote server without using a 'screen' or 'tmux' client, installing an operating system, and sshd is in a weird state so you can't create any new connections - breaking the PATH can can leave you in a terrible situation where you can permanently loose access to the machine), you need to be careful with the PATH. The easiest way to add a directory to the path is to use "PATH=/path/to/some/directory:$PATH" (to add the new directory to the front of the path) or "PATH=$PATH:/path/to/some/directory" (to add the new directory to the end of the path). You can also "echo $PATH", copy it to a text editor, and then modify it manually. Note that if you have a script that adds directories to the PATH without checking if it's already there, you can end up with repeat directories on a PATH like /usr/bin:/usr/bin:/usr/bin:/usr/bin:/usr/bin. The shell/PATH don't really care if that happens, but it can be a readability issue, and if it gets really extreme, your path can become so huge that it causes other issues, like using too much memory in long-running shell sessions (from having a huge variable), so just keep that in mind. This is how I usually add a directory to the path while preventing repeats:
echo $PATH | grep /path/to/directory || PATH=/path/to/directory:$PATH
Although I did not explain "||" yet, and if you do learn about || and &&, remember that exit code behavior doesn't always follow a standard, so you should always check behavior of a command under different circumstances before relying on its behavior in a script. I checked the behavior of this command that I posted regarding modifying the PATH variable to make sure it behaved as expected.

How to run multiple commands in a row: sometimes we want to run multiple commands in a row, often because they function as a single logical unit. To do this, we use a semicolon (";") between commands. Instead of running "pwd" and then running "ls" on another line, you could run this instead: "pwd ; ls". I tend to put a space on either side of the semicolon for readability, but the shell doesn't care. "pwd;ls" is equivalent. Because semicolon is treated specially by the shell, if you want to pass a literal semicolon, then it needs to be quoted, because otherwise the shell interpreter splits out the commands by semicolons.

mica-pro:~ mica$ ;
bash: syntax error near unexpected token `;'
mica-pro:~ mica$ echo;

mica-pro:~ mica$ echo;echo


mica-pro:~ mica$ echo ';'
;
mica-pro:~ mica$

(notice 2 empty lines after "echo;echo")

Back to the example from the beginning of this post:

Now that you know about man pages, pipes, and "less", you should be able to read the man page to find out what I'm doing, and try it out on your own computer (as long as you have a git repository somewhere that has some commit history) and think about what it does before I describe it. Note that man pages for git commands (aside from the base "man git" page) are found by using (e.g. for the "git log" command) as "man git-log" (notice the dash).

git log | grep current -C 2 | grep latitude

The basics: You can search your git repository code history by using the "git log" command. You can search through output of one command by piping to the "grep" command, and you can retain context lines (lines that appear near a line that matches your search) in the grep command by using the "-C" option with an argument (number of lines of context above and below matching lines).

Let's break it down:
"gig log": search all the code history of this repository. the output of this will be the entire output of every line of code ever written in your codebase. if you were to type this command by itself without piping the output, it would open in a "pager", usually in the one called "less" - there's another pager called "more", which less is based on. a pager allows you to read through something on the command line that's too big to fit on the screen all at once.
"|": this vertical bar is called "pipe" and means to pass the output of one command as the input to the next command.
"grep current -C 2": since "|" comes before this, it's taking the output of the "git log" command. "grep current" means to search for the word "current" (case sensitive - to use case-insensitive, use -i), and "-C 2" means to keep 2 context lines above and below any lines that match.
"|": again, pass the output of the previous command.
"grep latitude": search the output of the previous search for "latitude". now you will find lines that contain "latitude" that are within 2 lines of the word "current" anywhere in your git repository history.

Notes:
These are the only Google searches I ran while writing this blog post: "terminal linux", "list of shells bash tsh zsh", "putty windows", "mergetool", "heartbleed"
This blog post took me just over 6 consecutive hours to write
Writing this blog post has been on my todo list for at least 3 months.
This is a very introductory blog post, and I had to restrain myself from writing even more than this.

Additional bash/linux concepts I would like to write about (but they should probably not be in this blog post, to keep this post from getting any more complex than it already is) (and things listed here is just a huge jumble of notes, many things repeating):

mv (move), mkdir (make directory), rmdir (remove directory), rm (remove), cp (copy)
ps aux, pstree, tree, lsof, pgrep, top, !!, jobs -l, disown, c-z, fg 1, bg 1, c-c
users/groups, permissions, quoting (single quotes vs double quotes), touch, chmod, chown, variables (e.g. $RANDOM, $BASH, $HOME), absolute paths, ~, relative paths, xargs, positional arguments using $@ and $* (especially regarding quoting behavior, check the man pages), star/asterisk (*) (explain using echo, ls), ln and symbolic/hard links (especially with regards to "rm", cd, pwd, readlink -e), standard output and standard error (redirection, especially as it relates to pipes), tee, uniq and -c, wc, fold, echo (especially something like "echo stuff | grep s"), brew on Mac and apt on linux, control commands like a/e/k/y, meta-backspace (not always built in, sometimes it has to be configured manually), nano, $() and ``, sed, awk, grep -o --pcre/-P, escaping periods with grep, how bash interprets \\ vs '\', using $ in '' vs "", ls -1, head (esp -n-1 or -n+1) and tail (esp -n-2 or -n+2) and their -c bytes options, a better way to deal with a huge number of files instead of using glob to match all of the files, globbing things that return spaces, cat/less, alias, profile, time (pipeline), $(()), cut -d -f, find, "." on the path, sudo, su, whoami / $HOME, $PWD, adduser, usermod -aG, screen/tmux (especially as it relates to ssh over a spotty network connection on a long-running command), disown/&/jobs -l/nohup/signals (kill), ps aux, pstree, tree, diff (especially -u), -- (especially in the context of printf and git checkout), specifying "-" as a filename to indicate to read from standard input (especially in the context of gpg --import/--verify), C-d to indicate end of input, < and >>, <, /dev/null, environment variables (e.g. PATH=... python3 my_python_program.py), tail -f and "F" while inside of "less", ps aux | grep [s]omething, nano +lineno (+lineno MUST come before the filename!!), interactive vs non-interactive shell, array variables and iteration such as ${array[@]} (or whatever the syntax is; need to look it up again), setting environment variables and using export/source and running SOME_ENV_VAR=blahblah somecommand, environment (search man bash for "COMMAND EXECUTION ENVIRONMENT"),

how the command line tokenizes input: bash treats '"$*"' differently from '"$@"', everything gets split by whitespace before passing on to the command, there don't need to be spaces on either side of a pipe, but when using brackets for a command there has to be a space between the first open bracket and the command name (e.g. "{ echo stuff;}" is right while "{echo stuff;}" will complain that a command called "{echo" can't be found), "{ echo stuff}" doesn't work because it thinks that "stuff}" is meant as one thing instead of two things without a space:

{ echo stuff} ; }
stuff}
{ echo stuff }

still waiting for input because it didn't see a semicolon to indicate the end of the command.
quote a semicolon to print a literal semicolon, e.g. "echo ';'"
variables get expanded in double quotes but not in single quotes, e.g.:

mica-pro:~ mica$ echo '$BASH'
$BASH
mica-pro:~ mica$ echo "$BASH"
/usr/local/bin/bash

how semicolon (";") gets treated

while read line ; do echo $line ; done <somefile.txt
cat somefile.txt | while read line ; do echo $line ; done
nohup bash -c "/usr/bin/python3 process_data.py" &
echo "$(date): $(basename "$0"): input configuration file is missing required 'DESTINATION' parameter"

defining functions (especially numbered parameters, variable arguments using $* and $@), using absolute paths for calling commands, passing a command to bash instead of calling it directly with the executable bit and the hash-bang set (e.g. "#!/usr/bin/env bash" or "#!/usr/bin/env python3"), crontab (especially as it relates to absolute paths, the user that runs the cron tab, the current directory when a cron job is run, the PATH of the cron user, strange behavior when using date +"%Y-%m-%d" etc), variable expansion (see "man bash") e.g. 'printf "${PATH//:/\\n}"', aliases, true/false/||/&&, exit codes, spaces in filenames (e.g. when not using quotes, try autocompletion by typing "\ " in the location of a filename where there should be a space and then hit tab to see it autocomplete as expected, or try adding a quote, single or double, before typing the file/path name), $HOME/~ and their expansions/behaviors with quotes, ...

/dev/null
/dev/tty (can output results of a command to both stdout and a file! echo stuff | tee -a /dev/tty >> test.txt.)
stdout, stdin

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.