Overview

It’s easy to get intimidated, but once you get over the initial learning curve you will see it’s not as hard as it looks. Like every other aspect of programming, you will improve with lots of practice and experience.

If you are comfortable with programming in R or Python, then the Unix/Linux command line is not so different. We will look at several useful commands, which are equivalent to functions in R. Recall that in R functions have arguments, with the general form:

RFunction(Parm1=ARG1, Parm2=ARG2, ...)

I R, a function is always followed by a set of brackets, which can be used to define arguments that are specific to that function.

The Unix/Linux commands also have arguments, but they are defined with dashes and spaces instead of () and = :

UnixFunction --Parm1 ARG1 --Parm2 ARG2

This may look confusing at first but you will get used to it. Often there are one-letter short forms for these commands, which are used with just a single dash:

UnixFunction -A ARG1 -B ARG2

That’s all you need to get started!

WARNING!

‘With great power comes great responsibility’ – Uncle Ben

The command line is a powerful tool. It can be used to do a lot of wonderful things, but it can also completely mess up your computer if you aren’t careful. It is very easy to accidentally delete a file with no way of recovering it. Pay close attention to the files you are working with.

Moving Around

Often when we log onto a new computer it’s good to explore the file system. This can be a bit disorienting at first because we have to interpret a bunch of lines of text, rather than graphics of folders and files. It can help to imagine the folders just like you do on your regular graphical user interface (e.g. MacOS, Windows, Ubuntu).

pwd Present Working Directory

It’s important to check where you are in your file system. Doing this frequently will ensure that you don’t accidentally delete an important file. This will check what your directory is, equivalent to getwd() in R:

pwd

Your output will depend on your computer and the program you are using:

MobaXTerm might loook something like this:

/home/mobaxterm

If you are using MacOS it look more like this:

/Users/UserName

where UserName is the user name you defined when you set up the computer.

If you are using a Unix-based system like Ubuntu or CentOS, or logging onto a remote server with Compute Canada or the Queen’s University Centre for Advanced Computing (CAC), then it might look more like this:

/home/USERNAME

OR

/global/home/USERNAME

cd Change Directory

~ Getting Home

Regardless of where you are, you can use the ~ command to return to your home directory:

cd ~
pwd

Again, the output will depend on your file system. In this tutorial, we’ll represent your home directory with ~ in the output. So where you see ~ in the output, you should see one of the above directories in your own output.

mkdir Make a Directory

Let’s make a temporary directory:

mkdir tmp

cd Change Directory

Let’s move into our new directory and then see where we are at

cd tmp
pwd
~/tmp

Now we are inside the tmp directory inside of our home directory. If we wanted to move ‘up’ one level, back to our home directory, then we type two periods:

cd ..
pwd
~

Let’s go back into the tmp directory for now

cd tmp

ls List

Use the ls command to see all the files in the directory.

ls

You won’t see anything yet because there are no files in here. Let’s create a text file.

Text editing

The linux/unix environment has a number of text editors available. Two really basic and easy to understand text editors are nano and pico. Unfortunately these are not installed with MobaXTerm by default; see these instructions for installing nano. OR you can type a command to get and install the nano program:

apt-get install nano

If nano is installed, we can use it to create a new text file:

nano Text.txt

NOTE: If this doesn’t work, then just open the Notepad (Windows) or textEdit (MacOS) editor, type the lines below, and then manually save it as Text.txt inside of your ~/tmp folder. Then quit the text editor program.

Type the following into the text file:

Line 1
Line 2
Good-bye

To save from nano, we can press Ctl-O to Output or we can press Ctl-X to Exit and press y to save the changes. Hit Enter to confirm that we want to use the same file name, which will replace the olde Text.txt with a new file containing our three lines of text.

Now take another look inside the folder:

ls
Text.txt

We can use the -l argument to see more detail

ls -l
total 1
-rw-r--r--    1 UsrName    UsersGrp        24 Jan 27 11:34 Text.txt

Note that this argument doesn’t need any additional parameters.

Let’s change the name of the file:

mv Move a File

To change the name of a file we ‘move’ the contents from the old file name to the new one:

mv Text.txt .hiddenText

Note the period in front of hiddenText. File names that start with a period are hidden by default:

ls -l
total 0

However, hidden files can be seen with the -a parameter

ls -a -l
total 3
drwxr-xr-x    1 rob_c    UsersGrp         0 Jan 27 11:37 .
drwxr-xr-x    1 rob_c    UsersGrp         0 Jan 27 11:20 ..
-rw-r--r--    1 rob_c    UsersGrp        24 Jan 27 11:34 .hiddenText

Now we can see the hidden file, but also two other lines corresponding to one and two periods. We have already used the command cd .. to change directories, so the .. is just a symbol showing that the current ‘working’ directory is inside another ‘parent’ directory. The single dot is a symbol for the current working directory. For example, compare the output from these two lines. Why are they different?

ls -a
ls -a ./
ls -a ../

The first two lines list the files ‘linked’ to the current directory. This includes the current directory ., the parent directory .., and the file .hiddenText.

The second command lists the files in the parent directory. The specific output will depend on the operating system you are using.

cp Copy

Let’s copy our hidden file and turn it into a Bash Script. A bash script is a text file that can be used to run programs in the Unix/Linux environment.

cp .hiddenText BashScript.sh

rm Remove

Let’s delete the old file, which we don’t need. We do this with the rm command:

WARNING: be careful with rm because there is no ‘recycle bin’ in the command line. When you delete it, it’s pretty much gone!

rm .hiddenText.txt

Now open the text file with nano or a text editor.

Bash

A bash script always begins with the line

#!/bin/bash

The order is very important here. A good way to remember the order is the phrase Shazam!, like the comic book characer. Not to be confused with kazaam, the impressively bad Shaquille O’Neal movie.

# Comments

The first thing we can do is add comments to the document. Just like in R we can add comments with #. Note the exception is the special Shazam! combination, which starts with # and appears on the first line of the bash script.

echo Output

The Unix/Linux equivalent of the print() command in R is echo. Because we are going to print the other lines as strings, we use quotation marks to clarify this for the bash interpreter. Now your file should look like this:

#!/bin/bash

# Comment -- this line will not run

# echo returns output to the user

echo "Line 1"
echo "Line 2"
echo "Good-bye"

Next we can make a more interesting program using variables, a for loop and if/then statements.

$Variable

Bash variables are set with regular text, BUT when you want to use the variable then add the $ prefix. This will make sense when we use it in the context of a for loop

for

A for statement has two components:

  1. The for component sets up the looping variable
  2. The do and done statements define what to do for each iteration of the loop.

Let’s add a for loop with a variable counting from 1 to 5. There are two ways we can do this:

for Count in 1 2 3 4 5
for Count in {1..5}

In the first case, you just separate each iteration by a space and you could use other things besides numbers, for example, strings defined with quotation marks, or file names.

Once the for line is written, the next lines use do and done to define what to do in each iteration of the loop

for Count in 1 2 3 4 5
do
  echo "This is loop number $Count"
done

Now save and exit the script.

if

We won’t include an if statement, but they are similar, with arguments defined in square brackets. Instead of do and done if statements use if, then, elif, else and fi. Here’s an example:

if [ $Var1 == $Var2 ]
then
   echo "Variable 1 is equal to Variable 2"
elif [ $Var1 -gt $Var2 ]
then
   echo "Variable 1 is greater than Variable 2"
elif [ $Var1 -lt $Var2 ]
   echo "Variable 1 is less than Variable 2"
else
   echo "Variable 1 is not equal to or greater than Variable 2"
fi

In this case, specific statements are made with if and elif, whereas else captures anything else that doesn’t fit in any of the if or elif criteria.

chmod permissions

Before we can run our bash script, we have to modify the file permissions. The permissions for files are shown when you run the long version of the list command ls:

Here’s an example

total 3
drwxr-xr-x    1 rob_c    UsersGrp         0 Jan 27 11:37 .
drwxr-xr-x    1 rob_c    UsersGrp         0 Jan 27 11:20 ..
-rw-r--r--    1 rob_c    UsersGrp        36 Jan 27 11:34 BashScript.sh

The first letter ‘d’ indicates whether it is a directory, and the other nine letters define the user access privileges.

The nine characters are actually 3 sets of 3 characters corresponding to (from left to right):

  1. The user (you, or rob_c in the example above)
  2. Your group (UsrsGrp in our example)
  3. Others not in your group.

An example of a Group would be a lab or PI on Compute Canada, with each individual User in the group, and Others being everyone else who has access to the computer but are not in your group.

The dash - means that permission is denied, and the three letters correspond to different kinds of use for the file:

  • w - write (i.e. edit)
  • x - execute
  • r - read (look but don’t change or run)

To run our bash script, we need permission to execute the file, which is not set by default when we made our text file. We use the chmod command to do this.

chmod 755 BashScript.sh

Now take a look at the permissions. The three numbers correspond to permissions for each of the three levels, from 0 to 7:

Code Permission rwx
7 read, write and execute rwx
6 read and write rw-
5 read and execute r-x
4 read only r–
3 write and execute -wx
2 write only -w-
1 execute only –x
0 no access —

bash Run a Script

Now that the permissions are changed, run the file using the bash command

bash BashScript.sh

User variables

We can add user-defined variables in our bash script in two ways.

Command arguments

Use the $ with numbers to indicate the order of arguments entered by the user. For example, if you add this code to your bash script:

Var1=$1
Var2=$2

Then the user would input two arguments, for example:

bash BashScript.sh 10 15

Would set Var1=10 and Var2=15

The second method is to interactively ask the user for input. Usually, you would want to prompt the user with an echo command so that they know what you are looking for. Try this:

echo "Enter a number"
read Var1
echo "Enter another number"
read Var2
echo "Your numbers are $Var1 and $Var2"

head, tail and less

Sometimes we are working with files that are too big to open with nano. In these cases, we may want to look at the first few lines using head, the last few lines using tail, or browse through the file a few lines at a time using less

Note that to exit less you type q

man and help

To get more information on any command, use the man command or the –help parameters:

man ls
ls --help