diff --git a/aux/blank-markdown.md b/aux/blank-markdown.md index b4bc1d75d..1b6d2b782 100644 --- a/aux/blank-markdown.md +++ b/aux/blank-markdown.md @@ -17,21 +17,18 @@ Write paragraphs of text here. When you need to show input, output, and errors, use `
` and triple tildes as shown below: -
~~~ $ this is the input ~~~ -
-
+{:class="in"} ~~~ this is the output ~~~ -
-
+{:class="out"} ~~~ error message ~~~ -
+{:class="err"} The `div`'s are needed because Jekyll's Markdown processor will not let us put classes on code blocks. diff --git a/novice/extras/01-branching.md b/novice/extras/01-branching.md index 0818348ef..42df64c05 100644 --- a/novice/extras/01-branching.md +++ b/novice/extras/01-branching.md @@ -5,12 +5,10 @@ title: Branching in Git --- Here's where we are right now: -
~~~ $ git log ~~~ -
-
+{:class="in"} ~~~ commit 005937fbe2a98fb83f0ade869025dc2636b4dad5 Author: Vlad Dracula @@ -35,7 +33,7 @@ Cold and dry, but everything is my favorite color The two moons may be a problem for Wolfman But the Mummy will appreciate the lack of humidity ~~~ -
+{:class="out"} We can draw the history of the repository like this (we'll see in a second why there's a box called `master`): @@ -44,26 +42,23 @@ We can draw the history of the repository like this Let's run this command: -
~~~ $ git branch moons ~~~ -
+{:class="in"} It appears to do nothing, but behind the scenes it has created a new [branch](../../gloss.html#branch) called `moons`: -
~~~ $ git branch ~~~ -
-
+{:class="in"} ~~~ * master moons ~~~ -
+{:class="out"} Immediately After Creating Branch @@ -76,76 +71,65 @@ They both point to the same revision right now, but we can change that. Let's make `moons` the active branch: -
~~~ $ git checkout moons ~~~ -
-
+{:class="in"} ~~~ Switched to branch 'moons' ~~~ -
-
+{:class="out"} ~~~ $ git branch ~~~ -
-
+{:class="in"} ~~~ master * moons ~~~ -
+{:class="out"} After Switching to Branch Our file looks the same: -
~~~ $ cat mars.txt ~~~ -
-
+{:class="in"} ~~~ Cold and dry, but everything is my favorite color The two moons may be a problem for Wolfman But the Mummy will appreciate the lack of humidity ~~~ -
+{:class="out"} because it *is* the same: Let's add another line to it: -
~~~ $ echo "Maybe we should put the base on one of the moons instead?" >> mars.txt ~~~ -
+{:class="in"} and add an entirely new file: -
~~~ $ echo "Phobos is larger than Deimos" > moons.txt $ ls ~~~ -
-
+{:class="in"} ~~~ mars.txt moons.txt ~~~ -
+{:class="out"} Git now tells us that we have one changed file and one new file: -
~~~ $ git status ~~~ -
-
+{:class="in"} ~~~ # On branch moons # Changes not staged for commit: @@ -160,18 +144,16 @@ $ git status # moons.txt no changes added to commit (use "git add" and/or "git commit -a") ~~~ -
+{:class="out"} Let's add and commit those changes (the `-A` flag to `git commit` means "add everything"): -
~~~ $ git add -A $ git status ~~~ -
-
+{:class="in"} ~~~ # On branch moons # Changes to be committed: @@ -181,19 +163,17 @@ $ git status # new file: moons.txt # ~~~ -
-
+{:class="out"} ~~~ $ git commit -m "Thinking about the moons" ~~~ -
-
+{:class="in"} ~~~ [moons 62e7791] Thinking about the moons 2 files changed, 2 insertions(+) create mode 100644 moons.txt ~~~ -
+{:class="out"} Our repository is now in the state shown below: @@ -203,36 +183,31 @@ The `moons` branch has advanced to record the changes we just made, but `master` is still where it was. If we switch back to `master`: -
~~~ $ git checkout master ~~~ -
+{:class="in"} our changes seem to disappear: -
~~~ $ ls ~~~ -
-
+{:class="in"} ~~~ mars.txt ~~~ -
-
+{:class="out"} ~~~ $ cat mars.txt ~~~ -
-
+{:class="in"} ~~~ Cold and dry, but everything is my favorite color The two moons may be a problem for Wolfman But the Mummy will appreciate the lack of humidity ~~~ -
+{:class="out"} They're still in the repository—they're just not in the revision that `master` is currently pointing to. @@ -241,13 +216,11 @@ we've created a parallel timeline that shares some history with the original one Let's make some changes in the `master` branch to further illustrate this point: -
~~~ $ echo "Should we go with a classical name like Ares Base?" > names.txt $ git status ~~~ -
-
+{:class="in"} ~~~ # On branch master # Untracked files: @@ -256,20 +229,18 @@ $ git status # names.txt nothing added to commit but untracked files present (use "git add" to track) ~~~ -
-
+{:class="out"} ~~~ $ git add names.txt $ git commit -m "We will need a cool name for our secret base" ~~~ -
-
+{:class="in"} ~~~ [master dfcf908] We will need a cool name for our secret base 1 file changed, 1 insertion(+) create mode 100644 names.txt ~~~ -
+{:class="out"} Our repository is now in this state: @@ -281,22 +252,19 @@ They could continue independent existence indefinitely, but at some point we'll probably want to [merge](../../gloss.html#merge) our changes. Let's do that now: -
~~~ $ git branch ~~~ -
-
+{:class="in"} ~~~ * master moons ~~~ -
-
+{:class="out"} ~~~ $ git merge moons ~~~ -
+{:class="in"} When we run the `git merge` command, Git opens an editor to let us write a log entry about what we're doing. @@ -323,7 +291,6 @@ In this case, we'll stick with the default log message. When we save the file and exit the editor, Git displays this: -
~~~ Merge made by the 'recursive' strategy. mars.txt | 1 + @@ -331,20 +298,18 @@ Merge made by the 'recursive' strategy. 2 files changed, 2 insertions(+) create mode 100644 moons.txt ~~~ -
+{:class="out"} We now have all of our changes in one place: -
~~~ $ ls ~~~ -
-
+{:class="in"} ~~~ mars.txt moons.txt names.txt ~~~ -
+{:class="out"} and our repository looks like this: @@ -352,12 +317,10 @@ and our repository looks like this: We can ask Git to draw a diagram of the repository's history with this command: -
~~~ $ git log --oneline --topo-order --graph ~~~ -
-
+{:class="in"} ~~~ * e0cf8ab Merge branch 'moons' |\ @@ -368,7 +331,7 @@ $ git log --oneline --topo-order --graph * 34961b1 Concerns about Mars's moons on my furry friend * f22b25e Starting to think about Mars ~~~ -
+{:class="out"} This ASCII art is fine for small sets of changes, but for anything significant, diff --git a/novice/extras/02-review.md b/novice/extras/02-review.md index 48b46d2d3..d586a89c7 100644 --- a/novice/extras/02-review.md +++ b/novice/extras/02-review.md @@ -23,34 +23,30 @@ Dracula creates a repository on GitHub in exactly the same way as [we created the `planets` repository](../git/02-collab.html) and then [clones](../../gloss.html#repository-clone) it to his desktop: -
~~~ $ git clone https://github.com/vlad/undersea.git ~~~ -
-
+{:class="in"} ~~~ Cloning into 'undersea'... warning: You appear to have cloned an empty repository. ~~~ -
+{:class="out"} `git clone` automatically adds the original repository on GitHub as a remote of the local repository called `origin`—this is why we chose `origin` as a remote name in our previous example: -
~~~ $ cd undersea $ git remote -v ~~~ -
-
+{:class="in"} ~~~ origin https://github.com/vlad/undersea.git (fetch) origin https://github.com/vlad/undersea.git (push) ~~~ -
+{:class="out"} Dracula can now push and pull changes just as before. diff --git a/novice/extras/03-man.md b/novice/extras/03-man.md index e4cd7c160..93af2f9b0 100644 --- a/novice/extras/03-man.md +++ b/novice/extras/03-man.md @@ -8,11 +8,10 @@ We can get help for any Unix command with the `man` For example, here is the command to look up information on `cp`: -
~~~ $ man cp ~~~ -
+{:class="in"} The output displayed is referred to as the "man page". @@ -50,23 +49,21 @@ AUTHOR, REPORTING BUGS, COPYRIGHT, HISTORY, (known) BUGS, and COMPATIBILITY. Here is the is synopsis for the `cp` command on Ubuntu Linux: -
~~~ SYNOPSIS cp [OPTION]... [-T] SOURCE DEST cp [OPTION]... SOURCE... DIRECTORY cp [OPTION]... -t DIRECTORY SOURCE... ~~~ -
+{:class="out"} This tells the reader that there are three ways to use the command. Let's look at the first usage: -
~~~ cp [OPTION]... [-T] SOURCE DEST ~~~ -
+{:class="out"} `[OPTION]` means the `cp` command can be followed by one or more optional [flags](../../gloss.html#command-line-flag). @@ -90,7 +87,6 @@ Note that to use the last one, the `-t` option is mandatory The `DESCRIPTION` section starts with a few paragraphs explaining the command and its use, then expands on the possible options one by one: -
~~~ The following options are available: @@ -107,7 +103,7 @@ then expands on the possible options one by one: ... ... ~~~ -
+{:class="out"} #### Finding Help on Specific Options @@ -122,12 +118,11 @@ After that, we can use the 'n' key to navigate to the next match until we find the detailed information we need: -
~~~ -t, --target-directory=DIRECTORY copy all SOURCE arguments into DIRECTORY ~~~ -
+{:class="out"} This means that this option has the short form `-t` and the long form `--target-directory` and that it takes an argument. diff --git a/novice/extras/04-permissions.md b/novice/extras/04-permissions.md index e4b50ed5f..9198bf6e2 100644 --- a/novice/extras/04-permissions.md +++ b/novice/extras/04-permissions.md @@ -83,17 +83,15 @@ This is its way of telling us that `setup` is executable, i.e., that it's (probably) something the computer can run. -
~~~ $ cd labs $ ls -F ~~~ -
-
+{:class="in"} ~~~ safety.txt setup* waiver.txt ~~~ -
+{:class="out"} > #### Necessary But Not Sufficient > @@ -110,18 +108,16 @@ safety.txt setup* waiver.txt Now let's run the command `ls -l`: -
~~~ $ ls -l ~~~ -
-
+{:class="in"} ~~~ -rw-rw-r-- 1 vlad bio 1158 2010-07-11 08:22 safety.txt -rwxr-xr-x 1 vlad bio 31988 2010-07-23 20:04 setup -rw-rw-r-- 1 vlad bio 2312 2010-07-11 08:23 waiver.txt ~~~ -
+{:class="out"} The `-l` flag tells `ls` to give us a long-form listing. It's a lot of information, so let's go through the columns in turn. @@ -160,16 +156,14 @@ To change permissions, we use the `chmod` command (whose name stands for "change mode"). Here's a long-form listing showing the permissions on the final grades in the course Vlad is teaching: -
~~~ $ ls -l final.grd ~~~ -
-
+{:class="in"} ~~~ -rwxrwxrwx 1 vlad bio 4215 2010-08-29 22:30 final.grd ~~~ -
+{:class="out"} Whoops: everyone in the world can read it—and what's worse, modify it! @@ -178,11 +172,10 @@ which would almost certainly not work.) The command to change the owner's permissions to `rw-` is: -
~~~ $ chmod u=rw final.grd ~~~ -
+{:class="in"} The 'u' signals that we're changing the privileges of the user (i.e., the file's owner), @@ -190,45 +183,39 @@ and `rw` is the new set of permissions. A quick `ls -l` shows us that it worked, because the owner's permissions are now set to read and write: -
~~~ $ ls -l final.grd ~~~ -
-
+{:class="in"} ~~~ -rw-rwxrwx 1 vlad bio 4215 2010-08-30 08:19 final.grd ~~~ -
+{:class="out"} Let's run `chmod` again to give the group read-only permission: -
~~~ $ chmod g=r final.grd $ ls -l final.grd ~~~ -
-
+{:class="in"} ~~~ -rw-r--rw- 1 vlad bio 4215 2010-08-30 08:19 final.grd ~~~ -
+{:class="out"} And finally, let's give "all" (everyone on the system who isn't the file's owner or in its group) no permissions at all: -
~~~ $ chmod a= final.grd $ ls -l final.grd ~~~ -
-
+{:class="in"} ~~~ -rw-r----- 1 vlad bio 4215 2010-08-30 08:20 final.grd ~~~ -
+{:class="out"} Here, the 'a' signals that we're changing permissions for "all", @@ -239,28 +226,24 @@ We can search by permissions, too. Here, for example, we can use `-type f -perm -u=x` to find files that the user can execute: -
~~~ $ find . -type f -perm -u=x ~~~ -
-
+{:class="in"} ~~~ ./tools/format ./tools/stats ~~~ -
+{:class="out"} Before we go any further, let's run `ls -a -l` to get a long-form listing that includes directory entries that are normally hidden: -
~~~ $ ls -a -l ~~~ -
-
+{:class="in"} ~~~ drwxr-xr-x 1 vlad bio 0 2010-08-14 09:55 . drwxr-xr-x 1 vlad bio 8192 2010-08-27 23:11 .. @@ -268,7 +251,7 @@ drwxr-xr-x 1 vlad bio 8192 2010-08-27 23:11 .. -rwxr-xr-x 1 vlad bio 31988 2010-07-23 20:04 setup -rw-rw-r-- 1 vlad bio 2312 2010-07-11 08:23 waiver.txt ~~~ -
+{:class="out"} The permissions for `.` and `..` (this directory and its parent) start with a 'd'. But look at the rest of their permissions: diff --git a/novice/extras/05-shellvar.md b/novice/extras/05-shellvar.md index 8670e5400..70e4c7bff 100644 --- a/novice/extras/05-shellvar.md +++ b/novice/extras/05-shellvar.md @@ -10,12 +10,10 @@ you can change how the shell and other programs behave. Let's start by running the command `set` and looking at some of the variables in a typical shell session: -
~~~ $ set ~~~ -
-
+{:class="in"} ~~~ COMPUTERNAME=TURING HOME=/home/vlad @@ -30,7 +28,7 @@ UID=1000 USERNAME=vlad ... ~~~ -
+{:class="out"} As you can see, there are quite a few—in fact, four or five times more than what's shown here. And yes, @@ -69,7 +67,6 @@ As soon as it finds a match, it stops searching and runs the program. To show how this works, here are the components of `PATH` listed one per line: -
~~~ /Users/vlad/bin /usr/local/git/bin @@ -79,7 +76,7 @@ here are the components of `PATH` listed one per line: /sbin /usr/local/bin ~~~ -
+{:class="out"} On our computer, there are actually three programs called `analyze` @@ -97,31 +94,27 @@ since the directory `/users/vlad` isn't in `PATH`. Let's show the value of the variable `HOME`: -
~~~ $ echo HOME ~~~ -
-
+{:class="in"} ~~~ HOME ~~~ -
+{:class="out"} That just prints "HOME", which isn't what we wanted (though it is what we actually asked for). Let's try this instead: -
~~~ $ echo $HOME ~~~ -
-
+{:class="in"} ~~~ /home/vlad ~~~ -
+{:class="out"} The dollar sign tells the shell that we want the *value* of the variable rather than its name. @@ -134,31 +127,27 @@ which displays the right thing. Creating a variable is easy—we just assign a value to a name using "=": -
~~~ $ SECRET_IDENTITY=Dracula $ echo $SECRET_IDENTITY ~~~ -
-
+{:class="in"} ~~~ Dracula ~~~ -
+{:class="out"} To change the value, just assign a new one: -
~~~ $ SECRET_IDENTITY=Camilla $ echo $SECRET_IDENTITY ~~~ -
-
+{:class="in"} ~~~ Camilla ~~~ -
+{:class="out"} If we want to set some variables automatically every time we run a shell, we can put commands to do this in a file called `.bashrc` in our home directory. diff --git a/novice/extras/06-ssh.md b/novice/extras/06-ssh.md index fd7fcd865..3a490d182 100644 --- a/novice/extras/06-ssh.md +++ b/novice/extras/06-ssh.md @@ -47,67 +47,55 @@ To make it clearer which machine is doing what, we'll indent the commands sent to the remote machine and their output. -
~~~ $ pwd ~~~ -
-
+{:class="in"} ~~~ /users/vlad ~~~ -
-
+{:class="out"} ~~~ $ ssh vlad@moon.euphoric.edu Password: ******** ~~~ -
-
+{:class="in"} ~~~ moon> hostname ~~~ -
-
+{:class="in"} ~~~ moon ~~~ -
-
+{:class="out"} ~~~ moon> pwd ~~~ -
-
+{:class="in"} ~~~ /home/vlad ~~~ -
-
+{:class="out"} ~~~ moon> ls -F ~~~ -
-
+{:class="in"} ~~~ bin/ cheese.txt dark_side/ rocks.cfg ~~~ -
-
+{:class="out"} ~~~ moon> exit ~~~ -
-
+{:class="in"} ~~~ $ pwd ~~~ -
-
+{:class="in"} ~~~ /users/vlad ~~~ -
+{:class="out"} The secure shell is called "secure" to contrast it with an older program called `rsh`, which stood for "remote shell". @@ -137,37 +125,33 @@ For example, this command copies our latest results to the backup server in the basement, printing out its progress as it does so: -
~~~ $ scp results.dat vlad@backupserver:backups/results-2011-11-11.dat Password: ******** ~~~ -
-
+{:class="in"} ~~~ results.dat 100% 9 1.0 MB/s 00:00 ~~~ -
+{:class="out"} Copying a whole directory is similar: we just use the `-r` option to signal that we want copying to be recursive. For example, this command copies all of our results from the backup server to our laptop: -
~~~ $ scp -r vlad@backupserver:backups ./backups Password: ******** ~~~ -
-
+{:class="in"} ~~~ results-2011-09-18.dat 100% 7 1.0 MB/s 00:00 results-2011-10-04.dat 100% 9 1.0 MB/s 00:00 results-2011-10-28.dat 100% 8 1.0 MB/s 00:00 results-2011-11-11.dat 100% 9 1.0 MB/s 00:00 ~~~ -
+{:class="out"} Here's one more thing SSH can do for us. Suppose we want to check whether we have already created the file @@ -175,18 +159,16 @@ Suppose we want to check whether we have already created the file Instead of logging in and then typing `ls`, we could do this: -
~~~ $ ssh vlad@backupserver "ls results" Password: ******** ~~~ -
-
+{:class="in"} ~~~ results-2011-09-18.dat results-2011-10-28.dat results-2011-10-04.dat results-2011-11-11.dat ~~~ -
+{:class="out"} SSH takes the argument after our remote username and passes them to the shell on the remote computer. diff --git a/novice/extras/07-exceptions.md b/novice/extras/07-exceptions.md index ed9a6cffb..b67921a1a 100644 --- a/novice/extras/07-exceptions.md +++ b/novice/extras/07-exceptions.md @@ -17,7 +17,6 @@ here's a small piece of code that tries to read parameters and a grid from two separate files, and reports an error if either goes wrong: -
~~~ try: params = read_params(param_file) @@ -26,7 +25,7 @@ except: log.error('Failed to read input file(s)') sys.exit(ERROR) ~~~ -
+{:class="in"} We join the normal case and the error-handling code using the keywords `try` and `except`. @@ -45,12 +44,10 @@ trying to open a nonexistent file triggers a type of exception called an while trying to access a list element that doesn't exist causes an `IndexError`: -
~~~ open('nonexistent-file.txt', 'r') ~~~ -
-
+{:class="in"} ~~~ --------------------------------------------------------------------------- IOError Traceback (most recent call last) @@ -60,14 +57,12 @@ IOError Traceback (most recent call last) IOError: [Errno 2] No such file or directory: 'nonexistent-file.txt' ~~~ -
-
+{:class="err"} ~~~ values = [0, 1, 2] print values[999] ~~~ -
-
+{:class="in"} ~~~ --------------------------------------------------------------------------- IndexError Traceback (most recent call last) @@ -78,20 +73,22 @@ IndexError Traceback (most recent call last) IndexError: list index out of range ~~~ -
+{:class="err"} We can use `try` and `except` to deal with these errors ourselves if we don't want the program simply to fall over: -
~~~ try: reader = open('nonexistent-file.txt', 'r') except IOError: print 'Whoops!' -Whoops! ~~~ -
+{:class="in"} +~~~ +Whoops! +~~~ +{:class="err"} When Python executes this code, it runs the statement inside the `try`. @@ -111,7 +108,6 @@ We can also handle several different kinds of errors afterward. For example, here's some code to calculate the entropy at each point in a grid: -
~~~ try: params = read_params(param_file) @@ -123,7 +119,7 @@ except IOError: except ArithmeticError: report_error_and_exit('Arithmetic error') ~~~ -
+{:class="in"} Python tries to run the four functions inside the `try` as normal. If an error occurs in any of them, @@ -144,7 +140,6 @@ which file caused the problem. We can do better if we capture and hang on to the object that Python creates to record information about the error: -
~~~ try: params = read_params(param_file) @@ -156,7 +151,7 @@ except IOError as err: except ArithmeticError as err: report_error_and_exit(err.message) ~~~ -
+{:class="in"} If something goes wrong in the `try`, Python creates an exception object, @@ -181,26 +176,24 @@ For example, if this code can't read the grid file that the user has asked for, it creates a default grid instead: -
~~~ try: grid = read_grid(grid_file) except IOError: grid = default_grid() ~~~ -
+{:class="in"} Other programmers would explicitly test for the grid file, and use `if` and `else` for control flow: -
~~~ if file_exists(grid_file): grid = read_grid(grid_file) else: grid = default_grid() ~~~ -
+{:class="in"} It's mostly a matter of taste, but we prefer the second style. @@ -220,7 +213,6 @@ Exceptions can actually be thrown a long way: they don't have to be handled immediately. Take another look at this code: -
~~~ try: params = read_params(param_file) @@ -232,7 +224,7 @@ except IOError as err: except ArithmeticError as err: report_error_and_exit(err.message) ~~~ -
+{:class="in"} The four lines in the `try` block are all function calls. They might catch and handle exceptions themselves, @@ -271,7 +263,6 @@ Here, for example, is a function that reads a grid and checks its consistency: -
~~~ def read_grid(grid_file): data = read_raw_data(grid_file) @@ -280,7 +271,7 @@ def read_grid(grid_file): result = normalize_grid(data) return result ~~~ -
+{:class="in"} The `raise` statement creates a new exception with a meaningful error message. Since `read_grid` itself doesn't contain a `try`/`except` block, diff --git a/novice/extras/08-numbers.md b/novice/extras/08-numbers.md index 0f2f36d26..01f637a04 100644 --- a/novice/extras/08-numbers.md +++ b/novice/extras/08-numbers.md @@ -17,11 +17,10 @@ The first is that this scheme gives us two representations for zero (000002 This isn't necessarily fatal, but any claims this scheme has to being "natural" disappear when we have to write code like: -
~~~ if (length != +0) and (length != -0) ~~~ -
+{:class="in"} As for the other problem, it turns out that the circuits needed to do addition and other arithmetic on this @@ -142,7 +141,6 @@ it makes little sense to say that we're off by a hundredth when the value in que To see why this matters, let's have a look at a little program: -
~~~ nines = [] sums = [] @@ -156,7 +154,7 @@ for i in range(1, 10): for i in range(len(nines)): print '%.18f %.18f' % (nines[i], sums[i]) ~~~ -
+{:class="in"} The loop runs over the integers from 1 to 9 inclusive. Using those values, we create the numbers 0.9, 0.09, 0.009, and so on, and put them in the list `vals`. diff --git a/novice/git/01-backup.md b/novice/git/01-backup.md index 7e4e2307b..e49cdcaa2 100644 --- a/novice/git/01-backup.md +++ b/novice/git/01-backup.md @@ -34,14 +34,13 @@ The first time we use Git on a new machine, we need to configure a few things. Here's how Dracula sets up his new laptop: -
~~~ $ git config --global user.name "Vlad Dracula" $ git config --global user.email "vlad@tran.sylvan.ia" $ git config --global color.ui "auto" $ git config --global core.editor "nano" ~~~ -
+{:class="in"} (Please use your own name and email address instead of Dracula's, and please make sure you choose an editor that's actually on your system, @@ -66,44 +65,39 @@ Once Git is configured, we can start using it. Let's create a directory for our work: -
~~~ $ mkdir planets $ cd planets ~~~ -
+{:class="in"} and tell Git to make it a [repository](../../gloss.html#repository)—a place where Git can store old versions of our files: -
~~~ $ git init ~~~ -
+{:class="in"} If we use `ls` to show the directory's contents, it appears that nothing has changed: -
~~~ $ ls ~~~ -
+{:class="in"} But if we add the `-a` flag to show everything, we can see that Git has created a hidden directory called `.git`: -
~~~ $ ls -a ~~~ -
-
+{:class="in"} ~~~ . .. .git ~~~ -
+{:class="out"} Git stores information about the project in this special sub-directory. If we ever delete it, @@ -112,12 +106,10 @@ we will lose the project's history. We can check that everything is set up correctly by asking Git to tell us the status of our project: -
~~~ $ git status ~~~ -
-
+{:class="in"} ~~~ # On branch master # @@ -125,7 +117,7 @@ $ git status # nothing to commit (create/copy files and use "git add" to track) ~~~ -
+{:class="out"} #### Tracking Changes to Files @@ -135,52 +127,44 @@ about the Red Planet's suitability as a base. you can use whatever editor you like. In particular, this does not have to be the core.editor you set globally earlier.) -
~~~ $ nano mars.txt ~~~ -
+{:class="in"} Type the text below into the `mars.txt` file: -
~~~ Cold and dry, but everything is my favorite color ~~~ -
+{:class="in"} `mars.txt` now contains a single line: -
~~~ $ ls ~~~ -
-
+{:class="in"} ~~~ mars.txt ~~~ -
-
+{:class="out"} ~~~ $ cat mars.txt ~~~ -
-
+{:class="in"} ~~~ Cold and dry, but everything is my favorite color ~~~ -
+{:class="out"} If we check the status of our project again, Git tells us that it's noticed the new file: -
~~~ $ git status ~~~ -
-
+{:class="in"} ~~~ # On branch master # @@ -192,26 +176,23 @@ $ git status # mars.txt nothing added to commit but untracked files present (use "git add" to track) ~~~ -
+{:class="out"} The "untracked files" message means that there's a file in the directory that Git isn't keeping track of. We can tell Git that it should do so using `git add`: -
~~~ $ git add mars.txt ~~~ -
+{:class="in"} and then check that the right thing happened: -
~~~ $ git status ~~~ -
-
+{:class="in"} ~~~ # On branch master # @@ -223,25 +204,23 @@ $ git status # new file: mars.txt # ~~~ -
+{:class="out"} Git now knows that it's supposed to keep track of `mars.txt`, but it hasn't yet recorded any changes for posterity as a commit. To get it to do that, we need to run one more command: -
~~~ $ git commit -m "Starting to think about Mars" ~~~ -
-
+{:class="in"} ~~~ [master (root-commit) f22b25e] Starting to think about Mars 1 file changed, 1 insertion(+) create mode 100644 mars.txt ~~~ -
+{:class="out"} When we run `git commit`, Git takes everything we have told it to save by using `git add` @@ -258,28 +237,24 @@ so that we can write a longer message. If we run `git status` now: -
~~~ $ git status ~~~ -
-
+{:class="in"} ~~~ # On branch master nothing to commit, working directory clean ~~~ -
+{:class="out"} it tells us everything is up to date. If we want to know what we've done recently, we can ask Git to show us the project's history using `git log`: -
~~~ $ git log ~~~ -
-
+{:class="in"} ~~~ commit f22b25e3233b4645dabd0d81e651fe074bd8e73b Author: Vlad Dracula @@ -287,7 +262,7 @@ Date: Thu Aug 22 09:51:46 2013 -0400 Starting to think about Mars ~~~ -
+{:class="out"} `git log` lists all revisions made to a repository in reverse chronological order. The listing for each revision includes @@ -312,28 +287,24 @@ Now suppose Dracula adds more information to the file. (Again, we'll edit with `nano` and then `cat` the file to show its contents; you may use a different editor, and don't need to `cat`.) -
~~~ $ nano mars.txt $ cat mars.txt ~~~ -
-
+{:class="in"} ~~~ Cold and dry, but everything is my favorite color The two moons may be a problem for Wolfman ~~~ -
+{:class="out"} When we run `git status` now, it tells us that a file it already knows about has been modified: -
~~~ $ git status ~~~ -
-
+{:class="in"} ~~~ # On branch master # Changes not staged for commit: @@ -344,7 +315,7 @@ $ git status # no changes added to commit (use "git add" and/or "git commit -a") ~~~ -
+{:class="out"} The last line is the key phrase: "no changes added to commit". @@ -357,12 +328,10 @@ which shows us the differences between the current state of the file and the most recently saved version: -
~~~ $ git diff ~~~ -
-
+{:class="in"} ~~~ diff --git a/mars.txt b/mars.txt index df0654a..315bf3a 100644 @@ -372,7 +341,7 @@ index df0654a..315bf3a 100644 Cold and dry, but everything is my favorite color +The two moons may be a problem for Wolfman ~~~ -
+{:class="out"} The output is cryptic because it is actually a series of commands for tools like editors and `patch` @@ -391,12 +360,10 @@ If we can break it down into pieces: Let's commit our change: -
~~~ $ git commit -m "Concerns about Mars's moons on my furry friend" ~~~ -
-
+{:class="in"} ~~~ # On branch master # Changes not staged for commit: @@ -407,24 +374,22 @@ $ git commit -m "Concerns about Mars's moons on my furry friend" # no changes added to commit (use "git add" and/or "git commit -a") ~~~ -
+{:class="out"} Whoops: Git won't commit because we didn't use `git add` first. Let's fix that: -
~~~ $ git add mars.txt $ git commit -m "Concerns about Mars's moons on my furry friend" ~~~ -
-
+{:class="in"} ~~~ [master 34961b1] Concerns about Mars's moons on my furry friend 1 file changed, 1 insertion(+) ~~~ -
+{:class="out"} Git insists that we add files to the set we want to commit before actually committing anything @@ -453,25 +418,21 @@ and into long-term storage. First, we'll add another line to the file: -
~~~ $ nano mars.txt $ cat mars.txt ~~~ -
-
+{:class="in"} ~~~ Cold and dry, but everything is my favorite color The two moons may be a problem for Wolfman But the Mummy will appreciate the lack of humidity ~~~ -
-
+{:class="out"} ~~~ $ git diff ~~~ -
-
+{:class="in"} ~~~ diff --git a/mars.txt b/mars.txt index 315bf3a..b36abfd 100644 @@ -482,7 +443,7 @@ index 315bf3a..b36abfd 100644 The two moons may be a problem for Wolfman +But the Mummy will appreciate the lack of humidity ~~~ -
+{:class="out"} So far, so good: we've added one line to the end of the file @@ -490,12 +451,11 @@ we've added one line to the end of the file Now let's put that change in the staging area and see what `git diff` reports: -
~~~ $ git add mars.txt $ git diff ~~~ -
+{:class="in"} There is no output: as far as Git can tell, @@ -504,12 +464,10 @@ and what's currently in the directory. However, if we do this: -
~~~ $ git diff --staged ~~~ -
-
+{:class="in"} ~~~ diff --git a/mars.txt b/mars.txt index 315bf3a..b36abfd 100644 @@ -520,47 +478,41 @@ index 315bf3a..b36abfd 100644 The two moons may be a problem for Wolfman +But the Mummy will appreciate the lack of humidity ~~~ -
+{:class="out"} it shows us the difference between the last committed change and what's in the staging area. Let's save our changes: -
~~~ $ git commit -m "Thoughts about the climate" ~~~ -
-
+{:class="in"} ~~~ [master 005937f] Thoughts about the climate 1 file changed, 1 insertion(+) ~~~ -
+{:class="out"} check our status: -
~~~ $ git status ~~~ -
-
+{:class="in"} ~~~ # On branch master nothing to commit, working directory clean ~~~ -
+{:class="out"} and look at the history of what we've done so far: -
~~~ $ git log ~~~ -
-
+{:class="in"} ~~~ commit 005937fbe2a98fb83f0ade869025dc2636b4dad5 Author: Vlad Dracula @@ -580,7 +532,7 @@ Date: Thu Aug 22 09:51:46 2013 -0400 Starting to think about Mars ~~~ -
+{:class="out"} #### Exploring History @@ -589,12 +541,10 @@ we use `git diff` again, but refer to old versions using the notation `HEAD~1`, `HEAD~2`, and so on: -
~~~ $ git diff HEAD~1 mars.txt ~~~ -
-
+{:class="in"} ~~~ diff --git a/mars.txt b/mars.txt index 315bf3a..b36abfd 100644 @@ -605,13 +555,11 @@ index 315bf3a..b36abfd 100644 The two moons may be a problem for Wolfman +But the Mummy will appreciate the lack of humidity ~~~ -
-
+{:class="out"} ~~~ $ git diff HEAD~2 mars.txt ~~~ -
-
+{:class="in"} ~~~ diff --git a/mars.txt b/mars.txt index df0654a..b36abfd 100644 @@ -622,7 +570,7 @@ index df0654a..b36abfd 100644 +The two moons may be a problem for Wolfman +But the Mummy will appreciate the lack of humidity ~~~ -
+{:class="out"} In this way, we build up a chain of revisions. @@ -643,12 +591,10 @@ Our first commit was given the ID f22b25e3233b4645dabd0d81e651fe074bd8e73b, so let's try this: -
~~~ $ git diff f22b25e3233b4645dabd0d81e651fe074bd8e73b mars.txt ~~~ -
-
+{:class="in"} ~~~ diff --git a/mars.txt b/mars.txt index df0654a..b36abfd 100644 @@ -659,18 +605,16 @@ index df0654a..b36abfd 100644 +The two moons may be a problem for Wolfman +But the Mummy will appreciate the lack of humidity ~~~ -
+{:class="out"} That's the right answer, but typing random 40-character strings is annoying, so Git lets us use just the first few: -
~~~ $ git diff f22b25e mars.txt ~~~ -
-
+{:class="in"} ~~~ diff --git a/mars.txt b/mars.txt index df0654a..b36abfd 100644 @@ -681,7 +625,7 @@ index df0654a..b36abfd 100644 +The two moons may be a problem for Wolfman +But the Mummy will appreciate the lack of humidity ~~~ -
+{:class="out"} #### Recovering Old Versions @@ -690,27 +634,23 @@ we can save changes to files and see what we've changed---how can we restore older versions of things? Let's suppose we accidentally overwrite our file: -
~~~ $ nano mars.txt $ cat mars.txt ~~~ -
-
+{:class="in"} ~~~ We will need to manufacture our own oxygen ~~~ -
+{:class="out"} `git status` now tells us that the file has been changed, but those changes haven't been staged: -
~~~ $ git status ~~~ -
-
+{:class="in"} ~~~ # On branch master # Changes not staged for commit: @@ -721,24 +661,22 @@ $ git status # no changes added to commit (use "git add" and/or "git commit -a") ~~~ -
+{:class="out"} We can put things back the way they were by using `git checkout`: -
~~~ $ git checkout HEAD mars.txt $ cat mars.txt ~~~ -
-
+{:class="in"} ~~~ Cold and dry, but everything is my favorite color The two moons may be a problem for Wolfman But the Mummy will appreciate the lack of humidity ~~~ -
+{:class="out"} As you might guess from its name, `git checkout` checks out (i.e., restores) an old version of a file. @@ -748,11 +686,10 @@ which is the last saved revision. If we want to go back even further, we can use a revision identifier instead: -
~~~ $ git checkout f22b25e mars.txt ~~~ -
+{:class="in"} It's important to remember that we must use the revision number that identifies the state of the repository @@ -767,11 +704,10 @@ the commit in which we made the change we're trying to get rid of: > If you read the output of `git status` carefully, > you'll see that it includes this hint: > ->
> ~~~ > (use "git checkout -- ..." to discard changes in working directory) > ~~~ ->
+> {:class="in"} > > As it says, > `git checkout` without a version identifier restores files to the state saved in `HEAD`. @@ -796,21 +732,18 @@ like backup files created by our editor or intermediate files created during data analysis. Let's create a few dummy files: -
~~~ $ mkdir results $ touch a.dat b.dat c.dat results/a.out results/b.out ~~~ -
+{:class="in"} and see what Git says: -
~~~ $ git status ~~~ -
-
+{:class="in"} ~~~ # On branch master # Untracked files: @@ -822,7 +755,7 @@ $ git status # results/ nothing added to commit but untracked files present (use "git add" to track) ~~~ -
+{:class="out"} Putting these files under version control would be a waste of disk space. What's worse, @@ -831,18 +764,16 @@ so let's tell Git to ignore them. We do this by creating a file in the root directory of our project called `.gitignore`. -
~~~ $ nano .gitignore $ cat .gitignore ~~~ -
-
+{:class="in"} ~~~ *.dat results/ ~~~ -
+{:class="out"} These patterns tell Git to ignore any file whose name ends in `.dat` and everything in the `results` directory. @@ -852,12 +783,10 @@ Git would continue to track them.) Once we have created this file, the output of `git status` is much cleaner: -
~~~ $ git status ~~~ -
-
+{:class="in"} ~~~ # On branch master # Untracked files: @@ -866,7 +795,7 @@ $ git status # .gitignore nothing added to commit but untracked files present (use "git add" to track) ~~~ -
+{:class="out"} The only thing Git notices now is the newly-created `.gitignore` file. You might think we wouldn't want to track it, @@ -874,47 +803,41 @@ but everyone we're sharing our repository with will probably want to ignore the same things that we're ignoring. Let's add and commit `.gitignore`: -
~~~ $ git add .gitignore $ git commit -m "Add the ignore file" $ git status ~~~ -
-
+{:class="in"} ~~~ # On branch master nothing to commit, working directory clean ~~~ -
+{:class="out"} As a bonus, using `.gitignore` helps us avoid accidentally adding files to the repository that we don't want. -
~~~ $ git add a.dat ~~~ -
-
+{:class="in"} ~~~ The following paths are ignored by one of your .gitignore files: a.dat Use -f if you really want to add them. fatal: no files added ~~~ -
+{:class="out"} If we really want to override our ignore settings, we can use `git add -f` to force Git to add something. We can also always see the status of ignored files if we want: -
~~~ $ git status --ignored ~~~ -
-
+{:class="in"} ~~~ # On branch master # Ignored files: @@ -927,7 +850,7 @@ $ git status --ignored nothing to commit, working directory clean ~~~ -
+{:class="out"}
@@ -959,7 +882,6 @@ nothing to commit, working directory clean 2. The following sequence of commands creates one Git repository inside another: -
~~~ cd # return to home directory mkdir alpha # make a new directory alpha @@ -969,7 +891,7 @@ nothing to commit, working directory clean cd beta # go into alpha/beta git init # make the beta sub-directory a Git repository ~~~ -
+ {:class="in"} Why is it a bad idea to do this? diff --git a/novice/git/02-collab.md b/novice/git/02-collab.md index 070ab0a36..002cd4885 100644 --- a/novice/git/02-collab.md +++ b/novice/git/02-collab.md @@ -43,13 +43,12 @@ GitHub displays a page with a URL and some information on how to configure your This effectively does the following on GitHub's servers: -
~~~ $ mkdir planets $ cd planets $ git init ~~~ -
+{:class="in"} Our local repository still contains our earlier work on `mars.txt`, but the remote repository on GitHub doesn't contain any files yet: @@ -74,28 +73,25 @@ Copy that URL from the browser, go into the local `planets` repository, and run this command: -
~~~ $ git remote add origin https://github.com/vlad/planets ~~~ -
+{:class="in"} Make sure to use the URL for your repository rather than Vlad's: the only difference should be your username instead of `vlad`. We can check that the command has worked by running `git remote -v`: -
~~~ $ git remote -v ~~~ -
-
+{:class="in"} ~~~ origin https://github.com/vlad/planets.git (push) origin https://github.com/vlad/planets.git (fetch) ~~~ -
+{:class="out"} The name `origin` is a local nickname for your remote repository: we could use something else if we wanted to, @@ -105,12 +101,10 @@ Once the nickname `origin` is set up, this command will push the changes from our local repository to the repository on GitHub: -
~~~ $ git push origin master ~~~ -
-
+{:class="in"} ~~~ Counting objects: 9, done. Delta compression using up to 4 threads. @@ -121,7 +115,7 @@ To https://github.com/vlad/planets * [new branch] master -> master Branch master set up to track remote branch master from origin. ~~~ -
+{:class="out"} Our local and remote repositories are now in this state: @@ -135,18 +129,16 @@ Our local and remote repositories are now in this state: We can pull changes from the remote repository to the local one as well: -
~~~ $ git pull origin master ~~~ -
-
+{:class="in"} ~~~ From https://github.com/vlad/planets * branch master -> FETCH_HEAD Already up-to-date. ~~~ -
+{:class="out"} Pulling has no effect in this case because the two repositories are already synchronized. @@ -162,12 +154,11 @@ don't make `tmp` a subdirectory of the existing repository). Instead of creating a new repository here with `git init`, we will [clone](../../gloss.html#repository-clone) the existing repository from GitHub: -
~~~ $ cd /tmp $ git clone https://github.com/vlad/planets.git ~~~ -
+{:class="in"} `git clone` creates a fresh local copy of a remote repository. (We did it in `/tmp` or some other directory so that we don't overwrite our existing `planets` directory.) @@ -177,39 +168,33 @@ Our computer now has two copies of the repository: Let's make a change in the copy in `/tmp/planets`: -
~~~ $ cd /tmp/planets $ nano pluto.txt $ cat pluto.txt ~~~ -
-
+{:class="in"} ~~~ It is so a planet! ~~~ -
-
+{:class="out"} ~~~ $ git add pluto.txt $ git commit -m "Some notes about Pluto" ~~~ -
-
+{:class="in"} ~~~ 1 file changed, 1 insertion(+) create mode 100644 pluto.txt ~~~ -
+{:class="out"} then push the change to GitHub: -
~~~ $ git push origin master ~~~ -
-
+{:class="in"} ~~~ Counting objects: 4, done. Delta compression using up to 4 threads. @@ -219,7 +204,7 @@ Total 3 (delta 0), reused 0 (delta 0) To https://github.com/vlad/planets.git 9272da5..29aba7c master -> master ~~~ -
+{:class="out"} Note that we didn't have to create a remote called `origin`: Git does this automatically, @@ -234,13 +219,11 @@ Our three repositories now look like this: We can now download changes into the original repository on our machine: -
~~~ $ cd ~/planets $ git pull origin master ~~~ -
-
+{:class="in"} ~~~ remote: Counting objects: 4, done. remote: Compressing objects: 100% (2/2), done. @@ -254,7 +237,7 @@ Fast-forward 1 file changed, 1 insertion(+) create mode 100644 pluto.txt ~~~ -
+{:class="out"} which gives us this: diff --git a/novice/git/03-conflict.md b/novice/git/03-conflict.md index b1360f239..20c002cec 100644 --- a/novice/git/03-conflict.md +++ b/novice/git/03-conflict.md @@ -25,56 +25,48 @@ The file `mars.txt` currently looks like this in both local copies of our `planets` repository (the one in our home directory and the one in `/tmp`): -
~~~ $ cat mars.txt ~~~ -
-
+{:class="in"} ~~~ Cold and dry, but everything is my favorite color The two moons may be a problem for Wolfman But the Mummy will appreciate the lack of humidity ~~~ -
+{:class="out"} Let's add a line to the copy under our home directory: -
~~~ $ nano mars.txt $ cat mars.txt ~~~ -
-
+{:class="in"} ~~~ Cold and dry, but everything is my favorite color The two moons may be a problem for Wolfman But the Mummy will appreciate the lack of humidity This line added to our home copy ~~~ -
+{:class="out"} and then push the change to GitHub: -
~~~ $ git add mars.txt $ git commit -m "Adding a line in our home copy" ~~~ -
-
+{:class="in"} ~~~ [master 5ae9631] Adding a line in our home copy 1 file changed, 1 insertion(+) ~~~ -
-
+{:class="out"} ~~~ $ git push origin master ~~~ -
-
+{:class="in"} ~~~ Counting objects: 5, done. Delta compression using up to 4 threads. @@ -84,7 +76,7 @@ Total 3 (delta 1), reused 0 (delta 0) To https://github.com/vlad/planets 29aba7c..dabb4c8 master -> master ~~~ -
+{:class="out"} Our repositories are now in this state: @@ -94,45 +86,39 @@ Now let's switch to the copy under `/tmp` and make a different change there *without* updating from GitHub: -
~~~ $ cd /tmp/planets $ nano mars.txt $ cat mars.txt ~~~ -
-
+{:class="in"} ~~~ Cold and dry, but everything is my favorite color The two moons may be a problem for Wolfman But the Mummy will appreciate the lack of humidity We added a different line in the temporary copy ~~~ -
+{:class="out"} We can commit the change locally: -
~~~ $ git add mars.txt $ git commit -m "Adding a line in the temporary copy" ~~~ -
-
+{:class="in"} ~~~ [master 07ebc69] Adding a line in the temporary copy 1 file changed, 1 insertion(+) ~~~ -
+{:class="out"} but Git won't let us push it to GitHub: -
~~~ $ git push origin master ~~~ -
-
+{:class="in"} ~~~ To https://github.com/vlad/planets.git ! [rejected] master -> master (non-fast-forward) @@ -142,7 +128,7 @@ hint: its remote counterpart. Merge the remote changes (e.g. 'git pull') hint: before pushing again. hint: See the 'Note about fast-forwards' in 'git push --help' for details. ~~~ -
+{:class="out"} Git detects that the changes made in one copy overlap with those made in the other and stops us from trampling on our previous work. @@ -151,12 +137,10 @@ What we have to do is pull the changes from GitHub, and then push that. Let's start by pulling: -
~~~ $ git pull origin master ~~~ -
-
+{:class="in"} ~~~ remote: Counting objects: 5, done. remote: Compressing objects: 100% (2/2), done. @@ -168,17 +152,15 @@ Auto-merging mars.txt CONFLICT (content): Merge conflict in mars.txt Automatic merge failed; fix conflicts and then commit the result. ~~~ -
+{:class="out"} `git pull` tells us there's a conflict, and marks that conflict in the affected file: -
~~~ $ cat mars.txt ~~~ -
-
+{:class="in"} ~~~ Cold and dry, but everything is my favorite color The two moons may be a problem for Wolfman @@ -189,7 +171,7 @@ We added a different line in the temporary copy This line added to our home copy >>>>>>> dabb4c8c450e8475aee9b14b4383acc99f42af1d ~~~ -
+{:class="out"} Our change---the one in `HEAD`---is preceded by `<<<<<<<`. Git has then inserted `=======` as a separator between the conflicting changes @@ -206,31 +188,27 @@ write something new to replace both, or get rid of the change entirely. Let's replace both so that the file looks like this: -
~~~ $ cat mars.txt ~~~ -
-
+{:class="in"} ~~~ Cold and dry, but everything is my favorite color The two moons may be a problem for Wolfman But the Mummy will appreciate the lack of humidity We removed the conflict on this line ~~~ -
+{:class="out"} To finish merging, we add `mars.txt` to the changes being made by the merge and then commit: -
~~~ $ git add mars.txt $ git status ~~~ -
-
+{:class="in"} ~~~ # On branch master # All conflicts fixed but you are still merging. @@ -241,17 +219,15 @@ $ git status # modified: mars.txt # ~~~ -
-
+{:class="out"} ~~~ $ git commit -m "Merging changes from GitHub" ~~~ -
-
+{:class="in"} ~~~ [master 2abf2b1] Merging changes from GitHub ~~~ -
+{:class="out"} Our repositories now look like this: @@ -259,12 +235,10 @@ Our repositories now look like this: so we push our changes to GitHub: -
~~~ $ git push origin master ~~~ -
-
+{:class="in"} ~~~ Counting objects: 10, done. Delta compression using up to 4 threads. @@ -274,7 +248,7 @@ Total 6 (delta 2), reused 0 (delta 0) To https://github.com/vlad/planets.git dabb4c8..2abf2b1 master -> master ~~~ -
+{:class="out"} to get this: @@ -284,13 +258,11 @@ Git keeps track of what we've merged with what, so we don't have to fix things by hand again if we switch back to the repository in our home directory and pull from GitHub: -
~~~ $ cd ~/planets $ git pull origin master ~~~ -
-
+{:class="in"} ~~~ remote: Counting objects: 10, done. remote: Compressing objects: 100% (4/4), done. @@ -303,23 +275,21 @@ Fast-forward mars.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) ~~~ -
+{:class="out"} we get the merged file: -
~~~ $ cat mars.txt ~~~ -
-
+{:class="in"} ~~~ Cold and dry, but everything is my favorite color The two moons may be a problem for Wolfman But the Mummy will appreciate the lack of humidity We removed the conflict on this line ~~~ -
+{:class="out"} We don't need to merge again because GitHub knows someone has already done that. diff --git a/novice/shell/01-filedir.md b/novice/shell/01-filedir.md index 0f763dc20..b361c8b6b 100644 --- a/novice/shell/01-filedir.md +++ b/novice/shell/01-filedir.md @@ -26,11 +26,10 @@ Several commands are frequently used to create, inspect, rename, and delete file To start exploring them, let's open a shell window: -
~~~ $ ~~~ -
+{:class="in"} The dollar sign is a [prompt](../../gloss.html#prompt), which shows us that the shell is waiting for input; @@ -42,16 +41,14 @@ The command's output is the ID of the current user, i.e., it shows us who the shell thinks we are: -
~~~ $ whoami ~~~ -
-
+{:class="in"} ~~~ vlad ~~~ -
+{:class="out"} More specifically, when we type `whoami` the shell: @@ -73,16 +70,14 @@ Here, the computer's response is `/users/vlad`, which is Vlad's [home directory](../../gloss.html#home-directory): -
~~~ $ pwd ~~~ -
-
+{:class="in"} ~~~ /users/vlad ~~~ -
+{:class="out"} > #### Alphabet Soup > @@ -136,18 +131,16 @@ which is why `vlad` is the last part of the directory's name. Let's see what's in Vlad's home directory by running `ls`, which stands for "listing": -
~~~ $ ls ~~~ -
-
+{:class="in"} ~~~ bin data mail music notes.txt papers pizza.cfg solar solar.pdf swc ~~~ -
+{:class="out"} Vlad's Home Directory @@ -156,18 +149,16 @@ arranged neatly into columns. We can make its output more comprehensible by using the [flag](../../gloss.html#command-line-flag) `-F`, which tells `ls` to add a trailing `/` to the names of directories: -
~~~ $ ls -F ~~~ -
-
+{:class="in"} ~~~ bin/ data/ mail/ music/ notes.txt papers/ pizza.cfg solar/ solar.pdf swc/ ~~~ -
+{:class="out"} Here, we can see that `/users/vlad` contains seven [sub-directories](../../gloss.html#sub-directory). @@ -206,17 +197,15 @@ the command `ls` with the parameters `-F` and `data`. The second parameter—the one *without* a leading dash—tells `ls` that we want a listing of something other than our current working directory: -
~~~ $ ls -F data ~~~ -
-
+{:class="in"} ~~~ amino-acids.txt elements/ morse.txt pdb/ planets.txt sunspot.txt ~~~ -
+{:class="out"} The output shows us that there are four text files and two sub-sub-directories. Organizing things hierarchically in this way helps us keep track of our work: @@ -234,17 +223,15 @@ rather than from the root of the file system. If we run `ls -F /data` (*with* a leading slash) we get a different answer, because `/data` is an [absolute path](../../gloss.html#absolute-path): -
~~~ $ ls -F /data ~~~ -
-
+{:class="in"} ~~~ access.log backup/ hardware.cfg network.cfg ~~~ -
+{:class="out"} The leading `/` tells the computer to follow the path from the root of the filesystem, so it always refers to exactly one directory, @@ -255,28 +242,24 @@ Before we do this, `pwd` shows us that we're in `/users/vlad`, and `ls` without any parameters shows us that directory's contents: -
~~~ $ pwd ~~~ -
-
+{:class="in"} ~~~ /users/vlad ~~~ -
-
+{:class="out"} ~~~ $ ls ~~~ -
-
+{:class="in"} ~~~ bin/ data/ mail/ music/ notes.txt papers/ pizza.cfg solar/ solar.pdf swc/ ~~~ -
+{:class="out"} We can use `cd` followed by a directory name to change our working directory. `cd` stands for "change directory", @@ -284,11 +267,10 @@ which is a bit misleading: the command doesn't change the directory, it changes the shell's idea of what directory we are in. -
~~~ $ cd data ~~~ -
+{:class="in"} `cd` doesn't print anything, but if we run `pwd` after it, we can see that we are now in `/users/vlad/data`. @@ -296,55 +278,47 @@ If we run `ls` without parameters now, it lists the contents of `/users/vlad/data`, because that's where we now are: -
~~~ $ pwd ~~~ -
-
+{:class="in"} ~~~ /users/vlad/data ~~~ -
-
+{:class="out"} ~~~ $ ls ~~~ -
-
+{:class="in"} ~~~ amino-acids.txt elements/ morse.txt pdb/ planets.txt sunspot.txt ~~~ -
+{:class="out"} We now know how to go down the directory tree: how do we go up? We could use an absolute path: -
~~~ $ cd /users/vlad ~~~ -
+{:class="in"} but it's almost always simpler to use `cd ..` to go up one level: -
~~~ $ pwd ~~~ -
-
+{:class="in"} ~~~ /users/vlad/data ~~~ -
-
+{:class="out"} ~~~ $ cd .. ~~~ -
+{:class="in"} `..` is a special directory name meaning "the directory containing this one", @@ -353,32 +327,28 @@ the [parent](../../gloss.html#parent-directory) of the current directory. Sure enough, if we run `pwd` after running `cd ..`, we're back in `/users/vlad`: -
~~~ $ pwd ~~~ -
-
+{:class="in"} ~~~ /users/vlad ~~~ -
+{:class="out"} The special directory `..` doesn't usually show up when we run `ls`. If we want to display it, we can give `ls` the `-a` flag: -
~~~ $ ls -F -a ~~~ -
-
+{:class="in"} ~~~ ./ ../ bin/ data/ mail/ music/ notes.txt papers/ pizza.cfg solar/ solar.pdf swc/ ~~~ -
+{:class="out"} `-a` stands for "show all"; it forces `ls` to show us file and directory names that begin with `.`, @@ -437,30 +407,27 @@ All 1520 files will go into the same directory. If she is in her home directory, Nelle can see what files she has using the command: -
~~~ $ ls north-pacific-gyre/2012-07-03/ ~~~ -
+{:class="in"} This is a lot to type, but she can let the shell do most of the work. If she types: -
~~~ $ ls no ~~~ -
+{:class="in"} and then presses tab, the shell automatically completes the directory name for her: -
~~~ $ ls north-pacific-gyre/ ~~~ -
+{:class="in"} If she presses tab again, Bash will add `2012-07-03/` to the command, diff --git a/novice/shell/02-create.md b/novice/shell/02-create.md index 1c590951b..08862d2c6 100644 --- a/novice/shell/02-create.md +++ b/novice/shell/02-create.md @@ -19,37 +19,32 @@ Let's go back to Vlad's home directory, `/users/vlad`, and use `ls -F` to see what it contains: -
~~~ $ pwd ~~~ -
-
+{:class="in"} ~~~ /users/vlad ~~~ -
-
+{:class="out"} ~~~ $ ls -F ~~~ -
-
+{:class="in"} ~~~ bin/ data/ mail/ music/ notes.txt papers/ pizza.cfg solar/ solar.pdf swc/ ~~~ -
+{:class="out"} Let's create a new directory called `thesis` using the command `mkdir thesis` (which has no output): -
~~~ $ mkdir thesis ~~~ -
+{:class="in"} As you might (or might not) guess from its name, `mkdir` means "make directory". @@ -57,36 +52,32 @@ Since `thesis` is a relative path (i.e., doesn't have a leading slash), the new directory is made below the current working directory: -
~~~ $ ls -F ~~~ -
-
+{:class="in"} ~~~ bin/ data/ mail/ music/ notes.txt papers/ pizza.cfg solar/ solar.pdf swc/ thesis/ ~~~ -
+{:class="out"} However, there's nothing in it yet: -
~~~ $ ls -F thesis ~~~ -
+{:class="in"} Let's change our working directory to `thesis` using `cd`, then run a text editor called Nano to create a file called `draft.txt`: -
~~~ $ cd thesis $ nano draft.txt ~~~ -
+{:class="in"} > #### Which Editor? > @@ -119,35 +110,31 @@ we can use Control-X to quit the editor and return to the shell. `nano` doesn't leave any output on the screen after it exits, but `ls` now shows that we have created a file called `draft.txt`: -
~~~ $ ls ~~~ -
-
+{:class="in"} ~~~ draft.txt ~~~ -
+{:class="out"} Let's tidy up by running `rm draft.txt`: -
~~~ $ rm draft.txt ~~~ -
+{:class="in"} This command removes files ("rm" is short for "remove"). If we run `ls` again, its output is empty once more, which tells us that our file is gone: -
~~~ $ ls ~~~ -
+{:class="in"} > #### Deleting Is Forever > @@ -160,46 +147,39 @@ $ ls Let's re-create that file and then move up one directory to `/users/vlad` using `cd ..`: -
~~~ $ pwd ~~~ -
-
+{:class="in"} ~~~ /users/vlad/thesis ~~~ -
-
+{:class="out"} ~~~ $ nano draft.txt $ ls ~~~ -
-
+{:class="in"} ~~~ draft.txt ~~~ -
-
+{:class="out"} ~~~ $ cd .. ~~~ -
+{:class="in"} If we try to remove the entire `thesis` directory using `rm thesis`, we get an error message: -
~~~ $ rm thesis ~~~ -
-
+{:class="in"} ~~~ rm: cannot remove `thesis': Is a directory ~~~ -
+{:class="err"} This happens because `rm` only works on files, not directories. The right command is `rmdir`, @@ -207,34 +187,30 @@ which is short for "remove directory". It doesn't work yet either, though, because the directory we're trying to remove isn't empty: -
~~~ $ rmdir thesis ~~~ -
-
+{:class="in"} ~~~ rmdir: failed to remove `thesis': Directory not empty ~~~ -
+{:class="err"} This little safety feature can save you a lot of grief, particularly if you are a bad typist. To really get rid of `thesis` we must first delete the file `draft.txt`: -
~~~ $ rm thesis/draft.txt ~~~ -
+{:class="in"} The directory is now empty, so `rmdir` can delete it: -
~~~ $ rmdir thesis ~~~ -
+{:class="in"} > #### With Great Power Comes Great Responsibility > @@ -255,42 +231,36 @@ Let's create that directory and file one more time. (Note that this time we're running `nano` with the path `thesis/draft.txt`, rather than going into the `thesis` directory and running `nano` on `draft.txt` there.) -
~~~ $ pwd ~~~ -
-
+{:class="in"} ~~~ /users/vlad ~~~ -
-
+{:class="out"} ~~~ $ mkdir thesis ~~~ -
-
+{:class="in"} ~~~ $ nano thesis/draft.txt $ ls thesis ~~~ -
-
+{:class="in"} ~~~ draft.txt ~~~ -
+{:class="out"} `draft.txt` isn't a particularly informative name, so let's change the file's name using `mv`, which is short for "move": -
~~~ $ mv thesis/draft.txt thesis/quotes.txt ~~~ -
+{:class="in"} The first parameter tells `mv` what we're "moving", while the second is where it's to go. @@ -300,16 +270,14 @@ which has the same effect as renaming the file. Sure enough, `ls` shows us that `thesis` now contains one file called `quotes.txt`: -
~~~ $ ls thesis ~~~ -
-
+{:class="in"} ~~~ quotes.txt ~~~ -
+{:class="out"} Just for the sake of inconsistency, `mv` also works on directories—there is no separate `mvdir` command. @@ -323,35 +291,31 @@ but put the file somewhere new. In this case, the directory name we use is the special directory name `.` that we mentioned earlier. -
~~~ $ mv thesis/quotes.txt . ~~~ -
+{:class="in"} The effect is to move the file from the directory it was in to the current working directory. `ls` now shows us that `thesis` is empty: -
~~~ $ ls thesis ~~~ -
+{:class="in"} Further, `ls` with a filename or directory name as a parameter only lists that file or directory. We can use this to see that `quotes.txt` is still in our current directory: -
~~~ $ ls quotes.txt ~~~ -
-
+{:class="in"} ~~~ quotes.txt ~~~ -
+{:class="out"} The `cp` command works very much like `mv`, except it copies a file instead of moving it. @@ -359,17 +323,15 @@ We can check that it did the right thing using `ls` with two paths as parameters—like most Unix commands, `ls` can be given thousands of paths at once: -
~~~ $ cp quotes.txt thesis/quotations.txt $ ls quotes.txt thesis/quotations.txt ~~~ -
-
+{:class="in"} ~~~ quotes.txt thesis/quotations.txt ~~~ -
+{:class="out"} To prove that we made a copy, let's delete the `quotes.txt` file in the current directory @@ -377,17 +339,15 @@ and then run that same `ls` again. This time it tells us that it can't find `quotes.txt` in the current directory, but it does find the copy in `thesis` that we didn't delete: -
~~~ $ ls quotes.txt thesis/quotations.txt ~~~ -
-
+{:class="in"} ~~~ ls: cannot access quotes.txt: No such file or directory thesis/quotations.txt ~~~ -
+{:class="err"} > #### Another Useful Abbreviation > diff --git a/novice/shell/03-pipefilter.md b/novice/shell/03-pipefilter.md index b976aa5f9..517e7a7b5 100644 --- a/novice/shell/03-pipefilter.md +++ b/novice/shell/03-pipefilter.md @@ -22,17 +22,15 @@ that contains six files describing some simple organic molecules. The `.pdb` extension indicates that these files are in Protein Data Bank format, a simple text format that specifies the type and position of each atom in the molecule. -
~~~ $ ls molecules ~~~ -
-
+{:class="in"} ~~~ cubane.pdb ethane.pdb methane.pdb octane.pdb pentane.pdb propane.pdb ~~~ -
+{:class="out"} Let's go into that directory with `cd` and run the command `wc *.pdb`. `wc` is the "word count" command: @@ -40,13 +38,11 @@ it counts the number of lines, words, and characters in files. The `*` in `*.pdb` matches zero or more characters, so the shell turns `*.pdb` into a complete list of `.pdb` files: -
~~~ $ cd molecules $ wc *.pdb ~~~ -
-
+{:class="in"} ~~~ 20 156 1158 cubane.pdb 12 84 622 ethane.pdb @@ -56,7 +52,7 @@ $ wc *.pdb 15 111 825 propane.pdb 107 819 6081 total ~~~ -
+{:class="out"} > #### Wildcards > @@ -85,12 +81,10 @@ $ wc *.pdb If we run `wc -l` instead of just `wc`, the output shows only the number of lines per file: -
~~~ $ wc -l *.pdb ~~~ -
-
+{:class="in"} ~~~ 20 cubane.pdb 12 ethane.pdb @@ -100,7 +94,7 @@ $ wc -l *.pdb 15 propane.pdb 107 total ~~~ -
+{:class="out"} We can also use `-w` to get only the number of words, or `-c` to get only the number of characters. @@ -110,11 +104,10 @@ It's an easy question to answer when there are only six files, but what if there were 6000? Our first step toward a solution is to run the command: -
~~~ $ wc -l *.pdb > lengths ~~~ -
+{:class="in"} The `>` tells the shell to [redirect](../../gloss.html#redirect) the command's output to a file instead of printing it to the screen. @@ -124,16 +117,14 @@ or overwrite the contents of that file if it does. everything that `wc` would have printed has gone into the file `lengths` instead.) `ls lengths` confirms that the file exists: -
~~~ $ ls lengths ~~~ -
-
+{:class="in"} ~~~ lengths ~~~ -
+{:class="out"} We can now send the content of `lengths` to the screen using `cat lengths`. `cat` stands for "concatenate": @@ -141,12 +132,10 @@ it prints the contents of files one after another. There's only one file in this case, so `cat` just shows us what it contains: -
~~~ $ cat lengths ~~~ -
-
+{:class="in"} ~~~ 20 cubane.pdb 12 ethane.pdb @@ -156,18 +145,16 @@ $ cat lengths 15 propane.pdb 107 total ~~~ -
+{:class="out"} Now let's use the `sort` command to sort its contents. This does *not* change the file; instead, it sends the sorted result to the screen: -
~~~ $ sort lengths ~~~ -
-
+{:class="in"} ~~~ 9 methane.pdb 12 ethane.pdb @@ -177,7 +164,7 @@ $ sort lengths 30 octane.pdb 107 total ~~~ -
+{:class="out"} We can put the sorted list of lines in another temporary file called `sorted-lengths` by putting `> sorted-lengths` after the command, @@ -185,17 +172,15 @@ just as we used `> lengths` to put the output of `wc` into `lengths`. Once we've done that, we can run another command called `head` to get the first few lines in `sorted-lengths`: -
~~~ $ sort lengths > sorted-lengths $ head -1 sorted-lengths ~~~ -
-
+{:class="in"} ~~~ 9 methane.pdb ~~~ -
+{:class="out"} Using the parameter `-1` with `head` tells it that we only want the first line of the file; @@ -210,16 +195,14 @@ even once you understand what `wc`, `sort`, and `head` do, all those intermediate files make it hard to follow what's going on. We can make it easier to understand by running `sort` and `head` together: -
~~~ $ sort lengths | head -1 ~~~ -
-
+{:class="in"} ~~~ 9 methane.pdb ~~~ -
+{:class="out"} The vertical bar between the two commands is called a [pipe](../../gloss.html#pipe). It tells the shell that we want to use @@ -233,16 +216,14 @@ we don't have to know or care. We can use another pipe to send the output of `wc` directly to `sort`, which then sends its output to `head`: -
~~~ $ wc -l *.pdb | sort | head -1 ~~~ -
-
+{:class="in"} ~~~ 9 methane.pdb ~~~ -
+{:class="out"} This is exactly like a mathematician nesting functions like *sin(πx)2* and saying "the square of the sine of *x* times π". @@ -323,16 +304,14 @@ Nelle has run her samples through the assay machines and created 1520 files in the `north-pacific-gyre/2012-07-03` directory described earlier. As a quick sanity check, she types: -
~~~ $ cd north-pacific-gyre/2012-07-03 $ wc -l *.txt ~~~ -
+{:class="in"} The output is 1520 lines that look like this: -
~~~ 300 NENE01729A.txt 300 NENE01729B.txt @@ -342,16 +321,14 @@ The output is 1520 lines that look like this: 300 NENE01812A.txt ... ... ~~~ -
+{:class="out"} Now she types this: -
~~~ $ wc -l *.txt | sort | head -5 ~~~ -
-
+{:class="in"} ~~~ 240 NENE02018B.txt 300 NENE01729A.txt @@ -359,7 +336,7 @@ $ wc -l *.txt | sort | head -5 300 NENE01736A.txt 300 NENE01751A.txt ~~~ -
+{:class="out"} Whoops: one of the files is 60 lines shorter than the others. When she goes back and checks it, @@ -369,12 +346,10 @@ and she forgot to reset it. Before re-running that sample, she checks to see if any files have too much data: -
~~~ $ wc -l *.txt | sort | tail -5 ~~~ -
-
+{:class="in"} ~~~ 300 NENE02040A.txt 300 NENE02040B.txt @@ -382,7 +357,7 @@ $ wc -l *.txt | sort | tail -5 300 NENE02043A.txt 300 NENE02043B.txt ~~~ -
+{:class="out"} Those numbers look good—but what's that 'Z' doing there in the third-to-last line? All of her samples should be marked 'A' or 'B'; @@ -390,16 +365,14 @@ by convention, her lab uses 'Z' to indicate samples with missing information. To find others like it, she does this: -
~~~ $ ls *Z.txt ~~~ -
-
+{:class="in"} ~~~ NENE01971Z.txt NENE02040Z.txt ~~~ -
+{:class="out"} Sure enough, when she checks the log on her laptop, diff --git a/novice/shell/04-loop.md b/novice/shell/04-loop.md index 59bab5522..57fb98d55 100644 --- a/novice/shell/04-loop.md +++ b/novice/shell/04-loop.md @@ -22,19 +22,17 @@ When new files arrive, we'd like to rename the existing ones to `original-basilisk.dat` and `original-unicorn.dat`. We can't use: -
~~~ $ mv *.dat original-*.dat ~~~ -
+{:class="in"} because that would expand (in the two-file case) to: -
~~~ $ mv basilisk.dat unicorn.dat ~~~ -
+{:class="in"} This wouldn't back up our files: it would replace the content of `unicorn.dat` with whatever's in `basilisk.dat`. @@ -43,15 +41,13 @@ Instead, we can use a [loop](../../gloss.html#for-loop) to do some operation once for each thing in a list. Here's a simple example that displays the first three lines of each file in turn: -
~~~ $ for filename in basilisk.dat unicorn.dat > do > head -3 $filename > done ~~~ -
-
+{:class="in"} ~~~ COMMON NAME: basilisk CLASSIFICATION: basiliscus vulgaris @@ -60,7 +56,7 @@ COMMON NAME: unicorn CLASSIFICATION: equus monoceros UPDATED: 1738-11-24 ~~~ -
+{:class="out"} When the shell sees the keyword `for`, it knows it is supposed to repeat a command (or group of commands) once for each thing in a list. @@ -88,25 +84,23 @@ in order to make its purpose clearer to human readers. The shell itself doesn't care what the variable is called; if we wrote this loop as: -
~~~ for x in basilisk.dat unicorn.dat do head -3 $x done ~~~ -
+{:class="in"} or: -
~~~ for temperature in basilisk.dat unicorn.dat do head -3 $temperature done ~~~ -
+{:class="in"} it would work exactly the same way. *Don't do this.* @@ -116,7 +110,6 @@ increase the odds that the program won't do what its readers think it does. Here's a slightly more complicated loop: -
~~~ for filename in *.dat do @@ -124,7 +117,7 @@ do head -100 $filename | tail -20 done ~~~ -
+{:class="in"} The shell starts by expanding `*.dat` to create the list of files it will process. The [loop body](../../gloss.html#loop-body) @@ -132,26 +125,23 @@ then executes two commands for each of those files. The first, `echo`, just prints its command-line parameters to standard output. For example: -
~~~ $ echo hello there ~~~ -
+{:class="in"} prints: -
~~~ hello there ~~~ -
+{:class="out"} In this case, since the shell expands `$filename` to be the name of a file, `echo $filename` just prints the name of the file. Note that we can't write this as: -
~~~ for filename in *.dat do @@ -159,7 +149,7 @@ do head -100 $filename | tail -20 done ~~~ -
+{:class="in"} because then the first time through the loop, when `$filename` expanded to `basilisk.dat`, the shell would try to run `basilisk.dat` as a program. @@ -222,33 +212,30 @@ the `head` and `tail` combination selects lines 81-100 from whatever file is bei Going back to our original file renaming problem, we can solve it using this loop: -
~~~ for filename in *.dat do mv $filename original-$filename done ~~~ -
+{:class="in"} This loop runs the `mv` command once for each filename. The first time, when `$filename` expands to `basilisk.dat`, the shell executes: -
~~~ mv basilisk.dat original-basilisk.dat ~~~ -
+{:class="in"} The second time, the command is: -
~~~ mv unicorn.dat original-unicorn.dat ~~~ -
+{:class="in"} > #### Measure Twice, Run Once > @@ -285,7 +272,6 @@ she decides to build up the required commands in stages. Her first step is to make sure that she can select the right files—remember, these are ones whose names end in 'A' or 'B', rather than 'Z': -
~~~ $ cd north-pacific-gyre/2012-07-03 $ for datafile in *[AB].txt @@ -293,8 +279,7 @@ $ for datafile in *[AB].txt > echo $datafile > done ~~~ -
-
+{:class="in"} ~~~ NENE01729A.txt NENE01729B.txt @@ -303,22 +288,20 @@ NENE01736A.txt NENE02043A.txt NENE02043B.txt ~~~ -
+{:class="out"} Her next step is to decide what to call the files that the `goostats` analysis program will create. Prefixing each input file's name with "stats" seems simple, so she modifies her loop to do that: -
~~~ $ for datafile in *[AB].txt > do > echo $datafile stats-$datafile > done ~~~ -
-
+{:class="in"} ~~~ NENE01729A.txt stats-NENE01729A.txt NENE01729B.txt stats-NENE01729B.txt @@ -327,7 +310,7 @@ NENE01736A.txt stats-NENE01736A.txt NENE02043A.txt stats-NENE02043A.txt NENE02043B.txt stats-NENE02043B.txt ~~~ -
+{:class="out"} She hasn't actually run `goostats` yet, but now she's sure she can select the right files and generate the right output filenames. @@ -341,20 +324,18 @@ In response, the shell redisplays the whole loop on one line (using semi-colons to separate the pieces): -
~~~ $ for datafile in *[AB].txt; do echo $datafile stats-$datafile; done ~~~ -
+{:class="in"} Using the left arrow key, Nelle backs up and changes the command `echo` to `goostats`: -
~~~ $ for datafile in *[AB].txt; do bash goostats $datafile stats-$datafile; done ~~~ -
+{:class="in"} When she presses enter, the shell runs the modified command. @@ -365,11 +346,10 @@ She kills the job by typing Control-C, uses up-arrow to repeat the command, and edits it to read: -
~~~ $ for datafile in *[AB].txt; do echo $datafile; bash goostats $datafile stats-$datafile; done ~~~ -
+{:class="in"} > #### Beginning and End > @@ -380,14 +360,13 @@ $ for datafile in *[AB].txt; do echo $datafile; bash goostats $datafile stats-$d When she runs her program now, it produces one line of output every five seconds or so: -
~~~ NENE01729A.txt NENE01729B.txt NENE01736A.txt ... ~~~ -
+{:class="out"} 1518 times 5 seconds, divided by 60, diff --git a/novice/shell/05-script.md b/novice/shell/05-script.md index 723d45d21..799e6022c 100644 --- a/novice/shell/05-script.md +++ b/novice/shell/05-script.md @@ -38,12 +38,10 @@ Once we have saved the file, we can ask the shell to execute the commands it contains. Our shell is called `bash`, so we run the following command: -
~~~ $ bash middle.sh ~~~ -
-
+{:class="in"} ~~~ ATOM 14 C 1 -1.463 -0.666 1.001 1.00 0.00 ATOM 15 C 1 0.762 -0.929 0.295 1.00 0.00 @@ -51,7 +49,7 @@ ATOM 16 C 1 0.771 -0.937 1.840 1.00 0.00 ATOM 17 C 1 -0.664 -0.610 2.293 1.00 0.00 ATOM 18 C 1 -4.705 2.108 -0.396 1.00 0.00 ~~~ -
+{:class="out"} Sure enough, our script's output is exactly what we would get if we ran that pipeline directly. @@ -74,27 +72,23 @@ but that would probably take longer than just retyping the command. Instead, let's edit `middle.sh` and replace `cholesterol.pdb` with a special variable called `$1`: -
~~~ $ cat middle.sh ~~~ -
-
+{:class="in"} ~~~ head -20 $1 | tail -5 ~~~ -
+{:class="out"} Inside a shell script, `$1` means "the first filename (or other parameter) on the command line". We can now run our script like this: -
~~~ $ bash middle.sh cholesterol.pdb ~~~ -
-
+{:class="in"} ~~~ ATOM 14 C 1 -1.463 -0.666 1.001 1.00 0.00 ATOM 15 C 1 0.762 -0.929 0.295 1.00 0.00 @@ -102,16 +96,14 @@ ATOM 16 C 1 0.771 -0.937 1.840 1.00 0.00 ATOM 17 C 1 -0.664 -0.610 2.293 1.00 0.00 ATOM 18 C 1 -4.705 2.108 -0.396 1.00 0.00 ~~~ -
+{:class="out"} or on a different file like this: -
~~~ $ bash middle.sh vitamin-a.pdb ~~~ -
-
+{:class="in"} ~~~ ATOM 14 C 1 1.788 -0.987 -0.861 ATOM 15 C 1 2.994 -0.265 -0.829 @@ -119,28 +111,24 @@ ATOM 16 C 1 4.237 -0.901 -1.024 ATOM 17 C 1 5.406 -0.117 -1.087 ATOM 18 C 1 -0.696 -2.628 -0.641 ~~~ -
+{:class="out"} We still need to edit `middle.sh` each time we want to adjust the range of lines, though. Let's fix that by using the special variables `$2` and `$3`: -
~~~ $ cat middle.sh ~~~ -
-
+{:class="in"} ~~~ head $2 $1 | tail $3 ~~~ -
-
+{:class="out"} ~~~ $ bash middle.sh vitamin-a.pdb -20 -5 ~~~ -
-
+{:class="in"} ~~~ ATOM 14 C 1 1.788 -0.987 -0.861 ATOM 15 C 1 2.994 -0.265 -0.829 @@ -148,24 +136,22 @@ ATOM 16 C 1 4.237 -0.901 -1.024 ATOM 17 C 1 5.406 -0.117 -1.087 ATOM 18 C 1 -0.696 -2.628 -0.641 ~~~ -
+{:class="out"} This works, but it may take the next person who reads `middle.sh` a moment to figure out what it does. We can improve our script by adding some [comments](../../gloss.html#comment) at the top: -
~~~ $ cat middle.sh ~~~ -
-
+{:class="in"} ~~~ # Select lines from the middle of a file. # Usage: middle.sh filename -end_line -num_lines head $2 $1 | tail $3 ~~~ -
+{:class="out"} A comment starts with a `#` character and runs to the end of the line. The computer ignores comments, @@ -174,11 +160,10 @@ but they're invaluable for helping people understand and use scripts. What if we want to process many files in a single pipeline? For example, if we want to sort our `.pdb` files by length, we would type: -
~~~ $ wc -l *.pdb | sort -n ~~~ -
+{:class="in"} because `wc -l` lists the number of lines in the files and `sort -n` sorts things numerically. @@ -193,22 +178,18 @@ which means, "All of the command-line parameters to the shell script." Here's an example: -
~~~ $ cat sorted.sh ~~~ -
-
+{:class="in"} ~~~ wc -l $* | sort -n ~~~ -
-
+{:class="out"} ~~~ $ bash sorted.sh *.dat backup/*.dat ~~~ -
-
+{:class="in"} ~~~ 29 chloratin.dat 89 backup/chloratin.dat @@ -217,7 +198,7 @@ $ bash sorted.sh *.dat backup/*.dat 172 backup/sphag-merged.dat 182 girmanis.dat ~~~ -
+{:class="out"} > #### Why Isn't It Doing Anything? > @@ -273,11 +254,10 @@ Instead of typing them in again (and potentially getting them wrong) we can do this: -
~~~ $ history | tail -4 > redo-figure-3.sh ~~~ -
+{:class="in"} The file `redo-figure-3.sh` now contains: @@ -350,19 +330,17 @@ done She saves this in a file called `do-stats.sh` so that she can now re-do the first stage of her analysis by typing: -
~~~ $ bash do-stats.sh *[AB].txt ~~~ -
+{:class="in"} She can also do this: -
~~~ $ bash do-stats.sh *[AB].txt | wc -l ~~~ -
+{:class="in"} so that the output is just the number of files processed rather than the names of the files that were processed. diff --git a/novice/shell/06-find.md b/novice/shell/06-find.md index 506c9a920..2b317fb22 100644 --- a/novice/shell/06-find.md +++ b/novice/shell/06-find.md @@ -25,12 +25,10 @@ For our examples, we will use a file that contains three haikus taken from a 1998 competition in *Salon* magazine: -
~~~ $ cat haiku.txt ~~~ -
-
+{:class="in"} ~~~ The Tao that is seen Is not the true Tao, until @@ -44,7 +42,7 @@ Yesterday it worked Today it is not working Software is like that. ~~~ -
+{:class="out"} > #### Forever, or Five Years > @@ -54,18 +52,16 @@ Software is like that. Let's find lines that contain the word "not": -
~~~ $ grep not haiku.txt ~~~ -
-
+{:class="in"} ~~~ Is not the true Tao, until "My Thesis" not found Today it is not working ~~~ -
+{:class="out"} Here, `not` is the pattern we're searching for. It's pretty simple: @@ -75,17 +71,15 @@ The output is the three lines in the file that contain the letters "not". Let's try a different pattern: "day". -
~~~ $ grep day haiku.txt ~~~ -
-
+{:class="in"} ~~~ Yesterday it worked Today it is not working ~~~ -
+{:class="out"} This time, the output is lines containing the words "Yesterday" and "Today", @@ -94,28 +88,25 @@ If we give `grep` the `-w` flag, it restricts matches to word boundaries, so that only lines with the word "day" will be printed: -
~~~ $ grep -w day haiku.txt ~~~ -
+{:class="in"} In this case, there aren't any, so `grep`'s output is empty. Another useful option is `-n`, which numbers the lines that match: -
~~~ $ grep -n it haiku.txt ~~~ -
-
+{:class="in"} ~~~ 5:With searching comes loss 9:Yesterday it worked 10:Today it is not working ~~~ -
+{:class="out"} Here, we can see that lines 5, 9, and 10 contain the letters "it". @@ -125,12 +116,10 @@ since `-i` makes matching case-insensitive and `-v` inverts the match, using them both only prints lines that *don't* match the pattern in any mix of upper and lower case: -
~~~ $ grep -i -v the haiku.txt ~~~ -
-
+{:class="in"} ~~~ You bring fresh toner. @@ -140,7 +129,7 @@ Yesterday it worked Today it is not working Software is like that. ~~~ -
+{:class="out"} `grep` has lots of other options. To find out what they are, we can type `man grep`. @@ -148,12 +137,10 @@ To find out what they are, we can type `man grep`. it prints a description of a command and its options, and (if you're lucky) provides a few examples of how to use it: -
~~~ $ man grep ~~~ -
-
+{:class="in"} ~~~ GREP(1) GREP(1) @@ -189,7 +176,7 @@ Interpret PATTERN as a list of fixed strings, separated by newlines, any of whi matched. (-F is specified by POSIX.) ... ... ... ~~~ -
+{:class="out"} > #### Wildcards > @@ -238,12 +225,10 @@ Sure enough, `find`'s output is the names of the five directories in our little tree (including `.`): -
~~~ $ find . -type d -print ~~~ -
-
+{:class="in"} ~~~ ./ ./data @@ -251,17 +236,15 @@ $ find . -type d -print ./tools ./tools/old ~~~ -
+{:class="out"} If we change `-type d` to `-type f`, we get a listing of all the files instead: -
~~~ $ find . -type f -print ~~~ -
-
+{:class="in"} ~~~ ./data/one.txt ./data/two.txt @@ -269,7 +252,7 @@ $ find . -type f -print ./tools/format ./tools/stats ~~~ -
+{:class="out"} `find` automatically goes into subdirectories, their subdirectories, @@ -277,62 +260,54 @@ and so on to find everything that matches the pattern we've given it. If we don't want it to, we can use `-maxdepth` to restrict the depth of search: -
~~~ $ find . -maxdepth 1 -type f -print ~~~ -
-
+{:class="in"} ~~~ ./notes.txt ~~~ -
+{:class="out"} The opposite of `-maxdepth` is `-mindepth`, which tells `find` to only report things that are at or below a certain depth. `-mindepth 2` therefore finds all the files that are two or more levels below us: -
~~~ $ find . -mindepth 2 -type f -print ~~~ -
-
+{:class="in"} ~~~ ./data/one.txt ./data/two.txt ./tools/format ./tools/stats ~~~ -
+{:class="out"} Another option is `-empty`, which restricts matches to empty files and directories: -
~~~ $ find . -empty -print ~~~ -
-
+{:class="in"} ~~~ ./thesis ./tools/old ~~~ -
+{:class="out"} Now let's try matching by name: -
~~~ $ find . -name *.txt -print ~~~ -
-
+{:class="in"} ~~~ ./notes.txt ~~~ -
+{:class="out"} We expected it to find all the text files, but it only prints out `./notes.txt`. @@ -340,11 +315,10 @@ The problem is that the shell expands wildcard characters like `*` *before* comm Since `*.txt` in the current directory expands to `notes.txt`, the command we actually ran was: -
~~~ $ find . -name notes.txt -print ~~~ -
+{:class="in"} `find` did what we asked; we just asked for the wrong thing. @@ -354,18 +328,16 @@ put `*.txt` in single quotes to prevent the shell from expanding the `*` wildcar This way, `find` actually gets the pattern `*.txt`, not the expanded filename `notes.txt`: -
~~~ $ find . -name '*.txt' -print ~~~ -
-
+{:class="in"} ~~~ ./data/one.txt ./data/two.txt ./notes.txt ~~~ -
+{:class="out"} > #### Listing vs. Finding > @@ -384,19 +356,17 @@ How can we combine that with `wc -l` to count the lines in all those files? The simplest way is to put the `find` command inside `$()`: -
~~~ $ wc -l $(find . -name '*.txt' -print) ~~~ -
-
+{:class="in"} ~~~ 70 ./data/one.txt 420 ./data/two.txt 30 ./notes.txt 520 total ~~~ -
+{:class="out"} When the shell executes this command, the first thing it does is run whatever is inside the `$()`. @@ -404,11 +374,10 @@ It then replaces the `$()` expression with that command's output. Since the output of `find` is the three filenames `./data/one.txt`, `./data/two.txt`, and `./notes.txt`, the shell constructs the command: -
~~~ $ wc -l ./data/one.txt ./data/two.txt ./notes.txt ~~~ -
+{:class="in"} which is what we wanted. This expansion is exactly what the shell does when it expands wildcards like `*` and `?`, @@ -420,16 +389,14 @@ the second looks for lines inside those files that match another pattern. Here, for example, we can find PDB files that contain iron atoms by looking for the string "FE" in all the `.pdb` files below the current directory: -
~~~ $ grep FE $(find . -name '*.pdb' -print) ~~~ -
-
+{:class="in"} ~~~ ./human/heme.pdb:ATOM 25 FE 1 -0.924 0.535 -0.518 ~~~ -
+{:class="out"} > #### Binary Files > diff --git a/novice/teaching/02-shell.md b/novice/teaching/02-shell.md index a8faefb2b..2158ff6e1 100644 --- a/novice/teaching/02-shell.md +++ b/novice/teaching/02-shell.md @@ -117,12 +117,11 @@ as long as learners using Windows do not run into roadblocks such as: * On Windows, it appears that: -
~~~ $ cd $ cd Desktop ~~~ -
+ {:class="in"} will always put someone on their desktop. Have them create the example directory for the shell exercises there