Downloadable zip file for shell filesystem.

[claresloggett]

This PR is a proposal to add a downloadable zipfile containing the example filesystem used in the Unix shell lessons. This is to help take advantage of @rbeagrie 's recent fixes which make the example filesystem consistent with the lessons. Having this filesystem available as a zip file allows students to get it without needing to install or learn git first. Usually students cover the shell lessons before the git lessons.

I think that adding the zipfile itself is relatively uncontroversial. However I've also edited the first shell lesson to provide some instructions to students to download and unzip the zipfile, and I'm less sure that this is an appropriate thing to do, or that I've done it in the best way. Comments welcome!

The added files are

novice/shell/filesystem.zip
novice/shell/filesystem.tar.gz

although only filesystem.zip is referenced in the lesson text. The .tar.gz file is there so that it's available for instructors in case of problems.

The download-and-unzip instructions are necessarily slightly fuzzy since the location of a downloaded file will depend on what system you're on. So, for those workshops where instructors want to have students do this, I expect that some students will need help. My feeling though is that it's a worthwhile thing to be able to do.

[claresloggett]

One disadvantage of this approach is that the zip archive needs to be kept in sync by anyone who makes future changes to the lessons. This was already the case for the example filesystem itself though, so probably adding a zip step is not much more onerous.

[rgaiacs]

What about add instructions at Makefile to build the .zip? And also add instructions at bin/swc_index_validator.py to check if the leader instructor did

$ git add -f novice/shell/filesystem.zip novice/shell/filesystem.tar.gz

[wking]

On Sun, Sep 14, 2014 at 03:47:00AM -0700, claresloggett wrote:

One disadvantage of this approach is that the zip archive needs to
be kept in sync by anyone who makes future changes to the lessons.

To avoid this issue, I just suggested (in #646) that we keep the
filesystem in Git (like it is now), and point students that have
trouble with Git towards the archive snapshots 1. I'm happy to
continue that discussion in this PR if we think this is the more
appropriate location.

[DamienIrving]

It would be great if the zip file could also include the inflammation data files that are used in the novice programming lessons (they're used in Python, R and MATLAB). That would mean the zip file contains all the files required for the entire bootcamp.

[wking]

On Sun, Sep 14, 2014 at 02:34:47PM -0700, Damien Irving wrote:

It would be great if the zip file could also include the
inflammation data files that are used in the novice programming
lessons (they're used in Python, R and MATLAB). That would mean the
zip file contains all the files required for the entire bootcamp.

+1 to including all the files students will need for a workshop in one
repository.

Personally, I think bc is too large for this (since it will surely
also contain a bunch of files that students will not need). That's
why https://github.com/wking/2014-03-17-uw/ only merged the lessons
and scripts we needed, with as much history as I was able to assemble
in per-lesson-branch form (e.g. all of the Windows-installer history
with wking/2014-03-17-uw@29e5124, but none of the Python history with
wking/2014-03-17-uw@9293c77, see #102 and #374).

[claresloggett]

@r-gaia-cs great idea. We can't remove the need to keep the example filesystem in sync with the lesson text, but we can at least make it hard for the different "versions" of the example filesystem to get out of sync which could be confusing if it happened. I will try your suggestions out.

@wking it seems like the total bc archive snapshots are significantly bigger than just the example filesystem - that would be my main objection to using them. Possibly this is what you're saying in your second comment? The example filesystem alone is really quite small. I haven't yet tried adding in the inflammation datasets as suggested by @DamienIrving but I think they're also of a very feasible size.

[wking]

On Sun, Sep 14, 2014 at 07:55:52PM -0700, claresloggett wrote:

@wking it seems like the total bc archive snapshots are
significantly bigger than just the example filesystem - that would
be my main objection to using them. Possibly this is what you're
saying in your second comment? The example filesystem alone is
really quite small. I haven't yet tried adding in the inflammation
datasets as suggested by @DamienIrving but I think they're also of a
very feasible size.

Yeah. My solution to this is to keep them in separate branches
(e.g. a branch for the example filesystem, a branch for the
inflammation data, …, see #102), and then pull what you want into your
final student-facing repository (or have someone like @gvwilson pull
them into his template student-facing repo, and just reuse his
version ;). For an example of shell examples as separate branches,
see my earlier comment 1.

[jdblischak]

First, this is a great idea. Thanks for putting this together, @claresloggett.

Though I will argue against adding the inflammation data into this zip file from the standpoint of simplicity. Adding more data files would require one extra level of subdirectories that they would have to navigate. It would also require more coordination to combine data files across different lesson sets. In other words, if someone modifies the data files in the Python lessons, they are less likely to remember to update a zip file stored in some other location in the repo. Whereas if the zip file is in the same directory, it is easy to see and realize it also needs updated.

[claresloggett]

@jdblischak Thanks! So far as having more levels of subdirectories goes, right now I don't think this would be an issue. The archive I've got for the unix shell files already has

filesystem/
  data/
  users/
    nelle/

and so on, and students already need to cd filesystem/users/nelle, because the unix lesson text expects them to start under nelle/ and expects nelle to be under users/ . So I could add the programming lessons' csv files to a directory that's a sibling to filesystem/ within the same archive, and it wouldn't impact the shell lessons at all.

That said, I do agree it is confusing to have the files for one section stored in another section!

I'm not sure what the best answer is but I think possibly the answer depends on how we as instructors want to use the files. At the moment I have "download this zip file" instructions at the start of the first unix shell lesson. If we were going to add similar instructions to the start of the first Python and Matlab lessons, I don't think there'd be much drawback to separate archives stored in their logical respective locations.

But, are the "download this file" instructions at the start of the first unix shell lesson, as currently in the PR, a good way to go? Are people happy with the addition described at claresloggett@5f0b0e5 ? I wasn't sure this is the best way, definitely open to suggestions here.

[jdblischak]

But, are the "download this file" instructions at the start of the first unix shell lesson, as currently in the PR, a good way to go? Are people happy with the addition described at claresloggett/bc@5f0b0e5 ? I wasn't sure this is the best way, definitely open to suggestions here.

I am +1. I merged these changes into my bootcamp repo that we are teaching tomorrow.

I did find one issue with this on Windows. You can't just double-click on the zip file to unzip it (it just let's you view what is inside the zipped file). When I right click and choose "Extract All...", it creates another level. In other words, in order to get to users/nelle, they would have to run cd filesystem/filesystem/users/nelle. In order to get the same behavior as the unzip command in Unix, Windows users would need to modify the suggested path in the pop window to remove filesystem. For example, in my case the pop-up window suggests unzipping the contents in C:\Users\John\filesystem, so I would need to change it to just C:\Users\John.

So one solution would be to add this explanation. Another option would be to just direct everyone to use unzip (Windows users will have installed Git Bash in order to do the lesson).

[chendaniely]

I am +1 as other instructors who taught this summer have mentioned a quick way to have a unified file system is needed.

I think some proposed have a bash script that does everything, but the zipfile does seem like a more familiar way to get things up and ready.

@jdblischak the other way on windows is to double click the .zip, and drag the folder into the correct directory.

[wking]

On Wed, Sep 17, 2014 at 08:57:19AM -0700, John Blischak wrote:

I merged these changes into my bootcamp repo that we are teaching
tomorrow.

With the tarball in your master? Since bc is the development
repository, I'd at least prefer this to land in gh-pages, to avoid
even more auto-generated content in master than we already have (as
I've pointed out before, links available on request ;). The
Makefile-based approach would help here, if we don't want to go the
separate-Git-branch route.

[rgaiacs]

About the separate-Git-branch route, why not create another repository (e.g.
swcarpentry/filesystem)? Will be easy (and fast) to students clone it and
"easy" to keep tracking of it with submodule (I know that submodule isn't
perfect but maybe will help new contributors to our lessons).

[wking]

On Wed, Sep 17, 2014 at 10:25:30AM -0700, r-gaia-cs wrote:

About the separate-Git-branch route, why not create another repository (e.g.
swcarpentry/filesystem)?

Like the one's I've already got [1](and mentioned before [2])? ;)

Will be easy (and fast) to students clone it and "easy" to keep
tracking of it with submodule (I know that submodule isn't perfect
but maybe will help new contributors to our lessons).

No need for submodules; you can use pull if you like [3,4].

[jdblischak]

With the tarball in your master? Since bc is the development repository, I'd at least prefer this to land in gh-pages, to avoid even more auto-generated content in master than we already have

It's my repo for teaching a bootcamp. gh-pages is the only branch.

the other way on windows is to double click the .zip, and drag the folder into the correct directory.

OK. I didn't know about that. So that is a third option.

[wking]

On Wed, Sep 17, 2014 at 11:43:57AM -0700, John Blischak wrote:

With the tarball in your master? Since bc is the development
repository, I'd at least prefer this to land in gh-pages, to avoid
even more auto-generated content in master than we already have

It's my repo for [teaching][] a bootcamp. gh-pages is the only
branch.

That makes sense to me. I just get jumpy when we get a run of +1s
that someone will think “if it's ok for a per-workshop branch, it's ok
for bc's master” and hit the merge button without considering the
differences (bc would need newer tarballs as the material develops,
while a per-workshop repo is one-and-done).

[cboettig]

I think @wking 's suggestion of one repo per lesson has a lot to recommend it. Certainly this would add overhead in coordinating the different parts, but it's exactly the lack of any explicit structure in the current system that makes it more difficult particularly for individuals not deeply familiar with the whole SWC curriculum to either reuse or contribute to lessons (which are really two sides of the same coin).

I see that this is discussed in #102 in a long discussion regarding submodules and other git complexity. It seems to me that the task of assembling content from multiple repositories into a single website need not be so complicated and can be automated; so I'm referring more to the kind of basic metadata of knowing what material (files etc) are needed for a lesson and where to obtain them, as well as what prior knowledge is assumed, software installed, etc. By being more explicit about all of these things it would be much easier for someone to reuse or contribute to a particular lesson in a more modular fashion. We teach functional codes and declaring dependencies, but sometimes the teaching material feels like one long script ;-)

[wking]

On Tue, Sep 23, 2014 at 07:46:52PM -0700, Carl Boettiger wrote:

I see that this is discussed in #102 in a long discussion regarding
submodules and other git complexity.

There are some alternative assembly approaches in there too such as using
pulls 1 or Bower 2.

[gdevenyi]

The big question right now for this PR is before or after #759

I suggest we wait until after, and we can maintain the file structure in a separate branch which can be directly downloaded from github. The only challenge for this will be that I think we will need two PR's for each change that touches the lesson material and the filesystem.

[wking]

On Mon, Sep 29, 2014 at 02:48:15PM -0700, Gabriel A. Devenyi wrote:

I suggest we wait until after, and we can maintain the file
structure in a separate branch which can be directly downloaded from
github. The only challenge for this will be that I think we will
need two PR's for each change that touches the lesson material and
the filesystem.

I think you'll just need a PR against the filesystem branch.
Whoever's maintaining the lesson can merge the filesystem branch after
any changes land:

$ git checkout filesystem
$ git merge origin/pr/123
$ git checkout master
$ git merge filesystem
$ git push origin master filesystem

[claresloggett]

Thanks @gdevenyi . I didn't know about #759 ! I think the zip file idea is less important if the repo is to be broken up into lesson-sized pieces. As @wking points out, the github archive for each small repo is already a zip file which will then be of manageable size, and it needs no maintenance.

It still might be worthwhile to add instructions to the start of certain lessons to tell students to clone (or download and unzip) the relevant filesystem or data?

[wking]

On Sun, Oct 12, 2014 at 12:40:26AM -0700, claresloggett wrote:

It still might be worthwhile to add instructions to the start of
certain lessons to tell students to clone (or download and unzip)
the relevant filesystem or data?

This makes sense to me, although you'd probably want to set the
user/repository in the Jekyll metadata and use that to build the
link(s), so it's easy for folks to adjust them when they fork
per-workshop copies.