import history of setup/ from bc repo #61

pipitone · 2014-11-17T23:58:08Z

As discussed in #1, here is the history of the setup/ folder from swcarpentry/bc. I think it will be easier to merge the history of index.html in a separate PR, unless someone can tell me otherwise.

For reference, I loosely followed this and this to extract the history from the bc repo and prepare it for import. Specifically:

git clone https://github.com/swcarpentry/bc
cd bc

# split the history of setup/ into the setup-history branch
git subtree split -P setup -b setup-history
git checkout setup-history

# all of the files from setup/ are now in the root, so we need to move them into setup/
# before import
git filter-branch --tree-filter 'mkdir -p setup; mv * setup || true;' HEAD

# now pull into workshop-template
cd ..
git clone [email protected]:pipitone/workshop-template
cd workshop-template
git checkout -b pr-import-bc-setup-history
git pull ../bc setup

The content of this script was posted by Greg Wilson and attributed to Konrad Hinsen [1]. [1]: swcarpentry/website-archive#38 (comment)

This script was linked to by Eric Bray [1] and hosted on Fernando Perez's website [2]. [1]: swcarpentry/website-archive#38 (comment) [2]: http://fperez.org/py4science/workshop_checklist.py

Based on a list of possibilities mentioned by Cait Pickens in an internal email.

Often we don't care which editor is installed, so long as at least one is installed. This commit tweaks Dependency and adds a VirtualDependency class to support such virtual dependencies. You can't get very fancy with boolean logic, but an all-and and all-or lists will probably be sufficient for our needs.

Use distutils' new_compiler() to get the appropriate exe_extension for the user's system. This way command names can always be specified in their bare form, and we'll automatically add the right executable extension for other platforms.

Avoid running the same check (e.g. 'python') over and over.

For example, if you have an outdated 'python', you only want to hear about that once, not once for each PythonPackageDependency.

This makes it easy to see what a user has installed.

Exception chaining (PEP 3134, 'raise ... from') raises SyntaxErros in Python 2.x. Comment the chaining out until SWC starts teaching only Python 3.x.

Otherwise `print()` actually prints `()`, where we want it to print a blank line.

importlib is new in Python 2.7 / 3.1. Add a minimal workaround for earlier versions.

…sion Split it into pieces for easier subclassing.

The Mercurial Python package stores its version in strange places.

The 'sqlite3' package has the same version as the standard library. We only check for it because sometimes (e.g. on Gentoo) Python may be compiled without this package.

I have access to a machine running OS X 10.6.8, which has: $ bash --version GNU bash, version 3.2.48(1)-release (x86_64-apple-darwin10.0) Copyright (C) 2007 Free Software Foundation, Inc. That means no associative arrays, but for SWC purposes, it should still get the job done. All we really need is POSIX.2.

Commit to POSIX.2 [1] compatibility by allowing additional shells. If anyone has something besides sh, bash, or dash installed, they probably know how to use it ;). [1]: http://pubs.opengroup.org/onlinepubs/009695399/utilities/xcu_chap02.html IEEE Std 1003.2-1992

Distribute's easy_install supports --version (at least since 0.6.21), but Setuptool's original version does not (at least as of 0.6c9). Assume that if we get some kind of reasonable output from `easy_install --version` we're dealing with the Setuptools version.

Xcode 4.0.2 for OS X 10.6.8 shipped with Git 1.7.3.4.

Dodge the logic which (at least in Python 2.6.1) raises: DeprecationWarning: BaseException.message has been deprecated as of Python 2.6 We're setting the .message attribute explicitly, so the deprecation warning does not apply to us. The fix was suggested by Brett Cannon: On Sun Jul 8 02:42:18 CEST 2007, Brett Cannon wrote [1]: > You can get around this easily enough with a subclass that uses a > property for message:: > > class gerror(Exception): > def _get_message(self, message): return self._message > def _set_message(self, message): self._message = message > message = property(_get_message, _set_message) [1]: http://mail.python.org/pipermail/python-dev/2007-July/073777.html

We already told them how to do this in swc-installation-test-1.py, hopefully they remember ;).

This can link to the "howto setup Python" notes from SWC once those get a more precise link than: http://software-carpentry.org/setup/

I can't give package-specific links yet, but they will hopefully be in place soon. I made the "email your instructor" suggestion optional, because it is easy for me to imagine a course large enough that the instructor could not field all requests. To disable, change print_suggestions(instructor_fallback=True) to print_suggestions(instructor_fallback=False)

No need to test the other dependencies once you find one that matches.

Placing the often-symlinked `sh` at the end of the list. This way if a user has Bash installed, it doesn't show up as plain old Bourne.

… found'

Many people have spaces in their Windows user names, leading to trouble with an unquoted $PATH export. This file now quotes the $PATH addition accordingly.

…dows-installer into windows-installer-quote-path * 'bc-namespaced-python' of git://tremily.us/swc-setup-windows-installer: swc-windows-installer.py: Quote $PATH export to account for spaces.

Make writing Python source code (and a number of other languages) a bit more exciting for the eyes and easier on the brain by coloring strings, comments, and such. We need to install the source tarball because the compiled zip doesn't include the syntax highlighting scripts. To make that easier, I've extracted out the download / hash-check functionality into a new download(), which I use in both the old zip_install() and the new tar_install(). If the user doesn't have an existing ~/.nanorc, we populate it by adding 'include' options for each of the syntax files in the Nano tarball. If they do have an existing ~/.nanorc, I assume they know what they're doing ;). The default '*' compression is for transparent detection [1], but you can override it to explicitly select the compression type if you like. There's a bit of ugliness to work around the lack of a --strip-components analog in TarFile.extractall [2]. Instead, we iterate over the TarInfo members and strip leading components from their 'name' by hand [3]. [1]: http://docs.python.org/3/library/tarfile.html#tarfile.open [2]: http://docs.python.org/3/library/tarfile.html#tarfile.TarFile.extractall [3]: http://docs.python.org/3/library/tarfile.html#tarfile.TarInfo.name

The nano FAQ hints that this is the prefered path on MS Windows [1], and Ethan White confirmed it in testing [2]. [1]: http://www.nano-editor.org/dist/v2.2/faq.html#3.9.1 [2]: swcarpentry/DEPRECATED-bc#357 (comment)

On Wed, Mar 12, 2014 at 04:44:31PM -0700, Ethan White wrote [1]: > Got it. We just need to change the `/c/Users/River/` in the include > statements to `~/` and everything works great! [1]: swcarpentry/DEPRECATED-bc#357 (comment)

On Wed, Mar 12, 2014 at 06:34:43PM -0700, Ethan White wrote [1]: > The bash script in #228 is now working properly, but the Python > script is still failing. It reports errors like: > > Error reading ~\.swc\share\nanorc\doc\syntax\python.nanorc > > The obvious difference is that the nano.rc files produced by the > bash script have `/` for paths for the includes, but the Python > script produces includes that use `\`. [1]: swcarpentry/DEPRECATED-bc#357 (comment)

On Thu, Mar 13, 2014 at 03:37:12AM -0700, Ethan White wrote [1]: > We now have a line ending issue resulting in: > > Error reading /cygdrive/c/Users/... > > This is fixed by running `dos2unix` on the resulting nano.rc. This is also mentioned in the nano FAQ [2]: > Note that the nano.rc file must remain Unix formated in order for > nano to unerstand it. Since Python 3.0 and PEP 3116, Python's open() has taken a 'newline' argument that allows you to override the default line separator for a particular file [3,4]. However, Python 2.x does not support that argument [5]. This commit adds a local 'open3' which uses a terrible hack of a wrapper on Python 2 to fake that support. It opens the file in binary mode, and takes advantage of Python 2's lack of distinction between str and bytes to treat the \n-terminated strings as the raw binary output. That's going to crash and burn if you have any non-ASCII characters in the output content, but we explicitly set all the components in our relative include_path, so that shouldn't be a problem. Another side effect of binary-mode is that we are going to get ASCII-encoded output, not output encoded in the user's preferred encoding (locale.getpreferredencoding [3,6]). That's unfortunate, because nano does the right thing and uses your locale to decode input (at least for UTF-8 locales [7], presumably including your nano.rc file). [1]: swcarpentry/DEPRECATED-bc#357 (comment) [2]: http://www.nano-editor.org/dist/v2.2/faq.html#3.9.1 [3]: http://docs.python.org/3/library/functions.html#open [4]: http://legacy.python.org/dev/peps/pep-3116/ [5]: http://docs.python.org/2/library/functions.html#open [6]: http://docs.python.org/2/library/stdtypes.html#file.encoding [7]: http://www.nano-editor.org/dist/v2.2/faq.html#5.3

msysgit != Git for Windows

SWC's current setup instructions use both "Git for Windows" and "Git Bash" [1]. They link to the Git for Windows installer [2] and not the msysGit installer [3]. Since SWC has Make lessons [4], I think it makes more sense to have students installing msysGit (which has Make [5]) than to have them install Git for Windows (which does not have Make [6]). However, none of this matters for *this* Windows installer. Because this script works with both the full msysGit and the minimal Git for Windows, I'm sticking to msysGit (the more generic project name), and generalizing the suggested link to http://code.google.com/p/msysgit/downloads/list. [1]: https://github.com/swcarpentry/bc/blob/5c26313b0525e9d0b911d9558592a9e33cd03c1e/_includes/setup.html#L86 [2]: http://code.google.com/p/msysgit/downloads/list?q=full+installer+official+git [3]: http://code.google.com/p/msysgit/downloads/list?q=installer+hack+on+git [4]: https://github.com/swcarpentry/bc/tree/master/intermediate/make [5]: https://github.com/msysgit/msysgit#the-build-environment [6]: https://github.com/msysgit/msysgit#the-relationship-between-msysgit-and-git-for-windows

On Thu, Apr 03, 2014 at 09:38:38AM -0700, Ethan White wrote: > Some instructors like to teach `sqlite` from the command > line. Therefore we should have the Windows installer handle > downloading, installing and adding sqlite3 to the path. The novice/sql lessons currently revolve around some IPython magic, but even they use the SQLite shell to do the initial database setup [1]. The SQLite shell has also seen a bit more developer time than our one-off magic [2], so it's probably more robust ;). I've installed this in it's own directory (rather than just dumping it into ~/.swc/bin), because zip_install uses the existence of ${INSTALL_DIRECTORY} to decide whether or not the installation has already happened. create_nosetests_entry_point will have already created ${BIN_DIR}, so we need to use a different target directory for SQLite. We could also flip the order and run install_sqlite first, but that doesn't scale well if we install more packages in the future, and it's easier to keep the installs separate and keep adding to the path. [1]: https://github.com/swcarpentry/bc/blob/1aa0799ac6a48e9815d50bf2f49171b9eb5bf6d4/novice/sql/README.txt#L8 [2]: https://github.com/swcarpentry/bc/blob/1aa0799ac6a48e9815d50bf2f49171b9eb5bf6d4/novice/sql/sqlitemagic.py

…dows-installer * 'bc-namespaced-python' of git://tremily.us/swc-setup-windows-installer: swc-windows-installer.py: Install the sqlite3 shell swc-windows-installer.py: Generalize msysGit download link swc-windows-installer.py: Force UNIX-style \n line endings for ~/nano.rc swc-windows-installer.py: Use POSIX paths for ~/nano.rc swc-windows-installer.py: Use '~' in ~/nano.rc includes swc-windows-installer.py: Rename ~/.nanorc to ~/nano.rc swc-windows-installer.py: Install nano source syntax highlighters Conflicts: setup/swc-windows-installer.py This resolves a conflict in the Windows installer comments between my branch and changes from #389 [1]. Joon felt like we should match our suggested bootcamp installation instructions (which recommend the limited Git for Windows), but I prefer the more generic msysGit (as I explain in 4fe28c0e, swc-windows-installer.py: Generalize msysGit download link, 2014-03-27). Ethan feels like it isn't a big deal either way, so I'm going with my version ;). [1]: swcarpentry/DEPRECATED-bc#389

Following Amy Brown's [1]: > We borrowed the "bootcamp" terminology from Hacker Within and also > various fitness training camps, but as we branch out into parts of > the world with a more violent recent history I'm less comfortable > with the term. The consensus on that issue was to change back to our old 'workshop' terminology. The swcarpentry/bc repository hasn't made the shift yet, but there's no reason I can't embrace the future here ;). [1]: swcarpentry/DEPRECATED-bc#240

…on-test into installation-test-bc-namespaced * 'bc-namespaced' of git://tremily.us/swc-setup-installation-test: README.md: Convert 'boot camp' -> 'workshop' Conflicts: setup/README.md The conflics were due to Ivan's 'boot camp' -> 'bootcamp' changes [1] which I hadn't pulled into my upstream installation-test branch. The end result (using 'workshop' to refer to our sessions) is the same, but my upstream history just skips the 'bootcamp' phase. [1]: swcarpentry/DEPRECATED-bc#174

This setup.py file allows building an executable version of the installer that doesn't require an external Python installation.

This script tells Inno Setup how to build the stand alone installer

The Windows installer opens a shell while running. This adds some text to that shell so that it is clear that something is happening.

setup/README.md: Convert 'boot camp' -> 'workshop'

wking · 2014-11-19T06:19:22Z

On Mon, Nov 17, 2014 at 03:58:09PM -0800, Jon Pipitone wrote:

As discussed in #1, here is the history of the setup/ folder from
swcarpentry/bc.

These are already separate repositories 1:

The Windows installer 2.
The installation test scripts 3.

I'm happy to push the installation-test script somewhere on GitHub 4
if folks would find that easier than pulling from and contributing to
my self-hosted repository (which is what we eventually did for the
Windows installer 5), but so far there hasn't been any interest in
that.

I think it will be easier to merge the history of index.html in a
separate PR, unless someone can tell me otherwise.

Sorry, I'd thought you were just focused on index.html and the setup
instructions, or I would have pointed that out earlier. In this case,
I'd recommend:

Folks who are interested in the Windows installer to just use 6 or
link to 7.
Folks who are interested in the installation-test scripts to just
pull the appropriately-namespaced branch from my repository:

$ git pull git://tremily.us/swc-setup-installation-test.git bc-namespaced

I don't think you need the history massaging from this PR.

pipitone · 2014-11-19T16:58:43Z

As discussed in #1, here is the history of the setup/ folder from
swcarpentry/bc.

These are already separate repositories [1]:
...
Sorry, I'd thought you were just focused on index.html and the setup
instructions, or I would have pointed that out earlier.

Ah. Well, this is why I was asking for clarification on what history
needed to be transfered. My mistake, I was confused by the "and the
setup instructions" part.

So, to be clear. There already exists an index.html file. Is the work
that I agreed to do in #1 only to copy the history of that file in bc
to this file in this repo?

wking · 2014-11-19T17:08:45Z

On Wed, Nov 19, 2014 at 08:58:44AM -0800, Jon Pipitone wrote:

So, to be clear. There already exists an index.html file. Is the
work that I agreed to do in #1 only to copy the history of that
file in bc to this file in this repo?

I haven't been following this repository closely, so you might want to
confirm this with @gvwilson, but I think it's just that file. The
setup instructions (‘

Setup

…’) are in there now,
but before swcarpentry/DEPRECATED-bc@1f871402 (1. Putting common install
instructions in a single include file _includes/setup.html…,
2014-02-17) they were distributed under _includes/setup/. The bulk of
the bc setup is currently in _includes/setup.html 1 with a small
lead-in in setup.md 2.

gvwilson · 2014-11-19T18:02:05Z

That's correct - they used to be under _includes/setup/*.html.

gvwilson · 2014-11-20T14:23:24Z

@pipitone @wking Does more work need to be done on this to pull in _includes/setup/* from history?

gvwilson · 2014-12-17T14:41:15Z

Replaced by #83.

Konrad Hinsen and others added 30 commits December 29, 2012 14:41

swc-installation-test.py: Add installation-testing script

c0484a3

The content of this script was posted by Greg Wilson and attributed to Konrad Hinsen [1]. [1]: swcarpentry/website-archive#38 (comment)

workshop_checklist.py: Add installation-testing script

51aafd3

This script was linked to by Eric Bray [1] and hosted on Fernando Perez's website [2]. [1]: swcarpentry/website-archive#38 (comment) [2]: http://fperez.org/py4science/workshop_checklist.py

Merge Konrad Hinsen and Fernando Perez's installation-testing scripts

c3f2c3a

swc-installation-test: Consolidate and reorganize test scripts

c081808

swc-installation-test-2.py: sort CHECKS into topics

5318884

swc-installation-test-2.py: Add some editors and browsers

c17a5ec

Based on a list of possibilities mentioned by Cait Pickens in an internal email.

swc-installation-test-2.py: PythonDependencys should depend on Python

60e88c1

swc-installation-test-2.py: Cache check errors

5453afe

Avoid running the same check (e.g. 'python') over and over.

swc-installation-test-2.py: Don't print duplicate exceptions

f8008e7

For example, if you have an outdated 'python', you only want to hear about that once, not once for each PythonPackageDependency.

swc-installation-test-2.py: Print successfully-found versions

ab9d3af

This makes it easy to see what a user has installed.

swc-installation-test-2.py: Don't chain exceptions (yet)

ae70783

Exception chaining (PEP 3134, 'raise ... from') raises SyntaxErros in Python 2.x. Comment the chaining out until SWC starts teaching only Python 3.x.

swc-installation-test-2.py: import print_function for Python 2.6

a7b6d8b

Otherwise `print()` actually prints `()`, where we want it to print a blank line.

swc-installation-test-2.py: Don't parse missing versions

5df31aa

swc-installation-test-2.py: Add import_module for older Pythons

2314663

importlib is new in Python 2.7 / 3.1. Add a minimal workaround for earlier versions.

swc-installation-test-2.py: Refactor PythonPackageDependency._get_ver…

d9bf86f

…sion Split it into pieces for easier subclassing.

swc-installation-test-2.py: Fix 'mercurial' version extraction

4099f8b

The Mercurial Python package stores its version in strange places.

swc-installation-test-2.py: Fix version extraction for sqlite3-python

98c4d17

The 'sqlite3' package has the same version as the standard library. We only check for it because sometimes (e.g. on Gentoo) Python may be compiled without this package.

swc-installation-test-2.py: Only require Git >= 1.7 (not >= 1.8)

bcf4a24

Xcode 4.0.2 for OS X 10.6.8 shipped with Git 1.7.3.4.

swc-installation-test-2.py: Don't say how to get a terminal

89fbb1b

We already told them how to do this in swc-installation-test-1.py, hopefully they remember ;).

swc-installation-test-1.py: Give suggested install hints

067f325

This can link to the "howto setup Python" notes from SWC once those get a more precise link than: http://software-carpentry.org/setup/

swc-installation-test-2.py: Break after first successful or-dependency

f713ede

No need to test the other dependencies once you find one that matches.

swc-installation-test-2.py: Sort shells in order of preference

6ab7e25

Placing the often-symlinked `sh` at the end of the list. This way if a user has Bash installed, it doesn't show up as plain old Bourne.

swc-installation-test-1.py: Give instructions for 'python command not…

c0e144c

… found'

davis68 and others added 21 commits January 27, 2014 13:24

swc-windows-installer.py: Quote $PATH export to account for spaces.

d63439b

Many people have spaces in their Windows user names, leading to trouble with an unquoted $PATH export. This file now quotes the $PATH addition accordingly.

Merge branch 'bc-namespaced-python' of git://tremily.us/swc-setup-win…

c5fda54

…dows-installer into windows-installer-quote-path * 'bc-namespaced-python' of git://tremily.us/swc-setup-windows-installer: swc-windows-installer.py: Quote $PATH export to account for spaces.

swc-windows-installer.py: Rename ~/.nanorc to ~/nano.rc

828060a

The nano FAQ hints that this is the prefered path on MS Windows [1], and Ethan White confirmed it in testing [2]. [1]: http://www.nano-editor.org/dist/v2.2/faq.html#3.9.1 [2]: swcarpentry/DEPRECATED-bc#357 (comment)

swc-windows-installer.py: Use '~' in ~/nano.rc includes

7af68bb

On Wed, Mar 12, 2014 at 04:44:31PM -0700, Ethan White wrote [1]: > Got it. We just need to change the `/c/Users/River/` in the include > statements to `~/` and everything works great! [1]: swcarpentry/DEPRECATED-bc#357 (comment)

* replaced msysgit with Git for Windows

2ab7c79

msysgit != Git for Windows

Make separate directory for Windows installer

dbd1365

Add setup.py file for building Windows installer

f6cbe6a

This setup.py file allows building an executable version of the installer that doesn't require an external Python installation.

Add script for building Windows installer

9c947b8

This script tells Inno Setup how to build the stand alone installer

Add a Software Carpentry icon file for Windows installer

6dd223a

Add indication that the Windows installer is running

8bf14a8

The Windows installer opens a shell while running. This adds some text to that shell so that it is clear that something is happening.

Renaming Windows installer directory

1047381

Merge pull request #454 from wking/installation-test-bc-namespaced

56f5777

setup/README.md: Convert 'boot camp' -> 'workshop'

Merge history of setup/ folder from swcarpentry/bc

5148ac6

aaren mentioned this pull request Nov 18, 2014

transition to fit lesson-template aaren/swc-novice-python#1

Open

gvwilson mentioned this pull request Dec 16, 2014

Merging setup #83

Merged

gvwilson closed this Dec 17, 2014

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

import history of setup/ from bc repo #61

import history of setup/ from bc repo #61

pipitone commented Nov 17, 2014

wking commented Nov 19, 2014

pipitone commented Nov 19, 2014

wking commented Nov 19, 2014

gvwilson commented Nov 19, 2014

gvwilson commented Nov 20, 2014

gvwilson commented Dec 17, 2014

import history of setup/ from bc repo #61

import history of setup/ from bc repo #61

Conversation

pipitone commented Nov 17, 2014

wking commented Nov 19, 2014

pipitone commented Nov 19, 2014

wking commented Nov 19, 2014

Setup

gvwilson commented Nov 19, 2014

gvwilson commented Nov 20, 2014

gvwilson commented Dec 17, 2014