setup/swcarpentry-installation-test-2.py should test the tools we use most commonly #136
Comments
|
On Thu, Jan 15, 2015 at 10:57:14AM -0800, Azalee Bostroem wrote:
I don't think “standardly taught” is well defined ;). It's hard to It would be pretty easy for individual instructors to setup branches
Then before a new workshop, you'd:
|
|
@wking I agree there is no single testing script that works for every workshop (R vs python, git vs mercurial, are you teaching SQL...). However I think the installation testing script should reflect our core material and what is being taught in most workshops. mayavi is not in our core material (what lesson does that belong to?). Likewise - testing is not part of our required teaching material (and I don't think our core lessons on testing include using all of those packages). When mercurial is taught is it taught using hg or EasyMercurial (or do you need both)? My basic point is that while you can't write a general testing script tailored to each workshop there are some packages that are not used in most workshops which could be commented out so that even if a student uses the wrong version of the code they install fewer unnecessary packages. Also - I'm confused about your instructions. I think that if you follow the general workshop creation workflow you will get a personal copy of the testing script and directions to run it that you can point students to for an individual workshop. How is what you are proposing different/better? |
|
On Thu, Jan 15, 2015 at 12:37:58PM -0800, Azalee Bostroem wrote:
That's reasonable. Going back to your initial list, I'd recommend
That's right, but I maintain the script in a separate repository
It's not different/better (i.e. “instead of”), it's just an optional $ git pull https://github.com/you/swc-setup-installation-test.git bc-namespaced for each workshop is easier than trying to remember and apply your |
|
My take on this is that the most important uses of these tests will be for relative novices. Experienced programmers will generally be able to handle installing software. We're trying to avoid the issues that happen when a bunch of novices don't get things installed correctly. From that perspective all of the things that @abostroem listed are less important to include, with the possible exception of Mercurial. It seems to me that setting sensible defaults can accommodate both a more restricted set of default tests and being able to test lots of things. I'd propose that we comment out all of the things on @abostroem's list. The instructions can then indicate that if you want to add testing for less standard options that you can do so by uncommenting the relevant sections of the file. |
|
👍 |
|
On Thu, Mar 19, 2015 at 10:54:39AM -0700, Ethan White wrote:
Sure, but configuring the tests is up to the
The default is currently to check for everything that has at some virtual-shell That's a smaller set of tests than the current default, but I still virtual-shell because their's either R or Python, either Git or Mercurial, and Instead of trying to guess at a list of “standard” requirements, it |
|
I guess I'll just say that I disagree and that if it was up to me we'd test the most common workshop setup - shell, git, and Python - by default, and ask folks to modify for less common configurations. There are lots of things to do for any given workshop and minimizing the number of changes to be made by the average instructor is a good thing. With that, I'll leave this up to the maintainers. |
|
It's clear from @ethanwhite's description in #180 that the current procedure is not working, and that it's generating extra work for organizers and unnecessary scares for students. I don't think either of those outcomes are acceptable and we need to modify some part of this workflow to address that. Restricting the default tests to only the most commonly used items (as proposed by @ethanwhite and @abostroem) seems like an effective way to cut down on unnecessary reports of missing software. |
|
On Thu, Mar 19, 2015 at 02:29:39PM -0700, Ethan White wrote:
So: virtual-shell ?
Absolutely, but I'm concerned that “many instructors don't need to |
|
There should be some more explicit info/reminder for the instructors to tweak the CHECKS. Right now it's easy to miss the reference to checks (esp. for the new instructors) among lots of other instructions. We just had a case when at a workshop that uses R, one of the attendees following the instructions on the website (made from the template) tried to run the scripts and since he didn't have Python, got lots of errors. He (and one of the local helpers) then spend hours trying to fix that until they came to the conclusion that maybe something is wrong with the info on the website. I'd suggest that to avoid that, we do a simple redesign of the workshop template index.html and make a separate subsection "Install check" and add a note for instructors to comment it out/not include if the workshop doesn't use "xyz". |
|
We had the same problem (cc: @hdashnow) at an R workshop a few weeks ago in Melbourne. I'm in the process of creating a new workshop website now, and it doesn't look like swc-installation-test-2.py has any checks for R. Maybe we need an R specific script that checks for valid shell, git, sql, and R installations? |
I think having a parallel script written in, and focused on, R might be a good solution. |
|
On Wed, Jun 10, 2015 at 06:56:06AM -0700, Ethan White wrote:
The current Python scripts could drive R from the command line |
|
I see your point about duplication of effort @wking. My main concern about adding R workshop requirements to the current script would mean that students of R workshops would have to also install Python. Or do we think all students should download Python for all workshops irrespective of the language being taught? If that's the case, then we need to be adding Python to the list of R install instructions and explain why. |
|
Another thought, what if the script takes an optional argument like Or even Apologies if this has already been proposed elsewhere... |
|
On Wed, Jun 10, 2015 at 06:12:45PM -0700, Harriet Dashnow wrote:
Folks who are using non-Windows OSes already have a system Python.
Yes and no ;). They'll need Python to run the script, but if that a. Have instructors build their own installer/checker after editing |
|
I just forgot to change the requirements for a workshop I'm teaching next In any case, adapting the test script should be in the lead instructor list On Thu, Jun 11, 2015 at 10:39 AM, W. Trevor King [email protected]
|
|
On Thu, Jun 11, 2015 at 08:24:15AM -0700, Ivan Gonzalez wrote:
Sounds good to me. I wrote up a tool for creating checklist issues on |
On Wed, Mar 11, 2015 at 03:08:15PM -0700, Piotr Banaszkiewicz wrote [1]: > Maybe add workshop-template/requirements.txt (not necessarily a > Python dependencies file) that's read by installation testing script > and adjusts CHECKS entries accordingly? The old CHECKS approach didn't work very well, because many instructors would forget to edit CHECKS and learners would be confused when they failed checks for packages that they didn't actually need (but which were still listed in CHECKS) [2,3,4,5]. This commit makes: $ ./swc-installation-test-2.py a no-op. Users will either have to explicitly specify checks on the command line: $ ./swc-installation-test-2.py virtual-editor or point the tester at an instructor-provided config: $ ./swc-installation-test-2.py --url https://swcarpentry.github.io/2015-11-09-abc/lessons.json Because dependencies for a given lesson can be tricky and are shared by all lesson consumers, it offloads the bulk of the requirement-listing to lesson maintainers. Once that work is done, instructors can just list out the lessons they're covering (which is also useful for tools like AMY [6] who would like an automatic way to determine what is being taught). SWC generally prefers YAML to JSON, but I've gone with JSON here to stick with stock Python 2.6+ support. I've also changed some or_dependency definitions from tuples to lists so I can mutate them with .remove(). [1]: numfocus/gsoc#3 (comment) [2]: carpentries/workshop-template#136 [3]: carpentries/workshop-template#180 [4]: carpentries/workshop-template#181 [5]: carpentries/workshop-template#258 [6]: https://github.com/swcarpentry/amy
On Wed, Mar 11, 2015 at 03:08:15PM -0700, Piotr Banaszkiewicz wrote [1]: > Maybe add workshop-template/requirements.txt (not necessarily a > Python dependencies file) that's read by installation testing script > and adjusts CHECKS entries accordingly? The old CHECKS approach didn't work very well, because many instructors would forget to edit CHECKS and learners would be confused when they failed checks for packages that they didn't actually need (but which were still listed in CHECKS) [2,3,4,5]. This commit makes: $ ./swc-installation-test-2.py a no-op. Users will either have to explicitly specify checks on the command line: $ ./swc-installation-test-2.py virtual-editor or point the tester at an instructor-provided config: $ ./swc-installation-test-2.py --url https://swcarpentry.github.io/2015-11-09-abc/lessons.json Because dependencies for a given lesson can be tricky and are shared by all lesson consumers, it offloads the bulk of the requirement-listing to lesson maintainers. Once that work is done, instructors can just list out the lessons they're covering (which is also useful for tools like AMY [6] who would like an automatic way to determine what is being taught). SWC generally prefers YAML to JSON, but I've gone with JSON here to stick with stock Python 2.6+ support. I've also changed some or_dependency definitions from tuples to lists so I can mutate them with .remove(). [1]: numfocus/gsoc#3 (comment) [2]: carpentries/workshop-template#136 [3]: carpentries/workshop-template#180 [4]: carpentries/workshop-template#181 [5]: carpentries/workshop-template#258 [6]: https://github.com/swcarpentry/amy
On Wed, Mar 11, 2015 at 03:08:15PM -0700, Piotr Banaszkiewicz wrote [1]: > Maybe add workshop-template/requirements.txt (not necessarily a > Python dependencies file) that's read by installation testing script > and adjusts CHECKS entries accordingly? The old CHECKS approach didn't work very well, because many instructors would forget to edit CHECKS and learners would be confused when they failed checks for packages that they didn't actually need (but which were still listed in CHECKS) [2,3,4,5]. This commit makes: $ ./swc-installation-test-2.py a no-op. Users will either have to explicitly specify checks on the command line: $ ./swc-installation-test-2.py virtual-editor or point the tester at an instructor-provided config: $ ./swc-installation-test-2.py --url https://swcarpentry.github.io/2015-11-09-abc/lessons.json Because dependencies for a given lesson can be tricky and are shared by all lesson consumers, it offloads the bulk of the requirement-listing to lesson maintainers. Once that work is done, instructors can just list out the lessons they're covering (which is also useful for tools like AMY [6] who would like an automatic way to determine what is being taught). SWC generally prefers YAML to JSON, but I've gone with JSON here to stick with stock Python 2.6+ support. I've also changed some or_dependency definitions from tuples to lists so I can mutate them with .remove(). [1]: numfocus/gsoc#3 (comment) [2]: carpentries/workshop-template#136 [3]: carpentries/workshop-template#180 [4]: carpentries/workshop-template#181 [5]: carpentries/workshop-template#258 [6]: https://github.com/swcarpentry/amy
|
On Thu, Mar 19, 2015 at 02:44:40PM -0700, Matt Davis wrote:
So I finally got the time to implement @pbanaszkiewicz's suggestion |
On Wed, Mar 11, 2015 at 03:08:15PM -0700, Piotr Banaszkiewicz wrote [1]: > Maybe add workshop-template/requirements.txt (not necessarily a > Python dependencies file) that's read by installation testing script > and adjusts CHECKS entries accordingly? The old CHECKS approach didn't work very well, because many instructors would forget to edit CHECKS and learners would be confused when they failed checks for packages that they didn't actually need (but which were still listed in CHECKS) [2,3,4,5]. This commit makes: $ ./swc-installation-test-2.py a no-op. Users will either have to explicitly specify checks on the command line: $ ./swc-installation-test-2.py virtual-editor or point the tester at an instructor-provided config: $ ./swc-installation-test-2.py --url https://swcarpentry.github.io/2015-11-09-abc/lessons.json Because dependencies for a given lesson can be tricky and are shared by all lesson consumers, it offloads the bulk of the requirement-listing to lesson maintainers. Once that work is done, instructors can just list out the lessons they're covering (which is also useful for tools like AMY [6] who would like an automatic way to determine what is being taught). SWC generally prefers YAML to JSON, but I've gone with JSON here to stick with stock Python 2.6+ support. I've also changed some or_dependency definitions from tuples to lists so I can mutate them with .remove(). [1]: numfocus/gsoc#3 (comment) [2]: carpentries/workshop-template#136 [3]: carpentries/workshop-template#180 [4]: carpentries/workshop-template#181 [5]: carpentries/workshop-template#258 [6]: https://github.com/swcarpentry/amy
On Wed, Mar 11, 2015 at 03:08:15PM -0700, Piotr Banaszkiewicz wrote [1]: > Maybe add workshop-template/requirements.txt (not necessarily a > Python dependencies file) that's read by installation testing script > and adjusts CHECKS entries accordingly? The old CHECKS approach didn't work very well, because many instructors would forget to edit CHECKS and learners would be confused when they failed checks for packages that they didn't actually need (but which were still listed in CHECKS) [2,3,4,5]. This commit makes: $ ./swc-installation-test-2.py a no-op. Users will either have to explicitly specify checks on the command line: $ ./swc-installation-test-2.py virtual-editor or point the tester at an instructor-provided config: $ ./swc-installation-test-2.py --url https://swcarpentry.github.io/2015-11-09-abc/lessons.json Because dependencies for a given lesson can be tricky and are shared by all lesson consumers, it offloads the bulk of the requirement-listing to lesson maintainers. Once that work is done, instructors can just list out the lessons they're covering (which is also useful for tools like AMY [6] who would like an automatic way to determine what is being taught). SWC generally prefers YAML to JSON, but I've gone with JSON here to stick with stock Python 2.6+ support. I've also changed some or_dependency definitions from tuples to lists so I can mutate them with .remove(). The tuple cast when setting minimum_version from the JSON lets us avoid: TypeError: unorderable types: tuple() < list() when doing the version comparison. [1]: numfocus/gsoc#3 (comment) [2]: carpentries/workshop-template#136 [3]: carpentries/workshop-template#180 [4]: carpentries/workshop-template#181 [5]: carpentries/workshop-template#258 [6]: https://github.com/swcarpentry/amy
Sometimes instructors forget to customize their CHECKS, but learners still follow this link and run the stock checks, resulting in a bunch of confusing errors about missing packages that the learner doesn't actually need [1,2,3,4]. Instead of expecting instructors to edit CHECKS or remove the setup/ reference from index.html (a default that is right for nobody), this commit comments that setup/ reference out (a default that is right for folks who don't want installation-checks). Instructors who *do* want installation checks now need to do two things (uncomment the setup/ reference and update CHECKS), but with both steps listed in CUSTOMIZATION they are more likely to get that right. [1]: carpentries#136 [2]: carpentries#180 [3]: carpentries#181 [4]: carpentries#258
…tions Ivan brought this idea back up in maintainers@ recently [1]. I think the last time it was raised seriously was [2], which has reasonable links into earlier discussion. The main argument against tags was that it was too hard to find the source for a particular line you wanted to tweak [3]. This commit restores our old liquid templating to show/hide sections *without* splitting the sections out into sub-files (e.g. bc#738 had _includes/setup/linux-editor.html). If we keep everything in the index file, we can have tags and instructors can either adjust the tags or easily find/edit/delete as they see fit. A few notes on the implementation: * I've gone with double quotes in the YAML front matter for consistency with the other entries, but stuck with the original (from swcarpentry/bc) single quotes for the liquid conditionals. * I've kept "check" and "VM" out of the default tools list to match the current display, but we may want to enable everything and write a stronger message about removing stuff you don't need to avoid repeating the problems we had with check being visible by default [4]. Because folks will have to tweak the tools list if they want to enable the "check" or "VM" sections, I've added comments at the beginning of each section pointing instructors back up at the YAML front matter. * I've moved the "check" section out of the Python section, because while the tool doesn't currently test R packages, it does test Git, Bash, text editors, etc., and it could certainly be extended to test R if someone with R knowledge wanted to chip in (although it's harder to *run* the script on Windows without bundling Python) [5]. [1]: http://lists.software-carpentry.org/pipermail/maintainers_lists.software-carpentry.org/2016-March/000179.html Subject: Re: [Maintainers] Vote needed for setup instructions Date: Thu, 10 Mar 2016 12:05:33 -0500 Message-ID: <CAJpTZ0DPYKgu2fbi6dZ3XvZVyed0MwmS-jTR-e-Y3uYoZKoctQ@mail.gmail.com> [2]: swcarpentry/DEPRECATED-bc#738 [3]: swcarpentry/DEPRECATED-bc#729 (comment) [4]: carpentries#278 [5]: carpentries#136 (comment)
…tions Ivan brought this idea back up in maintainers@ recently [1]. I think the last time it was raised seriously was [2], which has reasonable links into earlier discussion. The main argument against tags was that it was too hard to find the source for a particular line you wanted to tweak [3]. This commit restores our old liquid templating to show/hide sections *without* splitting the sections out into sub-files (e.g. bc#738 had _includes/setup/linux-editor.html). If we keep everything in the index file, we can have tags and instructors can either adjust the tags or easily find/edit/delete as they see fit. A few notes on the implementation: * I've gone with double quotes in the YAML front matter for consistency with the other entries, but stuck with the original (from swcarpentry/bc) single quotes for the liquid conditionals. * I've kept "test" and "VM" out of the default tools list to match the current display, but we may want to enable everything and write a stronger message about removing stuff you don't need to avoid repeating the problems we had with the installation-test link being visible by default [4]. Because folks will have to tweak the tools list if they want to enable the "test" or "VM" sections, I've added comments at the beginning of each section pointing instructors back up at the YAML front matter. * I've moved the "test" section out of the Python section, because while the tool doesn't currently test R packages, it does test Git, Bash, text editors, etc., and it could certainly be extended to test R if someone with R knowledge wanted to chip in (although it's harder to *run* the script on Windows without bundling Python) [5]. The new header separates the test section from the previous section. [1]: http://lists.software-carpentry.org/pipermail/maintainers_lists.software-carpentry.org/2016-March/000179.html Subject: Re: [Maintainers] Vote needed for setup instructions Date: Thu, 10 Mar 2016 12:05:33 -0500 Message-ID: <CAJpTZ0DPYKgu2fbi6dZ3XvZVyed0MwmS-jTR-e-Y3uYoZKoctQ@mail.gmail.com> [2]: swcarpentry/DEPRECATED-bc#738 [3]: swcarpentry/DEPRECATED-bc#729 (comment) [4]: carpentries#278 [5]: carpentries#136 (comment)
|
Status? |
|
On Tue, Jul 26, 2016 at 03:50:54AM -0700, Greg Wilson wrote:
wking/swc-setup-installation-test#9 (mentioned in 1) works, but I a. someone has a good alternative idea, |
|
@abostroem and @wking Thanks to work on this one. Any progress? |
fmichonneau
added
type:enhancement
|
closing as stale. |
The current version of setup/swcarpentry-installation-test-2.py tests for everything we could possibly use including tools like Mayavi that are not standardly taught. It would be good to have only the packages that we teach standardly uncommented so that students who use this script rather than a customized one for the workshop. I would comment:
The text was updated successfully, but these errors were encountered: