Skip to content
This repository has been archived by the owner on Jan 3, 2018. It is now read-only.

swc-windows-installer.py: Install nano source syntax highlighters #357

Merged

Conversation

wking
Copy link
Contributor

@wking wking commented Mar 5, 2014

As suggested in #354. Currently untested on Windows ;).

Make writing Python source code (and a number of other languages) a bit more
exciting for the eyes and easier on the brain by coloring strings, comments,
and such.

We need to install the source tarball because the compiled zip doesn't include
the syntax highlighting scripts. To make that easier, I've extracted out the
download / hash-check functionality into a new download(), which I use in both
the old zip_install() and the new tar_install(). If the user doesn't have an
existing ~/.nanorc, we populate it by adding 'include' options for each of the
syntax files in the Nano tarball. If they do have an existing ~/.nanorc, I
assume they know what they're doing ;).

The default '*' compression is for transparent detection 1, but you can
override it to explicitly select the compression type if you like.

There's a bit of ugliness to work around the lack of a
--strip-components analog in TarFile.extractall 2. Instead, we iterate over
the TarInfo members and strip leading components from their 'name' by hand
3.

wking added 5 commits October 23, 2013 12:04
This is a no-op merge, explicitly considering and rejecting the
de-namespacing from the master branch.  After this merge, future
merges from the master branch will pick up any content changes.

  $ git merge --log --no-ff --strategy ours master

* master:
  swc-windows-installer.py: Remove nesting setup/
* master:
  swc-windows-installer.py: Restore nose entry point
* master:
  swc-windows-installer.py: Use regular expressions to POSIX-ify paths
  swc-windows-installer.py: Only create the python-scripts dir if it doesn't exist
  swc-windows-installer.py: Split out zip_install into its own function
* python:
  swc-windows-installer.py: Quote $PATH export to account for spaces.
* python:
  swc-windows-installer.py: Install nano source syntax highlighters
@gvwilson
Copy link
Contributor

@ethanwhite can you please test on Windows?

@ethanwhite
Copy link
Contributor

I did that last week, but it looks like I reported the result in the wrong place (#228 (comment)). Oops.

The installation runs smoothly on Windows 7, but I don't see any syntax highlighting on a Python file. What can I do to help you debug?

@wking
Copy link
Contributor Author

wking commented Mar 12, 2014

On Wed, Mar 12, 2014 at 08:28:21AM -0700, Ethan White wrote:

The installation runs smoothly on Windows 7, but I don't see any
syntax highlighting on a Python file. What can I do to help you
debug?

What's in your ~/.nanorc? It should be a series of 'include'
statements pointing at nano's doc/syntax/*.nanorc files. However, if
you already had a ~/.nanorc, the script won't change anything (and
you'll have to add in includes yourself).

The other possibility is that nano is looking somewhere other than
~/.nanorc for it's default config on MS Windows, or that our include
paths are somehow garbled.

@wking
Copy link
Contributor Author

wking commented Mar 12, 2014

On Wed, Mar 12, 2014 at 08:28:21AM -0700, Ethan White wrote:

I did that last week, but it looks like I reported the result in the
wrong place
(#228 (comment)). Oops.

Ah, sorry I missed that in my inbox. That's the right place too ;),
since I've got both the Python and POSIX-shell versions of this script
going in parallel until we pick a favorite.

@ethanwhite
Copy link
Contributor

On Wed, Mar 12, 2014 at 12:18 PM, W. Trevor King [email protected] wrote:

What's in your ~/.nanorc? It should be a series of 'include'
statements pointing at nano's doc/syntax/*.nanorc files. However, if
you already had a ~/.nanorc, the script won't change anything (and
you'll have to add in includes yourself).

The other possibility is that nano is looking somewhere other than
~/.nanorc for it's default config on MS Windows, or that our include
paths are somehow garbled.

I didn't have one before, and yes it includes a bunch of includes. For example,

include /c/Users/River/.swc/share/nanorc/doc/syntax/python.nanorc

These files do exist in the listed directories.

@wking
Copy link
Contributor Author

wking commented Mar 12, 2014

On Wed, Mar 12, 2014 at 01:24:40PM -0700, Ethan White wrote:

On Wed, Mar 12, 2014 at 12:18 PM, W. Trevor King [email protected] wrote:

What's in your ~/.nanorc? It should be a series of 'include'
statements pointing at nano's doc/syntax/*.nanorc files. However, if
you already had a ~/.nanorc, the script won't change anything (and
you'll have to add in includes yourself).

The other possibility is that nano is looking somewhere other than
~/.nanorc for it's default config on MS Windows, or that our include
paths are somehow garbled.

I didn't have one before, and yes it includes a bunch of includes. For example,

include /c/Users/River/.swc/share/nanorc/doc/syntax/python.nanorc

These files do exist in the listed directories.

That looks reasonable. My next guess is that either nano is not
finding your ~/.nanorc, or it's formatted incorrectly (nano seems
picky about endlines 1). Can you use the Windows equivalent of
strace [2](I assume it exists) to figure out which files nano is
opening to see if it's actually checking ~/.nanorc?

@ethanwhite
Copy link
Contributor

On Wed, Mar 12, 2014 at 4:40 PM, W. Trevor King [email protected] wrote:

That looks reasonable. My next guess is that either nano is not
finding your ~/.nanorc, or it's formatted incorrectly (nano seems
picky about endlines 1). Can you use the Windows equivalent of
strace [2](I assume it exists) to figure out which files nano is
opening to see if it's actually checking ~/.nanorc?

Well, add strace to the list of things I've learned about thanks to Software Carpentry. Cool. The best Windows analogue (non-cygwin) appears to be Process Monitor 1, which reports that:

  1. nano attempts to create a file called nanorc at C:\Users\River\.swc\lib\usr\local\etc\nanorc and reports (obviously) PATH NOT FOUND.
  2. nano appears to be trying to access ~/nano.rc, which doesn't exist

If I rename .nanorc to nano.rc then starting nano throws errors like:

Error reading /c/Users/River/.swc/share/nanorc/doc/syntax/python.nanorc

And Process Manager reports failed attempts to access:

C:\Users\River\.swc\lib\c\Users\River\.swc\share\nanorc\doc\syntax\python.nanorc

So it looks like maybe we want to rename to nano.rc but are still having a path issue.

@ethanwhite
Copy link
Contributor

Got it. We just need to change the /c/Users/River/ in the include statements to ~/ and everything works great!

@wking
Copy link
Contributor Author

wking commented Mar 12, 2014

On Wed, Mar 12, 2014 at 04:44:31PM -0700, Ethan White wrote:

Got it. We just need to change the /c/Users/River/ in the include
statements to ~/ and everything works great!

Thanks for figuring that out :). I'll work up fixes to the scripts
and post back.

* python:
  swc-windows-installer.py: Use '~' in ~/nano.rc includes
  swc-windows-installer.py: Rename ~/.nanorc to ~/nano.rc
wking added a commit to wking/swc-boot-camps-v2 that referenced this pull request Mar 13, 2014
The nano FAQ hints that this is the prefered path on MS Windows [1],
and Ethan White confirmed it in testing [2].

[1]: http://www.nano-editor.org/dist/v2.2/faq.html#3.9.1
[2]: swcarpentry#357 (comment)
wking added a commit to wking/swc-boot-camps-v2 that referenced this pull request Mar 13, 2014
On Wed, Mar 12, 2014 at 04:44:31PM -0700, Ethan White wrote [1]:
> Got it. We just need to change the `/c/Users/River/` in the include
> statements to `~/` and everything works great!

[1]: swcarpentry#357 (comment)
wking added a commit to wking/swc-boot-camps-v2 that referenced this pull request Mar 13, 2014
The nano FAQ hints that this is the prefered path on MS Windows [1],
and Ethan White confirmed it in testing [2].

[1]: http://www.nano-editor.org/dist/v2.2/faq.html#3.9.1
[2]: swcarpentry#357 (comment)
wking added a commit to wking/swc-boot-camps-v2 that referenced this pull request Mar 13, 2014
On Wed, Mar 12, 2014 at 04:44:31PM -0700, Ethan White wrote [1]:
> Got it. We just need to change the `/c/Users/River/` in the include
> statements to `~/` and everything works great!

I also replaced an:

  if [ ... ]

condition with:

  if test ...

to match the rest of the script.

The underscore prefixes in the string-equality test are to avoid miss
interpretation in the pathological case that RCPATH starts with '!' or
some such.  It's hard to imagine how this could happen with the
current script, but... safety first ;).

[1]: swcarpentry#357 (comment)
@wking
Copy link
Contributor Author

wking commented Mar 13, 2014

On Wed, Mar 12, 2014 at 04:45:56PM -0700, W. Trevor King wrote:

On Wed, Mar 12, 2014 at 04:44:31PM -0700, Ethan White wrote:

Got it. We just need to change the /c/Users/River/ in the include
statements to ~/ and everything works great!

Thanks for figuring that out :). I'll work up fixes to the scripts
and post back.

Updated script posted with fd2075f (Merge branch 'python' into
bc-namespaced-python, 2013-03-12). Remember to remove your ~/nano.rc
(and optionally ~/.swc) before running the script, or it won't change
anything ;).

@ethanwhite
Copy link
Contributor

The bash script in #228 is now working properly, but the Python script is still failing. It reports errors like:

Error reading ~\.swc\share\nanorc\doc\syntax\python.nanorc

The obvious difference is that the nano.rc files produced by the bash script have / for paths for the includes, but the Python script produces includes that use \.

* python:
  swc-windows-installer.py: Use POSIX paths for ~/nano.rc
wking added a commit to wking/swc-boot-camps-v2 that referenced this pull request Mar 13, 2014
On Wed, Mar 12, 2014 at 06:34:43PM -0700, Ethan White wrote [1]:
> The bash script in swcarpentry#228 is now working properly, but the Python
> script is still failing. It reports errors like:
>
>     Error reading ~\.swc\share\nanorc\doc\syntax\python.nanorc
>
> The obvious difference is that the nano.rc files produced by the
> bash script have `/` for paths for the includes, but the Python
> script produces includes that use `\`.

[1]: swcarpentry#357 (comment)
@wking
Copy link
Contributor Author

wking commented Mar 13, 2014

On Wed, Mar 12, 2014 at 06:34:43PM -0700, Ethan White wrote:

The obvious difference is that the nano.rc files produced by the
bash script have / for paths for the includes, but the Python
script produces includes that use \.

Should be fixed now with ee9a596 (swc-windows-installer.py: Use POSIX
paths for ~/nano.rc, 2014-03-12). Thanks for testing :).

@ethanwhite
Copy link
Contributor

Almost there. We now have a line ending issue resulting in:

Error reading /cygdrive/c/Users/...

This is fixed by running dos2unix on the resulting nano.rc.

* python:
  swc-windows-installer.py: Force UNIX-style \n line endings for ~/nano.rc
@wking
Copy link
Contributor Author

wking commented Mar 13, 2014

On Thu, Mar 13, 2014 at 03:37:12AM -0700, Ethan White wrote:

Almost there. We now have a line ending issue resulting in:

Error reading /cygdrive/c/Users/...

This is fixed by running dos2unix on the resulting nano.rc.

Hmm, the presence of '/cygdrive/' hints that you are running this from
Cygwin, and not the msysGit Bash shell that you're using for nano?
That may have been the source of the absolute-import-path problem,
which we worked around by using ~/ prefixes. With ~/, the same
nano.rc should work for both Cygwin-based shells and msysGit-based
shells, but we might want to test swc-windows-installer.py in both
environments to confirm that.

Anyhow, I've tried to force UNIX-style newlines (\n) with 21a58ad
(swc-windows-installer.py: Force UNIX-style \n line endings for
~/nano.rc, 2014-03-13). Can you test this with both Python 2 and 3?

@ethanwhite
Copy link
Contributor

On Thu, Mar 13, 2014 at 11:46 AM, W. Trevor King [email protected] wrote:

Hmm, the presence of '/cygdrive/' hints that you are running this from
Cygwin, and not the msysGit Bash shell that you're using for nano?
That may have been the source of the absolute-import-path problem,
which we worked around by using ~/ prefixes.

Nope. All the testing was in msysGit. Have a little faith :) I think msysGit
uses a cygwin dll somewhere which might be triggering this.

With ~/, the same
nano.rc should work for both Cygwin-based shells and msysGit-based
shells, but we might want to test swc-windows-installer.py in both
environments to confirm that.

I'm preferential to just supporting this for msysGit since that's our official
tool at the moment.

Anyhow, I've tried to force UNIX-style newlines (\n) with 21a58ad
(swc-windows-installer.py: Force UNIX-style \n line endings for
~/nano.rc, 2014-03-13). Can you test this with both Python 2 and 3?

I'll try to get to this tonight when I'm back near a Windows machine.

@wking
Copy link
Contributor Author

wking commented Mar 13, 2014

On Thu, Mar 13, 2014 at 08:57:33AM -0700, Ethan White wrote:

On Thu, Mar 13, 2014 at 11:46 AM, W. Trevor King wrote:

Hmm, the presence of '/cygdrive/' hints that you are running this
from Cygwin, and not the msysGit Bash shell that you're using for
nano? That may have been the source of the absolute-import-path
problem, which we worked around by using ~/ prefixes.

Nope. All the testing was in msysGit. Have a little faith :) I think
msysGit uses a cygwin dll somewhere which might be triggering this.

Ah good, that halves the number of cases we need to test :). Ignoring
Cygwin works for me.

@ethanwhite
Copy link
Contributor

Anyhow, I've tried to force UNIX-style newlines (\n) with 21a58ad (swc-windows-installer.py: Force UNIX-style \n line endings for ~/nano.rc, 2014-03-13). Can you test this with both Python 2 and 3?

The link to the commit is broken and I don't seem to be able to track it down quickly. Can you point me in the right direction?

wking added a commit to wking/swc-boot-camps-v2 that referenced this pull request Mar 13, 2014
On Thu, Mar 13, 2014 at 03:37:12AM -0700, Ethan White wrote [1]:
> We now have a line ending issue resulting in:
>
>     Error reading /cygdrive/c/Users/...
>
> This is fixed by running `dos2unix` on the resulting nano.rc.

This is also mentioned in the nano FAQ [2]:

> Note that the nano.rc file must remain Unix formated in order for
> nano to unerstand it.

Since Python 3.0 and PEP 3116, Python's open() has taken a 'newline'
argument that allows you to override the default line separator for a
particular file [3,4].  However, Python 2.x does not support that
argument [5].  This commit adds a local 'open3' which uses a terrible
hack of a wrapper on Python 2 to fake that support.  It opens the file
in binary mode, and takes advantage of Python 2's lack of distinction
between str and bytes to treat the \n-terminated strings as the raw
binary output.  That's going to crash and burn if you have any
non-ASCII characters in the output content, but we explicitly set all
the components in our relative include_path, so that shouldn't be a
problem.

Another side effect of binary-mode is that we are going to get
ASCII-encoded output, not output encoded in the user's preferred
encoding (locale.getpreferredencoding [3,6]).  That's unfortunate,
because nano does the right thing and uses your locale to decode input
(at least for UTF-8 locales [7], presumably including your nano.rc
file).

[1]: swcarpentry#357 (comment)
[2]: http://www.nano-editor.org/dist/v2.2/faq.html#3.9.1
[3]: http://docs.python.org/3/library/functions.html#open
[4]: http://legacy.python.org/dev/peps/pep-3116/
[5]: http://docs.python.org/2/library/functions.html#open
[6]: http://docs.python.org/2/library/stdtypes.html#file.encoding
[7]: http://www.nano-editor.org/dist/v2.2/faq.html#5.3
@wking
Copy link
Contributor Author

wking commented Mar 13, 2014

On Thu, Mar 13, 2014 at 04:45:17PM -0700, Ethan White wrote:

The link to the commit is broken and I don't seem to be able to
track it down quickly. Can you point me in the write direction?

I just forgot to push :p. Should be fixed now.

@ethanwhite
Copy link
Contributor

It's working great with Python 2. Nice work! I don't have Python 3 on Windows handy, but I'll see if I can get it installed if I have a few spare minutes this weekend.

@ethanwhite
Copy link
Contributor

And... drum roll... works like a charm on Python 3 as well. Go ahead and merge these in whenever you're ready.

@gvwilson
Copy link
Contributor

Happy to merge, but want to check there's no overlap/conflict with #228.

@wking
Copy link
Contributor Author

wking commented Mar 19, 2014

On Mon, Mar 17, 2014 at 05:04:20AM -0700, Greg Wilson wrote:

Happy to merge, but want to check there's no overlap/conflict with #228.

There is definitely a conflict with #228 ;). I don't think
maintaining two installers (one in Python and one in POSIX shell)
makes sense in the long run. We should pick the one we like best and
ditch the other. I think the currently-known trade off is:

  • Pro-POSIX: Only need msysGit to run the installer (no Python required).
  • Pro-Python: No need to reboot before double-clicking works 1.

Maintaining both scripts in parallel forever is possible, but I'd
rather only handle one side of that myself.

* python:
  swc-windows-installer.py: Install the sqlite3 shell
  swc-windows-installer.py: Generalize msysGit download link
@wking
Copy link
Contributor Author

wking commented Apr 4, 2014

Updated with 1ed39ee (swc-windows-installer.py: Generalize msysGit
download link, 2014-03-27, my take on the ongoing #389 discussion) and
488c87c (swc-windows-installer.py: Install the sqlite3 shell,
2014-04-04, #413).

@ethanwhite
Copy link
Contributor

As per discussion in #424 this should be merged in when someone gets the chance. It will need to be done manually.

@wking wking merged commit 6217603 into swcarpentry:master Apr 19, 2014
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants