Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for manpages via mandoc #1196

Closed
wants to merge 22 commits into from
Closed
Show file tree
Hide file tree
Changes from 14 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion .travis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,9 @@ notifications:
git:
depth: 10
before_install:
- sudo add-apt-repository ppa:jordemort/mandoc -y
- sudo apt-get update -qq
- sudo apt-get install perl
- sudo apt-get install mandoc perl xmlstarlet
- curl -L http://cpanmin.us | perl - --sudo App::cpanminus
- sudo cpanm --installdeps --notest Pod::Simple
- sudo pip install docutils
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ you wish to run the library. You can also run `script/bootstrap` to fetch them a
* [.asciidoc, .adoc, .asc](http://asciidoc.org/) -- `gem install asciidoctor` (http://asciidoctor.org)
* [.pod](http://search.cpan.org/dist/perl/pod/perlpod.pod) -- `Pod::Simple::XHTML`
comes with Perl >= 5.10. Lower versions should install Pod::Simple from CPAN.

* [.mdoc](http://mandoc.bsd.lv/man/mdoc.7.html), [.man](http://mandoc.bsd.lv/man/man.7.html) -- `apt-get` or `brew install mandoc xmlstarlet` (http://mandoc.bsd.lv/ http://xmlstar.sourceforge.net/)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like this suggested change has broken diff.


Installation
-----------
Expand Down
2 changes: 1 addition & 1 deletion lib/github-markup.rb
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
module GitHub
module Markup
VERSION = '3.0.1'
VERSION = '3.0.2'
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if this is appropriate to do here or if we want to roll out a release as a separate PR. Might also be worth more than a patch version; maybe 3.1.0? cc @kivikakk @lildude

Version = VERSION
end
end
16 changes: 16 additions & 0 deletions lib/github/commands/man2html
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
#!/bin/bash
set -e

# write the input to a tempfile first so we can look at it
TEMPFILE="$(mktemp)"
# shellcheck disable=SC2064
trap "rm -f $TEMPFILE" EXIT
cat "$@" > "$TEMPFILE"

# bail out if ths doesn't look like roff
grep -q '^[\.'\''][ \t]*[A-Za-z]\{2\}' "$TEMPFILE"

# process it, and print everything from the first <h1> to </body>
mandoc -Thtml -Ofragment "$TEMPFILE" \
| xmlstarlet tr --html --omit-decl "$(dirname "$0")/man2html.xslt" - \
| awk '/<h1/{flag=1}/<\/body>/{flag=0}flag'
93 changes: 93 additions & 0 deletions lib/github/commands/man2html.xslt
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
<?xml version="1.0" encoding="utf-8"?>
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
<!-- //
Based on the HTML output and default stylesheet of mandoc 1.14.4
YMMV with other versions of mandoc
// -->

<!-- Default: match anything and apply templates to it -->
<xsl:template match="node()|@*">
<xsl:copy>
<xsl:apply-templates select="node()|@*"/>
</xsl:copy>
</xsl:template>

<!-- Strip all the attributes off of these -->
<xsl:template match="h1">
<h1><xsl:apply-templates /></h1>
</xsl:template>

<!-- Turn these into paragraphs -->
<xsl:template match="div[@class='Pp']">
<p><xsl:apply-templates /></p>
Copy link

@ischwarze ischwarze Nov 11, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This replacement is WRONG, you can't do that. In mandoc HTML output, paragraphs often contain flow content, but HTML only permits phrasing content inside <p> elements. So what you are doing here is introducing HTML syntax errors.

Copy link

@Alhadis Alhadis Jan 25, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ischwarze You aren't wrong, but mistakes like these really, really, really common in GitHub's front-end code, which is yet another giant bowl of "tag-soup" for their designers to play in. Nobody's gonna notice this syntax violation when the entire site is a scrapbook of "do whatever, as long as it looks good". :(

</xsl:template>

<!-- Indent these -->
<xsl:template match="div[@class='Bd-indent' or @class='Bd Bd-indent']">
<ul><xsl:apply-templates /></ul>
</xsl:template>

<!-- Headers don't need to be linking to themselves -->
<xsl:template match="a[@class='permalink']">
<xsl:apply-templates />
</xsl:template>

<!-- Pull the manual-text div out -->
<xsl:template match="div[@class='manual-text']">
<xsl:apply-templates />
</xsl:template>

<!-- These were display: inline, so just give the content -->
<xsl:template match="div[@class='Nd' or @class='Bf' or @class='Op']">
<xsl:apply-templates />
</xsl:template>

<!-- These were display: italic -->
<xsl:template match="span[@class='Pa' or @class='Ad']">
<em><xsl:apply-templates /></em>
</xsl:template>

<!-- These were font-weight: bold -->
<xsl:template match="span[@class='Ms']">
<b><xsl:apply-templates /></b>
</xsl:template>

<xsl:template match="dl[@class='BL-diag']/dt">
<b><xsl:apply-templates /></b>
</xsl:template>

<xsl:template match="code[@class='Nm' or @class='Fl' or @class='Cm' or @class='Ic' or @class='In' or @class='Fd' or @class='Fn' or @class='Cd']">
<b><code><xsl:apply-templates /></code></b>
</xsl:template>

<!-- Remove header table -->
<xsl:template match="table[@class='head']"/>

<!-- Reformat footer table and pull some header stuff into it -->
<xsl:template match="table[@class='foot']">
<hr/>
<table>
<xsl:apply-templates select="//td[text() and text() != '()' and @class='head-ltitle']"/>
<xsl:apply-templates select="//td[text() and text() != '()' and @class='head-vol']"/>
<xsl:apply-templates select="//td[text() and text() != '()' and @class='foot-os']"/>
<xsl:apply-templates select="//td[text() and text() != '()' and @class='foot-date']"/>
</table>
</xsl:template>

<!-- Turn head header/footer cells into rows -->
<xsl:template match="td[@class='head-ltitle']">
<tr><td><em>Title:</em></td><td><xsl:apply-templates /></td></tr>
</xsl:template>

<xsl:template match="td[@class='head-vol']">
<tr><td><em>Volume:</em></td><td><xsl:apply-templates /></td></tr>
</xsl:template>

<xsl:template match="td[@class='foot-date']">
<tr><td><em>Date:</em></td><td><xsl:apply-templates /></td></tr>
</xsl:template>

<xsl:template match="td[@class='foot-os']">
<tr><td><em>Manual:</em></td><td><xsl:apply-templates /></td></tr>
</xsl:template>
</xsl:stylesheet>
1 change: 1 addition & 0 deletions lib/github/markup.rb
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ module Markups
MARKUP_RDOC = :rdoc
MARKUP_RST = :rst
MARKUP_TEXTILE = :textile
MARKUP_MANPAGE = :manpage
end

module Markup
Expand Down
2 changes: 2 additions & 0 deletions lib/github/markups.rb
Original file line number Diff line number Diff line change
Expand Up @@ -52,3 +52,5 @@
)

command(::GitHub::Markups::MARKUP_POD, :pod2html, /pod/, ["Pod"], "pod")

command(::GitHub::Markups::MARKUP_MANPAGE, :man2html, /(?:[1-9][a-z]*|0p|n|man|mdoc)(?:\.in)?/, ["Roff"], "manpage")
jordemort marked this conversation as resolved.
Show resolved Hide resolved
5 changes: 3 additions & 2 deletions test/markup_test.rb
Original file line number Diff line number Diff line change
Expand Up @@ -74,7 +74,7 @@ def call
message
end
end

def test_knows_what_it_can_and_cannot_render
assert_equal false, GitHub::Markup.can_render?('README.html', '<h1>Title</h1>')
assert_equal true, GitHub::Markup.can_render?('README.markdown', '=== Title')
Expand All @@ -92,8 +92,9 @@ def test_each_render_has_a_name
assert_equal "asciidoctor", GitHub::Markup.renderer('README.adoc', '== Title').name
assert_equal "restructuredtext", GitHub::Markup.renderer('README.rst', 'Title').name
assert_equal "pod", GitHub::Markup.renderer('README.pod', '=begin').name
assert_equal "manpage", GitHub::Markup.renderer('README.man', '.Dd $Mdocdate: March 23 1981 $').name
end

def test_rendering_by_symbol
assert_equal '<p><code>test</code></p>', GitHub::Markup.render_s(GitHub::Markups::MARKUP_MARKDOWN, '`test`').strip
end
Expand Down
6 changes: 6 additions & 0 deletions test/markups/README.man
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
.Dd $Mdocdate: March 23 1981 $
.Dt README 1
.Os GitHub
.Sh README.man

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is an mdoc(7) syntax error, which is probably not a good thing to have in an example/test file. The first section header in a manual page MUST be exactly ".Sh NAME". Your line is also invalid because it contains lower case letters.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And after that it should provide data for whatis(1)'s scraping; see lexgrog(1)'s WHATIS PARSING as an example.

.Sh NAME
.Nm foo
.Nd program to do something

.Nm README
.Nd This is an example readme in mandoc

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is an invalid example for a second reason. In addition to the NAME section, at least the DESCRIPTION section is also required in a manual page. So you should probably add

.SH DESCRIPTION
And one line of text.

or something like that.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ingo probably means .Sh

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oops, yes, of course, ".Sh DESCRIPTION", thanks for noticing.

23 changes: 23 additions & 0 deletions test/markups/README.man.html
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
<h1>README.man</h1>
<b><code>README</code></b> —
This is an example readme in mandoc

<hr>
<table>
<tr>
<td><em>Title:</em></td>
<td>README(1)</td>
</tr>
<tr>
<td><em>Volume:</em></td>
<td>General Commands Manual</td>
</tr>
<tr>
<td><em>Manual:</em></td>
<td>GitHub</td>
</tr>
<tr>
<td><em>Date:</em></td>
<td>March 23, 1981</td>
</tr>
</table>