Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixes for zimwiki writer #3403

Closed
wants to merge 92 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
92 commits
Select commit Hold shift + click to select a range
eade47b
Fixes for sourceview and table plugins
alexivkin Jan 31, 2017
dde9370
Init, link text and note fixes
alexivkin Feb 13, 2017
eae03db
Test fixes to reflect zimwiki bugfixes
alexivkin Feb 14, 2017
1945dd1
Update test output for skylighting >= 0.1.1.3.
jgm Jan 31, 2017
e74b089
Version bounds: require skylighting 0.1.1.4, bump bounds for blaze-*.
jgm Jan 31, 2017
ed0e55c
Name change OSX -> MacOS.
jgm Jan 31, 2017
d9cb41f
stack.yaml - use latest skylighting.
jgm Jan 31, 2017
9bff0bb
make_macos_package.sh - install cpphs if not present.
jgm Jan 31, 2017
dc9a81e
Reduce state in Org writer (#3404)
Feb 1, 2017
6f84dda
Added deb/.vagrant to gitignore
jgm Feb 1, 2017
1f446a6
Added skeletons for docs on customizing pandoc and using pandoc API.
jgm Feb 1, 2017
5869da7
Org.hs: remove misleading comment
Feb 2, 2017
f4e5add
HTML and DocBook writers: fix internal links with writerIdentifierPre…
mb21 Feb 3, 2017
ca87b34
Docx reader: Don't drop smartTag contents.
jgm Feb 3, 2017
bc7443d
Manual: Mention limitations of Literate Haskell Support (#3413)
nomeata Feb 3, 2017
bc50426
Docx reader: handle local namespace declarations.
jgm Feb 3, 2017
c6627b4
Added TODO comment.
jgm Feb 4, 2017
5f3f8c8
Better error messages for removed options.
jgm Feb 4, 2017
4dbcb44
Simplified Opt structure in cli option parsing.
jgm Feb 4, 2017
dc9633f
Small revision to deprecation message for --old-dashes.
jgm Feb 4, 2017
0da8209
Implemented +/-smart in rst writer.
jgm Feb 4, 2017
ac54fe9
Moved tests/ -> test/.
jgm Feb 4, 2017
7715866
Added skeleton for Tests.Command.
jgm Feb 4, 2017
b7ade9e
Expose setVerbosity in Text.Pandoc
jgm Feb 4, 2017
e6ff5f3
Markdown writer: Better escaping when +smart.
jgm Feb 4, 2017
5643e63
Added new test framework Tests.Command.
jgm Feb 4, 2017
0496ffc
Org writer: reduce to two spaces after bullets
tarleb Feb 4, 2017
96c0cd3
Added first command test to cabal metadata and repo.
jgm Feb 4, 2017
510d383
Consolidated some common functions in Tests.Helper.
jgm Feb 4, 2017
106a934
Improved escaping in RST writer with smart option.
jgm Feb 4, 2017
447ea7c
More smart escaping tests.
jgm Feb 4, 2017
781ccaa
More simplification of Opt in pandoc.hs.
jgm Feb 4, 2017
c108f2e
Changed writerEpubMetadata to a Maybe String.
jgm Feb 4, 2017
e7ec24e
More simplification of Opt in pandoc.hs.
jgm Feb 4, 2017
c260d9d
More simplification of pandoc.hs.
jgm Feb 5, 2017
6f5643c
Allow user to specify User-Agent (#3421)
thenaesh Feb 5, 2017
8490606
Removed redundant import.
jgm Feb 5, 2017
45bc069
Makefile: added BRANCH variable for winpkg
jgm Feb 5, 2017
ef1087e
pandoc.hs: make reader and writer Maybe values in Opt.
jgm Feb 5, 2017
6b8365a
Split pandoc.hs into a module, Text.Pandoc.App, and a small program.
jgm Feb 5, 2017
ecd77af
Handle language in inline code with --listings.
jgm Feb 5, 2017
e687d05
Fix an unneeded import warning.
jgm Feb 5, 2017
46b5daf
Removed another redundant import.
jgm Feb 6, 2017
ebafbf8
Use unicode-math by default in default.latex template.
jgm Feb 6, 2017
e9acb8f
Further refactoring of App.
jgm Feb 6, 2017
e9bc585
Consolidated file arguments into Opt.
jgm Feb 6, 2017
c0b5b94
Changed year on copyright message.
jgm Feb 6, 2017
9fc7583
Fix import
jgm Feb 6, 2017
1db6c79
default.beamer changes (Thomas Hodgson):
jgm Feb 6, 2017
dec95e0
Removed --parse-raw and readerParseRaw.
jgm Feb 6, 2017
a2f216a
Use latest dev version of pandoc-citeproc.
jgm Feb 7, 2017
d57da89
Update dev version of pandoc-citeproc in stack.yaml.
jgm Feb 7, 2017
3b8d645
Use latest pandoc-citeproc dev in stack builds.
jgm Feb 7, 2017
5eaf041
Makefile: make version overridable.
jgm Feb 7, 2017
e15ae8b
Refactored include file handling in LaTeX reader.
jgm Feb 7, 2017
05a4d9a
Moved readFileFromDirs to Text.Pandoc.Class.
jgm Feb 7, 2017
63fae14
Refactored some files formerly in LaTeX reader.
jgm Feb 7, 2017
98ce73d
RST reader: Improved admonition support.
jgm Feb 7, 2017
c93b35a
LaTeX reader: support `\lstinputlisting`.
jgm Feb 7, 2017
10883e6
Reverted deferred media bag code.
jgm Feb 9, 2017
0fc6308
LaTeX reader: Issue warnings when skipping unknown latex commands.
jgm Feb 9, 2017
cde6692
HTML reader: Added warnings for ignored material.
jgm Feb 10, 2017
99b59b7
LaTeX reader: Improved warning messages for unknown commands.
jgm Feb 10, 2017
5080ab6
LaTeX reader: Improved messages for skipped environments.
jgm Feb 10, 2017
b62b3da
Added Text.Pandoc.Logging (exported module).
jgm Feb 10, 2017
884197a
Logging: added ToJSON instance and showLogMessage.
jgm Feb 10, 2017
9edbdbf
Use new warnings throughout the code base.
jgm Feb 10, 2017
4b088e8
HTML writer: report when not rendering raw inline/block.
jgm Feb 10, 2017
37e2ed7
Class.report: Save all log messages in state.
jgm Feb 11, 2017
b5d65a7
Added --log option to save log messages in JSON format to a file.
jgm Feb 11, 2017
a652398
Logging: export logMessagesToJSON.
jgm Feb 11, 2017
cb56fde
Fixed small bug in RST list parsing.
jgm Feb 11, 2017
cc2471d
RST reader: Initial support of .. table directive.
jgm Feb 11, 2017
db35546
RST reader/writer: properly handle table captions.
jgm Feb 11, 2017
2bedd3f
RST reader: Support `.. line-block` directive.
jgm Feb 11, 2017
7e35ba4
Rename logMessagesToJSON -> encodeLogMessages.
jgm Feb 11, 2017
855e8a2
Add Org writer unit tests
Feb 12, 2017
3979c95
Do not strip # from Org anchor links
Feb 12, 2017
ff5fc7c
Comment out instruction to use cpphs in macos stack.yaml.
jgm Feb 12, 2017
6202488
stack.yamls: Use lts-8.0.
jgm Feb 12, 2017
6da044b
pandoc.cabal: use cpphs if embed_data_files and darwin.
jgm Feb 12, 2017
193dcce
More updates to macos/stack.yaml.
jgm Feb 12, 2017
30afa9a
Replaced {deb,macos,windows}/stack.yaml with stack.pkg.yaml.
jgm Feb 12, 2017
80ab982
Windows packaging fixes to use new stack.pkg.yaml.
jgm Feb 12, 2017
850300f
appveyor.yml: use -j1, in hopes that this will help with a GHC bug.
jgm Feb 12, 2017
471c606
deb/make_deb.sh fixes.
jgm Feb 12, 2017
f96ac49
appveyor.yml: Fixed some paths.
jgm Feb 12, 2017
4ba52ea
LaTeX reader: properly handle column prefixes/suffixes.
jgm Feb 13, 2017
abf3887
Removed unnecessary import.
jgm Feb 14, 2017
8d912d8
Catching up with upstream
alexivkin Feb 14, 2017
78d6603
Catching
alexivkin Feb 14, 2017
6284872
Resolved conflicts
alexivkin Feb 14, 2017
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion data/templates
Submodule templates updated 2 files
+1 −7 default.beamer
+1 −13 default.latex
78 changes: 54 additions & 24 deletions src/Text/Pandoc/Writers/ZimWiki.hs
Original file line number Diff line number Diff line change
Expand Up @@ -18,11 +18,11 @@ Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA

{- |
Module : Text.Pandoc.Writers.ZimWiki
Copyright : Copyright (C) 2008-2015 John MacFarlane, 2016 Alex Ivkin
Copyright : Copyright (C) 2008-2015 John MacFarlane, 2017 Alex Ivkin
License : GNU GPL, version 2 or above

Maintainer : Alex Ivkin <[email protected]>
Stability : alpha
Stability : beta
Portability : portable

Conversion of 'Pandoc' documents to ZimWiki markup.
Expand All @@ -44,20 +44,22 @@ import Data.Default (Default(..))
import Network.URI ( isURI )
import Control.Monad ( zipWithM )
import Control.Monad.State ( modify, State, get, evalState )
--import Control.Monad.Reader ( ReaderT, runReaderT, ask, local )
import Text.Pandoc.Class ( PandocMonad )
import qualified Data.Map as Map

data WriterState = WriterState {
stItemNum :: Int,
stIndent :: String -- Indent after the marker at the beginning of list items
stIndent :: String, -- Indent after the marker at the beginning of list items
stInTable :: Bool, -- Inside a table
stInLink :: Bool -- Inside a link description
}

instance Default WriterState where
def = WriterState { stItemNum = 1, stIndent = "" }
def = WriterState { stItemNum = 1, stIndent = "", stInTable = False, stInLink = False }

-- | Convert Pandoc to ZimWiki.
writeZimWiki :: PandocMonad m => WriterOptions -> Pandoc -> m String
writeZimWiki opts document = return $ evalState (pandocToZimWiki opts document) (WriterState 1 "")
writeZimWiki opts document = return $ evalState (pandocToZimWiki opts document) def

-- | Return ZimWiki representation of document.
pandocToZimWiki :: WriterOptions -> Pandoc -> State WriterState String
Expand Down Expand Up @@ -129,9 +131,15 @@ blockToZimWiki opts (Header level _ inlines) = do
return $ eqs ++ " " ++ contents ++ " " ++ eqs ++ "\n"

blockToZimWiki _ (CodeBlock (_,classes,_) str) = do
-- Remap languages into the gtksourceview2 convention that ZimWiki source code plugin is using
let langal = [("javascript", "js"), ("bash", "sh"), ("winbatch", "dosbatch")]
let langmap = Map.fromList langal
return $ case classes of
[] -> "'''\n" ++ cleanupCode str ++ "\n'''\n" -- no lang block is a quote block
(x:_) -> "{{{code: lang=\"" ++ x ++ "\" linenumbers=\"True\"\n" ++ str ++ "\n}}}\n" -- for zim's code plugin, go verbatim on the lang spec
[] -> "'''\n" ++ cleanupCode str ++ "\n'''\n" -- turn no lang block into a quote block
(x:_) -> "{{{code: lang=\"" ++
(case Map.lookup x langmap of
Nothing -> x
Just y -> y) ++ "\" linenumbers=\"True\"\n" ++ str ++ "\n}}}\n" -- for zim's code plugin, go verbatim on the lang spec

blockToZimWiki opts (BlockQuote blocks) = do
contents <- blockListToZimWiki opts blocks
Expand All @@ -145,7 +153,7 @@ blockToZimWiki opts (Table capt aligns _ headers rows) = do
return $ "" ++ c ++ "\n"
headers' <- if all null headers
then zipWithM (tableItemToZimWiki opts) aligns (rows !! 0)
else zipWithM (tableItemToZimWiki opts) aligns headers
else mapM (inlineListToZimWiki opts) (map removeFormatting headers) -- emphasis, links etc. are not allowed in table headers
rows' <- mapM (zipWithM (tableItemToZimWiki opts) aligns) rows
let widths = map (maximum . map length) $ transpose (headers':rows')
let padTo (width, al) s =
Expand All @@ -167,10 +175,10 @@ blockToZimWiki opts (Table capt aligns _ headers rows) = do
then replicate (width-1) '-' ++ ":"
else ":" ++ replicate (width-2) '-' ++ ":"
let underheader = "|" ++ intercalate "|" (zipWith borderCell (zip widths aligns) headers') ++ "|"
let renderRow sep cells = sep ++ intercalate sep (zipWith padTo (zip widths aligns) cells) ++ sep
let renderRow cells = "|" ++ intercalate "|" (zipWith padTo (zip widths aligns) cells) ++ "|"
return $ captionDoc ++
(if null headers' then "" else renderRow "|" headers' ++ "\n") ++ underheader ++ "\n" ++
unlines (map (renderRow "|") rows')
(if null headers' then "" else renderRow headers' ++ "\n") ++ underheader ++ "\n" ++
unlines (map renderRow rows')

blockToZimWiki opts (BulletList items) = do
indent <- stIndent <$> get
Expand Down Expand Up @@ -255,7 +263,9 @@ tableItemToZimWiki opts align' item = do
(if align' == AlignLeft || align' == AlignCenter
then " "
else "")
contents <- blockListToZimWiki opts item -- local (\s -> s { stBackSlashLB = True }) $
modify $ \s -> s { stInTable = True }
contents <- blockListToZimWiki opts item
modify $ \s -> s { stInTable = False }
return $ mkcell contents

-- | Convert list of Pandoc block elements to ZimWiki.
Expand Down Expand Up @@ -305,7 +315,15 @@ inlineToZimWiki opts (Cite _ lst) = inlineListToZimWiki opts lst

inlineToZimWiki _ (Code _ str) = return $ "''" ++ str ++ "''"

inlineToZimWiki _ (Str str) = return $ escapeString str
inlineToZimWiki _ (Str str) = do
inTable <- stInTable <$> get
inLink <- stInLink <$> get
if inTable
then return $ substitute "|" "\\|" . escapeString $ str
else
if inLink
then return $ str
else return $ escapeString str

inlineToZimWiki _ (Math mathType str) = return $ delim ++ str ++ delim -- note: str should NOT be escaped
where delim = case mathType of
Expand All @@ -318,7 +336,11 @@ inlineToZimWiki opts (RawInline f str)
| f == Format "html" = do cont <- indentFromHTML opts str; return cont
| otherwise = return ""

inlineToZimWiki _ LineBreak = return "\n" -- was \\\\
inlineToZimWiki _ LineBreak = do
inTable <- stInTable <$> get
if inTable
then return "\\n"
else return "\n"

inlineToZimWiki opts SoftBreak =
case writerWrapText opts of
Expand All @@ -329,30 +351,38 @@ inlineToZimWiki opts SoftBreak =
inlineToZimWiki _ Space = return " "

inlineToZimWiki opts (Link _ txt (src, _)) = do
label <- inlineListToZimWiki opts txt
inTable <- stInTable <$> get
modify $ \s -> s { stInLink = True }
label <- inlineListToZimWiki opts $ removeFormatting txt -- zim does not allow formatting in link text, it takes the text verbatim, no need to escape it
modify $ \s -> s { stInLink = False }
let label'= if inTable
then "" -- no label is allowed in a table
else "|"++label
case txt of
[Str s] | "mailto:" `isPrefixOf` src -> return $ "<" ++ s ++ ">"
| escapeURI s == src -> return src
_ -> if isURI src
then return $ "[[" ++ src ++ "|" ++ label ++ "]]"
else return $ "[[" ++ src' ++ "|" ++ label ++ "]]"
then return $ "[[" ++ src ++ label' ++ "]]"
else return $ "[[" ++ src' ++ label' ++ "]]"
where src' = case src of
'/':xs -> xs -- with leading / it's a
_ -> src -- link to a help page
inlineToZimWiki opts (Image attr alt (source, tit)) = do
alt' <- inlineListToZimWiki opts alt
let txt = case (tit, alt) of
("", []) -> ""
("", _ ) -> "|" ++ alt'
(_ , _ ) -> "|" ++ tit
inTable <- stInTable <$> get
let txt = case (tit, alt, inTable) of
("",[], _) -> ""
("", _, False ) -> "|" ++ alt'
(_ , _, False ) -> "|" ++ tit
(_ , _, True ) -> ""
-- Relative links fail isURI and receive a colon
prefix = if isURI source then "" else ":"
return $ "{{" ++ prefix ++ source ++ imageDims opts attr ++ txt ++ "}}"

inlineToZimWiki opts (Note contents) = do
-- no concept of notes in zim wiki, use a text block
contents' <- blockListToZimWiki opts contents
return $ "((" ++ contents' ++ "))"
-- note - may not work for notes with multiple blocks
return $ " **{Note:** " ++ trimr contents' ++ "**}**"

imageDims :: WriterOptions -> Attr -> String
imageDims opts attr = go (toPx $ dimension Width attr) (toPx $ dimension Height attr)
Expand Down
46 changes: 23 additions & 23 deletions test/tables.zimwiki
Original file line number Diff line number Diff line change
@@ -1,43 +1,43 @@
Simple table with caption:

Demonstration of simple table syntax.
| Right|Left | Center |Default|
|------:|:-----|:--------:|-------|
| 12|12 | 12 |12 |
| 123|123 | 123 |123 |
| 1|1 | 1 |1 |
|Right|Left |Center |Default|
|----:|:----|:-----:|-------|
| 12|12 | 12 |12 |
| 123|123 | 123 |123 |
| 1|1 | 1 |1 |

Simple table without caption:

| Right|Left | Center |Default|
|------:|:-----|:--------:|-------|
| 12|12 | 12 |12 |
| 123|123 | 123 |123 |
| 1|1 | 1 |1 |
|Right|Left |Center |Default|
|----:|:----|:-----:|-------|
| 12|12 | 12 |12 |
| 123|123 | 123 |123 |
| 1|1 | 1 |1 |

Simple table indented two spaces:

Demonstration of simple table syntax.
| Right|Left | Center |Default|
|------:|:-----|:--------:|-------|
| 12|12 | 12 |12 |
| 123|123 | 123 |123 |
| 1|1 | 1 |1 |
|Right|Left |Center |Default|
|----:|:----|:-----:|-------|
| 12|12 | 12 |12 |
| 123|123 | 123 |123 |
| 1|1 | 1 |1 |

Multiline table with caption:

Here’s the caption. It may span multiple lines.
| Centered Header |Left Aligned | Right Aligned|Default aligned |
|:-----------------:|:-------------|--------------:|:------------------------------------------------------|
| First |row | 12.0|Example of a row that spans multiple lines. |
| Second |row | 5.0|Here’s another one. Note the blank line between rows. |
|Centered Header|Left Aligned|Right Aligned|Default aligned |
|:-------------:|:-----------|------------:|:------------------------------------------------------|
| First |row | 12.0|Example of a row that spans multiple lines. |
| Second |row | 5.0|Here’s another one. Note the blank line between rows. |

Multiline table without caption:

| Centered Header |Left Aligned | Right Aligned|Default aligned |
|:-----------------:|:-------------|--------------:|:------------------------------------------------------|
| First |row | 12.0|Example of a row that spans multiple lines. |
| Second |row | 5.0|Here’s another one. Note the blank line between rows. |
|Centered Header|Left Aligned|Right Aligned|Default aligned |
|:-------------:|:-----------|------------:|:------------------------------------------------------|
| First |row | 12.0|Example of a row that spans multiple lines. |
| Second |row | 5.0|Here’s another one. Note the blank line between rows. |

Table without column headers:

Expand Down
12 changes: 4 additions & 8 deletions test/writer.zimwiki
Original file line number Diff line number Diff line change
Expand Up @@ -606,22 +606,18 @@ Here is a movie {{:movie.jpg|movie}} icon.

====== Footnotes ======

Here is a footnote reference,((Here is the footnote. It can go anywhere after the footnote reference. It need not be placed at the end of the document.
)) and another.((Here’s the long note. This one contains multiple blocks.
Here is a footnote reference, **{Note:** Here is the footnote. It can go anywhere after the footnote reference. It need not be placed at the end of the document.**}** and another. **{Note:** Here’s the long note. This one contains multiple blocks.

Subsequent blocks are indented to show that they belong to the footnote (as with list items).

'''
{ <code> }
'''

If you want, you can indent every line, but you can also be lazy and just indent the first line of each block.
)) This should //not// be a footnote reference, because it contains a space.[^my note] Here is an inline note.((This is //easier// to type. Inline notes may contain [[http://google.com|links]] and '']'' verbatim characters, as well as [bracketed text].
))
If you want, you can indent every line, but you can also be lazy and just indent the first line of each block.**}** This should //not// be a footnote reference, because it contains a space.[^my note] Here is an inline note. **{Note:** This is //easier// to type. Inline notes may contain [[http://google.com|links]] and '']'' verbatim characters, as well as [bracketed text].**}**

> Notes can go in quotes.((In quote.
> ))
> Notes can go in quotes. **{Note:** In quote.**}**

1. And in list items.((In list.))
1. And in list items. **{Note:** In list.**}**

This paragraph should not be part of the note, as it is not indented.