Merge pull request #48 from texmacs/develop

Minor updates
texmacs · Oct 24, 2023 · 57c881e · 57c881e
2 parents dce23ec + 5150813
commit 57c881e
Show file tree

Hide file tree

Showing 11 changed files with 1,078 additions and 270 deletions.
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
@@ -0,0 +1,58 @@
+# Contributing to the TeXmacs markdown plugin
+
+Follow the setup instructions in [README.md](README.md) to get started.
+
+**TODO:** this document is a stub.
+
+## Code organization
+
+Conversion is done in two stages:
+
+1. From TeXmacs to an internal stree representation: `tmmarkdown.scm`. 
+2. From the internal representation to markdown: `markdownout.scm`.
+
+The plugin is loaded in `init-markdown.scm`, and some utilities are provided in
+`markdown-utils.scm`. Extensions to TeXmacs menus are in `markdown-menus.scm`.
+
+Tests are in `tests/` and are can be run from within TeXmacs with the markdown
+menu.
+
+## Conversion from TeXmacs to internal representation
+
+The entry point is `tm->markdown*` in `tmmarkdown.scm`. The function takes a
+TeXmacs tree and returns an slist of markdown elements. This is also a 
+pre-processing step does things like dropping TeXmacs-specific info, adding
+labels to equations, setting up counters, etc.
+
+Parsing functions always return an slist. Some arguments are processed and
+`tm->markdown*` is typically called on the remaining ones recursively.
+
+
+## Conversion from internal representation to markdown
+
+The entry point is `serialize-markdown` in `markdownout.scm`.
+
+## Some suggestions for development
+
+Like all TeXmacs code, the entry point for each module is located at the
+bottom of the file.
+
+Scheme sessions inside TeXmacs documents are very useful to prototype ideas.
+First you want to add whatever tag it is that you are interested, say `mytag`
+to the top of the document, then you read it into scheme with:
+
+```scheme
+(select (buffer-get-body (current-buffer)) '(:* mytag))
+```
+
+This will return a list of trees with the tag `mytag`. You can then pick any
+one of them and process it with the `tm-` prefixed built-in functions:
+* `(tm-ref x pos)`: returns the `pos`th child of `x`.
+* `(tm->string x)`: returns a string representation of `x`.
+* `(tm->markdown* x)`: returns an slist of markdown elements.
+
+## Testing
+
+Tests are located in `tests/` and are run from within TeXmacs with the markdown
+menu. Each test reads a TeXmacs file and compares it with a homonymous markdown
+file in the same directory.
diff --git a/LICENSE b/LICENSE
diff --git a/README.md b/README.md
@@ -9,17 +9,18 @@ possibility of exporting them as html into the markdown file.
 
 The plugin has been developed for its use in two specific websites, 
 [Paperwhy](https://paperwhy.8027.org/) and appliedAI's 
-[TransferLab](https://transferlab.appliedai.de/), and can use multiple 
-extensions specific to the static website generator [Hugo](https://gohugo.io/). 
-There might still be some code very specific to those sites, YMMV.
+[TransferLab](https://transferlab.ai/), and can use multiple extensions 
+specific to the static website generator [Hugo](https://gohugo.io/). There 
+might still be some code very specific to those sites, YMMV.
 
 ## Setup
 
-Clone this repository into your `~/.TeXmacs/plugins` directory as `markdown`. 
-For Linux / OSX this is:
+Clone this repository into your `~/.TeXmacs/plugins` directory as `markdown` 
+and checkout the `master` branch. For Linux / OSX this is:
 
 ```shell
-git clone https://github.com/texmacs/markdown.git ~/.TeXmacs/plugins/markdown
+git clone -b master https://github.com/texmacs/markdown.git 
+~/.TeXmacs/plugins/markdown
 ```
 
 For Windows, the path (usually?) is
@@ -55,7 +56,9 @@ package -> Markdown -> markdown` to enable:
 
 * Support for alternate image formats between TeXmacs and markdown via 
   `<md-alt-image|tm-image|md-image>`. This is useful e.g. to provide 
-  SVG and EPS/PDF versions of images for print and web respectively.
+  SVG and EPS/PDF versions of images for print and web respectively. The same 
+  effect can be achieved with `<specific|markdown|…>` and 
+  `<specific|texmacs|…>`.
 * Macros for labels in equation arrays. Using positioning tricks with 
   `<htab>` results in LaTeX code that MathJax does not support, namely 
   `\hfill`. For this reason, we provide two macros 
@@ -107,7 +110,11 @@ All custom shortcodes are in `extensions/hugo/`.
 
 * Figures are converted to `{{< figure  >}}`.
 * For arbitrary shortcodes, use `<hugo-short>`, e.g. 
-  `<hugo-short|toc>` for `{{< toc >}}`.
+  `<hugo-short|toc>` for `{{< toc >}}`. In order to pass 
+  named arguments, use 
+  `<hugo-short|shortcode-name|argname1|argvalue1|…>`. For 
+  positional arguments, replace the argument names by `#f` as in 
+  `<hugo-short|shortcode-name|#f|posvalue1|#f|posvalue2|…>`.
 * Citations using `<cite>` and `<cite-detail>` (of any arity) 
   are automatically detected and converted to `{{< cite ref >}}`, and 
   all of them are gathered in the frontmatter as well, for indization by Hugo's 

diff --git a/doc/markdown.en.tm b/doc/markdown.en.tm
@@ -1,9 +1,9 @@
-<TeXmacs|2.1>
+<TeXmacs|2.1.2>
 
 <style|tmdoc>
 
 <\body>
-  <tmdoc-title|Markdown plug-in>
+  <specific|texmacs|<tmdoc-title|Markdown plug-in>>
 
   <section|<TeXmacs> markdown plugin>
 
@@ -16,18 +16,19 @@
 
   The plugin has been developed for its use in two specific websites,
   <hlink|Paperwhy|https://paperwhy.8027.org/> and appliedAI's
-  <hlink|TransferLab|https://transferlab.appliedai.de/>, and can use multiple
+  <hlink|TransferLab|https://transferlab.ai/>, and can use multiple
   extensions specific to the static website generator
   <hlink|Hugo|https://gohugo.io/>. There might still be some code very
   specific to those sites, YMMV.
 
   <subsection|Setup>
 
   Clone this repository into your <shell|~/.TeXmacs/plugins> directory as
-  <shell|markdown>. For Linux / OSX this is:
+  <shell|markdown> and checkout the <shell|master> branch. For Linux / OSX
+  this is:
 
   <\shell-code>
-    git clone https://github.com/texmacs/markdown.git
+    git clone -b master https://github.com/texmacs/markdown.git
     ~/.TeXmacs/plugins/markdown
   </shell-code>
 
@@ -73,7 +74,9 @@
     <item>Support for alternate image formats between <TeXmacs> and markdown
     via <explain-macro|md-alt-image|tm-image|md-image>. This is useful e.g.
     to provide SVG and EPS/PDF versions of images for print and web
-    respectively.
+    respectively. The same effect can be achieved with
+    <explain-macro|specific|markdown|<text-dots>> and
+    <explain-macro|specific|texmacs|<text-dots>>.
 
     <item>Macros for labels in equation arrays. Using positioning tricks with
     <explain-macro|htab> results in <LaTeX> code that <name|MathJax> does not
@@ -143,7 +146,10 @@
     \<gtr\>}}>.
 
     <item>For arbitrary shortcodes, use <explain-macro|hugo-short>, e.g.
-    <explain-macro|hugo-short|toc> for <code*|{{\<less\> toc \<gtr\>}}>.
+    <explain-macro|hugo-short|toc> for <code*|{{\<less\> toc \<gtr\>}}>. In
+    order to pass named arguments, use <explain-macro|hugo-short|shortcode-name|argname1|argvalue1|<text-dots>>.
+    For positional arguments, replace the argument names by <tt|#f> as in
+    <explain-macro|hugo-short|shortcode-name|#f|posvalue1|#f|posvalue2|<text-dots>>.
 
     <item>Citations using <explain-macro|cite> and
     <explain-macro|cite-detail> (of any arity) are automatically detected and
@@ -189,4 +195,4 @@
   <\collection>
     <associate|markdown-auto-export|../README.md>
   </collection>
-</initial>
+</initial>
diff --git a/packages/markdown/hugo.ts b/packages/markdown/hugo.ts
@@ -1,4 +1,4 @@
-<TeXmacs|2.1>
+<TeXmacs|2.1.2>
 
 <style|source>
 
@@ -28,6 +28,8 @@
     </src-comment>
   </active*>
 
+  <assign|hugo-short|<xmacro|args|<flag|<merge|hugo-|<arg|args|0>>|green>>>
+
   <assign|hugo-front|<xmacro|args|<flag|frontmatter parameters|orange>>>
 
   <assign|dict|<xmacro|kvs|dict (to do: display keys and values)>>

diff --git a/progs/markdown-menus.scm b/progs/markdown-menus.scm
@@ -102,3 +102,8 @@
 (menu-bind texmacs-extra-menu
   (if (and (supports-markdown?) (markdown-menu-show?))
        (=> "Markdown" (link markdown-menu))))
+
+(menu-bind specific-menu
+  (when (not (selection-active-large?))
+    (former)
+    ("Markdown" (make-specific "markdown"))))
diff --git a/progs/markdownout.scm b/progs/markdownout.scm
@@ -136,10 +136,9 @@
 (define (md-abstract x)
   (if (hugo-extensions?)
       (with-md-globals 'paragraph-width #f
-        (with-md-globals 'disable-shortcodes #t
-          (md-hugo-frontmatter
+        (md-hugo-frontmatter
            `(hugo-front "summary" ,(serialize-markdown* (cdr x))))))
-      (md-paragraph `(concat (strong "Abstract: ") (em ,(cdr x))))))
+      (md-paragraph `(concat (strong "Abstract: ") (em ,(cdr x)))))
 
 (define (must-adjust? t)
   (tm-in? t '(strong em tt strike math concat cite cite-detail 
@@ -394,7 +393,7 @@
       (with citations 
           (filter (lambda (x) (and (string? x) (not (string-null? x)))) (cdr x))
         (md-set 'refs (append (md-get 'refs) citations))
-        (md-hugo-shortcode 
+        (md-hugo-shortcode* 
          (cons 'cite (map (lambda (s) `(#f . ,s)) citations))))))
 
 (define (md-cite-detail x)
@@ -423,8 +422,8 @@
           (begin
             (set! args (assoc-remove-many args '(body name caption)))
             (set! args (assoc-append? args 'class (md-get 'html-class)))
-            (md-hugo-shortcode `(,type ,@args)
-                               `(document ,body (concat ,name ,caption))))
+            (md-hugo-shortcode* `(,type ,@args)
+                                `(document ,body (concat ,name ,caption))))
           (with content (if (assoc 'src args)
                             `(image ,(assoc-ref args 'src) ,body)
                             body)
@@ -464,7 +463,12 @@
           (map set-pair! (list->assoc (cdr x))))))
   "")
 
-(define (md-hugo-shortcode x . inner)
+(define (md-hugo-shortcode x)
+  "Processes '(hugo-short shortcode-name (args)) where args is a list of tuples (name val). For unnamed arguments, use (#f val)"
+  (md-hugo-shortcode* (cdr x)))
+
+(define (md-hugo-shortcode* x . inner)
+  "Inner processing of shortcodes"
   (if (not (md-get 'disable-shortcodes))
       (let ((shortcode (symbol->string (car x)))
             (args (cdr x))
@@ -485,14 +489,14 @@
 
 (define (md-bibliography x)
   (if (hugo-extensions?) 
-      (md-hugo-shortcode '(references))
+      (md-hugo-shortcode* '(references))
       (md-style '(strong "Bibliography not implemented for raw Markdown"))))
 
 (define (md-sidenote-sub x numbered?)
   (if (hugo-extensions?)
       (let ((numbered (if numbered? '((numbered . "numbered")) '()))
             (args (cdr x)))
-        (md-hugo-shortcode
+        (md-hugo-shortcode*
          (append `(sidenote (halign . ,(md-marginal-style (first args)))
                             (valign . ,(md-marginal-style (second args))))
                  numbered)
@@ -574,71 +578,71 @@
 
 (define serialize-hash (make-ahash-table))
 (map (lambda (l) (apply (cut ahash-set! serialize-hash <> <>) l))
-     (list (list 'identity skip)
-           (list 'markdown md-markdown)
-           (list 'localize md-translate)
-           (list 'labels md-labels)
-           (list 'strong md-style)
-           (list 'em md-style)
-           (list 'tt md-style)
-           (list 'strike md-style)
-           (list 'underline md-style)
-           (list 'block md-block)
-           (list 'quotation md-quotation)
-           (list 'document md-document)
-           (list 'std-env (md-make-environment 'em))
-           (list 'std-env* (md-make-environment* 'em))
-           (list 'plain-env (md-make-environment 'identity))
-           (list 'plain-env* (md-make-environment* 'identity))
-           (list 'dueto md-dueto)
-           (list 'math md-math)
-           (list 'equation md-numbered-equation)
-           (list 'equation* md-equation)
-           (list 'eqnarray md-numbered-equation)
-           (list 'eqnarray* md-equation)
-           (list 'concat md-concat)
-           (list 'item md-item)
-           (list 'itemize md-list)
-           (list 'enumerate md-list)
-           (list 'h1 (md-header 1))
-           (list 'h2 (md-header 2))
-           (list 'h3 (md-header 3))
-           (list 'para md-para)
-           (list 'doc-date md-doc-date)
-           (list 'doc-title md-doc-title)
-           (list 'doc-subtitle md-doc-subtitle)
-           (list 'doc-author md-doc-author)
-           (list 'abstract md-abstract)
-           (list 'paper-author-name paper-author-add)  ; Paperwhy extension
-           (list 'cite md-cite)
-           (list 'cite-detail md-cite-detail)
-           (list 'eqref md-eqref)
-           (list 'label md-label)
-           (list 'reference md-reference)
-           (list 'footnote md-footnote)
-           (list 'todo md-todo)
-           (list 'image md-image)
-           (list 'html-class md-html-class)
-           (list 'small-figure (md-figure 'tmfigure '(class . "small-figure")))
-           (list 'big-figure
-             (md-figure 'tmfigure
-                        '(marginal-caption . #t) '(class . "big-figure")))
-           (list 'wide-figure (md-figure 'tmfigure '(class . "wide-figure")))
-           (list 'marginal-figure (md-figure 'sidefigure))
-           (list 'small-table (md-figure 'tmfigure))
-           (list 'big-table (md-figure 'tmfigure ))
-           (list 'tabular md-tabular)
-           (list 'hlink md-hlink)
-           (list 'tags md-hugo-tags)  ; Hugo extension (DEPRECATED)
-           (list 'hugo-short md-hugo-shortcode)  ; Hugo extension
-           (list 'hugo-front md-hugo-frontmatter)  ; Hugo extension
-           (list 'table-of-contents md-toc) ; Hugo extension
-           (list 'bibliography md-bibliography)  ; TfL extension
-           (list 'marginal-note md-sidenote) ; TfL extension
-           (list 'marginal-note* md-sidenote*) ; TfL extension
-           (list 'explain-macro md-explain-macro)
-           (list 'tmdoc-copyright md-tmdoc-copyright)
-           ))
+     (list
+      (list 'abstract md-abstract)
+      (list 'bibliography md-bibliography)  ; TfL extension
+      (list 'big-figure
+        (md-figure 'tmfigure '(marginal-caption . #t) '(class . "big-figure")))
+      (list 'big-table (md-figure 'tmfigure ))
+      (list 'block md-block)
+      (list 'cite-detail md-cite-detail)
+      (list 'cite md-cite)
+      (list 'concat md-concat)
+      (list 'doc-author md-doc-author)
+      (list 'doc-date md-doc-date)
+      (list 'doc-subtitle md-doc-subtitle)
+      (list 'doc-title md-doc-title)
+      (list 'document md-document)
+      (list 'dueto md-dueto)
+      (list 'em md-style)
+      (list 'enumerate md-list)
+      (list 'eqnarray* md-equation)
+      (list 'eqnarray md-numbered-equation)
+      (list 'eqref md-eqref)
+      (list 'equation* md-equation)
+      (list 'equation md-numbered-equation)
+      (list 'explain-macro md-explain-macro)
+      (list 'footnote md-footnote)
+      (list 'h1 (md-header 1))
+      (list 'h2 (md-header 2))
+      (list 'h3 (md-header 3))
+      (list 'hlink md-hlink)
+      (list 'html-class md-html-class)
+      (list 'hugo-front md-hugo-frontmatter)  ; Hugo extension
+      (list 'hugo-short md-hugo-shortcode)  ; Hugo extension
+      (list 'identity skip)
+      (list 'image md-image)
+      (list 'itemize md-list)
+      (list 'item md-item)
+      (list 'label md-label)
+      (list 'labels md-labels)
+      (list 'localize md-translate)
+      (list 'marginal-figure (md-figure 'sidefigure))
+      (list 'marginal-note md-sidenote) ; TfL extension
+      (list 'marginal-note* md-sidenote*) ; TfL extension
+      (list 'markdown md-markdown)
+      (list 'math md-math)
+      (list 'paper-author-name paper-author-add)  ; Paperwhy extension
+      (list 'para md-para)
+      (list 'plain-env (md-make-environment 'identity))
+      (list 'plain-env* (md-make-environment* 'identity))
+      (list 'quotation md-quotation)
+      (list 'reference md-reference)
+      (list 'small-figure (md-figure 'tmfigure '(class . "small-figure")))
+      (list 'small-table (md-figure 'tmfigure))
+      (list 'std-env (md-make-environment 'em))
+      (list 'std-env* (md-make-environment* 'em))
+      (list 'strike md-style)
+      (list 'strong md-style)
+      (list 'table-of-contents md-toc) ; Hugo extension
+      (list 'tabular md-tabular)
+      (list 'tags md-hugo-tags)  ; Hugo extension (DEPRECATED)
+      (list 'tmdoc-copyright md-tmdoc-copyright)
+      (list 'todo md-todo)
+      (list 'tt md-style)
+      (list 'underline md-style)
+      (list 'wide-figure (md-figure 'tmfigure '(class . "wide-figure")))
+    ))
 
 ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
 ;; Public interface