fix(#4460): refactor post escape #4472

SukkaW · 2020-08-11T12:47:18Z

What does it do?

Basically the PR is reverting #4254.

Given a code block like this:

```html
<body>
  <!-- here goes the rest of the page -->
</body>
```

backtick_code_block will use prismjs to convert the code block into html:

<!--hexoPostRenderEscape:<pre class="line-numbers language-html" data-language="html"><code class="language-html"><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span>body</span><span class="token punctuation">></span></span>
<span class="token comment">&lt;!-- here goes the rest of the page --></span>
<span class="token tag"><span class="token tag"><span class="token punctuation">&lt;/</span>body</span><span class="token punctuation">></span></span><span aria-hidden="true" class="line-numbers-rows"><span></span><span></span><span></span></span></code></pre>:hexoPostRenderEscape-->

 is added by Hexo to avoid missing paragraph issue.

Why they are needed, what is "missing paragraph" issue: #4171 (comment)

However, instead of ignoring the comment, marked.js.org will add extra <p> tag:

<!--hexoPostRenderEscape:<pre class="line-numbers language-html" data-language="html"><code class="language-html"><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span>body</span><span class="token punctuation">></span></span>
  <span class="token comment">&lt;!-- here goes the rest of the page --></span>
<p><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;/</span>body</span><span class="token punctuation">&gt;</span></span><span aria-hidden="true" class="line-numbers-rows"><span></span><span></span><span></span></span></code></pre>:hexoPostRenderEscape--&gt;</p>

With extra <p> tag the code block is now broken and :hexoPostRenderEscape--> can not be matched & removed.

Ref: marked.js demo: https://marked.js.org/demo/

So to prevent marked.js from breaking the html, the PR reverts the behavior introduced by #4254

The performance is not affected.

How to test

git clone -b fix-4460 https://github.com/sukkaw/hexo.git
cd hexo
npm install
npm test

Screenshots

Pull request tasks

Add test cases for the changes.
Passed the CI test.

cc @jerryc127.

coveralls · 2020-08-11T12:50:29Z

Coverage increased (+0.0005%) to 98.295% when pulling 75acca6 on SukkaW:fix-4460 into b7d15b9 on hexojs:master.

SukkaW · 2020-08-11T13:46:17Z

@curbengh @stevenjoezhang

We should also suggret user stick to highlight.js, since prismjs is not completely safe and robust enough: PrismJS/prism#2516

stevenjoezhang · 2020-08-11T14:07:53Z

Yes, the HTML escaping issue may destroy the structure of the entire page.

curbengh · 2020-08-12T08:32:11Z

We should also suggret user stick to highlight.js, since prismjs is not completely safe and robust enough:

Syntax highlight still says highlight,js is the default. Configuration page hasn't mention prism yet.

Do you want to switch the priority?

hexo/lib/plugins/filter/before_post_render/backtick_code_block.js

Lines 53 to 64 in b7d15b9

    
           // Since prismjs have better performance, so prismjs should have higher priority. 
        
           if (prismCfg.enable) { 
        
             if (!prismHighlight) prismHighlight = require('hexo-util').prismHighlight; 
        
             const options = { 
        
               lineNumber: prismCfg.line_number, 
        
               tab: prismCfg.tab_replace, 
        
               isPreprocess: prismCfg.preprocess, 
        
               lang 
        
             }; 
        
             content = prismHighlight(content, options);

test/scripts/hexo/post.js

jerryc127 · 2020-08-12T09:20:05Z

如果书写codeblock在列表下，会同时出现‘<!–hexoPostRenderEscape:’ 和‘:hexoPostRenderEscape–>’

1.在md里，列表与代码块没有空行，会渲染出错

在md里，列表与代码块有空行，但是代码块前面有空格，会渲染出错

在md里，列表与代码块有空行，代码块前面没有空格，渲染才正常

测试代码

1. 测试

```diff
+comments:
+  # Up to two comments system, the first will be shown as default
+  # Choose: Disqus/Disqusjs/Livere/Gitalk/Valine/Utterances/Facebook Comments
+  use:
+  # - Valine
+  # - Disqus
+  text: true # Display the comment name next to the button
+  # lazyload: The comment system will be load when comment element enters the browser's viewport.
+  # If you set it to false, the comment count will be invalid
+  lazyload: false
+  count: false # Display comment count in top_img

disqus:
-  enable: false
-  count: false # dispaly comment count in top_img

disqusjs:
-  enable: false
-  count: false # dispaly comment count in top_img

livere:
-  enable: false

gitalk:
-  enable: false
-  count: false # dispaly comment count in top_img

valine:
-  enable: false # if you want use valine,please set this value is true
-  count: false # dispaly comment count in top_img

utterances:
-  enable: false

facebook_comments:
-  enable: false
-  count: false
```

curbengh · 2020-08-13T03:15:51Z

@jerryc127

May I have your prismjs config? I only tested with:

# _config.yml
prismjs:
  enable: true

Have you tested this PR? $ npm install hexo@SukkaW/hexo#fix-4460

jerryc127 · 2020-08-13T05:31:01Z

@jerryc127

May I have your prismjs config? I only tested with:
# _config.yml
prismjs:
  enable: true
Have you tested this PR? $ npm install hexo@SukkaW/hexo#fix-4460

I test the pr. it solved.
but i find the problem still appear in Tag Plugins.
It hasn't show the ‘<!–hexoPostRenderEscape:’ and ‘:hexoPostRenderEscape–>’, still cannot render correct

In markdown, use the Tag Plugins, such as 'pullquote', there is no blank line in the list and code block, it will rendering errors
In markdown, use the Tag Plugins, such as 'pullquote', the list and code block have blank lines, but there are spaces in front of the code block, it will rendering errors

{% pullquote %}

1. Test Test
```diff
+comments:
+  # Up to two comments system, the first will be shown as default
+  # Choose: Disqus/Disqusjs/Livere/Gitalk/Valine/Utterances/Facebook Comments
+  use:
+  # - Valine
+  # - Disqus
+  text: true # Display the comment name next to the button
+  # lazyload: The comment system will be load when comment element enters the browser's viewport.
+  # If you set it to false, the comment count will be invalid
+  lazyload: false
+  count: false # Display comment count in top_img

disqus:
-  enable: false
-  count: false # dispaly comment count in top_img

disqusjs:
-  enable: false
-  count: false # dispaly comment count in top_img

livere:
-  enable: false

gitalk:
-  enable: false
-  count: false # dispaly comment count in top_img

valine:
-  enable: false # if you want use valine,please set this value is true
-  count: false # dispaly comment count in top_img

utterances:
-  enable: false

facebook_comments:
-  enable: false
-  count: false
```

{% endpullquote %}
```

the setting

highlight:
  enable: false
  line_number: true
  auto_detect: false
  tab_replace: ''
  wrap: true
  hljs: false
prismjs:
  enable: true
  preprocess: true
  line_number: true
  tab_replace: ''

SukkaW · 2020-08-13T06:32:29Z

Do you want to switch the priority?

@curbengh No worry, only prismjs.enable = true will let prismjs has higher priority.

SukkaW · 2020-08-13T06:50:54Z

@stevenjoezhang @curbengh

Since the prismjs escaping has been fixed, the PR is ready for review now.

curbengh · 2020-08-13T07:32:32Z

but i find the problem still appear in Tag Plugins.

Rendered fine for me. I noticed you had an extra triple backtick at the last line.

jerryc127 · 2020-08-13T10:28:58Z

但我發現問題仍然出現在標籤插件中。

對我來說很好。我注意到您在最後一行有一個額外的三重奏。

umm...
i try the landscape theme
it still cannot render correct
i dont konw why

SukkaW · 2020-08-13T10:32:49Z

@jerryc127 Would you mind update your status at #4460 instead? The PR is mainly focused on fixing hexoPostRenderEscape being shown.

curbengh

WFM

SukkaW requested review from seaoak, stevenjoezhang and curbengh August 11, 2020 12:47

curbengh requested changes Aug 12, 2020

View reviewed changes

test/scripts/hexo/post.js Outdated Show resolved Hide resolved

fix(hexojs#4460): refactor post escape

75acca6

SukkaW force-pushed the fix-4460 branch from 5605c14 to 75acca6 Compare August 13, 2020 06:47

SukkaW requested a review from curbengh August 13, 2020 06:50

curbengh approved these changes Aug 14, 2020

View reviewed changes

SukkaW merged commit 463025c into hexojs:master Aug 14, 2020

curbengh mentioned this pull request Aug 14, 2020

使用prism渲染出现的Bugs #4460

Closed

5 tasks

SukkaW mentioned this pull request Aug 14, 2020

perf(backtick_code): avoid duplicated escaping #4478

Merged

2 tasks

ChrAlpha mentioned this pull request Aug 18, 2020

unespected hexoPostRenderEscape appeared #4481

Closed

5 tasks

curbengh mentioned this pull request Aug 21, 2020

release: 5.1.0 #4487

Merged

2 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix(#4460): refactor post escape #4472

fix(#4460): refactor post escape #4472

SukkaW commented Aug 11, 2020 •

edited

Loading

coveralls commented Aug 11, 2020 •

edited

Loading

SukkaW commented Aug 11, 2020 •

edited

Loading

stevenjoezhang commented Aug 11, 2020

curbengh commented Aug 12, 2020

jerryc127 commented Aug 12, 2020 •

edited

Loading

curbengh commented Aug 13, 2020

jerryc127 commented Aug 13, 2020

SukkaW commented Aug 13, 2020 •

edited

Loading

SukkaW commented Aug 13, 2020

curbengh commented Aug 13, 2020

jerryc127 commented Aug 13, 2020

SukkaW commented Aug 13, 2020

curbengh left a comment

fix(#4460): refactor post escape #4472

fix(#4460): refactor post escape #4472

Conversation

SukkaW commented Aug 11, 2020 • edited Loading

What does it do?

How to test

Screenshots

Pull request tasks

coveralls commented Aug 11, 2020 • edited Loading

SukkaW commented Aug 11, 2020 • edited Loading

stevenjoezhang commented Aug 11, 2020

curbengh commented Aug 12, 2020

jerryc127 commented Aug 12, 2020 • edited Loading

curbengh commented Aug 13, 2020

jerryc127 commented Aug 13, 2020

SukkaW commented Aug 13, 2020 • edited Loading

SukkaW commented Aug 13, 2020

curbengh commented Aug 13, 2020

jerryc127 commented Aug 13, 2020

SukkaW commented Aug 13, 2020

curbengh left a comment

Choose a reason for hiding this comment

SukkaW commented Aug 11, 2020 •

edited

Loading

coveralls commented Aug 11, 2020 •

edited

Loading

SukkaW commented Aug 11, 2020 •

edited

Loading

jerryc127 commented Aug 12, 2020 •

edited

Loading

SukkaW commented Aug 13, 2020 •

edited

Loading