Optimizing refresh rate on 7.8'' Waveshare e-ink

[javierdepascual]

Hello and thank you to the people involved in this project. I found this looking for a cheap substitute of the Freewrite product line. I had a Raspberry 3B+ lying around, no experience in python or linux, but I felt building a portable, usable word processor for writing drafts and brain dumping would really make a difference to me.

Anyway, yesterday I spent the whole day trying to figure everything out. I got to make PaperTTY work after hours of browsing on past issues and documentation here. It really is simpler than it seems but I feel the key points of the documentation are scattered around. I know you guys are pretty aware of this and I hope by the time I finish my project I can make suggestions or even a tutorial for people at my level (newb).

My current problem is that the refresh rate for my display (https://www.waveshare.com/wiki/7.8inch_e-Paper_HAT) is supposed to be 350ms but right now it's very inconsistent, which got me worried about how manageable the whole thing can be from a usability standpoint. After writing a few sentences it can go from very responsive to more than a couple of seconds of delay. Just to be thorough: I cranked the max refresh rate parameter in the driver to 20Mhz (apparently in the Waveshare documentation it says the maximum is 24Mhz, but I saw on another issue that it might make the Pi crash) and tried running the Virtual Terminal after reboot to see if it's a Raspberry Pi performance issue. Since I have a LCD monitor mirroring the VT and a Full Raspberry OS I was also thinking it might be a performance issue of the system itself, since I feel the fluctuation of the refresh rate might indicate that.

I'm trying to attack this problem from every angle: I saw on reddit there was an user that got a workable word processor from a Waveshare 5.5'' without using PaperTTY, just the Waveshare drivers, and it's much more responsive than mine (with a similar refresh rate). I also saw there's similar projects that could work better than this one. In both cases I feel like I'm missing something to make mine work. So, I just want you guys to help me answering a couple of questions:

1. Is there a way to force B/W mode in terminal for a faster refresh rate? (I saw this popping up on other issues but didn't find how to make it work on terminal)
2. Does font choice matter when considering refresh rate? (Right now I'm using Courier Prime)
3. I barely understand the "working terminal on e-ink on startup" part of the readme. I want to try it out to see if having just the e-ink connected and terminal mode on startup might make it work faster. Am I wrong assuming that? If it's worth trying, please give me any additional guidance for a total newb on this system.
4. Would switching to a Raspbian OS Lite improve the refresh rate significantly? I saw the microSD storage can make a difference but mine is A1, so from what I've read performance should be as good as it can be.
5. How long it's been since you guys checked this or other similar projects? Should I try my luck elsewhere?

Thank you to anyone that can help me. I hope I can get something usable out of this to share my findings and help other people that might want to make this work without any prior knowledge.

[mcarr823]

I'm not one of the maintainers of this project, but:
2. font choice shouldn't matter hugely. Font size, on the other hand, definitely does. Smaller font size = fewer pixels to draw = faster.
3. I'm not sure which bit of the documentation you're referring to. Are you currently running in terminal mode or VNC mode? Terminal mode should be faster.

If you are using terminal mode, you could take a look at my fork of papertty.
I've been working on performance improvements for terminal mode on this branch here: https://github.com/mcarr823/PaperTTY/tree/single-line-refresh
There's more work and testing to do (I'll create a pull request here when it's done), but it's already significantly faster for single-line changes.
Full screen updates (eg. scrolling a document) are still slow though. I'm not sure what can really be done about that.

[javierdepascual]

Here's a very low res version of the test I did yesterday, which already was a lot better than when I wrote this issue, but I'm really interested in your fork @mcarr823.

Hardware: RPI 3B+ with A1 microSD card and Full Raspberry OS
Screen: https://www.waveshare.com/wiki/7.8inch_e-Paper_HAT

Changes that I've done:

1. Changed the font from CourierPrime to Courier New (it actually made a difference, I guess because it's a thinner front and therefore takes less time to update.
2. Changed VCOM from 2000 to 1560, which matches the indications of my Waveshare 7,8'' screen
3. Changed Max refresh rate from 1MHz to 18MHz as another user indicated.

Yes, I'm using PaperTTY to run terminal. The idea is to make a very basic word processor for writing drafts without thinking too much about editing, like the Freewrite line of products.

Here's a bigger version of the video on reddit with more info about the project, in case you're interested.

I really want to try your build @mcarr823 but I don't have any prior knowledge of python, Linux, RPi and I just dabbled installing software from my MacOS Terminal. If I could have a 10/20% improvement from what you can see in this video it would be a world of difference to me!

IMG_5296.mp4

[mcarr823]

@javierdepascual Ah, good point on the font. I didn't think about font thinness/thickness. Yeah, it makes sense that a thinner font would perform better.

The screen you're using is 1872×1404, which is the same as the 10" waveshare screen I've been testing with.
I'd say that my fork should be notably faster while typing. Moving the text up (on Enter) is harder to say, since that appears to shift the entire paragraph up in the word processor you're using. Moving the text up is the same as changing it, as far as the screen is concerned, which means redrawing and is thus bad for performance.
Being able to keep previous lines in place (if wordgrinder has a setting for that) might make a notable difference).

The main difference in that branch on my fork is that I've changed the partial refresh implementation.
I'm planning to do a full write-up when I finish development of that branch and submit a pull request, but basically I found that terminal mode is being slowed down by the way that comparisons are being done.
Currently papertty compares the entire screen from last draw to the entire screen this draw, then only writes the parts that changed.
Doing a full-screen image comparison of a screen with lots of pixels (1872x1404) on a low-powered device (rpi) is slow.
My fork swaps that for doing text comparisons instead of image comparisons, which can make a big difference depending on the circumstances.

If you did want to test it out, you can just copy papertty.py from my fork
https://github.com/mcarr823/PaperTTY/blob/single-line-refresh/papertty/papertty.py
and put it where you currently have papertty installed (after backing up your current papertty.py file, just to be safe).
eg. on my rpi, having installed papertty through pip, it's installed in
~/papertty_venv/lib/python3.9/site-packages/papertty/papertty.py

*Note that you might need to grab
https://github.com/mcarr823/PaperTTY/blob/single-line-refresh/papertty/drivers/drivers_color.py
as well. The git version of this file appears to be ahead of the pip/poetry version.

[javierdepascual]

Thank you so much @mcarr823 for keeping this project alive. I think there's a lot of writers like me that would love to have a single-purpose device without spending a fortune and this is definitely the way to do it. when do you think you'll finish development? I'm gonna test it today after backing it up and will post my results. I'll also make my font smaller to keep optimizing.

I'm also gonna search for alternatives to wordgrinder. Right now I'm feeling like I'm wasting half of my screen real state just because the way it behaves, and maybe there's software that behaves the way you're describing.

[mcarr823]

@javierdepascual It depends on when I stop finding things to try to improve lol.
At the moment I'm part way through implementing two more changes.
The first will improve the speed of redrawing non-sequential lines (eg. if you moved your cursor from line 1 to line 20 with a shortcut).
The second is to only draw the part of a line which has changed (currently it redraws the whole line).
After that I plan on testing different combinations of flags (--flipy, --flipx, --portrait) and running some benchmarks, then creating a pull request.

The first is mostly done and the second shouldn't take long. I'm hoping to finish it off this weekend.

Edit: regarding writing software, you could try using nano. It's not specifically intended for writing. It's just a simple terminal-based text editor. It'll still slow down when you reach the end of the screen and the whole page has to move up. I'm not sure how best to handle that.

[javierdepascual]

Thank you! Excited about those changes. I'll let you know how it works at the moment when I test it and I'll try nano in the meantime.

[joukos]

Thanks for the improvements and discussion. FWIW, with some minor tweaks my 6" HD one on a Pi4(00) has been in a "pixels per second" way probably the fastest configuration so far of the displays I have. A couple of vids can be found via here #81 (comment)

With a beefier RPi the bottleneck is just pushing the data out and there's a lot of potential to improve that. This may be of interest: #36 (comment)

[mcarr823]

I've done a bit more work on my fork on that branch.
I haven't figured out rotation yet, so --portrait is required for the time being.
I did do some work to make wordgrinder and similar applications faster though.

The work referenced in issue #36 looks interesting @joukos
I'll have to try that out.

[javierdepascual]

Hi @mcarr823, unfortunately while testing your branch I'm not able to make it work, but I get the following error:

Started displaying /dev/vcsa1, minimum update interval 0.1 s, exit with Ctrl-C
Traceback (most recent call last):
File "/home/javierdepascual/papertty_venv/bin/papertty", line 8, in
sys.exit(cli())
File "/home/javierdepascual/papertty_venv/lib/python3.9/site-packages/click/core.py", line 829, in call
return self.main(*args, **kwargs)
File "/home/javierdepascual/papertty_venv/lib/python3.9/site-packages/click/core.py", line 782, in main
rv = self.invoke(ctx)
File "/home/javierdepascual/papertty_venv/lib/python3.9/site-packages/click/core.py", line 1259, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/javierdepascual/papertty_venv/lib/python3.9/site-packages/click/core.py", line 1066, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/home/javierdepascual/papertty_venv/lib/python3.9/site-packages/click/core.py", line 610, in invoke
return callback(*args, **kwargs)
File "/home/javierdepascual/papertty_venv/lib/python3.9/site-packages/click/decorators.py", line 33, in new_func
return f(get_current_context().obj, *args, **kwargs)
File "/home/javierdepascual/papertty_venv/lib/python3.9/site-packages/papertty/papertty.py", line 1271, in terminal
oldimage = ptty.showtext(buff, fill=ptty.black, cursor=cursor if not nocursor else None,
File "/home/javierdepascual/papertty_venv/lib/python3.9/site-packages/papertty/papertty.py", line 410, in showtext
image = self.showtext_line_by_line(text=text, fill=fill, cursor=cursor, portrait=portrait, flipx=flipx, flipy=flipy, oldimage=oldimage, oldtext=oldtext, oldcursor=oldcursor)
File "/home/javierdepascual/papertty_venv/lib/python3.9/site-packages/papertty/papertty.py", line 499, in showtext_line_by_line
for i in range(self.rows):
TypeError: 'str' object cannot be interpreted as an integer

Do you know why this could be happening? thank you!

[mcarr823]

@javierdepascual Based on the error message, are you perhaps passing in a string as the number of rows when running papertty?
eg. passing in --rows "50" instead of --rows 50?
If not, could you paste the command you're using to start papertty?

Edit: I've done a bit more work on this today, so you should grab the updated version if you're using my fork.
Landscape mode works now (except with --flipy) and other minor fixes/speedups have been added.
Non-monospace fonts don't work currently.
Once I get landscape with --flipy working and maybe non-monospace fonts I'll do some cleanup and submit a PR.

[javierdepascual]

@mcarr823 this is the command I'm using

sudo papertty_venv/bin/papertty --driver it8951 terminal --font /home/javierdepascual/.fonts/couriernew.ttf --size 49 --rows 31 --cols 62 --spacing 3 --portrait

Specifically I get this error whenever I try to access the VM through my terminal. I'm using Raspbian OS, I have a terminal window open with the virtual environment to launch the command, and from there I press CTRL + ALT + F1 to switch to the Virtual Machine Window so I can write into the e-ink screen. I don't know why, but there's no error before I press CTRL + ALT + F1.

Also to clarify I used --flipy and it didn't go to landscape mode on my e-ink model. I used your most recent build and still get the same error.

[mcarr823]

Thanks @javierdepascual it looks like the problem is that manually specifying the rows/columns (as opposed to using --autofit) automatically changes the type from string to int. Papertty then casts it to an int every time it's used.
I've updated my fork to resolve this.

I'm not sure I understand what you mean about --flipy. If you want to change to landscape you should omit --portrait.
--flipy flips the text upside down. You would use it if you wanted to change from portrait to reverse-portrait, or landscape to reverse-landscape.

Edit: I've just pushed a couple more commits to handle odd font sizes and spacing correctly.

Edit2: I've just created a new branch https://github.com/mcarr823/PaperTTY/tree/A2-mode
This is a separate change (to drivers/driver_it8951.py instead of papertty.py) to introduce A2 mode and use that instead of DU mode.
Per https://www.waveshare.com/w/upload/c/c4/E-paper-mode-declaration.pdf
DU mode has a typical update time of 260ms.
A2 mode has a typical update time of 120ms.
Could you try that out as well? I have the 10.3" model and it does seem faster, but it would be good to get confirmation before I create a pull request.

[javierdepascual]

So... just tested it right now. What I did and the problems I'm having:

1. I updated the it8951 drivers with your version @mcarr823.
2. I also updated papertty.py and the drivers_color.py files with the most recent ones from your fork.
3. I first tried the same command I was using until now. It didn't give me any errors but full refresh is not working properly (i.e. I go from terminal to wordgrinder and even though wordgrinder should be appearing on the e-ink screen, it stays on the terminal indefinitely. There's no error message and when I write on terminal the partial refresh, I'm guessing, works properly).
4. I tried the command: sudo papertty_venv/bin/papertty --driver it8951 terminal --font /home/javierdepascual/.fonts/couriernew.ttf --size 49 --autofit. There I can tell it works perfectly. It does seem noticeably faster and more consistent. Only issue is when it has to update multiple line breaks in a short period of time, it lags for a bit then updates. I guess this is impossible to improve, as I feel partial/one-character-at-a-time updates are working really well right now.

EDIT: I just uploaded a 3min test to my Google Drive to show you how it looks: https://drive.google.com/file/d/1buWPD53bs6rYyHf8MFdLnijClFj0M0Am/view?usp=sharing

So @mcarr823 the only issue for me would be to use it in landscape mode. In a previous comment you told me I should omit --portrait for landscape mode. What should I use then? i feel that in order to properly use this I would need to switch to landscape mode, as in portrait line breaks happen way too often, therefore slowing down the overall refresh rate considerably.

In any case, tremendous work and definitely an improvement over the original drivers. I'm really excited to get the batteries, case and keyboard going and finishing the build to start typing away!

[javierdepascual]

Ok, I just tested it using --autofit and --portrait and it works almost perfectly on landscape mode. The only issue happens occasionally when doing anything that requires a full refresh (i.e. when going from terminal to wordgrinder or viceversa, as I previously said). It slows down a lot and even freezes randomly, something that doesn't happen when in non-portrait mode, I don't know why. For the most part it works, even though it doesn't seem as consistently faster as the non-portrait mode.

[mcarr823]

Thanks for the feedback @javierdepascual
Based on the video I'm not sure how much can be done to improve the speed in that particular scenario.
There are two main issues affecting the speed there:

1. the word count. It's down the bottom of the screen, separate from the rest of the text, being updated independently. So this incurs a second write every time you type a word.
2. the way that text moves up the screen whenever you type, so papertty has to redraw every line (since all of them moved) instead of just 1-2 lines.

Removing the word count would speed things up a bit.
Or alternatively if you wanted to keep the word count you could change maxRedraw from 2 to 1 in papertty.py, which would make it behave more like the old partial redraw method.
Either way, what's shown in your video is unfortunately pretty much the worst-case scenario performance-wise.
The text moves up every time you do a line break, forcing most of the screen to redraw.
The word count is right down the bottom, forcing either a second draw (maxRedraw = 2) or a full-screen redraw (maxRedraw = 1)

Landscape mode works for me except with --flipy.
I'll see what I can do about that. Should just be a matter of figuring out the math.
I think something could also be done about some full-screen refresh scenarios where it's actually like 90% of the screen that's updated, not 100%... but I'll need to experiment a bit.

Anyway, I've taken a couple of videos below to show how it's working for me.
I don't know how easy it is to tell the speed difference from a recording though.
I think it's more noticeable when you're typing, since the delay between pressing a key and having it appear is immediately obvious.

Still, you should be able to get a general idea of the speed improvement by 2 things.

1. how often it refreshes. This would be an indication of if A2 mode is faster. But we're talking about 260ms vs 120ms, and it's not something you can measure programmatically, so it's hard to say definitively if it's faster.
2. how many letters appear each refresh. More letters appearing in one go means more text was typed within that refresh interval, which would indicate that papertty was going slower. ie. Lag in the papertty execution means more time for text to pile up before being displayed.

I probably shouldn't have applied both patches. That makes it harder to say what's improved where.
But whatever. Here are recordings with a pi4b
Old https://drive.google.com/file/d/1mHleiWokLkb-eop_jjuYOvqOvbQfu2e7/view?usp=sharing
New https://drive.google.com/file/d/1tBOjr2taI2XSEHq2MABObOBzDBsC4MXn/view?usp=sharing

And here with a pi2b
Old https://drive.google.com/file/d/139lAcfd4u5hPsaKVEtviAIO5buY2d70H/view?usp=sharing
New https://drive.google.com/file/d/1hEdQ0nyNG8LC8vAh6zPPiV2CgT_Bzgq9/view?usp=sharing
(In the video I say 800ms -> 200ms for the 2b on a full screen, but it's more like 1.2s -> 250ms)

*To clarify, when I mention speeds in the video (200ms, 8ms, etc.) I'm not including the panel refresh rate or the IT8951 board speed, since I can't measure those. I'm only referring to the speed of papertty's logic and the it_8951 driver.
So when I say the speed has gone from 200ms to 8ms, that actually means something like 570ms to 378ms. (120ms for A2 mode on the IT8951 board, 250ms for panel refresh, 200-8ms for papertty itself).
My single-line-refresh branch is all about improving the speed of papertty itself and minimizing its overhead, so as much as possible the speed is only bottlenecked by the panel's refresh rate instead of papertty.
Because of that, any improvements are much more noticeable on slower devices, like the rpi2b or pi zero, where the CPU of the device is the limiting factor. Or (hypothetically) on faster panels, since the panel speed would make up less of the overall delay time.

[javierdepascual]

Thank you for this detailed account of your progress @mcarr823. Right now and as per your suggestion, I'm gonna try out vim, since it has a bunch of plugins that can turn it into a very capable markdown processor, and since it's so customizable, I feel I might be able to optimize it for this type of e-ink screen (i.e. instead of scrolling down whenever I write beyond the bottom limit of the screen, maybe turning a "page" and going back to a blank screen). Do you think that might make it work faster on these type of displays?

On another note, something that worries me is the fact that when using --portrait --autofit to make it work on landscape mode it becomes noticeably slower, and it almost freezes when going in/out of terminal and/or the word processor. Is this an issue you're aware of when working in landscape mode? Do you need me to test it out so you can see it and update the fork?

I'll keep sharing my tests if they're helpful. Let me know if you have any other suggestions or if you need anything else to be tested. I'm really greatful by everyone's work and I hope this project stays alive for the sake of all of us, distracted writers in the digital era lol.

[mcarr823]

@javierdepascual Turning a page would certainly be faster.
Firstly because it's faster to draw a blank page than it is to draw a page of text shifted up a few rows.
Secondly because a blank page means it'll take longer until your next page turn.
Anything involving moving elements around the screen (scrolling text) will be noticeably slower than simply drawing.

Could you take a video of the freezing which occurs when entering/exiting a word processor?
I would expect it to be slow for a second, since the entire screen is being updated as you enter/exit a program, but I can't say that I've seen any slowdown with landscape mode or had any freezing.
The closest thing I can think of is when exiting wordgrinder the screen doesn't clear. But that appears to happen even when not using my fork, so it appears to be unrelated.
Could you temprorarily undo any changes you've made to max_speed_hz in driver_it8951.py, if you've made any, and see if that causes any change in regards to the freezing? Just to make sure it's not something unrelated to the changes in the fork, since changing max_speed_hz can potentially cause freezing (based on other comments on this repo).

Thank you for your testing, it's all a big help. I only have one panel to test with, and the way I use it will be different than how you and other people use theirs, so any input should help to make the program better for everyone.

[javierdepascual]

Hi @mcarr823, I'll test it by monday as I'm travelling this week. In any case, I don't think I've changed max_speed_hz. The value I have right now is 200000. Let me know which one would be safer and I'll test both. i also changed VCOM from 2000 to 1560, as Waveshare documentation told me I should change that value as indicated by the screen label.

[mcarr823]

I've made a few more changes.
The first two are 1bpp mode and A2 refresh support. They've been submitted as a PR.
The single-line-refresh branch has been rebased on that one, and I've also added in support for performing multiple writes in a single refresh.

I haven't taken a video, but the short version is that the single-line-refresh branch is substantially faster for the scenarios it was bad at before. Things like opening a text editor or using an application like wordgrinder which updates multiple parts of the screen simultaneously are now significantly faster.

[javierdepascual]

Hi @mcarr823. Sorry it took me so long to give it another try, but I did try today the most recent branch and it works wonders. Definitely comparable to the Freewrite and similar e-ink devices. Right now I'm struggling to set up terminal at boot. Will create a separate issue for that matter. Thank you so much for everything.

[mcarr823]

Thanks for the feedback @javierdepascual
I'll try to get that branch into a state where it can handle other orientations (that's where I'm currently stuck...) and then clean up the code a bit and submit a PR when it's done.

[javierdepascual]

@mcarr823 I have a question about fonts and the tty limitations: since I'm using vim as my text editor I was wondering if the Virtual Console/tty is able to display underlined characters, italics and bold. Since I'm using a couple of plugins for vim, ideally I'd like to use them sparingly for certain purposes. Thing is right now VIM doesn't display them, and I feel it's a limitation of the environment. I'm also wondering if maybe I could load the tty with a font that includes different weights.

[mcarr823]

@javierdepascual I'm not really sure.
Papertty grabs a text buffer from the vcs. As far as I'm aware the vcs only gives raw text, not font information.
If that's the case, then I don't think it would be possible unless the text was retrieved from somewhere other than /dev/vca*.

[gw]

@javierdepascual were you ever able to work through the issues you were having in getting a terminal at boot?

I'm grateful for all your work here—I also will be trying @mcarr823 's branch on a Waveshare 7.8" screen. Will let you know how it goes!