Convert Micropython to JavaScript in order to create an online simulator #792

kisvegabor · 2019-02-02T17:17:46Z

In continue of #687

Discuss how to concert micropython to javascript to have an online simulator where people can easily try LittlevGL in their browser without downloading anything.

Summarizing what's left before release:

Standard import support for scripts (waiting on py: handle builtins.__import__ micropython/micropython#4929)
Finalize the method(s) for sharing scripts
Potentially improve frame rates
Ability to open files (images, fonts, etc)

Later (once the initial version is up on https://littlevgl.com):

Ability to resize the canvas (up to a set maximum)
Possibly more

The text was updated successfully, but these errors were encountered:

embeddedt · 2019-02-02T22:49:40Z

@amirgon That port of Micropython builds properly but I ran into errors while trying to run it. I think that Emscripten has evolved since it was made.

amirgon · 2019-02-02T23:10:50Z

@embeddedt Did you see micropython-ports-wasm?
It looks rather new and active (initial commit 3 days ago), @pmp-p claims he got Micropython working in WASM.

embeddedt · 2019-02-03T00:27:02Z

@amirgon That's working out a lot better. Thanks! I'll let you know about my progress.

stale · 2019-02-24T01:11:47Z

This issue or pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

embeddedt · 2019-02-24T03:46:04Z

I haven't had a lot of time to work on this lately, but I hope to get to it at some point.

kisvegabor · 2019-02-24T11:39:18Z

Shall we "pin" this issue to leave it opened as a reminder?

embeddedt · 2019-02-24T15:21:24Z

@kisvegabor Sure.

amirgon · 2019-03-13T13:35:32Z

micropython/micropython#3575
Micropython now officially has an Emscripten port! micropython/micropython@7d675f3

lvgl+micropython could make a nice interactive lvgl demo on lvgl web page.

embeddedt · 2019-03-13T14:04:29Z

@amirgon I'll see what I can do, but I have limited time right now.

kisvegabor · 2019-03-13T14:12:38Z

@amirgon Awesome! It should be much easier with this.

kisvegabor · 2019-04-14T13:37:44Z

@embeddedt Did you have a chance to make some experiments with Emscripten and Micropython?

embeddedt · 2019-04-14T21:44:01Z

@kisvegabor I tried to work with it but I found that I don't understand Micropython well enough to be able to make it work with LittlevGL.

I haven't had time to look at it again lately.

kisvegabor · 2019-04-15T13:48:49Z

Would you like to warm it up? Probably @amirgon could help to understand the Mpy specific parts.
Or shall we postpone it? (It could be a great thing though but I understand if you have more important things now)

embeddedt · 2019-04-15T13:50:16Z

I will try to experiment with it again this week.

kisvegabor · 2019-04-15T13:53:48Z

Awesome!

embeddedt · 2019-04-19T22:49:53Z

@amirgon I made some more progress on this, but it looks like Emscripten doesn't support SDL threads properly at the moment... could you figure out if the binding can be modified to avoid the need for another thread?

amirgon · 2019-04-20T20:34:16Z

@embeddedt The micropython binding was designed to be single threaded. You can run everything (micropython+lvgl+display driver) on the same thread.
I don't think you should use SDL at all. You probably need a special display driver (and input device driver) for Emscripten.
How does Emscripten write to the screen and read mouse input? If you want I can help creating the drivers for it.

embeddedt · 2019-04-20T20:42:10Z

@amirgon Emscripten uses SDL2 for graphics.

The micropython binding was designed to be single threaded. You can run everything (micropython+lvgl+display driver) on the same thread.

Is SDL_CreateThread supposed to be called here then?

https://github.com/littlevgl/lv_binding_micropython/blob/da8588210df7e34644f94fe3bf605d70cb01058a/driver/SDL/modSDL.c#L32

amirgon · 2019-04-20T20:56:51Z

Is SDL_CreateThread supposed to be called here then?

@embeddedt The only purpose of the SDL thread over there is to call lv_tick_inc and lv_task_handler indirectly, using mp_sched_schedule.
This will schedule lv_task_handler to run next time micropython is ready, on the same thread micropython is running.
If you have other ways to schedule things in Emscripten (something like javascript's setTimeout), that would be good enough instead of SDL thread for calling these functions periodically.

embeddedt · 2019-04-20T21:01:03Z

@amirgon Okay, I'll see what I can do.

amirgon · 2019-04-20T21:06:49Z

@embeddedt How do you call SDL main loop (monitor_sdl_refr_core) if you don't use SDL threads?

embeddedt · 2019-04-20T22:28:35Z

@amirgon I'm now using an Emscripten callback function to call the lv_tick_inc, lv_task_handler, and monitor_sdl_refr_core functions. I no longer use mp_sched_schedule.

Of course, I don't fully understand the Micropython architecture yet, so it's possible this will all fall apart shortly. 😄

amirgon · 2019-04-20T22:37:07Z

I no longer use mp_sched_schedule

@embeddedt mp_sched_schedule ensures that a function is called when micropython is ready, and not in the middle of the execution of some micropython statement.
Depending on how you schedule micropython main loop and how you schedule lv_task_handler, this might be ok (or not).

embeddedt · 2019-04-20T22:42:18Z

I got it to work with mp_sched_schedule. Now I have to figure out how to get the REPL working so that people can put in their own code.

amirgon · 2019-04-20T22:47:47Z

Nice progress @embeddedt ! 👍

embeddedt · 2019-12-03T15:02:24Z

It seems to work for me now as well; I think the problem is fixed.

kisvegabor · 2019-12-06T13:59:28Z

@amirgon @embeddedt
What is still missing to release the simulator? The "inti script" feature is still not added, right?

embeddedt · 2019-12-06T15:08:46Z

I need to update the simulator to work with the latest version of lv_micropython. Currently it's using an old snapshot from September/October.

The init script feature should be supported in lvgl/lv_micropython#15.

amirgon · 2019-12-06T20:22:13Z

The init script feature should be supported in littlevgl/lv_micropython#15.

@embeddedt what about lvgl/lv_micropython#17? I think it's important to include this as well.

Did you get a chance to make some progress with other features we discussed?

stepansnigirev · 2019-12-06T20:41:42Z

@amirgon, regarding lvgl/lv_micropython#17
I am traveling at the moment and unfortunately can't work on this pull request until ~mid next week.

amirgon · 2019-12-06T20:44:15Z

@amirgon, regarding littlevgl/lv_micropython#17
I am traveling at the moment and unfortunately can't work on this pull request until ~mid next week.

No problem @stepansnigirev, we can add this later.

embeddedt · 2019-12-06T21:12:22Z

Did you get a chance to make some progress with other features we discussed?

I haven't had a chance to work on them yet.

kisvegabor · 2019-12-07T06:07:49Z

@embeddedt
Are there any difficulties with updating the project or is it just a simple submodule update?

@amirgon @embeddedt
It'd be great to add a blog post and an Announcement on the forum about the simulator. You know it much better than me, so can you help with that?

I'll write to the hackaday too. They were already interested in the Micropythin binding once.

barbiani · 2019-12-07T06:11:22Z

You guys ever heard of moddable.com? Maybe emscripten?

…

On Sat, Dec 7, 2019, 04:07 Gabor Kiss-Vamosi ***@***.***> wrote: @embeddedt <https://github.com/embeddedt> Are there any difficulties with updating the project or is it just a simple submodule update? @amirgon <https://github.com/amirgon> @embeddedt <https://github.com/embeddedt> It'd be great to add a blog post and an Announcement on the forum about the simulator. You know it much better than me, so can you help with that? I'll write to the hackaday too. They were already interested in the Micropythin binding once. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#792>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABYLTJF5DINM475NNB5G2DLQXM4TVANCNFSM4GT6XZWA> .

pmp-p · 2019-12-07T07:47:41Z

@barbiani while wasm is now an official standard https://www.w3.org/2019/12/pressrelease-wasm-rec.html.en and emscripten uses clang 10.

The javascript simulator actually uses the (soon to be deprecated) emscripten fastcomp compiler (clang 8) because the project wants to match what official micropython javascript port may do.

amirgon · 2019-12-07T07:59:04Z

You guys ever heard of moddable.com?

Could be interesting in other context - js binding to lvgl. If anyone is willing to take this effort, because I'm too invested in Micropython for now...

embeddedt · 2019-12-07T13:34:29Z

Are there any difficulties with updating the project or is it just a simple submodule update?

For some reason it doesn't seem to work properly if I just merge master. I haven't figured out what the culprit is, but I'll most likely have it working by the end of the weekend.

Could be interesting in other context - js binding to lvgl.

I thought of this as well, but then I realized that most people using JavaScript with a user interface will be using it in a web browser, where LittlevGL isn't very necessary or practical.

The javascript simulator actually uses the (soon to be deprecated) emscripten fastcomp compiler (clang 8) because the project wants to match what official micropython javascript port may do.

We're trying to remain aligned with upstream MicroPython as much as possible to ease the merging process.

amirgon · 2019-12-07T20:06:32Z

I thought of this as well, but then I realized that most people using JavaScript with a user interface will be using it in a web browser, where LittlevGL isn't very necessary or practical.

@embeddedt I think that JavaScript in this context is used like Micopython and Lua on embedded device, where there is no web browser.
It's part of the trend these days to move from C to higher level languages (python, js) on embedded devices. I've seen multiple js implementations for embedded such as low.js, Espruino, jerryscript, duktape and now moddable.
On each of these, JavaScript is used on an embedded device with no web browser, so lvgl binding is applicable just like we did for Micropython.
I don't know how popular JavaScript-on-embedded-devices is going to get, but it is definitely gaining some momentum.

barbiani · 2019-12-07T21:39:39Z

Moddable js is great and fast as it runs precompiled bytecode. There is also a preliminary web simulator built by another user.

…

On Sat, Dec 7, 2019, 18:06 Amir Gonnen ***@***.***> wrote: I thought of this as well, but then I realized that most people using JavaScript with a user interface will be using it in a web browser, where LittlevGL isn't very necessary or practical. @embeddedt <https://github.com/embeddedt> I think that JavaScript in this context is used like Micopython and Lua on *embedded device*, where there is *no web browser*. It's part of the trend these days to move from C to higher level languages (python, js) on embedded devices. I've seen multiple js implementations for embedded such as low.js <https://github.com/neonious/lowjs>, Espruino <https://www.espruino.com/>, and now moddable. On each of these, JavaScript is used on an embedded device with no web browser, so lvgl binding is applicable just like we did for Micropython. I don't know how popular JavaScript-on-embedded-devices is going to get, but it definitely gaining some momentum. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#792>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABYLTJFMS34BP57DZYLGUJTQXP64TANCNFSM4GT6XZWA> .

embeddedt · 2019-12-08T13:57:03Z

QuickJS is another small and fast implementation of JS. Someone was able to make React run on a Raspberry Pi using it. I think it would fit for sure on an STM32F7 and probably even an F4.

I'd be willing to work on making a JS binding for LittlevGL if there was substantial interest. The main problem I see is that there are so many different implementations (JerryScript, QuickJS, etc.) which don't use the same API for bindings. I suppose using a script like what you use for MicroPython would help with that.

I don't know how popular JavaScript-on-embedded-devices is going to get, but it is definitely gaining some momentum.

It's useful because it enables IoT developers to write the same type of code for both the browser interface and the device backend.

amirgon · 2019-12-08T14:56:13Z

QuickJS is another small and fast implementation of JS. Someone was able to make React run on a Raspberry Pi using it. I think it would fit for sure on an STM32F7 and probably even an F4.

Not sure about "fast". It's fully interpreted, as opposed to others which run bytecode.
It's not small. See this. The core is 700KB and he needed another 10MB of "auxiliary environment" to make it work...
He is running it on a Linux OS on a Raspberry Pi. If you call that an "embedded device" this is really the very high end.

I'd be willing to work on making a JS binding for LittlevGL if there was substantial interest. The main problem I see is that there are so many different implementations (JerryScript, QuickJS, etc.) which don't use the same API for bindings.

If you are thinking about this, I suggest taking a close look at the other options. Which devices are supported? ROM/RAM requirements? Performance?

One great thing about Micropython is that it can run in a variety of ways:

Parse and run micropython source code on the device (REPL)
Run precompiled bytecode
Compile Micropython to native instructions (native code emitter)
Compile Micropython to non compliant but very fast native instructions (viper code emitter)

I wonder if any of the Javascript engines above expose such range of options.

I suppose using a script like what you use for MicroPython would help with that.

Probably, but I'm not sure how much you could reuse the existing Micropython binding script for that. It's really geared towards Micropython.

embeddedt · 2019-12-08T21:18:10Z

It's fully interpreted, as opposed to others which run bytecode.

I'm pretty sure it can compile to bytecode as well. See page 3 of this PDF.

he needed another 10MB of "auxiliary environment" to make it work...

Much of that "auxiliary environment" is compilation infrastructure which won't be in the final bytecode/script. JavaScript keeps compilers and libraries together in one folder; his measurement includes both.

The core is 700KB

Most of that is probably React (which is fairly large). The author of QuickJS claims that it takes up "190 KiB of x86 code for a simple hello world program."

He is running it on a Linux OS on a Raspberry Pi. If you call that an "embedded device" this is really the very high end.

My point is that it could easily be run on a smaller device, just like how MicroPython runs on Linux but can also run on STM32, ESP32, etc.

I'm not sure how much you could reuse the existing Micropython binding script for that. It's really geared towards Micropython.

Of course. I would have to write a new script that does the same type of thing, but for JavaScript instead.

kisvegabor · 2019-12-09T13:50:04Z

I also see great potential in a JS binding. As Python and JavaScript are the two most popular languages it'd be great to have bindings to them.

BTW, JerryScript also claims that it runs with 64k RAM, 200 kB ROM.

I'm not sure how much you could reuse the existing Micropython binding script for that. It's really geared towards Micropython.

Of course. I would have to write a new script that does the same type of thing, but for JavaScript instead.

At least the parser of the MicroPython can be used, can't it? And we already know lvgl has the conventions to generate a binding.

Next year lvgl will have a new website where the bindings (and the simulator(s)) should be clearly highlighted. Some videos and tutorials also will be created as the bindings and the simulator really makes a difference compared to other gui libraries. Actually I don't know about any serious embedded UI library which has binding to any higher-level languages.

amirgon · 2019-12-09T14:06:46Z

BTW, JerryScript also claims that it runs with 64k RAM, 200 kB ROM.

Selecting the right js engine is a critical decision, since there are so many options.
Imagine you create binding for js engine X but everyone are using js engine Y so they can't use your js binding.
Beside RAM/ROM/performance consideration, another important consideration is the ecosystem and adoption. How much is it used and adopted? Which hardware platforms are supported? What libraries / drivers are available?

At least the parser of the MicroPython can be used, can't it? And we already know lvgl has the conventions to generate a binding.

The parser itself (pycparser) can be used, of course.

amirgon · 2019-12-17T22:51:43Z

We need to take into account lvgl versions on https://sim.littlevgl.com/.
Suppose we move to v7.0 which is not backward compatible - we wouldn't want all the simulator links of v6 scripts to stop working.

So I suggest something like:

sim.littlevgl.com/v6/micropython/ports/javascript/lvgl_editor.html
sim.littlevgl.com/v7/micropython/ports/javascript/lvgl_editor.html

What do you think?

embeddedt · 2019-12-17T22:55:05Z

As long as it's not a requirement to continue updating the "v6" folder once "v7" is released, it shouldn't be any extra work. It just means adding some subdirectories to https://github.com/littlevgl/sim.

I've moved the existing lv_micropython submodule to a new "v6" directory.

amirgon · 2019-12-17T22:59:24Z

I've moved the existing lv_micropython submodule to a new "v6" directory.

Once you deploy it - it will break existing links. (I fixed the one I posted today on lvgl/blog#20)

You would also need to update the html on https://sim.littlevgl.com/, because micropython link is now broken.

embeddedt · 2019-12-18T01:12:09Z

Once you deploy it - it will break existing links.

Good point. I will put back the old micropython in the root directory, but it will be permanently fixed at v6, for backwards compatibility. All changes will only happen in the subdirectories.

amirgon · 2019-12-18T05:59:39Z

How about using a symbolic link to v6 instead?

…

On Wed, Dec 18, 2019, 03:12 embeddedt ***@***.***> wrote: Once you deploy it - it will break existing links. Got it. I will put back the old micropython in the root directory, but it will be permanently fixed at v6, for backwards compatibility. All changes will only happen in the subdirectories. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#792>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACZS3LQ3Q3CPH5VINTBNM3DQZF2GVANCNFSM4GT6XZWA> .

embeddedt · 2019-12-18T14:28:24Z

Yes; that would probably be a lot easier.

embeddedt · 2019-12-19T23:31:43Z

I've added the symlink; it seems to work properly.

kisvegabor · 2019-12-22T07:10:30Z

I've added the symlink; it seems to work properly.

Great!

Is there any major things with the simulator?
I know there is a list with the possible improvements but the current version can be considered a v1.0, right?
If you agree I suggest closing this issue and open a new one with the remaining ideas/things to do.

amirgon · 2019-12-22T07:30:31Z

If you agree I suggest closing this issue and open a new one with the remaining ideas/things to do.

Ok. The simulator is usable, but I think should be polished and styled a little.
For the reference here is a list of improvements we discussed.

embeddedt · 2019-12-22T13:56:29Z

Opened #1320.

kisvegabor mentioned this issue Feb 2, 2019

Emscripten port #687

Closed

stale bot added the stale label Feb 24, 2019

stale bot removed the stale label Feb 24, 2019

embeddedt added the pinned label Feb 24, 2019

amirgon mentioned this issue Mar 5, 2019

Architecture design lvgl/lv_gui_builder#1

Open

embeddedt closed this as completed Dec 22, 2019

Convert Micropython to JavaScript in order to create an online simulator #792

Convert Micropython to JavaScript in order to create an online simulator #792

Comments

kisvegabor commented Feb 2, 2019 • edited by embeddedt Loading

embeddedt commented Feb 2, 2019

amirgon commented Feb 2, 2019

embeddedt commented Feb 3, 2019

stale bot commented Feb 24, 2019

embeddedt commented Feb 24, 2019

kisvegabor commented Feb 24, 2019

embeddedt commented Feb 24, 2019

amirgon commented Mar 13, 2019

embeddedt commented Mar 13, 2019

kisvegabor commented Mar 13, 2019

kisvegabor commented Apr 14, 2019

embeddedt commented Apr 14, 2019 • edited Loading

kisvegabor commented Apr 15, 2019 • edited Loading

embeddedt commented Apr 15, 2019

kisvegabor commented Apr 15, 2019

embeddedt commented Apr 19, 2019

amirgon commented Apr 20, 2019

embeddedt commented Apr 20, 2019

amirgon commented Apr 20, 2019

embeddedt commented Apr 20, 2019

amirgon commented Apr 20, 2019

embeddedt commented Apr 20, 2019 • edited Loading

amirgon commented Apr 20, 2019

embeddedt commented Apr 20, 2019

amirgon commented Apr 20, 2019

embeddedt commented Dec 3, 2019

kisvegabor commented Dec 6, 2019

embeddedt commented Dec 6, 2019

amirgon commented Dec 6, 2019

stepansnigirev commented Dec 6, 2019

amirgon commented Dec 6, 2019

embeddedt commented Dec 6, 2019 • edited Loading

kisvegabor commented Dec 7, 2019

barbiani commented Dec 7, 2019 via email

pmp-p commented Dec 7, 2019 • edited Loading

amirgon commented Dec 7, 2019

embeddedt commented Dec 7, 2019

amirgon commented Dec 7, 2019 • edited Loading

barbiani commented Dec 7, 2019 via email

embeddedt commented Dec 8, 2019

amirgon commented Dec 8, 2019

embeddedt commented Dec 8, 2019

kisvegabor commented Dec 9, 2019

amirgon commented Dec 9, 2019

amirgon commented Dec 17, 2019

embeddedt commented Dec 17, 2019

amirgon commented Dec 17, 2019 • edited Loading

embeddedt commented Dec 18, 2019 • edited Loading

amirgon commented Dec 18, 2019 via email

embeddedt commented Dec 18, 2019

embeddedt commented Dec 19, 2019

kisvegabor commented Dec 22, 2019

amirgon commented Dec 22, 2019

embeddedt commented Dec 22, 2019

kisvegabor commented Feb 2, 2019 •

edited by embeddedt

Loading

embeddedt commented Apr 14, 2019 •

edited

Loading

kisvegabor commented Apr 15, 2019 •

edited

Loading

embeddedt commented Apr 20, 2019 •

edited

Loading

embeddedt commented Dec 6, 2019 •

edited

Loading

pmp-p commented Dec 7, 2019 •

edited

Loading

amirgon commented Dec 7, 2019 •

edited

Loading

amirgon commented Dec 17, 2019 •

edited

Loading

embeddedt commented Dec 18, 2019 •

edited

Loading