Make packaging JSON asynchronous

[wbamberg]

See #14 (comment): currently file IO is synchronous, but we should make it all asynch.

[peterbe]

I think this might be overkill. Sorry JS, but writing "promising" async/await code is more complicated and the speed won't matter since at best we're doing a little bit of disk IO.

[wbamberg]

OK, let's close this one then.

[peterbe]

@davidflanagan Or, what do you think?

For context; stumptown-experiment is primarily about structured content. An organized bunch of .md and .yaml files. But we've also started adding some node scripts. Basically, they iterate over the files on disk and apply various transformations functions such as turning markdown into HTML. The primary "output" is that it creates directories and in those directories we create .json files. Right now we're doing a bunch of sync fs operations such as fs.existsSync(elementPath).

Suppose we switch to async functions and replace these fs.existsSync() or fs.mkdirSync(destDir, { recursive: true }); with await fs.existsSync(), would it be worth it? Or is my comment above valid?

Async is usually (almost) a must if you have a bunch of tasks that need to be done and some of those tasks might get "stuck" on slow network I/O. But here it's all file system I/O.

Also, the general intention here is that we run these ops in a build step. Like once a day or something like that. Perhaps more often when we figure out a strategy to rebuild after deployment, from like a cron job or something.

Don't spend too much time on it but it would be nice to hear some thoughts from a JS-thought-leader like yourself.

[davidflanagan]

The way I see it, using async is mandatory for server code where you want to be able to handle more than one request at a time. But for scripts that are doing filesystem stuff, I would stick with the synchronous calls. It will make the code easier to write and to read.

If you've got a plan to parallelize your build process and making it blazing fast is a priority, then maybe it would be a good idea to start using async IO from the very start. But if it was me, I think I'd stick with simple synchronous operations.

[peterbe]

And I supposed that even if you have 99% disk I/O if you have that 1% that depends on network I/O you can make those async away. E.g. fetch(url).then(r => {...})