Build Proposal/Specification

[kitsonk]

Story

Dojo 2 needs a set of tooling to make building Dojo 2 applications straight forward, as well as an easy way of managing dependencies of Dojo 2 packages.

Features

The proposal should express the following features:

• Ability to easily include dependent packages
• Ability to build from TypeScript source, including all source packages that are written in TypeScript
• Build time concatenation/layering
• Build time minification
• Generation of typings for a package

Considerations

There are several things that should be considered:

• Minification for TypeScript could benefit from a minifier that is integrated into the language services (see: Suggestion: minification microsoft/TypeScript#8)
• Overall dependency management and typings is still up in the air (see: Package typing inclusion and discovery microsoft/TypeScript#5537 and: Create a validate command typings/typings#3), although we have dts-generator this is likely limiting in the future.
• There seems to be more support for typings than DefinitelyTyped going forward.
• GruntJS is our preferred task runner, but are their other considerations?
• Should we support the RequireJS extensions to AMD for build time plugin resolution? (see: Adopt builder plugin resolver API from RequireJS loader#31)

[kitsonk]

@kfranqueiro raised some valid concerns around discovery which should be addressed in any proposal.

@novemberborn also mentioned the need to provide project scaffolding in the tooling.

[tomdye]

Approaches

dojo-cli

A dojo-cli package on npm to be installed globally similar to ember-cli and angular-cli.
Users would install using npm install -g dojo-cli and would run the cli by running dojo <command>.

To create a new app you would simply run dojo new my-app. This would pull down from template files on github and create project folder structures etc...
Grunt would be used under the covers to perform builds / dist / scaffolding etc... and build tools would live inside dojo/build.

Users would have the option to use dojo build, dojo dist etc as shortcuts to the underlying grunt build calls and so on.

This approach would mean that all the user would have to do to create a dojo 2 project and import the required packages is download and install dojo-cli and use it to get started keeping the barriers to entry low.

yeoman

Yeoman provides project scaffolding via the use of custom generators. It provides a cli with a wizard format for selecting options and supplying information when creating your application. Users install via npm install -g yo and run by calling yo <generator-name>.

@bryanforbes started writing a generator for dojo 1x here. This could be a base for creating the dojo 2 generator.

Similar to the proposition for dojo-cli, yo creates grunt tasks under the covers if you ask it to (via the generator) and allows the creator to build complex commands what create routes, controllers etc..
For example, the angular-generator allows you to run yo angular:controller user --typescript which creates a new typescript controller named user and adds the appropriate routes and imports automatically.

If we chose not to go down this route as our official starting point, it may be worth investing time in creating a usable generator anyway that simply instantiates whatever base setup is required to proceed.

grunt-init

grunt-init provides project scaffolding as an extension to grunt. It gets installed globally via npm install -g grunt-init. In order to use this, you must also install appropriate templates. A barrier to entry here is that the user would need to be instructed on where to acquire the templates from. (This could be automated if grunt-init is used under the covers of a dojo-cli.).

We will need to create the templates required. Looks like some members of the community have had a go at creating grunt-init templates for dojo before: dojo-widget & dojo-class.

It is worth noting that both these examples state that they have been superseded by yeoman-generator versions.

dojo/build

Dojo build will contain the tools and resources required to create, build and distribute a dojo 2 project.

• Grunt tasks
• Templates
• Scripts

dojo-build would likely be installed locally within a project as a dev dependency in order to provide a project with the tools it needs.

If we chose not to go ahead with dojo-cli then this package will include the scaffolding cli also. However, this would mean that a user would need to create the project and install dojo/build themselves before having local access to the scripts.

It is intended that dojo-cli will utilise the tools within dojo/build.

[kitsonk]

Some questions.

What do we see the technology/structure of dojo-cli? What platforms are expected to be supported with dojo-cli? What pre-reqs would be required for dojo-cli?

What are the advantages of having dojo-cli versus just embracing something like yeoman or grunt-init? I agree that potentially investing in having decent scaffolding and making it easier for people to integrate into other tooling/build systems, but is there a decent system we can already customise (which seems to be the summary of yeoman)? And if so, why not just embrace it. Is there a reason we need to mark our territory outside of "because we can"?

I find the scoping of dojo/build to make a lot of sense. Essentially a dependency which would contain the right sort of custom build tooling we need and whatever dependencies we need upstream.

@dylans I would also really appreciate your input on this one as well.

[tomdye]

An initial search doesn't shed much light on out of the box technology for a customisable CLI, angular and ember's cli's both appear to be bespoke with the latter using broccoli behind the scenes.

If we DID do a CLI, it would most likely just be a wrapper for a yeoman scaffolding system as there's no point reinventing the wheel and writing all of that structure ourselves. The cli could then provide the build / dist functionality (whilst making calls to grunt under the covers). It would just be sugar really.

I believe that yeoman is superior to grunt-init in it's wizard style offerings and easy access, it would only require the user to install yo and our dojo-generator.

Basically, i'm all for using yeoman as a scaffolding tool and grunt for our other tasks. We could always decide to wrap these in a dojo-cli at a later date if we feel it's important enough.

[novemberborn]

Wrapping yeoman may improve discoverability for those that do not ordinary use yeoman, and reduce friction in terms of how many packages you need to install before you can get started.

[dylans]

In an ideal world, we want to solve both the problem of make it easy, and make it flexible for people to work this into whatever build environment they choose, without creating substantial work for us to maintain.

In my mind, this means creating something simple for new users, such as a dojo-cli wrapper for Yeoman, but then also providing reasonable out of the box help (in the form of tasks or example or whatever makes sense) for people using Grunt and Gulp and perhaps broccoli or other tools.

I think the only thing we know for certain is that whatever is state of the art today with build systems is unlikely to be so at the end of Dojo 2's life, so being flexible is key here. I guess the best analogy I can make is our approach to reporters or functional testing providers with Intern. While not the same, it is that level of choice that will reduce complaints over time, and also keep us honest in our approach to builds.

[tomdye]

I am glad that we appear to be in agreement over this

[kitsonk]

To to summerise:

• dojo-cli will be an globally (ideally) installed package which essentially wraps some pre-configured yoeman generator(s).
• We will make yoeman generator(s) available for those who prefer to just work with yeoman directly
• dojo-build will largely be a grunt based build system
• We will continue to try to leverage npm and bower as package managers (though we need to flesh out our story around bower).

[tomdye]

Building Dojo1 dijits into Dojo2 app

Have been working with the latest app-example to try and create a built version using r.js.

The problem I came across is with Dojo1 plugins. It's unlikely that we will back port the build time resolution back to them, so a work around is to alias the require.js versions on i18n and text during the build.

The only other plugin issue that came up is with dojo/selector/_loader which needs an alias to be written which splits it into a separate module and plugin and calls loaded within the load function properly.

app.build.js example

({
    baseUrl: ".",
    paths: {
        "src": "_build/src",
        "dojo-actions": "node_modules/dojo-actions",
        "dojo-compose": "node_modules/dojo-compose",
        "dojo-core": "node_modules/dojo-core",
        "dojo-widgets": "node_modules/dojo-widgets",
        "immutable": "node_modules/immutable/dist",
        "maquette": "node_modules/maquette/dist",
        "rxjs": "node_modules/@reactivex/rxjs/dist/amd",
        "dojo": "node_modules/dojo",
        "dijit": "node_modules/dijit"
    },
    name: "_build/src/index.js",
    out: "_build/src/index-built.js",
    map: {
        '*': {
            'dojo/i18n': 'legacyDojoShim/i18n',
            'dojo/text': 'legacyDojoShim/text'
        }
    },
    include: [
        'dojo/dojo'
    ]
});

Conclusion

Dont do this at all!
If nesting Dojo1 dijits in Dojo2 it would be better to perform a custom build on the Dojo1 code as a pre-build step and reference that built version in our r.js config file. Not tried it yet, but it should work.

[tomdye]

Update to dojo1 in dojo2 config

({
    paths: {
        "src": ".",
        "dojo-actions": "../../node_modules/dojo-actions",
        "dojo-compose": "../../node_modules/dojo-compose",
        "dojo-core": "../../node_modules/dojo-core",
        "dojo-widgets": "../../node_modules/dojo-widgets",
        "immutable": "../../node_modules/immutable/dist",
        "maquette": "../../node_modules/maquette/dist",
        "rxjs": "../../node_modules/@reactivex/rxjs/dist/amd",
        "dojo": "../../node_modules/dojo",
        "dijit": "../../node_modules/dijit",
        "legacyPlugins": "../../legacyPlugins"
    },
    baseUrl: ".",
    appDir: ".",
    dir: "built/",
    modules: [
        {
            name: "index"
        }
    ],
    map: {
        '*': {
            'dojo/i18n': 'legacyPlugins/requirejs/i18n',
            'dojo/text': 'legacyPlugins/requirejs/text'
        }
    }

});

[kitsonk]

dojo/cli is now a thing, which can be used for issue tracking for now.

[kitsonk]

We should make sure that any thoughts or output of this is covered by the constitute parts of this and once validated we should close this in lieu of other more discreet issues.

[agubler]

We can close this in favour of discreet issues in cli-build now we have a working build