Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tsify + babelify + browserify with threejs modules: SyntaxError: 'import' and 'export' may appear only with 'sourceType: module' (9:0) #286

Closed
JohannesDeml opened this issue Jun 22, 2019 · 17 comments

Comments

@JohannesDeml
Copy link

JohannesDeml commented Jun 22, 2019

Hello there,

I'm trying to use threejs with typescript for a web application. Since I would like to use modules (threejs introduced them in r105), I need to get down to es5 for browserify again for which I thought babel might be a solution. Sadly I still get the error
SyntaxError: 'import' and 'export' may appear only with 'sourceType: module' (9:0)

I have the following configuration:

package.json

    "@babel/cli": "^7.4.4",
    "@babel/core": "^7.4.5",
    "@babel/preset-env": "^7.4.5",
    "babelify": "^10.0.0",
    "browserify": "^16.2.3",
    "gulp": "^4.0.2",
    "tsify": "^4.0.1",
    "typescript": "^3.5.2",

.babelrc

{
    "presets": [
        ["@babel/preset-env",
            {
                "targets": {
                    "esmodules": true
                }
            }
        ],
    ],
    "extensions": [ ".js", ".ts", ".tsx" ]
}

.tsconfig

{
    "include": [
        "./src/**/*"
    ],
    "compilerOptions": {
        "baseUrl": ".",
        "paths": {
            "*": ["types/*"] },
        "noImplicitAny": true,
        "noImplicitReturns": true,
        "noImplicitThis": true,
        "alwaysStrict": true,
        "strictNullChecks": true,
        "module": "commonjs",
        "moduleResolution": "node",
        "target": "es6"
    }
}

gulpfile.js

function buildBundle() {
    return browserify({
        debug: true
    })
        .add("./src/main.ts")
        .plugin(tsify, { target: 'es6'})
        .transform(babelify)
        .bundle()
}

File at which the error is triggered

...
import { ResizeSignal } from "../Signals/InputSignals/ResizeSignal";
import { GLTFLoader } from 'three/examples/jsm/loaders/GLTFLoader.js';  // this is the line
...

private loader: GLTFLoader;

What am I missing? I'm completely stuck.

@goto-bus-stop
Copy link
Contributor

In your babel config, I think you're looking for the "modules": "commonjs" option instead of "esmodules": true.

Browserify transforms only run on "local" code (not inside node_modules) by default, to encourage more self-contained modules in the ecosystem. To allow transforms to run on files inside node_modules, use global: true: .transform(babelify, { global: true }).

@JohannesDeml
Copy link
Author

JohannesDeml commented Jun 23, 2019

Hello @goto-bus-stop

Thanks a lot for the reply. Good point, the modules should be commonjs from what I can tell. I changed my babelrc and gulp file to the following:

{
    "presets": [
        ["@babel/preset-env",
            {
                "targets": {
                    "modules": "commonjs"
                }
            }
        ],
    ],
    "plugins": ["@babel/plugin-transform-modules-commonjs"], // Tried it with and without the plugin
    "extensions": [ ".js", ".ts", ".tsx" ]
}
function buildBundle() {
    return browserify({
        debug: true
    })
        .add("./src/main.ts")
        .plugin(tsify, { target: 'es6'})
        .transform(babelify, {"compact": false, "global": "true"}) // compact false since three.js is larger than 500kb
        .bundle()
}

Sadly I still get the same error:

[12:16:46] Starting 'buildBundle'...
[12:16:57] 'buildBundle' errored after 11 s
[12:16:57] SyntaxError: 'import' and 'export' may appear only with 'sourceType: module' (8:0) while parsing C:\project\node_modules\three\examples\jsm\loaders\GLTFLoader.js while parsing file: C:\project\node_modules\three\examples\jsm\loaders\GLTFLoader.js     
    at DestroyableTransform.end [as _flush] (C:\project\node_modules\insert-module-globals\index.js:114:21)
    at DestroyableTransform.prefinish (C:\project\node_modules\through2\node_modules\readable-stream\lib\_stream_transform.js:138:10)
    at emitNone (events.js:106:13)
    at DestroyableTransform.emit (events.js:208:7)
    at prefinish (C:\project\node_modules\through2\node_modules\readable-stream\lib\_stream_writable.js:619:14)
    at finishMaybe (C:\project\node_modules\through2\node_modules\readable-stream\lib\_stream_writable.js:627:5)
    at endWritable (C:\project\node_modules\through2\node_modules\readable-stream\lib\_stream_writable.js:638:3)
    at DestroyableTransform.Writable.end (C:\project\node_modules\through2\node_modules\readable-stream\lib\_stream_writable.js:594:41)
    at BabelifyStream.onend (_stream_readable.js:595:10)
    at Object.onceWrapper (events.js:313:30)

Any idea what else I can try?

@goto-bus-stop
Copy link
Contributor

goto-bus-stop commented Jun 23, 2019

Oh, I think babel in v7 also changed how its babelrc resolution works, so a local babelrc is no longer used for modules inside node_modules(?). You might try:

b.transform(babelify, { global: true, plugins: ['@babel/plugin-transform-modules-commonjs'] })

that should not be affected by the new babelrc resolution. you can then use .babelrc only for the transforms you need in your own code.

You can also add https://www.npmjs.com/package/tfilter to optimize things a bit and only transform the three.js modules:

b.transform(
  tfilter(babelify, { include: /\/three\/examples\/jsm/ }),
  { global: true, plugins: ['@babel/plugin-transform-modules-commonjs'] }
)

@JohannesDeml
Copy link
Author

Wow, thanks a lot for the quick response!
Sadly now it seems like there is a syntax problem, not quite sure why though:
TypeError: browserify(...).add(...).plugin(...).transform(...).pipe is not a function

function buildBundle() {
    return browserify({
        debug: true
    })
        .add("./src/main.ts")
        .plugin(tsify, { target: 'es6'})
        .transform(babelify, { global: true, presets: ["@babel/preset-env"], plugins: ['@babel/plugin-transform-modules-commonjs'] })
        .pipe(source('bundle.js'))
        .pipe(buffer())
        .pipe(terser())
        .pipe(gulp.dest("dist"));
}

I tried it with and without the preset setting. Thanks also for the info with optimizing. I will do that once the pipeline works again :)

@goto-bus-stop
Copy link
Contributor

you're missing a .bundle() call between transform and pipe to start the bundling:

.transform(babelify, ...)
.bundle()
.pipe(source(...))

@JohannesDeml
Copy link
Author

Oh, just saw that as well. 😊 It works again, thank you so much for your guidance!

@JohannesDeml JohannesDeml changed the title tsify + babelify + browserify with threejs modules: yntaxError: 'import' and 'export' may appear only with 'sourceType: module' (9:0) tsify + babelify + browserify with threejs modules: SyntaxError: 'import' and 'export' may appear only with 'sourceType: module' (9:0) Jun 23, 2019
@JohannesDeml
Copy link
Author

Sorry to post once again, but sadly the new setup created a problem: It seems to me like three.js is now included two times in the bundle. When I'm counting for specific lines of code in the new bundle I always get two matches (for the bundles I created before babel it was only one hit). Also the heap size seems to fit with the theory and the problem described here.

That being said, I guess the problem could be solved with defining tfilter correctly to only run on the jsm folder. I tried the following:

b.transform(
  tfilter(babelify, { include: /\/three\/examples\/jsm/ }),
  { global: true, plugins: ['@babel/plugin-transform-modules-commonjs'] }
)
b.transform(
  tfilter(babelify, { include: "*/three/examples/jsm*" }),
  { global: true, plugins: ['@babel/plugin-transform-modules-commonjs'] }
)
.transform(babelify,
            { compact: false,
              only: [
                  "./src",
                  "./node_modules/three/examples/jsm"
              ],
              plugins: ['@babel/plugin-transform-modules-commonjs'], 
              sourceMaps: false })

For all of them I get the same error again:

SyntaxError: 'import' and 'export' may appear only with 'sourceType: module' (9:0) while parsing C:\project\node_modules\three\examples\jsm\loaders\GLTFLoader.js while parsing file: C:\project\node_modules\three\examples\jsm\loaders\GLTFLoader.js

Is there something other than tfilter you can recommend or what is wrong with the configuration you suggested?

folder structure:

-dist/bundle.js
-src/mySourceFiles
-node_modules/three/...
-gulpfile.js
-tsconfig.js
-.babelrc

@nicolo-ribaudo
Copy link
Member

Try adding sourceType: "unambiguous" to your babel config, which makes Babel infer the source type (module or script) based on the presence of import/export declarations.

@JohannesDeml
Copy link
Author

Try adding sourceType: "unambiguous" to your babel config, which makes Babel infer the source type (module or script) based on the presence of import/export declarations.

Thanks for the suggestion, I added it to the config and to the gulpfile, sadly it seems to change nothing.

{
    "sourceType": "unambiguous",
    "presets": [
        ["@babel/preset-env",
            {
                "targets": {
                    "esmodules": true
                }
            }
        ],
    ],
    "extensions": [ ".js", ".ts", ".tsx" ]
}
function buildBundle() {
    return browserify({
        debug: false
    })
        .add("./src/main.ts")
        .plugin(tsify, { noImplicitAny: true })
        .transform(
            tfilter(babelify, { include: /(\\three\\examples\\jsm|\/three\/examples\/jsm|three.module.js)/ }),
            { global: true, sourceType: "unambiguous", presets: ["@babel/preset-env"], plugins: ['@babel/plugin-transform-modules-commonjs'] }
          )
        .bundle()
}

I did solve the tfilter riddle: You need to use backslashes for windows paths, and the jsm are also in the build folder which also needs to be included. That however didn't solve my problem with a lot of things being included twice.

Is this a problem on my pipeline or is this maybe caused by threejs?

@goto-bus-stop
Copy link
Contributor

three.js includes different bundle versions, so it might be that you're accidentally including the commonjs version somewhere and the ESM version somewhere else.

@JohannesDeml
Copy link
Author

JohannesDeml commented Jun 24, 2019

three.js includes different bundle versions, so it might be that you're accidentally including the commonjs version somewhere and the ESM version somewhere else.

That sounds quite likely. On the other hand I don't know how that would happen... From what I can tell from the Documentation I can import the commonjs version with var THREE = require('three'); and the ESM version with import { Scene } from 'three';

I went through my project and I never use require for threejs
grafik

The imports I'm doing for threejs look like that:

import { Scene, Group, PerspectiveCamera, CubeTexture, Object3D, AmbientLight } from 'three';
import { GLTFLoader } from "three/examples/jsm/loaders/GLTFLoader";

Sorry, I'm not doing that much webdev, how can I find out when commonjs is imported?

@goto-bus-stop
Copy link
Contributor

Babelify transforms imports to CommonJS before browserify looks at the file, so browserify sees require() calls. Those will use the CommonJS build of three.js. (browserify does not support ES modules yet).

The only way to get to the ESM build of three.js is by explicitly require()ing its path. The jsm examples may(?) do that, not sure!

@nicolo-ribaudo
Copy link
Member

I'm not familiar with browserify, but if it supports es modules you can pass the modules: false option to preset-env to avoid transforming import/export to require()/exports.*.

@JohannesDeml
Copy link
Author

Babelify transforms imports to CommonJS before browserify looks at the file, so browserify sees require() calls. Those will use the CommonJS build of three.js. (browserify does not support ES modules yet).

The only way to get to the ESM build of three.js is by explicitly require()ing its path. The jsm examples may(?) do that, not sure!

Ah, that makes sense. I took a look in one of the examples, they are also using import from the build/three.module.js:

import {
	BufferGeometry,
	DefaultLoadingManager,
	Euler,
	FileLoader,
	Float32BufferAttribute,
	Group,
	LineBasicMaterial,
	LineSegments
} from "../../../build/three.module.js";

In my code I'm just pointing to three (import { Scene, Group, PerspectiveCamera, CubeTexture, Object3D, } from 'three';), so probably that import is pointing to the normal three version. So I guess in the end I have to find out how to point to build/three.module.js instead of build/three.min.js

I'm not familiar with browserify, but if it supports es modules you can pass the modules: false option to preset-env to avoid transforming import/export to require()/exports.*.

I think that is sadly not an option. I got the error from browserify because of using import and export.

@goto-bus-stop
Copy link
Contributor

goto-bus-stop commented Jun 24, 2019

you can explicitly do import 'three/build/three.module.js' to get the ESM version, or do

b.require(require.resolve('three/build/three.module.js'), { expose: 'three' })

to alias the .module.js version to the bare module name.

@JohannesDeml
Copy link
Author

Wow, thanks a lot. The require setting indeed solved the problem. Here is the final gulp script:

function es6Bundle() {
    log("✳️  ES6 Bundling!");
    return browserify({
        debug: true
    })
    .add("./src/main.ts")
    .require(require.resolve('three/build/three.module.js'), { expose: 'three' })
    .plugin(tsify, { noImplicitAny: true })
    .transform(
        babelify,
        {
            only: [
                "./node_modules/three/build/three.module.js",
                "./node_modules/three/examples/jsm/*"
              ],
            global: true,
            sourceType: "unambiguous",
            presets: ["@babel/preset-env"],
            plugins: ['@babel/plugin-transform-modules-commonjs']
        }
      )
    .bundle()
    .on('error', function(e) {log.error('Error when updating the Bundle: \n' + e);})
    .on('end', function() {log("➡️  Bundle created, uploading to dist")})
    .pipe(source('bundle.js'))
    .pipe(gulp.dest("dist"))
    .on('end', function() {log("✅  Bundle Updated")});
}

and the following dependencies:

"devDependencies": {
    "@babel/cli": "^7.4.4",
    "@babel/core": "^7.4.5",
    "@babel/preset-env": "^7.4.5",
    "@babel/plugin-transform-modules-commonjs": "^7.4.4",
    "babelify": "^10.0.0",
    "browserify": "^16.2.3",
    "fancy-log": "^1.3.3",
    "gltf-pipeline": "^2.1.3",
    "gulp": "^4.0.2",
    "gulp-cli": "^2.2.0",
    "gulp-sourcemaps": "^2.6.5",
    "gulp-typescript": "^5.0.1",
    "tsify": "^4.0.1",
    "typedoc": "^0.14.2",
    "typescript": "^3.5.2",
    "vinyl-buffer": "^1.0.1",
    "vinyl-source-stream": "^2.0.0",
  },
  "dependencies": {
    "three": "git://github.com/JohannesDeml/three.js.git#a2caed8481085e3fe72142f3708d77a7ed5c09d5"
  }

Sadly the build time now went up from 8 to 15 seconds for me, but at least it works now :)

@gAusWeb
Copy link

gAusWeb commented Dec 21, 2021

Oh, I think babel in v7 also changed how its babelrc resolution works, so a local babelrc is no longer used for modules inside node_modules(?). You might try:

b.transform(babelify, { global: true, plugins: ['@babel/plugin-transform-modules-commonjs'] })

that should not be affected by the new babelrc resolution. you can then use .babelrc only for the transforms you need in your own code.

You can also add https://www.npmjs.com/package/tfilter to optimize things a bit and only transform the three.js modules:

b.transform(
  tfilter(babelify, { include: /\/three\/examples\/jsm/ }),
  { global: true, plugins: ['@babel/plugin-transform-modules-commonjs'] }
)

I messed with this for days - this resolved all my issues - TY so much! :)

These parts in particular helped me:
yarn add -D @babel/plugin-transform-modules-commonjs

Then in my typescript gulp task:
{ global: true, plugins: ['@babel/plugin-transform-modules-commonjs'] }

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants