[v2] not transpiling cache-dir breaks jest tests

[ascorbic]

Description

Since upgrading from beta.12 to beta.44, jest tests fail with a syntax error, as gatsby-browser-entry is not being processed through Babel. Previously, cache-dir was transpiled before publishing as a package, but this was changed recently. All my other settings are as before. My project is mostly TypeScript, with ts-jest, but .js files are configured to use babel-jest. The repo can be found here.

Steps to reproduce

Run jest

Expected result

Tests pass as before

Actual result

Jest fails on non-transpiled gatsby code:

    Details:

... /node_modules/gatsby/cache-dir/gatsby-browser-entry.js:1
    ({"Object.<anonymous>":function(module,exports,require,__dirname,__filename,global,jest){import React from "react"
                                                                                             ^^^^^^

    SyntaxError: Unexpected token import

Environment

System:
OS: macOS High Sierra 10.13.4
CPU: x64 Intel(R) Core(TM) i7-7660U CPU @ 2.50GHz
Shell: 5.3 - /bin/zsh
Binaries:
Node: 9.11.1 - /usr/local/bin/node
Yarn: 1.7.0 - /usr/local/bin/yarn
npm: 6.1.0 - /usr/local/bin/npm
Browsers:
Chrome: 67.0.3396.99
Safari: 11.1
npmPackages:
gatsby: 2.0.0-beta.44 => 2.0.0-beta.44
gatsby-image: ^2.0.0-beta.6 => 2.0.0-beta.6
gatsby-plugin-manifest: ^2.0.2-beta.2 => 2.0.2-beta.2
gatsby-plugin-netlify-cms: ^2.0.0-beta.6 => 2.0.0-beta.6
gatsby-plugin-offline: next => 2.0.0-beta.2
gatsby-plugin-react-helmet: ^3.0.0-beta.3 => 3.0.0-beta.3
gatsby-plugin-sass: ^2.0.0-beta.5 => 2.0.0-beta.5
gatsby-plugin-sharp: ^2.0.0-beta.5 => 2.0.0-beta.5
gatsby-plugin-typescript: ^2.0.0-beta.5 => 2.0.0-beta.5
gatsby-source-filesystem: ^2.0.1-beta.5 => 2.0.1-beta.5
gatsby-transformer-json: ^2.1.1-beta.2 => 2.1.1-beta.2
gatsby-transformer-sharp: ^2.1.1-beta.5 => 2.1.1-beta.5
npmGlobalPackages:
gatsby-cli: 2.0.0-beta.6

File contents (if changed)

gatsby-config.js:

const siteConfig = require("./site-config");

module.exports = {
    pathPrefix: siteConfig.pathPrefix,
    siteMetadata: siteConfig.siteMetadata,
    mapping: {
        "ProjectsJson.client": "ClientsJson"
    },
    plugins: [
        `gatsby-plugin-react-helmet`,
        {
            resolve: `gatsby-source-filesystem`,
            options: {
                path: `${__dirname}/static/assets`,
                name: "uploads"
            }
        },
        {
            resolve: `gatsby-source-filesystem`,
            options: {
                path: `${__dirname}/src/pages`,
                name: "pages"
            }
        },
        // This plugin identifies file nodes that are images and
        // transforms these to create new “ImageSharp” nodes.
        // With them you can resize images and
        // generate responsive image thumbnails.
        `gatsby-transformer-sharp`,
        // transform JSON file nodes
        `gatsby-transformer-json`,
        // This plugin exposes helper functions for processing
        // images with the NPM package “sharp”. It's used by
        // several plugins.
        `gatsby-plugin-sharp`,
        // Manifest for AppCache and PWA compatibility
        {
            resolve: `gatsby-plugin-manifest`,
            options: siteConfig.manifest
        },
        {
            resolve: `gatsby-plugin-netlify-cms`,
            options: {
                // One convention is to place your Netlify CMS customization code in a
                // `src/cms` directory.
                modulePath: `${__dirname}/src/cms/cms.ts`
            }
        },
        `gatsby-plugin-typescript`
    ]
};

package.json:

{
    "name": "aerian-site-rebuild",
    "description": "Aerian Studios LLC.",
    "version": "0.0.1",
    "author": "[email protected]",
    "dependencies": {
        "@fortawesome/fontawesome-svg-core": "^1.2.0-14",
        "@fortawesome/free-brands-svg-icons": "^5.1.0-11",
        "@fortawesome/free-solid-svg-icons": "^5.1.0-11",
        "@fortawesome/react-fontawesome": "^0.1.0-11",
        "@researchgate/react-intersection-observer": "^0.7.3",
        "deep-map": "^1.5.0",
        "extract-text-webpack-plugin": "^3.0.2",
        "gatsby": "2.0.0-beta.44",
        "gatsby-image": "^2.0.0-beta.6",
        "gatsby-plugin-manifest": "^2.0.2-beta.2",
        "gatsby-plugin-netlify-cms": "^2.0.0-beta.6",
        "gatsby-plugin-offline": "next",
        "gatsby-plugin-react-helmet": "^3.0.0-beta.3",
        "gatsby-plugin-sass": "^2.0.0-beta.5",
        "gatsby-plugin-sharp": "^2.0.0-beta.5",
        "gatsby-plugin-typescript": "^2.0.0-beta.5",
        "gatsby-source-filesystem": "^2.0.1-beta.5",
        "gatsby-transformer-json": "^2.1.1-beta.2",
        "gatsby-transformer-sharp": "^2.1.1-beta.5",
        "leaflet": "^1.3.1",
        "netlify-cms": "^1.9.2",
        "react": "^16.4.1",
        "react-dom": "^16.4.1",
        "react-helmet": "^5.2.0",
        "react-leaflet": "^2.0.0",
        "react-markdown": "^3.3.4"
    },
    "keywords": [
        "gatsby"
    ],
    "license": "MIT",
    "scripts": {
        "build": "gatsby build",
        "start": "gatsby develop",
        "test": "jest",
        "updateSnapshot": "jest --updateSnapshot",
        "storybook": "start-storybook -p 9001 -c .storybook",
        "storybook-build": "build-storybook -c .storybook -o docs",
        "precommit": "lint-staged",
        "prepush": "jest --ci",
        "docz": "docz dev"
    },
    "devDependencies": {
        "@babel/plugin-syntax-dynamic-import": "^7.0.0-beta.51",
        "@storybook/addon-actions": "^3.4.8",
        "@storybook/addon-info": "^3.4.8",
        "@storybook/react": "^4.0.0-alpha.10",
        "@types/jest": "^23.1.1",
        "@types/leaflet": "^1.2.8",
        "@types/node": "^10.5.2",
        "@types/react-helmet": "^5.0.6",
        "@types/react-leaflet": "^1.1.5",
        "@types/react-test-renderer": "^16.0.1",
        "@types/storybook__addon-actions": "^3.0.3",
        "@types/storybook__addon-info": "^3.2.3",
        "@types/storybook__react": "^3.0.7",
        "autoprefixer": "^8.6.5",
        "coveralls": "^3.0.2",
        "css-loader": "^0.28.11",
        "docz": "^0.3.3",
        "extract-text-webpack-plugin": "^3.0.2",
        "gh-pages": "^1.2.0",
        "husky": "^0.14.3",
        "hygen": "^1.6.2",
        "hygen-react-typescript": "^1.0.2",
        "identity-obj-proxy": "^3.0.0",
        "intersection-observer": "^0.5.0",
        "jest": "^23.1.0",
        "lint-staged": "^7.2.0",
        "node-sass": "^4.9.0",
        "prettier": "^1.13.5",
        "react-docgen-typescript-loader": "^2.1.1",
        "react-docgen-typescript-webpack-plugin": "^1.1.0",
        "react-test-renderer": "^16.4.1",
        "ts-jest": "^22.4.6",
        "ts-loader": "^4.4.1",
        "tslint": "^5.10.0",
        "tslint-config-aerian": "^1.0.2",
        "typescript": "^2.9.2",
        "typings-for-css-modules-loader": "^1.7.0"
    },
    "lint-staged": {
        "*.{js,jsx,css,md,scss}": [
            "prettier --write",
            "git add",
            "jest --ci --findRelatedTests"
        ],
        "*.{ts,tsx}": [
            "tslint --fix",
            "git add",
            "jest --ci --findRelatedTests"
        ]
    },
    "repository": {
        "type": "git",
        "url": "git+https://github.com:aerian-studios/aerian-site-rebuild.git"
    },
    "jest": {
        "transform": {
            "^.+\\.tsx?$": "ts-jest",
            "^.+\\.js$": "<rootDir>/jestPreprocess.js"
        },
        "testRegex": "(/__tests__/.*|(\\.|/)(test|spec))\\.(tsx?)$",
        "moduleFileExtensions": [
            "ts",
            "tsx",
            "js",
            "jsx",
            "json",
            "node"
        ],
        "moduleNameMapper": {
            "^.+\\.(css|less|scss|png)$": "identity-obj-proxy",
            "^./pages.json$": "<rootDir>/__mocks__/pages.json"
        }
    }
}

gatsby-node.js:

const path = require("path");
const { createFilePath } = require("gatsby-source-filesystem");
const deepMap = require("deep-map");
// Implement the Gatsby API “createPages”. This is
// called after the Gatsby bootstrap is finished so you have
// access to any information necessary to programmatically
// create pages.
exports.createPages = ({ actions, graphql }) => {
    const { createPage } = actions;

    /**
     * Work out the necessary to generate disntinct pages
     * @param {object} edge - The data for the distinct page
     */
    const generateDistinctPage = data => {
        const template = data.path && data.path.replace("/", "");
        const { id, sections, staff } = data;

        // for the time being we can just assume that there are different teplates for each of the pages, but we can add logic here to reuse page templates
        createPage({
            path: template,
            component: path.resolve(`src/templates/${String(template)}.tsx`),
            // additional data can be passed via context
            context: {
                id
            }
        });
    };

    /**
     *
     * @param {string} id - JSON id, generally a path to the file
     * @param {string} template - the string name of the template without the `.tsx`
     * @param {string} slug - generally the unique name of the JSON file (without path or file type)
     */
    const generatePage = (id, template, slug) => {
        createPage({
            path: slug,
            component: path.resolve(`src/templates/${String(template)}.tsx`),
            // additional data can be passed via context
            context: {
                id
            }
        });
    };

    /**
     * Run queries to get all the types of pages for which we need to make static pages
     *
     * `allPagesJson` needs a bit more information if we want to control how they are
     * processed in the future
     */
    return graphql(
        `
            {
                allProjectsJson(limit: 1000) {
                    edges {
                        node {
                            id
                            slug
                        }
                    }
                }
                allPagesJson(limit: 1000) {
                    edges {
                        node {
                            id
                            path
                            staff {
                                name
                            }
                            sections {
                                title
                            }
                        }
                    }
                }
            }
        `
    ).then(result => {
        if (result.errors) {
            result.errors.forEach(e => console.error(e.toString()));
            return Promise.reject(result.errors);
        }

        // Gatsby uses Redux to manage its internal state.
        // Plugins and sites can use functions like "createPage"
        // to interact with Gatsby.
        result.data.allProjectsJson.edges.forEach(edge => {
            const id = edge.node.id;
            const template = "project";
            const slug = `our-work/project/${edge.node.slug}`;

            generatePage(id, template, slug);
        });

        result.data.allPagesJson.edges.forEach(edge => {
            generateDistinctPage(edge.node);
        });
        return Promise.resolve();
    });
};

const excluded = new Set(["internal", "children", "parent", "id"]);

exports.onCreateNode = ({ node, getNode, getNodes }) => {
    if (node.internal.owner === "gatsby-transformer-json") {
        const parent = getNode(node.parent);
        const makeRelative = value => {
            if (typeof value === "string") {
                const pathToFile = path.join(__dirname, "static", value);
                const foundFileNode = getNodes().find(
                    n => n.absolutePath === pathToFile
                );

                if (foundFileNode) {
                    const p = path.relative(
                        parent.dir,
                        foundFileNode.absolutePath
                    );
                    if (p) {
                        return p;
                    }
                }
            }
            return value;
        };
        Object.keys(node).forEach(key => {
            if (excluded.has(key)) {
                return;
            }

            if (typeof node[key] === "string") {
                node[key] = makeRelative(node[key]);
            }
            deepMap(node[key], makeRelative, { inPlace: true });
        });
    }
};

exports.onCreateWebpackConfig = (
    { actions, stage, loaders, getConfig },
    { postCssPlugins, ...sassOptions }
) => {
    const sassLoader = {
        loader: require.resolve(`sass-loader`),
        options: {
            sourceMap: stage === "develop",
            ...sassOptions
        }
    };

    const cssLoader = {
        loader: require.resolve("typings-for-css-modules-loader"),
        options: {
            modules: true,
            camelCase: true,
            banner: `/* tslint:disable */
// This file is automatically generated from your CSS. Any edits will be overwritten.`,
            namedExport: true,
            silent: true,
            importLoaders: 2,
            localIdentName: "[path][name]__[local]--[hash:base64:5]"
        }
    };

    const sassModuleRule = {
        test: /\.s(a|c)ss$/,
        use: [
            loaders.miniCssExtract(),
            cssLoader,
            loaders.postcss({
                plugins: [require("autoprefixer")({ browsers: [">1%"] })]
            }),
            sassLoader
        ]
    };

    let configRules = [];
    switch (stage) {
        case `develop`:
        case `build-html`:
        case `develop-html`:
            configRules.push(sassModuleRule);
            break;

        case `build-javascript`:
            configRules = [
                sassModuleRule,
                {
                    test: /\.mjs$/,
                    include: /node_modules/,
                    type: "javascript/auto"
                }
            ];
            break;
        default:
            return;
    }
    actions.setWebpackConfig({
        module: {
            rules: configRules
        },
        resolve: {
            extensions: [
                ".webpack.js",
                ".web.js",
                ".mjs",
                ".js",
                ".json",
                ".jsx",
                ".ts",
                ".tsx"
            ]
        }
    });
};

gatsby-browser.js: N/A
gatsby-ssr.js: N/A

[ascorbic]

OK, I've fixed this by putting a fuller babel config in the jest preprocessor. It would be nice to have a minimal jest setup either documented or included in the start projects, as it can be a bit of a fiddle to set up.

[KyleAMathews]

Yeah, that's definitely something we'd like. Do you think yours is a good starting point (now that you've gone through the pain of figuring things out)? Would you like to PR a start to a new docs page?

[ascorbic]

Once I'm happy that it's working robustly then yes, I'll do that.