promises support in tests (#307)

* Added expected and actual to TestResults class * First pass of MochaTestReporter * Added actual and expected to assertTrue and assertFalse * New string function and more assert messages * Cleaned up message generation * Fixed failing unit test * Cleaned up some messaging output * Do not report empty actual and expected diffs * actual and expected are invalid by default * whitespace clean up * Made temp logs less spammy * More assertion updates * Updates to most aa and array asserts * clean up on displaying types * Updates to assertArrayCount * Updates to assertArrayCount to fix a crash case * Messaging updates * Updated array count assertions * updated assertNotEmpty and assertEmpty * Fixed a actual/expected filting bug * Added tests to sample project * Fixed a bug with assertArrayNotCount * Updates to assertArrayContainsOnlyValuesOfType and assertType * Fixed a bug in assertArrayContainsOnlyValuesOfType with AA's * Removed stop statement * updates to assertArrayNotContainsSubset, assertArrayContainsSubset, and more tests for AssertArrayContains * updates to assertClass and assertSubType * Fixed a bugt in assertArrayNotContainsSubset that would always fail for aa subsets * updated assertNodeCount * More updates * Updates assets to respect custom failure messages when supplied * remove only annotation * Fixed a crash in eqAssocArray * Fixed some unexpected failing tests * Fixed some failure messages for assert true and false * Tweeks to empty multiline aa and array * Fixed a crash reporting node tests due to missing actual and expected results * Fixed a bug related to ignoredFields * Fixed a error log that would print even when there was no error for node tests * Added new test reporter hooks * implimented new hooks and moved logs into reporters * Fixed a falure case that would pass * Include ignored tests but skip their exicution when found * Moved tests that should fail to their own suite * Removed old commited code * Removed an ignore anotation * Fixed some issues with skipped tests * Updated the console test reporter to handle skipped and ignored tests/groups/suites * Fixed a regression in the console test reporter * Fixed some broken tests * Re-enabled catch on crash * Added new colorizeOutput setting * Formatting fixes * linking fixes * replaced short hand prints * Moved diffing logic into an internal namespace to prevent code conflicts * Code clean up * Updated reporters to take event objects * Updated brighterscript to 0.68.3 * Updated brighterscript to 0.68.3 for test workspace * Added a log that will always fire after test reporters * Removed some unnessisary prints from mocha reporter * Doc blocks * Made stacktraces easier to read and added test file links in crashes * Typo * More tests * Removed actual and expected from assertNotEqual * More failure tests * More failure tests for assertArrayNotContains * Cleaned up some tests * Fixed assertAANotHasKey not marking tests as failed in some cases * Fixed crash in assertInvalid * Added actual and expected diff for assertAAHasKeys * Cleaned up some failure messages * More tests and clean up for failed assertions * Updated asMultilineString to print invalid and uninitialized as their types * comment typo * Code clean up and lots more comments in mocha test reporter * Fixed the reports namespace name to be reporter * Moved some logs to a new warning log rather then error * updated assertClass to remove actual and expected diff * Fixed uninitilized not being comparable with eqValues * Fixed mocha results printing * Fixed most failing tests * Update CommonUtils.bs * Update CommonUtils.bs * Update TestGroup.bs * Update CodeCoverageProcessor.spec.ts * installed ropm and promises * added fast glob npm module * updated FileFactory to use flog to find framework files * Updated the framework fold structure to match what will be on device * Fixed build copy command * Code clean up and removed unessisary props following the structure change * Removed options from FileFactory constructor * Updated the promises prefix and FileFactory to auto import it * Moved MochaTestReporter class into source/rooibos * Added a couple passing tests for assertInvalid * Added example test showing promises are imported with rooibos tests * Always sort keys in aa when generating diff strings * Added support for reporting slow tests * Fixed failing test * Fixed a rehydration issue * Create differed promises in node tests * Added some examples returning promises from tests * First pass as promises in node/async tests * Added more tests that levrage promises * Added support for using native nodes in node tests * Async tests are now automaticly flagged as Node tests * Fixed a thread ownership issue if your node test uses a non-renderable node * Reduced the delay in some tests as they did not need to be so long * renammed vars differed to deferred * Removed threadinto prints * Fixed some bugs and removed some unused callbacks and node fields * Updated how results from node tests are passed back to the main thread test runner * Updated how the test time is recorded and moved some logic into functions for clarity * removed some commented out code * Fixed failing test * Fixed a bug where function names that start with a number would transpile invalid output * Added a suite to test suite level time outs * Added the new timeout logic to the suite and removed lagicy code * Fixed an issue where identical diffs would attempt to be rendered * Update BaseTestSuite.bs * Update Test.bs * Moved back to brighterscript 0.67.4 * Cleaned up unused field and unnecessary call back * Added some tests to prove returning promises fails tests when in non-node tests * Removed only annotation in spec file * Updated docs with some information on promises * Added an example of how to use assertAsyncField to watch a field on a task in a non-node test * package-lock update * Log message update * Fixed typos in docs * Added doc for slow annotation * Added tons of tests for passing usecases * Update bsc-plugin/src/lib/rooibos/CodeCoverageProcessor.spec.ts * New line when tests start * Remove the `postinstall` script during packaging * Give slow test more time to complete * Fix lint issue * Added assertNotInvalid test for roInvalid * Updated assertInvalid test for roInvalid * hardened assertNotInvalid and fixed assertInvalid for roInvalid * Fixed some off by oneline number reporting bugs * fixes after updating with reporting line number fixes from mocha branch --------- Co-authored-by: Bronley Plumb <[email protected]>
rokucommunity · Jan 28, 2025 · e68b898 · e68b898
1 parent ebfd429
commit e68b898
Show file tree

Hide file tree

Showing 23 changed files with 1,555 additions and 216 deletions.
diff --git a/bsc-plugin/package-lock.json b/bsc-plugin/package-lock.json
diff --git a/bsc-plugin/package.json b/bsc-plugin/package.json
@@ -16,7 +16,10 @@
         "publish-npm:beta": "npm run test && npm publish --tag=beta",
         "local": "ts-node scripts/install-local.js",
         "remote": "ts-node scripts/install-npm.js",
-        "cli": "ts-node src/cli.ts"
+        "cli": "ts-node src/cli.ts",
+        "prepack": "node scripts/pack.js --pre",
+        "postpack": "node scripts/pack.js --post",
+        "postinstall": "ropm copy"
     },
     "repository": {
         "type": "git",
@@ -35,6 +38,7 @@
     "dependencies": {
         "roku-debug": "^0.21.10",
         "roku-deploy": "^3.12.1",
+        "rooibos_promises": "npm:@rokucommunity/promises@^0.5.0",
         "source-map": "^0.7.3",
         "undent": "^0.1.0",
         "vscode-languageserver": "~6.1.1",
@@ -63,6 +67,7 @@
         "mocha": "^9.1.3",
         "nyc": "^15.1.0",
         "release-it": "^17.6.0",
+        "ropm": "^0.10.30",
         "source-map-support": "^0.5.13",
         "trim-whitespace": "^1.3.3",
         "ts-node": "^9.0.0",
@@ -121,6 +126,9 @@
             "ts"
         ]
     },
+    "ropm": {
+        "rootDir": "../framework/src"
+    },
     "config": {
         "commitizen": {
             "path": "./node_modules/cz-conventional-changelog"

diff --git a/bsc-plugin/scripts/pack.js b/bsc-plugin/scripts/pack.js
@@ -0,0 +1,20 @@
+var fsExtra = require('fs-extra');
+var path = require('path');
+
+var cachePath = path.join(__dirname, 'pack-cache.txt');
+var packageJsonPath = path.join(__dirname, '../package.json');
+
+var packageJson = fsExtra.readJsonSync(packageJsonPath);
+
+//store the script and remove it from package.json
+if (process.argv.includes('--pre')) {
+    fsExtra.outputFileSync(cachePath, packageJson.scripts.postinstall);
+    delete packageJson.scripts.postinstall;
+
+    //restore the script
+} else if (process.argv.includes('--post')) {
+    packageJson.scripts.postinstall = fsExtra.readFileSync(cachePath, packageJson.scripts.postinstall).toString();
+    fsExtra.removeSync(cachePath);
+}
+
+fsExtra.outputJsonSync(packageJsonPath, packageJson, { spaces: 4 });
diff --git a/bsc-plugin/src/lib/rooibos/Annotation.ts b/bsc-plugin/src/lib/rooibos/Annotation.ts
@@ -141,13 +141,18 @@ export class RooibosAnnotation {
                 isSolo = getAnnotationsOfType(AnnotationType.Solo).length > 0;
                 isIgnore = getAnnotationsOfType(AnnotationType.Ignore).length > 0;
 
+                for (const annotation of getAnnotationsOfType(AnnotationType.NodeTest)) {
+                    nodeName = annotation.getArguments()[0] as string;
+                }
+
                 for (const annotation of getAnnotationsOfType(AnnotationType.Async)) {
                     async = true;
                     asyncTimeout = annotation.getArguments().length === 1 ? parseInt(annotation.getArguments()[0] as any) : -1;
-                }
-
-                for (const annotation of getAnnotationsOfType(AnnotationType.NodeTest)) {
-                    nodeName = annotation.getArguments()[0] as string;
+                    if (nodeName === null) {
+                        // If the test is async, it must be a node test
+                        // Default to `Node` if no node is specified
+                        nodeName = 'Node';
+                    }
                 }
 
                 for (const annotation of getAnnotationsOfType(AnnotationType.Tags)) {

diff --git a/bsc-plugin/src/lib/rooibos/FileFactory.ts b/bsc-plugin/src/lib/rooibos/FileFactory.ts
@@ -24,7 +24,7 @@ export class FileFactory {
         this.coverageComponentBrsTemplate = fs.readFileSync(path.join(this.frameworkSourcePath, '/source/rooibos/CodeCoverage.brs'), 'utf8');
     }
 
-    public addedSourceFrameworkFilePaths: string[] = [];
+    public sourceFilesToAutoImport: string[] = [];
     public addedFrameworkFiles: BscFile[] = [];
 
     public addFrameworkFiles(program: Program) {
@@ -43,10 +43,10 @@ export class FileFactory {
         });
 
         for (let filePath of globedFiles) {
-            if (/^source[/\\]rooibos[/\\]/g.test(filePath)) {
+            if (this.shouldAddFileToImportList(filePath)) {
                 // Save a list of all source files added to the program
                 // to be imported by node test components
-                this.addedSourceFrameworkFilePaths.push(filePath);
+                this.sourceFilesToAutoImport.push(filePath);
             }
             let sourcePath = path.resolve(this.frameworkSourcePath, filePath);
             let fileContents = fs.readFileSync(sourcePath, 'utf8').toString();
@@ -67,7 +67,7 @@ export class FileFactory {
 
     public createTestXML(name: string, baseName: string, suite?: TestSuite): string {
         let scriptImports = [];
-        for (let filePath of this.addedSourceFrameworkFilePaths) {
+        for (let filePath of this.sourceFilesToAutoImport) {
             scriptImports.push(`<script type="text/brighterscript" uri="pkg:/${filePath}" />`);
         }
 
@@ -117,6 +117,16 @@ export class FileFactory {
         return result !== undefined;
     }
 
+    private shouldAddFileToImportList(destFilePath): boolean {
+        const pathDetails = path.parse(destFilePath);
+        if (pathDetails.dir === 'source' || pathDetails.dir.startsWith('source\\') || pathDetails.dir.startsWith('source/')) {
+            if (pathDetails.ext === '.brs' || (pathDetails.ext === '.bs' && !pathDetails.name.endsWith('.d'))) {
+                return true;
+            }
+        }
+        return false;
+    }
+
     public addFile(program, projectPath: string, contents: string) {
         try {
             const file = program.setFile({

diff --git a/bsc-plugin/src/lib/rooibos/RooibosSession.ts b/bsc-plugin/src/lib/rooibos/RooibosSession.ts
@@ -219,6 +219,12 @@ export class RooibosSession {
         }
         let brsFile = this.fileFactory.addFile(program, suite.bsPkgPath, undent`
             function init()
+                m.top.addField("rooibosRunSuite", "boolean", false)
+                m.top.observeFieldScoped("rooibosRunSuite", "rooibosRunSuite")
+            end function
+
+            function rooibosRunSuite()
+                m.top.unobserveFieldScoped("rooibosRunSuite")
                 nodeRunner = Rooibos_TestRunner(m.top.getScene(), m)
                 m.top.rooibosTestResult = nodeRunner.runInNodeMode("${suite.name}")
             end function

diff --git a/bsc-plugin/src/lib/rooibos/TestGroup.ts b/bsc-plugin/src/lib/rooibos/TestGroup.ts
@@ -220,6 +220,7 @@ export class TestGroup extends TestBlock {
                 name: ${sanitizeBsJsonString(this.name)}
                 isSolo: ${this.isSolo}
                 isIgnored: ${this.isIgnored}
+                isAsync: ${this.isAsync}
                 filename: "${this.pkgPath}"
                 lineNumber: "${this.annotation.annotation.range.start.line + 1}"
                 setupFunctionName: "${this.setupFunctionName || ''}"

diff --git a/bsc-plugin/src/lib/rooibos/TestSuite.ts b/bsc-plugin/src/lib/rooibos/TestSuite.ts
@@ -1,5 +1,6 @@
 import * as path from 'path';
 import type { AstEditor, BrsFile, ClassStatement } from 'brighterscript';
+import { nodes } from 'brighterscript/dist/roku-types';
 
 import { diagnosticNodeTestIllegalNode, diagnosticNodeTestRequiresNode } from '../utils/Diagnostics';
 
@@ -9,6 +10,8 @@ import type { TestGroup } from './TestGroup';
 import { addOverriddenMethod, sanitizeBsJsonString } from './Utils';
 import type { RooibosSession } from './RooibosSession';
 
+const nativeNodeNames = Object.keys(nodes);
+
 /**
  * base of test suites and blocks..
  */
@@ -124,7 +127,7 @@ export class TestSuite extends TestBlock {
         if (this.isNodeTest) {
             if (!this.nodeName) {
                 diagnosticNodeTestRequiresNode(this.file, this.annotation.annotation);
-            } else if (!this.file.program.getComponent(this.nodeName)) {
+            } else if (!this.file.program.getComponent(this.nodeName) && !nativeNodeNames.includes(this.nodeName.toLowerCase())) {
                 diagnosticNodeTestIllegalNode(this.file, this.annotation.annotation, this.nodeName);
             }
         }
@@ -138,6 +141,7 @@ export class TestSuite extends TestBlock {
       isSolo: ${this.isSolo}
       noCatch: ${this.annotation.noCatch}
       isIgnored: ${this.isIgnored}
+      isAsync: ${this.isAsync}
       pkgPath: "${this.pkgPath}"
       filePath: "${this.filePath}"
       lineNumber: ${this.classStatement.range.start.line + 1}

diff --git a/bsc-plugin/src/lib/rooibos/TestSuiteBuilder.ts b/bsc-plugin/src/lib/rooibos/TestSuiteBuilder.ts
@@ -186,6 +186,9 @@ export class TestSuiteBuilder {
     }
 
     private sanitizeFunctionName(name: string) {
+        if (/^\d/.test(name)) {
+            name = '_' + name;
+        }
         return name.replace(/[^0-9_a-z]/ig, '_');
     }
     public createTestCases(statement: ClassMethodStatement, annotation: RooibosAnnotation): boolean {

diff --git a/bsc-plugin/src/plugin.spec.ts b/bsc-plugin/src/plugin.spec.ts
@@ -488,6 +488,7 @@ describe('RooibosPlugin', () => {
                             isSolo: false
                             noCatch: false
                             isIgnored: false
+                            isAsync: false
                             pkgPath: "${s`source/test.spec.bs`}"
                             filePath: "${s`${tmpPath}/rootDir/source/test.spec.bs`}"
                             lineNumber: 3
@@ -510,6 +511,7 @@ describe('RooibosPlugin', () => {
                                     name: "groupA"
                                     isSolo: false
                                     isIgnored: false
+                                    isAsync: false
                                     filename: "${s`source/test.spec.bs`}"
                                     lineNumber: "4"
                                     setupFunctionName: ""
@@ -556,6 +558,124 @@ describe('RooibosPlugin', () => {
             })).not.to.exist;
         });
 
+        it('handles groups that start with numbers', async () => {
+            plugin.afterProgramCreate(program);
+            // program.validate();
+            const file = program.setFile<BrsFile>('source/test.spec.bs', `
+                @suite
+                class ATest extends rooibos.BaseTestSuite
+                    @describe("1groupA")
+                    @it("is test1")
+                    @slow(1000)
+                    function Test_3()
+                    end function
+                end class
+            `);
+            program.validate();
+            await builder.transpile();
+            console.log(builder.getDiagnostics());
+            expect(builder.getDiagnostics()).to.have.length(1);
+            expect(builder.getDiagnostics()[0].severity).to.equal(DiagnosticSeverity.Warning);
+            expect(plugin.session.sessionInfo.testSuitesToRun).to.not.be.empty;
+            expect(plugin.session.sessionInfo.suitesCount).to.equal(1);
+            expect(plugin.session.sessionInfo.groupsCount).to.equal(1);
+            expect(plugin.session.sessionInfo.testsCount).to.equal(1);
+
+            expect(
+                getContents('rooibosMain.brs')
+            ).to.eql(undent`
+                function main()
+                    Rooibos_init("RooibosScene")
+                end function
+            `);
+            expect(
+                getContents('test.spec.brs')
+            ).to.eql(undent`
+                function __ATest_builder()
+                    instance = __rooibos_BaseTestSuite_builder()
+                    instance.super0_new = instance.new
+                    instance.new = sub()
+                        m.super0_new()
+                    end sub
+                    instance._1groupA_is_test1 = function()
+                    end function
+                    instance.super0_getTestSuiteData = instance.getTestSuiteData
+                    instance.getTestSuiteData = function()
+                        return {
+                            name: "ATest"
+                            isSolo: false
+                            noCatch: false
+                            isIgnored: false
+                            isAsync: false
+                            pkgPath: "${s`source/test.spec.bs`}"
+                            filePath: "${s`${tmpPath}/rootDir/source/test.spec.bs`}"
+                            lineNumber: 3
+                            valid: true
+                            hasFailures: false
+                            hasSoloTests: false
+                            hasIgnoredTests: false
+                            hasSoloGroups: false
+                            setupFunctionName: ""
+                            tearDownFunctionName: ""
+                            beforeEachFunctionName: ""
+                            afterEachFunctionName: ""
+                            isNodeTest: false
+                            isAsync: false
+                            asyncTimeout: 60000
+                            nodeName: ""
+                            generatedNodeName: "ATest"
+                            testGroups: [
+                                {
+                                    name: "1groupA"
+                                    isSolo: false
+                                    isIgnored: false
+                                    isAsync: false
+                                    filename: "${s`source/test.spec.bs`}"
+                                    lineNumber: "4"
+                                    setupFunctionName: ""
+                                    tearDownFunctionName: ""
+                                    beforeEachFunctionName: ""
+                                    afterEachFunctionName: ""
+                                    testCases: [
+                                        {
+                                            isSolo: false
+                                            noCatch: false
+                                            funcName: "_1groupA_is_test1"
+                                            isIgnored: false
+                                            isAsync: false
+                                            asyncTimeout: 2000
+                                            slow: 1000
+                                            isParamTest: false
+                                            name: "is test1"
+                                            lineNumber: 7
+                                            paramLineNumber: 0
+                                            assertIndex: 0
+                                            rawParams: invalid
+                                            paramTestIndex: 0
+                                            expectedNumberOfParams: 0
+                                            isParamsValid: true
+                                        }
+                                    ]
+                                }
+                            ]
+                        }
+                    end function
+                    return instance
+                end function
+                function ATest()
+                    instance = __ATest_builder()
+                    instance.new()
+                    return instance
+                end function
+            `);
+
+            //verify the AST was restored after transpile
+            const cls = file.ast.statements[0] as ClassStatement;
+            expect(cls.body.find((x: ClassMethodStatement) => {
+                return x.name?.text.toLowerCase() === 'getTestSuiteData'.toLowerCase();
+            })).not.to.exist;
+        });
+
         it('test full transpile with complex params', async () => {
             plugin.afterProgramCreate(program);
             // program.validate();
@@ -2174,7 +2294,8 @@ describe('RooibosPlugin', () => {
             [['mocha'], 'rooibos_MochaTestReporter'],
             [['JUnit', 'MyCustomReporter'], `rooibos_JUnitTestReporter${sep}MyCustomReporter`]
         ];
-        it('adds custom test reporters', async () => {
+        it('adds custom test reporters', async function test() {
+            this.timeout(10_000);
             for (const [reporters, expected] of params) {
                 setupProgram({
                     rootDir: _rootDir,

diff --git a/docs/index.md b/docs/index.md
@@ -1064,7 +1064,7 @@ The behavior of your unit tests is identical to unit testing any other class, wi
 ### Async testing
 To indicate a test suite will run in async mode you will mark the test suite with the @async annotation or mark one ore more tests with @async. This will cause rooibos to run the tests in async mode. If all tests in the suite do not complete within the timeout, then the suite will fail. The default time out is 60 seconds.
 
-When you mark any test in a suite with the @async annotation this will keep the test running in the background, waiting for a call to m.done(). If the call is not made within the timeout, then the test fails. Again, the default timeout for each test is 2 seconds.
+When you mark any test in a suite with the @async annotation this will keep the test running in the background, waiting for a call to `m.done()`. If the call is not made within the timeout, then the test fails. Again, the default timeout for each test is 2 seconds.
 
 Note: you are not required to add @async to your testSuite, it is implied when you add it one more more tests. However, you may wish to add tge async annotation to your suite, to override the default of 60 seconds (e.g. with the annotation `async(12000)` we are instructing rooibos to wait up to 2 minutes for the _whole_ suite).
 
@@ -1142,7 +1142,26 @@ function OnTimer()
   m.testSuite.done()
 end function
 ```
+#### Working with Promises
 
+Alternately, instead of using the `done()` callback, you may return a `Promise` from your test. The test will automaticly complete when the promise is completed. If the returned promise rejects the test will be considered failed.
+
+This is useful if the APIs you are testing return promises:
+
+```
+  @it('respond with matching records')
+  function _()
+    return rooibos.promises.chain(db.find({type: 'User'})).then(sub(result)
+      m.testSuite.assertEqual(result.count(), 3)
+    end sub).catch(sub(error)
+      m.testSuite.fail("should not reject")
+    end sub).toPromises()
+  end function
+```
+
+Rooibos implements the [rokucomunity/promises](https://github.com/rokucommunity/promises) library to enable this support. Please visit that project for more details and examples of promises.
+
+Note: `Promises` are only supported in Node tests and returning a promise from a non-Node test will automaticly fail the test.
 
 ## Advanced setup