diff --git a/docs/howto/skills/addingskills.md b/docs/howto/skills/addingskills.md index fa2e299d9c..2b79369bc6 100644 --- a/docs/howto/skills/addingskills.md +++ b/docs/howto/skills/addingskills.md @@ -74,4 +74,11 @@ botskills disconnect --skillId SKILL_ID > Note: The id of the Skill can also be aquired using the `botskills list` command. You can check the [Skill CLI documentation](/lib/typescript/botskills/docs/list.md) on this command. ## Updating an existing Skill to reflect changes to Actions or LUIS model -> A botskills refresh command will be added shortly. In the meantime, run the above disconnect command and then connect the skill again. \ No newline at end of file +## Refresh Connected Skills +To refresh the dispatch model with any changes made to connected skills use the following command, specifying the `--cs` (for C#) or `--ts` (for TypeScript) argument for determining the coding language of your assistant, since each language takes different folder structures that need to be taken into consideration. + +botskills: + +```bash +botskills refresh --cs +``` diff --git a/lib/typescript/botskills/README.md b/lib/typescript/botskills/README.md index ccf91ac3fa..138c8e03b2 100644 --- a/lib/typescript/botskills/README.md +++ b/lib/typescript/botskills/README.md @@ -21,8 +21,9 @@ npm uninstall -g botskills ``` ## Botskills functionality -- [Connect](./docs/connect-disconnect.md) a Skill to your assistant -- [Disconnect](./docs/connect-disconnect.md) a Skill from your assistant +- [Connect](./docs/connect-disconnect.md#connect-a-skill-to-your-assistant) a Skill to your assistant +- [Disconnect](./docs/connect-disconnect.md#disconnect-a-skill-to-your-assistant) a Skill from your assistant +- [Refresh](./docs/refresh.md) connected skills - [List](./docs/list.md) all Skills connected to your assistant ## Daily builds @@ -36,3 +37,5 @@ npm install -g botskills --registry https://botbuilder.myget.org/F/aitemplates/n ## Further Reading - [Create and customize Skills for your assistant](../../../docs/tutorials/typescript/skill.md) - [Connect a Skill to your Assistant](../../../docs/howto/skills/addingskills.md) +- [Disconnect a Skill to your Assistant](../../../docs/howto/skills/addingskills.md#remove-a-skill-from-your-virtual-assistant) +- [Refresh Connected Skills](../../../docs/howto/skills/addingskills.md#refresh-connected-skills) diff --git a/lib/typescript/botskills/docs/connect-disconnect.md b/lib/typescript/botskills/docs/connect-disconnect.md index a622cba8b9..25f390e11b 100644 --- a/lib/typescript/botskills/docs/connect-disconnect.md +++ b/lib/typescript/botskills/docs/connect-disconnect.md @@ -18,8 +18,8 @@ botskills connect [options] | -l, --localManifest \ | Path to local Skill Manifest file | | -r, --remoteManifest \ | URL to remote Skill Manifest | | --cs | Determine your assistant project structure to be a csharp-like structure | -| --ts | Determine your assistant project structure to be a TypeScript-like structure -| --noTrain | (OPTIONAL) Determine whether the skills connected are not going to be trained (by default they are trained) | +| --ts | Determine your assistant project structure to be a TypeScript-like structure | +| --noRefresh | (OPTIONAL) Determine whether the model of your skills connected are not going to be refreshed (by default they are refreshed) | | --dispatchName [name] | (OPTIONAL) Name of your assistant's '.dispatch' file (defaults to the name displayed in your Cognitive Models file) | | --language [language] | (OPTIONAL) Locale used for LUIS culture (defaults to 'en-us') | | --luisFolder [path] | (OPTIONAL) Path to the folder containing your Skills' '.lu' files (defaults to './deployment/resources/skills/en' inside your assistant folder) | @@ -65,8 +65,8 @@ botskills disconnect [option] |-------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------| | -i, --skillId \ | Id of the skill to remove from your assistant (case sensitive) | | --cs | Determine your assistant project structure to be a csharp-like structure | -| --ts | Determine your assistant project structure to be a TypeScript-like structure -| --noTrain | (OPTIONAL) Determine whether the skills connected are not going to be trained (by default they are trained) | +| --ts | Determine your assistant project structure to be a TypeScript-like structure | +| --noRefresh | (OPTIONAL) Determine whether the model of your skills connected are not going to be refreshed (by default they are refreshed) | | --dispatchName [name] | (OPTIONAL) Name of your assistant's '.dispatch' file (defaults to the name displayed in your Cognitive Models file) | | --dispatchFolder [path] | (OPTIONAL) Path to the folder containing your assistant's '.dispatch' file (defaults to './deployment/resources/dispatch/en' inside your assistant folder) | | --outFolder [path] | (OPTIONAL) Path for any output file that may be generated (defaults to your assistant's root folder) | diff --git a/lib/typescript/botskills/docs/refresh.md b/lib/typescript/botskills/docs/refresh.md new file mode 100644 index 0000000000..c88d9d87e0 --- /dev/null +++ b/lib/typescript/botskills/docs/refresh.md @@ -0,0 +1,33 @@ +# Refresh Connected Skills + +The `refresh` command allows you to refresh the model of your connected skills specifying the assistant's coding language using `--cs` or `--ts`. + +> **Tip:** It's highly advisable to execute this command from the root folder of your assistant bot, so if you are using the suggested folder structure from the Templates, you may ommit most of the optional arguments, as they default to the expected values from the Templates' folder structure. + +The basic command to refresh your connected skills: + +```bash +botskills refresh [options] +``` + +### Options + +| Option | Description | +|-------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------| +| --cs | Determine your assistant project structure to be a csharp-like structure | +| --ts | Determine your assistant project structure to be a TypeScript-like structure | +| --dispatchName [name] | (OPTIONAL) Name of your assistant's '.dispatch' file (defaults to the name displayed in your Cognitive Models file) | +| --language [language] | (OPTIONAL) Locale used for LUIS culture (defaults to 'en-us') | +| --luisFolder [path] | (OPTIONAL) Path to the folder containing your Skills' '.lu' files (defaults to './deployment/resources/skills/en' inside your assistant folder) | +| --dispatchFolder [path] | (OPTIONAL) Path to the folder containing your assistant's '.dispatch' file (defaults to './deployment/resources/dispatch/en' inside your assistant folder) | +| --outFolder [path] | (OPTIONAL) Path for any output file that may be generated (defaults to your assistant's root folder) | +| --lgOutFolder [path] | (OPTIONAL) Path for the LuisGen output (defaults to a 'service' folder inside your assistant's folder) | +| --cognitiveModelsFile [path] | (OPTIONAL) Path to your Cognitive Models file (defaults to 'cognitivemodels.json' inside your assistant's folder) | +| --verbose | (OPTIONAL) Output detailed information about the processing of the tool | +| -h, --help | Output usage information | + +An example on how to use: + +```bash +botskills refresh --cs --verbose +``` \ No newline at end of file diff --git a/lib/typescript/botskills/src/botskills-connect.ts b/lib/typescript/botskills/src/botskills-connect.ts index 396858dbb9..f309673c7f 100644 --- a/lib/typescript/botskills/src/botskills-connect.ts +++ b/lib/typescript/botskills/src/botskills-connect.ts @@ -36,7 +36,7 @@ program .option('-r, --remoteManifest ', 'URL to remote Skill Manifest') .option('--cs', 'Determine your assistant project structure to be a CSharp-like structure') .option('--ts', 'Determine your assistant project structure to be a TypeScript-like structure') - .option('--noTrain', '[OPTIONAL] Determine whether the skills connected are not going to be trained (by default they are trained)') + .option('--noRefresh', '[OPTIONAL] Determine whether the model of your skills connected are not going to be refreshed (by default they are refreshed)') .option('--dispatchName [name]', '[OPTIONAL] Name of your assistant\'s \'.dispatch\' file (defaults to the name displayed in your Cognitive Models file)') .option('--language [language]', '[OPTIONAL] Locale used for LUIS culture (defaults to \'en-us\')') .option('--luisFolder [path]', '[OPTIONAL] Path to the folder containing your Skills\' .lu files (defaults to \'./deployment/resources/skills/en\' inside your assistant folder)') @@ -58,7 +58,7 @@ if (process.argv.length < 3) { } logger.isVerbose = args.verbose; -let noTrain: boolean = false; +let noRefresh: boolean = false; // Validation of arguments // cs and ts validation @@ -73,9 +73,9 @@ if (csAndTsValidationResult) { const projectLanguage: string = args.cs ? 'cs' : 'ts'; -// noTrain validation -if (args.noTrain) { - noTrain = true; +// noRefresh validation +if (args.noRefresh) { + noRefresh = true; } // botName validation @@ -103,7 +103,7 @@ const configuration: Partial = { botName: args.botName, localManifest: args.localManifest, remoteManifest: args.remoteManifest, - noTrain: noTrain, + noRefresh: noRefresh, lgLanguage: projectLanguage }; diff --git a/lib/typescript/botskills/src/botskills-disconnect.ts b/lib/typescript/botskills/src/botskills-disconnect.ts index 7c2113edec..bf43363e2e 100644 --- a/lib/typescript/botskills/src/botskills-disconnect.ts +++ b/lib/typescript/botskills/src/botskills-disconnect.ts @@ -34,7 +34,7 @@ program .option('-i, --skillId ', 'Id of the skill to remove from your assistant (case sensitive)') .option('--cs', 'Determine your assistant project structure to be a CSharp-like structure') .option('--ts', 'Determine your assistant project structure to be a TypeScript-like structure') - .option('--noTrain', '[OPTIONAL] Determine whether the skills connected are not going to be trained (by default they are trained)') + .option('--noRefresh', '[OPTIONAL] Determine whether the model of your skills connected are not going to be refreshed (by default they are refreshed)') .option('--dispatchName [name]', '[OPTIONAL] Name of your assistant\'s \'.dispatch\' file (defaults to the name displayed in your Cognitive Models file)') .option('--dispatchFolder [path]', '[OPTIONAL] Path to the folder containing your assistant\'s \'.dispatch\' file (defaults to \'./deployment/resources/dispatch/en\' inside your assistant folder)') .option('--outFolder [path]', '[OPTIONAL] Path for any output file that may be generated (defaults to your assistant\'s root folder)') @@ -51,7 +51,7 @@ if (process.argv.length < 3) { } logger.isVerbose = args.verbose; -let noTrain: boolean = false; +let noRefresh: boolean = false; // Validation of arguments // cs and ts validation @@ -65,9 +65,9 @@ if (csAndTsValidationResult) { } const projectLanguage: string = args.cs ? 'cs' : 'ts'; -// noTrain validation -if (args.noTrain) { - noTrain = true; +// noRefresh validation +if (args.noRefresh) { + noRefresh = true; } // skillId validation @@ -79,7 +79,7 @@ if (!args.skillId) { const configuration: Partial = { skillId: args.skillId, lgLanguage: projectLanguage, - noTrain: noTrain, + noRefresh: noRefresh, logger: logger }; diff --git a/lib/typescript/botskills/src/botskills-refresh.ts b/lib/typescript/botskills/src/botskills-refresh.ts new file mode 100644 index 0000000000..6da167017e --- /dev/null +++ b/lib/typescript/botskills/src/botskills-refresh.ts @@ -0,0 +1,102 @@ +/** + * Copyright(c) Microsoft Corporation.All rights reserved. + * Licensed under the MIT License. + */ + +import * as program from 'commander'; +import { join, resolve } from 'path'; +import { RefreshSkill } from './functionality'; +import { ConsoleLogger, ILogger} from './logger'; +import { ICognitiveModelFile, IRefreshConfiguration } from './models'; +import { validatePairOfArgs } from './utils'; + +function showErrorHelp(): void { + program.outputHelp((str: string) => { + logger.error(str); + + return ''; + }); + process.exit(1); +} + +const logger: ILogger = new ConsoleLogger(); + +program.Command.prototype.unknownOption = (flag: string): void => { + logger.error(`Unknown arguments: ${flag}`); + showErrorHelp(); +}; + +// tslint:disable: max-line-length +program + .name('botskills refresh') + .description('Refresh the connected skills.') + .option('--cs', 'Determine your assistant project structure to be a CSharp-like structure') + .option('--ts', 'Determine your assistant project structure to be a TypeScript-like structure') + .option('--dispatchName [name]', '[OPTIONAL] Name of your assistant\'s \'.dispatch\' file (defaults to the name displayed in your Cognitive Models file)') + .option('--language [language]', '[OPTIONAL] Locale used for LUIS culture (defaults to \'en-us\')') + .option('--luisFolder [path]', '[OPTIONAL] Path to the folder containing your Skills\' .lu files (defaults to \'./deployment/resources/skills/en\' inside your assistant folder)') + .option('--dispatchFolder [path]', '[OPTIONAL] Path to the folder containing your assistant\'s \'.dispatch\' file (defaults to \'./deployment/resources/dispatch/en\' inside your assistant folder)') + .option('--outFolder [path]', '[OPTIONAL] Path for any output file that may be generated (defaults to your assistant\'s root folder)') + .option('--lgOutFolder [path]', '[OPTIONAL] Path for the LuisGen output (defaults to a \'service\' folder inside your assistant\'s folder)') + .option('--cognitiveModelsFile [path]', '[OPTIONAL] Path to your Cognitive Models file (defaults to \'cognitivemodels.json\' inside your assistant\'s folder)') + .option('--verbose', '[OPTIONAL] Output detailed information about the processing of the tool') + .action((cmd: program.Command, actions: program.Command) => undefined); + +const args: program.Command = program.parse(process.argv); + +if (process.argv.length < 3) { + program.help(); +} + +logger.isVerbose = args.verbose; + +// Validation of arguments +// cs and ts validation +const csAndTsValidationResult: string = validatePairOfArgs(args.cs, args.ts); +if (csAndTsValidationResult) { + logger.error( + csAndTsValidationResult.replace('{0}', 'cs') + .replace('{1}', 'ts') + ); + process.exit(1); +} + +const projectLanguage: string = args.cs ? 'cs' : 'ts'; +// Initialize an instance of IConnectConfiguration to send the needed arguments to the connectSkill function +const configuration: Partial = { + lgLanguage: projectLanguage +}; + +// language validation +const language: string = args.language || 'en-us'; +configuration.language = language; +const languageCode: string = (language.split('-'))[0]; + +// outFolder validation -- the const is needed for reassuring 'configuration.outFolder' is not undefined +const outFolder: string = args.outFolder || resolve('./'); +configuration.outFolder = outFolder; + +// cognitiveModelsFile validation +const cognitiveModelsFilePath: string = args.cognitiveModelsFile || join(configuration.outFolder, (args.ts ? join('src', 'cognitivemodels.json') : 'cognitivemodels.json')); +configuration.cognitiveModelsFile = cognitiveModelsFilePath; +// luisFolder validation +configuration.luisFolder = args.luisFolder || join(configuration.outFolder, 'Deployment', 'Resources', 'Skills', languageCode); + +// dispatchFolder validation +configuration.dispatchFolder = args.dispatchFolder || join(configuration.outFolder, 'Deployment', 'Resources', 'Dispatch', languageCode); + +// lgOutFolder validation +configuration.lgOutFolder = args.lgOutFolder || join(configuration.outFolder, (args.ts ? join('src', 'Services') : 'Services')); + +// dispatchName validation +if (!args.dispatchName) { + // try get the dispatch name from the cognitiveModels file + // tslint:disable-next-line + const cognitiveModelsFile: ICognitiveModelFile = require(cognitiveModelsFilePath); + configuration.dispatchName = cognitiveModelsFile.cognitiveModels[languageCode].dispatchModel.name; +} + +configuration.logger = logger; +// End of arguments validation + +new RefreshSkill(logger).refreshSkill( configuration); diff --git a/lib/typescript/botskills/src/botskills.ts b/lib/typescript/botskills/src/botskills.ts index 433a8087b2..91a0237a32 100644 --- a/lib/typescript/botskills/src/botskills.ts +++ b/lib/typescript/botskills/src/botskills.ts @@ -38,6 +38,7 @@ program program .command('connect', 'connect any skill to your assistant bot') .command('disconnect', 'disconnect a specific skill from your assitant bot') + .command('refresh', 'refresh the connected skills') .command('list', 'list the connected skills in the assistant'); const args: program.Command = program.parse(process.argv); diff --git a/lib/typescript/botskills/src/functionality/connectSkill.ts b/lib/typescript/botskills/src/functionality/connectSkill.ts index d0af0414b1..784f768364 100644 --- a/lib/typescript/botskills/src/functionality/connectSkill.ts +++ b/lib/typescript/botskills/src/functionality/connectSkill.ts @@ -7,17 +7,20 @@ import { existsSync, readFileSync, writeFileSync } from 'fs'; import { isAbsolute, join, resolve } from 'path'; import { get } from 'request-promise-native'; import { ConsoleLogger, ILogger} from '../logger'; -import { IAction, IConnectConfiguration, ISkillFIle, ISkillManifest, IUtteranceSource } from '../models'; +import { IAction, IConnectConfiguration, IRefreshConfiguration, ISkillFIle, ISkillManifest, IUtteranceSource } from '../models'; import { AuthenticationUtils, ChildProcessUtils } from '../utils'; +import { RefreshSkill } from './refreshSkill'; export class ConnectSkill { private logger: ILogger; private childProcessUtils: ChildProcessUtils; + private refreshSkill: RefreshSkill; private authenticationUtils: AuthenticationUtils; constructor(logger: ILogger) { this.logger = logger || new ConsoleLogger(); this.childProcessUtils = new ChildProcessUtils(); + this.refreshSkill = new RefreshSkill(this.logger); this.authenticationUtils = new AuthenticationUtils(); } @@ -82,14 +85,11 @@ export class ConnectSkill { } } - // tslint:disable-next-line:max-func-body-length public async updateDispatch(configuration: IConnectConfiguration, manifest: ISkillManifest): Promise { try { // Initializing variables for the updateDispatch scope const dispatchFile: string = `${configuration.dispatchName}.dispatch`; - const dispatchJsonFile: string = `${configuration.dispatchName}.json`; const dispatchFilePath: string = join(configuration.dispatchFolder, dispatchFile); - const dispatchJsonFilePath: string = join(configuration.dispatchFolder, dispatchJsonFile); const intentName: string = manifest.id; let luisDictionary: Map; @@ -161,29 +161,14 @@ export class ConnectSkill { await this.runCommand(dispatchAddCommand, `Executing dispatch add for the ${luisApp} LU file`); })); - // Check if it is necessary to train the skill - if (!configuration.noTrain) { - this.logger.message('Running dispatch refresh...'); - const dispatchRefreshCommand: string[] = ['dispatch', 'refresh']; - dispatchRefreshCommand.push(...['--dispatch', dispatchFilePath]); - dispatchRefreshCommand.push(...['--dataFolder', configuration.dispatchFolder]); - - this.logger.message(await this.runCommand( - dispatchRefreshCommand, - `Executing dispatch refresh for the ${configuration.dispatchName} file`)); - - if (!existsSync(dispatchJsonFilePath)) { - // tslint:disable-next-line: max-line-length - throw(new Error(`Path to ${dispatchJsonFile} (${dispatchJsonFilePath}) leads to a nonexistent file. Make sure the dispatch refresh command is being executed successfully`)); + // Check if it is necessary to refresh the skill + if (!configuration.noRefresh) { + const refreshConfiguration: IRefreshConfiguration = {...{}, ...configuration}; + if (!await this.refreshSkill.refreshSkill(refreshConfiguration)) { + throw new Error(`There was an error while refreshing the Dispatch model.`); } - - this.logger.message('Running LuisGen...'); - const luisgenCommand: string[] = ['luisgen']; - luisgenCommand.push(dispatchJsonFilePath); - luisgenCommand.push(...[`-${configuration.lgLanguage}`, `"DispatchLuis"`]); - luisgenCommand.push(...['-o', configuration.lgOutFolder]); - - await this.runCommand(luisgenCommand, `Executing luisgen for the ${configuration.dispatchName} file`); + } else { + this.logger.warning(`Run 'botskills refresh --${configuration.lgLanguage}' command to refresh your connected skills`); } this.logger.success('Successfully updated Dispatch model'); } catch (err) { @@ -213,7 +198,7 @@ export class ConnectSkill { // Take VA Skills configurations //tslint:disable-next-line: no-var-requires non-literal-require - const assistantSkillsFile: ISkillFIle = require(configuration.skillsFile); + const assistantSkillsFile: ISkillFIle = JSON.parse(readFileSync(configuration.skillsFile, 'UTF8')); const assistantSkills: ISkillManifest[] = assistantSkillsFile.skills || []; // Check if the skill is already connected to the assistant diff --git a/lib/typescript/botskills/src/functionality/disconnectSkill.ts b/lib/typescript/botskills/src/functionality/disconnectSkill.ts index 6e6ffa87c3..003d89feb7 100644 --- a/lib/typescript/botskills/src/functionality/disconnectSkill.ts +++ b/lib/typescript/botskills/src/functionality/disconnectSkill.ts @@ -6,38 +6,23 @@ import { existsSync, readFileSync, writeFileSync } from 'fs'; import { join } from 'path'; import { ConsoleLogger, ILogger } from '../logger'; -import { IDisconnectConfiguration, IDispatchFile, IDispatchService, ISkillFIle, ISkillManifest } from '../models/'; -import { ChildProcessUtils } from '../utils'; +import { IDisconnectConfiguration, IDispatchFile, IDispatchService, IRefreshConfiguration, ISkillFIle, ISkillManifest } from '../models/'; +import { RefreshSkill } from './refreshSkill'; export class DisconnectSkill { public logger: ILogger; - private childProcessUtils: ChildProcessUtils; + private refreshSkill: RefreshSkill; + constructor(logger?: ILogger) { this.logger = logger || new ConsoleLogger(); - this.childProcessUtils = new ChildProcessUtils(); - } - private async runCommand(command: string[], description: string): Promise { - this.logger.command(description, command.join(' ')); - const cmd: string = command[0]; - const commandArgs: string[] = command.slice(1) - .filter((arg: string) => arg); - - try { - return await this.childProcessUtils.execute(cmd, commandArgs); - } catch (err) { - - throw err; - } + this.refreshSkill = new RefreshSkill(this.logger); } public async updateDispatch(configuration: IDisconnectConfiguration): Promise { try { // Initializing variables for the updateDispatch scope const dispatchFile: string = `${configuration.dispatchName}.dispatch`; - const dispatchJsonFile: string = `${configuration.dispatchName}.json`; const dispatchFilePath: string = join(configuration.dispatchFolder, dispatchFile); - const dispatchJsonFilePath: string = join(configuration.dispatchFolder, dispatchJsonFile); - this.logger.message('Removing skill from dispatch...'); // dispatch remove(?) @@ -67,28 +52,14 @@ Run 'botskills list --assistantSkills ""' in or 1); writeFileSync(dispatchFilePath, JSON.stringify(dispatchData, undefined, 4)); - // Check if it is necessary to train the skill - if (!configuration.noTrain) { - this.logger.message('Running Dispatch refresh'); - const dispatchRefreshCommand: string[] = ['dispatch', 'refresh']; - dispatchRefreshCommand.push(...['--dispatch', dispatchFilePath]); - dispatchRefreshCommand.push(...['--dataFolder', configuration.dispatchFolder]); - await this.runCommand(dispatchRefreshCommand, `Executing dispatch refresh for the ${configuration.dispatchName} file`); - - if (!existsSync(dispatchJsonFilePath)) { - // this.logger.error(`Path to ${dispatchJsonFile} (${dispatchJsonFilePath}) leads - // to a nonexistent file. Make sure the dispatch refresh command is being executed successfully`); - // tslint:disable-next-line: max-line-length - throw(new Error(`Path to ${dispatchJsonFile} (${dispatchJsonFilePath}) leads to a nonexistent file. Make sure the dispatch refresh command is being executed successfully`)); + // Check if it is necessary to refresh the skill + if (!configuration.noRefresh) { + const refreshConfiguration: IRefreshConfiguration = {...{}, ...configuration}; + if (!await this.refreshSkill.refreshSkill(refreshConfiguration)) { + throw new Error(`There was an error while refreshing the Dispatch model.`); } - - this.logger.message('Running LuisGen...'); - - const luisgenCommand: string[] = ['luisgen']; - luisgenCommand.push(dispatchJsonFilePath); - luisgenCommand.push(...[`-${configuration.lgLanguage}`, '"DispatchLuis"']); - luisgenCommand.push(...['-o', configuration.lgOutFolder]); - await this.runCommand(luisgenCommand, `Executing luisgen for the ${configuration.dispatchName} file`); + } else { + this.logger.warning(`Run 'botskills refresh --${configuration.lgLanguage}' command to refresh your connected skills`); } return true; diff --git a/lib/typescript/botskills/src/functionality/index.ts b/lib/typescript/botskills/src/functionality/index.ts index 7e639c76f5..b5427f0763 100644 --- a/lib/typescript/botskills/src/functionality/index.ts +++ b/lib/typescript/botskills/src/functionality/index.ts @@ -6,3 +6,4 @@ export { ConnectSkill } from './connectSkill'; export { DisconnectSkill } from './disconnectSkill'; export { ListSkill } from './listSkill'; +export { RefreshSkill } from './refreshSkill'; diff --git a/lib/typescript/botskills/src/functionality/refreshSkill.ts b/lib/typescript/botskills/src/functionality/refreshSkill.ts new file mode 100644 index 0000000000..5d7c938362 --- /dev/null +++ b/lib/typescript/botskills/src/functionality/refreshSkill.ts @@ -0,0 +1,96 @@ +/** + * Copyright(c) Microsoft Corporation.All rights reserved. + * Licensed under the MIT License. + */ +import { existsSync } from 'fs'; +import { join } from 'path'; +import { ConsoleLogger, ILogger } from '../logger'; +import { IRefreshConfiguration } from '../models'; +import { ChildProcessUtils } from '../utils'; + +export class RefreshSkill { + public logger: ILogger; + private childProcessUtils: ChildProcessUtils; + private dispatchFile: string = ''; + private dispatchJsonFile: string = ''; + private dispatchFilePath: string = ''; + private dispatchJsonFilePath: string = ''; + + constructor(logger?: ILogger) { + this.logger = logger || new ConsoleLogger(); + this.childProcessUtils = new ChildProcessUtils(); + } + + private async runCommand(command: string[], description: string): Promise { + this.logger.command(description, command.join(' ')); + const cmd: string = command[0]; + const commandArgs: string[] = command.slice(1) + .filter((arg: string) => arg); + try { + return await this.childProcessUtils.execute(cmd, commandArgs); + } catch (err) { + throw err; + } + } + + private async updateDispatch(configuration: IRefreshConfiguration): Promise { + try { + this.logger.message('Running dispatch refresh...'); + + const dispatchRefreshCommand: string[] = ['dispatch', 'refresh']; + dispatchRefreshCommand.push(...['--dispatch', this.dispatchFilePath]); + dispatchRefreshCommand.push(...['--dataFolder', configuration.dispatchFolder]); + + await this.runCommand( + dispatchRefreshCommand, + `Executing dispatch refresh for the ${configuration.dispatchName} file`); + + if (!existsSync(this.dispatchJsonFilePath)) { + // tslint:disable-next-line: max-line-length + throw new Error(`Path to ${this.dispatchJsonFile} (${this.dispatchJsonFilePath}) leads to a nonexistent file. Make sure the dispatch refresh command is being executed successfully`); + } + } catch (err) { + throw new Error(`There was an error in the dispatch refresh command:\n${err}`); + } + } + + private async runLuisGen(configuration: IRefreshConfiguration): Promise { + try { + this.logger.message('Running LuisGen...'); + + const luisgenCommand: string[] = ['luisgen']; + luisgenCommand.push(this.dispatchJsonFilePath); + luisgenCommand.push(...[`-${configuration.lgLanguage}`, `"DispatchLuis"`]); + luisgenCommand.push(...['-o', configuration.lgOutFolder]); + + await this.runCommand(luisgenCommand, `Executing luisgen for the ${configuration.dispatchName} file`); + } catch (err) { + throw new Error(`There was an error in the luisgen command:\n${err}`); + } + } + + public async refreshSkill(configuration: IRefreshConfiguration): Promise { + try { + this.dispatchFile = `${configuration.dispatchName}.dispatch`; + this.dispatchJsonFile = `${configuration.dispatchName}.json`; + this.dispatchFilePath = join(configuration.dispatchFolder, this.dispatchFile); + this.dispatchJsonFilePath = join(configuration.dispatchFolder, this.dispatchJsonFile); + + if (!existsSync(configuration.dispatchFolder)) { + throw(new Error(`Path to the Dispatch folder (${configuration.dispatchFolder}) leads to a nonexistent folder.`)); + } else if (!existsSync(this.dispatchFilePath)) { + throw(new Error(`Path to the ${this.dispatchFile} file leads to a nonexistent file.`)); + } + + await this.updateDispatch(configuration); + await this.runLuisGen(configuration); + this.logger.success('Successfully refreshed Dispatch model'); + + return true; + } catch (err) { + this.logger.error(`There was an error while refreshing any Skill from the Assistant:\n${err}`); + + return false; + } + } +} diff --git a/lib/typescript/botskills/src/models/connectConfiguration.ts b/lib/typescript/botskills/src/models/connectConfiguration.ts index 31cb7cd4b7..28e275076e 100644 --- a/lib/typescript/botskills/src/models/connectConfiguration.ts +++ b/lib/typescript/botskills/src/models/connectConfiguration.ts @@ -9,7 +9,7 @@ export interface IConnectConfiguration { botName: string; localManifest: string; remoteManifest: string; - noTrain: boolean; + noRefresh: boolean; dispatchName: string; language: string; luisFolder: string; diff --git a/lib/typescript/botskills/src/models/disconnectConfiguration.ts b/lib/typescript/botskills/src/models/disconnectConfiguration.ts index 9c9380706f..d1b81f765a 100644 --- a/lib/typescript/botskills/src/models/disconnectConfiguration.ts +++ b/lib/typescript/botskills/src/models/disconnectConfiguration.ts @@ -9,7 +9,7 @@ export interface IDisconnectConfiguration { skillId: string; skillsFile: string; outFolder: string; - noTrain: boolean; + noRefresh: boolean; cognitiveModelsFile: string; language: string; luisFolder: string; diff --git a/lib/typescript/botskills/src/models/index.ts b/lib/typescript/botskills/src/models/index.ts index 863e9e1963..ac46d81c3a 100644 --- a/lib/typescript/botskills/src/models/index.ts +++ b/lib/typescript/botskills/src/models/index.ts @@ -26,3 +26,4 @@ export { ITriggers, IUtterance, IUtteranceSource } from './skillManifest'; +export { IRefreshConfiguration } from './refreshConfiguration'; diff --git a/lib/typescript/botskills/src/models/refreshConfiguration.ts b/lib/typescript/botskills/src/models/refreshConfiguration.ts new file mode 100644 index 0000000000..ae0e2dd5fa --- /dev/null +++ b/lib/typescript/botskills/src/models/refreshConfiguration.ts @@ -0,0 +1,18 @@ +/** + * Copyright(c) Microsoft Corporation.All rights reserved. + * Licensed under the MIT License. + */ + +import { ILogger } from '../logger'; + +export interface IRefreshConfiguration { + dispatchName: string; + dispatchFolder: string; + language: string; + luisFolder: string; + lgLanguage: string; + outFolder: string; + lgOutFolder: string; + cognitiveModelsFile: string; + logger?: ILogger; +} diff --git a/lib/typescript/botskills/test/flow/connectTest.js b/lib/typescript/botskills/test/flow/connectTest.js index 338bc98605..f43790fe99 100644 --- a/lib/typescript/botskills/test/flow/connectTest.js +++ b/lib/typescript/botskills/test/flow/connectTest.js @@ -11,27 +11,35 @@ const testLogger = require('../models/testLogger'); const botskills = require('../../lib/index'); let logger; let connector; +const filledSkillsArray = JSON.stringify( + { + "skills": [ + { + "id": "testSkill" + }, + { + "id": "testDispatch" + } + ] + }, + null, 4); + describe("The connect command", function () { beforeEach(function () { - fs.writeFileSync(path.resolve(__dirname, path.join('..', 'mockedFiles', 'filledSkillsArray.json')), - JSON.stringify( - { - "skills": [ - { - "id": "testSkill" - }, - { - "id": "testDispatch" - } - ] - }, - null, 4)); + fs.writeFileSync(path.resolve(__dirname, path.join('..', 'mockedFiles', 'filledSkillsArray.json')), filledSkillsArray); + fs.writeFileSync(path.resolve(__dirname, path.join('..', 'mockedFiles', 'successfulConnectFiles', 'filledSkillsArray.json')), filledSkillsArray); + logger = new testLogger.TestLogger(); connector = new botskills.ConnectSkill(logger); }) + after(function () { + fs.writeFileSync(path.resolve(__dirname, path.join('..', 'mockedFiles', 'filledSkillsArray.json')), filledSkillsArray); + fs.writeFileSync(path.resolve(__dirname, path.join('..', 'mockedFiles', 'successfulConnectFiles', 'filledSkillsArray.json')), filledSkillsArray); + }) + describe("should show an error", function () { it("when there is no skills File", async function () { const config = { @@ -99,11 +107,11 @@ describe("The connect command", function () { logger: logger }; const errorMessages = [ -`Missing property 'name' of the manifest`, -`Missing property 'id' of the manifest`, -`Missing property 'endpoint' of the manifest`, -`Missing property 'authenticationConnections' of the manifest`, -`Missing property 'actions' of the manifest` + `Missing property 'name' of the manifest`, + `Missing property 'id' of the manifest`, + `Missing property 'endpoint' of the manifest`, + `Missing property 'authenticationConnections' of the manifest`, + `Missing property 'actions' of the manifest` ] await connector.connectSkill(config); const ErrorList = logger.getError(); @@ -277,7 +285,7 @@ describe("The connect command", function () { assert.strictEqual(ErrorList[ErrorList.length - 1], `There was an error while connecting the Skill to the Assistant:\nError: An error ocurred while updating the Dispatch model:\nError: Path to ${config.dispatchName}.luis (${path.join(config.luisFolder, config.dispatchName)}.luis) leads to a nonexistent file. Make sure the ludown command is being executed successfully`); }); - it("when the dispatch refresh fails and the dispatch .json file is missing", async function () { + it("when the refreshSkill fails", async function () { const config = { botName: '', localManifest: path.resolve(__dirname, path.join('..', 'mockedFiles', 'connectableSkill.json')), @@ -298,9 +306,14 @@ describe("The connect command", function () { sandbox.replace(connector.childProcessUtils, 'execute', (command, args) => { return Promise.resolve('Mocked function successfully'); }); + sandbox.replace(connector.refreshSkill, 'refreshSkill', (command, args) => { + return Promise.resolve(false); + }); await connector.connectSkill(config); const ErrorList = logger.getError(); - assert.strictEqual(ErrorList[ErrorList.length - 1], `There was an error while connecting the Skill to the Assistant:\nError: An error ocurred while updating the Dispatch model:\nError: Path to ${config.dispatchName}.json (${path.join(config.dispatchFolder, config.dispatchName)}.json) leads to a nonexistent file. Make sure the dispatch refresh command is being executed successfully`); + assert.strictEqual(ErrorList[ErrorList.length - 1], `There was an error while connecting the Skill to the Assistant: +Error: An error ocurred while updating the Dispatch model: +Error: There was an error while refreshing the Dispatch model.`); }); }); @@ -328,13 +341,46 @@ describe("The connect command", function () { assert.strictEqual(WarningList[WarningList.length - 1], `The skill 'Test Skill' is already registered.`); }); + it("when the noRefresh argument is present", async function () { + const config = { + botName: '', + localManifest: path.resolve(__dirname, path.join('..', 'mockedFiles', 'connectableSkill.json')), + remoteManifest: '', + noRefresh: true, + dispatchName: 'connectableSkill', + language: '', + luisFolder: path.resolve(__dirname, path.join('..', 'mockedFiles', 'successfulConnectFiles')), + dispatchFolder: path.resolve(__dirname, path.join('..', 'mockedFiles', 'successfulConnectFiles')), + outFolder: '', + lgOutFolder: '', + skillsFile: path.resolve(__dirname, path.join('..', 'mockedFiles', 'filledSkillsArray.json')), + resourceGroup: '', + appSettingsFile: '', + cognitiveModelsFile: '', + lgLanguage: '', + logger: logger + }; + sandbox.replace(connector.childProcessUtils, 'execute', (command, args) => { + return Promise.resolve('Mocked function successfully'); + }); + sandbox.replace(connector.authenticationUtils, 'authenticate', (configuration, manifest, logger) => { + return Promise.resolve('Mocked function successfully'); + }); + sandbox.replace(connector.refreshSkill, 'refreshSkill', (command, args) => { + return Promise.resolve(true); + }); + await connector.connectSkill(config); + const WarningList = logger.getWarning(); + assert.strictEqual(WarningList[WarningList.length - 1], `Appending 'Test Skill' manifest to your assistant's skills configuration file.`); + assert.strictEqual(WarningList[WarningList.length - 2], `Run 'botskills refresh --${config.lgLanguage}' command to refresh your connected skills`); + }); }); describe("should show a message", function () { it("when the skill is successfully connected to the Assistant", async function () { const config = { botName: '', - localManifest: path.resolve(__dirname, path.join('..', 'mockedFiles', 'connectableSkill.json')), + localManifest: path.resolve(__dirname, path.join('..', 'mockedFiles', 'successfulConnectFiles', 'connectableSkill.json')), remoteManifest: '', dispatchName: 'connectableSkill', language: '', @@ -342,7 +388,7 @@ describe("The connect command", function () { dispatchFolder: path.resolve(__dirname, path.join('..', 'mockedFiles', 'successfulConnectFiles')), outFolder: '', lgOutFolder: '', - skillsFile: path.resolve(__dirname, path.join('..', 'mockedFiles', 'filledSkillsArray.json')), + skillsFile: path.resolve(__dirname, path.join('..', 'mockedFiles', 'successfulConnectFiles', 'filledSkillsArray.json')), resourceGroup: '', appSettingsFile: '', cognitiveModelsFile: '', @@ -355,6 +401,9 @@ describe("The connect command", function () { sandbox.replace(connector.authenticationUtils, 'authenticate', (configuration, manifest, logger) => { return Promise.resolve('Mocked function successfully'); }); + sandbox.replace(connector.refreshSkill, 'refreshSkill', (command, args) => { + return Promise.resolve(true); + }); await connector.connectSkill(config); const MessageList = logger.getMessage(); assert.strictEqual(MessageList[MessageList.length - 1], `Configuring bot auth settings`); diff --git a/lib/typescript/botskills/test/flow/disconnectTest.js b/lib/typescript/botskills/test/flow/disconnectTest.js index 3418fa563c..3e545ab683 100644 --- a/lib/typescript/botskills/test/flow/disconnectTest.js +++ b/lib/typescript/botskills/test/flow/disconnectTest.js @@ -6,17 +6,21 @@ const assert = require('assert'); const fs = require('fs'); const path = require('path'); -const sinon = require('sinon'); const sandbox = require('sinon').createSandbox(); const testLogger = require('../models/testLogger'); const botskills = require('../../lib/index'); -let logger; +let refreshSkillStub; describe("The disconnect command", function () { - + afterEach(function (){ + sandbox.restore(); + }); + beforeEach(function () { this.logger = new testLogger.TestLogger(); disconnector = new botskills.DisconnectSkill(this.logger); + refreshSkillStub = sandbox.stub(disconnector.refreshSkill, "refreshSkill"); + refreshSkillStub.returns(Promise.resolve('Mocked function successfully')); fs.writeFileSync(path.resolve(__dirname, path.join('..', 'mockedFiles', 'filledDispatch.dispatch')), JSON.stringify( { @@ -121,7 +125,7 @@ describe("The disconnect command", function () { assert.strictEqual(ErrorList[ErrorList.length - 1], `Could not find file ${config.dispatchName}.dispatch. Please provide the 'dispatchName' and 'dispatchFolder' parameters.`); }); - it("when the dispatch refresh fails to create the Dispatch JSON file", async function () { + it("when the refreshSkill fails", async function () { const config = { skillId : "testDispatch", skillsFile: path.resolve(__dirname, '../mockedFiles/filledSkillsArray.json'), @@ -136,12 +140,14 @@ describe("The disconnect command", function () { logger : this.logger }; - sandbox.replace(disconnector.childProcessUtils, 'execute', (command, args) => { - return Promise.resolve('Mocked function successfully'); + sandbox.replace(disconnector.refreshSkill, 'refreshSkill', (command, args) => { + return Promise.resolve(false); }); await disconnector.disconnectSkill(config); const ErrorList = this.logger.getError(); - assert.strictEqual(ErrorList[ErrorList.length - 1], `There was an error while disconnecting the Skill ${config.skillId} from the Assistant:\nError: An error ocurred while updating the Dispatch model:\nError: Path to ${config.dispatchName}.json (${path.join(config.dispatchFolder, config.dispatchName)}.json) leads to a nonexistent file. Make sure the dispatch refresh command is being executed successfully`); + assert.strictEqual(ErrorList[ErrorList.length - 1], `There was an error while disconnecting the Skill ${config.skillId} from the Assistant: +Error: An error ocurred while updating the Dispatch model: +Error: There was an error while refreshing the Dispatch model.`); }); it("when the lgOutFolder argument is invalid ", async function () { @@ -159,9 +165,6 @@ describe("The disconnect command", function () { logger : this.logger }; - sandbox.replace(disconnector.childProcessUtils, 'execute', (command, args) => { - return Promise.resolve('Mocked function successfully'); - }); await disconnector.disconnectSkill(config); const ErrorList = this.logger.getError(); assert.strictEqual(ErrorList[ErrorList.length - 1], `The 'lgOutFolder' argument is absent or leads to a non-existing folder.\nPlease make sure to provide a valid path to your LUISGen output folder.`); @@ -182,15 +185,13 @@ describe("The disconnect command", function () { logger : this.logger }; - sandbox.replace(disconnector.childProcessUtils, 'execute', (command, args) => { - return Promise.resolve('Mocked function successfully'); - }); await disconnector.disconnectSkill(config); const ErrorList = this.logger.getError(); assert.strictEqual(ErrorList[ErrorList.length - 1], `The 'lgLanguage' argument is incorrect.\nIt should be either 'cs' or 'ts' depending on your assistant's language.`); }); it("when the execution of a command fails", async function () { + refreshSkillStub.returns(Promise.reject(new Error('Mocked function throws an Error'))); const config = { skillId : "testDispatch", skillsFile: path.resolve(__dirname, '../mockedFiles/filledSkillsArray.json'), @@ -205,9 +206,6 @@ describe("The disconnect command", function () { logger : this.logger }; - sandbox.replace(disconnector.childProcessUtils, 'execute', (command, args) => { - return Promise.reject(new Error('Mocked function throws an Error')); - }); await disconnector.disconnectSkill(config); const ErrorList = this.logger.getError(); assert.strictEqual(ErrorList[ErrorList.length - 1], `There was an error while disconnecting the Skill ${config.skillId} from the Assistant:\nError: An error ocurred while updating the Dispatch model:\nError: Mocked function throws an Error`); @@ -277,9 +275,6 @@ describe("The disconnect command", function () { logger : this.logger }; - sandbox.replace(disconnector.childProcessUtils, 'execute', (command, args) => { - return Promise.resolve('Mocked function successfully'); - }); await disconnector.disconnectSkill(config); const SuccessList = this.logger.getSuccess(); assert.strictEqual(SuccessList[SuccessList.length - 1], `Successfully removed '${config.skillId}' skill from your assistant's skills configuration file.`); diff --git a/lib/typescript/botskills/test/flow/refreshTest.js b/lib/typescript/botskills/test/flow/refreshTest.js new file mode 100644 index 0000000000..db995ff1cd --- /dev/null +++ b/lib/typescript/botskills/test/flow/refreshTest.js @@ -0,0 +1,113 @@ +/** + * Copyright(c) Microsoft Corporation.All rights reserved. + * Licensed under the MIT License. + */ + +const assert = require("assert"); +const { join, resolve } = require("path"); +const sandbox = require('sinon').createSandbox(); +const testLogger = require("../models/testLogger"); +const botskills = require("../../lib/index"); +let childProcessStub; + +describe("The refresh command", function () { + afterEach(function (){ + sandbox.restore(); + }); + + beforeEach(function () { + this.logger = new testLogger.TestLogger(); + this.refresher = new botskills.RefreshSkill(this.logger); + }); + + describe("should show an error", function() { + it("when the dispatchFolder points to a nonexistent folder", async function () { + const config = { + dispatchName : "", + dispatchFolder : resolve(__dirname, "../nonexistentDispatchFolder"), + language: "ts", + luisFolder: "", + lgLanguage: "cs", + outFolder: "", + lgOutFolder: resolve(__dirname, "../mockedFiles"), + cognitiveModelsFile: "", + logger: this.logger + }; + + await this.refresher.refreshSkill(config); + const ErrorList = this.logger.getError(); + assert.strictEqual(ErrorList[ErrorList.length - 1], `There was an error while refreshing any Skill from the Assistant: +Error: Path to the Dispatch folder (${config.dispatchFolder}) leads to a nonexistent folder.`); + }); + + it("when the dispatchName points to a nonexistent file", async function () { + const config = { + dispatchName : "filledDispatchNoJson", + dispatchFolder : resolve(__dirname, join("..", "mockedFiles", "dispatchFolder")), + language: "ts", + luisFolder: "", + lgLanguage: "cs", + outFolder: "", + lgOutFolder: resolve(__dirname, "../mockedFiles"), + cognitiveModelsFile: "", + logger: this.logger + }; + + await this.refresher.refreshSkill(config); + const ErrorList = this.logger.getError(); + assert.strictEqual(ErrorList[ErrorList.length - 1], `There was an error while refreshing any Skill from the Assistant: +Error: Path to the ${config.dispatchName}.dispatch file leads to a nonexistent file.`); + }); + + it("when the external calls fails", async function () { + sandbox.replace(this.refresher, 'updateDispatch', (configuration) => { + return Promise.resolve("Mocked function successfully"); + }); + + sandbox.replace(this.refresher.childProcessUtils, 'execute', (command, args) => { + return Promise.reject(new Error("Mocked function throws an Error")); + }); + + const config = { + dispatchName: "connectableSkill", + dispatchFolder : resolve(__dirname, join("..", "mockedFiles", "dispatchFolder")), + language: "ts", + luisFolder: "", + lgLanguage: "cs", + outFolder: "", + lgOutFolder: "", + cognitiveModelsFile: "", + logger: this.logger + }; + + await this.refresher.refreshSkill(config); + const ErrorList = this.logger.getError(); + assert.strictEqual(ErrorList[ErrorList.length - 1], `There was an error while refreshing any Skill from the Assistant: +Error: There was an error in the luisgen command: +Error: Mocked function throws an Error`); + }); + }); + + describe("should show a successfully message", function() { + it("when the refreshSkill is executed successfully", async function () { + const config = { + dispatchName : "connectableSkill", + dispatchFolder : resolve(__dirname, join("..", "mockedFiles", "successfulConnectFiles")), + language: "ts", + luisFolder: "", + lgLanguage: "cs", + outFolder: "", + lgOutFolder: resolve(__dirname, "../mockedFiles"), + cognitiveModelsFile: "", + logger: this.logger + }; + + sandbox.replace(this.refresher.childProcessUtils, 'execute', (command, args) => { + return Promise.resolve('Mocked function successfully'); + }); + await this.refresher.refreshSkill(config); + const SuccessList = this.logger.getSuccess(); + assert.strictEqual(SuccessList[SuccessList.length - 1], `Successfully refreshed Dispatch model`); + }); + }); +}); diff --git a/lib/typescript/botskills/test/mockedFiles/successfulConnectFiles/connectableSkill.json b/lib/typescript/botskills/test/mockedFiles/successfulConnectFiles/connectableSkill.json index e69de29bb2..7b9256186e 100644 --- a/lib/typescript/botskills/test/mockedFiles/successfulConnectFiles/connectableSkill.json +++ b/lib/typescript/botskills/test/mockedFiles/successfulConnectFiles/connectableSkill.json @@ -0,0 +1,27 @@ +{ + "id": "connectableSkill", + "msaAppId": "00000000-0000-0000-0000-000000000000", + "name": "Test Skill", + "endpoint": "https://bftestskill.azurewebsites.net/api/skill/messages", + "description": "This is a test skill to use for testing purpose.", + "authenticationConnections": [], + "actions": [ + { + "id": "connectableSkill_testAction", + "definition": { + "description": "Test Action", + "slots": [], + "triggers": { + "utteranceSources": [ + { + "locale": "en", + "source": [ + "connectableSkill#Test" + ] + } + ] + } + } + } + ] +} \ No newline at end of file