From caa759724d2ac91885273ee3b61dac731e5bb140 Mon Sep 17 00:00:00 2001 From: Alexander Biemann Date: Wed, 10 Jul 2019 00:18:01 +0200 Subject: [PATCH] Latest Android Assistant updates (#1771) * blue background for users' requests in chat * added demo feature to update widgets of 3rd party app * added ISO 8601 timestamps to generated activity events * pull from upstream (#11) * Added initial readme for Android Assistant app (#1667) * Virtual Assistant Client for Android - initial public commit * updated .gitignore to ensure the /src/debug folder is not ignored * implemented feedback from PR 1493 * code cleanup after merge * fixes possible exception due to race condition * fix: release created and stopped audio records * added timeout event so that client app can notify user * WIP adaptive card click events * WIP adaptive card clicking * finished implementation of adaptive card clicking * Updated Location event so that the data is consumed properly * additional IPA -> VA changes * logcat logging of complete JSON data * using suggested actions for adaptive card click * undo incorrect commit * added suggested actions feature * updates to allow the service to receive connections from other apps * small code updates + locationProvider threading fix * finished feature to broadcast data to other apps * added initial readme for Android app * Update readme.md * [Botskills] Add a new Known Issue of the CLI tool (#1665) * Adapt luisgen commands to trailing backslashs * Add issue in KnownIssues doc * Add an example of the issue * [DevOps] Disable builds in pull-request by default (#1661) * Update YAML files in TypeScript and Csharp * Update readme in both versions * Minor changes in YAML files * Minor changes in readme files * Updated default comment with sections for bots/deployment scripts (#1669) * News skill: adding trending articles, favorite topics, searching by market, searching by website (#1660) * Added trending articles, changing search market, search specific sites, choosing favorite topics * Changes from comments * fis newsLuis * update publish scripts to fix skill deployment filenotfound issues with kudu (#1668) * Create release-drafter.yml * [TypeScript][Virtual Assistant/Skill samples] Migrate from TSLint to ESLint (#1673) * Add ESLint to sample assistant * Fix ESLint issues in sample-assistant * Add ESLint to sample skill * Fix ESLint issues in sample-skill * Update dependencies * Update templates * Update generator * [SkillsLib]remove http transport and related (#1676) * remove http transport and related * try to get RestoreAddtionalProjectSources working * trigger build * add package source for Skills test project * Android app update: Ability to change view between Full Conversation and Conversation (#1681) * Virtual Assistant Client for Android - initial public commit * updated .gitignore to ensure the /src/debug folder is not ignored * implemented feedback from PR 1493 * code cleanup after merge * fixes possible exception due to race condition * fix: release created and stopped audio records * added timeout event so that client app can notify user * WIP adaptive card click events * WIP adaptive card clicking * finished implementation of adaptive card clicking * Updated Location event so that the data is consumed properly * additional IPA -> VA changes * logcat logging of complete JSON data * using suggested actions for adaptive card click * undo incorrect commit * added suggested actions feature * updates to allow the service to receive connections from other apps * small code updates + locationProvider threading fix * finished feature to broadcast data to other apps * added initial readme for Android app * Update readme.md * added feature to show full conversation * latest upstream changes (#8) * Added initial readme for Android Assistant app (#1667) * Virtual Assistant Client for Android - initial public commit * updated .gitignore to ensure the /src/debug folder is not ignored * implemented feedback from PR 1493 * code cleanup after merge * fixes possible exception due to race condition * fix: release created and stopped audio records * added timeout event so that client app can notify user * WIP adaptive card click events * WIP adaptive card clicking * finished implementation of adaptive card clicking * Updated Location event so that the data is consumed properly * additional IPA -> VA changes * logcat logging of complete JSON data * using suggested actions for adaptive card click * undo incorrect commit * added suggested actions feature * updates to allow the service to receive connections from other apps * small code updates + locationProvider threading fix * finished feature to broadcast data to other apps * added initial readme for Android app * Update readme.md * [Botskills] Add a new Known Issue of the CLI tool (#1665) * Adapt luisgen commands to trailing backslashs * Add issue in KnownIssues doc * Add an example of the issue * [DevOps] Disable builds in pull-request by default (#1661) * Update YAML files in TypeScript and Csharp * Update readme in both versions * Minor changes in YAML files * Minor changes in readme files * Updated default comment with sections for bots/deployment scripts (#1669) * News skill: adding trending articles, favorite topics, searching by market, searching by website (#1660) * Added trending articles, changing search market, search specific sites, choosing favorite topics * Changes from comments * fis newsLuis * update publish scripts to fix skill deployment filenotfound issues with kudu (#1668) * Create release-drafter.yml * [TypeScript][Virtual Assistant/Skill samples] Migrate from TSLint to ESLint (#1673) * Add ESLint to sample assistant * Fix ESLint issues in sample-assistant * Add ESLint to sample skill * Fix ESLint issues in sample-skill * Update dependencies * Update templates * Update generator * [Botskills] Improve error messages (#1683) * Improve error messages for more verbosity * Continue improving error messages * Finish improving error messages * Improve error handling for authenticationUtils * Revert changes to arguments arrays * Add link to documentation when auth fails * Allow ITriggers with nullable properties (#1685) * [Botskills] Move location to tools folder (#1666) * Move botskills cli tool to the tools root folder * Update doc references * Update yaml files * Update package-lock file * Update build.botskills.ts.ps1 * Update speechenablement.md * [Botskills] Add re-publish warning (#1686) * Add warning to republish VA after refresh * Fix tests * add flag to include nuget pkgs so nuspec can pick up packages needed (#1690) * add flag to include nuget pkgs so nuspec can pick up packages needed * force trigger build * remove redundant space * [VA] Updated deployment scripts to assign luis subscription key automatically (#1693) * add luis sub key * add azure account * deploy updates * deploy fix * [Botskills] Update documentation (#1688) * Update botskills documentation * Remove style of botskills word * Fix link reference * Fix references to botskills documentation * Fix typos * Apply suggestions from code review Co-Authored-By: David Favretto * Apply suggestions from code review * upgrade versions of skills and solutions for templates (#1695) * [TypeScript][BotBuilder-Libs] Migrate from TSLint to ESLint (#1696) * Add ESLint to botbuilder-solutions * Add ESLint to botbuilder-skills * Update dependencies * Fix ESLint issues in botbuilder-solutions * Fix ESLint issues in botbuilder-skills * upgrade skills to latest lib version (#1698) * [Calendar] Add config for slot filling attribute skip (#1647) * add slot filling default value config * fix test cases * add default value config of start date and duration * add default content and location in common string * update doc * Update productivity-calendar.md * Update productivity-email.md * Update knownissues.md * removing template projects from sln as msbuild on devops can't skip certain projects. (#1703) * VA Deployment Updates (#1694) * write full validation result to log * handle spaces in path * fixed publish with different bot name * added luis region to all commands * removed extra registration from skill (#1702) * Fix broken link to botskills cli (#1708) * move backgroundImage into column (#1711) (#1712) They are used in ResponseManager.GetCardResponse as containerItems, so backgroundImage should be kept in Column. * [TypeScript][Botskills] Refactor in Mocks folder (#1706) * Fix typos for errors * Refactor mocks folder * Remove unused dependencies in generator-botbuilder-assistant (#1707) * Update README.md * Update CODEOWNERS (#1709) * removed timestamp from Activity events * added Timezone feature. Fixes #1678 * removed unnecessary Directline constant * updates: 1. new API sendTimeZoneEvent 2. Settings UI update 3. fixes bug in Synthesizer 4. bot will listen again when expectingInput is received * Settings screen cleanup * fix merge for these files with special chars * Improved broadcasting to 3rd-party apps --- .../configuration/DefaultConfiguration.java | 11 +-- .../app/src/main/AndroidManifest.xml | 7 +- .../AppConfigurationActivity.java | 85 ------------------- .../activities/main/MainActivity.java | 9 +- .../SettingsActivity.java} | 60 +++++++++++-- .../service/SpeechService.java | 45 ++++++++-- .../res/layout/activity_app_configuration.xml | 72 ---------------- ...onfiguration.xml => activity_settings.xml} | 57 ++++++++----- .../src/main/res/layout/item_chat_user.xml | 2 +- .../app/src/main/res/menu/nav_drawer_menu.xml | 4 - .../app/src/main/res/values/strings.xml | 14 +-- .../solutions/directlinespeech/SpeechSdk.java | 8 +- .../directlinespeech/Synthesizer.java | 7 ++ .../directlinespeech/model/Configuration.java | 17 ++-- .../main/java/events/SynthesizerStopped.java | 7 ++ 15 files changed, 165 insertions(+), 240 deletions(-) delete mode 100644 solutions/android/VirtualAssistantClient/app/src/main/java/com/microsoft/bot/builder/solutions/virtualassistant/activities/configuration/AppConfigurationActivity.java rename solutions/android/VirtualAssistantClient/app/src/main/java/com/microsoft/bot/builder/solutions/virtualassistant/activities/{botconfiguration/BotConfigurationActivity.java => settings/SettingsActivity.java} (65%) delete mode 100644 solutions/android/VirtualAssistantClient/app/src/main/res/layout/activity_app_configuration.xml rename solutions/android/VirtualAssistantClient/app/src/main/res/layout/{activity_bot_configuration.xml => activity_settings.xml} (87%) create mode 100644 solutions/android/VirtualAssistantClient/directlinespeech/src/main/java/events/SynthesizerStopped.java diff --git a/solutions/android/VirtualAssistantClient/app/src/debug/java/com/microsoft/bot/builder/solutions/virtualassistant/activities/configuration/DefaultConfiguration.java b/solutions/android/VirtualAssistantClient/app/src/debug/java/com/microsoft/bot/builder/solutions/virtualassistant/activities/configuration/DefaultConfiguration.java index d802765857..190dbfc847 100644 --- a/solutions/android/VirtualAssistantClient/app/src/debug/java/com/microsoft/bot/builder/solutions/virtualassistant/activities/configuration/DefaultConfiguration.java +++ b/solutions/android/VirtualAssistantClient/app/src/debug/java/com/microsoft/bot/builder/solutions/virtualassistant/activities/configuration/DefaultConfiguration.java @@ -3,17 +3,14 @@ public class DefaultConfiguration { // Replace below with your own subscription key - public static final String COGNITIVE_SERVICES_SUBSCRIPTION_KEY = "YOUR_KEY_HERE";//TODO + public static final String SPEECH_SERVICE_SUBSCRIPTION_KEY = "YOUR_KEY_HERE";//TODO - public static final String BOT_ID = "YOUR_BOT_ID_HERE";//TODO + public static final String DIRECT_LINE_SPEECH_SECRET_KEY = "YOUR_DIRECTLINE_SPEECH_KEY_HERE";//TODO - public static final String SPEECH_REGION = "westus2"; + public static final String SPEECH_SERVICE_SUBSCRIPTION_KEY_REGION = "westus2";//TODO - public static final String VOICE_NAME = "Microsoft Server Speech Text to Speech Voice (en-US, JessaNeural)"; - - public static final String DIRECT_LINE_CONSTANT = "directline"; public static final String USER_NAME = "User"; - public static final String USER_ID = "YOUR_USER_ID_HERE";//TODO + public static final String USER_FROM_ID = "YOUR_USER_FROM_ID_HERE";//TODO public static final String LOCALE = "en-us"; diff --git a/solutions/android/VirtualAssistantClient/app/src/main/AndroidManifest.xml b/solutions/android/VirtualAssistantClient/app/src/main/AndroidManifest.xml index da0c7c8fab..40027f012d 100644 --- a/solutions/android/VirtualAssistantClient/app/src/main/AndroidManifest.xml +++ b/solutions/android/VirtualAssistantClient/app/src/main/AndroidManifest.xml @@ -44,12 +44,7 @@ - - diff --git a/solutions/android/VirtualAssistantClient/app/src/main/java/com/microsoft/bot/builder/solutions/virtualassistant/activities/configuration/AppConfigurationActivity.java b/solutions/android/VirtualAssistantClient/app/src/main/java/com/microsoft/bot/builder/solutions/virtualassistant/activities/configuration/AppConfigurationActivity.java deleted file mode 100644 index a191bf779b..0000000000 --- a/solutions/android/VirtualAssistantClient/app/src/main/java/com/microsoft/bot/builder/solutions/virtualassistant/activities/configuration/AppConfigurationActivity.java +++ /dev/null @@ -1,85 +0,0 @@ -package com.microsoft.bot.builder.solutions.virtualassistant.activities.configuration; - -import android.content.Context; -import android.content.Intent; -import android.os.Bundle; -import android.support.design.widget.TextInputEditText; -import android.view.KeyEvent; -import android.view.inputmethod.EditorInfo; - -import com.microsoft.bot.builder.solutions.directlinespeech.ConfigurationManager; -import com.microsoft.bot.builder.solutions.directlinespeech.model.Configuration; -import com.microsoft.bot.builder.solutions.virtualassistant.R; -import com.microsoft.bot.builder.solutions.virtualassistant.activities.BaseActivity; - -import butterknife.BindView; -import butterknife.ButterKnife; -import butterknife.OnClick; -import butterknife.OnEditorAction; - -/** - * App Configuration Activity - settings to change the way the app behaves - * Note: settings are saved when OK is pressed - */ -public class AppConfigurationActivity extends BaseActivity { - - // VIEWS - @BindView(R.id.history_linecount) TextInputEditText historyLinecount; - - // CONSTANTS - private static final int CONTENT_VIEW = R.layout.activity_app_configuration; - - // STATE - private Configuration configuration; - private ConfigurationManager configurationManager; - - public static Intent getNewIntent(Context context) { - return new Intent(context, AppConfigurationActivity.class); - } - - @Override - protected void onCreate(Bundle savedInstanceState) { - super.onCreate(savedInstanceState); - setContentView(CONTENT_VIEW); - ButterKnife.bind(this); - - configurationManager = new ConfigurationManager(this); - configuration = configurationManager.getConfiguration(); - - showConfiguration(); - } - - @OnEditorAction({R.id.history_linecount}) - boolean onEditorAction(int actionId, KeyEvent key){ - boolean handled = false; - if (actionId == EditorInfo.IME_ACTION_SEND || (key != null && key.getKeyCode() == KeyEvent.KEYCODE_ENTER)) { - hideKeyboardFrom(getCurrentFocus()); - handled = true; - } - return handled; - } - - @OnClick(R.id.btn_cancel) - public void onClickCancel() { - finish(); - } - - @OnClick(R.id.btn_ok) - public void onClickOk() { - saveConfiguration();// must save updated config first - finish(); - } - - private void showConfiguration(){ - int iHistoryLinecount = configuration.historyLinecount==null?1:configuration.historyLinecount; - String historyLineCount = String.valueOf(iHistoryLinecount); - historyLinecount.setText(historyLineCount); - } - - private void saveConfiguration(){ - configuration.historyLinecount = Integer.valueOf(historyLinecount.getText().toString()); - if (configuration.historyLinecount == 0) configuration.historyLinecount = 1;//do not allow 0 - configurationManager.setConfiguration(configuration); - } - -} diff --git a/solutions/android/VirtualAssistantClient/app/src/main/java/com/microsoft/bot/builder/solutions/virtualassistant/activities/main/MainActivity.java b/solutions/android/VirtualAssistantClient/app/src/main/java/com/microsoft/bot/builder/solutions/virtualassistant/activities/main/MainActivity.java index cc2f6b17de..1c26206f16 100644 --- a/solutions/android/VirtualAssistantClient/app/src/main/java/com/microsoft/bot/builder/solutions/virtualassistant/activities/main/MainActivity.java +++ b/solutions/android/VirtualAssistantClient/app/src/main/java/com/microsoft/bot/builder/solutions/virtualassistant/activities/main/MainActivity.java @@ -37,8 +37,7 @@ import com.microsoft.bot.builder.solutions.directlinespeech.model.Configuration; import com.microsoft.bot.builder.solutions.virtualassistant.R; import com.microsoft.bot.builder.solutions.virtualassistant.activities.BaseActivity; -import com.microsoft.bot.builder.solutions.virtualassistant.activities.botconfiguration.BotConfigurationActivity; -import com.microsoft.bot.builder.solutions.virtualassistant.activities.configuration.AppConfigurationActivity; +import com.microsoft.bot.builder.solutions.virtualassistant.activities.settings.SettingsActivity; import com.microsoft.bot.builder.solutions.virtualassistant.activities.main.actionslist.ActionsAdapter; import com.microsoft.bot.builder.solutions.virtualassistant.activities.main.actionslist.ActionsViewholder; import com.microsoft.bot.builder.solutions.virtualassistant.activities.main.chatlist.ChatAdapter; @@ -168,7 +167,6 @@ protected void onResume() { // Unregister EventBus messages and SpeechService @Override public void onStop() { - Log.v("BaseActivity","onStop() finished"); EventBus.getDefault().unregister(this); if (speechServiceBinder != null) { unbindService(myConnection); @@ -251,10 +249,7 @@ public boolean onNavigationItemSelected(MenuItem item) { switch (id) { case R.id.nav_menu_configuration: - startActivity(BotConfigurationActivity.getNewIntent(this)); - break; - case R.id.nav_menu_app_configuration: - startActivity(AppConfigurationActivity.getNewIntent(this)); + startActivity(SettingsActivity.getNewIntent(this)); break; case R.id.nav_menu_reset_bot: speechServiceBinder.resetBot(); diff --git a/solutions/android/VirtualAssistantClient/app/src/main/java/com/microsoft/bot/builder/solutions/virtualassistant/activities/botconfiguration/BotConfigurationActivity.java b/solutions/android/VirtualAssistantClient/app/src/main/java/com/microsoft/bot/builder/solutions/virtualassistant/activities/settings/SettingsActivity.java similarity index 65% rename from solutions/android/VirtualAssistantClient/app/src/main/java/com/microsoft/bot/builder/solutions/virtualassistant/activities/botconfiguration/BotConfigurationActivity.java rename to solutions/android/VirtualAssistantClient/app/src/main/java/com/microsoft/bot/builder/solutions/virtualassistant/activities/settings/SettingsActivity.java index 5443a4c909..6c76aeaad2 100644 --- a/solutions/android/VirtualAssistantClient/app/src/main/java/com/microsoft/bot/builder/solutions/virtualassistant/activities/botconfiguration/BotConfigurationActivity.java +++ b/solutions/android/VirtualAssistantClient/app/src/main/java/com/microsoft/bot/builder/solutions/virtualassistant/activities/settings/SettingsActivity.java @@ -1,20 +1,28 @@ -package com.microsoft.bot.builder.solutions.virtualassistant.activities.botconfiguration; +package com.microsoft.bot.builder.solutions.virtualassistant.activities.settings; import android.content.Context; import android.content.Intent; import android.os.Bundle; +import android.os.Handler; +import android.os.Looper; import android.os.RemoteException; +import android.support.annotation.Nullable; import android.support.design.widget.TextInputEditText; import android.util.Log; import android.view.KeyEvent; import android.view.inputmethod.EditorInfo; +import android.widget.ArrayAdapter; +import android.widget.Spinner; import com.google.gson.Gson; import com.google.gson.reflect.TypeToken; +import com.microsoft.bot.builder.solutions.directlinespeech.ConfigurationManager; import com.microsoft.bot.builder.solutions.directlinespeech.model.Configuration; import com.microsoft.bot.builder.solutions.virtualassistant.R; import com.microsoft.bot.builder.solutions.virtualassistant.activities.BaseActivity; +import java.util.TimeZone; + import butterknife.BindView; import butterknife.ButterKnife; import butterknife.OnClick; @@ -24,27 +32,29 @@ * Bot Configuration Activity - settings to change the connection to the Bot * Note: settings are saved when OK is pressed */ -public class BotConfigurationActivity extends BaseActivity { +public class SettingsActivity extends BaseActivity { // VIEWS @BindView(R.id.service_key) TextInputEditText serviceKey; @BindView(R.id.service_region) TextInputEditText serviceRegion; @BindView(R.id.bot_id) TextInputEditText botId; - @BindView(R.id.voice_name) TextInputEditText voiceName; @BindView(R.id.user_id) TextInputEditText userId; @BindView(R.id.locale) TextInputEditText locale; @BindView(R.id.geolocation_lat) TextInputEditText locationLat; @BindView(R.id.geolocation_lon) TextInputEditText locationLon; + @BindView(R.id.history_linecount) TextInputEditText historyLinecount; + @BindView(R.id.spinner_timezone) Spinner spinnerTimezone; // CONSTANTS - private static final int CONTENT_VIEW = R.layout.activity_bot_configuration; + private static final int CONTENT_VIEW = R.layout.activity_settings; // STATE private Configuration configuration; + private ArrayAdapter tzAdapter; private Gson gson; public static Intent getNewIntent(Context context) { - return new Intent(context, BotConfigurationActivity.class); + return new Intent(context, SettingsActivity.class); } @Override @@ -53,6 +63,7 @@ protected void onCreate(Bundle savedInstanceState) { setContentView(CONTENT_VIEW); ButterKnife.bind(this); gson = new Gson(); + initTimezoneAdapter(); } @Override @@ -77,7 +88,7 @@ protected void serviceConnected() { showConfiguration(); } - @OnEditorAction({R.id.service_key, R.id.service_region, R.id.bot_id, R.id.voice_name, R.id.user_id, R.id.locale, R.id.geolocation_lat, R.id.geolocation_lon}) + @OnEditorAction({R.id.history_linecount, R.id.service_key, R.id.service_region, R.id.bot_id, R.id.user_id, R.id.locale, R.id.geolocation_lat, R.id.geolocation_lon}) boolean onEditorAction(int actionId, KeyEvent key){ boolean handled = false; if (actionId == EditorInfo.IME_ACTION_SEND || (key != null && key.getKeyCode() == KeyEvent.KEYCODE_ENTER)) { @@ -99,6 +110,27 @@ public void onClickOk() { finish(); } + private void initTimezoneAdapter() { + + //populate spinner with all timezones + String[] idArray = TimeZone.getAvailableIDs(); + tzAdapter = new ArrayAdapter<>(this, android.R.layout.simple_spinner_dropdown_item, idArray); + tzAdapter.setDropDownViewResource(android.R.layout.simple_spinner_dropdown_item); + spinnerTimezone.setAdapter(tzAdapter); + } + + private void selectTimezone(@Nullable String tzId) { + if (tzId == null){ + tzId = TimeZone.getDefault().getID(); + } + for (int i = 0; i < tzAdapter.getCount(); i++) { + if (tzAdapter.getItem(i).equals(tzId)) { + spinnerTimezone.setSelection(i); + break; + } + } + } + private void showConfiguration(){ try { final String json = speechServiceBinder.getConfiguration(); @@ -106,11 +138,16 @@ private void showConfiguration(){ serviceKey.setText(configuration.serviceKey); serviceRegion.setText(configuration.serviceRegion); botId.setText(configuration.botId); - voiceName.setText(configuration.voiceName); userId.setText(configuration.userId); locale.setText(configuration.locale); locationLat.setText(configuration.geolat); locationLon.setText(configuration.geolon); + + int iHistoryLinecount = configuration.historyLinecount==null?1:configuration.historyLinecount; + String historyLineCount = String.valueOf(iHistoryLinecount); + historyLinecount.setText(historyLineCount); + + selectTimezone(configuration.currentTimezone); } catch (RemoteException exception){ Log.e(LOGTAG, exception.getMessage()); } @@ -121,11 +158,18 @@ private void saveConfiguration(){ configuration.serviceKey = serviceKey.getText().toString(); configuration.serviceRegion = serviceRegion.getText().toString(); configuration.botId = botId.getText().toString(); - configuration.voiceName = voiceName.getText().toString(); configuration.userId = userId.getText().toString(); configuration.locale = locale.getText().toString(); configuration.geolat = locationLat.getText().toString(); configuration.geolon = locationLon.getText().toString(); + + // history linecount + configuration.historyLinecount = Integer.valueOf(historyLinecount.getText().toString()); + if (configuration.historyLinecount == 0) configuration.historyLinecount = 1;//do not allow 0 + + // timezone + configuration.currentTimezone = (String)tzAdapter.getItem(spinnerTimezone.getSelectedItemPosition()); + String json = gson.toJson(configuration); speechServiceBinder.setConfiguration(json); } catch (RemoteException exception){ diff --git a/solutions/android/VirtualAssistantClient/app/src/main/java/com/microsoft/bot/builder/solutions/virtualassistant/service/SpeechService.java b/solutions/android/VirtualAssistantClient/app/src/main/java/com/microsoft/bot/builder/solutions/virtualassistant/service/SpeechService.java index 4b76d71ddf..f5b78f18ac 100644 --- a/solutions/android/VirtualAssistantClient/app/src/main/java/com/microsoft/bot/builder/solutions/virtualassistant/service/SpeechService.java +++ b/solutions/android/VirtualAssistantClient/app/src/main/java/com/microsoft/bot/builder/solutions/virtualassistant/service/SpeechService.java @@ -41,10 +41,12 @@ import java.io.IOException; import client.model.BotConnectorActivity; +import client.model.InputHints; import events.ActivityReceived; import events.Recognized; import events.RecognizedIntermediateResult; import events.RequestTimeout; +import events.SynthesizerStopped; /** * The SpeechService is the connection between bot and activities and widgets @@ -68,6 +70,7 @@ public class SpeechService extends Service { private ConfigurationManager configurationManager; private LocationProvider locationProvider; private Gson gson; + private boolean shouldListenAgain; // CONSTRUCTOR public SpeechService() { @@ -104,7 +107,8 @@ public void startLocationUpdates() { @Override public void resetBot(){ - speechSdk.resetBot(); + shouldListenAgain = false; + speechSdk.resetBot(configurationManager.getConfiguration()); } @Override @@ -180,13 +184,11 @@ public void onCreate() { Configuration configuration = configurationManager.getConfiguration(); if (configuration.isEmpty()){ // set up defaults - configuration.serviceKey = DefaultConfiguration.COGNITIVE_SERVICES_SUBSCRIPTION_KEY; - configuration.botId = DefaultConfiguration.BOT_ID; - configuration.serviceRegion = DefaultConfiguration.SPEECH_REGION; - configuration.voiceName = DefaultConfiguration.VOICE_NAME; - configuration.directlineConstant = DefaultConfiguration.DIRECT_LINE_CONSTANT; + configuration.serviceKey = DefaultConfiguration.SPEECH_SERVICE_SUBSCRIPTION_KEY; + configuration.botId = DefaultConfiguration.DIRECT_LINE_SPEECH_SECRET_KEY; + configuration.serviceRegion = DefaultConfiguration.SPEECH_SERVICE_SUBSCRIPTION_KEY_REGION; configuration.userName = DefaultConfiguration.USER_NAME; - configuration.userId = DefaultConfiguration.USER_ID; + configuration.userId = DefaultConfiguration.USER_FROM_ID; configuration.locale = DefaultConfiguration.LOCALE; configuration.geolat = DefaultConfiguration.GEOLOCATION_LAT; configuration.geolon = DefaultConfiguration.GEOLOCATION_LON; @@ -330,6 +332,17 @@ private void initializeSpeechSdk(boolean haveRecordAudioPermission){ speechSdk.initialize(configuration, haveRecordAudioPermission, directory.getPath()); } + // EventBus: the synthesizer has stopped playing + @Subscribe(threadMode = ThreadMode.MAIN) + public void onEventSynthesizerStopped(SynthesizerStopped event) { + + if(shouldListenAgain){ + shouldListenAgain = false; + speechSdk.listenOnceAsync(); + } + + } + // EventBus: the previous request timed out @Subscribe(threadMode = ThreadMode.MAIN) public void onEventRequestTimeout(RequestTimeout event) { @@ -373,8 +386,16 @@ public void onEventActivityReceived(ActivityReceived activityReceived) throws IO openDefaultApp(botConnectorActivity); break; default: + broadcastWidgetUpdate(botConnectorActivity); break; } + + // make the bot automatically listen again + if(botConnectorActivity.getInputHint() != null){ + if(botConnectorActivity.getInputHint().equals(InputHints.EXPECTINGINPUT)){ + shouldListenAgain = true; + } + } } } @@ -411,7 +432,6 @@ private void openDefaultApp(BotConnectorActivity botConnectorActivity){ private void broadcastActivity(BotConnectorActivity botConnectorActivity){ final String json = gson.toJson(botConnectorActivity); - final Intent intent=new Intent(); intent.setAction("com.microsoft.broadcast"); intent.putExtra("BotConnectorActivity",json); @@ -420,13 +440,20 @@ private void broadcastActivity(BotConnectorActivity botConnectorActivity){ private void broadcastTimeout(RequestTimeout event){ final String json = gson.toJson(event); - final Intent intent=new Intent(); intent.setAction("com.microsoft.broadcast"); intent.putExtra("RequestTimeout",json); sendBroadcast(intent); } + private void broadcastWidgetUpdate(BotConnectorActivity botConnectorActivity){ + final String json = gson.toJson(botConnectorActivity); + final Intent intent=new Intent(); + intent.setAction("com.microsoft.broadcast"); + intent.putExtra("WidgetUpdate",json); + sendBroadcast(intent); + } + private void updateBotResponseWidget(String text){ Log.v(TAG_FOREGROUND_SERVICE, "updateBotResponseWidget("+text+")"); Context context = this; diff --git a/solutions/android/VirtualAssistantClient/app/src/main/res/layout/activity_app_configuration.xml b/solutions/android/VirtualAssistantClient/app/src/main/res/layout/activity_app_configuration.xml deleted file mode 100644 index 49087eda05..0000000000 --- a/solutions/android/VirtualAssistantClient/app/src/main/res/layout/activity_app_configuration.xml +++ /dev/null @@ -1,72 +0,0 @@ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - \ No newline at end of file diff --git a/solutions/android/VirtualAssistantClient/app/src/main/res/layout/activity_bot_configuration.xml b/solutions/android/VirtualAssistantClient/app/src/main/res/layout/activity_settings.xml similarity index 87% rename from solutions/android/VirtualAssistantClient/app/src/main/res/layout/activity_bot_configuration.xml rename to solutions/android/VirtualAssistantClient/app/src/main/res/layout/activity_settings.xml index ba283d6738..fdc980b669 100644 --- a/solutions/android/VirtualAssistantClient/app/src/main/res/layout/activity_bot_configuration.xml +++ b/solutions/android/VirtualAssistantClient/app/src/main/res/layout/activity_settings.xml @@ -1,13 +1,12 @@ + tools:context=".activities.settings.SettingsActivity"> - - - - - - - + - + + + + + + + + + + + + + + + + diff --git a/solutions/android/VirtualAssistantClient/app/src/main/res/layout/item_chat_user.xml b/solutions/android/VirtualAssistantClient/app/src/main/res/layout/item_chat_user.xml index dd0584caf4..d74f207969 100644 --- a/solutions/android/VirtualAssistantClient/app/src/main/res/layout/item_chat_user.xml +++ b/solutions/android/VirtualAssistantClient/app/src/main/res/layout/item_chat_user.xml @@ -9,7 +9,7 @@ - Show Textinput Show full conversation Inject Adaptive Card Event - Reset Bot + Restart Conversation Enable KWS - Bot Configuration + Settings App Configuration Show Assistant Settings - Service Key - Service Region - Bot ID + Speech Service Subscription Key + Speech Service Subscription Key Region + Direct Line Speech Secret Key Voice Name - User ID + User From Id Locale Lat Lon Chat history line-count + Timezone + Talk diff --git a/solutions/android/VirtualAssistantClient/directlinespeech/src/main/java/com/microsoft/bot/builder/solutions/directlinespeech/SpeechSdk.java b/solutions/android/VirtualAssistantClient/directlinespeech/src/main/java/com/microsoft/bot/builder/solutions/directlinespeech/SpeechSdk.java index ae3c6dec94..e90385722e 100644 --- a/solutions/android/VirtualAssistantClient/directlinespeech/src/main/java/com/microsoft/bot/builder/solutions/directlinespeech/SpeechSdk.java +++ b/solutions/android/VirtualAssistantClient/directlinespeech/src/main/java/com/microsoft/bot/builder/solutions/directlinespeech/SpeechSdk.java @@ -82,6 +82,7 @@ public void initialize(Configuration configuration, boolean haveRecordAudioPermi intializeAppLogFile(); initializeSpeech(configuration, haveRecordAudioPermission); handler = new Handler(Looper.getMainLooper()); + if (configuration.currentTimezone != null) sendTimeZoneEvent(TimeZone.getTimeZone(configuration.currentTimezone));//only do this once per session } private void intializeAppLogFile() { @@ -353,8 +354,7 @@ public void sendLocationEvent(String latitude, String longitude) { /* * Send the VA.TimeZone event to the bot */ - public void sendTimeZoneEvent() { - TimeZone tz = TimeZone.getDefault(); + private void sendTimeZoneEvent(TimeZone tz) { Activity activityTemplate = createEventActivity("VA.Timezone", null, tz.getDisplayName()); final String activityJson = gson.toJson(activityTemplate); @@ -373,12 +373,13 @@ public void reset() { final Future task = botConnector.disconnectAsync(); } - public void resetBot() { + public void resetBot(Configuration configuration) { isConnected = false; final Future task = botConnector.disconnectAsync(); setOnTaskCompletedListener(task, result -> { Log.d(LOGTAG,"disconnected"); connectAsync(); + sendTimeZoneEvent(TimeZone.getTimeZone(configuration.currentTimezone));//only do this once per session }); } @@ -421,7 +422,6 @@ private Activity createEventActivity(String eventname, Object channelData, Objec activity.setType(ActivityTypes.EVENT); activity.setLocale(configuration.locale); if (from_user != null) activity.setFrom(from_user); - activity.setChannelId(configuration.directlineConstant); activity.setChannelData(channelData); activity.setName(eventname); activity.setValue(value); diff --git a/solutions/android/VirtualAssistantClient/directlinespeech/src/main/java/com/microsoft/bot/builder/solutions/directlinespeech/Synthesizer.java b/solutions/android/VirtualAssistantClient/directlinespeech/src/main/java/com/microsoft/bot/builder/solutions/directlinespeech/Synthesizer.java index cd2c5b3ed3..b5e4886193 100644 --- a/solutions/android/VirtualAssistantClient/directlinespeech/src/main/java/com/microsoft/bot/builder/solutions/directlinespeech/Synthesizer.java +++ b/solutions/android/VirtualAssistantClient/directlinespeech/src/main/java/com/microsoft/bot/builder/solutions/directlinespeech/Synthesizer.java @@ -40,6 +40,8 @@ import com.microsoft.cognitiveservices.speech.audio.PullAudioInputStream; import com.microsoft.cognitiveservices.speech.audio.PullAudioOutputStream; +import org.greenrobot.eventbus.EventBus; + import java.io.IOException; import java.io.PipedInputStream; import java.io.PipedOutputStream; @@ -53,6 +55,8 @@ import java.util.concurrent.locks.Lock; import java.util.concurrent.locks.ReentrantLock; +import events.SynthesizerStopped; + public class Synthesizer { static final String logTag = "Synthesizer"; @@ -223,7 +227,10 @@ public void run() { //audioTrack.write(buffer, 0, readSize); } audioTrack.stop(); + audioTrack.release(); isPlaying.set(false); + // trigger event that playback is stopped + EventBus.getDefault().post(new SynthesizerStopped()); } }.start(); } diff --git a/solutions/android/VirtualAssistantClient/directlinespeech/src/main/java/com/microsoft/bot/builder/solutions/directlinespeech/model/Configuration.java b/solutions/android/VirtualAssistantClient/directlinespeech/src/main/java/com/microsoft/bot/builder/solutions/directlinespeech/model/Configuration.java index e3b4459f6a..5728035982 100644 --- a/solutions/android/VirtualAssistantClient/directlinespeech/src/main/java/com/microsoft/bot/builder/solutions/directlinespeech/model/Configuration.java +++ b/solutions/android/VirtualAssistantClient/directlinespeech/src/main/java/com/microsoft/bot/builder/solutions/directlinespeech/model/Configuration.java @@ -17,10 +17,6 @@ public class Configuration { @Expose public String botId; - @SerializedName("voice_name") - @Expose - public String voiceName; - @SerializedName("user_id") @Expose public String userId; @@ -41,26 +37,25 @@ public class Configuration { @Expose public String userName; - @SerializedName("directline_constant") - @Expose - public String directlineConstant; - @SerializedName("history_linecount") @Expose public Integer historyLinecount; + @SerializedName("current_timezone") + @Expose + public String currentTimezone;//stores the TZ ID + public boolean isEmpty(){ return serviceKey==null&& serviceRegion==null&& botId==null&& - voiceName==null&& userId==null&& locale==null&& geolat==null&& geolon==null&& userName==null&& - directlineConstant==null&& - historyLinecount==null; + historyLinecount==null&& + currentTimezone==null; } } diff --git a/solutions/android/VirtualAssistantClient/directlinespeech/src/main/java/events/SynthesizerStopped.java b/solutions/android/VirtualAssistantClient/directlinespeech/src/main/java/events/SynthesizerStopped.java new file mode 100644 index 0000000000..616e6ed0cb --- /dev/null +++ b/solutions/android/VirtualAssistantClient/directlinespeech/src/main/java/events/SynthesizerStopped.java @@ -0,0 +1,7 @@ +package events; + +public class SynthesizerStopped { + + public SynthesizerStopped() { + } +}