2916: Add tts on android #2950
2916: Add tts on android #2950
Conversation
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
There was a problem hiding this comment.
Works pretty well so far! A couple of points about the functionality:
- If I change the volume while it's playing, it starts re-reading the current fragment. If possible, I think it should keep reading at the same spot.
- In using, I thought the volume bar was for updating the speed of reading. Maybe that's just me but maybe not, we should see if other people have that problem.
- For Persian (and any other languages where it doesn't work), I think we should hide the button to read the text aloud, and close the overlay if it's open when someone switches languages.
- In emulated iOS, no sound comes out and I get this warning:
Excessive number of pending callbacks: 501. Some pending callbacks that might have leaked by never being called from native code: {"1000":{},"1001":{},"1002":{},"1003":{},"1004":{},"1005":{},"1006":{},"1007":{},"1008":{},"1009":{},"1010":{},"1011":{},"1012":{},"1013":{},"1014":{},"1015":{},"1016":{},"1017":{},"1018":{},"1019":{},"1020":{},"1021":{},"1022":{},"1023":{},"1024":{},"1025":{},"1026":{},"1027":{},"1028":{},"1029":{},"1030":{},"1031":{},"1032":{},"1033":{},"1034":{},"1035":{},"1036":{},"1037":{},"1038":{},"1039":{},"1040":{},"1041":{},"1042":{},"1043":{},"1044":{},"1045":{},"1046":{},"1047":{},"1048":{},"1049":{},"...(truncated keys)...":451}But I guess you did say that it's not yet tested for iOS :D
As discussed, I only half-checked the PR, let me know when it's ready for review again :)
There was a problem hiding this comment.
Works for me on android. good work.
Since its a draft some improvements are missing i guess
Shouldn't here stand the title of what will be read?
Even add shadows/elevation for the container and the play button
The icons should switch i guess since the direction arrows are wrong
I think it would be better to create a fixed area for this player instead of overlapping content, but this may be also done in an additional tasks, since it will need some effort.
Good catch @steffenkleinle – The player should be kept open but we should also offer feedback. This solution avoids abrupt ui changes, the user retains control and there is no unexpected closing behaviour: ➡️ The player remains open but is shown as disabled. If a user clicks anyway, it reads “There is nothing to read on this page” or similar. If the user navigates back to a page with content, the player resumes normal behavior. |
There was a problem hiding this comment.
I tested in on android and works great so far with one issue.
Is it intended that first level pages can not be 'read aloud'?
Design
Some general opinions. I would keep the size of playing and not playing container equal size. I think this jumping of the size is a bit annoying.
With more space we could even display the complete headline (title) maybe in two lines.
The alignment of the "play" button should be more left aligned to have a bigger spacing between the close button. Here it can easily happen that the close button will be touched instead of the play button.
These are all things that could be done in an additional design improvement task
| @@ -57,8 +60,6 @@ const Events = ({ cityModel, language, navigateTo, events, slug, refresh }: Even | |||
| } | |||
|
|
|||
| if (slug) { | |||
| const event = events.find(it => it.slug === slug) | |||
There was a problem hiding this comment.
why this change was needed?
There was a problem hiding this comment.
for useTtsPlayer and for less redundant code..
I followed the design from here:
|
yes but now there is not "Vorlesefunktion" but title which may be longer. Unfortunately its not working for ios. Player can't be stopped. Could be related to this but not sure |
There is a separate issue for iOS (#2917) and it is disabled on production (as the feature flag for everything but integreat-test-cms is set to false. |
|
Ok i put some investigation into this. The issue is that the action I put also some effort in checking the native modules and thought this shouldn't be too difficult to patch but in the java and swift code the particular functions send the correct events. So i'm not pretty sure where is the issue and how to fix that |
|
Yes we can do that. As far as i can see this issue was even in older version (tested several), so i'm not sure if it will be fixed or we have to think about a different implementation. It looks like this is an ios internal problem since ios15 as you can find here, so the bindings in the native modules are correct but ios internally uses the wrong methods. |
How is the behavior of tts-cancel can we use it for ios to keep the chain of speech going? |
@bahaaTuffaha One question i didn't get an answer was why the first level texts will not be read or why they are disabled for tts. |
That was part of the original design and discussion of the issue. Perhaps @hauf-toni can shed some more light on this. Personally I also think we should perhaps change this decision and allow to read all content (not counting children or similar). I think otherwise it is not really obvious for users how the feature works. |
|
|
Alright, I think everything clear here for now, lets get this merged. @bahaaTuffaha can you check the failing CI? |
and what about this? |
That is a separate issue/task, so we are good to merge this (if it is disabled on ios as suggested here: #2950 (comment)) |
Yes, I agree. This enhances predictability & clarity for users and they don't have to guess, when content is likely to be read aloud. What is the status of the design decision concerning the sizing, do you still need some input on this? @bahaaTuffaha |
@hauf-toni we have a new issue for that, see #3064. I think @f1sh1918 said he'll take a look at it, so perhaps you can discuss the necessary changes (in the new issue)? |
Short description
Adding text-to-speech functionality.
Proposed changes
TtsPlayer.tsx: deals with html content and it passes it toreact-native-ttsas sentences. used debounce function from lodash to handle rapid volume change. I added an expandPlayer state just to keep player expanded when changes from sentence to sentence. This component located at app level where it wraps around the NavigationContainer to be able to be floating at the bottom of the screen while scrolling.useTtsPlayer.ts: Is a hook used to pass data (content, title, visibility, volume) from across the app toTtsPlayer.tsxusinguseContext.Categories.tsx, News.tsx, Events.tsx each one is using the
useTtsPlayer.tshook to pass content and title.Created a component
Slider.tsxinstead of adding more libraires and used the ones we have:react-native-reanimated&react-native-gesture-handlerto create slider that can be used anywhere other than the volume.Added ReadAloud HeaderButtonTitle to be selected from the kebab menu to show the player. This button will show up only in Integreat/IntegreatTestCms and at third level pages.
The TtsPlayer will stop when the app is in the background also if you back out from a 3rd level page.
Notes:
tts.speak()? due toreact-native-ttsdoesn't have a functional pause also passing word by word will sound more robotic.react-native-tts.Side effects
Testing
Resolved issues
Fixes: #2916