0.12.0 search item renderer TypeError #124

harrytouche · 2020-12-16T15:29:20Z

Hi there. Thanks for 0.12.0! I'm going through and getting a lot of unexpected failed searches. See below

Using this snippet (ytmusic initialised elsewhere)

song_list = [
    "Fleetwood Mac - Big Love",
    "Mike And The Mechanics - All I Need Is A Miracle",
    "Police - Walking On The Moon",
    "Walk The Moon - Shut Up And Dance",
    "U2 - Angel Of Harlem",
    "Annie Lennox - No More 'i Love You's'",
    "Jam - Eton Rifles",
    "Black Eyed Peas / Will.i.am - I Gotta Feeling",
    "Survivor - Eye Of The Tiger",
    "Alanis Morissette - You Learn",
    "Who - Who Are You",
    "Del Amitri - Roll To Me"
]

output = []
for song in song_list:
    try:
        output.append(ytmusic.search(song, "songs")[0]["videoId"])
        print("{} \t SUCCESS".format(song))

    except Exception as exc:
        print("FAIL \t".format(song))

Returns the following

FAIL             Fleetwood Mac - Big Love
SUCCESS          Mike And The Mechanics - All I Need Is A Miracle
SUCCESS          Police - Walking On The Moon
SUCCESS          Walk The Moon - Shut Up And Dance
SUCCESS          U2 - Angel Of Harlem
FAIL             Annie Lennox - No More 'i Love You's'
SUCCESS          Jam - Eton Rifles
SUCCESS          Black Eyed Peas / Will.i.am - I Gotta Feeling
SUCCESS          Survivor - Eye Of The Tiger
SUCCESS          Alanis Morissette - You Learn
FAIL             Who - Who Are You
SUCCESS          Del Amitri - Roll To Me

For all of the failed searches, the following error is thrown

The same is seen if I search without the "songs" filter

These are songs which never failed before, very well known! For example

In a wider test I had 23 failures from 151 songs. Before the YT update I was getting maximum 5 for the same number of songs, usually fewer

The text was updated successfully, but these errors were encountered:

sigma67 · 2020-12-16T15:53:29Z

Thanks for these search strings, quite useful for debugging. All of them are due to the unavailable songs issue #123, although I managed to catch another small thing (missing album id).

Can you check again with latest master? I want to try to fix all edge cases before releasing 0.12.1.

harrytouche · 2020-12-16T16:39:13Z

Still seeing a couple of failures there

Complete! Added 143 songs
8 songs failed. These were:
 * Fleetwood Mac - Big Love
 * Who - Who Are You
 * Cranberries - Dreams
 * Cast - Alright
 * Jamiroquai - Cosmic Girl
 * U2 - Even Better Then The Real Thing
 * Fleetwood Mac - Little Lies
 * Oasis - Roll With It

So a massive improvement but not quite there
Getting a 'NoneType' object is not subscriptable error with all of them

Will do a little more investigation on my side too

harrytouche · 2020-12-16T16:42:26Z

Yeah still looks like the same method causing trouble there

sigma67 · 2020-12-16T16:51:43Z

That's strange, all of these pass for me. Are you sure you're on 8f64cf7 ?

Edit: Perhaps it is because you try to get the videoId of the item but there is none because it's unplayable? The exception might originate within your code when you try to subscript with "videoId"

phirestalker · 2020-12-16T19:01:17Z

I am also getting this.

$ git status
On branch master
Your branch is up to date with 'origin/master'.
$ git show
commit 8f64cf7 (HEAD -> master, origin/master, origin/HEAD)
Author: sigma67 [email protected]
Date: Wed Dec 16 16:53:12 2020 +0100

trace below

  File "ytmusic-sync/./sync.py", line 99, in processFile
    song = searchYT(f'{artist} - {title}', 'songs', title, artist, track.info.length)
           │           │          │                 │      │       └ {'trackc': ['13'], 'acoustid_fingerprint': ['AQADtBu1LIoqNPuRZsOL__ggHv-Fw4JuHEjx40gPnUGOuzjxGj-OwxGSI9eBH4dxHDF0BMeLAz98on9xDz9...
           │           │          │                 │      └ 'Libera'
           │           │          │                 └ 'Sing Forever'
           │           │          └ 'Sing Forever'
           │           └ 'Libera'
           └ <function searchYT at 0x110adc280>
  File "ytmusic-sync/./sync.py", line 117, in searchYT
    results = ytmusic.search(query, type, 50)
              │              │      └ 'songs'
              │              └ 'Libera - Sing Forever'
              └ <ytmusicapi.ytmusic.YTMusic object at 0x10fdf70d0>
  File "/usr/local/lib/python3.9/site-packages/ytmusicapi-0.12.0-py3.9.egg/ytmusicapi/mixins/browsing.py", line 128, in search
    get_continuations(res['musicShelfRenderer'], 'musicShelfContinuation',
  File "/usr/local/lib/python3.9/site-packages/ytmusicapi-0.12.0-py3.9.egg/ytmusicapi/parsers/utils.py", line 86, in get_continuations
    contents = get_continuation_contents(results, parse_func)
               │                         │        └ <function BrowsingMixin.search.<locals>.<lambda> at 0x11453f430>
               │                         └ {'contents': [{'musicResponsiveListItemRenderer': {'trackingParams': 'CJMCEOFnGAAiEwjAq8W5ldPtAhVElsQKHY7tA7U=', 'thumbnail': {'...
               └ <function get_continuation_contents at 0x110aaa820>
  File "/usr/local/lib/python3.9/site-packages/ytmusicapi-0.12.0-py3.9.egg/ytmusicapi/parsers/utils.py", line 129, in get_continuation_contents
    return parse_func(continuation[term])
           │          │            └ 'contents'
           │          └ {'contents': [{'musicResponsiveListItemRenderer': {'trackingParams': 'CJMCEOFnGAAiEwjAq8W5ldPtAhVElsQKHY7tA7U=', 'thumbnail': {'...
           └ <function BrowsingMixin.search.<locals>.<lambda> at 0x11453f430>
  File "/usr/local/lib/python3.9/site-packages/ytmusicapi-0.12.0-py3.9.egg/ytmusicapi/mixins/browsing.py", line 125, in <lambda>
    parse_func = lambda contents: self.parser.parse_search_results(contents, type)
                                  │                                │         └ 'song'
                                  │                                └ [{'musicResponsiveListItemRenderer': {'trackingParams': 'CJMCEOFnGAAiEwjAq8W5ldPtAhVElsQKHY7tA7U=', 'thumbnail': {'musicThumbnai...
                                  └ <ytmusicapi.ytmusic.YTMusic object at 0x10fdf70d0>
  File "/usr/local/lib/python3.9/site-packages/ytmusicapi-0.12.0-py3.9.egg/ytmusicapi/helpers.py", line 78, in _impl
    return method(self, *method_args, **method_kwargs)
           │      │      │              └ {}
           │      │      └ ([{'musicResponsiveListItemRenderer': {'trackingParams': 'CJMCEOFnGAAiEwjAq8W5ldPtAhVElsQKHY7tA7U=', 'thumbnail': {'musicThumbna...
           │      └ <ytmusicapi.parsers.browsing.Parser object at 0x10fdf7100>
           └ <function Parser.parse_search_results at 0x110aaac10>
  File "/usr/local/lib/python3.9/site-packages/ytmusicapi-0.12.0-py3.9.egg/ytmusicapi/parsers/browsing.py", line 78, in parse_search_results
    search_result['feedbackTokens'] = parse_song_menu_tokens(toggle_menu)
    │                                 │                      └ None
    │                                 └ <function parse_song_menu_tokens at 0x110aaa310>
    └ {'resultType': 'song', 'videoId': 'EcQABso__-Q', 'title': 'Obsessed', 'artists': [{'name': 'Mariah Carey', 'id': 'UCKRnq8aBOCanY...
  File "/usr/local/lib/python3.9/site-packages/ytmusicapi-0.12.0-py3.9.egg/ytmusicapi/parsers/utils.py", line 33, in parse_song_menu_tokens
    toggle_menu = item['toggleMenuServiceItemRenderer']
                  └ None
TypeError: 'NoneType' object is not subscriptable

harrytouche · 2020-12-16T20:08:24Z

For sure on that commit, but good idea to make sure

$ git show
commit 8f64cf7a7ac19c7bdfb66c621554b1386243398e (HEAD -> master, origin/master, origin/HEAD)
Author: sigma67 <[email protected]>
Date:   Wed Dec 16 16:53:12 2020 +0100

The error occurs even if I don't select the 0th element and get the videoId, and indeed even if I don't have the "songs" filter as well
SNIP

song_list = [
    "Fleetwood Mac - Big Love",
    "Mike And The Mechanics - All I Need Is A Miracle",
    "Police - Walking On The Moon",
    "Walk The Moon - Shut Up And Dance",
    "U2 - Angel Of Harlem",
    "Annie Lennox - No More 'i Love You's'",
    "Jam - Eton Rifles",
    "Black Eyed Peas / Will.i.am - I Gotta Feeling",
    "Survivor - Eye Of The Tiger",
    "Alanis Morissette - You Learn",
    "Who - Who Are You",
    "Del Amitri - Roll To Me"
]

output = []
for song in song_list:
    try:
        output.append(ytmusic.search(song))
        print("SUCCESS \t {}".format(song))

    except Exception as exc:
        print("FAIL \t\t {} \n --->{}".format(song, exc))


print('",\n"'.join(song_list))

OUTPUT

FAIL             Fleetwood Mac - Big Love 
 --->'NoneType' object is not subscriptable
SUCCESS          Mike And The Mechanics - All I Need Is A Miracle
SUCCESS          Police - Walking On The Moon
SUCCESS          Walk The Moon - Shut Up And Dance
SUCCESS          U2 - Angel Of Harlem
SUCCESS          Annie Lennox - No More 'i Love You's'
SUCCESS          Jam - Eton Rifles
SUCCESS          Black Eyed Peas / Will.i.am - I Gotta Feeling
SUCCESS          Survivor - Eye Of The Tiger
SUCCESS          Alanis Morissette - You Learn
FAIL             Who - Who Are You 
 --->'NoneType' object is not subscriptable
SUCCESS          Del Amitri - Roll To Me

sigma67 · 2020-12-16T22:29:43Z

Unfortunately I still can't reproduce the issue on my end. However, thanks to your helpful debug output I believe I have identified the fix. Can you checkout the branch search_menu_fix and report back if that helps?

phirestalker · 2020-12-17T05:32:32Z

Yep, that fixed it.

harrytouche · 2020-12-17T08:56:54Z

Looks good to me, nice work!

Complete! Added 148 songs
---------------------------
{'success': 1, 'n_tracks': 148, 'n_failed': 0}

harrytouche · 2020-12-17T12:06:07Z

Confirmed with v0.12.1 through pip, thanks so much 🎉

sigma67 pushed a commit that referenced this issue Dec 16, 2020

Fix error on missing album browseId of unplayable item (#124)

8f64cf7

sigma67 pushed a commit that referenced this issue Dec 16, 2020

Fix issues with retrieving feedback tokens from search results (#124)

8b2fbd9

sigma67 closed this as completed Dec 17, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

0.12.0 search item renderer TypeError #124

0.12.0 search item renderer TypeError #124

harrytouche commented Dec 16, 2020

sigma67 commented Dec 16, 2020 •

edited

Loading

harrytouche commented Dec 16, 2020

harrytouche commented Dec 16, 2020

sigma67 commented Dec 16, 2020 •

edited

Loading

phirestalker commented Dec 16, 2020 •

edited

Loading

harrytouche commented Dec 16, 2020

sigma67 commented Dec 16, 2020

phirestalker commented Dec 17, 2020

harrytouche commented Dec 17, 2020

harrytouche commented Dec 17, 2020

0.12.0 search item renderer TypeError #124

0.12.0 search item renderer TypeError #124

Comments

harrytouche commented Dec 16, 2020

sigma67 commented Dec 16, 2020 • edited Loading

harrytouche commented Dec 16, 2020

harrytouche commented Dec 16, 2020

sigma67 commented Dec 16, 2020 • edited Loading

phirestalker commented Dec 16, 2020 • edited Loading

harrytouche commented Dec 16, 2020

sigma67 commented Dec 16, 2020

phirestalker commented Dec 17, 2020

harrytouche commented Dec 17, 2020

harrytouche commented Dec 17, 2020

sigma67 commented Dec 16, 2020 •

edited

Loading

sigma67 commented Dec 16, 2020 •

edited

Loading

phirestalker commented Dec 16, 2020 •

edited

Loading