Scraping google search result

[swapnali42]

I want scrape all data from html page but I get result which is not seen on that page

import requests
from bs4 import BeautifulSoup
import re
from selenium import webdriver

query = input("What would you like to search: ")
query = query.replace(" ","+")
base = "http://www.google.de"
query = "https://www.google.com/search?q=" + query
#patterns = ["More places"]

driver = webdriver.Firefox(executable_path=r'C:\Users\Swapnali\AppData\Local\geckodriver.exe')
driver.get(query)
elem1 = driver.find_element_by_link_text("More places")
elem1.click()
Current_Url = driver.current_url
print('Current_Url')
print(Current_Url)
r = requests.get(Current_Url)
html_doc = r.text
soup = BeautifulSoup(html_doc, 'html.parser')
for s in soup.find_all('p'):
allDetail = s.text
print('allDetail')
print(allDetail)

My code is like above

Expected behavior

I want all details from that page
e.g if my query is

list of meter manufacturing companies in pune

Result will be like Compnay Names and all details

Actual behavior

D:\Web-Scrapping>python more-places-link.py
What would you like to search: list of meter manufacturing companies in pune
Current_Url
https://www.google.com/search?q=list+of+meter+manufacturing+companies+in+pune&npsic=0&rflfq=1&rlha=0&rllag=18493799,73906966,2651&tbm=lcl&ved=0ahUKEwjoqqmL2pjbAhWKvY8KHQ9yBU8QjGoIWg&tbs=lrf:!2m1!1e2!2m1!1e3!3sIAE,lf:1,lf_ui:2&rldoc=1#rlfi=hd:;si:;mv:!1m3!1d55336.219470077114!2d73.85998475!3d18.49845465!2m3!1f0!2f0!3f0!3m2!1i516!2i351!4f13.1;tbs:lrf:!2m1!1e2!2m1!1e3!3sIAE,lf:1,lf_ui:2
allDetail
Your search - list of meter manufacturing companies in pune - did not match any documents. Reset search tools
allDetail
Suggestions:Make sure that all words are spelled correctly.Try different keywords.Try more general keywords.Try fewer keywords.
allDetail

D:\Web-Scrapping>

Environment settings (for bug reports)

• OS: windows 10
• Python version: 3.7
• Locust version:

[heyman]

As far as I can tell, this has nothing to do with Locust.

[swapnali42]

I am not using Locust

[swapnali42]

is there any other way to get that output

[heyman]

You're asking in the wrong forum. This is the issue tracker for the Locust projects. Perhaps you could try with stackoverflow.com or something similar.

[swapnali42]

ok. thank you