Using google suggest with BeautifulSoup python library to get matching keywords

Beatutifulsoup
If we enter any keyword in google search box , we will get a set of suggestions from google matching the input keyword for us to use as autocomplete.
Using this keyword we can collect the suggestions from google by using BeautifulSoup.

Here is the link to get XML data from google.

Check xml data from google

import requests
from bs4 import BeautifulSoup
str1='Python' # search string 
link = "https://suggestqueries.google.com/complete/search?output=toolbar&hl=en&q="+str1
content = requests.get(link)
soup = BeautifulSoup(content.text, "xml") # creating bs4 object 
for d in soup.find_all('CompleteSuggestion'): # loop through all suggestions 
        print(d.suggestion['data'])

Output is here

python
python online compiler
python download
python interview questions
python compiler
python tutorial
python programming
python w3schools
python interview questions for freshers
python basics

User Input Search string

#str1='Python' # search string 
str1=input("Enter keyword: ") # user input for search string

Using multiple keywords

import requests
from bs4 import BeautifulSoup
kw=['Tkinter','Python']
kw_data=[]
for s in kw:
    link = "https://suggestqueries.google.com/complete/search?output=toolbar&hl=en&q="+s
    content = requests.get(link)
    soup = BeautifulSoup(content.text, "xml")
    my_list=[] 
    for d in soup.find_all('CompleteSuggestion'):
        #print(d.suggestion['data']) # printing the data
        my_list.append(d.suggestion['data']) # adding to list
    kw_data.append(my_list)
print(kw_data)

We can store the above data in a google sheet, add this part at the end of the above code.
Python- Pygsheets

import pygsheets
path='G:\\My drive\\testing\\google-sheet\\creds1.json'
gc=pygsheets.authorize(service_account_file=path)
sh=gc.open('my_gsheets1')
wk1=sh[0]
wk1.append_table(kw_data,start='A1')
wk1.adjust_column_width(start=1,end=10,pixel_size=None)

Before scraping read the website’s legal use of data and avoid frequent request for data to website.

BeautifulSoup find_all children and parent tags select Python- Tutorials

Subscribe to our YouTube Channel here

Google suggest Keywords

User Input Search string

Using multiple keywords

Subscribe