Module error

[ghost]

Hi,
I am new to data science (and programming).
I tried to analyze an instagram account but Python tells me there is "no module named 'dynamic_profile'.
Though I imported the full insta-scrape from github.
Could you please help me out?

[chris-greening]

Hello @hartard and welcome to programming/data science!

dynamic_profile is a module that only appears in some of the examples (i.e. here) and is not an official module that can be imported from the library. The reason it's there is to show how dynamic content can be acquired by extending instascrape with a tool such as selenium. Check out the docs to see what tools are officially supported!

Out-of-the-box, instascrape does not render any JavaScript and only scrapes data served back by Instagram in the form of static HTML content. You're certainly not alone in wanting to scrape dynamic content as I've had quite a few people asking for a step-by-step tutorial on writing a dynamic content scraper so I'll be releasing that tutorial sometime in the near future 😄

I'm not sure if there is a specific user you're looking to analyze but if all you want to do is analyze some Instagram data for learning purposes, I've included a dataset and full analysis tutorial in the repo's tutorial/examples section that might be of some use to you!

[ghost]

Thank you very much for your fast and kind reply!
I will look into it (and hopefully continue to learn) 👍

Edit: actually, I want to analyze an instagram account (for example: the last 20 posts) and then plot the data (likes, comments).
I have seen that in your "get_recent_posts" function the limit is 12. Is there a reason for this or can I change it to any other number?

[chris-greening]

No problem at all 😄!

12 posts is unfortunately the max you can get without some sort of JavaScript rendering since 12 is what Instagram serves back in the static HTML.

It isn't a great solution but you can copy-paste the first 20 URL's from a profile into a text file or stick them in a list and then instantiate Post objects that way for scraping.

i.e.

url_list = ["https://www.instagram.com/p/CI8sdF0BOd8/", "https://www.instagram.com/p/CI6G2JNBfRy/", "https://www.instagram.com/p/CIoI9j6BpFT/"]
scraped_posts = [] 
for url in url_list:
    post = Post(url)
    post.scrape()
    scraped_posts.append(post)

Alternatively, I'm not sure how experienced you are with programming yet but if you're comfortable you can hack up a selenium script as I've mentioned and get as many as you want that way. Since Instagram serves more posts back as you scroll, the way I typically do this is:

• inject JavaScript into an automated selenium browser to scroll the page
• collect the HTML at every scroll and store in a list
• instantiate BeautifulSoup for each collected HTML
• parse the unique shortcode out of the <script> tags
• instantiate instascrape Post objects for each shortcode and scrape

It's kind of a hacky solution but I've had success scraping hundreds of posts at a clip with this method.

[ghost]

Those are great ways. I think, I am going with the first (easy) one, maybe I'll try the other later.
You helped me a lot. Thank you!