How do I scrape data from all pages

Hey Forum,

I want to scrape data till last page. How can i do it?

@Pooja Hi!
In order to scrape data from all pages, it is necessary to do data processing in a loop through the pages. For each iteration, open each page using a URL value you can save from the next page’s href value.

Here you can find some useful information about Web Scraping Activities:

2 Likes

Thank you @d.motta.

I’ll try.

3 Likes

Hi @d.motta,

I gone through the video. In that scraping is done for a single page. Here, i have multiple pages.

Is there any other method to scrape data from multiple pages?

@Pooja Hi! In the first message, I mentioned that you need to build a cycle with iterations for every page. After scraping from one page the bot clicks on the next page and scrapes again. Save a link from the button on the next page and open the browser with the URL value taken from the saved link.

1 Like

Hi @Pooja

As mentioned @d.motta , try that and save into excel based on the iteration keep loop counter for that. In the loop keep element exist and check Next button is enabled or not. Below is the screenshot for your reference.

Work flow :


File saving for each file in loop:
image

2 Likes

Thank you for sharing your workflow @Kiran_Yadavalli. Please keep contributing to other questions.

@Pooja, please let us know if this works.

Thank you @Kiran_Yadavalli

1 Like

Sure @Mabwa_Neek .

I’ll try and let you know.

1 Like