How do I scrape data from all pages

RPA Automation Studio Pro
1

Hey Forum,

I want to scrape data till last page. How can i do it?

1 Like
2

@Pooja Hi!
In order to scrape data from all pages, it is necessary to do data processing in a loop through the pages. For each iteration, open each page using a URL value you can save from the next page’s href value.

image
image960×874 33.4 KB

Here you can find some useful information about Web Scraping Activities:

2 Likes
3

Thank you @d.motta.

I’ll try.

3 Likes
4

Hi @d.motta,

I gone through the video. In that scraping is done for a single page. Here, i have multiple pages.

Is there any other method to scrape data from multiple pages?

5

@Pooja Hi! In the first message, I mentioned that you need to build a cycle with iterations for every page. After scraping from one page the bot clicks on the next page and scrapes again. Save a link from the button on the next page and open the browser with the URL value taken from the saved link.

1 Like
6

Hi @Pooja

As mentioned @d.motta , try that and save into excel based on the iteration keep loop counter for that. In the loop keep element exist and check Next button is enabled or not. Below is the screenshot for your reference.

Work flow :

image
image874×871 27.9 KB

File saving for each file in loop:
image

4 Likes
7

Thank you for sharing your workflow @Kiran_Yadavalli. Please keep contributing to other questions.

@Pooja, please let us know if this works.

8

Thank you @Kiran_Yadavalli

1 Like
9

Sure @Mabwa_Neek .

I’ll try and let you know.

1 Like
10

That looks like a very good solution. I don’t understand why the Append to file activity is connected to the little red dot in the Counter. What is the red dot for? (I’m new to using ElectroNeek and so far I’ve never used this red dot)

2 Likes
11

@cris-dsc, while I’m not sure why @Kiran_Yadavalli did that :thinking:, the red dots are to capture exceptions if that specific activity fails. I recommend you reading the following article: Exceptions handling – ElectroNeek Help Center

1 Like
12

I usually make a first loop through all the items on a page, making sure that it’s working well.

Only after that we put a loop outside that one going from page to page. We need to loop if there’s a URL change each page of we’ll always click on the next page button. In the end, we need to understand the best way to stop on the last page.

1 Like