I have a web page and with nested items. I’m using scrape structured data to get the data and then I loop through it and check for a condition with and If activity. If the condition is true, the bot expands that item, scrapes the new data exhibited (a sublist) and checks for another condition. I’m having trouble making the bot click in the correct item because some details in it changes depending on which item fulfills the condition.
This is the workflow:
I’ve tried using XPath for the element the bot must click and it seemed to work, but there are still cases in which a number in the XPath varies for each element and [*] isn’t working. Maybe the syntax is incorrect?
Examples of possible elements the bot must click:
/html/body/div[1]/div/div/div/div[5]/div[3]/div[2]/div[1]/div/button
/html/body/div[1]/div/div/div/div[5]/div[7]/div[2]/div[2]/div/button
/html/body/div[1]/div/div/div/div[5]/div[4]/div[2]/div[3]/div/button
The counter j used to control the loop that chesck each scraped item solves half the problem:
But I can’t use a counter for the other number that changes because it won’t necessarily follow and order. It depends whether or not the condition is fufilled. So I’ve tried this:
But it doesn’t work… Does anybody know what I’m missing here?
@cris-dsc Can you try with below xpath. Just by changing the div[*] with *
- /html/body/div[1]/div/div/div/div[5]/*/div[2]/div[“+j+”]/div/button
Thank you for trying to help.
I tried it and it clicked the wrong element. I tried what I was using before (div[*]) and it did the exact same thing. I realised the first element of the second list starts at
/html/body/div[1]/div/div/div/div[5]/div[5]/div[2]/div[2]/div/button instead of
/html/body/div[1]/div/div/div/div[5]/vid[5]/div[2]/div[0]/div/button
So now I’m using "/html/body/div[1]/div/div/div/div[5]/*/div[2]/div[“+j+2+”]/div/button". But when I tried running the bot after this modification, I get the following error:
It’s the same error I was getting before. No matter what I use - div[*], \*\ or \\div - I always get this same error. I don’t understand what’s happening. I got really hopeful after the bot clicked on the wrong element because at least it was clicking on something…
I’ve already opened a support ticket, but I’ll have to wait until friday for a zoom meeting.
Okay, I think I’ve found a way to make it work. But it’s not a solution to the question I asked…
I’m working with two lists here: the main list (counter is i) and the sublist (counter is j). My question focuses on the sublist, but the solution for my problem is in the main list. Upon studying the XPath of more elements from the sublist, I’ve realized there is a pattern:
“/html/body/div[1]/div/div/div/div[5]/div[1]/div[2]/div[”+(j+2)+“]/div/button”
“/html/body/div[1]/div/div/div/div[5]/div[2]/div[2]/div[”+(j+2)+“]/div/button”
“/html/body/div[1]/div/div/div/div[5]/div[3]/div[2]/div[”+(j+2)+“]/div/button”
The divs in bold mark the line of the main list, whereas the divs with j+2 mark the lines of the sublist. First, the bot must click on the line from the main list that fulfills a certain condition. Then it must click on the line from the sublist that fulfills another certain condition. So all I had to do was to use the counter from the main list as the number inside the divs in bold:
“/html/body/div[1]/div/div/div/div[5]/div[“+i+”]/div[2]/div[”+(j+2)+“]/div/button”
That still doesn’t really answer thw original question of how to make [*] work with a variable in the XPath. I’ve tried several different ways and all of them resulted in the error mentioned above.
@cris-dsc Could you please try with this.
/html/body/div[1]/div/div/div/div[5]/*/div[2]/div[“+(j+2)+”]/div/button
We are just changing this [“+j+2+”] as [“+(j+2)+”] because, () are needed also, because it would be a string with two numbers, not one.
Well, I’m pretty sure I had already tried that before and it didn’t work. I tried many different options and nothing worked. But for some reason it works now. Not only does your suggestion work, the following XPath also works (and if I remember correctly, I tried that too):
"/html/body/div[1]/div/div/div/div[5]/div[*]/div[2]/div["+(j+2)+"]/div/button"
