Web Scraping Using RPA Tool UiPath..!

Darwin gives Survival of fittest thoery for living being but it also applicable on AI as AI keeps updating or adopting itself.

Crawl the Web & get exactly what you want..!

INTRODUCTION

Web Scraping, Web Data Extraction & Web Harvesting is the collection of data from the web. Finding the data is good; extracting it even better; but doing it using automation is perfect.

Get to know the Tool

What is UiPath..?

UiPath is a RPA Tool.

What is RPA..?

Robotic Process Automation is the technology that allows anyone today to configure computer softwares. Or in other words it is a “robot” to emulate and integrate the actions of a human interacting within digital systems to execute a business process. RPA robots utilize the user interface to capture data and manipulate applications just like humans do. They interpret, trigger responses and communicate with other systems in order to perform on a vast variety of repetitive tasks.

Experiential-Session

Performed on Versions

UiPath — 20.4.3

Let’s perform web scraping using UiPath. Just check the website for the data you want to scrape and check the list of parent and child HTML tags for better understanding.

Steps to follow to do Web Scrape

· Select the Website and the Data

· Create a Project in your desired directory

· Create a Flowchart file for Webscraping flow design

· Design the Flow

· Run the Automation flow

· Cross-check the Scraped Data

Step 1- Select the Website and the Data

I select this website “https://www.bullion-rates.com/gold/INR/2007-1-history.htm” and want to scrape data of gold rates along with dates.

Data to Scrape

Step 2- Create a Project in your desired directory

Provide the Name, path & short description of your project.

Create the Project

Step 3- Create a Flowchart file

Now create a flowchart file to design your Webscraping flow.

Create a flowchart file

Step 3- Design the Flow

a) Choose the open browser from activity pane

Open Browser

b) Set the Property of Open Browser

i) Choose browser type as Chrome

ii) Set the URL- Insert your url within quotes here https://www.bullion-rates.com/gold/INR/2007-1-history.htm

iii) Choose New Session as True

Open Browser Properties

iv) Add a delay activity with duration of 6 sec in the format of 00:00:06, so that page will load properly. There are others option as well but for now I’m using delay option.

Delay activity

v) Choose Data Scraping option

a) Select Element option is coming

b) Select Next option

Select Element option

c) Now the Element selector highlighter will come to select the element. Once the elements are selected, you can see the preview data. If the data are coming as expected select the finish button else re-select the data.

Preview Selected Data

d) Now a pop-up box appears which ask for multiple pages scraping, so if you want to do multiple pages scraping then select Yes & choose the element which will redirect you to next page. I want to scrape one page only so I’m using No option.

Multiple Page Scrape

e) Data Scraping activity will appear in the flow design. Select the Extract Structured Data ‘TABLE dtDGrid’ activity & you can notice two things in properties

i) Max number of results default is 100, you can change it as per the records in the page.

ii) In the output section, you can see the Data Table variable is Extract Data Table.

Data Scraping activity Properties

f) Now, we have to write the scraped data in excel format. So we use Write Range activity.

i) 1st field is for the path of excel sheet, provide it as per your excel sheet location.

ii) 2nd field is for sheet name & Cell name, provide the sheet name in quotes & remove the cell name. So that it will create the sheet & write the whole data.

iii) Last field is for variable name, in my case variable name is ExtractDataTable.

Write Range Activity

Step 4- Run the Automation flow

Click on the run option or press ctrl +f6 to run the automation flow.

Step 5- Open the Excel file & Cross-check the Scraped Data

Scraped Data

Conclusion- I tried to explain Web Scraping using RPA Tool, UiPath in a very simple way, Hope this will help you.

Find full code on

access full code on github

If you have any questions about the code or web scraping in general, reach out to me on

Connect to Gyan on Linkedin

We will meet again with something new.

Till then,

Happy Coding..!

Stay Home..Stay Safe

Data Scientist at Private Organization