Web Scraping Using Beautiful Soup..!

4 min readSep 20, 2020

How to collect or scrape data using Beautiful Soup in Python..?

To start a business or to do a project in any field, the most basic & fundamental requirement is the data. Now the major problem arises, how to gather bits and pieces of information to achieve our goal.

Hi, I’m Gyan Vardhan & will provide you the basic idea, how to scrape the data using Beautiful Soup in Python.

Step 1:- Choose the website you want to scrape.

Today I scrape the gold prices from the website

Gold Prices - Indian Rupees (INR) - January 2007

Gold Price History in Indian Rupees (INR) for January 2007

www.bullion-rates.com

This is data we need to scrape.

Step 2:- Understand the website better in terms of their HTML tags. In a website all the information resides within the tags, we just need to look closely & find the exact tags in which our data are present. Let’s get started with finding the correct HTML tags. If you right click on the page & select inspect option then you can see all the HTML tags. But we need to find the exact tags so go to gold price table & do inspect.

In above screenshot you can see the HTML tags we need. The data we need from the website are in table format so our 1st approach is to check, Is there any table in the HTML tags & then find its respective table rows & table data. So I already highlighted the tags we need to scrape the data.

Step 3:-

Python Code

Importing important libraries

We need these 3 libraries to run the code.

Step 4:-

Set the url of the website & access it through requests library.

URL setup

If the requests get passed or successful then response will be 200; check below images.

success status response code

Now, (a) we have to write the code to find the table in which our data resides, there might be more than one table in the website, so in that case we have to give the class of the table to identify the exact table. In our case class of the table is “Grid History”. We use find() function & it will return our desired table.

Code will be like:-

(b) Identify the rows of the table:- Tag for the row is “tr”. To get the row we will use find_all() function, it is just like find() function but it will give all the matching results.

Code will be like:-

(C) Finally, we reached our last destination & Now, we have to find the body which resides in tag “td”. And to get all the data of all row, the code will be same but we need to loop the code to extract all the information at once.

Code will be like:-

Our Final Scraped Data

Conclusion:- Web Scraping is an amazing tool, that makes anyone’s life simpler.

Find full code on

vgyaan/Webscrape