Pandas for Data Science: A Step-by-Step Guide

 Pandas is an open-source Python library widely used for data pre-processing. This library offers numerous predefined functions for managing raw data efficiently we will discuss about it in detail using the best example.

First of all, we need a dataset we can create by our own using Excel or SQL or download from various sources like Kaggle,aws, or other websites

https://www.kaggle.com/datasets/saadharoon27/diwali-sales-dataset

Diwali Data Set

When you receive data, it often comes in raw and isn’t directly usable for machine learning models. This is where data processing comes in. Pandas is a Python library specifically designed for data analysis, organization, and cleaning. It helps in structuring and preparing the data so that it can be effectively utilized in machine learning models. With Pandas, you can easily handle data manipulation tasks such as sorting, filtering, and cleaning, making it a crucial tool for preprocessing raw data into a format suitable for machine learning algorithms.

To use Pandas, you need to install it first using pip, which stands for Python Install Package. You can do this by running the command pip install pandas in your command line or terminal. After installing Pandas, you can import it into your Python script or notebook.

Pandas provide various functions to handle null values, remove unused columns, and convert categories into proper formats. These functions make data preprocessing tasks easier and more efficient. With Pandas, you can clean and prepare your data for machine learning tasks effectively.

In the below example, we will analyze sales data using pandas:

This file was created by us in Excel

Import Python Libraries

Note: we are using jupyter notebook to write these commands:

Read data=>

We are calling sales files here with the title “Salese2.csv” It should keep in the same folder

Data Information

Using the shape command we can know how many rows and columns are in the data table

According to the above command, there are 10 rows (first value in shape ) and 10 columns (second value is shape)

In info, we can get complete information regarding the dataset like how many columns, rows, and types of each value, etc..

In datasets many times some values are missing these are called nulls it's a very common problem we have to face mostly, nulls are like poison for our dataset cause of nulls our insights of data get biased results or wrong predictions so it is highly recommended first remove or fill nulls (it depends on your domain of dataset)

In this pic, we can see we have 3 nulls in Brand and 1 in dis

In the above dataset, we filled null brands with ‘“others”

Here we are filling dis with 0%

Sometimes we have a big size of dataset like we have around 25000 rows and we have 2 or 3 nulls then we can drop nulls instead of fill nulls

ex=> data.dropna(inplace=True) depends on domain and dataset

Filter Data

We can filter our data like in the above pic we are calling only Samsung brand

Categorization data

A lot of functions are in pandas so Click on the below link video tutorial on pandas

PLAY ALL

Comments

Post a Comment

Popular posts from this blog

From Data Collection to Model Deployment: A Step-by-Step Guide in Machine Learning

Image
1 Collect Data => To train we need data so we collect data from different sources like Excel, SQL, Web Scraping or many other sources... There are many sources like Kaggle or Stats, these are websites From where we can download the datasets, most of the datasets are available in CSV (Comma Separated Values), which is the Excel format that is most used in ML models. 2) Data Cleaning => Whenever we get data it is available in raw format which is not directly usable in ML purpose, it means some data is missing or there is null in data or some in wrong format. Is. Is. Are there any or some duplicates and much more. ,This type of data is called messy data. If we use it directly in ML models we may get wrong or binary predictions, confusing insights or low accuracy, so it is extremely important that we avoid using it in ML. First of all the data should be cleaned, for this we have many tools like pandas, sklearn. In Python, cloud options and many others but in initial ML journey pandas...
Read more

What is Machine Learning?

Image
Machine learning (ML) is a process where we provide input to a machine, and in return, the machine generates answers or responses. However, the machine doesn't respond randomly; it relies on patterns and information provided to it in advance. It's similar to teaching a child about different fruits by showing them various examples. The child learns to distinguish between fruits based on features such as color and shape. In the same manner, we expose the machine to different features and data. The machine comprehends each feature and input. Through a combination of mathematical and statistical approaches, and with the help of programming languages like Python, the machine learns to recognize patterns within the data. This understanding enables the machine to make predictions for future inputs. Overall, the entire process is known as machine learning, where the machine learns from data, understands patterns, and applies this knowledge to make predictions. In today's world, ...
Read more