Data cleaning by using Pandas methods

Data we get from different sources may not be in the form or format for direct use in our applications. We need to correct these data by using various methods available in Pandas.

thresh	Using thresh option of dropna() removing rows and columns
dropna	Delete NaN rows or columns
df.drop_duplicates	Delete Duplicate rows from DataFrame
duplicated	duplicate rows from DataFrame
Series.duplicated	duplicate value from Series
Series.drop_duplicates	Delete Duplicate data from Series
replace	Replace data
notnull	Check for Not Null and NaN data
fillna	Fill NA/NaN values

dtypes	Pandas Data types
select_dtypes	Subset of DataFrame based on data type

Removing comma from a string column

df['p_view']=df['p_view'].apply(lambda x: x.replace(',',''))

Converting to integer data type

df['p_view'] = df['p_view'].astype('int')

A regex which selects only characters in UTF-8, removing the rest, for each field in the dataframe.

df.replace({r'[^\x00-\x7F]+':''}, regex=True, inplace=True)

Pandas columns() add_prefix() add_suffix()

Numpy arrays Python & MySQL Python- Tutorials

Subscribe to our YouTube Channel here

Cleaning Data

Subscribe