How to Drop Rows or Columns with missing data (NaN) in Pandas

by

You can drop rows or columns with missing data (e.g., with NaN) using dropna() in Pandas.

Drop rows with NaN:

df.dropna()

Drop columns with NaN:

df.dropna(axis=”columns”)

Example of dropping rows with NaN

By default, dropna() will drop rows that at least have 1 NaN. The following is an example. 

import pandas as pd
import numpy as np

# Create a dataframe with NaN
df = pd.DataFrame({'Col_1': [100, np.nan, 200, np.nan, 500],
'Col_2': [np.nan, 30, 100, 88, 55],
'Col_3': [88, 87, 79, 88, 55]})

# print out the dataframe with NaN
print('Original dataframe with NaN: \n', df)

# drop rows with NaN
df_new=df.dropna()

# print out updated dataframe removed rows with NaN
print('Dataframe removed rows with NaN: \n',df_new)

The following shows the original dataframe with NaN and the updated dataframe after being removed rows with NaN. 

Original dataframe with NaN: 
    Col_1  Col_2  Col_3
0  100.0    NaN     88
1    NaN   30.0     87
2  200.0  100.0     79
3    NaN   88.0     88
4  500.0   55.0     55

Dataframe removed rows with NaN: 
    Col_1  Col_2  Col_3
2  200.0  100.0     79
4  500.0   55.0     55

Example of dropping columns with NaN

You can drop columns that have at least 1 NaN by using dropna(axis='columns'). The following is the example showing that.

import pandas as pd
import numpy as np

# Create a dataframe with NaN
df = pd.DataFrame({'Col_1': [100, np.nan, 200, np.nan, 500],
'Col_2': [np.nan, 30, 100, 88, 55],
'Col_3': [88, 87, 79, 88, 55]})

# print out the dataframe with NaN
print('Original dataframe with NaN: \n', df)

# drop columns with NaN
df_new=df.dropna(axis="columns")

# print out updated dataframe removed columns with NaN
print('Dataframe removed columns with NaN: \n',df_new)

The following shows the original dataframe with NaN and the updated dataframe after being removed columns with NaN. 

Original dataframe with NaN: 
    Col_1  Col_2  Col_3
0  100.0    NaN     88
1    NaN   30.0     87
2  200.0  100.0     79
3    NaN   88.0     88
4  500.0   55.0     55

Dataframe removed columns with NaN: 
    Col_3
0     88
1     87
2     79
3     88
4     55

Further Reading

The following is a tutorial showing how to drop rows or columns with NaN in Pandas.