# duplicate_num = df3.duplicated().sum()
# duplicate_num
df3.drop_duplicates()
finding out the duplicated values and their sum and dropping out them

df3.InvoiceNo.str.contains('C').sum() specially categorizing by column and using .str.contains('C') which stands for finding out the C letter in that column

df3 = df3[df3['Quantity'] > 0]
filtering out where Quantity is more than

by_country = df3 \
.query("Country == 'Germany'") \
.aggregate({'CustomerID': 'count'})

taking firstly database then .querying('column == "Country"')
then aggregating

percentile_80 = df3['InvoiceNo'].value_counts().quantile(0.8)
filtered_users = df3['InvoiceNo'].value_counts()[df3['InvoiceNo'].value_counts() > percentile_80]
germany_top = filtered_users.index.unique()
germany_top


using filtering function with percentile

top_retail_germany = df3[df3['InvoiceNo'].isin(germany_top)]
top_retail_germany



finding out the column which contains in another column also

df3_g = df3 \
.groupby(['StockCode', 'Description'], as_index=False) \
.aggregate({'Quantity': 'count'}) \
.sort_values('Quantity', ascending=False)
df3_g


grouping by sorting and agg

df3.loc[:, 'Revenue'] = df3['Quantity'] * df3['UnitPrice']
df3

using loc with: only which stands for all then with comma, using column name for assigning it

df3_total_Revenue = df3 \
.groupby('InvoiceNo')['Revenue'].sum()

using groupby by Invoice No which means individually and by Revenue which totally means Individual Income

pd.to_datetime(ads_data.time, unit='s') with this you will make seconds carefully look

pd.to_datetime(ads_data.date) date only

ads_data.groupby('date') \
.agg({'ad_id': 'count'}).plot()

using groupby and aggregating and ploting

ads_data.groupby(['date', 'event'], as_index=False) \
.agg({'ad_id': 'count'}) \
.pivot(index='date', columns='event', values='ad_id')\
.reset_index()


using groupby and pivoting by date and event and ad_id

ads_data[ads_data.date == '2019-04-05'] \
.groupby('ad_id') \
.agg({'time': 'count'}) \
.sort_values('time', ascending=False) \
.head()

specifiying time and aggregating and sorting out the value time

ads_data.query('ad_id == @ad_id').head(1) with query you will find out the this one

.pivot(index='ad_id', columns='event', values='time').reset_index() this with reset_index helps to start index by again

ads_data_by_ad.assign(ctr = ads_data_by_ad.click / ads_data_by_ad.view,
ctr_per = 100 * ads_data_by_ad.click / ads_data_by_ad.view)

when there is 2 variables and you are assinging that in one table new variables aare in here

ads_data_by_ad.query('view == 0').ad_id finding out the missing column

ads_data[ads_data.ad_id.isin(ads_ids_bug)] finding out the wrong things

ads_data[ads_data.ad_id == 16548].sort_values('time') sorting out with time

okay download the dataset work with unproper columns like convert them or with usecols you can filter out in the begining then work with isnull values then find out what we need and use groupby query agg and use why not debugs then quantiles after finding out the problem issue we will look at the only data that we need and find out the root cause