logo
Tags down

shadow

Pandas - Comparing two dataframes by date and find missing entires


By : folecr
Date : October 16 2020, 08:10 AM
it fixes the issue You need merge with default inner join and then reindex by all unique values of emp_id and date, last merge with parameter indicator=True for filtering not reported rows:
code :
reported['date'] = pd.to_datetime(reported['date'])

df1 = reported.set_index(['emp_id', 'date'])
mux = pd.MultiIndex.from_product(df1.index.levels, names=df1.index.names)
df3 = (df1.reindex(mux)
         .reset_index()
         .merge(reported, indicator=True, how='outer')
         .query('_merge != "both"')
         .drop('_merge', axis=1))
print (df3)
   emp_id       date
2    1001 2018-11-03
4    1002 2018-11-02
6    1003 2018-11-01
7    1003 2018-11-02
df2 = df1.reindex(mux)
df3 = df2[~df2.index.isin(df1.index)].reset_index()
print (df3)

   emp_id       date
0    1001 2018-11-03
1    1002 2018-11-02
2    1003 2018-11-01
3    1003 2018-11-02


Share : facebook icon twitter icon

Comparing two DataFrames in Pandas


By : Haidar Haroun
Date : March 29 2020, 07:55 AM
I hope this helps . hello people (I am new to python) Question: how can I compare two DataFrames that have the same index but different formats. , Solution used
Better is use:
code :
dfres = res0.pivot(index='datum_von', columns='produktname', values='max')
dfdiff=dfres.sub(finaldf)
mask = dfdiff < 0
df = dfdiff.loc[mask.any(axis=1), mask.any()]]
dfa = dfres.loc[mask.any(axis=1), mask.any()]]
dfb = finaldf.loc[mask.any(axis=1), mask.any()]]

Need to find missing column value by comparing from 2 dataframes


By : anth
Date : March 29 2020, 07:55 AM
like below fixes the issue , This can also works :
code :
df1.loc[~df1['A'].isin(df2['A'])]
Out[34]:
    A   B
3   4   d
4   5   e

Comparing dataframes in pandas


By : Futshi
Date : March 29 2020, 07:55 AM
this will help I have two separate pandas dataframes (df1 and df2) which have multiple columns with some common columns. , This works:
code :
# set index (as selecting columns)
df1 = df1.set_index(['A','B'])
df2 = df2.set_index(['A','B'])

# now .isin will work
df2[~df2.index.isin(df1.index)].reset_index()

    A   B   D   text
0   45  3   1   shot
1   10  2   3   miss

Comparing two different sized pandas Dataframes and to find the row index with equal values


By : Erik Bij de Vaate
Date : March 29 2020, 07:55 AM
This might help you I need some help with comparing two pandas dataframe , You can using merge
code :
df1.merge(df2,indicator=True,how='left').loc[lambda x : x['_merge']=='both'].index
Out[459]: Int64Index([0, 3, 4, 6], dtype='int64')

Find the missing rows by comparing before and after when merging dataframes


By : user3720005
Date : March 29 2020, 07:55 AM
Hope that helps You simply need to do an outer join and set indicator to TRUE. Then you need to filter the required rows by your indicator column.
code :
import pandas as pd
df_prev = pd.DataFrame({'code':[111,222,555,666],
                   'col':['A','B','B','C']})
df_after = pd.DataFrame({'code':[111,222,555,777,888],
                   'col':['A','B','B','D','E']})

df_merged = pd.merge(df_prev, 
                     df_after,
                     how = 'outer',
                     left_on = ['code','col'],
                     right_on = ['code','col'], 
                     indicator = True)

deleted_rows = df_merged.loc[df_merged._merge == "left_only",]
 >> deleted_rows
    code    col     _merge
3   666     C   left_only
shadow
Privacy Policy - Terms - Contact Us © voile276.org