Data Analysis: Identifying Duplicate Rows in Excel


In Excel, there is funciton to remove duplicates; however, didn't find direct method to identifying duplicates based on specific columns.

Below is step created to find out duplicate ID columns for further analysis:

Column A contains all Customer IDs. We need to identify how many rows are generated for a customer id.

Step 1: Sort based on Column A
Step 2: Add 3 Columns with below calculations

=COUNT(MATCH(A10,A11,0)) as B10
=COUNT(MATCH(A10,A9,0))  as C10
=IF(B10=1,1,IF(C10=1,1,0)) or B10+C10  as D10

Now You can use D10 to filter on 1 to list down all duplicate records for further analysis for reason of what values are different or any other purpose.


Popular posts from this blog

Cognos Excel Text Wrapping Issue - Row Merge and Column Merge

Below are the properties to set to keep column size and width with text wrapping: Column Merge issue  http://www-01.ibm.com/support/docview.wss?uid=swg21674298 Set the advanced server property RSVP.EXCEL.XLS2007_COLUMN_WIDTH_CONTROL to TRUE for both BatchReportService and ReportService services.        Row Merge issue http://www-01.ibm.com/support/docview.wss?uid=swg21440556 RSVP.EXCEL.XLS2007_ALLOW_WRAPPING_SINGLE_CELL also to be set to TRUE  for both BatchReportService and ReportService services.   Once done, you can set any column width to be fixed and see same result in Excel as well as HTML. Please note that above properties will not be honored if you have unchecked Grouping repeating cells Report property in Report Properties menu. 
Read more

Cognos: How to replace an existing report without breaking links?

Below IBM document explains behavior of copy/move or overwriting impact of objects:- https://www.ibm.com/support/knowledgecenter/en/SSEP7J_11.0.0/com.ibm.swg.ba.cognos.wig_cr.doc/c_gtstd_copy_move_items.html If you copy and overwrite an existing report, the copied report ID and links replace those of the existing report. In this case, you might need to update links, such as links to job schedules. If you move and overwrite an existing report, the existing report ID and links are maintained. In this case, references to the moved entry are broken. However, this doesn't work in all cases and we see drill through links broken in some cases. Best approach in such case is to use cognos deploy method where you update reports in separate folder in same environment and import it to destination location. You can create single export and import tasks to use for all such updates.
Read more

How to change package for a report in IBM Cognos Analytics (ICA)?

In Cognos 10.x and 8.x , this was the simple to handle on Cognos Connection or via Repot Studio interface.  In Cognos 11.0.6, You  need to load classic view (Cognos Connection) to go back to this interface to change the package. In Cognos Cloud environment, we have single environment for Development and UAT. In such cases, when we move content from Dev folder to UAT area, we need to switch to classic mode to change the package.
Read more