Combine Columns in multiple Excel workbooks and auto-de-dupe

Question

I have three workbooks with IDs in column A. I want to create a fourth workbook, which should combine the IDs and de-dupe them automatically so that I can perform a vlookup on t

Answer 1

You can retrieve a unique list of the id numbers in [ID_First.xlsx]Sheet1!$A$2:$A$999 using the following array formula in the master worksheet's A2 (needs a row above it to avoid circular references).

=IFERROR(INDEX([ID_First.xlsx]Sheet1!$A$2:$A$999,MATCH(0, IF(LEN([ID_First.xlsx]Sheet1!$A$2:$A$999),COUNTIF(A$1:A1,[ID_First.xlsx]Sheet1!$A$2:$A$999),1),0)),"")

If you stack similar formulas consecutively, passing calculation on to them with IFERROR(), you can gain a unique list from three separate workbooks.

=IF(LEN(A1),IFERROR(INDEX([ID_First.xlsx]Sheet1!$A$2:$A$999,MATCH(0, IF(LEN([ID_First.xlsx]Sheet1!$A$2:$A$999),COUNTIF(A$1:A1,[ID_First.xlsx]Sheet1!$A$2:$A$999),1),0)),IFERROR(INDEX([ID_Second.xlsx]Sheet1!$A$2:$A$999,MATCH(0, IF(LEN([ID_Second.xlsx]Sheet1!$A$2:$A$999),COUNTIF(A$1:A1,[ID_Second.xlsx]Sheet1!$A$2:$A$999),1),0)),IFERROR(INDEX([ID_Third.xlsx]Sheet1!$A$2:$A$999,MATCH(0, IF(LEN([ID_Third.xlsx]Sheet1!$A$2:$A$999),COUNTIF(A$1:A1,[ID_Third.xlsx]Sheet1!$A$2:$A$999),1),0)),""))),"")

Array formulas require Ctrl+Shift+Enter to finalize. Once entered correctly, fill down as necessary to collect all unique IDs.

With a unique list of id numbers, you can use the same method of nested IFERROR functions to look through a series of three workbooks for additional data.

=IFERROR(VLOOKUP($A2, [ID_First.xlsx]Sheet1!$A$2:$Z$999, 2, FALSE),IFERROR(VLOOKUP($A2, [ID_Second.xlsx]Sheet1!$A$2:$Z$999, 2, FALSE),IFERROR(VLOOKUP($A2, [ID_Third.xlsx]Sheet1!$A$2:$Z$999, 2, FALSE),"")))

I'm offering this as you've mentioned a total of 50 member IDs. This method can quickly (and logrythmically) eat up calculation resources when applied to larger groups of numbers.

Answer 2

If you've got Excel 2016 or later, you could unpivot the data using PowerQuery, which is now built in to Excel under the Get & Transform section of the Data tab in the ribbon. Plenty of examples of how to do this if you search Google for 'unpivot' and 'Powerquery'.

If you have Excel 2010 or 2013, you can download the PowerQuery addin for free from https://www.microsoft.com/en-nz/download/details.aspx?id=39379 (assuming IT let you do so).

PowerQuery is a revolution in Excel data transformation, and the learning curve is a lot less steep than advanced formulas or VBA.