Pythonic way to merge two overlapping lists, preserving order

Question

Alright, so I have two lists, as such:

• They can and will have overlapping items, for example, [1, 2, 3, 4, 5], [4, 5, 6, 7].
• The

Answer 1

All the above solutions are similar in terms of using a for / while loop for the merging task. I first tried the solutions by @JuniorCompressor and @TankorSmash, but these solutions are way too slow for merging two large-scale lists (e.g. lists with about millions of elements).

I found using pandas to concatenate lists with large size is much more time-efficient:

import pandas as pd, numpy as np

trainCompIdMaps = pd.DataFrame( { "compoundId": np.random.permutation( range(800) )[0:80], "partition": np.repeat( "train", 80).tolist()} )

testCompIdMaps = pd.DataFrame( {"compoundId": np.random.permutation( range(800) )[0:20], "partition": np.repeat( "test", 20).tolist()} )

# row-wise concatenation for two pandas
compoundIdMaps = pd.concat([trainCompIdMaps, testCompIdMaps], axis=0)

mergedCompIds = np.array(compoundIdMaps["compoundId"])