Method 1 (OP)
pd.DataFrame([[item]+list(df.loc[line,'B':]) for line in df.index for item in df.loc[line,'A']],
columns=df.columns)
Method 2 (pir)
df1 = df.A.apply(pd.Series).stack().rename('A')
df2 = df1.to_frame().reset_index(1, drop=True)
df2.join(df.B).reset_index(drop=True)
Method 3 (pir)
A = np.asarray(df.A.values.tolist())
B = np.stack([df.B for _ in xrange(A.shape[1])]).T
P = np.stack([A, B])
pd.Panel(P, items=['A', 'B']).to_frame().reset_index(drop=True)
Thanks @user113531 for the reference to Alexander's answer. I had to modify it to work.
Method 4 (@Alexander) LINKED ANSWER
(Follow link and Up Vote if this was helpful)
rows = []
for i, row in df.iterrows():
for a in row.A:
rows.append([a, row.B])
pd.DataFrame(rows, columns=df.columns)
Timings
Method 4 (Alexander's) is the best followed by Method 3
