read csv vias pandas of different amount of columns and merge them to one file

Question

I\'m struggeling in the last step of building a crawler. If I try to crawl a couple of files with the same structure it works perfect. But if I try to grab ones with an olde