AWS Glue: How to handle nested JSON with varying schemas

Question

Objective: We\'re hoping to use the AWS Glue Data Catalog to create a single table for JSON data residing in an S3 bucket, which we would then query and parse v

Answer 1

you should add a glue classifier preferably $[*]

When you crawl the json file in s3, it will read the first line of the file.

You can create a glue job in order to load the data catalog table of this json file into the redshift.

My only problem with here is that Redshift Spectrum has problems reading json tables in the data catalog..

let me know if you have found a solution