Error when brace-expansion includes more than ~25 files in amazon-s3 read from spark

Question

I have just upgraded to using spark 3 instead of spark 2.4.

The following code ran fine in spark 2.4

df = spark.read.parquet(\'s3a://bucket/path/{\'+