How to grep into files stored in S3

Question

Do anybody know how to perform grep on S3 files with aws S3 directly into the bucket? For example I have FILE1.csv, FILE2.csv with many rows and want to look for the rows that

Answer 1

You can also use the GLUE/Athena combo which allows you to execute directly within AWS. Depending on data volumes, queries' cost can be significant and take time.

Basically

• Create a GLUE classifier that reads byline
• Create a crawler to your S3 data directory against a database (csvdumpdb) - it will create a table with all the lines across all the csvs found

• Use Athena to query, e.g.

  select "$path",line from where line like '%some%fancy%string%'

• and get something like

  $path line

  s3://mybucket/mydir/my.csv "some I did find some,yes, "fancy, yes, string"

Saves you from having to run any external infrastructure.