How to Reference the External Jar in Flink

Question

everyone. I tried to reference my company jar in Flink in the way of copying it to $FLINK/lib in all of taskmanagers, but failed. And I don\'t want to package a fat jar, which i

Answer 1

Flink's Command Line Interface (CLI) allows passing additional jar location paths using the -C option. We use it to pass dependencies to each job.

Our problem: Given that usually our jobs evolve during the whole project lifetime and that their external dependencies change their versions and that we run several processes in the same cluster, we wanted to select the exact jar versions to load in each run. Therefore, the $FLINK/lib directory was not enough for us.

Details: What we do is to distribute the jars to a fixed directory (different from $FLINK/lib) on every node. Later we use the CLI to start the job (not directly as the call is quite long, but using a bash script to abbreviate the call).