Spark: optimise writing a DataFrame to SQL Server

Question

I am using the code below to write a DataFrame of 43 columns and about 2,000,000 rows into a table in SQL Server:

dataFrame
  .write
  .format(\"jdbc\")
  .mode(         


        

Answer 1

is converting data to CSV files and copying those CSV's is an option for you? we have automated this process for bigger tables and transferring those in GCP in CSV format. rather than reading this through JDBC.