Export JSON from Spark and input into R

Question

I\'ve written some data from Spark to a JSON file and I am struggling to import it into R.

I cannot import it with any of the traditional JSON packages in R:

<         


        

Answer 1

It turns out that Spark exports streaming json files like those discussed in the following question: Error parsing JSON file with the jsonlite package

The solution is to use jsonlite's streaming function:

library(jsonlite)
json_file <- stream_in(file("win_rate_sample.json"))