In Clojure 1.3, How to read and write a file

Question

I\'d like to know the \"recommended\" way of reading and writing a file in clojure 1.3 .

1. How to read the whole file
2. How to read a file line by line

Answer 1

If the file fits into memory you can read and write it with slurp and spit:

(def s (slurp "filename.txt"))

(s now contains the content of a file as a string)

(spit "newfile.txt" s)

This creates newfile.txt if it doesnt exit and writes the file content. If you want to append to the file you can do

(spit "filename.txt" s :append true)

To read or write a file linewise you would use Java's reader and writer. They are wrapped in the namespace clojure.java.io:

(ns file.test
  (:require [clojure.java.io :as io]))

(let [wrtr (io/writer "test.txt")]
  (.write wrtr "hello, world!\n")
  (.close wrtr))

(let [wrtr (io/writer "test.txt" :append true)]
  (.write wrtr "hello again!")
  (.close wrtr))

(let [rdr (io/reader "test.txt")]
  (println (.readLine rdr))
  (println (.readLine rdr)))
; "hello, world!"
; "hello again!"

Note that the difference between slurp/spit and the reader/writer examples is that the file remains open (in the let statements) in the latter and the reading and writing is buffered, thus more efficient when repeatedly reading from / writing to a file.

Here is more information: slurp spit clojure.java.io Java's BufferedReader Java's Writer

Answer 2

This is how to read the whole file.

If the file is in the resource directory, you can do this:

(let [file-content-str (slurp (clojure.java.io/resource "public/myfile.txt")])

remember to require/use clojure.java.io.

Answer 3

(require '[clojure.java.io :as io])
(io/copy (io/file "/etc/passwd") \*out*\)

Answer 4

Regarding question 2, one sometimes wants the stream of lines returned as a first-class object. To get this as a lazy sequence, and still have the file closed automatically on EOF, I used this function:

(use 'clojure.java.io)

(defn read-lines [filename]
  (let [rdr (reader filename)]
    (defn read-next-line []
      (if-let [line (.readLine rdr)]
       (cons line (lazy-seq (read-next-line)))
       (.close rdr)))
    (lazy-seq (read-next-line)))
)

(defn echo-file []
  (doseq [line (read-lines "myfile.txt")]
    (println line)))

Answer 5

To read a file line by line you no longer need to resort to interop:

(->> "data.csv"
      io/resource
      io/reader
      line-seq
      (drop 1))

This assumes that your data file is kept in the resources directory and that the first line is header information that can be discarded.

Answer 6

Assuming we're only doing text files here and not some crazy binary stuff.

Number 1: how to read an entire file into memory.

(slurp "/tmp/test.txt")

Not recommended when it is a really big file.

Number 2: how to read a file line by line.

(use 'clojure.java.io)
(with-open [rdr (reader "/tmp/test.txt")]
  (doseq [line (line-seq rdr)]
    (println line)))

The with-open macro takes care that the reader is closed at the end of the body. The reader function coerces a string (it can also do a URL, etc) into a BufferedReader. line-seq delivers a lazy seq. Demanding the next element of the lazy seq results into a line being read from the reader.

Note that from Clojure 1.7 onwards, you can also use transducers for reading text files.

Number 3: how to write to a new file.

(use 'clojure.java.io)
(with-open [wrtr (writer "/tmp/test.txt")]
  (.write wrtr "Line to be written"))

Again, with-open takes care that the BufferedWriter is closed at the end of the body. Writer coerces a string into a BufferedWriter, that you use use via java interop: (.write wrtr "something").

You could also use spit, the opposite of slurp:

(spit "/tmp/test.txt" "Line to be written")

Number 4: append a line to an existing file.

(use 'clojure.java.io)
(with-open [wrtr (writer "/tmp/test.txt" :append true)]
  (.write wrtr "Line to be appended"))

Same as above, but now with append option.

Or again with spit, the opposite of slurp:

(spit "/tmp/test.txt" "Line to be written" :append true)

PS: To be more explicit about the fact that you are reading and writing to a File and not something else, you could first create a File object and then coerce it into a BufferedReader or Writer:

(reader (file "/tmp/test.txt"))
;; or
(writer (file "tmp/test.txt"))

The file function is also in clojure.java.io.

PS2: Sometimes it's handy to be able to see what the current directory (so ".") is. You can get the absolute path in two ways:

(System/getProperty "user.dir") 

or

(-> (java.io.File. ".") .getAbsolutePath)