Clojure EDN Walkthrough

When I set myself the task of learning how to use EDN, or Extensible Data Notation,  in Clojure, I couldn’t find a simple tutorial on how it worked, so I figured I would write one for anyone else having the same troubles I did. I was keen to learn EDN, as I am working on a Clojure based web application in my spare time, and wanted a serialisation format I could send data in that could be easily understood by both my Clojure and ClojureScript programs.

Disclaimer: This is my first blog post on Clojure, and I’m still learning the language. So if you see anything that can be improved / is incorrect, please let me know.

If you’ve never looked at EDN, it looks a lot like Clojure, which is not surprising, as it is actually a subset of Clojure notation.   For example, this is what a vector looks like in EDN:

Which should be fairly self explanatory.  If you want to know more about the EDN specification, have a read. You should be able to read through in around ten minutes, as it’s nice and lightweight.

The first place I started with EDN, was with the clojure.edn namespace, which has a very short API documentation and this was my first point of confusion. I could see a read and read-string method… but couldn’t see how I would actually write EDN?  Coming from a background that was used to JSON, I expected there to be some sort of equivalent Clojure to-edn function lying around, which I could not seem to find. The concept connection I was missing, was that the since EDN was a subset of Clojure, and the Clojure Reader supports, EDN, you only had to look at the IO Clojure functions to find that there are functions pr and prn whose job it is to take an object and “By default, pr and prn print in a way that objects can be read by the reader.”  Now pr and prn output to the current output stream. However we can use either pr-str or prn-str to give us a string output, which is far easier to use for our examples.  Let’s have a quick look at an example of that, with a normal Clojure Map:

Obviously this could have been written simpler, but I wanted to break it down into individual chunks to make learning a little bit easier. As you can see, to convert our sample-map into EDN, all we had to do was call prn-str on it to return the EDN string.  From there, to convert it back to EDN, it’s as simple as passing that string into the edn/read-string function, and we get back a new map with the same values as before.

So far, so good, but the next tricky bit (I found) comes when we want to actually extend EDN for our own usage. The immediate example that came to mind for me, was for use with defrecords. Out of the box, prn-str will convert a defrecord into EDN without any external intervention.  For example:

The nice thing here, is that prn-str provides us with a EDN tag for our defrecord out of the box: “#edn_example.core.Goat”. This let’s the EDN reader know that this is not a standard Clojure type, and that it will need to be handled differently from normal. The EDN reader makes it very easy to tell it how to handle this new EDN tag:

You can see that when we call edn/read-string we pass through an option of :readers with a map of our custom EDN tag as the key ( 'edn_example.core.Goat ), to functions that return what we finally want from our EDN deserialisation as the values ( map->Goat ).  This is the magic glue that tells the EDN reader what to do with your custom EDN tag, and we can tell it whatever we want it to do from that point forward. Since we have used custom EDN tags, if we didn’t do this, the EDN reader would throw an exception saying “No reader function for tag edn_example.core.Goat”, when we attempted to deserialise the EDN.

Therefore, as the EDN reader parses:

#edn_example.core.Goat{:stuff I love Goats, :things Goats are awesome}

It first looks at #edn_example.core.Goat, and matches that tag to the map->Goat function.  This function is then passed the deserialised value of {:stuff “I love Goats”, :things “Goats are awesome”} to it. map->Goat takes that map, and converts it into our Goat defrecord, and presto we are able to serialise and deserialise our new Goat defrecord.

This isn’t just limited to derecords, it could be any custom EDN we want to use. For example, if we wanted to write our own crazy EDN string for our Goat defrecord, rather than use the default, we could flatten out the map into a sequence, like so:

We can then apply the same strategy as we did before to convert it back:

Finally, there is the question of what do you do when you don’t know all the EDN tags that you will be required to parse? Thankfully the EDN reader handles this elegantly as well. You are able to provide the reader with a :default option, that gets called if a custom namespace can’t be found for the given EDN string, and gets passed the tag and the deserialised value as well.

For example, here is a function that simply converts the incoming EDN string into a map with :tag and :value values for the incoming EDN:

That about wraps up EDN. Hopefully that makes things easier for people who want to start using EDN, and encountered some of the stumbling blocks I did when trying to fit all the pieces together. EDN is a very nice data format for usage with Clojure communication.

The full source code for this example can be found on Github as well, if you want to have a look. Simply clone it and execute lein run to watch it go.

Leave a Comment

Comments

  • James Sofra | December 4, 2013

    Nice write up!

    For people looking for extra performance reading EDN they may be interested in looking at libraries such as https://github.com/ptaoussanis/nippy

  • Fred | October 3, 2014

    Nice examples, could you post (or give permission to post) these to http://clojuredocs.org ?

  • Mark Mandel | October 3, 2014

    @Fred –
    This article is already linked already from http://clojuredocs.org/clojure.edn, but if you think there is a better place it could be linked from, or copied to? I don’t know if the code would make much sense without the full article?

    Happy to have suggestions though!

  • Fred | October 3, 2014

    I added a fragment here… http://clojuredocs.org/clojure.edn/read-string
    I think this supplies the essential message (i.e. read-string prn-str).

  • Mark Mandel | October 3, 2014

    @Fred

    Nice! Works for me 🙂

  • Alexis Gallagher | March 29, 2016

    I believe what you’ve written here is incorrect, and that if using pr, prn, pr-str, etc. does not generate a valid edn representation of user-defined defrecord types.

    In your example, you use defrecord to create a record type Goat. When you prn-str it, you get the following representation: #edn_example.core.Goat{:stuff "I love Goats", :things "Goats are awesome"}. Notice that the tag is “edn_example.core.Goat”.

    However, if you read the EDN spec ( https://github.com/edn-format/edn ), it has this to say about tags: “Tag symbols without a prefix are reserved by edn for built-ins defined using the tag system…. User tags must contain a prefix component, which must be owned by the user (e.g. trademark or domain) or known unique in the communication context.”

    In other words, the tag produced by prn-str is not valid edn for a user-defined tag, because it lacks a prefix component. To be a valid tag, it would need to be “#edn_example.core/Goat”.

    This is why you get an error if you try to do the round-trip: (clojure.edn/read (pr-str (->Goat "I love goats" "goats are awesome")))

    In other words, Clojure’s native de/serialization format is supplied by read-str and pr-str, and it can handle defrecords automatically. To use edn, you need to write a custom encoder/decoder to do so.

    I hope I am wrong about this.

  • Juan | July 12, 2016

    Finally i could get into edn format
    thanks for sharing!!

  • llsouder | June 18, 2017

    Thank you for writing this. I love the examples and the way you get right to the point.

    Why did read-str convert the string? What if I wanted strings? You wrote:
    “we get back a new map with the same values as before.”

    so I entered you example into the repl and got the same result shown here.

    Is {:foo bar :bar foo}) exactly the same as {:foo “bar” :bar “foo”})?

  • llsouder | June 18, 2017

    Thank you for writing this. I love the examples and the way you get right to the point.
    Why did read-str convert the string? What if I wanted strings? You wrote:
    “we get back a new map with the same values as before.”
    so I entered you example into the repl and got the same result shown here.
    Is {:foo bar :bar foo}) exactly the same as {:foo “bar” :bar “foo”})?