Home > English > JSON.parse and the reviver function

JSON.parse and the reviver function

Building applications that store their data in NoSQL databases like MongoDB can sometimes be a challenge when dealing with dates and, even, IDs. In your front-end application, you could have put together the nicest form and have your dates formatted exactly as you like them but when you perform a POST to create a new record, the date ends up getting stored as a string.

Problem Statements

MongoDB really wants your dates in your documents to be date types. If you ever need to do any complex queries or aggregates, you will find that dates stored as a string type just do not work.

The same is true for the _id that MongoDB manages for you. It internally becomes an ObjectId. If you want to perform a PUT to update an existing record, you will need to pass a filter criteria. If that criteria contains the _id property, it needs to be represented as an ObjectId.

Why are these a problem?

The problem arises when we try to serialize our data and send it to our REST endpoints. The most common way to serialize JSON objects is to use JSON.stringify. Consider, for example, we have the following employee object from our front-end:

{
  first_name: "Matthew",
  last_name: "Duffield",
  start_date: Sun Jun 30 2019 17:00:00 GMT-0700 (Pacific Daylight Time)
}

As you can see, the start_date is represented as a native Date object. If we were to serialize this object using JSON.stringify and send it to our REST endpoint, we would see the following:

{
  "first_name": "Matthew",
  "last_name": "Duffield",
  "start_date": "2019-07-01T00:00:00.000Z"
}

Our date is no longer a native object, but a string representation. When we process this on our server, what do you think will be stored? We will get a string instead of a date.

OK, I get it. So, how do we fix this?

On the server, when we receive a request we will want to parse the body of the request. I am sure you have used JSON.parse a thousand times and never even thought about the second parameter. I know that is what I have done. Well, there is a second optional reviver function that you can pass. This function is used to perform a transformation on the resulting object before it is returned. This means that this function will be run for every property you have in a given JSON object. So, be careful what you put into this function.

You will see that the function has two parameters: key and value. This makes sense as we are simply traversing over all the properties on the object.

Let’s take a look at what an implementation might look like:

function reviver(key, value) {
  if (typeof value === 'string') {
    if (Date.parse(value)) {
      return new Date(value);
    } else if (key === '_id') {
      return ObjectId(value);
    }
  }
  return value;
}

Okay, what are we doing here? This reviver function is checking the type of the value. If it is of type string, then it performs an additional check to see if the value is a valid date. If it is a valid date, then we return a new date object. If the value is not a valid date, then we check to see if the key has the name of _id. If true, then we return an ObjectId of the value. Finally, if neither of these checks pass, then we simply return the value as is.

If you use the reviver function on the sample payload above, everything works just fine. This will now deserialize your date strings back to what they should be, date objects.

{
  first_name: "Matthew", 
  last_name: "Duffield", 
  start_date: Sun Jun 30 2019 17:00:00 GMT-0700 (Pacific Daylight Time)
}

Warning, Date.parse can be misleading

It turns out that Date.parse will eagerly accept values that aren’t really meant to be Dates. For example, if we were to add a postal_code property to our object and then call the JSON.stringify function, we would see the following:

{
  "first_name": "Matthew",
  "last_name": "Duffield",
  "start_date": "2019-07-01T00:00:00.000Z",
  "postal_code": "28210"
}

That looks like a perfect representation of our object and after we send our request over the wire to persist, our server would want to deserialize the string representation using our new reviver function. The following is what you would see:

{
  first_name: "Matthew", 
  last_name: "Duffield", 
  postal_code: Mon Jan 01 28210 00:00:00 GMT-0800 (Pacific Standard Time), 
  start_date: Sun Jun 30 2019 17:00:00 GMT-0700 (Pacific Daylight Time)
}

As you can see, our postal_code has been converted over to a date! That just won’t work for us. So, let’s go back to the reviver function and modify it ever so slightly. Here is what we have now:

function reviver(key, value) {
  if (typeof value === 'string') {
    if (/\d{4}-\d{1,2}-\d{1,2}/.test(value) ||
        /\d{4}\/\d{1,2}\/\d{1,2}/.test(value)) {
      return new Date(value);
    } else if (key === '_id') {
      return ObjectId(value);
    }
  }
  return value;
}

This time, we don’t use the Date.parse function. Instead we have decided to force the front-end to always serialize dates in the following to formats: yyyy-mm-dd or yyyy/mm/dd. You may want to enforce a different rule, just make sure that you uniquely identify dates.

Now that we have this change, let’s go ahead and deserialize our string again:

{
  first_name: "Matthew", 
  last_name: "Duffield", 
  postal_code: "28210", 
  start_date: Sun Jun 30 2019 17:00:00 GMT-0700 (Pacific Daylight Time)
}

Things are now looking exactly as they should! The same thing will happen if you were to perform a PUT and wanted to update a record based on the _id. If you pass in a filter criteria in the form of JSON, then when the server deserializes it into an object, it would automatically be converted to an ObjectId.

As you can see, the reviver function can really help out and keep your code clean. It removes the need to clutter your server side code with all the ceremony of remembering what properties were supposed to be dates and manually converting your _id properties.

  1. No comments yet.
  1. No trackbacks yet.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: