"embedded" data.frame from JSON

Question

I have the following code, which extracts data from a JSON file.

library(jsonlite)

file_path <- 'C:/some/file/path.json'

df <- jsonlite::fromJSON(txt = file_path , 
                        simplifyVector = FALSE,
                        simplifyDataFrame = TRUE,
                        simplifyMatrix = FALSE,
                        flatten = FALSE)

The data structure is highly nested. My approach extracts 99% of it just fine, but in one particular part of the data I came across a phenomenon that I would describe as an "embedded" data frame:

df <- structure(
  list(
    ID = c(1L, 2L, 3L, 4L, 5L),
    var1 = c('a', 'b', 'c', 'd', 'e'),
    var2 = structure(
      list(
        var2a = c('v', 'w', 'x', 'y', 'z'),
        var2b = c('vv', 'ww', 'xx', 'yy', 'zz')),
      .Names = c('var2a', 'var2b'),
      row.names = c(NA, 5L),
      class = 'data.frame'),
    var3 = c('aa', 'bb', 'cc', 'dd', 'ee')),
  .Names = c('ID', 'var1', 'var2', 'var3'),
  row.names = c(NA, 5L),
  class = 'data.frame')

# Looks like this:
#   ID var1 var2.var2a var2.var2b var3
# 1  1    a          v         vv   aa
# 2  2    b          w         ww   bb
# 3  3    c          x         xx   cc
# 4  4    d          y         yy   dd
# 5  5    e          z         zz   ee

This looks like a normal data frame, and it behaves like that for the most part.

class(df)
# [1] "data.frame"

df[1,]
# ID var1 var2.var2a var2.var2b var3
# 1     a          v         vv   aa

dim(df)
# [1] 5 4
# One less than expected due to embedded data frame

lapply(df, class)
# $ID
# [1] "integer"
# 
# $var1
# [1] "character"
# 
# $var2
# [1] "data.frame"
# 
# $var3
# [1] "character"

str(df)
# 'data.frame': 5 obs. of  4 variables:
#   $ ID  : int  1 2 3 4 5
# $ var1: chr  "a" "b" "c" "d" ...
# $ var2:'data.frame':  5 obs. of  2 variables:
#   ..$ var2a: chr  "v" "w" "x" "y" ...
# ..$ var2b: chr  "vv" "ww" "xx" "yy" ...
# $ var3: chr  "aa" "bb" "cc" "dd" ...

What is going on here, why is jsonlite creating this odd structure instead of just a simple data.frame? Can I avoid this behaviour, and if not how can I most elegantly rectify this? I've used the approach below, but it feels very hacky, at best.

# Any columns with embedded data frame?
newX <- X[,-which(lapply(X, class) == 'data.frame')] %>%
  # Append them to the end
  cbind(X[,which(lapply(X, class) == 'data.frame')])

---

Update

The suggested workaround solves my issue, but I still feel like I don't understand the strange embedded data.frame structure. I would have thought that such a structure would be illegal by R data format conventions, or at least behave differently in terms of subsetting using [. I have opened a separate question on that.

Answer 1

I think you want to flatten your df object:

json <- toJSON(df)
flat_df <- fromJSON(json, flatten = T)

str(flat_df)

'data.frame':   5 obs. of  5 variables:
 $ ID        : int  1 2 3 4 5
 $ var1      : chr  "a" "b" "c" "d" ...
 $ var3      : chr  "aa" "bb" "cc" "dd" ...
 $ var2.var2a: chr  "v" "w" "x" "y" ...
 $ var2.var2b: chr  "vv" "ww" "xx" "yy" ...

Is that closer to what you're looking for?