2021年2月5日星期五

how to read a multiline nested json in spark scala

I have a json file as below,

[   {    "WHO": "Joe",    "WEEK": [      {        "NUMBER": 3,        "EXPENSE": [          {            "WHAT": "BEER",            "AMOUNT": 18.00          },          {            "WHAT": "Food",            "AMOUNT": 12.00          },          {            "WHAT": "Food",            "AMOUNT": 19.00          },          {            "WHAT": "Car",            "AMOUNT": 20.00          }        ]      }    ]   }  ]  

I executed the below set of code,

import org.apache.spark.sql.SQLContext  val sqlContext = new SQLContext(sc)  val jsonRDD = sc.wholeTextFiles("/test.json").map(x => x._2)  val jason = sqlContext.read.json(jsonRDD)  jason.show  

Output:

multiline nested json

It shows WrappedArray in the output. How can we explode the data?

https://stackoverflow.com/questions/66073220/how-to-read-a-multiline-nested-json-in-spark-scala February 06, 2021 at 11:07AM

没有评论:

发表评论