convert json perline to panads data frame

Search…

Sometimes we have a file that contains a json object per line. For example log file in json format
1
{foo: 1, bar: 2}
2
{foo: 3, bar: 4}
Copied!
if we want to read this in python pandas we need to convert it to
1
[
2
  {foo: 1, bar: 2},
3
  {foo: 3, bar: 4}
4
]
Copied!
easy way to do this is with jq --slurp
1
$ cat file.json | jq --slurp . > one_array.json
Copied!
then you can read it in python pandas (notebook) like this
1
sf = pd.read_json('one_array.json')
Copied!

analytics

hierarchical data format

Last modified 1mo ago

Copy link