convert json data to Pandas DataFrame
Project description
json2df is a library batch processing a lists of json data (multiple instances of same structured json data) into Pandas DataFrame
installation
$ pip install json2df
usage example
e.g. when you scrape some users info data from a website, usually some fields contains json data format
import pandas as pd
df = pd.read_csv('https://raw.githubusercontent.com/shichaoji/json2df/master/sample.csv')
df.shape
(100, 5)
df.head(3)
e.g. we want to extract the location field and convert into a dataframe
select location
df['location'].head()
0 {u'administrative_area': None, u'city': u'Sing... 1 {u'administrative_area': None, u'city': u'Bris... 2 {u'administrative_area': None, u'city': u'Ambo... 3 {u'administrative_area': None, u'city': u'Drob... 4 {u'administrative_area': None, u'city': u'Torr... Name: location, dtype: object
view first row
first_row = df['location'].head()[0]
print type(first_row)
<type 'str'>
first_row
"{u'administrative_area': None, u'city': u'Singapore', u'country': {u'highres_flag_url': u'/img/flags/highres_png/singapore.png', u'code': u'sg', u'name': u'Singapore', u'seo_url': None, u'flag_url_cdn': u'//cdn2.f-cdn.com/img/flags/png/sg.png', u'highres_flag_url_cdn': u'//cdn6.f-cdn.com/img/flags/highres_png/singapore.png', u'phone_code': None, u'language_code': None, u'demonym': None, u'language_id': None, u'person': None, u'iso3': None, u'sanction': None, u'flag_url': u'/img/flags/png/sg.png', u'flag_class': u'singapore', u'region_id': None}, u'vicinity': None, u'longitude': None, u'full_address': None, u'latitude': None}"
convert the string representation into a python dictionary
as you can see the json data has inner loop
import ast
ast.literal_eval(first_row)
{u'administrative_area': None, u'city': u'Singapore', u'country': {u'code': u'sg', u'demonym': None, u'flag_class': u'singapore', u'flag_url': u'/img/flags/png/sg.png', u'flag_url_cdn': u'//cdn2.f-cdn.com/img/flags/png/sg.png', u'highres_flag_url': u'/img/flags/highres_png/singapore.png', u'highres_flag_url_cdn': u'//cdn6.f-cdn.com/img/flags/highres_png/singapore.png', u'iso3': None, u'language_code': None, u'language_id': None, u'name': u'Singapore', u'person': None, u'phone_code': None, u'region_id': None, u'sanction': None, u'seo_url': None}, u'full_address': None, u'latitude': None, u'longitude': None, u'vicinity': None}
user json2df to convert the entire location field (Series) into a DataFrame
from json2df import series2df
extract_df = series2df(df['location'])
print (extract_df.shape)
extract_df.head(5)
(100, 22)
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
json2df-0.1.3.7.tar.gz
(4.1 kB
view hashes)
Built Distributions
json2df-0.1.3.7-py2.7.egg
(6.5 kB
view hashes)
Close
Hashes for json2df-0.1.3.7-py2-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | aec593643bc52c911247d51d3cd5aad4926d3e84ccc12934154bcbff5a078a79 |
|
MD5 | d64907460e289fe3aebc873371ba3f0d |
|
BLAKE2b-256 | e1e901fbce64392e4c7fe1b7c983389a56fef2073d17fca2f8ed4500099f1542 |