Charles Karpati | API's and loading data

Note how raw text can be loaded

And the JSON structure may vary.

Here the Json txt takes the form of an array of objects, commonly refered to as an 'object array'.

Notice the difference?

Heres a Really Simple Example API

This is the census' ones

You can get a full list of tableId's from downloads here.

or explore the available ones by not submitting a tableID in the query

Notice how the response varys when we actually give it a valule.

In this case we are given back meta data on table columns.

The response to any query is given in the acs documentation

Sometimes Json doesn't come in the form you want.

In times like those, you can download the data directly to a file using the wget terminal command

This is the json - machine readable - file provided by the Census bureau.

It has the data we want stored as an attribute within a larger 'object'.

Here is one way to read in the json, access the attribute, and then load it up for use.

! wget https://api.census.gov/data.json

import json
 url = 'data.json'
 with open(url) as data_file:    
     data = json.load(data_file) 
 
 # print( json.dumps(data['dataset']) );
 
 metaDataTable = pd.DataFrame(data['dataset'])
 metaDataTable.head()

! wget --output-document=todos.json https://jsonplaceholder.typicode.com/todos/2

From there, you'd do whatever operations you need on it (either manuall or with code).

And then it can be loaded up as a dataset

import json
 with open('todos.json') as data_file:    
     json = json.load(data_file)
 
 print(f"This is what it looks like: {json}")
 
 df = pd.DataFrame(json)
 df.head(1)