Home 00 Dataguide 01 Github 02 Colabs 03 Shell Basics 04 Apis05 Webscraping 06 Nbdev 07 Javascript Packag... 08 Cloud Functions 09 Browser Extension

Don't Look! I'm changing!

URL Copied

Musical loops

BinderBinderBinderOpen Source Love svg3

NPM LicenseActiveGitHub last commit

GitHub starsGitHub watchersGitHub forksGitHub followers

TweetTwitter Follow

CSV

You don't need an API to pull remote data!

You can publish you data in a Google Sheets and pull it via url in your code.

Likewise, github hosted files are accessible in a similar manner.

categorydescriptionshortname
0CDTotal PopulationTpop
1CDTotal Male PopulationMale
2CDTotal Female PopulationFemale
3CDPercent of Residents - Black/African-American ...Paa
4CDPercent of Residents - White/Caucasian (Non-Hi...Pwhite
TRACTCE10GEOID10CSA2010
01010024510010100Canton
11020024510010200Patterson Park North & East
21030024510010300Canton
31040024510010400Canton
41050024510010500Fells Point

JSON

Simple JSON

Note how raw text can be loaded

And the JSON structure may vary.

Here the Json txt takes the form of an array of objects, commonly refered to as an 'object array'.

Notice the difference?

col1col2
row11x
row22y
row33z
col1col2
01x
12y
23z

Json by URL (First API Example)

Heres a Really Simple Example API

providerWARNING_UPGRADE_TO_V6termsbasedatetime_last_updatedrates
AEDhttps://www.exchangerate-api.comhttps://www.exchangerate-api.com/docs/freehttps://www.exchangerate-api.com/termsALL2021-06-2916249248020.0357
AFNhttps://www.exchangerate-api.comhttps://www.exchangerate-api.com/docs/freehttps://www.exchangerate-api.com/termsALL2021-06-2916249248020.7700
ALLhttps://www.exchangerate-api.comhttps://www.exchangerate-api.com/docs/freehttps://www.exchangerate-api.com/termsALL2021-06-2916249248021.0000

This is the census' ones

You can get a full list of tableId's from downloads here.

or explore the available ones by not submitting a tableID in the query

https://api.census.gov/data/2019/acs/acs5/groups/.json
groups
0{'name': 'B17015', 'description': 'POVERTY STA...
1{'name': 'B18104', 'description': 'SEX BY AGE ...
2{'name': 'B17016', 'description': 'POVERTY STA...
3{'name': 'B18105', 'description': 'SEX BY AGE ...
4{'name': 'B17017', 'description': 'POVERTY STA...

Notice how the response varys when we actually give it a valule.

In this case we are given back meta data on table columns.

The response to any query is given in the acs documentation

https://api.census.gov/data/2019/acs/acs5/groups/B17015.json
variables
B17015_001E{'label': 'Estimate!!Total:', 'concept': 'POVE...
B17015_001EA{'label': 'Annotation of Estimate!!Total:', 'c...
B17015_001M{'label': 'Margin of Error!!Total:', 'concept'...
B17015_001MA{'label': 'Annotation of Margin of Error!!Tota...
B17015_002E{'label': 'Estimate!!Total:!!Income in the pas...

Json by File

Sometimes Json doesn't come in the form you want.

In times like those, you can download the data directly to a file using the wget terminal command

This is the json - machine readable - file provided by the Census bureau.

It has the data we want stored as an attribute within a larger 'object'.

Here is one way to read in the json, access the attribute, and then load it up for use.

From there, you'd do whatever operations you need on it (either manuall or with code).

And then it can be loaded up as a dataset

Esri (Second API Example)

BNIA uses ARCGIS/ ESRI to store its data for programmatic retrieval in a bunch of our applicatons.

For certain API services, tools can be provided to help you build and model your query.

Here is ESRIs

To read in the Geospatial information provided by Esri requires the use of a speical library

OBJECTIDNameShape__AreaShape__Lengthgeometry
01Abell1.890312e+051796.137499POLYGON ((-76.61113 39.32345, -76.61167 39.323...
12Allendale1.052283e+064351.591130POLYGON ((-76.67263 39.29184, -76.67263 39.291...
23Arcadia5.854937e+053739.317928POLYGON ((-76.56853 39.33595, -76.56814 39.335...
34Arlington4.677565e+052973.669980POLYGON ((-76.68627 39.34791, -76.68646 39.348...
45Armistead Gardens1.224526e+065155.929124POLYGON ((-76.55880 39.30646, -76.55892 39.306...