Charles Karpati | Importing Data

Whats inside?

In this notebook, we build and test a basic data-intaker.

• A .CSV file will be loaded into pandas

• We will import geospatial data from Esri then load this data into geo-pandas.

  Help on module dataplay.intaker in dataplay:

NAME
    dataplay.intaker

CLASSES
    builtins.object
        Intake
    
    class Intake(builtins.object)
     |  # The intaker class retrieves data into a pandas dataframe.
     |  # Can read in a CSV URL but uses dataplay.geom.readInGeometryData() for Geojson endpoints.
     |  # Otherwise this tool assumes shp or pgeojson files have geom='geometry', in_crs=2248. 
     |  # Depending on interactivity the values should be 
     |  # coerce fillna(-1321321321321325)
     |  
     |  Methods defined here:
     |  
     |  checkColumn(dataset, column)
     |      # a2. Returns Bool
     |  
     |  coerce(ds1, ds2, col1, col2, interactive)
     |      # b1. Used by Merge Lib. Returns Both Datasets and Coerce Status
     |  
     |  coerceDtypes(isNum, dt, ds, col, interactive)
     |  
     |  getAndCheck(url, col='geometry', interactive=False)
     |      # a1. Used by Merge Lib. Returns valid (df, column) or (df, False) or (False, False).
     |  
     |  getAndCheckColumn(df, col, interactive)
     |      # a3. Returns False or col. Interactive calls self
     |  
     |  getData(url, interactive=False)
     |      # 1. Recursively calls self/getData until something valid is given.
     |      #    Returns df or False. Calls readInGeometryData. or pulls csv directly.
     |      # Returns df or False.
     |  
     |  getdTypeAndFillNum(ds, col, interactive)
     |  
     |  isPandas(df)
     |      # 1ai. A misnomer. Returns Bool.
     |  
     |  ----------------------------------------------------------------------
     |  Data descriptors defined here:
     |  
     |  __dict__
     |      dictionary for instance variables (if defined)
     |  
     |  __weakref__
     |      list of weak references to the object (if defined)

FILE
    c:\users\charl\documents\github\karpatic\src\ipynb\dataplay\dataplay\intaker.py


 

And the same thing works without a url

Using Esri and the Geoms handler directly:

Again but with the Intake class:

This getAndCheck function is useful for checking for a required field.

We could also retrieve from a file.