Charles Karpati | Geo-Data Intake and Operations

Geographic data must be encoded properly order to attain the full potential of the spatial nature of your geographic data.

If you have read in a dataset using pandas it's data type will be a Dataframe.

It may be converted into a Geo-Dataframe using Geopandas as demonstrated in the sections below.

You can check a variables at any time using the dtype command:

Make sure the appropriate spatial Coordinate Reference System (CRS) is used when reading in your data!

ala wiki:

A spatial reference system (SRS) or coordinate reference system (CRS) is a coordinate-based local, regional or global system used to locate geographical entities

CRS 4326 is the CRS most people are familar with when refering to latiude and longitudes.

Baltimore's 4326 CRS should be at (39.2, -76.6)

BNIA uses CRS 2248 internally

Additional Information: https://docs.qgis.org/testing/en/docs/gentle_gis_introduction/coordinate_reference_systems.html

Ensure your geodataframes' coordinates are using the same CRS using the geopandas command:

When first recieving a spatial dataset, the spatial column may need to be encoded to convert its 'text' data type values into understood 'coordinate' data types before it can be understood/processed accordingly.

Namely, there are two ways to encode text into coordinates:

• df[geom] = df[geom].apply(lambda x: loads( str(x) ))

• df[geom] = [Point(xy) for xy in zip(df.x, df.y)]

The first approach can be used for text taking the form "Point(-76, 39)" and will encode the text too coordinates. The second approach is useful when creating a point from two columns containing lat/lng information and will create Point coordinates from the two columns.

More on this later

There exists two types of Geospatial Data, Raster and Vector. Both have different file formats.

This lab will only cover vector data.

Vector Data

Vector Data: Individual points stored as (x,y) coordinates pairs. These points can be joined to create lines or polygons.

Format of Vector data

Esri Shapefile — .shp, .dbf, .shx Description - Industry standard, most widely used. The three files listed above are needed to make a shapefile. Additional file formats may be included.

Geographic JavaScript Object Notation — .geojson, .json Description — Second most popular, Geojson is typically used in web-based mapping used by storing the coordinates as JSON.

Geography Markup Language — .gml Description — Similar to Geojson, GML has more data for the same amount of information.

Google Keyhole Markup Language — .kml, .kmz Description — XML-based and predominantly used for google earth. KMZ is a the newer, zipped version of KML.

Raster Data

Raster Data: Cell-based data where each cell represent geographic information. An Aerial photograph is one such example where each pixel has a color value

Raster Data Files: GeoTIFF — .tif, .tiff, .ovr ERDAS Imagine — .img IDRISI Raster — .rst, .rdc

Information Sourced From: https://towardsdatascience.com/getting-started-with-geospatial-works-1f7b47955438

Vector Data: Census Geographic Data:

• Geographic Coordinate Data is provided by the census and compliments their census geographies

• https://www.census.gov/geographies/mapping-files/time-series/geo/tiger-line-file.2010.html

• https://www.census.gov/programs-surveys/acs/geography-acs/geography-boundaries-by-year.html

• Bnia created and provides for free geographic boundary data that compliment these CSA's

If you are using Geopandas, direct imports only work with geojson and shape files.

spatial coordinate data is properly encoded with these types of files soas to make them particularly easy to use.

You can perform this using geopandas' undefined function.

As you can see, the resultant variable is of type GeoDataFrame.

GeoDataFrames are only possible when one of the columns are of a 'Geometry' Datatype

Awesome. So that means, now you can plot maps all prety like:

And now lets take a peak at the raw data:

	OBJECTID	CSA2010	hhchpov14	hhchpov15	hhchpov16	hhchpov17	hhchpov18	hhchpov19	CSA2020	hhchpov20	hhchpov21	Shape__Area	Shape__Length	geometry
0	1	Allendale/Irvington/S. Hilton	41.55	38.93	34.73	32.77	35.27	32.6	Allendale/Irvington/S. Hilton	21.42	21.42	6.38e+07	38770.17	POLYGON ((-76.65726 39.27600, -76.65726 39.276...

I'll show you more ways to save the data later, but for our example in the next section to work, we need a csv.

We can make one by saving the geo-dataframe avove using the undefined function.

The spatial data will be stored in an encoded form that will make it easy to re-open up in the future.

In the following example pulls point geodata from a Postgres database.

We will pull the postgres point data in two manners.

• SQL query where an SQL query uses ST_Transform(the_geom,4326) to transform the_geom's CRS from a DATABASE Binary encoding into standard Lat Long's

• Using a plan SQL query and performing the conversion using gpd.io.sql.read_postgis() to pull the data in as 2248 and convert the CRS using .to_crs(epsg=4326)

• These examples will not work in colabs as their is no local database to connect to and has been commented out for that reason

Boundary

envelope

convex_hull

simplify

buffer

rotate

scale

skew