ERCOT Information Extraction Tool (ERCOT_scraper)

The Electric Reliability Council of Texas (ERCOT) utilizes a Management Information System (MIS) to distribute market and grid information for market participants. Using their MIS tool, this Python code serves to extract all available information pertaining to Day-Ahead markets, Real-Time markets, and various operational related information.

The ercot_scraper Python project extracts all available market and grid information for the following areas:

  • Day Ahead Markets
  • Real-Time Markets
  • LMP prices and associated nodes
  • System load
  • Generation

Future work for this project mainly involves the analysis. This data will be utilized to assess trading patterns across market participants at various locations and will rely on spatio-temporal methods and will utilize computational methods that will be able to tackle such vast quantities of data. Once the data collection process is streamlined, a dashboard will be created to visualize such patterns.

NFL Stats via ESPN (NFL_stats)

This project is currently under development.

ESPN offers statistics that can be accessed for personal use for both individual athletes as well as team-wide. The NFL_stats packages seeks to extract pertinent information regarding schedules, team stats, rosters / depth charts, etc.

NYC Open Data - NYPD Crime Analysis (NYC_crime)

This project is currently under development.

With the abundance of publicly available and open-source data from NYC agencies, NYC Open Data offers an API that is utilized to extract geo-referenced spatio-temporal data surrounding various crimes. This project seeks to take this data a step further by obtaining text information from the NYPD Crime Stoppers Twitter account to help predict the most likely regions of crime for a given “tip” or crime tweet. Given the immense quantity of available data, both mathematical and computational understanding of such data structures are crucial.

From a statistics perspective, Bayesian methods will be used and spatial interaction networks will model the dependencies between the locations of the initial complaints and the arrests. The ultimate goal is to create an interactive dashboard that will incorporate and visualize all necessary information.