Construction Bid Result Scraper
Code
The repo for this project can be found here.Hawaii Bid Result Scraper
Bid results for many public works construction projects are publicly available and I had this idea to mine the bid result data from the past 10 years.
I wrote this scraper using Python with Scrapy for scraping public bid results to collect bid information such as bid amounts, listed subcontractors that can be used for analyzing business trends.
For example, the data collected can offer a rough approximation of how much work each of the major general contractors has at present so that estimators can have an idea how aggressive these companies are likely to be in terms of pursuing new work. Secondly, the data can also be analyzed to show how likely each of these general contractors is going to use certain subcontractors based on historic records.