Finance faculty and student use web scraping for research on innovation
By Cara Daltwas ’25
Throughout the 2023-2024 school year, a Providence College School of Business faculty member has collaborated with a student on two impressive projects based on data from websites. Lydia Morgan ’24 is a finance major and computer science minor in the Honors Program from Richmond, Virginia. She has been working alongside David Cortés, Ph.D., associate professor of finance, who teaches undergraduate and graduate courses in management science, operations management, and business analytics. This dynamic team with a shared passion for data analytics has been conducting research using web scraping, which is used to extract data from websites.
In one project, they categorize various types of innovation that a company pursues over time and how perceptions of novelty associated with those innovations impact product performance, using the video game industry as context. The two are building a large database of video game reviews using R code and, through text mining and content analysis algorithms, investigate both user and expert reviewer comments to assess the relationships between the company’s perceived innovation effort and the game’s scores as well the videogame developer’s overall firm performance.
This project has allowed Morgan and Cortés to use their research methods to study topics that interest them outside of the analytical world. “We not only provide insight into game innovation and its users’ perceptions but I, personally, also feel a connection to my past lives as an avid player of Mario World, FIFA, and Metal Gear, among others,” Cortés said.
A second project seeks to provide an in-depth analysis of the business corporate landscape for business graduates. Here, Cortés and Morgan are building a database with information from digital job marketplaces, such as Indeed and LinkedIn. With this information, they will subsequently devise insights using various data analytics tools, including trend analyses and different visualization and clustering techniques. This work has important implications in that it can help determine the most desirable skills that different industries seek in business graduates, as well as general industry trends or possible disruptions in a post Covid-19 world and beyond.
“I was drawn to this project to further my analytics skills as I enter the workforce. Combining my PCSB curriculum and hands-on research was a great way to learn more about the data analytics space,” Morgan said. “I have really enjoyed working to better understand the inner workings of the code that I developed and the sites themselves.”
After graduating from Providence College, Morgan hopes to use her finance degree and strong analytical skills to assist non-profits or other start-ups. She has her sights set on working in business analytics or operations management.
