Fire Fighter Standard Operating Procedure (SOP) Peer Production Site: FireCrowd
Jump to: Building the Communication, Independent Study, or Publication
I. Building the Communication
My academic work on FireCrowd began in 2013 when I took Online Communities and Crowds (OCC) taught by Professor Aaron Shaw. I had started a project called FireCrowd late in the summer, and as Project Manager, I was continuing to develop the site over the course of Aaron's class by incorporating what I learned about crowdsourcing and peer production directly into the system.
FireCrowd is a national fire fighter health and safety platform. The goal of the site is to peer produce best practice Standard Operating Procedures (SOPs). The following description is taken from the first page of FireCrowd.com. It describes SOPs and why they are important resources for fire fighters:
“Emergency responders commonly utilize SOPs, and the less prescriptive Standard Operating Guidelines (SOGs), as documents that ultimately represent the best practice solutions for emergency response organizations….Currently there are no standard SOPs that are uniform across the fire departments in the United States…Online peer production offers a novel approach to synthesis and coordinated information on a common technical topic (the individual SOP) based on broad and on-going input from directly impacted stakeholders, in this case the fire fighter community and health and safety personnel. The FireCrowd project focuses on the use of online peer production techniques to develop and refine SOPs on a national scale for the fire service.”
I designed the FireCrowd prototype for Custos Fratris LLC with the assistance of a part-time web developer, and the first SOP we created was for Electric Vehicle and Hybrid Vehicle fires. FireCrowd allows fire fighters from across the country to share best practices and to update SOPs quickly, making it a great resource in areas where technology changes quickly. For example, the lithium ion batteries found in some electric and hybrid vehicles require a different method of fire extinguishing than traditional car batteries. Through class research for my final paper, I realized that the user commenting method that I had originally built into the site should be enhanced with a Wikipedia-inspired text-edit function. This would allow more individual ownership of the text, and should also allow the page to be updated more frequently. After stating this conclusion in my final class paper, I proceeded to change to the site to match the new recommendations.
During the winter, work on FireCrowd continued and I presented the site to fire chiefs across the country. In collaboration with Casey Grant, my contact at the Fire Protection Research Foundation (FPRF), I led a conference call with fire chiefs as well as a few law enforcement members. Talking about my project to such an esteemed audience was a great opportunity to become comfortable sharing my work and results. I presented FireCrowd in a simple way, went through the site flow, and then asked my list of prepared questions. I was expecting lots of feedback from the chiefs, but they did not have many concrete criticisms or praises. Overall people thought it was “cool” and “interesting,” but I did not receive specific answers to my questions about, for example, site security. I realized that I should not have asked this decision-maker audience specific questions about site functions because they probably did not give much thought to those details. I have learned that I must be careful to think of the viewer’s level of engagement with the site in order to construct effective questions.
These (primarily) men on the call saw a technology that would possibly change the way that they created and maintained SOPs. Some were happy to see a new model for the process, but others liked the way they had “always done it.” If I could go back now I would ask them a new set of questions: How can this site be best implemented in your fire department? What benefits would a national online platform such as this provide for your fire fighters? What about this prototype appeals to you? Do all your fire houses/stations have Internet access and a computer capable of running sophisticated programs? (More on this last question below). Overall, I realized that I needed to talk to the users themselves – the individual fire fighters, to find out answers to specific technical questions. Therefore, Aaron agreed to be my advisor for an Independent Study during spring quarter where I would do just that: talk to and conduct research with the local fire department in Evanston, IL.
II. Spring 2014 Independent Study “FireCrowd: A Usability Study of an Online Peer Production System.”
When I started formal research on FireCrowd in the spring of 2014 under Aaron's supervision, I had already been developing the site for seven months and I knew that it had some challenging pieces that needed to be addressed including naming of buttons and the wiki-style editing. However, I didn’t know exactly how new users would think about and interact with the site, so I began research using user-centered design methods to gather data. While conducting my independent study, I was also taking Professor Darren Gergle’s Technology and Human Interaction class. From day one in his class, I realized I had created FireCrowd in entirely the wrong way. The end user should be observed and consulted first, and the user environment should also be scrutinized to produce an application best suited for user use. I had designed FireCrowd entirely in isolation with only two clients (neither fire fighters themselves) to advise me. Understandably quite sheepish, I explained the FireCrowd saga to Professor Gergle. An hour later I walked out of his office feeling much better, armed with several HCI-methods of research to conduct with my fellow peers, experts, and the fire fighters themselves. At the same time, Professor Shaw was corresponding with Professor Haoqi who also had some good ideas to help form my research agenda.
Next, I needed to reach out to the Evanston Fire and Life Safety Services and convince them to collaborate with me on my project. As a paramilitary organization, it was crucial to get the green light from the fire chief, Chief Greg Klaiber. Through this effort I realized that it takes time to set up trust and create connections with community organizations. About halfway through the quarter I received a thumbs-up, and Chief Klaiber passed me on to the most appropriate person for the study, head of the SOG (Standard Operating Guideline) committee Division Chief Brian Scott, who I worked with for the rest of my project. I met with him several times to identify how SOGs were created within the Evanston Fire Department, how they were maintained, and where they were stored. I acquired a huge amount of information about how fire fighters think about and use SOPs.
I used usability studies to determine issues with FireCrowd, and then rapidly iterated to fix these problems. I conducted three of these tests with peers at Northwestern. I also asked Technology and Communication graduate students to provide expert evaluations of site. Their specific, informed feedback gave me even more ideas improve the site even further. Unfortunately, they suggested changes that would have involved complete site design changes, so I could not implement all of their suggestions. In the future, I will look for expert advice during the paper prototyping phase where such drastic changes are much easier to implement.
In early June I presented FireCrowd at the monthly department Standard Operating Guideline (SOG) meeting to about 30 fire fighters. I brought coffee from Einstein’s, my phone to record discussions, a clipboard to sign up for 1:1 Usability studies, and lots of paper and pens. After going though the site on a big projector, I had everyone stay quiet and write answers to questions individually (suggested by Darren). That way I could maximize both the number and variety of responses before I opened the floor to discussion. In both the written responses and the ensuing discussion, I received a lot of great feedback from the fire fighters both about the system and how they saw it fitting into the structure of their department. (See my Global Health research poster in the appendix for a summary of their comments). Three fire fighters also signed up for individual meetings with me, and so, over the course of two weekends, I hopped on my bike and visited Evanston Fire Stations 4, 5, and 1 where I conducted usability studies with the volunteering fire fighter on the station computer.
As soon as the first fire fighter and I sat down at Station 4, I realized that their technology was below what I had expected. The thick desktop computer ran only Internet Explorer, and the fire fighter was quick to inform me, “only one tab can be open at a time.” I am glad that I made FireCrowd very simple without flashy graphics or videos, because the station computers were not ready to handle a fancy site. They showed me the huge SOG Binder where all the SOGs are kept in each station, and I saw where the fire fighters relax and spend their free time in the station. I made a plan that for the next project I would create, it would be my first priority to meet with the users and observe their environment before I began designing.
communication technology to solve problems in heath care.
Presentation to a Danish Fire Chief
I studied abroad in Copenhagen, Denmark during the fall of 2014. I sought to understand cross-cultural communication while continuing to promote past projects. While in Denmark, I presented FireCrowd to a Danish Fire Chief, and had a very positive reaction. He was thrilled with the idea, telling me that "best practices" is a buzz-word in the European fire fighting community and that a system like FireCrowd could really help health and safety collaboration between fire safety organizations from the many European nations.
III. Publication - Evaluating Open Collaboration Opportunities in the Fire Service with FireCrowd
Paper Abstract:
In emergency response organizations like the fire service, personnel require easy access to reliable, up-to-date safety protocols. Systems for creating and managing Standard Operating Procedures (SOPs) within these command and control organizations are often rigid, inaccessible, and siloed. Open collaboration systems like wikis and social computing tools have the potential to address these limitations, but have not been analyzed for intra-organizational use in emergency services. In response to a request from the Fire Protection Research Foundation (FPRF) we evaluated a high-fidelity open collaboration system prototype, FireCrowd, that was designed to manage SOPs within the U.S. fire service. We use the prototype as a technology probe and apply human-centered design methods in a suburban fire department in the Chicago area. We find that organizational factors would inhibit the adoption of some open collaboration practices and identify points in current practices that offer opportunities for open collaboration in the future.
