How to Build, When to Buy: Scalable Tactics for Digital Projects and Services
Chad Curtis, Saint Louis Art Museum, USA
AbstractKnowing when to build or buy software is an ongoing topic that has existed for decades, but answers evolve alongside trends in museum staffing and software business models. How do you respond when vendors and agencies are filling your inbox with listicles on why you should buy their solutions? What if an energetic developer wants to build sharable and open solutions that will require maintainers? How can museums with very different resources and personnel share tactics in a meaningful way? This paper aims to provoke conversation and encourage museums to address the question of "build versus buy" case by case and not as a global institutional decision.
Keywords: competition, tactics, software, teams, partners
It is critical to outline circumstances that surround and shape software development decisions made by digital teams at cultural institutions before approaching the central topic of this paper. Digital projects and services provided by cultural institutions compete both directly and indirectly with solutions created by the private sector. Competing for time users spend on their devices is as challenging as sustaining physical attendance at galleries, libraries, archives, and museums. Cultural institutions are expected to provide exceptional digital experiences alongside competing solutions from the private sector despite gaps in staffing and business models.
The highest-ranking factor for a decision to build or buy software in the private sector is competitive advantage (Daneshgar et al. 2011). Within the frame of competition digital teams at cultural institutions, regardless of size or structure, have to decide when to build or buy solutions based on a variety of parameters. Interesting questions arise when competitive advantage is viewed through perspectives of cultural institutions and surrounding agencies that specialize in galleries, libraries, archives, and museums. What tangibles and intangibles are gained or lost when deciding to build or buy? How should cultural institutions weigh short-term and long-term outcomes to ensure tactics fulfill digital strategies?
Lastly, an essential distinction for the phrase “build or buy”: it is a swift way to summarize a range of options among two obvious and opposing points. This paper does not aim to address and solve a binary decision, distilled into a single method or tool. However, examining layers of decisions related to building and buying, in accumulation, helps us better understand how cultural institutions are (and are not) applying digital strategies through the internal work of digital teams.
History of Software Economics and Cost Estimation
The concept of software economics, as explored in the research of Barry Boehm, from the 1970s to present, is an entry point in understanding how the private sector chooses to build or buy software. Based on previous research in software development, Boehm reviewed estimation models and ultimately created his own (a well-known example being the Constructive Cost Model). The complexity of mathematics in formal estimation models are out of the scope of this paper, however it is essential to consider how methods of cost estimation move from software engineering and ultimately to cultural institutions. Formal estimation models, despite being driven by mathematics and meant for large-scale production, have connections to decision tools that many cultural institutions use alongside expert judgment to decide whether to build or buy software.
Boehm (1984) defines economics, as “how people make decisions in resource-limited situations,” which provides a flexible yet focused scope, as it does not limit types of resources, whether it be money, employees, time, or knowledge. Six years later, in an evaluation of the effectiveness of cost estimation models used by Dutch organizations, Fred Heemstra (1990) states “judging by reports from everyday practice and findings in the literature, it regularly happens that software projects get out of hand.” This universal conflict of planning and failure that afflicts software project managers is an important dynamic in buying and building. If a project runs over internally with a digital team or externally with an agency, the cost will go up regardless.
Periodic literature reviews of software cost estimation have continued due to the proliferation of research. In 2007 Magne Jørgensen and Martin Shepperd published “A Systematic Review of Software Development Cost Estimation Studies,” which identifies over 300 journal and conference papers that address software cost estimation. Jørgensen, like Heemstra, finds that software projects frequently overrun, or “overshoot,” despite the use of estimation models. Jørgensen, a scholar who emerged among the established research of Boehm, joined Boehm in a 2009 interview comparing estimation models and expert judgment. The debate provides thoughtful exchanges but little closure or hope on there being a clear leader in cost estimation methodology. Although the statement is now more than twenty years old, one might still argue that the lack of closure is due the fact that compared to manufacturing “software is an immature engineering discipline” (Rushton and Gardiner 1997). The interest in ongoing research foreshadows that there are no definitive answers and that the field of software economics and cost estimation are worth revisiting as cultural institutions revise tactics for building and buying software.
Digital Teams and Software Economics
An instrumental survey of digital teams across cultural institutions was published and presented at MuseWeb 2018 by Kati Price and Dafydd James. Five staffing models were used to identify trends in team structure and maturity: Outsourced, Decentralized, Centralized, Hub and Spoke, and Holistic (Price and James 2018). The tendency to buy, rather than build, is apparent in the Outsourced model, but the other four models do not constrain how or where resources for software development are managed. Additional findings from the survey provide insight into how the other models could build or buy software to achieve their goals. 74% of the 56 responding institutions use the Centralized or Hub and Spoke models. Within the same 74% subset of respondents, 60.7% of institutions contained teams made of 5 or fewer employee that worked exclusively on digital activities.
Price and James concluded “the majority of our respondents feel their digital activity is not adequately resourced relative to the organization’s ambition” and recommended “breaking resource requirements down, and mapping these against organizational priorities” in order to “build the case for any additional investment needed to adequately resource these areas.” Later in their paper Price and James state “Web and app development is among the most underrepresented of skills, but this is an area that is likely to be outsourced, typically by smaller organizations.” A last counterpoint is offered as advice: “technical leadership and Web/app development are . . . ripe for investment.”
Based on the findings of Price and James it can be presumed that the majority of cultural institutions employ five or fewer employees dedicated to digital activities and that software development is underrepresented and likely to be outsourced. Connecting Bohem’s flexible scope of software economics and considering the context of competitive advantages, most cultural institutions are likely constrained to buy solutions and a smaller segment of institutions possess the ability to build solutions when it aligns with strategic plans.
Recent examples of what is possible while leveraging the ability to build and create a competitive advantage is contained in “Ticketing 2017: Two New Projects Take on Complex Challenges.” The Minneapolis Institute of Art (Mia) utilized software developers to build Hive, a ticketing platform, after “researching and testing several available ticketing products” (Hegley et al. 2018). A table comparing options to build or buy a ticketing system is shared by Mia and offers insight into layers that the team had to assess in order to arrive at a final decision. The Australian Centre for the Moving Image (ACMI) also decided to build software in-house, but with a distinction of building a connection with an application programming interface (API) of an existing licensed system. The combined layers of buying one platform (Tessitura) and building a frontend demonstrates that decisions for building and buying are not always mutually exclusive.
In comparison to the decisions made at Mia and ACMI, in or upon which ticketing software was built, Museums Victoria (MV) made a decision to “buy commercially available business applications such as ticketing” (Pryor 2016). MV is Australia’s largest public museum organization, comprised of Melbourne Museum, Immigration Museum, Scienceworks, and the Royal Exhibition Building. In brief, MV states its “solution to sourcing is broadly based on uniqueness versus ubiquity.” The decision to build software within MV would be more likely for “unique products . . . or highly individualised products” such as their Web interface to their collections or internal Web dashboards containing institutional metrics.
Looking at the decisions to build or buy software at Mia, ACMI, and MV reveals an elevated goal for cultural institutions: the flexibility to build or buy software as needed within the context of project or service requirements. Each institution’s documented tactical decisions, built upon their respective digital strategies, demonstrate how unique value and competitive advantage can lead to one institution building a ticketing system and another institution to buying a solution in the same software category.
Scalable Tactics for Digital Teams
Although no single model or method can meet all requirements of an imminent decision to build or buy, in the interest of sharing and inspiring discourse across institutions and teams, the following are recommended parameters for cultural institutions to consider. The parameters have either been adapted from a review of existing literature (Langer 2011; Rushton and Gardiner 1997; Daneshgar et al. 2011; Pryor 2016) or created from the author’s professional experience in museums and libraries. Weighting should be applied to each parameter, but it is only implied and not explicit in this paper. Each cultural institution needs to arrive at their own hierarchy of prioritization based on established institutional planning.
The first question that needs to be addressed is how a project or service will contribute to an institution’s strategic plan. The higher the strategic value, the more likely building or contracting to build should be considered. The lower the strategic value, the more likely buying should be considered.
Direct Impact to Cultural Objects or Knowledge
Much like strategic importance, the more direct impact a project or service will have on objects or knowledge an institution cares for, the more likely building or contracting to build should be considered. The lower the impact, the more likely buying should be considered.
Availability and Sufficiency of Commercial Off-the-Shelf (COTS)
An environmental scan of available software should always be conducted, followed by a comparison of functional requirements. The need to build or buy will be guided by whether COTS solutions are an exact fit, under-deliver, or over-deliver for cost.
Cost of Labor
Whether building software internally or contracting an external partner to build a solution, a calculation of labor compared to the value of commercially available software must be evaluated. In the case of “contract to build” the difference between the time to complete a project and the amount of labor required has to be calculated, then a comparison needs to be made between labor rates of staff and contractors.
Cost and Type of License
Maximize the use of open-source software licenses, remembering that commercial services and support can be offered alongside open licenses. Closed-source software licenses will be necessary at various levels and should not be dismissed, however potential closed-source solutions should be scrutinized to avoid vendor lock-in, vague roadmaps, unclear pricing, or any issue that creates unnecessary dependencies between a provider and client.
Time to Public
The private sector uses the term “time to market,” so it seems fitting to adapt the term for cultural institutions. The concept, however, is the same. The shorter the deadline, the more likely the need to buy, while a longer and more flexible timeline allows for building.
Scope and Complexity
The features provided by software, number of stakeholders, and dependent services are just some factors that make the difference between a small feature request and a discrete project or service. A larger scope and complexity, compounded by a short timeline, means buying is more likely. However, with enough time and staff capability even large scopes can be addressed in-house.
Capability is a team’s theoretical potential to complete a project. If a team has unlimited time to apply current skills and knowledge, this is a team’s capability. Capability is often summarized and conveyed through rank and roles, such as comparing a junior developer and senior developer or the difference between a graphic designer and an interface designer. The overall size and cumulative skills of a team is going to be weighted heavily compared to other parameters. While capability is an important high-level parameter, it has to be converted to capacity to arrive at a decision to build or buy.
Capacity is a quantifiable output, discoverable when staff capability is placed within time restrictions. Staff capacity can be applied in software cost estimation, during the decision to build or buy. Like staff capability, staff capacity is a crucial parameter. An institution can be in a position of having a highly capable and skilled team, but if they are not available they simply do not have the capacity to build software.
Language / Architecture / Platform
In correlation with staff capability, software is created with a language, library, or framework that a team either has previous experience with or will need to learn. The more accumulated experience a team possess the more likely they can build across languages and adapt to frameworks as they go in and out of style. Limited experience across fewer languages means buying is more likely when a project or service requires an unfamiliar language or platform.
Parameters and Use Cases
While strategy addresses scalability at a high level, tactics are more easily dragged into dependencies of specific scenarios. Using the parameters alongside tactics assists with scaling, allowing for changes to address varying team and project sizes. The following use case at the Saint Louis Art Museum (SLAM) demonstrates how the parameters were used during a project with a large scope and short timeline.
SLAM launched a new website in December 2018 in partnership with Cuberis, an agency that focuses on museums and galleries. The decision to engage with an external partner was based on a short timeline (see Time to Public) and current availability of the small internal team, despite existing experience (see Staff Capacity). The global decision was classified as a “contract to build” (see Cost of Labor) in which the team at SLAM would take over development after the foundation was established by Cuberis.
The project was initiated in April 2018 through a series of discovery meetings facilitated by Cuberis to confirm functional requirements alongside a new design based on SLAM’s visual guidelines. SLAM and Cuberis worked together to decide when to build or buy solutions, with the goal to reduce perceptible seams among different software layers. The following are just three layers that were evaluated for “build or buy” decisions within the global scope of the “contract to build” project:
- Adopting a CMS: All Web content would be stored or displayed through a content management system (CMS), which made the selection of a new CMS a key decision. SLAM needed a CMS that would be a flexible development platform, easy for content managers to use, and had a large global community which SLAM could leverage for long-term sustainability. We did not want to build a CMS, adopt a proprietary system, or join a small community for support and development (see Cost and Type of License). After reviewing a short list of open source solutions WordPress was selected as the best foundation for additional development.
- Building the Online Collection: A new information architecture was created by Cuberis during the project. The type of content with the highest complexity and top priority were not surprisingly art objects that comprise our encyclopedic collection (see Direct Impact to Cultural Objects or Knowledge). For context, SLAM adopted a Digital Strategy in 2016 that was created in partnership with Balboa Park Online Collaborative (BPOC). SLAM’s Digital Strategy emphasizes the creation of Web content around the collection, therefore the new Online Collection required substantial planning and resources (see Strategic Importance). A review of existing commercial solutions revealed features that were not needed and requirements that were not addressed (see Availability and Sufficiency of Commercial off-the-shelf). Knowing the weight of the decision and the fact that a previously bought solution became more of a hindrance and liability than an investment, the construction of a new abstraction layer was the best path forward.
- Buying an events calendar: The original scope the project included building an integration layer with commercial event management software already deployed internally at SLAM. The scope of the integration had to be reduced to allow SLAM more time to adjust internal procedures and ensure data entry would match requirements for the integration (see Scope and Complexity). SLAM and Cuberis decided to license a commercially available WordPress plugin for the launch of the website (see Time to Public) and let SLAM determine an appropriate timeline to handle further event integration as a separate project.
Cultural institutions that have invested in a digital strategy have likely adopted a team structure that delegates decisions related to building and buying software solutions. Faced with competition among peer institutions and the private sector, digital teams are faced with the challenge of pushing their own capabilities and capacity while knowing when to ask for help from external partners. Implicit competition amplifies the importance of how cultural institutions provide and maintain digital projects and services. Tactics are the primary method for digital teams to realize digital strategy and deploying them through a combination of shared models and tools is vital to compete for time on user devices. By continuing to examine our decisions to build or buy software cultural institutions can gain a greater awareness of how experiences are shaped by software and how to best utilize our teams to serve our communities and collections.
Boehm, B.W. (1984). “Software Engineering Economics.” IEEE Transactions on Software Engineering SE-10, no. 1: 4–21. https://doi.org/10.1109/TSE.1984.5010193
Daneshgar, F., L. Worasinchai, & G. Low. (2011). “An Investigation of ‘Build vs. Buy’ Decision for Software Acquisition in Small to Medium Enterprises.” Society of Interdisciplinary Business Research (SIBR). Conference on Interdisciplinary Business Research. http://dx.doi.org/10.2139/ssrn.1867839
Hegley, D., A. Serong, A. David, M. Tongen, & K. Olsen. (2018). “Ticketing 2017: Two new projects take on complex challenges.” MW18: MW 2018. Published January 22, 2018. Consulted January 04, 2020. https://mw18.mwconf.org/paper/ticketing-2017-two-new-projects-take-on-complex-challenges/
Heemstra, F. J. (1990). “Software cost estimation models,” Proceedings of the 5th Jerusalem Conference on Information Technology, 1990. ‘Next Decade in Information Technology’, Jerusalem, Israel, 1990, pp. 286–297.
Jørgensen M., B.W. Boehm, & S. Rifkin. (2009). “Software Development Effort Estimation: Formal Models or Expert Judgment?” IEEE Software 26, no. 2: 14–19. https://doi.org/10.1109/MS.2009.47
Jørgensen, M., & M. Shepperd. (2007). “A Systematic Review of Software Development Cost Estimation Studies,” in IEEE Transactions on Software Engineering, vol. 33, no. 1, pp. 33–53. http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=4027147&isnumber=4027141
Price, K. & D. James. (2018). “Structuring for digital success: A global survey of how museums and other cultural organizations resource, fund, and structure their digital teams and activity.” MW18: MW 2018. Published January 31, 2018. Consulted January 04, 2020. https://mw18.mwconf.org/paper/structuring-for-digital-success-a-global-survey-of-how-museums-and-other-cultural-organisations-resource-fund-and-structure-their-digital-teams-and-activity/
Pryor, W. (2016). “Digital strategy in evolution: Issues and responses emerging from the project to develop a digital transformation strategy for Museum Victoria.” MW2016: Museums and the Web 2016. Published January 24, 2016. Consulted January 04, 2020. https://mw2016.museumsandtheweb.com/paper/digital-strategy-in-evolution-issues-and-responses-emerging-from-the-project-to-develop-a-digital-transformation-strategy-for-museum-victoria/
Rushton, P.J., & G.S. Gardiner. (1997). “A design or buy process for determining software sourcing strategy,” Fifth International Conference on Factory 2000 – The Technology Exploitation Process, Cambridge, UK, pp. 251–256. http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=608074&isnumber=13329
Curtis, Chad. "How to Build, When to Buy: Scalable Tactics for Digital Projects and Services." MW20: MW 2020. Published February 16, 2020. Consulted .