Tangible Augmented Reality for Archival Research: Using Augmented Reality to Research Cultural Heritage Items

Daniella Kalinda, Ryerson University, Canada, Lucas Hrynyshyn, Ryerson University, Canada, Gabby Resch, Ryerson University; University of Toronto, Canada, Anitha Nathan, Ryerson, Canada, Ravit David, Scholars Portal, Univ. of Toronto, Ontario, Canada, Ali Mazalek, Ryerson University, Canada

Abstract

Keywords: augmented reality, cultural heritage, tangible augmented reality

Introduction

Cultural heritage items can be interacted with in novel ways using emerging digital technologies like augmented reality (AR), virtual reality (VR), and mixed reality (MR). The rise in popularity of augmented reality, in particular, shows promising opportunities for integration of this media for people from all walks of life- including cultural heritage researchers. With respect to cultural heritage research practices, current methods for presenting digital archival content often lack the richness, versatility, and longevity that is necessary for researching physical artifacts or ephemera (Champion & Rahaman, 2019). Our project aims to address some of the challenges that scholars face when working with ephemera and memorabilia collections, and to enhance scholarly engagement with cultural artifacts by enabling research and discovery through emerging technologies, and augmented reality in particular.

In this paper, we describe advancements in digital recreations of cultural heritage and present a formative study we undertook to understand current challenges in cultural heritage research, as well as to understand current practices in research and insights gained from interaction with physical cultural heritage materials. We then present the design of a mobile augmented reality prototype based on analysis of data from the formative study. We conclude with possible directions for the integration of augmented reality in cultural heritage research. 

Background and Related Work

This section provides an overview of the roles, value, and challenges of cultural heritage preservation, and an assessment of how augmented reality can be used in cultural heritage research.

Cultural Heritage

Cultural heritage refers to the traditions (tangible and intangible) that are transferred from one generation to another. Tangible heritage includes objects such as artifacts, ephemera, and buildings, whereas intangible heritage involves norms, practices, and stories (Brumann, 2015; Kenderdine, 2015).

 Tangible objects such as artifacts (three-dimensional cultural heritage objects) and ephemera (paper-based cultural heritage) are kept in archives, libraries, and museums where they are preserved and can be used for research. However, with limitations in storage and access to these archives, the cultural heritage field has recently shifted toward digital methods and emerging technologies to improve access.

Challenges Facing Cultural Heritage Access and Interactions

First-hand public access to cultural heritage is limited by several factors. For travel and tourism, cultural heritage centres are attempting to increase their number of visitors, improve the visitor experience, and reimagine public perception of cultural heritage sites as exciting, innovative, and entertaining (Chung, Han and Joun, 2015). One of the ways the industry has achieved this is by incorporating emerging technology as part of the visitor experience, with a specific focus on augmented reality. Methods for preserving cultural heritage have also evolved with the advancement of technology. Many libraries are taking initiatives to digitize their collections and increase access to their historical items so as to remain relevant in a digital world and to keep the library as the center for knowledge acquisition (Jones, 2017).

Some heritage researchers assert that ephemera are most meaningfully understood in a collection, posing another challenge to visualization and representation of ephemera through technology (See Appendix D; Table 1.2). For example, a single postcard with “John” as the sender’s name does not illustrate any sort of need for privacy or identity protection, until the viewer of the collection understands that numerous postcards sent from concentration camps all use this same name as an alias for their personal protection. Therefore, the value of ephemera as understood as a part of a whole collection makes them both unique and difficult to work with. 

The cataloguing of donated special collections can often be non-linear. Items in a collection may be randomly assorted in a scrapbook, carefully selected by an experienced collector, or inherited by a family member (Anghelescu, 2001). While a collection can provide valuable insights on a collector and the motivations behind the collections, librarians and archivists often struggle to classify these special collections in a meaningful way that supports the research process.

A Collaborative Preservation Initiative 

Various libraries have taken the initiative to digitize their archives (Smith, 2016). The Thomas Fisher Rare Book Library at the University of Toronto, which holds part of the Mernick Farley collection of Jewish Ephemera, is currently seeking to digitize its collection in an innovative way that will help heritage researchers undertake insightful research of primary source cultural heritage from remote places. Our project is being conducted in partnership with this initiative. Our goal is to create intuitive and engaging interfaces for meaningful digital research experiences. In order to do so, we investigated ways that innovative technologies have been used for the dissemination and preservation of cultural heritage, including which modern technological methods were most appropriate for this task.

Augmented Reality 

Augmented reality is an enhanced digital visualization technique that superimposes virtual objects onto a physical environment, through handheld or wearable display, giving the appearance of being integrated in the physical environment (Milgram et al.,1994; Wu, Lee, Chang and Liang, 2012). This dynamic interaction with the physical environment may be the reason augmented reality has become increasingly popular in many areas including entertainment and education  (Kim, Kang, Choi, Choi and Hong, 2017). 

In a systematic review of augmented reality for educational purposes, Akcayir and Akcayir (2017) reported sixty percent of augmented reality applications are built for mobile devices, twenty-four percent for desktops, and sixteen percent are created for head mounted devices specifically designed for augmented reality. The authors argue that there are five factors that may make mobile devices more prominent: they are more cost-effective, easier to learn to use, portable, more likely to encourage social collaboration, and built for independent operability (one’s own ability to utilize the physical hardware). Furthermore, it is possible that the dynamic interaction between the physical and virtual environment may improve processing and problem-solving skills, critical thinking and on-the-spot decision making (Akcayir and Akcayir, 2017). The growing prevalence of augmented reality also demonstrates that the technology is maturing and will become easier to use. This trend suggests that augmented reality could be applied beyond the field of entertainment and used in areas such as education, training, and research. 

Some of the commonly reported challenges with augmented reality are “lagginess” (a term indicating a delay between player action and the action’s digital result) and issues with rendering visualizations using trigger (or target) recognition (Wu, Lee, Chang and Liang, 2012; Akcayir and Akcayir, 2017). Improvements in wireless connections may decrease these challenges. However, most studies of augmented reality have been short-term and some researchers suggest that more assessment protocols should be taken to analyze long-term effects of AR usage (Botden and Jakimowicz, 2009).  

Design and Implementation of Augmented Reality

The integration of PTC’s Vuforia plug-in with Unity 3D has helped novice developers build augmented reality applications fairly easily. They can quickly prototype different applications and test them using this game engine. For example, researchers with little to no programming experience built a mobile augmented reality application for children with obesity and tested its effectiveness in decreasing obesity and iterating as necessary (Kim, Suk, Kang, Jung, Laine and Westin, 2014). Game engines such as Unity 3D have increased access to developing mobile augmented reality applications for less experienced developers. Furthermore, this game engine can also be used to interact with tangible objects in a virtual environment.

Annexing Reality, a toolkit developed by Hettiarachchi and Wigdor (2016) on Unity3D, enables designers and developers to match augmented virtual objects to similarly-shaped physical objects in one’s environment. An AR user can grab a physical object and see the corresponding virtual object move in the simulated environment. The developers and designers who assessed this toolkit found it easy to learn and were satisfied with the tool overall. However, they reported that it was difficult to predict the weight of physical objects. Furthermore, for the AR user, erroneous head movements led to lagging, making interactions more difficult. Overall, despite some challenges, AR users found interacting with physical objects in the simulated environment to be more realistic.

Augmented Reality in Cultural Heritage and Tourism

Augmented reality applications have been used in the tourism industry to help improve the visitor experience. These applications are generally location-based and often intended for guided tours (Jung and tom Dieck, 2017; Dow et al., 2005). Augmented reality has successfully been integrated in creating innovative experiences in institutions like museums and heritage sites. Examples include integrating augmented reality with outdoor heritage in parks for sustainable tours (Ciolfi, 2013) and cultural visits (Kenderdine, 2015), incorporating natural interaction in augmented reality in museums (Kyriakou and Herman, 2018), and interactive and context-based applications in Egyptian museums (Nofal, 2013). Such designs may be useful for increasing public engagement, making the site experience more engaging for the modern visitor. They also point to interesting research applications, including the use of augmented reality for the interpretation of archaeological sites (Amin et al., 2012). These various examples suggest that augmented reality may be a relevant medium for academic cultural heritage research.

Formative Study

Methodology

The goal of this study was to discover the current working procedures researchers use, and to identify challenges in the interactions for researching cultural heritage. We hypothesized that researchers are looking for more intuitive and interactive ways of researching artifacts. We expected the interviews to provide insights into the areas where heritage researchers are lacking intuitive and interactive opportunities for engaging with digital artifacts. All the interviews were conducted in person and were audio-recorded. This ongoing research is in partnership with the Thomas Fisher Library at the University of Toronto. These findings are used to inform the design of an augmented reality prototype for research purposes.

Data Collection

The method for data collection used was open-ended interviewing (Yin, 2015, p. 138). Excluding demographic questions, the questions were open-ended and followed a qualitative interview format. A questionnaire was used as a guide for the interview but the order and phrasing of the questions differed depending on the context of the interview (Yin, 2015, p.142). Conversational-style interviews are suggested to help researchers recollect their work candidly (Yin, 2015, p. 142). Finally, data was analyzed using a five-step process: compiling, disassembling, reassembling, interpreting, and concluding (Yin, 2015, p. 185). These approaches are reviewed in the data analysis.

Participant Recruitment

We recruited participants using snowball sampling. All participants were sent a consent form to review once they had agreed to join the study through email response. They were offered compensation of $10 for their participation. Participation in this study included both senior and junior researchers with a graduate degree, as well as those currently working on their graduate degree who work with cultural heritage items. Researchers without a graduate degree and who do not work with cultural heritage items were excluded from participation. 

Interview

A total of 8 participants were involved in this study. This includes 4 curators, 2 history professors, and 2 digital archaeologists. The study was conducted in either the participant’s office or in a booked meeting room to ensure privacy and an audio-recording of high fidelity. The study lasted between 30 to 50 minutes. The signed copy was kept by the investigator and a copy of the consent form was given to the participant. The investigator used the questionnaire as a guide for the interview. Out of the 37 available questions (see Appendix A), a total of 14 interview questions and five demographic questions were used between interviews. Any follow-up question was assigned to its original interview question. For example, questions 22 to 26, under the insights category of the questionnaire (see Appendix B), were all placed under question 22 for data analysis. Asking each question during the interview separately became redundant whereas grouping such questions into one question facilitated candid answers on the part of the interviewee (heritage researcher). These “grouped” questions (having a total of 14) were analyzed to find trends (see Appendix B).

Data Analysis

To analyze our findings, audio-recorded interviews were transcribed and, once verified, the recordings were deleted. Each interviewee was assigned an alphanumeric code. The questions were numbered to correspond to the questionnaire. The interview answers for each participant were reorganized into a new spreadsheet. Answers to a question that appeared twice or more were highlighted and summarized in an “emergent themes” section for that answer. The themes found across various answers were summarized to find common trends (see Appendix D). These steps were repeated as necessary.

All participants’ answers for each question were used to find trends in the workflows, desired insights, and common challenges during cultural heritage research. In the next section, the results will be discussed, then they will be interpreted in the following discussion section.

Results

After conducting our data analysis, we categorized our findings into three main sections: common practices in research, practices in tangible versus digital search, and researchers’ proposed solutions for enhancing research.

Research Workflow: Advantages and Disadvantages

The reported tools researchers use during research include pen, paper, laptops and desktops for note-taking. The software used include Excel, Word, Dropbox (Appendix D; Table 1.3; T7 and T8), and email applications. Only digital archaeologists in our study reported using 3D scanners (Table 1.4).

The motivations for research varied between researchers. While three curators and both digital archaeologists claimed to work collaboratively (Table 1.2 ; A4-A8) on projects as opportunities came, the history professors and one curator reported that their work was self-driven and they selected what they researched based on their area of specialty (Table 1.2; A1-A3). Therefore, while the ways that researchers collaborate vary, collaboration generally happens for the identification of miscatalogued items, for advice and clarification as part of a larger project, or to get another expert’s take on a topic they are less familiar with (Table 1.2; T20, T32 and T33). These conversations may take place by email as they analyze the cultural heritage item, or in person, about the general topic overall.

Pre-visit: Researchers reported starting their research online to find cultural heritage items (whether ephemera or artifacts) that are relevant to their project (Table 1.2; T4). During the pre-visit phase, researchers do as much preparatory work before the archival visit. This includes accessing various databases online, usually on a museum, library, or another archival institution website. It is reported to be more  efficient due to time and budget constraints (Table 1.2; T15), but less reliable because the available information online is not “necessarily accurate” (Table 1.4; T8). The researcher will also generally contact colleagues within their network for clarification on whether certain collections are in their institution although not visible online (Table 1.2; T33 &T20). During this time, they begin organizing their findings in Excel spreadsheets, Dropbox folders or Word documents (Table 1.3; T7 and T10). They may also collaborate with colleagues either in person, over the phone or via email to decipher the provenance of an ephemeral item. However, researchers mentioned that not all cultural heritage items that are in an archive are available online (Table 1.4; T12), which can lead to a necessary archival visit for the researcher to find the archives onsite (Table 1.2; T2, T3A, T3B, T11 and T25). During the pre-visit stage, they decide which archive is worthwhile visiting and make preparations accordingly. Therefore, the pre-archival visit stage is used by the researcher to gather general data about their project themes and to find relevant archives for further research.

Archival visit: A researcher sifts through various boxes of data to find archives relevant to their research. Here, researchers take an “exploratory” approach to sort through the archives and analyze each item and create their own filing system for reference. Researchers generally take photos of items in the collection with their own mobile devices and create their own database for items that are relevant to their topic of research (Table 1.2; T18). They note details such as the box number, date, and object number of the collection. This way, they may easily refer back to the physical collection and know how to find each item easily.

Researchers use either an object and device-centric focus or a narrative focus to studying cultural heritage. Some researchers were more interested in the history of the object and its implications in society while others were more focused on the individual lives impacted by these objects. For example, one researcher studied the introduction of consumer cameras on social life, whereas another researcher studied the letters of a photojournalist at different stages of her career. While both used cultural heritage to make inferences, they took different focal points to arrive at their conclusions. All researchers, therefore, agree that the story behind the object is what is most important, but they do so according to different focal points (Table 1.3; T12-T14). Therefore, the purpose of studying any item of cultural heritage is not only to study the physical object but to draw implications about its significance.

Post-visit: Once researchers have visited the archives, they use their new-found insights to either redirect or reinforce the stories they are telling about these objects. They link the items to greater social structures and address the implications of the items carried within their historical context.

Heritage researchers who are professors use the photos they took at an archive for use in their publications (Table 1.1, T10; Table 1.2, T18; Table 1.3, T10). Curators generally opt to use the physical objects themselves in their exhibitions (Table 1.2; T19). In other cases, the research is ongoing and can lead to other archival visits for long-term projects, such as books.

Research Search Methodology

One of the main reported advantages of visiting archives and working with cultural heritage objects is that it offers researchers a more meaningful research experience (Table 1.2; T2, T3A, T3B, T11 and T25). The primary motivating factor for visiting archival institutions is to find ephemeral cultural heritage that is of relevance to a participant’s/researcher’s topic of study. Generally, a collection is sought out because it is unique, relevant to one’s research, and/or inaccessible online. This visit may be a part of collaborative work or a self-driven project. The visit to the archival institution was considered pivotal to most researchers and as the moment in their research that either solidifies their hypotheses or challenges them. Depending on their research specialization, researchers travelled either within the country or overseas to see these artifacts (Table 1.2; T3A and T3B). Although central to their research, most reported that one visit was enough for their research. Whether one’s work is more self-driven or collaborative seemed to be contingent on the researcher’s style. Some curators preferred the collaborative approach and worked mainly on requested items. However, other curators were more self-driven and enjoyed an independent research process. Professors tend to be more self-driven having a few collaborations on occasion.

Neither historians, nor digital archaeologists or curators, visited archives only out of curiosity or interest. However, they did mention visiting public exhibitions out of curiosity or interest. That is, while each researcher is personally motivated and curious about their work and tends to work on self-driven projects, the visits to an archival institution tend to be driven by a strong external motivation such as an upcoming publication or curated exhibition.

The archival visit is considered very important and researchers strategize (through researching online and contacting their network) to ensure their visit is worthwhile (Table 1.2; T4). Physical archives give the researcher the opportunity to make conclusions about the special collections that they have limited ability to make prior.

Participants stated that the majority of their research was conducted online because of ease, access, and vastness of directed search (Table 1.2; T27 and T29). Furthermore, it is considered to be more resourceful as it requires less time and a smaller budget (Table 1.2; T5). As a participant stated, “the find function is very valuable.” That is, being able to type pointed keywords in the search function helps researchers navigate copious amounts of information in a directed and efficient way. Online research, therefore, is considered a necessary precursor to archival research (Table 1.2; T4). This helps prepare the researcher with as much background information about the artifact in order to prepare for the tangible search. Further, while the archival search may be copious, the tangible analysis is considered more meaningful and more rewarding for their research.

Another advantage of visiting archives is that researchers have access to all the cultural heritage that is not available online. Further, the way that the collections are organized, researchers can make uncanny connections between items that usually are not possible online. Therefore, currently miscatalogued cultural heritage items in special collections can be beneficial for making unexpected discoveries. From the benefits of online and tangible research, as well as their disadvantages, researchers reported the solutions they thought would work best for their research.

Reported Possible Solutions for Current Challenges

The most common proposed solution was to have more cultural heritage items accessible and available online (Table 1.4; T12, T21-23). The second proposed area for improvement was to have a more accurate cataloguing system. The third suggested solution was to have one common platform for researching cultural heritage across multiple institutions. Many found that pictures online of the object in all angles sufficed for what they were working with. As long as they had the option to zoom in and out and see the recto and verso of documents and other two-dimensional ephemeral items, they found it sufficient for their pre- and post-research. For three-dimensional cultural heritage items such as artifacts, they found that the physical visit is necessary if the item is indeed relevant to their research. Some agreed that three-dimensional imaging of the artifact could be helpful but maintained that it would probably not replace the physical visit. One mentioned, “I’m very comfortable dealing with materials.”

Researchers were aware of how augmented reality has been used for cultural heritage. Most only saw a use for it for exhibitive or teaching purposes. None had used augmented reality for their own work but understood how it worked. Even less experience was reported about the use of virtual reality. One curator mentioned, “my imagination is probably better than VR.” When commenting on the use of augmented reality or virtual reality in their research they reported that they only saw a use for it in exhibitions or for teaching purposes, but not for research. One curator had incorporated AR in one of their exhibitions, portraying the exhibition in three different time stages. She stated that the project was “successful and beautiful” but, with a large group using one provided mobile device, the quality of the experience diminished as people waited longer than expected to use the application. However, another curator found that using AR for her exhibitions on African art pieces would be less appropriate for her audience. She found that the AR “looked too much like a video game” and would take away from the experience she was trying to create.  They were unsure how AR or VR could be relevant to researching cultural heritage. However, researchers remained open to the possibility of such technologies for research because, “it could be a generational thing.” Another mentioned, “I started my career before there was an online.” While most participants who are experts in their field do not envision using a different technology to do their research, they have adapted to researching online even though they started their careers only researching through physical archives. The possibilities afforded by this reality are explored further in the prototype design.

Mobile Augmented Reality Prototype

From our study, we found that researchers were looking to have greater access and better information about cultural heritage items. While doing large digitization projects and item-by-item re-cataloguing of the items were beyond the scope of this project, we decided to start our development by tackling a different angle of the project. We sought to make the pre-archival visit digital search more meaningful by adding tangibility to the digital search experience, as well as adding features for collaboration and recommendations for an easier cross-platform search. In this section, the design of an augmented reality application for researchers is discussed. We call the application TARA, for “Tangible Augmented Reality Archives.” The design explores how virtual objects that represent artifacts from a cultural heritage archive can be superimposed over a tangible cube using a mobile augmented reality application. For simplicity, the cultural heritage in the applications only include artifacts and not ephemera.

Design

TARA (see Appendix C) consists of an augmented reality application on a mobile device that is coupled with a tangible paper cube with one QR code on each of the sides. The purpose of this interactive cube is to offer researchers a tangible experience during their digital search. In the application, the camera on a mobile device detects one side of the QR code and a three-dimensional visualization of an artifact is then activated. The digital artifact covers the cube so that turning the cube around in the physical environment also turns the artifact in the virtual environment (Figure 1). In this way, users can manipulate the virtual artifact by performing the corresponding physical manipulation on the cube, e.g., turning the cube upside down to see the bottom side of the virtual artifact, or bringing the cube closer to zoom in on the virtual artifact. Users also have the option to scroll between artifacts, get more information about an artifact, share or save the artifact, get recommendations of similar artifacts, and search through a database (Figure 2 and 3).

Low accessibility and inaccurate cataloguing were often mentioned as needs for improvement in the digital search. Therefore, this application is intended to be used by researchers during the pre-archival visit stage. We expect this to give them a better visualization of the cultural heritage they are researching, so that this may help them decide whether or not to make the archival visits. However, some questions we still found ourselves with were: how do we help make more cultural heritage accessible digitally? How can we help both researchers and librarians with their cataloguing of data? 

Our next steps with this prototype involve user testing with cultural researchers and investigating if, and how, this application would be useful for research.

Future Directions

We aim to finalize the implementation of the Tangible Augmented Reality Archives (TARA) application and conduct user studies. From these studies, we will be able to learn how researchers might integrate TARA in their work. Future iterations of the TARA application could include the archival visit in its user experience—namely, helping with accessibility and cataloguing. During their archival visit, researchers would be able to use their mobile device with the augmented reality application and could take a three-dimensional scan of the artifact or ephemera they are interested in evaluating. With this, they would have the option of saving it to their own gallery and/or adding it to the repository. They could then create their own catalogue for each artifact, take notes, and with smart recognition of these notes, the application itself could detect keywords found in the researcher’s analysis and automatically create categories (or tags) to facilitate finding this artifact for future research on multiple platforms. Users could write their own artifact description and request changes in the current cataloguing of the artifact to be sent to the archivist of the library. This application could be useful for librarians as well in the digitization of their files.

Conclusion

In this paper, we investigated the value of using emerging digital technologies like augmented reality in cultural heritage research. We presented a formative study investigating current practices and challenges in archival research of artifacts and ephemera. Results of this study influenced the design of an augmented reality application called TARA (Tangible Augmented Reality Archives) that, we propose, might assist with cultural heritage research. In our formative study, researchers in the heritage field were interviewed about their workflow, insights, and challenges during their research. We found that there are 3 major stages in research, the pre-archival visit, the archival visit, and the post-archival visit. Each stage has a focus, namely, directed research in the pre-archival visit, exploratory search when visiting archives, and the post-archival search brings together the insights from the previous stages for an exhibition or publication. While the archival visit tends to only happen once, it was considered especially valuable for studying the cultural heritage item. Researchers noted that working with the tangible object allowed them to draw insights about the item that is not possible through a two-dimensional, primarily visual, digital search. Researchers suggested improvements centered around availability and accessibility of archives online, and they expressed skepticism about the relevance of AR for research. However, they also acknowledged the way in which online search has transformed their research practices, and were optimistic that AR has the potential to be used in cultural heritage research. More broadly, the reported desire for greater access to information about cultural heritage items online, as well as the reported importance of tangibility and online visualization of cultural heritage items, together suggest that emerging technologies like AR may present promising approaches for online cultural heritage research. 

Acknowledgements 

We thank Loryl MacDonald and Brock Silversides from the Thomas Fisher Rare Book Library for their collaboration and extending the usage of the collection of Jewish ephemera. We would also like to thank student collaborators Ben Ashley, Zeeanna Ibrahim, and Milad Monavvarian for their work on this project.

References

Akçayır, M., & Akçayır, G. (2017). Advantages and challenges associated with augmented reality for education: A systematic review of the literature. Educational Research Review, 20, 1-11.

Mohammed-Amin, R. K., Levy, R. M., & Boyd, J. E. (2012). Mobile augmented reality for interpretation of archaeological sites. In Proceedings of the second international ACM workshop on Personalized access to cultural heritage (pp. 11-14). ACM.

Anghelescu, H. G. (2001). A bit of history in the library attic: challenges of ephemera research. Collection management, 25(4), 61-75.

Botden, S. M., & Jakimowicz, J. J. (2009). What is going on in augmented reality simulation in laparoscopic surgery?. Surgical endoscopy, 23(8), 1693.

Brumann, C. (2015). Cultural heritage. In International encyclopedia of the social & behavioral sciences (pp. 414-419). Elsevier.

Chung, N., Han, H., & Joun, Y. (2015). Tourists’ intention to visit a destination: The role of augmented reality (AR) application for a heritage site. Computers in Human Behavior, 50, 588-599.

Champion, E., & Rahaman, H. (2019). 3D Digital Heritage Models as Sustainable Scholarly Resources. Sustainability, 11(8), 2425.

Ciolfi, L., Petrelli, D., Goldberg, R., Dulake, N., Willox, M., Marshall, M., & Caparrelli, F. (2013). Exploring historical, social and natural heritage: challenges for tangible interaction design at Sheffield General Cemetery.

Dow, S., Lee, J., Oezbek, C., Maclntyre, B., Bolter, J. D., & Gandy, M. (2005). Exploring spatial narratives and mixed reality experiences in Oakland Cemetery. In Proceedings of the 2005 ACM SIGCHI International Conference on Advances in computer entertainment technology (pp. 51-60). ACM.

Hettiarachchi, A., & Wigdor, D. (2016). Annexing reality: Enabling opportunistic use of everyday objects as tangible proxies in augmented reality. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (pp. 1957-1967). ACM.

Jones, E. (2017). The public library movement, the digital library movement, and the large-scale digitization initiative: Assumptions, intentions, and the role of the public. Information & Culture, 52(2), 229-263. doi:http://dx.doi.org.ezproxy.lib.ryerson.ca/10.7560/IC52205

Jung, T. H., & tom Dieck, M. C. (2017). Augmented reality, virtual reality and 3D printing for the co-creation of value for the visitor experience at cultural heritage places. Journal of Place Management and Development, 10(2), 140-151.

Kenderdine, S. (2015). Embodiment, entanglement, and immersion in digital cultural heritage. A New Companion to Digital Humanities, 22-41.

Kim, S. K., Kang, S. J., Choi, Y. J., Choi, M. H., & Hong, M. (2017). Augmented-Reality Survey: from Concept to Application. KSII Transactions on Internet & Information Systems, 11(2).

Kim, S. L., Suk, H. J., Kang, J. H., Jung, J. M., Laine, T. H., & Westlin, J. (2014). Using Unity 3D to facilitate mobile augmented reality game development. In 2014 IEEE World Forum on Internet of Things (WF-IoT) (pp. 21-26). IEEE.

Kyriakou, P., & Hermon, S. (2019). Can I Touch This? Using Natural Interaction in a Museum Augmented Reality System. Digital Applications in Archaeology and Cultural Heritage, 12, e00088.

Milgram, P., & Kishino, F. (1994). A taxonomy of mixed reality visual displays. IEICE TRANSACTIONS on Information and Systems, 77(12), 1321-1329.

Reitsma, L., Smith, A., & Van Den Hoven, E. (2013). Storybeads: preserving indigenous knowledge through tangible interaction design. In 2013 International conference on culture and computing (pp. 79-85). IEEE.

Nofal, E. (2013). “Taking advantages of augmented reality technology in museum visiting experience.” In Proceedings of 6th International Congress “Science and Technology for the Safeguard of Cultural Heritage in the Mediterranean Basin,” 3, 1-8. Also available at https://lirias.kuleuven.be/2802267?limo=0 

Wu, H. K., Lee, S. W. Y., Chang, H. Y., & Liang, J. C. (2013). Current status, opportunities and challenges of augmented reality in education. Computers & education, 62, 41-49.

Yin, R. K. (2015). Qualitative research from start to finish. Guilford Publications. 

Appendix

Appendix A: Questionnaire

DEMOGRAPHIC:

1. Name
2. Research institution/University
3. Research specialization?
4. Expertise – number of years as researcher/specialist?
5. What gender do you identify with?
   DEFINITION:
6. Do you know what ephemera and archives are?
7. How would you define both terms?
   MOTIVATION:
8. [For researchers], what are the determining factors that motivate you to visit a distant library to see its ephemera?
   PROCESS: [For researchers, librarians, and metadata specialists]
9. Could you give us a ratio of how often you handle physical ephemera versus how often you search online through online publications? For example, ejournals, and museum’s websites
10. Can you explain what a typical research session [for a paper] looks like on any given day? (Get case studies).
11. When researching, are you generally looking for one specific ephemera or do you search by another category? (E.g. era, location, usage, evolution in technology)
12. If there were no limitations to how you’d research, what would a research session look like? (E.g. start with geographical map to pick location, timeline, etc.)
13. What would you imagine are the benefits of working with physical ephemera?
14. What do you think are the benefits of researching digitally?
15. What would you say are the top 3 problems you have when researching about ephemera through online publications?
    COLLABORATION: [For researchers]
16. How do you collaborate with other researchers while doing your own research? / How do you discuss what you discover from literature reviews or your findings from the physical collections?VISUALIZATIONS: [For all]
17. [For researchers], how is the data you are working with currently presented to you? (E.g.  open-source linked data vs. library’s search engines).
    1. What are the benefits of the current publications?
    2. What are their constraints?
18. [For librarians], if a researcher is unable to make it to your library but is interested in one of your objects, what would make it possible for them to access it remotely?
19. [For metadata specialists], what is the current categorization process for the ephemera at your institution?
20. [For metadata specialists], Can you break down the digitization process in steps from archives to final digital platform?
21. [For all], how do you visualize the data you are working with?
    INSIGHTS: [For researchers]
22. What insights are you looking for when researching through a specific collection of ephemera?
23. Do you look for more information about the context the ephemera was in? If so, what contextual information are you looking for?
24. What physical properties are important for a researcher to know- especially if they cannot access the object in person but only digitally?
25. Are you interested in knowing the meaningfulness of the object to the original owner? The collector?
26. How do you make the relation between a piece of ephemera and contemporary society? Through literature reviews? Discussions? Are there other means?
    TECHNOLOGY: [For researchers]
27. How do you record your research findings? This includes note-taking, database management, etc.
    1. What technologies are you used to using?
       1. E.g. Hardware: e.g. laptop and mouse, archaeological tools;
       2. E.g. software: Excel, MySQL, Omeka, Islandora, etc.
28. How do you approach various ephemera during the digitization process?
29. What are the visualizations you encounter for metadata? (e.g. digital repositories, open-source linked data visual representations, etc.)
    1. How have these visualizations helped your research?
    2. Do you have any struggles with the current visualization techniques?
    3. If so, what do you think could help with this process?
30. Is there a different way of working with the data that you would find more suitable for research on ephemera?
    1. 1. What functions and features would you want to have that would make the tool valuable in your research?
       2. For example, would you want the interface to work differently? To use different kinds of technology or tools to interact with digitized ephemera?
31. What is your experience with AR/VR?
32. Do you think an AR/VR experience will benefit or inhibit your research process? How do you imagine this would happen?
33. Would you be interested in incorporating new or different technologies in your research process that can help simplify working with digital ephemera?

Appendix B: Analyzed Interview Questions

1) How do you define ephemera, archives, and artifacts?

2) What are the determining or motivating factors for you to visit ephemera or artifacts at an archival institution?

3) How often do you handle physical ephemera or artifacts compared to how often do you research online?

4) Can you explain what a typical research session looks like for you (in preparation of a publication or exhibition)?

5) What are the benefits of working with physical ephemera or artifacts?

6) What would you say are the top 3 problems you have when researching about ephemera through online publications?

7) How do you collaborate with other researchers while doing your own research?

8) How is the data you are working with currently presented to you? What is the cataloguing like? What are the benefits and constraints of this system?

9) How do you organize or visualize the data you are working with?

10) What kind of insights are you looking for when researching through a specific collection of ephemera?

11) How do you record your research findings? What tools do you use? This includes note-taking, data management, and the digital hardwares and softwares used.

12) Is there a different way of working with the data that you would find more suitable for research on ephemera?

13) What is your experience with augmented reality and virtual reality?

14) Would you be interested in incorporating new or different technologies in your research process that can help simplify working with digital ephemera?

Appendix C: Figure of User Journey with the TARA application

Appendix D: Results Tables with Emergent Themes

Table 1.1: Participant Definitions of the Terms “Ephemera,” “Artifacts,” and “Archives”

Table 1.2: Emergent Themes for Participants Collaboration, Motivations and Process

Table 1.3: Methods that Participants use for Visualization and Insights

Table 1.4: Participants Opinions on Emerging and Preferred Technologies for Research