Paper

Saturday, April 04, 2020: 11:00am - 12:20pm -

Brendan Ciecko, Cuseum, USA

Recently, as AI has become more widespread and accessible, museums have begun to make use of this technology. One tool in particular, machine vision, has made a considerable splash in museums in recent years. Machine vision is the ability for computers to understand what they are seeing. Although the application of machine vision to museums is still in its early stages, the results show promise. In this session, we will explore the strengths and successes of this new technology, as well as the areas of concern and ethical dilemmas it produces as museums look towards machine vision as a move to effortless aid in the generation of metadata and descriptive text for their collections.

Over the course of several months, we have collected data on how machine vision perceives collection images. This study represents a sustained effort to analyze the performance and accuracy of various machine vision tools (such as Google Cloud Vision, Microsoft Cognitive Services, AWS Rekognition, etc.) at describing images in museum collection databases. In addition to thoroughly assessing the AI-generated outputs, we have shared the results with several prominent curators, and museum digital technology specialists, collecting expert commentary from such museum professionals on the fruits of this research.

Now, we strive to share our results. Our study represents over 100 hours worth of time invested in technical analysis, data collection, and interpretation, and we want to share this knowledge to advance the conversation in the museum field.

The goal of this session is to spark a discussion around machine vision in museums and encourage the community to engage with ongoing ethical considerations related to this technology. While machine vision may unlock new potentials for the cultural sector, when it comes to analyzing culturally-sensitive artifacts, it is essential to scrutinize the ways that machine vision can perpetuate biases, conflate non-Western cultures, and generate confusion

Bibliography:
Fenstermaker, W., “How Artificial Intelligence Can Change the Way We Explore Visual Collections.” Published February 4, 2019. Consulted September 2019. https://www.metmuseum.org/blogs/now-at-the-met/2019/artificial-intelligence-machine-learning-art-authorship.

Ciecko, B., “Examining The Impact Of Artificial Intelligence In Museums.” Museums and the Web 2017. https://mw17.mwconf.org/paper/exploring-artificial-intelligence-in-museums/.

“Art+Tech Summit at Christie’s: The A.I. Revolution.” Art+Tech Summit at Christie’s 2019. https://www.christies.com/exhibitions/art-tech-summit-ai-revolution#overview_Nav.

Knott, J., “Using AI to analyze collections.” Published February 14, 2017. Consulted September 2019. https://www.museumsassociation.org/museum-practice/artificial-intelligence/14022017-using-ai-to-analyse-collections.

Hao, K., “This is how AI bias really happens—and why it’s so hard to fix.” Published February 4, 2019. Consulted September 2019. https://www.technologyreview.com/s/612876/this-is-how-ai-bias-really-happensand-why-its-so-hard-to-fix/.

Shu, C., “Why inclusion in the Google Arts & Culture selfie feature matters.” Published January 21, 2018. Consulted September 2019. https://techcrunch.com/2018/01/21/why-inclusion-in-the-google-arts-culture-selfie-feature-matters/.

Nunez, M., “The Google Arts and Culture app has a race problem.” Published January 16, 2018. Consulted September 2019. https://mashable.com/2018/01/16/google-arts-culture-app-race-problem-racist/.

Merritt, E., “Exploring the Explosion of Museum AI.” Published October 2, 2018. Consulted September 2019.
https://www.aam-us.org/2018/10/02/exploring-the-explosion-of-museum-ai/.

Robinson, S., “When art meets big data: Analyzing 200,000 items from The Met collection in BigQuery.” Published August 7, 2017. Consulted September 2019. https://cloud.google.com/blog/products/gcp/when-art-meets-big-data-analyzing-200000-items-from-the-met-collection-in-bigquery.

Bailey, J., "Solving Art's Data Problem - Part One, Museums.” Published April 29, 2019. Consulted September 2019. https://www.artnome.com/news/2019/4/29/solving-arts-data-problem-part-one-museums.