Losing our Heads: Leveraging Headless Content Management Systems to Bridge Dynamic Web Interfaces and Backend Content
Radha Nagaraja, National September 11 Memorial and Museum, USA
Keywords: content management system (CMS), headless Drupal, dynamic websites, APIs, front-end, back-end
The natural question therefore is whether it would be possible to somehow bridge the best of both worlds (i.e., the easy content maintainability of CMSs and the rapid and flexible implementation capabilities of dynamic web development frameworks). Such a hybrid decoupled methodology would indeed provide an ideal solution. To this end, an elegant strategy that has been gaining popularity in recent years (Schadler & Grannan, 2017; Heslop, 2018) is to leverage the content management functionalities of CMSs via a headless mode and combine the headless CMS with a front-end rich web interface that is separately developed and communicates with the headless CMS via an Application Programming Interface (API). With this strategy (illustrated in Figure 1), the front-end web interface uses CMS APIs to query for data that is required for rendering the web pages. All the templates/styles for the actual rendering are implemented within the front-end layer while the content is managed from within the CMS. This decouples the presentation from the back-end data in the CMS and enables a modular solution wherein the front-end interface and the content are each managed using the most conducive tools for the purpose. This is essentially the headless CMS approach in a nutshell.
In this paper, we will outline the broad architectural principles and implementation strategies of this hybrid methodology and specific details of a particular recently developed website, namely our “Timeline” website (developed as a replacement for https://timeline.911memorial.org) that presents an interactive timeline of the events of 9/11 along with various media content within an interactive user interface. This paper is organized as follows. We will first discuss the current trend of dynamic websites and more broadly rich user interfaces which bring together multiple aspects including interactivity, incremental updates, responsive interfaces, accessibility, and multi-modal/multi-platform interfaces to enable a seamless and rich user experience. We will then discuss the content management capabilities of current-day CMSs and the functionalities they provide for facilitating storage, administration, and retrieval of structured content of various types. We will then describe strategies for interfacing a headless CMS back-end to a separately developed front-end. We will then take a closer look at the technical details involved in development of the Timeline website, which provides an illustrative example of this hybrid methodology. Finally, we provide concluding remarks, acknowledgments, and references.
Figure 1: Coupled and decoupled design architectures.
Dynamic Websites and Rich User Interfaces
Over recent years, dynamic and rich user interfaces are becoming more prevalent to enable seamless user experiences. This ongoing trend of user interface evolution involves two primary overarching facets: interactivity and adaptability. These two facets can be seen to be the key motivating factors from multiple viewpoints. Firstly, unlike the static web pages of the 90s, current-day websites provide rich interactivity, which is progressively becoming more seamless by enabling fast interactive updates of elements in a web page without visible reloads or indeed without even making it apparent to the user that any additional data was fetched from the server or that any code was run to update the page display. This is particularly seen in the evolution from web sites to web applications wherein a browser-hosted web page provides the visual interactive feel of a native (e.g., desktop or mobile) application. Secondly, the trend of increasing adaptability is evident in multiple directions including to the device platform (e.g., mobile, audio, augmented and virtual reality), situational context (e.g., location-dependent), and specific user needs (e.g., accessibility).
Modern user interfaces have evolved significantly beyond browser-based interfaces and now encompass a wide range of modalities (i.e., user interaction methods and presentation formats) such as audio (e.g., conversational interfaces such as Amazon Alexa and Google Home), augmented reality and virtual reality systems such as Google Glass and Oculus Quest, smart watches and other gadgets, and Internet-of-Things (IoT) devices, in addition to computer and mobile/handheld platforms. While these user interfaces are extremely varied in their “form factors” and the types of technology stacks used to develop and deploy these interfaces, a requirement essentially common to all interface modalities is to be able to configure the underlying content to be “displayed” via these interfaces. Furthermore, since it is becoming increasingly common for businesses (specifically museums) to provide apps via multiple interface modalities (e.g., mobile and desktop apps, web app, and an Alexa app, as well as via business calendar and social media platforms), it would be highly desirable to be able to configure the underlying content in a single back-end system. This would enable scalability in terms of front-ends (Figure 1) while retaining ease of content administration via a single back-end so that the underlying content can be reflected onto multiple front-end interfaces. A headless CMS therefore provides a very attractive solution by enabling leveraging the ease-of-use content administration functionalities of a CMS (as discussed further in the next section below) while being able to integrate the content into different technology “stacks” for various front-end interfaces.
While a headless CMS provides an attractive solution approach for a variety of front-end interfaces as discussed above, we focus below on dynamic websites in particular with the Timeline website being one representative recent example. A key part of the “dynamic” nature of modern websites is seamless interactivity wherein the user can interact with a page and load desired content with minimal delays. Another key part is the ability of the web page to adapt itself to the device (e.g., responsive to the device form factor and resolution) and user needs (e.g., accessibility for visually impaired users). This dynamism relies upon context-aware code execution both on the server-side and the client-side and includes, in general, the following components:
Using the approach outlined above for dynamic webpage updates, it is, in fact, possible to make an entire website a single page wherein after initial page load, all subsequent updates are done as incremental changes to a page without ever explicitly navigating to a different page. Such a “single-page application” enables a much more dynamic and responsive feel to the user wherein instead of navigating from page to page, a single page is dynamically rewritten wherein the parts (e.g., header, footer, sidebars, menus) that remain the same are not ever fetched again from the server and indeed not even “drawn” again on the browser. In order to address some of the nuances with single page applications such as enabling bookmarking specific pages or sharing select links to social channels, the location of the user within the single-page application can be encoded in the URL path and routes can be defined to automatically reopen the same location that was bookmarked/shared. Current-day web application frameworks such as Angular (https://angular.io) and React (https://reactjs.org) enable efficient development of dynamic web pages including single page applications based on the strategies outlined above.
Figure 2: Interaction of client-side and server-side code to enable dynamic webpage updates.
Content Management Systems (CMSs)
CMSs both for specific domains (e.g., Collective Access for museums, https://www.collectiveaccess.org) and for general use cases (e.g., Drupal, https://www.drupal.org, WordPress, https://wordpress.com) have been developed over several years. Current-day CMSs integrate a wide variety of functionalities that facilitate content management and are therefore highly preferable to use instead of a special-purpose custom CMS built for managing content for a specific website. While it might initially appear that a simple repository of media files and associated metadata (e.g., via key-value stores in, for example, a schemaless MongoDB (https://www.mongodb.com) database) could be sufficient for back-end content management for a website, it is often the case that websites very quickly outgrow such simple needs and eventually require various CMS-standard functionalities. Ease of content administration is, of course, the most important benefit of using an actual CMS since managing media repositories and metadata directly is not a user-friendly solution for a content administrator. CMSs such as Drupal provide a rich structure for defining taxonomies (i.e., vocabularies) to accurately represent the semantics of the content. Furthermore, CMSs provide features such as content tagging and integrated search.
Additionally, some features built into the back-end admin/editorial UI (user interface) include, for example, user authentication, management of permissions by role and author, revision history and change management, log messages, available security updates, cache statistics, and overall site status reports. Such functionalities come “for free” compared to managing flat content as media repositories with metadata wherein keeping track of when and by whom a particular content was reconfigured itself requires custom solutions. Also, CMSs typically allow individual settings to be accommodated in the admin UI to fit individual workflows and preferences. The admin UI is usually optimized for speed of response to facilitate management of large content repositories. Furthermore, even if a separate system is used for front-end page layout in a headless solution, parts of the CMS front-end functionalities can still be used to configure aspects of the front-end display (e.g., high-level structures such as menus, integration of specific CMS-provided modules such as stores and blogs, using the CMS admin UI to configure metadata for the front-end such as theme settings).
Hence, instead of developing a custom data store, it is often a good strategy to use a CMS with a well-developed admin/editorial user interface (UI) and associated functionalities to manage the back-end content, but in a headless mode, i.e., without using the CMS to manage the webpage layout and display, but instead retrieving needed content using an API as illustrated in Figure 3. While the webpage front-end development in a coupled mode of the CMS utilizes, for example, a WYSIWYG (What You See Is What You Get) editor or other CMS-inbuilt page layout functionalities, the headless mode relies instead upon the CMS API (e.g., using REST — Representational State Transfer) to access data and other key functions of the CMS and relies upon a separate system for the actual front-end display. Thus, the CMS is used for its main purpose of storing, administering, and retrieving structured content (i.e., content such as media files along with its semantic context) while however decoupling the CMS from where or how the content will be displayed. This approach also enables separation of concerns wherein the content administrator does not need to be necessarily concerned about how the content is going to be displayed. This can also be seen to better fits the “agile” development philosophy. Content development and front-end development can happen simultaneously based on a structured API-based interface between the back-end and front-end systems. Incremental updates on both the back-end and front-end can happen independently. Furthermore, this decoupled approach enables the workflow and development tools/interfaces to be less cluttered since each group of people only needs to care about their aspect of the work (content creation vs. front-end development). Also, this architecture intrinsically enables scalability and plurality of front-ends since content configured by the content administrator from within a CMS admin UI can be transparently reflected onto multiple front-ends without additional effort.
Figure 3: Decoupling of the back-end CMS and the front-end interface using a headless CMS approach.
In addition to headless modes of more traditional CMSs such as Drupal and WordPress, more recent CMSs built primarily/only for headless operation (i.e., “API-first” CMSs) such as Storyblok (https://www.storyblok.com), Contentful (https://www.contentful.com), and Prismic (https://prismic.io) are attractive choices for development of Content as a Service (CaaS) solutions, which can be viewed as an adaptation of the Software as a Service (SaaS) model to content delivery.
Interfacing the Headless CMS to the Web Front-End
Current-day CMSs that support headless operation typically provide inbuilt REST API and JSON serialization functionalities and often also interfaces to configure REST API endpoints (e.g., Drupal’s REST Web Services module). While these inbuilt functionalities can provide quick and easy access to content entities and their fields, one aspect to consider is that a web application typically requires specific subsets of content when rendering different pages. To query for the required data via the built-in API of the CMS might not be directly possible or at least not efficient. Instead, a custom module (as, for example, in the Timeline website discussed in the next section) might need to be developed integrated with the CMS to specifically service requests from the front-end for the application as illustrated in Figure 3. The custom module implements essentially a domain-specific language (i.e., DSL) embedded into the CMS and custom-designed for efficient interaction with the front-end for the specific application (e.g., to retrieve the specific hierarchical data structures that are required for a website, to simultaneously retrieve sets of related data that are all required at the same time for a webpage update rather than requiring separate queries, etc.). Technologies such as GraphQL, https://graphql.org (a data query and manipulation language that can be used, for example, to precisely specify what data structures to retrieve regarding, for instance, entities and entity collections in the CMS) can be used to facilitate development of a custom module.
Regarding performance, there are some subtle trade-offs to be cognizant about. A coupled solution (wherein the front-end is built using layout functionalities integrated into the CMS) can benefit from multi-level caching capabilities built into the CMS. In particular, since the CMS is aware of when each piece of content has been updated, the cache invalidation can utilize logic based on content updates to optimize the use of caches. In a decoupled solution, the page load performance can be better since the front-end pages to be sent to the browser can be stored as static files rather than being built at load time by PHP code in the CMS. However, the retrieving of content from the CMS can be slower unless care is taken to ensure that the CMS caching capabilities are being used effectively. While not necessarily out-of-the-box as in coupled mode, the caching capabilities can be enabled by making the REST API functions cognizant of cached contents and fetching from cache whenever possible. For optimal performance, load balancing and third-party caching systems for DDOS (Distributed Denial of Service) protection can also be utilized in addition to caching at the CMS and web server levels. Another point to note is the overhead associated with communication between a custom module and the back-end CMS. In particular, for example, the custom module which essentially mediates communication between the web server and the CMS back-end might be hosted on a machine different from the CMS itself in which case the interaction would be via a web-based API (e.g., REST) and network latency and bandwidth limitations would need to be carefully addressed. Prerendering and caching strategies can help in alleviating some of this overhead. Commonly accessed pages/sub-pages/views can be prerendered by the custom module and cached so that new requests are serviced directly from the cache instead of having to query the CMS. Depending on the application, the prerendering and caching can be at different levels of granularity ranging from small snippets (e.g., corresponding to responses to specific queries) to complete pages.
To test functionality and performance, automated testing methodologies (e.g., using the Selenium framework) can be used as with any other websites. In the case of decoupled websites based on headless CMSs, one additional aspect to consider during automated testing is to ensure that all the REST API endpoints and interactions between the custom module and the CMS are covered in the testing including any corner cases in these code modules.
The Timeline Website: A Case Study of the Headless CMS Approach
Figure 4: Technology stack used in the Timeline website.
Figure 5: Timeline website architecture from the user’s point of view.
To enable a flexible architecture of this website, a “timeline” is represented as an abstract concept via labels (using the taxonomy module in Drupal). Additional labels can be created anytime in the Drupal administration back-end interface. Creating an additional label will automatically instantiate a new timeline within the front-end and allow interactive access to it. The user interactivity, carousel views, timeline item views, etc., are all seamlessly scalable so that new timelines can be added by simply configuring the timeline items in the admin back-end without changing any of the front-end code. From the back-end admin interface, each timeline is specified as a sequence of timeline items (“events” or “elements”) and the temporal sequence of timeline items within a timeline is configured via weights. Events can be assigned to multiple timelines. The view of each timeline item is configured as a paragraph in Drupal along with entity references to model relationships between entities represented in the CMS. The view of each timeline item can have multiple components including audio, video, images, etc. These components can be configured from within the admin back-end and once configured, any combination of media types will be automatically shown in the front-end view. Thus, the front-end view provides a unified framework that automatically scales to addition of new timelines, new timeline items, new media components for timeline items, reconfiguration of temporal sequence of timeline items, etc. Also, timeline items in each timelines can be easily toggled to visible/disabled via a simple toggle button in the admin back-end. The admin UI is implemented as a custom module based on Drupal Views to facilitate the back-end content management. The admin UI allows arbitrary linkages between timelines, timeline items, and media (e.g., multiple timeline items can reference the same media item). The admin UI also includes an integrated search to enable quick finding of timeline items with a specific text, tag, etc. The admin UI also includes functionalities for revision history and change management, tagging (for search and optionally display in the front-end view), and configuration of aspects of the front-end such as menus.
The decoupled architecture discussed in this paper enables leveraging the specific key strengths of CMS and dynamic web front-end technologies. More broadly, this architecture based on a headless CMS will enable development of a multitude of front-ends (e.g., native desktop/mobile applications, websites, audio, VR, etc.) that all draw from the same content configured in a single back-end CMS. This can greatly reduce the work entailed in developing heterogeneous front-ends while also simplifying both the back-end content management by content administrators and the front-end interface implementation by developers skilled in each of the specific technologies.
I would like to express my appreciation for Rui Yang of the 9/11 Memorial and Museum and the 2019 summer interns (Kai Heng Chen and Andy Wang) who worked on various aspects of the implementation of the Timeline website. I would also like to thank Ms. Nancy Morrissey (SVP of Digital Products and Technology at the 9/11 Memorial and Museum) for always encouraging me to take on challenging projects and providing crucial support throughout the development and implementation process. Finally, I would like to give a big shout-out to my husband Prashanth Krishnamurthy for always motivating me to expand my boundaries and also my two small kids who have been very patient while this paper has been competing with them for my attention.
Buytaert, D. (2019). “How to decouple Drupal in 2019.” Consulted January 15, 2020. Available https://dri.es/how-to-decouple-drupal-in-2019
Heslop, B. (2018). “History of content management systems and rise of headless CMS.” Consulted January 15, 2020. Available https://www.contentstack.com/blog/all-about-headless/content-management-systems-history-and-headless-cms
Mayekar, D. (2019). Decoupling Drupal: a decoupled design approach for web applications. New York: Springer.
Schadler, T. & M. Grannan. (2017). “The rise of the headless content management system.” Forrester. Consulted January 15, 2020. Available https://www.forrester.com/report/The+Rise+Of+The+Headless+Content+Management+System/-/E-RES132202
So, P. (2018). Decoupled Drupal in practice: architect and implement decoupled Drupal architectures across the stack. New York: Apress.
Nagaraja, Radha. "Losing our Heads: Leveraging Headless Content Management Systems to Bridge Dynamic Web Interfaces and Backend Content." MW20: MW 2020. Published January 16, 2020. Consulted .