Context

Research context

In cities such as Hong Kong, there is a constant struggle between human traditions, forces of technologically driven desires (e.g. ‘smart cities’), and natural contextual constraints (e.g. climate change). The sites that a community values combine physically persistent and ephemeral qualities. As discussed in our recent study, urban places that are culturally valued typically present an “intertwined tangible-intangible duality, expressed both as a physical construction and as a set of social, traditional practices” (Lenzi, Sabada, & Lindborg 2021). 

In the present proposal, we adopt the notion of ‘sensory heritage’. France has recently adopted a law that explicitly aims at protecting culturally valuable sounds and smells (patrimoine sensoriel sur les sons et odeurs; see Morel-à-L’Huissier 2020). The background to this concept lies in tangible heritage which emphasises “architecture, their homogeneity or their place in the landscape” (UNESCO 1972), and intangible cultural heritage [ICH], which is “any non-corporeal manifestation of tradition-based creativity [that reflects] the community’s social or cultural identity. It includes… the social, intellectual and cultural processes that… have made possible the development of a distinct cultural tradition whose preservation and protection is important…” (UNESCO 2021b p. 5). The latter is typically related to rituals, festive events, and crafts within a community, yet it is not so much about the cultural manifestations themselves but rather the wealth of knowledge and skills being transmitted (UNESCO 2003–). Moreover, social enactment is an “essential and defining aspect of intangible heritage, in the sense that this heritage exists and is sustained through the acts of people” (UNESCO 2021a p. 2). Soundscape is recognised in UNESCO (2017, p. 57)

However, the tangible/intangible framework is not without critique (see e.g. Caust & Vecco 2017) and the dichotomy enshrined in these conventions may appear artificial in the way it is based in an entrenched hierarchisation of the senses. It has proved easier to find international agreement about values that can be perceived with the eyes, over those that can only be appreciated from touch or hearing, and certainly over those which come to us via the ‘lower’ olfactory and gustatory senses. How can we frame a research project to avoid conceptual challenges of this kind? Firat (2020) argued that sounds, and acoustics in general, are in fact tangible, because “any sensory modality which can be preserved with digital methods should be regarded as tangible heritage” (p. 3). His socio-historic analysis argued that insisting on this dichotomy perpetuates a Western worldview dominating the Eastern; the former is oculo-centric, while in the latter, imagination is “beyond the visual” or any other sense (p. 5), noting that “ephemerality is one of the main attributes of auditory [and] olfactory… cultural objects, which cause them to be considered as abstract phenomena and associated with the intangible heritage” (p. 9). 

For the proposed project we avoid the tangible/intangible dichotomy altogether by adopting the notion of sensory heritage (Morel-à-L’Huissier 2020). Furthermore, we are influenced by the Burra Charter’s definition of cultural significance as the “sum of aesthetic, historic, scientific and social values”, in particular because ‘aesthetic value’ here includes the “smells and sounds associated with the place and its use” (ICOMOS 1999 p. 120). A practical example is given by Bembibre and Strlič (2017) who discuss smellscape as a constituent part of place-making in cultural heritage sites as well as its use in museum exhibition design, and soundscape as cultural heritage is discussed by Jordan (2017; also Jordan & Fiebrig 2020).

Pilot survey

To gain a better understanding of potential sites in Hong Kong, the PI conducted a pilot survey (n = 60) to estimate the kinds of urban places that people would find are the most important in relation to ICH in Hong Kong (or another similar Asian city). See Figure 3 and the last page of the Project Statement (‘Appendix’). Interpreting preliminary results, we believe it is reasonable for the proposed research project to focus on the top three ranked types of places: 

Street food (街頭小食). It has been noted that “food is culture when it is produced because humans do not use only what they find in nature but want to create their own food” (Maffei 2012 p. 225-6). Indeed, food production, preparation, and consumption are practices that are simultaneously traditional and expressive of contemporary living culture. It is paradoxical that “something as perishable as food constitutes a living legacy of the past… The transmission of food practices from one generation to the other contributes to constructing identity and memory of both individuals and communities [and constitutes] a form of intangible cultural heritage” (Vadi 2013 p. 3; cf. Kong 2015). 

Chinese temples (寺廟 [佛祖, 天后). As indicated by the Chinese characters, we focus on Taoist and Tin Hau (‘joss houses’) because these have developed rituals and expressions that are idiosyncratic to Hong Kong (and in the larger diaspora of overseas Chinese, 海外華人). Therefore we exclude places of worship of other religions while being aware of the large range of religions active in contemporary Hong Kong. Other kinds of temples might form the subject of future research.

Wet markets (傳統市場) are “open-air, partially sheltered spaces populated with dozens of vendors selling fresh food products, such as fish, poultry, pork, beef, vegetables and fruits, as well spices and other sundries. The name, ‘wet market’, refers to the routine acts of hosing down and washing the market’s floors with water to keep areas clean and sanitary.” (Mele et al. 2015 p. 106). Wet markets in Asia are rather like the local grocery stores in Western countries. The seasonal availability of items govern the rhythm of culinary practices and calendar events in people’s lives. Examining social identity in Singapore, Mele (2015) contended that the “collective attachment to wet markets anchors Singaporeans and provides a fragile basis for consistency and stability.” (p. 105). Their analysis, if applied to Hong Kong, supports the argument that it is both valuable and urgent to document such places, to empower the cultural understanding of residents today and for the future. Moreover, note in this context that a clear distinction must be made to ‘wildlife markets’, which trade in living animals that are often caught wild. Wildlife markets have received considerable bad press during the COVID-19 pandemic as it is believed that zoonotic diseases can spread to humans at such places (e.g. Zhu & Zhu 2020). In the proposed project we do not include ‘wildlife markets’ in the sample. 

Research question

The knowledge gained so far leads us to ask: How do sounds and smells contribute to our appreciation of culturally significant places? Can we really claim to know urban places without thoroughly considering, and documenting, the sensory cultural heritage represented by sounds and smells? How could a digital archive of environmental factors be constructed to contribute as broadly as possible to research and development? The simple answer is that we must build knowledge that is as rich as possible. Noting the current lack of data, two Co-Is of the present proposal, Xiao and Aletta, and collaborators (2021) concluded that “the biggest challenge to the advance of smellscape research and practice in the built environment is the lack of applicable smell databases to aid the design process and predict outcomes” (Xiao et al. 2021 p. 2). Therefore the proposed project addresses the need for an open-access ‘smellscape archive’ via Objective 1.

Methodology

To deepen our understanding of meaning-creation in complex physical environments, the project adopts a multimethodology, combining quantitative and qualitative methods. This reflects the way that, in people’s experience of living environments throughout life, their senses combine to generate memories with cross-modally interrelated information (Spence 2020, p. 2). Congruence between sensory information channels  – especially auditory, visual, and olfactory – leads to perceptual processing fluency, which in turn is associated with positive evaluation (Reber et al. 2004). More knowledge is needed on how people are affected by sound (noise, music, signals, soundmarks) and smell (odours, scents, flavours, smellmarks) throughout their everyday activities (Xiao et al. 2021). Based within a paradigm of embodied ecological perception, our project aims at accurately documenting and describing complex real-life environments in their multi-sensorial richness.

Soundscape studies have a natural affinity with environmental psychology. The soundscape is both an indicator of environmental quality and a component of cultural identity. The reliance on traditional descriptors of the acoustic environment is giving way to perceptual qualities such as Pleasantness/Eventfulness (Aletta et al. 2016) or Calmness/Vibrancy (Aletta & Kang 2018), and context-dependent sound source identification (Lindborg 2016). See also Jordan and Fiebrig (2020) who assess ‘significance’ and ‘meaning’ in historical soundscapes. The PI and collaborators examined the soundscape of a city plaza in the Basque Country as it changed during Covid-19 social restrictions (Lenzi et al. 2021), collecting daily soundscape recordings and other data over a two-month period. Two expert panels (n = 11, 12) annotated sound sources and estimated perceptual qualities. Computational descriptors such as acoustic richness and loudness, and the ratio between perceived natural, vocal, and technological sounds, gave evidence to how new regulations obliged people to adapt and reshape their social activities, by reducing the use of motor vehicles and spending more time outdoors. 

Smellscape is the olfactory environment as perceived and understood, consisting of odours and scents from multiple smell sources (Lindborg & Liew 2021, forthcoming). The term was introduced by Porteous (1985), who based his methodology on soundscape research. While a ‘soundmark’ (as defined by Schafer) is a sound that is intimately linked to a site and carries meaning for its community, a ‘smellmark’ is a culturally highly important smell. A smellscape should be understood as the sum total of numerous smell sources that each may connote cause and effect (Porteous 1985, p. 360). When considering living spaces and everyday environments, the quality of the olfactory environment affords stress recovery (Hedblom et al. 2019) and affects people’s general wellbeing (Naddeo et al. 2013). Xiao and collaborators (2020, p. 14) wrote that “smellscapes are representations of individuals’ imaginations of places, triggered by smells in a space-time structure” (see also Xiao 2018 p. 106). The PI and Co-I (Liew) have recently completed a study of seven locations at Tiong Bahru market, Singapore (Lindborg & Liew 2021, forthcoming). We conducted a sensory walk with one group (n = 15) who made evaluative ratings of the smellscape and annotations of specific smell sources. Another group (n = 53) was presented with audiovisual reproductions of the same seven locations and asked to imagine the smellscape and specific smell sources (but with no differentiation to their actual olfactory environment). Comparing ratings between real and imagined smellscapes using canonical correlation showed that people are able to correctly infer smells not only from visual information but also from audio-only recordings of the soundscape. This points to the existence of cross-modal associative mechanisms, likely based in long-term memory recall, emotion, and semantic mediation. We will investigate these mechanisms for multisensorial integration via further experiments.

Methods

We employ four data elicitation components (Bauer & Gaskell. 2012 ch. 1), namely Measurements, Recordings, Sensory walks and Interviews. The main outputs are a database and a case study. The research purpose is to empower residents through a deeper understanding of their sensory cultural heritage. Figure 1 gives an overview of the methods that we employ in relation to each of the four main Objectives.

Database

To achieve Objective 1, we create a multimodal database that documents salient physical and intangible aspects of ~100 sites of cultural value to Hong Kong. Methods will build on our previous work on restaurant soundscapes (Lindborg 2015, 2016) and marketplace smells (Lindborg & Liew 2021, forthcoming). The field measurements and recordings of acoustic, visual, olfactory, ambient, and physical environmental aspects are outlined as follows.

Acoustic measurements include Sound Pressure Level [SPL] meter (certified Type-1 or calibrated Type-2 device. Audio recordings will be 3D [Ambisonics] using microphones [Brahmamic 2nd order HOA] and other equipment available in the PI’s current lab.

Visual measurement involves illuminance and colour metering. Video recordings will be in 360˚ [spherical] format, using equipment [GoPro Fusion] available in the PI’s current lab. We will systematically anonymise video footage (e.g. blurring of people’s faces using tools such as https://brighter.ai/, which we are currently testing in the pilot database; see Appendix) before any archiving or further analysis, to comply with Hong Kong privacy laws and best international practices (see Graham 2012 for a user-friendly manual explaining the EU Data Protection Act of 1998). Note that audiovisual recordings do not replace objective onsite audiovisual measurements, since such comparison cannot reliably be made between movie clips from different locales. 

Making olfactory measurements, ‘smellprints’, represents a significant part of the proposed research. The Chemistry Lab at City University of Hong Kong is equipped to perform one-dimensional gas chromatography – mass spectrometry [GCMS] to yield detailed chemical profiles (see Belgiorno et al. 2013, esp. Chapter 3). The initial scope of ~100 sites will produce around 300 to 400 air samples for GCMS analysis; some of this work would be suitable for student Final Year Projects, thus anchoring the project in interdisciplinary pedagogy (see Education Plan). A detailed protocol for sampling and measurement procedures will be developed by PI and Co-Is (Matsuda, Xiao). One important avenue of future research lies in psychophysical linking of GCMS data with human assessment of perceived smells and smellscapes (see Xiao et al. 2021 p. 2; cf. ‘dynamic olfactometry’, e.g. Belgiorno et al. 2013 p. 40-3). A possible approach follows Zarzo (2021), starting with domain-blind classification (e.g. multivariate clustering analysis using principal component analysis), then identification against known chemical compounds (‘perfume raw materials’) by non-negative matrix factorisation. 

Documentation of physical aspects will include layout (blueprints, photos, maps etc), observational annotation of people activity (flow, headcounts, crowdedness), cost levels (in the case of Street food places and Wet markets; comparing specific items across locations, as in Lindborg 2015), annotations of building materials, and so forth. Part of this would also include a ‘softer’ aspect, utilising retrieval of social media or blog postings about these sites as an archival of collective experiences of visitors to the locations (see also Case study, below).

To keep the project budget within reasonable limits, we will as far as possible employ off-the-shelf multimeters or smartphone apps for acoustic, visual, and other ambient measurements (e.g. temperature, humidity). For example, it is possible to calibrate apps (e.g. SPL Pro, LightSpectrum Pro) against professional instruments (e.g. certified Type-1 SPL meter; Aumond et al. 2020 provides a method) to achieve high-quality measurement. 

The integration of these data in a multimodal record is crucial to achieving Objective 4. The project will build on Co-I (Aletta)’s ongoing work with the Soundscape Indices project, which aims to achieve a database of similar level of complexity (Mitchell et al. 2020). 

Case study

The reach Objective 3, the in-depth case study of social habits and servicescape features, we focus on three sites within the larger database sample, such as one in each of the three main categories as described above, and employ qualitative methods (Chen et al. 2020a, especially Appendices A–C for procedures; Lindborg 2021). They are:

Sensory walks is a systematic research method whereby people move through a physical environment while activating all their senses, and make observations (smellscape: Henshaw et al. 2009, 2010; McLean 2017; Bembibre & Strlič 2017; Xiao et al. 2020; for soundscape: Koseoglu 2016; Aletta et al. 2016; and for both: Bruce et al. 2015), complementing objective methods for the database. In the recent work by PI and Co-I, the observers received instructions to make structured annotations of significant sounds (i.e. soundmarks) and smells (i.e. smallmarks) at seven different locations in a wet market (Lindborg & Liew 2021, forthcoming). To anchor the research project in the general public, we will invite local residents and visitors (i.e. stakeholders) to join the walks. These sessions would include a post-walk focus group discussion to understand how the smell- and soundmarks relate to their everyday life and cultural values (cf. Rae & Hong 2003). 

Social media messages that are tagged or geo-located to the sites will be sampled and analysed using natural language processing (NLP), such as Latent Dirichlet Analysis (LDA), topic modeling, and sentiment analysis, for quantitative analysis. This will yield further information as to stakeholder perception of the sites being studied. Co-I (Liew) and PI will also crowdsource for narratives of memories and experiences (Manabe, Liew, et al. 2021) associated with these places through social media, such as Twitter (Liew & Lindborg 2020), in comparison with interviews (see below), to contextualize the collective memories associated with these places and add richness to the collected dataset.

Interviews. To determine social practices, behaviours, and perceptions of place, interviews will be made with market workers, i.e. store keepers, cleaners, owners, and with returning customers/residents, occasional tourists/visitors, and other participatory actors at the sites. The protocol will establish the purpose of the study, confidentiality, data protection, and other matters relating to best practices. An example is given in PI’s study of perceptions of a sound installation (Lindborg 2021). We developed a protocol for semi-structured group interviews in Mandarin and Cantonese (conducted by assistants), which were recorded and translated into English for content analysis using a coding frame method (Bauer & Gaskell. 2012 ch. 8). In the proposed project, crowdsourced narratives will also be analysed by PI and Co-I (Liew) using NLP techniques applied to the interview transcripts.

A stakeholder workshop will be used to produce actions and strategies on how this research could be applied in policy and management (see Objective 4). This will help us to understand how multimodal data can support future development of ‘virtual tourism’ applications (and for exhibition spatial design, consider MOMA /New York, Louvre /Paris, MOA /Macao (Lai 2015), integrating historically informed scent design in some of their tours and displays). 

Appendix

Details about the pilot survey

To gain a better understanding of potential sites in Hong Kong, the PI conducted a pilot survey (n = 60) to estimate the kinds of urban places that people would find are the most important in relation to ICH in Hong Kong (or another similar Asian city). 

One group of participants was sourced from Prolific.co, preselected by nationality: either Hong Kong, Singapore, or Taiwan (n = 51), and another group by snowball (convenience) sampling starting in Hong Kong (n = 21). Note that Prolific.co does not at present enlist people based outside the OECD in their pool of subjects. Participants could choose to take the survey in either English or Cantonese (i.e. traditional Chinese; translations were independently verified by four native Cantonese speakers). They were asked to rank eight types of locations using an interactive box-dragging method. A definition of ICH was provided (UNESCO 2021a), and the instruction: “Below are different TYPES OF PLACES that are relevant to the lifestyle in Hong Kong and similar Asian cities. From your own perspective and lived experience, how IMPORTANT do you see each one in the context of intangible cultural heritage?”

To filter out ‘junk responses’ (Oppenheimer et al. 2009), the ranking task was presented twice, at the beginning and end of the survey, sandwiching demographic questions. We estimated internal consistency with Spearman’s non-parametric correlation between rankings within each participant. Those with rho > 0.1 were discarded, and the remaining merged (n = 60). Mean age was 29 years (range 20…54), and there were 38 females. There were 25 Hong Kong nationals, 20 Singaporeans, and 12 Taiwanese (and 3 others). Twenty-eight had lived the longest time in Hong Kong, 16 in Singapore, and 9 in Taiwan. The duration they had lived in this city (after the age of 12), was 5-10 years for 15 participants, and more than 10 years for 35. These statistics show that our sample had substantial living experience in Hong Kong or another comparable Asian city, so that they can be relied upon to have gained a personal, lived opinion on which kinds of places would be important.

Means were calculated across all rankings. This yielded scores for the eight place-types (where a high number means more important for ICH), as follows: Street food (6.1, 街頭小食), Temple (5.7, 寺廟), Wet market (5.2, 傳統市場), Night market (4.8, 夜市), Hawker centre (4.5, 熟食中心), Cemetery (3.5, 墓園), Urban park (3.2, 城市公園), Shopping mall (1.7, 購物商場). See bar graph plot in Figure 3.

Pilot database

The PI and assistants have initiated a collection of audiovisual recordings and smellscape annotations at sites in Hong Kong, especially wet markets, using presently available equipment: Zoom HR-VR for 1st order Ambisonic audio, GoPro Fusion for 360˚ video, and onsite olfactory annotations using a ‘smellwheel’ (McGinley & McGinley 2002; see also Lindborg & Liew 2021, forthcoming).