Towards an Impact Evaluation Framework to Measure Urban Resilience in Food Practices

Abstract

The relationship among agriculture, food and cities is experiencing profound transformations that led us to reflect on causes and processes. Our research questions regarded the role of agriculture and food in territorial resilience, the relationship between global problems and local solutions (i.e., urban scale), the relationship between the action scales and the results of a practice, and the means to measure the effectiveness of a practice. The following paper adopts the coevolutive approach, which recognizes territorial dynamics as products of biunivocal relations between social and environmental components. We also outline an impact evaluation framework for assessing territorial resilience of urban food systems. The paper includes an analysis conducted on 50 local practices regarding the relationship between food and city. This analysis was collected within the Observatory of Resilience Practices, a project funded by the Cariplo Foundation and conducted by the Polytechnic University of Milan. The paper concludes by suggesting implementation of the methodology for assessing the impact of practices, and includes broader reasoning regarding the role of local bottom-up practices in territorial governance.

1. Introduction

Currently, the relationship among agriculture, food and cities is experiencing profound transformations that led us to reflect on the causes, effects, factors and processes [1,2,3]. It is clear that patterns of food production, consumption and wellbeing develop not only from economic and social relations of urban systems, but also depend on their capacity to sustain them (i.e., their resilience) [4,5]. We investigated the physical dimension of urban systems, their role in relation to the use of natural resources, the opportunity that they offer to the innovation [6], and particularly to their reaction capacity to pressures and fragilities (e.g., climate change, globalization of markets, the generation change of farmers, changes in dietary habits, changes in the relationship between places of production and consumption, etc.) [7].

Bauman sees cities as burdened with problems born at a global scale [8]. Since citizens have to find solutions at an urban scale, could agriculture and food be local instruments for urban resilience in the face of global problems [9]? Could agriculture and food production be a connecting link between management of ecosystem services, landscape protection, preservation of biodiversity, social inclusion and other issues of complex territorial systems [10,11]? To answer these questions, it is necessary to recognize agro-ecosystems not as antithetical entities to urban areas, but as integrated phenomena, able to play a key role in the development of territorial systems [3,12].

Using these questions as a starting point, we investigated economic, social, and environmental aspects of urban food practices, as well as their capacity and effectiveness to improve urban resilience [13]. For this investigation, we adopted the co-evolutionary paradigm as a holistic approach towards a theoretical framework to understand the transformations in social-ecological systems such as cities and urban food systems [14,15,16,17].

In addition, the following paper describes a replicable holistic framework that can assess and measure the impact and the effectiveness of practices [18] on urban resilience with a co-evolutionary optics, i.e., with social and ecological components and implications.

To achieve this, we decided to use 50 local practices related to food–city relationships. The examples were collected with the public call for “Resilient Communities”, as part of the Italian Observatory of Resilience Practices (ORP) project born in April 2015. This project was funded by the Italian bank foundation Fondazione Cariplo, and conducted by Polytechnic University of Milan, with a scientific committee of people from various organizations and locations in Italy [19].

The Observatory of Resilience Practices consists of several axes. The Mapping Path, which is the mapping of collected practices and is a strategic axis of the ORP, requires a tight integration between the construction of a conceptual framework, the direct involvement of the promoters of initiatives, and an innovation in the identification of methodological and design tools. The Mapping Path provided the opportunity to the whole local community to propose and subsequently validate practices that fall within Holling’s definition of ecosystem resilience [20]. “Ecosystem resilience” assumes a dynamic and evolutionary vision of possible response mechanisms of complex territorial systems [20]. Strengthening resilience properties of complex systems (such as territorial ones) allows the same systems to activate responses, adaptation, and change in face of exogenous or endogenous disturbances. With strengthened resilience properties, systems may reconfigure towards new equilibrium conditions, as well as acquire new capacities to face the future.

With that theoretical reference point, the Observatory collected 100 practices: 24 at Lombardy scale, more than 40 from within national context (i.e., Italian scale) and 30 scientific applied research studies. The practices are concerned with varying themes, dealing with both social and environmental dynamics, but they have the same objective of strengthening resilience against fragilities recognized in the system as follows; the issue of social inclusion, the support for disadvantaged groups, the prevention of hydrogeological instability, the support for the local economy, the redevelopment of degraded public spaces, the protection of endangered species, and much more.

We decided to use 50 out of the 100 practices collected concerning food and agro-ecosystems because, with a starting point of Holling’s definition of ecosystem resilience, agro-ecosystems turn out to be integrated socio-ecological systems suitable to be considered as bearers of a key role for the whole territorial system. Therefore, this sample can become an investigative opportunity on the evaluation method of the effectiveness of bottom-up practices.

2. Materials and Methods

We emphasize that, within this context, the concept of mapping does not attempt to question the degree of resilience of practices. Rather, starting from the assumption that the practices are all recognized in Holling’s definition, mapping primarily attempts to describe geographies, endogenous and exogenous factors, and characteristics of the practices purview.

With this assumption, we chose to adopt the co-evolutionary paradigm of Norgaard [14,15], which states that economic history is a process of adaptation to changes in the environment, and that such transformations are bijective. For this dynamic relationship between environmental and social systems, the co-evolutionary paradigm is the key to better understanding the transformations of socio-ecological systems, such as agro-food systems.

By adopting this theoretical approach as the main reading key, we analyzed these 50 initiatives in a large and detailed database. The 50 practices we investigated use food and agriculture to strengthen resilience of different fragilities. The practices include strategies for the inclusion of urban agriculture for the reappropriation of degraded territories and/or for the support of disadvantaged groups, the launching of alternative quality practices to traditional production–distribution systems, establishment of support systems for local quality agriculture and networks, strategies to restore traditional crops, and many others.

Every practice was analyzed by examining the individual actions (which may be similar in different practices), the contingent geographical and socio-economic contexts, the fragility being addressed, the objectives that were set, and the expected results.

Accordingly, the analysis of the database described in detail the following: the geographical location, the problems being addressed (described here as “pressures”), the objectives that were proposed, the individual actions planned and implemented (which, as previously mentioned, can be similar for different practices), the expected results, and the planned and/or performed monitoring.

The descriptive database that was obtained is the main source material for this investigation, and represents a potential foundation and starting point for further research. Starting from the same definition of resilience, the database represents a homogeneous sample to experiment with new evaluation methodologies.

The investigation of the database took place at three different times: (i) at the completion of the database both from an analytical–descriptive point of view and from a general interpretative re-elaboration; (ii) at the analysis of the origin of the destabilization of the system, from where these project initiatives are derived; and (iii) upon the interpretation and re-elaboration of practices, with the main goal being implementation of an impact assessment framework.

Here, we focus on Points (ii) and (iii), which are closely related to each other on the interpretative level, given that, as a problem arises, a specific solution inherent to the problem is described. These two areas are also related in terms of expected results, as well as spatial and temporal action scales.

With regard to Point (ii), we have already said that the practices we analyzed are born as solutions to an identified problem. For this reason, the analysis of the practices should start from the analysis of the problem from which they derive, as well as if and how they solve it, especially if oriented to the implementation of a framework that investigates the impact. Additionally, analysis included monitoring the effectiveness of the practice(s) over time.

To do this, we identified two classes of categories that define the nature of the problem: (a) endogenous/exogenous, which serves to understand if the cause is internal or external to the territorial system, and therefore help us discern if it is a local and/or global scale problem (this identification is useful for further investigations on the relationship between the scale of the problem, the scale of the practice and the scale of the reverberation of the practice); or (b) social/ecological, which tries to understand whether the problem derives from anthropic activities or from causes more directly related to the natural system.

The application of this double pair of categories led to the definition of the sample in terms of the origin of destabilization. This characterization led to a fundamental structuring of the evaluation framework. In fact, the individual actions may be the same in different practices (e.g., urban gardens), but the origin of pressures or fragilities on which they are implemented (e.g., economic support for disadvantaged categories or degradation of public space) define them differently.

Following a detailed analysis of the individual actions conceived for each practice, four macro-categories were then identified, representing areas of resilience on which the practices are oriented.

The four categories also coincide with different reference systems:

• Food system, i.e., the practices that aim to support local short supply chains;
• Territorial system, i.e., the practices aimed at redevelopment, enhancement and protection of urban space, territory and landscape;
• Social system, which includes practices that aim to raise awareness and strengthen the resilience of the local community; and
• Risk management, a category that focuses on the broad concept of “risk” (e.g., hydrogeological risk, risk of loss of animal species, consumption of soil, risk of economic crisis).

Having already said that resilience embraces very different dynamics and objectives, these macro-categories allowed us to characterize the evaluation framework by describing the individual actions of the practices on different systems (i.e., food, landscape, social, risk).

Having defined the material, how we evaluated it and what we wanted to derive from it, the main reference used for the method adopted to develop the investigation was the EKLIPSE Expert Working Group report “An impact evaluation framework to support planning and evaluation of nature-based solutions (NBS) projects” [21], prepared by the EKLIPSE Expert Working Group on Nature-based Solutions to Promote Climate Resilience in Urban Area (EWG). Nature-based Solutions (NBS) are solutions to societal challenges that are inspired and supported by nature. The European Commission requested the EKLIPSE project to help build evidence and knowledge base on the benefits and challenges of applying NBS. In response to the request, the EKLIPSE document is not intended to define NBS, but rather provide examples of indicators and methods for assessing impacts of NBS that may be applied in a range of different ways across urban areas in Europe. EWG specifies that the type of NBS impacts may vary according to the context in which they are applied, and the same applies to our practices.

The choice to have this document as main reference for our investigation was born both for the nature of analyzed content, which is congruous with our database, and for the shared objectives that this document shares with our work.

Regarding the processing of indicators, since literature analysis is not the main goal of this paper, we have referred to previous research activities on the bibliography in the field of food policy and food systems assessment and evaluation [10,22].

Whilst Prosperi et al. [22] proposed a review of eight initiatives focusing on the assessment of the sustainability of the food system implemented at different scales, Mazzocchi and Marino [10] integrated such list to gather the 14 main experiences of assessment of the sustainability of food systems, internationally and at different scales, reaching a list of 10 goals and 54 objectives, possibly exhaustive of all the possible objectives that can be pursued in an urban food policy. The two works, together, represent an exhaustive picture of the state of the art of indicators on food policies, useful to a conscious starting point for new reasoning on the topic.

For example, regarding the indicators we have developed to measure the increase of local community awareness and of local community involvement, we have based them on the indicators of the City Region Food System Indicator Framework [23,24]; among them, we focused on those oriented towards “Health and well-being and increasing access to food and nutrition” with the specific impact area on education and awareness, such as “increase in number of community residents involved in community-based food activities”, “change in consumer knowledge on healthy diets for different age and income groups” and “change in consumer awareness on healthy diets food/environmental impacts of their food consumption among different groups”.

For our environmental indicators, for example those regarding the water saving, the reduction of waste, the requalification of territory, and the increase in the use of renewable energy, we referred to the Sustainability Assessment of Food and Agriculture Systems indicators [25], in particular to the Environmental Integrity framework for indicators on Water, Land, Biodiversity and Energy.

It has to be specified that the developed of some indicators need to be adapted to the local situation and, being many of them of the composite type, require more detail in terms of definition (for example: “Growth rate of consumption of local products” definition of spatial scales) or flows of information and data for their calculation (for example: “Growth rate of local community perception” or “Growth rate of flows of people”).

As regards data collection, we can think of both quantitative (through the monitoring of practices over time) and qualitative methods (through interviews and questionnaires). Where data will be difficult to collect, existing case studies and research may be useful.

The collective process is configured as a research process in itself and even identifying where data are missing would be an important finding [24].

3. Results

Since the data of the database analysis: (i) have already been partly described previously within the description of the practices, this section focuses on the results concerning the analyses (ii) and (iii), which are closely related to each other on the interpretative level: if a problem (ii) arises, a specific solution (iii) inherent to the problem is relayed. These two analyses, (ii) and (iii), led to the definition of assessment frameworks for their impact.

3.1. Analysis of the Pressures Categories

Figure 1 below describes the results of analysis (i), representing the overview of the destabilization of the socio-ecological systems we are referring to.

The criteria for the attribution of the two pairs of categories are based on two principles: (i) the pressures deriving from an anthropic origin are only those problems directly connected to human activities; and (ii) the pressures of environmental origin are those that directly affect the natural components (even if indirectly the origin is anthropic).

For example:

-

Pressures assessed as being of anthropic origin were: infrastructural pressure, lack of care of public spaces, abandonment and depopulation of agricultural areas, and economic fragility of local farms.

-

Pressures assessed as being of environmental origin were: presence of invasive species, risk of hydrogeological instability, loss of biodiversity and ecosystem functionalities, and environmental pollution.

Regarding the second couple (internal/external origin), the phenomena of global origin were evaluated as “exogenous” pressures, i.e., external to the system. The causes of local origin, strictly connected to the local characteristics, situations or behaviors, were evaluated as “endogenous” pressures, that is internal to the system.

Figure 1 shows a homogeneous distribution with respect to the two categories crossed in the four quadrants, as a result of a weighting carried out by the authors following an expert assessment. This means that the origin of the fragility has been identified with a qualitative evaluation by the authors that compared and verified the problems declared by the designers of the practices (to which the practices tend to provide a solution) with the expected results, i.e., the expected impact.

For example, in a village of the suburbs of Mantua (Italy), a cultural association warns of a growing deterioration in an area, with the abandonment and neglect of public spaces. The practices consisted of the activation of urban gardens and the launching of training activities aimed at the local community of the area. These practices led to an impact on the recovery of the territory and the involvement of the local community. From the description of the reference system, the action, and the expected impact, it was deduced that the origin of the problem was the negligence of public spaces, as well as the loss of the sense of belonging by the local community. Therefore, a qualitative evaluation led to identifying a social and endogenous origin of the problem.

3.2. Analysis of the Resilience Categories

For each category, a framework has been set up that analyzes the actions of each practice with respect to the origin of the problem, as well as the expected impact and the indicator that would verify its effectiveness.

In this way, four different frameworks are obtained, with the practices distributed homogeneously: in the first framework there are 12 practices (Figure 2), in the second framework there are 13 practices (Figure 3), in the third framework there are 15 practices (Figure 4), and in the fourth framework there are 10 practices (Figure 5); for each practice two main actions are analyzed. Sometimes the expected actions or impacts can be repeated in different practices and/or frameworks. This is because the type of action can be the same but the context (evident from the categories of pressures, endogenous/exogenous-ecological/social) or the objectives and purposes (i.e., frameworks) are different.

For example, “activation of urban gardens” is an action present in the framework of territorial requalification (Figure 3) and that of strengthening the social system (Figure 4), as well as in the framework regarding risk management (Figure 5). In the first case (Figure 3, territorial requalification), the expected impact is the recovery of the territory and the impact indicator is qualitative, based on the perception of the local community. In the second case (Figure 4, strengthening of the social system), the action is present several times but with different expected impacts; the involvement of the local community, the support to poor families, the increase of territorial fruition, and the dissemination of good practices. At the same time, the indicators are both quantitative, as in the case to support poor families or in the case of practice replicability, and qualitative, as in the case of community involvement. In the third case (Figure 5, risk management), the activation of urban gardens is oriented to prepare a new local economy and therefore the indicator are sales and consumption linked to that new production.

4. Discussion

The survey revealed many topics and points of reference worthy of further investigation.

The first point of reference describes the relationship between the first part of the results concerning the origin of the problem (internal or external to the system, ecological or social) and the second part of the individual actions of the practices. What is clear from a cross-sector analysis is that, with the problems of exogenous origin (external to the system, primarily “global”), the actions are mainly macro actions on dilated temporal and spatial scales, with long impact terms and potentially involving local institutions, namely the establishment of agreements or networks between different actors, improvement in management plans, etc.

For example, Practice 11, Framework 1 (food system—support for local quality chains), describes a weakness in the local economy due to external pressure, and one of the relative actions was the establishment of a pact between producers and markets.

When the problems are of endogenous origin (therefore internal to the system and “local”), the actions are mainly micro small-scale actions and with imminently expected impacts from a temporal point of view; for example, the reactivation of agricultural supply chains, the reactivation of agricultural activity, public space care activities, etc.

In Practice 9 of Framework 4 (risk management), green space care and training, as well as involvement activities for the local community, were implemented remedially following the excessive consumption of local soil.

The second point of reference concerns the fact that, with problems of ecological-environmental origin, actions have been proposed that act on the social system, such as training or awareness activities. This indicates the close biunivocal relationship between the social system and the ecological system [14].

In Practice 7 of Framework 2 (requalification, valorization or protection of the territory) with an impoverishment of soil fertility and agro-biodiversity of landscapes due to intensive agriculture, meetings were organized between producers, consumers, and sellers, as well as laboratory activities to increase the production and consumption of quality local products.

The third point of reference concerns the nature of the individual actions with respect to the practice; that is, between two actions of a practice, often one action concerns the information, promotion, training, and involvement of the local community inside of the practice itself. This underlines the participatory, bottom-up and procedural nature of the food practices.

In Framework 3 (social system—sensitization and resilience of local community), in 11 out of 15 practices, one of the two actions undertaken always concerns the realization of educational, educational, and involvement activities of the local community.

From these results, we can deduce how effectively the relationship between food and the city (city being defined here as a complex system of physical and social relations) can be the reference framework of many transformations at both a local and global scale. Moreover, it is precisely from the subdivision of the frameworks that focuses on very different problems (e.g., social system, risk management, etc.) that we can deduce how food can actually be declined as a flexible and transversal tool oriented towards territorial resilience. Food represents a possibility for specific solutions to specific problems but also, given the holistic, complex, and transversal nature of the concept of food itself (linked to the environmental, social and economic system), it represents an adequate and congenial tool for interscalar relationships such as those of resilience in urban systems.

5. Conclusions

The survey provides the opportunity to reflect on several fields of innovation of the scientific research. The first point is related to the initial objective, in that we obtained a first implementation of a replicable holist framework for assessing and measuring the impact and the effectiveness of practices on urban resilience with a co-evolutionary optics, i.e., with social and ecological components and implications. The second point of reflection is the theme of agriculture not as an antithetical entity to the urban system but as an innovation tool for strengthening urban resilience. Then, from this analysis, we further evaluated the relationship between the scale of the problem, the scale of the practice, and the scale of material and immaterial reverberation. The next step could be to integrate the framework proposed here with spatial indicators, and to compare the measure of the origin problem with the measure of the final impact of the practice, as well as its reverberation over time [22]. Lastly, the following document raises the issue of territorial governance in relation to the role of local bottom-up practices, such as in solving problems that are beyond the institutional regulatory instruments. Therefore, the possibility exists to see an inductive model in this framework; namely, starting from the territorial practices and through the evaluation and monitoring of the impact of these over time, we can begin to extrapolate and suggest new orientations for adaptive and resilient local governance systems.

These fields of innovation are closely linked to the open scenarios proposed by this survey: in the short term on the implementation of frameworks with indicators that evaluate the relationship between different spatial and temporal scales (between practice and impact); in the long term on the role of the effectiveness of bottom-up practice with a greater spatial reverberation than its activation, in a territorial governance that crosses the administrative boundaries and returns to the natural geography of places and settlements.