Next Article in Journal
Circular Economy Concept in the Context of Economic Development in EU Countries
Next Article in Special Issue
Comparative Assessment of Different Modelling Schemes and Their Applicability to Inland Small Reservoirs: A Central Europe Case Study
Previous Article in Journal
Integrating Environmental, Geographical and Social Data to Assess Sustainability in Hydrographic Basins: The ESI Approach
Previous Article in Special Issue
Mapping Cultural Ecosystem Services Enables Better Informed Nature Protection and Landscape Management
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Hyperspectral Reflectance as a Basis to Discriminate Olive Varieties—A Tool for Sustainable Crop Management

1
MED-Mediterranean Institute for Agriculture, Environment and Development, Institute for Advanced Studies and Research, University of Évora, 7006-554 Évora, Portugal
2
MED-Mediterranean Institute for Agriculture, Environment and Development & Department of Rural Engineering, School of Science and Technology, University of Évora, 7006-554 Évora, Portugal
3
MED-Mediterranean Institute for Agriculture, Environment and Development & Departament of Phytotechnics, School of Science and Technology, University of Évora, 7006-554 Évora, Portugal
*
Author to whom correspondence should be addressed.
Sustainability 2020, 12(7), 3059; https://0-doi-org.brum.beds.ac.uk/10.3390/su12073059
Submission received: 11 March 2020 / Revised: 2 April 2020 / Accepted: 3 April 2020 / Published: 10 April 2020

Abstract

:
Worldwide sustainable development is threatened by current agricultural land change trends, particularly by the increasing rural farmland abandonment and agricultural intensification phenomena. In Mediterranean countries, these processes are affecting especially traditional olive groves with enormous socio-economic costs to rural areas, endangering environmental sustainability and biodiversity. Traditional olive groves abandonment and intensification are clearly related to the reduction of olive oil production income, leading to reduced economic viability. Most promising strategies to boost traditional groves competitiveness—such as olive oil differentiation through adoption of protected denomination of origin labels and development of value-added olive products—rely on knowledge of the olive varieties and its specific properties that confer their uniqueness and authenticity. Given the lack of information about olive varieties on traditional groves, a feasible and inexpensive method of variety identification is required. We analyzed leaf spectral information of ten Portuguese olive varieties with a powerful data-mining approach in order to verify the ability of satellite’s hyperspectral sensors to provide an accurate olive variety identification. Our results show that these olive varieties are distinguishable by leaf reflectance information and suggest that even satellite open-source data could be used to map them. Additional advantages of olive varieties mapping were further discussed.

1. Introduction

Sustainable development at a worldwide scale is crucially dependent on changes in land use structure [1], particularly with respect to the increasing global food demand and increasing land scarcity for agricultural production [2,3]. In terms of landscape dynamics, there have been three dominant agricultural land change processes with impact on biodiversity and nature values over the last half-century [4,5]: (1) In big cities surrounding areas, agricultural land was converted to urban use associated with rapid urbanization processes as a response to growing demographic demands [6]; (2) more economically productive areas have been intensified, incorporated into larger assemblages particularly within developed countries [7]; and (3) conversely, unproductive farm areas were increasingly abandoned, reforested, or included in rewilding for nature values with the creation of nature reserves or parks [8,9,10].
Agricultural land change phenomena—rural farmland abandonment and agricultural intensification—are ongoing processes all over the world and particularly in Europe, with potential important negative social and environmental impacts. Agricultural land abandonment has been an increasing problem, mostly in Southern Europe throughout the last decades [11], contrasting with Northern and Western Europe where agriculture intensification/expansion is the dominating land-use change process [12]. The extent of such landscape changes raises deep concerns in European authorities since they could entail significant negative impacts. Despite the potential positive role of abandoning areas on, for example, forest regrowth, natural regeneration, biodiversity and carbon sequestration, there is great concern on increased fire risk due to homogenization of woody vegetation [13,14,15]. Likewise, the loss of landscape heterogeneity, the decline in resource diversity and changes in disturbance regime associated with land abandonment may equally be responsible for significant biodiversity losses [16,17]. The integration of traditional agricultural lands into a more cost-efficient and intensive agricultural model capable of a higher food production also raises concerns as such practices often carry higher negative implications compromising ecosystem functioning, biodiversity, fresh water supply, preservation of the soil and natural restoration [16,17,18,19]. Additional concerns are raised regarding the loss of regional identities linked to the threat of cultural landscapes, essential for the sustainable development of specific communities [18,20,21,22].
In the Mediterranean agriculture scenario, both abandonment and intensification have been particularly relevant in traditional olive groves [23,24,25,26]. World olive growing occupies an area of 10.2 million hectares with more than 90% of the total area located in the Mediterranean Basin [27]. Traditional olive farming is a low-intensity farming system [28], associated with a low density of old olive trees, absence of irrigation, minimal pesticides and fungicides inputs and a low degree of mechanization. They are highly environmentally sustainable, supporting high levels of biodiversity and low rates of soil erosion [28], and play an important social-economic role in rural areas while providing income and employment [29]. Among other agro-economic systems, ancient traditional olive groves are recognized as Globally Important Agricultural Heritage Systems since they play a crucial role for agrobiodiversity conservation and livelihood [30]. The traditional olive groves may play an important role in building ecological and social resilience to climate change while maintaining ecological diversity, improving adaptability and putting into practice a sustainable model of land use which may go beyond business-as-usual logics [31,32]. In a food security realm, at a time when the actual intensification processes lead to the installation of monovarietal olive groves, preservation of traditional local adapted varieties plays an important role in environmental and climate change crop adaptation as well as aid in coping with genetic vulnerability issues by acting as diversity reservoirs [33,34,35]. The abandonment and the intensification of traditional olive groves entails environmental and social costs; it likely threatens the local economy, rural employment and agroecosystem’s resilience to climate change, as well as to other environmental disturbances, inherently affecting the food production ability in the future.
The abandonment of traditional olive farming practices is clearly linked with its economic trade-offs. The fragmented structure hampers farm competitiveness by escalating production costs [36], while intensive farming pulled down the overall market price making most olive traditional farms barely viable or even unviable [37,38,39]. Notwithstanding the complexity of current socio-economic dynamics, it is very likely that Mediterranean “traditional” olive farmers will continue following one of two main trajectories: (1) leaving the traditional farming practices and moving towards more profitable models, like intensive (200–400 plant/ha) or super-intensive (600–2000 plant/ha) olive farming, or even switching to a more promising and profitable crop, like almonds [23,25,26]; (2) searching for alternative economic opportunities outside the agricultural sector which consequently results in further abandonment of farmlands [40,41].
New approaches have been suggested to farmers in order to make traditional olive groves sustainable and viable again, in the context of the actual highly competitive international markets. Strategies of income diversification recommended to traditional agricultural businesses [42] can equally be implemented to traditional olive farms, such as (1) olive oil differentiation (by shifting from traditional to organic management or by adoption of specific production process and/or quality requirements [39,43,44]), (2) crop diversity (promoting diversification of agricultural production [45,46]), (3) development of non-agricultural products (such as olive oil related agro-tourism activities [47]) and (4) development of new agricultural-related products (innovative value-added by-products [48,49]). For implementation of the promising diversification strategies based on olive oil differentiation or on the development of innovative value-added products, a high quality of palatable products is required; this in turn relies on the olive varieties used, as each variety produces olive fruits with distinct organoleptic proprieties and chemical compositions. It seems thus clear that the sustainability of traditional olive groves in the Mediterranean region is dependent on the knowledge of the olive varieties and its specific properties that confer their uniqueness and authenticity.
Bearing in mind that traditional production is based on centennial olives which were empirically selected by growers centuries ago, the identification of the varieties can no longer be guaranteed with certainty. In this context, an automatic, feasible, low-cost and accurate technique to determine olive varieties is highly valuable for both farmers and authorities (e.g., assisting in the decision-making process of the olive crop management system and of the best value-added product, enabling higher control of the composition of products, or the implementation of certification or labelling processes). At a national/European level, an olive variety identification technique that could be generally applied to a broad area with limited resources would allow for assessing the olive germplasm status and its geographical distribution, being an important tool in landscape management and sustainability.
Currently, as far as we know there is no prompt, effective and feasible technique that can determine olive varieties with reliable accuracy, independent of orchard dimension. Until recent years, variety identification has been based on olive morphological and agronomic traits, classically [50,51,52,53,54] (made difficult by morphological changes raising from the age of the trees, the phenological stage of the plants or even the specificities of the local environmental conditions) or aided by image analyses tools and a semi-automatic algorithm [55,56] (to accommodate variability issues). Pattern recognition through molecular methods has also been achieved [57]. Both approaches are promising in terms of olive variety discrimination, however, they are both not applicable to large olive grove areas as they rely on the collection of individual biological and visual data in the field (which is extremely costly, spatially limited and time-consuming).
Remarkable advances in recent years in satellites’ remote sensors technology, particularly with the launch of satellites with hyperspectral sensors with very-high resolution [58], enable its use to small-scale applications like trees species identification (e.g., [59,60,61]). We hypothesized that the appropriate use of such remote technology could provide feasible and accurate olive variety identification on traditional olive groves. Our objective was to disclose the existence of different patterns in spectral reflectance signatures among olive varieties to support this hypothesis. Therefore, we carried out an intensive data mining classification approach using several machine learning classifiers and leaf spectral reflectance data. We further show that these spectral signatures can be used in a classification process that allows olive variety discrimination.

2. Materials and Methods

2.1. Olive Leaves Spectral Data

Leaf spectral reflectance data of ten representative Iberian olive varieties were collected using a handheld non-imaging spectroradiometer, the FieldSpec® 3 (Analytical Spectral Devices (ASD), Inc., Boulder, CO, USA), coupled with both Plant Probe and Leaf Clip accessories. ASD FieldSpec® 3 portable spectroradiometer consists of three detectors specifically designed to acquire different electromagnetic radiation. One covers the visible (VIS, 350 to 700 nm) and near-infrared (NIR, 701 to 1000 nm) regions of the electromagnetic spectrum with a spectral resolution of 3 nm. The other two cover the short-wave infrared (SWIR1, 1001 to 1830 nm; SWIR2,1831 to 2500 nm) spectral range with a spectral resolution of 10nm (ASD, 2007. FieldSpec® 3 user manual) The Leaf Clip is designed to minimize measurement errors associated with the stray light while using the Plant Probe accessory, which integrates a halogen bulb that emits radiation over the 350 to 2500 nm spectral range.
The measurements were carried out on October 2018 in the National Institute for Agricultural and Veterinary Research (INIAV) experimental station in Elvas, Portugal. One hundred fresh olive leaves were measured (10 random leaves per olive tree) for each of the following olive varieties: Arbequina, Azeiteira, Carrasquenha, Cobrançosa, Cordovil da Serra, Galega, Koroneiki, Picual, Redondil and Verdeal. The exception was Picual, for which only 85 leaves were measured. For each leaf, reflectance value was obtained for each 1 nm interval over 350–2500 nm wavelength range, which means we obtained 2150 reflectance values.

2.2. Machine Learning Classifier Algorithms

The discrimination task of olive leaf spectral information between distinct olive varieties relies on a data mining approach in which several classification supervised learning methods were tested in order to obtain an accurate classification model. Given the strengths and weaknesses of each algorithm and how well they fit both the dataset and classification problem, a set of six classification algorithms was selected for testing, among the most common ones: Classification And Regression Trees (CART) [62], Stochastic Gradient Boosting Machine (GBM) [63,64], Extreme Gradient Boost (XGBoost) [65,66], Random Forest (RF) [67], k-Nearest Neighbor (kNN) [68,69] and Support Vector Machine (SVM) [70,71].
Algorithms with distinct functional principles were included in our set. Decision Tree Algorithms, like CART, construct a model of decisions made based on actual values of attributes in the data. The decisions branch in tree structures until a prediction decision is made for a given record. Instance-based Algorithms, like kNN and SVM, typically build up a database of example data and compare new data to the database using a similarity measure to find the best match and make a prediction. GBM, RF and XGBoost algorithms are Ensemble Algorithms where final models composed of multiple weaker models that are independently trained are used and whose predictions are combined in some way to make the overall prediction. Those require an enormous effort on the selection of types of weak learners to combine and the ways in which to combine them.
The entire analysis was carried out using mlr package [72] implemented in R statistical software which provides an object-oriented and extensible framework for classification for the R language. For each tested algorithm, mlr package implements specific additional packages for the modelling process, namely rpart (CART) [73], class (kNN) [74], e1071 (SVM) [75], gbm (GBM) [76], randomForest (RF) [77] and xgboost (XGBoost) [77].

2.3. Hyperparameters Optimization

Optimization of all classification models involves a hyperparameter-tuning process. For each classifier algorithm tested there is a different set of hyperparameters that should be tuned in order to maximise model predictive accuracy (Table 1).
Hyperparameter-tuning is a truly hardware-demanding and time-consuming process when carried out with most common methods such as Grid Search or Random Search, especially with large parameters spaces. Those methods roam the full space of available parameter values in an isolated way without paying attention to past results. The search space grows exponentially with the number of tuned parameters, while for each hyperparameter combination a model needs to be trained, predictions must be generated in the validation data and the validation metric must be calculated. As a best combination between time consumption and suitability of results, we have chosen to tune the hyperparameters using a Sequential Model-Based Optimization, also known as Bayesian optimization, implemented in the mlrMBO package [78]. Bayesian Optimisation typically requires less iterations to get to the optimal set of hyperparameter values since it selects combinations in an “informed” way, considering past evaluations when choosing the hyperparameter set to evaluate next [79]. This limits the number of times a model is trained for validation since only those settings that are expected to generate a better validation score are passed through for evaluation. The upper and lower limits of search spaces used in mlrMBO for tuning each one of the hyperparameters are presented in Table 1.

2.4. Model Training and Validation

The full dataset is composed of 985 leaves reflectance percentage data for each one of the 2150 1nm intervals between 350–2500 nm, distributed by 10 olive variety classes. For the modelling, we randomly divided the dataset into training and validation sets, stratified by class, containing respectively 80% and 20% of the data. As the name says, the training set was used for training, i.e., to fit parameters of each classifier algorithm tested in order to produce a classification model. The validation set was then used to judge model performance by achieving their predictive accuracy in independent data. Confusion matrices resulting from the validation task were used to provide an estimation of the model’s classification accuracy. Two predictive accuracy assessment measures were computed for each matrix: overall classification accuracy and Kappa coefficient [80].

2.5. Dimensionality Reduction and Performance Improvement

The hyper-dimensionality of our 2150 features dataset constitute a severe constraint to model training and validation processes, with high computational demands. Commonly, feature extraction and dimensional reduction approaches are implemented to lower computational needs and the time spent. Pre-processing a full dataset with principal components analysis (PCA), linear discriminant analysis (LDA), or a sequential combination of both (PCA+LDA) to obtain a set of “most relevant” features to feed machine learning classification algorithms has been showed to efficiently reduce time involved in model training and validation processes [81,82,83], without model accuracy costs. Interestingly, such approaches can even significantly increase the performance of the classification models.
Despite training and validating our models with a full feature dataset, we repeated the modelling process with distinct datasets resulting from the implementation of several dimensional reduction techniques over our original features to attempt enhancing the predictive ability of the classification algorithms. In the first and second cases we respectively used PCA and LDA resulting features. In the third case we used the final features dataset resulting from the PCA+LDA two steps approach, in which we get the linear discriminant features of our original dataset principal components. In addition to the previous and most common approaches, we also proposed and tested an alternative approach, a pairwise fashion dimensional reduction method we called Class-Paired LDA. Such a method implies that linear discriminant features are obtained for each pair of classes—in this case, olive varieties—and then the computation of coefficients was calculated for the entire dataset using those new features. Those final features were the ones used in the modelling procedure. Principal components and linear discriminant features were computed using MASS and psych packages respectively, both implemented with R statistical software. In our data mining approach, 30 final models were produced covering all the combinations of algorithms and dimensionality reduction strategies.

3. Results

The spectral reflectance signatures of the 10 olive varieties are presented in Figure 1. Despite following a similar profile, a careful visual inspection provides evidence for a relative separability of reflectance signatures in several wavelength ranges among most varieties, such as at 780–1300 nm, 1420–1870 nm and 2000–2400 nm (Figure 2). Those discrepancies between classes are likely incorporated in the different models produced by tested algorithms to achieve an accurate classification of olive reflectance.
The hyperparameter optimization process preformed for the different algorithms and datasets resulted in the sequence of settings presented in Table 2.
Of the 30 models generated in our data mining approach. ranges from models with lower predictive accuracy (like the worst performer produced by the kNN algorithm with principal components dataset (PCA) (overall accuracy = 0.2792; Kappa = 0.1983)) to the best performer model (produced by the SVM algorithm with Class-Paired linear discriminants (Class-Paired LDA) (overall accuracy = 0.8173; Kappa = 0.7970))(Table 3). In our olive leaf reflectance classification task with the present dataset. The SVM algorithm always produced the most accurate model regardless of the implementation of any dimensional reduction strategy (Table 3). Similarly. the use of Class-Paired LDA has shown to be the most effective dimensional reduction strategy in terms of improving the model’s performance. since models produced using Class-Paired linear discriminants always achieved the higher overall accuracy regardless of the machine learning algorithm used in the classification task (Table 3).
The best performer model was produced by the SVM algorithm with a dataset of 45 Class-Paired linear discriminants (Table 4) and configured with the following hyperparameter values: cost = 1549.45 and gamma = 4.10 × 10−2. This model struggled specially with the classification of Azeiteira and Cordovil varieties. both with an accuracy of 70%. the lowest accuracy achieved. Azeiteira was misclassified mostly as Cobrançosa. The model handled the classification of Carrasquenha. Picual and Koroneiki extremely well. achieving respectively an accuracy of 95%. 94% and 90% (Table 5). The remaining classes were classified with an acceptable accuracy of 80% (Table 5).

4. Discussion

The diversity of olive cultivars is very high. showing differences in the bioclimatic envelope and limiting factors [84]. and also in plant morphology. traits and phenology (e.g., Bacelar et al. [85]). Despite this. the distinction among olive varieties is not an easy task. The extraordinary accuracy achieved by our best performer model in the classification of most tested olive varieties is a clear substantiation that olive varieties yield distinguishable spectral reflectance patterns that can be used to identify them. Even an accuracy of 70%. the lowest achieved for individual variety classification (Azeiteira and Cordovil). is acceptable considering the challenging task of distinguishing leaf reflectance patterns among very closely related conspecifics originating from a recursive selection-hybridization process among domesticated and existing wild Olea forms [86]. Indeed. several authors have previously found differences between crop cultivars using non-destructive procedures based on spectral data. Gutierrez et al. [87] and Gizaw et al. [88] highlighted the potential of multispectral radiometers to detect differences between wheat genotypes. Silva Júnior et al. [89] used a non-imaging hyperspectral sensor to discriminate four soybean varieties through their spectral profile. Good results have been achieved for discriminating cultivars in permanent crops. A Fourier transform NIR spectrometer was used by Guo et al. [90] to distinguish four peach varieties commonly used in China. Suphamitmongkol et al. [91] also differentiated three varieties of Thai orange through the use of a short-wavelength spectrometer.
A careful analysis of olive leaf reflectance data in the spectral bands range of the satellites integrating three open-data earth observation projects. the Sentinel 2 (Figure 3a). Landsat 8 (Figure 3b) and MODIS-Moderate Resolution Imaging Spectroradiometer (Figure 3c). reveal the relative separability among all olive varieties’ spectral reflectance. At least in one or two of the analyzed spectral bands there was no overlapping between the average standard error bands of each variety in relation to each other. a strong indicator that. in such ranges. varietal information is quite dissimilar. This clearly suggests. in turn. that data produced by these satellites provide the adequate spectral discrimination to support an olive variety identification process. The combination of multi-satellite and/or multi-temporal data obtained by different sensors may further increase the classification results obtained with snapshot single-satellite datasets. since it integrates additional spectral resolution and phenologic differences between olive varieties (in processes like flowering or fructification) that clearly temporarily affect the reflectance captured by satellites [92,93,94]. Concerning spatial resolution limitations. the monovarietal fashion of traditional olives groves and its typical patch size and distribution enable the use of such imagery for variety identification purposes. Particularly. Sentinel 2 data and Landsat 8 data. in which most band information has a spatial resolution that ranges from 10m to 30m. are strong candidates for the task. Even in lower tree density traditional groves. the olive canopy can easily fill the majority of the pixel area since in such orchards trees typically handle a much wider canopy than in higher density olive orchards. Additionally. the coarser spatial resolution information. like that obtained by the MODIS satellite. could be very helpful in increasing classification model accuracy by adding valuable additional information.
These findings support our hypothesis that remote sensing data could be used to identify olive cultivars. Unmanned aerial vehicle (UAV) imagery was used by Avola et al. [95] to distinguish two olive scions using vegetation indexes. Images obtained by UAV were also used by Kyratzis et al. [96] for durum wheat phenotyping in Mediterranean drylands. However. Matese et al. [97] showed that for precision viticulture. the advantage in using UAV is only cost-effective for areas under five hectares and above such threshold airborne and satellite provide better solutions. Thus. for large-scale areas. the solution may involve data provided by different sensors. particularly data from the Sentinel-2 mission with high temporal. spatial and spectral resolutions. and which have already been tested to discriminate and map small-scale crop types (e.g., Griffiths et al. [98]).

4.1. On the Importance of Olive Variety Discrimination

Mapping olive cultivars may be critical for the future of the southern European regions where natural resources are scarce. especially available water. and where large-scale negative impacts resulting from climate change are expected for olive yields [99]. Although the olive tree has stomatal regulation mechanisms to survive in drought conditions [100]. its eco-physiological response to irrigation is very high. particularly in critical moments of its vegetative cycle [101,102,103,104,105,106,107]. This allows for stabilization of the inter-annual variability in olive production which is a marked characteristic of olive trees. The results of Gómez-Rico et al. [106] for the cultivar Olea europaea L. cv. Cornicabra (for virgin olive oil) showed that the production in rain-fed conditions was 35% lower than the one obtained through different irrigation regimes. but that the results achieved based on regulated deficit irrigation were similar to the 100% restitution of the crop evapotranspiration (ETc). Patumi’s et al. [107] findings. obtained in an intensive olive grove with Olea europaea L. cv. Kalamata (for olive oil and table olives). highlight that a restitution of 66% of ETc allows for achieving higher yields. and that the rate of ETc is a threshold above which yield increases were insignificant. The efficient. rational and sustained use of water is therefore mandatory. and deficit irrigation (application of water below the total crop needs defined by ETc) is a potential strategy to reduce excess consumption and to avoid severe and prolonged drought stress in plants. However. given the above-mentioned differences in the phenology of olive cultivars. the optimization of the irrigation scheme is also varietal dependent. highlighting the relevance in knowing the spatial distribution of the different cultivars.
The optimization of management practices resulting from knowledge of the spatial distribution of varieties goes well beyond irrigation issues. Given the differences between olive cultivars in flowering. fruiting and fruit ripening. and also in fruit retention and detachment forces. the spatial distribution of the varieties will enhance mechanical harvest efficiency. guaranteeing better quality fruits and reducing losses [108]. This is even more relevant since fruits ripen earlier in rainfed or poorly irrigated olive trees [109]. and in this way all the optimal adjustment of the irrigation and harvesting schemes depend on the integration of this information since the ripening stage has an important role in olive oil acidity and total phenol content [110]. In referring to pest and diseases. differences in susceptibility are attributed to tree varieties (e.g., [111,112,113]) with a high impact on crop management.
Moreover. the olive varieties also differ in the characteristics of the fruit that limit or enhance multiple uses [114] and in the quality of the olive oil. namely in its stability and chemical composition [115,116]. Since the quality of the final products are an added value for the sector. ensuring its authenticity is imperative. particularly for the extra virgin olive oils [117]. The spatial distribution of cultivars can be one of the phases of a hierarchical process of traceability and authenticity. in addition to genomic approaches [118]. whose usefulness may lie in the identification of fraudulent practices and for varietal and geographical certification.

4.2. Portugal as a Case Study

Olive trees are well adapted to the Mediterranean climate of southern Portugal and have been traditionally cultivated in dryland areas and managed as non-irrigated farming systems. In the mid-1980s the National Plan for Oliviculture was approved. with the aim of restructuring the olive sector by planting new areas and also densifying and/or converting existing ones. Under the reform of the Common Organization of the Market in Oils and Fats (Council Regulation (EC) n.º 1638/98 of 20 July 1998) was established as an incentive for the production of olive oil and. subsequently. the European Commission Decision 2000/406/CE of 9 June 2000 allowed Portugal to expand the area of olive groves by 30,000 hectares.
However. changes at the landscape scale only started to emerge after 2005 [119]. with an increase of 25,000 hectares in the area covered by olive groves between 2005 and 2008 [120]. According to data from the Portuguese Institute for Statistics. olive groves are the permanent crop with the largest area in mainland Portugal. covering 343,557 hectares of agricultural land (and a share of 48%) [121]. About 98% of the Portuguese olive groves are dedicated to the production of olive oil. and only 2% to the production of table olives [119]. The above-mentioned changes were reflected both in the production of olives and olive oil. Portugal is currently the fourth largest European producer of olive oil. with 1.0x106 hl in 2013 and 0.7x106 hl in 2014 [91]. In the period 2000–2007. the average annual production was 390,493.62 hl of olive oil and 252,247.50 ton of olives [121]. and between 2008 and 2014 these values substantially increased to 677,249.14 hl of olive oil and 421,386.42 ton of olives [121]. Most of the Portuguese production of olives and olive oil comes from the Alentejo region (69.58% and 68.55%. respectively; [121]).
This production growth was not only the result of increased area but also due to changes in management. which is presently much more intensive. The tree density range is between 30–173 trees ha−1 in traditional rainfed systems to 1700–3000 trees ha−1 in drip irrigated super-intensive olive orchards [122]. In Portugal. more than 46% of the total area covered by olive groves is irrigated. including 45,000 hectares occupied by intensive [123] and 4000 hectares covered by super-intensive olive groves [124].
However. not all olive cultivars can be managed under this high intensity. This is the reason behind the expansion of imported varieties in Portugal. such as Olea europea cv. Picual and Olea europaea cv. Arbequina. which are fast-growth and high-reach yields [125]. In this conversion process. we are losing varietal composition that contributes to the quality and singularity of the national product. Its unique organoleptic characteristics and crop resilience.
Thus. changes are taking place very quickly. To counterbalance the tendency and mitigate the losses. spatial data is needed to discriminate traditionally managed crops making use of regional well-adapted cultivars. This information is critical for the sustainable management of olive groves in the future and for the guarantee of quality products with high market value.

5. Conclusions

We provided an approach based on distinguishable spectral reflectance patterns that allow low-cost. high-sensitivity identification of olive varieties. It supports the ability of satellite remote sensing that is being used to identify olive varieties in traditional and non-traditional groves in a cost-free fashion. enabling its mapping and monitoring across time. This approach. by identifying their olive tree patrimony and their inherent characteristics and uniqueness. can assist traditional olive farmers in decision-making processes both on crop management strategies and on the best value-added products to invest towards business viability and sustainability. This is a needed first step to counteract the abandonment of traditional olive farming practices and to promote their sustainability. paired with landscape diversification and ecosystem resilience. The optimization of management processes in areas that have undergone land use intensification and the valorization of products derived from endogenous varieties may contribute to a more rational use of the available resources to reduce the negative effects on ecological systems and related functions and services. and to decrease the conflicts between contrasting territorial policies.

Author Contributions

Conceptualization. L.G., T.N. and N.G.; methodology. L.G., T.N. and N.G.; formal analysis. L.G., T.N. and N.G.; funding acquisition. F.R., A.S. and T.N.; writing—original draft preparation. L.G.; writing—review and editing. L.G., T.N. and N.G. All authors have read and agreed to the published version of the manuscript.

Funding

L.G. was supported by the project “Integrated protection of the Alentejo olive grove. Contributions to its innovation and improvement against its key enemies” with the reference ALT20-03-0145-FEDER-000029. co-financed by the European Union through the European Regional Development Fund. under the ALENTEJO 2020 (Regional Operational Program of the Alentejo). T.N. was supported by PTDC/ASP-PLA/30650/2017 (“Fundação para a Ciência e Tecnologia”. FCT Portugal). This work is funded by National Funds through FCT-Foundation for Science and Technology under the Project UIDB/05183/2020.

Acknowledgments

The authors wish to thank João Tiago Marques for the help with R programming and Maria Teresa Carvalho by providing the access to INIAV’s olive cultivar reference collection.

Conflicts of Interest

The author declares no conflict of interest.

References

  1. Pereira, H.M.; Leadley, P.W.; Proença, V.; Alkemade, R.; Scharlemann, J.P.W.; Fernandez-Manjarrés, J.F.; Araújo, M.B.; Balvanera, P.; Biggs, R.; Cheung, W.W.L.; et al. Scenarios for Global Biodiversity in the 21st Century. Science 2010, 330, 1496–1501. [Google Scholar] [CrossRef] [Green Version]
  2. Lambin, E.F.; Meyfroidt, P. Global land use change. economic globalization. and the looming land scarcity. Proc. Natl. Acad. Sci. USA 2011, 108, 3465–3472. [Google Scholar] [CrossRef] [Green Version]
  3. State of the World’s Forests 2016|FAO|Food and Agriculture Organization of the United Nations. Available online: http://www.fao.org/publications/sofo/2016/en/ (accessed on 9 February 2020).
  4. Tilman, D.; Fargione, J.; Wolff, B.; D’Antonio, C.; Dobson, A.; Howarth, R.; Schindler, D.; Schlesinger, W.H.; Simberloff, D.; Swackhamer, D. Forecasting agriculturally driven global environmental change. Science 2001, 292, 281–284. [Google Scholar] [CrossRef] [Green Version]
  5. Izquierdo, A.E.; Grau, H.R. Agriculture adjustment. land-use transition and protected areas in Northwestern Argentina. J. Environ. Manag. 2009, 90, 858–865. [Google Scholar] [CrossRef]
  6. D’Amour, C.B.; Reitsma, F.; Baiocchi, G.; Barthel, S.; Güneralp, B.; Erb, K.H.; Haberl, H.; Creutzig, F.; Seto, K.C. Future urban land expansion and implications for global croplands. Proc. Natl. Acad. Sci. USA 2017, 114, 8939–8944. [Google Scholar] [CrossRef] [Green Version]
  7. Stoate, C.; Báldi, A.; Beja, P.; Boatman, N.D.; Herzon, I.; van Doorn, A.; de Snoo, G.R.; Rakosy, L.; Ramwell, C. Ecological impacts of early 21st century agricultural change in Europe—A review. J. Environ. Manag. 2009, 91, 22–46. [Google Scholar] [CrossRef]
  8. Correia, T.P. Land Abandonment: Changes in the Land Use Patterns around the Mediterranean Basin; CIHEAM-IAMZ: Zaragoza, Spain, 1993. [Google Scholar]
  9. MacDonald, D.; Crabtree, J.R.; Wiesinger, G.; Dax, T.; Stamou, N.; Fleury, P.; Gutierrez Lazpita, J.; Gibon, A. Agricultural abandonment in mountain areas of Europe: Environmental consequences and policy response. J. Environ. Manag. 2000, 59, 47–69. [Google Scholar] [CrossRef]
  10. Navarro, L.M.; Pereira, H.M. Rewilding Abandoned Landscapes in Europe. Ecosystems 2012, 15, 900–912. [Google Scholar] [CrossRef] [Green Version]
  11. Ustaoglu, E.; Collier, M.J. Farmland abandonment in Europe: An overview of drivers. consequences. and assessment of the sustainability implications. Environ. Rev. 2018, 26, 396–416. [Google Scholar] [CrossRef]
  12. Plieninger, T.; Draux, H.; Fagerholm, N.; Bieling, C.; Bürgi, M.; Kizos, T.; Kuemmerle, T.; Primdahl, J.; Verburg, P.H. The driving forces of landscape change in Europe: A systematic review of the evidence. Land Use Policy 2016, 57, 204–214. [Google Scholar] [CrossRef] [Green Version]
  13. Uchida, K.; Ushimaru, A. Land abandonment and intensification diminish spatial and temporal β-diversity of grassland plants and herbivorous insects within paddy terraces. J. Appl. Ecol. 2015, 52, 1033–1043. [Google Scholar] [CrossRef] [Green Version]
  14. Leal Filho, W.; Mandel, M.; Al-Amin, A.Q.; Feher, A.; Chiappetta Jabbour, C.J. An assessment of the causes and consequences of agricultural land abandonment in Europe. Int. J. Sustain. Dev. World Ecol. 2017, 24, 554–560. [Google Scholar] [CrossRef] [Green Version]
  15. Van der Zanden, E.H.; Verburg, P.H.; Schulp, C.J.E.; Verkerk, P.J. Trade-offs of European agricultural abandonment. Land Use Policy 2017, 62, 290–301. [Google Scholar] [CrossRef]
  16. Uchida, K.; Ushimaru, A. Biodiversity declines due to abandonment and intensification of agricultural lands: Patterns and mechanisms. Ecol. Monogr. 2014, 84, 637–658. [Google Scholar] [CrossRef]
  17. Beilin, R.; Lindborg, R.; Stenseke, M.; Pereira, H.M.; Llausàs, A.; Slätmo, E.; Cerqueira, Y.; Navarro, L.; Rodrigues, P.; Reichelt, N.; et al. Analysing how drivers of agricultural land abandonment affect biodiversity and cultural landscapes using case studies from Scandinavia. Iberia and Oceania. Land Use Policy 2014, 36, 60–72. [Google Scholar] [CrossRef]
  18. Plieninger, T.; Höchtl, F.; Spek, T. Traditional land-use and nature conservation in European rural landscapes. Environ. Sci. Policy 2006, 9, 317–321. [Google Scholar] [CrossRef]
  19. Altieri, M.; Nicholls, C.; Montalba, R. Technological Approaches to Sustainable Agriculture at a Crossroads: An Agroecological Perspective. Sustainability 2017, 9, 349. [Google Scholar] [CrossRef] [Green Version]
  20. Schulp, C.J.E.; Levers, C.; Kuemmerle, T.; Tieskens, K.F.; Verburg, P.H. Mapping and modelling past and future land use change in Europe’s cultural landscapes. Land Use Policy 2019, 80, 332–344. [Google Scholar] [CrossRef]
  21. Plieninger, T.; Bieling, C.; Ohnesorge, B.; Schaich, H.; Schleyer, C.; Wolff, F. Exploring futures of ecosystem services in cultural landscapes through participatory scenario development in the Swabian Alb. Germany. Ecol. Soc. 2013, 18, 39. [Google Scholar] [CrossRef] [Green Version]
  22. Benjamin, K.; Bouchard, A.; Domon, G. Abandoned farmlands as components of rural landscapes: An analysis of perceptions and representations. Landsc. Urban. Plan. 2007, 83, 228–244. [Google Scholar] [CrossRef]
  23. Brunori, E.; Salvati, L.; Antogiovanni, A.; Biasi, R. Worrying about ‘Vertical Landscapes’: Terraced Olive Groves and Ecosystem Services in Marginal Land in Central Italy. Sustainability 2018, 10, 1164. [Google Scholar] [CrossRef] [Green Version]
  24. de Graaff, J.; Duarte, F.; Fleskens, L.; de Figueiredo, T. The future of olive groves on sloping land and ex-ante assessment of cross compliance for erosion control. Land Use Policy 2010, 27, 33–41. [Google Scholar] [CrossRef]
  25. Stroosnijder, L.; Mansinho, M.I.; Palese, A.M. OLIVERO: The project analysing the future of olive production systems on sloping land in the Mediterranean basin. J. Environ. Manag. 2008, 89, 75–85. [Google Scholar] [CrossRef]
  26. Urieta, D.; Menor, A.; Caño, S.; Barreal, J.; del MVelasco Gámez, M.; Puentes Poyatos, R. International Olive Growing: Worldwide Analysis and Summary, 1st ed.; Vilar, J., Pereira, J.E., Eds.; Fundación Caja Rural de Jaén: La Carolina, Spain, 2018; ISBN 8494639498. [Google Scholar]
  27. IOC. World Olive Oil Figures—Production; International Olive Council; IOC: Madrid, Spain, 2017. [Google Scholar]
  28. Beaufoy, G.; Beopoulos, N.; Bignal, E.; Dubien, I.; Koumas, D.; Klepacki, B.; Louloudis, L.; Markus, F.; McCracken, D.; Petretti, F.; et al. The Nature of Farming Low Intensity Farming Systems in Nine European Countries; Institute for European Environmental Policy: London, UK, 1994. [Google Scholar]
  29. Fleskens, L.; Duarte, F.; Eicher, I. A conceptual framework for the assessment of multiple functions of agro-ecosystems: A case study of Trás-os-Montes olive groves. J. Rural Stud. 2009, 25, 141–155. [Google Scholar] [CrossRef] [Green Version]
  30. FAO. Globally Important Agricultural Heritage Systems. Combining Agricultural Biodiversity. Resilient Ecosystems. Traditional Farming Practices and Cultural Identity; FAO: Rome, Italy, 2018. [Google Scholar]
  31. Ferrara, V.; Ekblom, A.; Wästfelt, A. Biocultural Heritage in Sicilian Olive Groves; The Importance of Heterogeneous Landscapes over the Long Term. In Reference Module in Earth Systems and Environmental Sciences; Elsevier: Amsterdam, The Netherlands, 2019. [Google Scholar]
  32. Schermer, M.; Stotten, R.; Strasser, U.; Meißl, G.; Marke, T.; Förster, K.; Formayer, H. The Role of Transdisciplinary Research for Agricultural Climate Change Adaptation Strategies. Agronomy 2018, 8, 237. [Google Scholar] [CrossRef] [Green Version]
  33. Veloso, M. Os agroecosistemas tradicionais na conservação da diversidadegenética da oliveira (Olea europaea) em Vila Verde de Ficalho. In OlivalTradicional: Contextos. Realidades e Sustentabilidade; Rota do Guadiana: Serpa, Portugal, 2014; pp. 147–153. ISBN 978-989-98484-1-2. [Google Scholar]
  34. Govindaraj, M.; Vetriventhan, M.; Srinivasan, M. Importance of genetic diversity assessment in crop plants and its recent advances: An overview of its analytical perspectives. Genet. Res. Int. 2015, 2015, 431487. [Google Scholar] [CrossRef] [Green Version]
  35. Dez, C.M.; Trujillo, I.; Barrio, E.; Belaj, A.; Barranco, D.; Rallo, L. Centennial olive trees as a reservoir of genetic diversity. Ann. Bot. 2011, 108, 797–807. [Google Scholar] [CrossRef] [Green Version]
  36. Colombo, S.; Villanueva, M.P. Inefficiency due to parcel fragmentation in olive orchards. Acta Hortic. 2018, 1199, 159–164. [Google Scholar] [CrossRef]
  37. Loumou, A.; Giourga, C. Olive groves: “The life and identity of the Mediterranean”. Agric. Hum. Values 2003, 20, 87–95. [Google Scholar] [CrossRef]
  38. Stillitano, T.; De Luca, A.; Iofrida, N.; Falcone, G.; Spada, E.; Gulisano, G. Economic analysis of olive oil production systems in Southern Italy. Qual.-Access to Success 2017, 18, 107–112. [Google Scholar]
  39. Baptista, A.; Biswas, P. Quality Differentiation as a Strategy for the Viability of Traditional Olive Farming in Trás-os-Montes Region. In Proceedings of the 116th Seminar European Association of Agricultural Economists, Parma, Italy, 27–30 October 2010. [Google Scholar]
  40. Benayas, J.M.R.; Martins, A.; Nicolau, J.M.; Schulz, J.J. Abandonment of agricultural land: An overview of drivers and consequences. CAB Rev. Perspect. Agric. Vet. Sci. Nutr. Nat. Resour. 2007, 2. [Google Scholar] [CrossRef] [Green Version]
  41. Van Meijl, H.; Van Rheenen, T.; Tabeau, A.; Eickhout, B. The impact of different policy environments on agricultural land use in Europe. Agric. Ecosyst. Environ. 2006, 114, 21–38. [Google Scholar] [CrossRef]
  42. FAO. Farm commercialisation and income diversification on the road to EU accession. In Proceedings of the FAO Workshop, Prague, Czech Republic, 2–6 November 2003. [Google Scholar]
  43. Pleguezuelo, C.R.R.; Zuazo, V.H.D.; Martínez, J.R.F.; Peinado, F.J.M.; Martín, F.M.; Tejero, I.F.G. Organic olive farming in Andalusia. Spain. A review. Agron. Sustain. Dev. 2018, 38, 20. [Google Scholar] [CrossRef] [Green Version]
  44. Duarte, F.; Jones, N.; Fleskens, L. Traditional olive orchards on sloping land: Sustainability or abandonment? J. Environ. Manag. 2008, 89, 86–98. [Google Scholar] [CrossRef]
  45. Rosati, A.; Caporali, S.; Paoletti, A. Olive, Asparagus and animals: An agroforestry model for temperate climate in developed countries. In Proceedings of the III OLIVEBIOTEQ (Fora Renovated. Profitable and Competitive Mediterranean Olive Growing Sector), Sfax, Tunisia, 15–19 December 2009; pp. 229–233. [Google Scholar]
  46. Mantovani, D.; Benincasa, P. Rosati A Olive (Olea europea L.) and wild asparagus (Asparagus acutifolius L.) agroforestry system: Asparagus performance and its best positioning in the olive orchard. In Proceedings of the 3rd European Agroforestry Conference, Montpellier, France, 23–25 May 2016; pp. 267–269. [Google Scholar]
  47. Millán, G.; Arjona, J.M.; Amador, L. A new market segment for olive oil: Olive oil tourism in the south of Spain. Agric. Sci. 2014, 5, 179–185. [Google Scholar] [CrossRef] [Green Version]
  48. D’Adamo, I.; Falcone, P.M.; Gastaldi, M.; Morone, P. A Social Analysis of the Olive Oil Sector: The Role of Family Business. Resources 2019, 8, 151. [Google Scholar] [CrossRef] [Green Version]
  49. Pisanelli, A.; Russo, G.; Consalvo, C. New Bio-Products and Innovative Value Chain from Olive Processing: Promote the Bio-Economy of the Olive Oil Value Chain|Agroforestry Innovation Networks Project. Factsheet No.9; Mosquera-Losada., M.R., Ed.; CNR-IRET: Porano, Italy, 2019. [Google Scholar]
  50. Al-Ruqaie, I.; Al-Khalifah, N.S.; Shanavaskhan, A.E. Morphological cladistic analysis of eight popular Olive (Olea europaea L.) cultivars grown in Saudi Arabia using Numerical Taxonomic System for personal computer to detect phyletic relationship and their proximate fruit composition. Saudi J. Biol. Sci. 2016, 23, 115–121. [Google Scholar] [CrossRef] [Green Version]
  51. Rotondi, A.; Magli, M.; Ricciolini, C.; Baldoni, L. Morphological and molecular analyses for the characterization of a group of Italian olive cultivars. Euphytica 2003, 132, 129–137. [Google Scholar] [CrossRef]
  52. Malheiro, R.; Casal, S.; Baptista, P.; Pereira, J.A. Physico-chemical characteristics of olive leaves and fruits and their relation with Bactrocera oleae (Rossi) cultivar oviposition preference. Sci. Hortic. 2015, 194, 208–214. [Google Scholar] [CrossRef] [Green Version]
  53. Trujillo, I.; Ojeda, M.A.; Urdiroz, N.M.; Potter, D.; Barranco, D.; Rallo, L.; Diez, C.M. Identification of the Worldwide Olive Germplasm Bank of Córdoba (Spain) using SSR and morphological markers. Tree Genet. Genomes 2014, 10, 141–155. [Google Scholar] [CrossRef]
  54. Belaj, A.; Satovic, Z.; Rallo, L.; Trujillo, I. Genetic diversity and relationships in olive (Olea europaea L.) germplasm collections as determined by randomly amplified polymorphic DNA. Theor. Appl. Genet. 2002, 105, 638–644. [Google Scholar] [CrossRef] [PubMed]
  55. Blazakis, K.L.B.; Abdelmajid, M.; Bufacchi, M.; Kalaitzis, P. Advanced Mathematical Algorithms to Characterize Olive Varieties through Morphological Parameters. In Proceedings of the International Workshop on Image Analysis Methods for the Plant sciences, Angers, France, 5–6 September 2016. [Google Scholar]
  56. Blazakis, K.N.; Kosma, M.; Kostelenos, G.; Baldoni, L.; Bufacchi, M.; Kalaitzis, P. Description of olive morphological parameters by using open access software. Plant. Methods 2017, 13, 111. [Google Scholar] [CrossRef] [Green Version]
  57. Beiki, A.H.; Saboor, S.; Ebrahimi, M. A New Avenue for Classification and Prediction of Olive Cultivars Using Supervised and Unsupervised Algorithms. PLoS ONE 2012, 7, e44164. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  58. Zhu, L.; Suomalainen, J.; Liu, J.; Hyyppä, J.; Kaartinen, H.; Haggren, H. A Review: Remote Sensing Sensors. In Multi-Purposeful Application of Geospatial Data; InTechOpen: Rijeka, Croatia, 2018. [Google Scholar]
  59. Carleer, A.; Wolff, E. Exploitation of Very High Resolution Satellite Data for Tree Species Identification. Photogramm. Eng. Remote Sens. 2004, 70, 135–140. [Google Scholar] [CrossRef] [Green Version]
  60. Godinho, S.; Guiomar, N.; Gil, A. Estimating tree canopy cover percentage in a mediterranean silvopastoral systems using Sentinel-2A imagery and the stochastic gradient boosting algorithm. Int. J. Remote Sens. 2018, 39, 4640–4662. [Google Scholar] [CrossRef]
  61. Immitzer, M.; Vuolo, F.; Atzberger, C. First Experience with Sentinel-2 Data for Crop and Tree Species Classifications in Central Europe. Remote Sens. 2016, 8, 166. [Google Scholar] [CrossRef]
  62. Breiman, L.; Friedman, J.; Stone, C.J.; Olshen, R.A. Classification and Regression Trees; Chapman & Hall/CRC Press: Boca Raton, FL, USA, 2012; Volume 66, ISBN 9780412048418. [Google Scholar]
  63. Friedman, J.H. Greedy function approximation: A gradient boosting machine. Ann. Stat. 2001, 29, 1189–1232. [Google Scholar] [CrossRef]
  64. Friedman, J.H. Stochastic gradient boosting. Comput. Stat. Data Anal. 2002, 38, 367–378. [Google Scholar] [CrossRef]
  65. Chen, T.; He, T. xgboost: eXtreme Gradient Boosting. Documentation on the R Package ‘xgboost’. Version 0.4-2. 2015. Available online: https://cran.r-project.org/web/packages/xgboost/vignettes/xgboost.pdf (accessed on 1 February 2020).
  66. Chen, T.; Guestrin, C. XGBoost: A scalable tree boosting system. In Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA, 13–17 August 2016; pp. 785–794. [Google Scholar]
  67. Breiman, L. Random forests. Mach. Learn. 2001, 45, 5–32. [Google Scholar] [CrossRef] [Green Version]
  68. Venables, W.N.; Springer, B.D.R. Modern Applied Statistics with S, 4th ed.; Springer: Berlin/Heidelberg, Germany, 2002. [Google Scholar]
  69. Ripley, B.D. Pattern Recognition and Neural Networks; Cambridge University Press: Cambridge, UK, 2014; ISBN 9780511812651. [Google Scholar]
  70. Cortes, C.; Vapnik, V. Support-vector networks. Mach. Learn. 1995, 20, 273–297. [Google Scholar] [CrossRef]
  71. Boser, B.E.; Boser, B.E.; Guyon, I.M.; Vapnik, V.N. A Training Algorithm for Optimal Margin Classifiers. In Proceedings of the 5th Annual ACM Workshop on Computer Learning Theory, Pittsburgh, PA, USA, 27–29 July 1992; pp. 144–152. [Google Scholar]
  72. Bischl, B.; Lang, M.; Kotthoff, L.; Schiffner, J.; Richter, J.; Jones, Z.; Casalicchio, G.; Gallo, M.; Schratz, P.; Bossek, J.; et al. Machine Learning in R. Documentation on the R Package ‘mlr’. Version 2.17.0. 2020. Available online: https://cran.r-project.org/web/packages/mlr/mlr.pdf (accessed on 1 February 2020).
  73. Therneau, T.; Atkinson, B.; Ripley, B. Recursive Partitioning and Regression Trees. Documentation on the R Package ‘Rpart’. Version 4.1–15. 2019. Available online: https://cran.r-project.org/web/packages/rpart/rpart.pdf (accessed on 1 February 2020).
  74. Ripley, B.; Venables, W. Functions for Classification. Documentation on the R Package ‘Class’. Version 7.3–15. 2019. Available online: https://cran.r-project.org/web/packages/class/class.pdf (accessed on 1 February 2020).
  75. Meyer, D.; Dimitriadou, E.; Hornik, K.; Weingessel, A.; Leisch, F.; Chang, C.-C.; Lin, C.-C. Misc Functions of the Department of Statistics. Probability Theory Group (Formerly: E1071). TU Wien. Documentation on the R Package ‘e1071’. Version 1.7-3. 2019. Available online: https://cran.r-project.org/web/packages/e1071/e1071.pdf (accessed on 1 February 2020).
  76. Ridgeway, G. Generalized Boosted Regression Models. Documentation on the R Package ‘Gbm’. Version 2.1.5. 2019. Available online: http://cran.r-project.org/web/packages/gbm/gbm.pdf (accessed on 1 February 2020).
  77. Breiman, L.; Cutler, A.; Liaw, A.; Wiener, M. Breiman and Cutler’s Random Forests for Classification and Regression. Documentation on the R Package ‘randomForest’. version 4.6–14. 2018. Available online: https://cran.r-project.org/web/packages/randomForest/randomForest.pdf (accessed on 1 February 2020).
  78. Bischl, B.; Richter, J.; Bossek, J.; Horn, D.; Lang, M.; Thomas, J. Bayesian Optimization and Model-Based Optimization of Expensive Black-Box Functions. Documentation on the R Package ‘mlrMBO’. Version 1.1.3. 2019. Available online: https://cran.r-project.org/web/packages/mlrMBO/mlrMBO.pdf (accessed on 1 February 2020).
  79. Bischl, B.; Richter, J.; Bossek, J.; Horn, D.; Thomas, J.; Lang, M. mlrMBO: A Modular Framework for Model-Based Optimization of Expensive Black-Box Functions. 2017. Available online: https://arxiv.org/abs/1703.03373 (accessed on 1 February 2020).
  80. Congalton, R.G.; Green, K. Assessing the Accuracy of Remotely Sensed Data; CRC Press: Boca Raton, FL, USA, 2019. [Google Scholar]
  81. Fukunaga, K. Introduction to Statistical Pattern Recognition; Elsevier: Boca Raton, FL, USA, 1990. [Google Scholar]
  82. Pechenizkiy, M.; Tsymbal, A.; Puuronen, S. On combining principal components with fisher’s linear discriminants for supervised learning. Found. Comput. Decis. Sci. 2006, 31, 59–74. [Google Scholar]
  83. Karthikeyan, T.; Thangaraju, P. PCA-NB algorithm to enhance the predictive accuracy. Int. J. Eng. Technol. 2014, 6, 381–387. [Google Scholar]
  84. Barranco, D.; Ruiz, N.; Gómez-Del Campo, M. Frost Tolerance of Eight Olive Cultivars. HortScience 2005, 40: 3, 558–560. [Google Scholar] [CrossRef] [Green Version]
  85. Bacelar, E.A.; Correia, C.M.; Moutinho-Pereira, J.M.; Gonçalves, B.C.; Lopes, J.I.; Torres-Pereira, J.M.G. Sclerophylly and leaf anatomical traits of five field-grown olive cultivars growing under drought conditions. Tree Physiol. 2004, 24, 233–239. [Google Scholar] [CrossRef]
  86. Contento, A.; Ceccarelli, M.; Gelati, M.T.; Maggini, F.; Baldoni, L.; Cionini, P.G. Diversity of Olea genotypes and the origin of cultivated olives. Theor. Appl. Genet. 2002, 104, 1229–1238. [Google Scholar] [CrossRef]
  87. Gutierrez, M.; Reynolds, M.P.; Klatt, A.R. Association of water spectral indices with plant and soil water relations in contrasting wheat genotypes. J. Exp. Bot. 2010, 61, 3291–3303. [Google Scholar] [CrossRef] [Green Version]
  88. Gizaw, S.A.; Garland-Campbell, K.; Carter, A.H. Evaluation of agronomic traits and spectral reflectance in Pacific Northwest winter wheat under rain-fed and irrigated conditions. Field Crops Res. 2016, 196, 168–179. [Google Scholar] [CrossRef] [Green Version]
  89. da Silva Junior, C.A.; Nanni, M.R.; Shakir, M.; Teodoro, P.E.; de Oliveira-Júnior, J.F.; Cezar, E.; de Gois, G.; Lima, M.; Wojciechowski, J.C.; Shiratsuchi, L.S. Soybean varieties discrimination using non-imaging hyperspectral sensor. Infrared Phys. Technol. 2018, 89, 338–350. [Google Scholar] [CrossRef]
  90. Guo, W.; Gu, J.; Liu, D.; Shang, L. Peach variety identification using near-infrared diffuse reflectance spectroscopy. Comput. Electron. Agric. 2016, 123, 297–303. [Google Scholar] [CrossRef]
  91. Suphamitmongkol, W.; Nie, G.; Liu, R.; Kasemsumran, S.; Shi, Y. An alternative approach for the classification of orange varieties based on near infrared spectroscopy. Comput. Electron. Agric. 2016, 91, 87–93. [Google Scholar] [CrossRef]
  92. Vuletin Selak, G.; Goreta Ban, S.; Perica, S. Flowering phenology. flower sterility and pollen germination in olive cultivars. Acta Hortic. 2019, 1231, 11–15. [Google Scholar] [CrossRef]
  93. Joshi, P.K.; Gupta, B.; Roy, P.S. Spectral evaluation of vegetation features using multi-satellite sensor system (Terra ASTER. Landsat ETM+ and IRS 1D LISS III) in man-made and natural landscape. Sens. Rev. 2008, 28, 52–61. [Google Scholar] [CrossRef] [Green Version]
  94. Ghamisi, P.; Rasti, B.; Yokoya, N.; Wang, Q.; Hofle, B.; Bruzzone, L.; Bovolo, F.; Chi, M.; Anders, K.; Gloaguen, R.; et al. Multisource and multitemporal data fusion in remote sensing: A comprehensive review of the state of the art. IEEE Geosci. Remote Sens. Mag. 2019, 7, 6–39. [Google Scholar] [CrossRef] [Green Version]
  95. Avola, G.; Di Gennaro, S.F.; Cantini, C.; Riggi, E.; Muratore, F.; Tornambè, C.; Matese, A. Remotely Sensed Vegetation Indices to Discriminate Field-Grown Olive Cultivars. Remote Sens. 2019, 11, 1242. [Google Scholar] [CrossRef] [Green Version]
  96. Kyratzis, A.C.; Skarlatos, D.P.; Menexes, G.C.; Vamvakousis, V.F.; Katsiotis, A. Assessment of Vegetation Indices Derived by UAV Imagery for Durum Wheat Phenotyping under a Water Limited and Heat Stressed Mediterranean Environment. Front. Plant. Sci. 2017, 8, 1114. [Google Scholar] [CrossRef] [Green Version]
  97. Matese, A.; Toscano, P.; Di Gennaro, S.; Genesio, L.; Vaccari, F.; Primicerio, J.; Belli, C.; Zaldei, A.; Bianconi, R.; Gioli, B. Intercomparison of UAV. Aircraft and Satellite Remote Sensing Platforms for Precision Viticulture. Remote Sens. 2015, 7, 2971–2990. [Google Scholar] [CrossRef] [Green Version]
  98. Griffiths, P.; Nendel, C.; Hostert, P. Intra-annual reflectance composites from Sentinel-2 and Landsat for national-scale crop and land cover mapping. Remote Sens. Environ. 2019, 220, 135–151. [Google Scholar] [CrossRef]
  99. Fraga, H.; Pinto, J.G.; Viola, F.; Santos, J.A. Climate change projections for olive yields in the Mediterranean Basin. Int. J. Climatol. 2020, 40, 769–781. [Google Scholar] [CrossRef] [Green Version]
  100. Fernández, J.-E. Understanding olive adaptation to abiotic stresses as a tool to increase crop performance. Environ. Exp. Bot. 2014, 103, 158–179. [Google Scholar] [CrossRef] [Green Version]
  101. Gonçalves, A.; Silva, E.; Brito, C.; Martins, S.; Pinto, L.; Dinis, L.T.; Luzio, A.; Martins-Gomes, C.; Fernandes-Silva, A.; Ribeiro, C.; et al. Olive tree physiology and chemical composition of fruits are modulated by different deficit irrigation strategies. J. Sci. Food Agric. 2020, 100, 682–694. [Google Scholar] [CrossRef] [Green Version]
  102. García, J.M.; Hueso, A.; Gómez-del-Campo, M. Deficit irrigation during the oil synthesis period affects olive oil quality in high-density orchards (cv. Arbequina). Agric. Water Manag. 2020, 230, 105858. [Google Scholar] [CrossRef]
  103. Jiménez-Herrera, R.; Pacheco-López, B.; Peragón, J. Water stress. irrigation and concentrations of pentacyclic triterpenes and phenols in Olea europaea L. Cv. picual olive trees. Antioxidants 2019, 8, 294. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  104. Palese, A.M.; Nuzzo, V.; Favati, F.; Pietrafesa, A.; Celano, G.; Xiloyannis, C. Effects of water deficit on the vegetative response. yield and oil quality of olive trees (Olea europaea L. cv Coratina) grown under intensive cultivation. Sci. Hortic. 2010, 125, 222–229. [Google Scholar] [CrossRef]
  105. Correa-Tedesco, G.; Rousseaux, M.C.; Searles, P.S. Plant growth and yield responses in olive (Olea europaea) to different irrigation levels in an arid region of Argentina. Agric. Water Manag. 2010, 97, 1829–1837. [Google Scholar] [CrossRef]
  106. Gómez-Rico, A.; Salvador, M.D.; Moriana, A.; Pérez, D.; Olmedilla, N.; Ribas, F.; Fregapane, G. Influence of different irrigation strategies in a traditional Cornicabra cv. olive orchard on virgin olive oil composition and quality. Food Chem. 2007, 100, 568–578. [Google Scholar] [CrossRef]
  107. Patumi, M.; D’Andria, R.; Marsilio, V.; Fontanazza, G.; Morelli, G.; Lanza, B. Olive and olive oil quality after intensive monocone olive growing (Olea europaea L. cv. Kalamata) in different irrigation regimes. Food Chem. 2002, 77, 27–34. [Google Scholar] [CrossRef]
  108. Camposeo, S.; Vivaldi, G.A.; Gattullo, C.E. Ripening indices and harvesting times of different olive cultivars for continuous harvest. Sci. Hortic. 2013, 151, 1–10. [Google Scholar] [CrossRef]
  109. Ramos, A.F.; Santos, F.L. Yield and olive oil characteristics of a low-density orchard (cv. Cordovil) subjected to different irrigation regimes. Agric. Water Manag. 2010, 97, 363–373. [Google Scholar] [CrossRef]
  110. Gutiérrez, F.; Jímenez, B.; Ruíz, A.; Albi, M.A. Effect of olive ripeness on the oxidative stability of virgin olive oil extracted from the varieties picual and hojiblanca and on the different components involved. J. Agric. Food Chem. 1999, 47, 121–127. [Google Scholar] [CrossRef]
  111. Baù, A.; Delbianco, A.; Stancanelli, G.; Tramontini, S. Susceptibility of Olea europaea L. varieties to Xylella fastidiosa subsp. pauca ST53: Systematic literature search up to 24 March 2017. EFSA J. 2017, 15, e04772. [Google Scholar]
  112. Malheiro, R.; Casal, S.; Cunha, S.C.; Baptista, P.; Pereira, J.A. Olive Volatiles from Portuguese Cultivars Cobrançosa. Madural and Verdeal Transmontana: Role in Oviposition Preference of Bactrocera oleae (Rossi) (Diptera: Tephritidae). PLoS ONE 2015, 10, e0125070. [Google Scholar] [CrossRef] [PubMed]
  113. Iannotta, N.; Noce, M.E.; Ripa, V.; Scalercio, S.; Vizzarri, V. Assessment of susceptibility of olive cultivars to the Bactrocera oleae (Gmelin. 1790) and Camarosporium dalmaticum (Thüm.) Zachos & Tzav.-Klon. attacks in Calabria (Southern Italy). J. Environ. Sci. Health B 2007, 42, 789–793. [Google Scholar]
  114. Barranco, D.; Rallo, L. Olive cultivars in Spain. Horttechnology 2000, 10, 107–110. [Google Scholar] [CrossRef]
  115. Beltrán, G.; Jiménez, A.; del Rio, C.; Sánchez, S.; Martínez, L.; Uceda, M.; Aguilera, M.P. Variability of vitamin E in virgin olive oil by agronomical and genetic factors. J. Food Compos. Anal. 2010, 23, 633–639. [Google Scholar] [CrossRef]
  116. Aguilera, M.P.; Beltrán, G.; Ortega, D.; Fernández, A.; Jiménez, A.; Uceda, M. Characterisation of virgin olive oil of Italian olive cultivars: “Frantoio” and “Leccino”. grown in Andalusia. Food Chem. 2005, 89, 387–391. [Google Scholar] [CrossRef]
  117. Rabiei, Z.; Enferadi, S.T. Traceability of origin and authenticity of olive oil. In Olive Oil—Constituents. Quality, Health Properties and Bioconversions; Boskou, D., Ed.; InTechOpen: Rijeka, Croatia, 2012; ISBN 978-953-307-921-9. [Google Scholar]
  118. Jiménez-Morillo, N.T.; Cabrita, M.J.; Dias, C.B.; González-Vila, F.J.; González-Pérez, J.A. Pyrolysis-compound-specific hydrogen isotope analysis (δ2H Py-CSIA) of Mediterranean olive oils. Food Control. 2020, 110. [Google Scholar] [CrossRef]
  119. GPP—Gabinete Planeamento Políticas e Administração Geral SIAZ—Sistema de Informação do Azeite e Azeitona de Mesa. Available online: http://www.gpp.pt/index.php/siaz/siaz-sistema-de-informacao-sobre-o-azeite-e-a-azeitonade-mesa (accessed on 7 March 2017).
  120. Reis, P.; Rolo, J.; Calha, I.; Martins, V. Environmental pressure on intensive olive groves in the Alentejo region (Portugal)–the use of pesticides. In Proceedings of the 120th EAAE Seminar, Chania, Greece, 2–4 September 2010. [Google Scholar]
  121. INE. Estatísticas AGRÍCOLAS do Instituto Nacional de Estatística, 2001 to 2014; INE: Lisboa, Portugal, 2015. [Google Scholar]
  122. Vossen, O. Olive oil: History. production. and characteristics of the world’s classic oils. Hortscience 2007, 42, 1093–1100. [Google Scholar] [CrossRef] [Green Version]
  123. INE. Recenseamento Geral da Agricultura—2009; INE: Lisboa, Portugal, 2009. [Google Scholar]
  124. Cameira, M.R.; Pereira, A.; Ahuja, L.; Ma, L. Sustainability and environmental assessment of fertigation in an intensive olive grove under Mediterranean conditions. Agric. Water Manag. 2014, 146, 346–360. [Google Scholar] [CrossRef]
  125. Silveira, A.; Ferrão, J.; Muñoz-Rojas, J.; Pinto-Correia, T.; Guimarães, M.H.; Schmidt, L. The sustainability of agricultural intensification in the early 21st century: Insights from the olive oil production in Alentejo (Southern Portugal). In Changing Societies: Legacies and Challenges. Vol. iii. The Diverse Worlds of Sustainability; Delicado, A., Domingos, N., de Sousa, L., Eds.; Imprensa de Ciências Sociais da Universidade de Lisboa: Lisboa, Portugal, 2018; pp. 247–275. [Google Scholar]
Figure 1. Leaf reflectance of olive varieties over 1nm intervals (n ≈ 100). Band central lines represent average values by variety and shaded area represents the error envelope (Mean Standard Error).
Figure 1. Leaf reflectance of olive varieties over 1nm intervals (n ≈ 100). Band central lines represent average values by variety and shaded area represents the error envelope (Mean Standard Error).
Sustainability 12 03059 g001
Figure 2. Separability of leaf reflectance signatures among olive varieties in three wavelength ranges of the electromagnetic spectrum, where the visual discrimination between classes are more evident: (a) 700 nm–1400 nm, (b) 1400 nm–1900 nm and (c). 1900 nm–2500 nm. Band central lines represent average values by variety and shaded area represents the error envelope (Mean Standard Error).
Figure 2. Separability of leaf reflectance signatures among olive varieties in three wavelength ranges of the electromagnetic spectrum, where the visual discrimination between classes are more evident: (a) 700 nm–1400 nm, (b) 1400 nm–1900 nm and (c). 1900 nm–2500 nm. Band central lines represent average values by variety and shaded area represents the error envelope (Mean Standard Error).
Sustainability 12 03059 g002
Figure 3. Separability of leaf reflectance signatures among olive varieties in the band range of the satellites Sentinel 2 (a) Landsat 8 (b) and MODIS—Moderate Resolution Imaging Spectroradiometer (c). Reflectance band central lines represent average values by variety and the shaded area represents the error envelope (Mean Standard Error).
Figure 3. Separability of leaf reflectance signatures among olive varieties in the band range of the satellites Sentinel 2 (a) Landsat 8 (b) and MODIS—Moderate Resolution Imaging Spectroradiometer (c). Reflectance band central lines represent average values by variety and the shaded area represents the error envelope (Mean Standard Error).
Sustainability 12 03059 g003aSustainability 12 03059 g003b
Table 1. Hyperparameter’s type and searching spaces used in the tuning process of each classifier algorithm tested.
Table 1. Hyperparameter’s type and searching spaces used in the tuning process of each classifier algorithm tested.
Classifier AlgorithmHyperparameters
IDTypeSearch Space Limits
LowerUpper
CARTcpnumeric0−6.6439
maxdepthinteger330
minbucketinteger550
minsplitinteger550
kNNkinteger1
SVMcostnumeric0
gammanumeric0
GBMn.treesinteger1
interaction depthinteger1
shrinkagenumeric0
n.minobsinnodeinteger1
RFnodesizeinteger1
mtryinteger1
XGBoostnroundsinteger1
maxdepthinteger1
gammanumeric0
colsamples bytreenumeric01
min child weightnumeric0
subsamplenumeric01
Table 2. Hyperparameters selected for the final models in each algorithm for the different datasets.
Table 2. Hyperparameters selected for the final models in each algorithm for the different datasets.
Classifier Algorithm HyperparameterDimensional Reduction Approach
Original FeaturesPCALDAPCA-LDAClass-paired LDA
CART cp3.207201 × 10−39.770812 × 10−46.315712 × 10−21.240644 × 10−35.031421× 10−3
maxdepth2413122527
minbucket553455
minsplit58211422
kNNk5919
SVMcost32741.832743.65108.02613.0505491549.45
gamma3.058368 × 10−51.492811× 10−33.053304 × 10−54.249793 × 10−24.102502 × 10−2
GBMn.trees317264307328153
interaction depth859103
shrinkage0.045505250.16889380.13479960.034508260.1730139
n.minobsinnode18917105
RFnodesize28221
mtry25210527
XGBoostnrounds9610972116757968
maxdepth680.269077446
gamma0.82342881.1566599.3577910.31968651.658614
colsamples bytree0.6119750.64052060.57333290.50857810.4264549
min child weight3.86469512.359230.36326262.4215580.8988217
subsample0.74168910.87730870.3741450.91388810.7521763
Table 3. Preditive accuracy of the models generated in our data mining approach.
Table 3. Preditive accuracy of the models generated in our data mining approach.
Classifier Algorithm Dimensional Reduction ApproachModel Preditive Accuracy
AcuraccyKappa
CART Original features0.35530.2833
PCA0.30460.2267
LDA0.45180.3904
PCA-LDA0.53810.4864
Class-paired LDA0.64970.6109
kNNOriginal features0.37560.3058
PCA0.27920.1983
LDA0.71570.684
PCA-LDA0.63450.5936
Class-paired LDA0.79190.7686
SVMOriginal features0.76650.7403
PCA0.69040.6557
LDA0.68020.6445
PCA-LDA0.68020.6444
Class-paired LDA **0.81730.797
GBMOriginal features0.57360.526
PCA0.48220.4243
LDA0.5330.4806
PCA-LDA0.63960.5993
Class-paired LDA0.77160.7461
RFOriginal features0.54820.4977
PCA0.45180.3902
LDA0.63450.5938
PCA-LDA0.64970.6105
Class-paired LDA0.75630.7292
XGBoostOriginal features0.54820.4978
PCA0.4670.4074
LDA0.47210.4134
PCA-LDA0.67010.6331
Class-paired LDA0.75650.7292
** Best performer model.
Table 4. Relative importance of Class-Paired linear discriminants used with the SVM algorithm to produce the best performer model.
Table 4. Relative importance of Class-Paired linear discriminants used with the SVM algorithm to produce the best performer model.
Linear DiscriminantClassesPermutation Importance
LD45Redondil&Verdeal3.44 × 10−2
LD11Azeiteira&Cobrançosa2.63 × 10−2
LD34Cordovil&Redondil1.18 × 10−2
LD6Arbequina&Koroneiki7.81 × 10−3
LD42Koroneiki&Verdeal7.58 × 10−3
LD31Cordovil&Galega6.77 × 10−3
LD9Arbequina&Verdeal6.54 × 10−3
LD12Azeiteira&Cordovil5.75 × 10−3
LD15Azeiteira&Picual5.22 × 10−3
LD17Azeiteira&Verdeal4.94 × 10−3
LD35Cordovil&Verdeal4.16 × 10−3
LD18Carasquenha&Cobrançosa4.03 × 10−3
LD26Cobrançosa&Galega3.93 × 10−3
LD36Galega&Koroneiki3.78 × 10−3
LD19Carasquenha&Cordovil3.42 × 10−3
LD27Cobrançosa&Koroneiki3.24 × 10−3
LD8Arbequina&Redondil2.66 × 10−3
LD16Azeiteira&Redondil2.66 × 10−3
LD21Carasquenha&Koroneiki2.66 × 10−3
LD2Arbequina&Carasquenha2.64 × 10−3
LD4Arbequina&Cordovil2.59 × 10−3
LD1Arbequina&Azeiteira2.56 × 10−3
LD5Arbequina&Galega2.51 × 10−3
LD38Galega&Redondil2.33 × 10−3
LD29Cobrançosa&Redondil2.26 × 10−3
LD7Arbequina&Picual2.18 × 10−3
LD14Azeiteira&Koroneiki2.15 × 10−3
LD3Arbequina&Cobrançosa2.13 × 10−3
LD30Cobrançosa&Verdeal2.13 × 10−3
LD43Picual&Redondil2.08 × 10−3
LD25Cobrançosa&Cordovil1.67 × 10−3
LD41Koroneiki&Redondil1.50 × 10−3
LD13Azeiteira&Galega1.32 × 10−3
LD28Cobrançosa&Picual1.09 × 10−3
LD23Carasquenha&Redondil9.89 × 10−4
LD10Azeiteira&Carasquenha6.59 × 10−4
LD39Galega&Verdeal5.58 × 10−4
LD20Carasquenha&Galega5.07 × 10−4
LD40Koroneiki&Picual3.80 × 10−4
LD24Carasquenha&Verdeal2.79 × 10−4
LD37Galega&Picual2.53 × 10−4
LD22Carasquenha&Picual1.01 × 10−4
LD33Cordovil&Picual1.01 × 10−4
LD32Cordovil&Koroneiki2.53 × 10−5
LD44Picual&Verdeal0.00 × 10
Table 5. Confusion matrix (in percentage) obtained with SVM algorithm applied to the Class-Paired linear discriminants dataset.
Table 5. Confusion matrix (in percentage) obtained with SVM algorithm applied to the Class-Paired linear discriminants dataset.
Reference Data
Classified DataArbequinaAzeiteiraCarrasquenhaCobrançosaCordovilGalegaKoroneikiPicualRedondilVerdealTotal User’s Accuracy
Arbequina161000010001888.89
Azeiteira014001100222070.00
Carrasquenha001901000002095.00
Cobrançosa040160101002272.73
Cordovil010014100011782.35
Galega100111610002080.00
Koroneiki100000180001994.74
Picual100121016002176.19
Redondil101200001612176.19
Verdeal000010002161984.21
Total20202020202020172020197
Producer’s accuracy80.0070.0095.0080.0070.0080.0090.0094.1280.0080.00
Overall accuracy81.73Kappa0.797

Share and Cite

MDPI and ACS Style

Gomes, L.; Nobre, T.; Sousa, A.; Rei, F.; Guiomar, N. Hyperspectral Reflectance as a Basis to Discriminate Olive Varieties—A Tool for Sustainable Crop Management. Sustainability 2020, 12, 3059. https://0-doi-org.brum.beds.ac.uk/10.3390/su12073059

AMA Style

Gomes L, Nobre T, Sousa A, Rei F, Guiomar N. Hyperspectral Reflectance as a Basis to Discriminate Olive Varieties—A Tool for Sustainable Crop Management. Sustainability. 2020; 12(7):3059. https://0-doi-org.brum.beds.ac.uk/10.3390/su12073059

Chicago/Turabian Style

Gomes, Luis, Tânia Nobre, Adélia Sousa, Fernando Rei, and Nuno Guiomar. 2020. "Hyperspectral Reflectance as a Basis to Discriminate Olive Varieties—A Tool for Sustainable Crop Management" Sustainability 12, no. 7: 3059. https://0-doi-org.brum.beds.ac.uk/10.3390/su12073059

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop