Ground-Based Thermal Imaging for Assessing Crop Water Status in Grapevines over a Growing Season

Abstract

The quality of wine grapes in dry climates greatly depends on utilizing optimal amounts of irrigation water during the growing season. Robust and accurate techniques are essential for assessing crop water status in grapevines so that both over-irrigation and excessive water deficits can be avoided. This study proposes a robust strategy to assess crop water status in grapevines. Experiments were performed on Riesling grapevines (Vitis vinfera L.) planted in rows oriented north–south and subjected to three irrigation regimes in a vineyard maintained at an experimental farm in southeastern Washington, USA. Thermal and red–green–blue (RGB) images were acquired during the growing season, using a thermal imaging sensor and digital camera installed on a ground-based platform such that both cameras were oriented orthogonally to the crop canopy. A custom-developed algorithm was created to automatically derive canopy temperature (T_c) and calculate crop water stress index (CWSI) from the acquired thermal-RGB images. The relationship between leaf water potential (Ψ_leaf) and CWSI was investigated. The results revealed that the proposed algorithm combining thermal and RGB images to determine CWSI can be used for assessing crop water status of grapevines. There was a correlation between CWSI and Ψ_leaf with an R-squared value of 0.67 for the measurements in the growing season. It was also found that CWSI from the shaded (east) side of the canopy achieved a better correlation with Ψ_leaf compared to that from the sunlit (west) side around solar noon. The created algorithm allowed real-time assessment of crop water status in commercial vineyards and may be used in decision support systems for grapevine irrigation management.

1. Introduction

Irrigated agriculture is the biggest consumer of freshwater in arid and semi-arid areas, with a share of 70–80% of the total consumption. With the growing water scarcity related to global climate change, increasing the efficiency of water utilization has become a critical issue in irrigated regions. Many vineyards are located in semi-arid areas which require precise regulation of the water supply [1]. Since both the yield and quality of berries are sensitive to changes in water availability to vines [2], precise and robust methods to accurately and precisely detect grapevine water status are becoming increasingly important in commercial vineyards. Traditional measurement approaches, such as pressure chambers, used to measure leaf water potential (Ψ_leaf), are time-consuming and require skilled operators. To address such challenges, interest in the use of infrared thermal imagery for irrigation scheduling has increased with the accessibility of remote sensing technology. Early studies have indicated that stomatal closure related to water stress leads to canopy temperature increases. Since then, canopy temperature has been recognized as an important indicator of crop water status. Infrared thermography has been employed to obtain canopy temperature for assessing grapevine water status since crop water stress index (CWSI) was first proposed in the early 1980s [3]. The CWSI is defined as in Equation (1):

$$CWSI=\frac{T_c-T_{wet}}{T_{dry}-T_{wet}}$$

(1)

where T_c is the average temperature (in °C) of the canopy, T_dry and T_wet represent the reference temperatures corresponding to the dry surface and the wet surface, respectively. A dry surface and wet surface were used to simulate a non-transpiring leaf with closed stomata and fully transpiring leaf with open stomata. However, effective utilization of CWSI for evaluating crop water stress was limited by the accessibility to infrared thermal sensors for a long period. In recent years, development of thermal cameras has provided an opportunity for more robust water stress detection in commercial fields.

The CWSI has been recognized as an effective thermal index for quantifying water stress in plants, and was employed to assess water status for a variety of crops such as grapevine [4,5], cotton [6], and rice [7]. Empirical approaches using artificial references to obtain T_wet and T_dry have been universally used for calculating CWSI [4,8,9]. Artificial references simulating a fully transpiring leaf and a non-transpiring leaf are widely adopted to obtain T_wet and T_dry [10,11,12]. T_wet is determined from a wet artificial reference surface, such as a water-sprayed leaf or damp fabric surface, while T_dry is obtained by measuring the temperature of a dry reference, such as a leaf fully covered with Vaseline on both sides [13]. As natural reference surfaces can be easily disturbed by meteorological factors and the location of reference leaves, an alternative strategy has been developed for the determination of T_wet and T_dry using a canopy temperature histogram. In this statistical calculation of CWSI, T_wet and T_dry correspond to the mean temperatures of the lowest part and highest part in the canopy temperature histogram, respectively [6]. This approach significantly reduces the complexity of CWSI calculations and has been applied to thermal images collected using unmanned aerial vehicles (UAVs) at the plot level [6,10]. UAVs improve the efficiency of data collection by covering a large field in an individual image, providing a platform for crop water status monitoring on an aerial basis. UAV-based thermal imagery has showed a great potential to map variations of crop water status at the field and farm levels [12]. However, thermal information of crops in large plots is squeezed into pixels at the centimeter level, resulting in coarse spatial resolution of individual canopies. Mounting sensors (RGB camera and thermal sensor) on an all-terrain vehicle (ATV) or ground-based platforms can be an alternative for crop water status monitoring to obtain higher spatial resolution at the canopy level. Deficit drip irrigation strategies aim to save water to improve fruit quality [14]. As vineyards are subject to considerable spatial and temporal variation, ground-based thermal imagery for assessing crop water status may provide a means to achieve precision irrigation at the canopy level in vineyards.

In general, canopy temperature is extracted from thermal images to calculate CWSI. Reported research has found that thermal imagery could be an ideal approach for the measurement of canopy temperature in field environments [15,16,17]. Region of interest (ROI) analysis has been utilized to estimate the mean temperature of canopies using ground platform-based images. Different strategies were applied to select ROIs to minimize the influence of background in the thermal imagery of a grapevine canopy. A fully exposed leaf on a given part of a shaded or sunlit canopy was outlined in thermal images, and the temperature of the selected leaf was extracted to represent the mean temperature of the canopy [1]. The middle section of the canopy in the thermal imagery, which was mainly comprised of leaves, was manually selected to determine the average canopy temperature [18]. However, non-vegetation pixels would remain in ROIs without background elimination. A bi-modal histogram of temperatures was used to exclude background (soil) due to the great difference reflection between the canopy and soil in thermal imagery from a zenithal view [19]. Unlike the thermal imagery from the zenithal view—mostly consisting of canopy and soil—sky, soil, and artificial objects are all included in thermal imagery at the canopy level. Determination of the threshold with temperature histograms is limited for thermal images, since there are not obvious boundary values with which to determine the threshold between these non-vegetation objects from the canopy. Thermal imagery, which can only be converted to grey-scale and false color images, is limited by its inability to distinguish canopy from non-green vegetation (sky, soil, artificial objects, trunks, and shoots) pixels. Nonetheless, manual selection of ROIs for the determination of canopy temperature has limited the robust application of thermal imagery to assess crop water stress in commercial fields.

In this study, a canopy-level technique was developed to assess grapevine water status using thermal-RGB images, and its performance was examined by the correlation between CWSI and Ψ_leaf. The main goal of this study was to create a custom algorithm that could be reliably implemented for the segmentation of the grapevine canopy from thermal imagery and for calculating the average canopy temperature and CWSI. Consequently, RGB imagery was employed to distinguish the canopy in thermal imagery. A canopy binary mask was created by removing non-green vegetation pixels from the RGB image using color information. Segmentation of pure canopy pixels from thermal imagery was achieved by registering the binary mask with its corresponding thermal imagery. The average temperature of the canopy in the registered image was calculated to obtain the CWSI, using Equation (1) for crop water status assessment.

2. Material and Methods

2.1. Study Site

The experiment was carried out from mid-July to early-October during the 2019 growing season in a Washington State University research vineyard located near Prosser, Washington, USA. The experimental field is in a semi-arid climate zone, with an average daily temperature of 12.05 °C and mean annual rainfall of 229 mm. Grapevines (Vitis vinifera L.) cv. Riesling were planted in 2010 in rows oriented north–south. The vines were trained to a vertical shoot positioned (VSP) trellis system with the main trellis wire mounted 95 cm above the ground. The inter-vine and inter-row distances were 182 cm and 274 cm, respectively. A drip irrigation system was designed to apply varying levels of water to three different irrigation treatments: full irrigation (FULL, no water stress) as a control, regulated deficit-irrigation (RDI, moderate water stress over time), and partial root-zone drying (PRD, moderate water stress over space). Soil water was replenished before budbreak and after harvest for each treatment to prevent water stress before bloom time and during winter. From budbreak to harvest, FULL vines were irrigated weekly to a level that would cause no water stress (soil moisture of 16%, and midday leaf water potential between −8 to −10 bar of the vines). Each treatment was replicated in four random blocks; each block was comprised of 15 vines.

2.2. Image Acquisition and Field Measurement

Thermal images were acquired with an infrared thermal camera (FLIR Vue Pro R, FLIR Systems, Wilsonville, OR, USA). The thermal camera has a resolution of 640 (horizontal) × 512 (vertical) pixels and a microbolometer sensor with a field of view (FOV) of 69°, a reported accuracy of ±5 °C, frame rate of 30 Hz, and a spectral response wavelength range of 7.5–13.5 μm. RGB images were acquired with a Sony Alpha camera (a6000, Sony Inc., Tokyo, Japan). Both cameras were mounted on a ground-based utility vehicle platform at a height of 156 cm above ground, as shown in Figure 1. Thermal and RGB images were taken within 30 min before and after solar noon in a clear-sky and breeze-less day, since it has been shown that the stomata are essentially closed, and the canopy temperature is at its daily maximum at this time in grapevines that are subjected to a water deficit at this location [20,21].

Weather data (air temperature, soil temperature, wind speed, vapor pressure deficit (VPD), relative humidity) were obtained from an on-site AgWeatherNet station (http://weather.wsu.edu (accessed on 30 September 2019) located 415 m to the north of the vineyard. Canopy temperatures were measured using a handheld thermal sensor. As the reference indicator of crop water status, Ψ_leaf was measured in the center row of each block with a Scholander pressure chamber (Model 615, PMS Instruments Co., Albany, OR, USA) in the field periodically, along with the acquisition of thermal imagery and visible images. Ψ_leaf was measured as described elsewhere [20] on four well-illuminated and fully expanded middle-aged leaves per plot, from shoots near the main trunk per plot. Ψ_leaf was measured on Day of Year (DOY) 218, DOY 225, DOY 232, DOY239, DOY 247, DOY 253, and DOY 267 around solar noon (from 12:30 h to 13:30 h local time).

2.3. Image Processing

Data acquired in the FLIR Systems’ proprietary data format was converted to pixelated temperature data (.csv format) using FLIR Tools. Electronic component aging can cause calibration shift and produce inaccurate temperature measurements by infrared thermal imaging. Calibration of the thermal camera was performed with a blackbody reference source set to different known temperatures ranging from 5 to 65 °C in an incremental step of 5 °C, which covers the range of leaf temperatures in a field environment [21]. Temperatures read by the thermal sensor were captured along with each reference temperature value to obtain the calibration coefficient; then, the coefficient was used to correct the pixelated temperature in thermal imagery. To segment the desired pure canopy regions from the thermal imagery, a custom algorithm was developed and implemented in MATLAB (R2018b, The MathWorks, Natick, MA, USA) using pixelated temperature data and RGB images, which were then used to calculate CWSI.

Image analysis steps followed in the CWSI calculation are shown in Figure 2. First, raw binary thermal data were converted into a matrix (512 × 640) of actual temperature. Calibration coefficients determined using blackbody were applied to the temperature values of each pixel. After resizing and cropping the original RGB images, RGB images were transformed into grayscale images. An intensity-based image registration, nonreflective similarity transformation consisting of translation, rotation, and scale, was employed to align thermal images to the corresponding RGB images. In the next step, binary masks were created using individual RGB images to segment out canopies in thermal images. To create the canopy masks, RGB images were first converted to Hue–Saturation–Intensity (HSI) color space and a threshold in the hue band determined based on trial-and-error was used to remove non-green vegetation pixels, including sky, soil, and artificial objects. Then, RGB images were converted to the color space defined by the International Commission on Illumination (CIE-Lab) for setting the threshold via visual inspection to remove trunks and shoots. Two morphological operations, erosion and dilation, were then used to remove small noisy areas and to fill small unwanted holes in the segmented images. Desired canopy regions in the thermal images were then delineated by multiplying the registered images with corresponding binary masks. Consistent with the location of the ψ_leaf measurements, ROIs used for average temperature calculations were selected from the middle of the overlapped thermal images, which focused on the middle zone of the canopy. A script (in Matlab) was implemented to perform these procedures automatically, which has the potential to improve the simplicity and robustness of thermography in field conditions.

2.4. CWSI Calculation

2.4.1. Determination of Canopy Temperature (T_c)

Canopy temperature has been used as indicator of Ψ_leaf, which indicates crop water status. To reduce the background effects on the calculation of average canopy temperature, ROIs have traditionally been selected by manual inspection, referring to the corresponding visible images [9]. Manual selection of ROIs has impeded the real-time monitoring of crop water stress, and non-green vegetation information cannot be completely eliminated in the ROI. To overcome these limitations, the algorithm proposed in this study allows the automatic selection of an ROI with pure vegetation pixels in the central canopy, improving the applicability of thermal imagery for assessing crop water status in real-time. In this study, T_c is the average temperature of the canopy area within the selected ROI.

2.4.2. Determination of CWSIe and CWSIs

Previous studies have suggested that CWSI could be calculated by empirical and statistical methods, resulting in indices which are referred to as CWSIe (empirical CWSI) and CWSIs (statistical CWSI), respectively [6,22]. T_dry and T_wet in CWSIe are acquired by measuring the temperatures of the wet and dry reference surfaces, while T_dry and T_wet in CWSIs are adaptive approximations of the highest and lowest parts of the canopy temperature histogram. Recently, CWSIs were reported to be able to represent crop water status at the plot level [6,23]. This is based on the knowledge that the variance of leaf temperature is directly related to crop water stress [24]. In this study, CWSIs and CWSIe were correlated with field measurements of Ψ_leaf to identify the CWSI type that better represents the crop water status at the canopy level. T_dry and T_wet for CWSIe calculation were determined using artificial references during the period of imaging acquisition; a green sponge soaked in water was used as wet reference, and dry wood bark was used as a dry reference. As suggested by Bian et al. [6], T_dry and T_wet for CWSI calculation were the mean of the highest 5% and the lowest 5% of canopy temperature in the histogram. With the determination of T_c, T_dry and T_wet, CWSI was calculated using Equation (1).

With the aim of improving the efficiency of using infrared thermal imagery for crop water status assessment, correlations between thermal indices and Ψ_leaf were performed. T_c and CWSI derived from thermal imagery for the shaded and sunlit sides of the canopy were correlated with Ψ_leaf. Since vines in RDI and PRD treatments presented similar trends of Tc, as expected, samples from FULL and RDI were used for the correlation between Ψ_leaf and CWSI. Ψ_leaf, T_c and CWSI are the average values of the consecutive vines in each block. Ψ_leaf, T_c and CWSI acquired in cloudy and windy conditions were eliminated. A total of 42 samples of Ψ_leaf, T_c and CWSI were used in the correlation. Artificial references and statistical references were used to determine T_dry and T_wet for CWSIe and CWSIs calculation. Linear correlations between CWSIe and Ψ_leaf, and CWSIs and Ψ_leaf were then assessed and compared.

3. Results and Discussion

3.1. ROI Identification

Non-green vegetation pixels (sky, soil, artificial objects, trunks, and shoots) and shaded areas of the canopy were removed from thermal images using the proposed algorithm. The canopy segmentation results are shown in Figure 3, including the false color thermal imaging (each pixel is expressed in °C; Figure 3a), corresponding original RGB image (Figure 3b), the binary mask created using RGB imaging (Figure 3c), and the false color thermal image after removal of non-green vegetation pixels (Figure 3d). The rectangular ROI was set in the canopy region of the masked thermal image (Figure 3d) for further analysis and CWSI calculation.

Average canopy temperature and CWSI were determined using the proposed image processing algorithm. Canopy temperatures in the same ROIs were manually extracted from thermal imaging, with visual inspection to validate the T_c determination. The performance of the proposed algorithm for average T_c calculation is shown in Figure 4. Linear regression of calculated T_c versus measured T_c resulted in correlation coefficients 0.98 for both the sunlit and shaded sides of vine canopies. Root mean square errors (RMSE) of the calculated T_c for the sunlit and shaded sides were 0.60 and 0.56, respectively. The linear regression versus measured T_c illustrates that the T_c obtained with the proposed algorithm could accurately represent the actual average canopy temperature of grapevines. Temperature histograms of the original thermal image and the masked thermal image are illustrated in Figure 5. The average temperatures of the original thermal imagery prior to removal of non-green vegetation pixels and of the processed thermal imagery were 36.5 °C and 30.6 °C, respectively. Pixels representing non-green vegetation parts of the canopy exhibited higher (soil and artificial objects) or lower (sky) temperatures in thermal imagery. Since the non-green vegetation pixels were eliminated from the thermal imagery, the temperature distribution of the masked thermal image was found to be more concentrated around the actual canopy temperature than in the original thermal image. As solar radiation heats exposed tissues, the canopy temperature is typically higher during the day than the surrounding air temperature, despite the evaporative cooling effect provided by transpiration [20].

3.2. Variation of Ψ_leaf and CWSI

Changes of Ψ_leaf, T_c and CWSIe from DOY 218 to 267 for three irrigation regimes are illustrated in Figure 6. The Ψ_leaf, T_c and CWSIe shown in Figure 6 are the average values of the samples for each treatment. Ψ_leaf of FULL and PRD vines exhibited similar behaviors, as shown in the curves. This outcome was expected since the alternating wet sides of the root zone in PRD maintains plant water status [25]. The variation of Ψ_leaf over the entire season depended on the irrigation schedule. Three treatments had higher values on DOY 225 and DOY 267, when the vineyard was being irrigated. The deficit irrigation of RDI treatment was applied from the first irrigation (DOY 119) until DOY 239, when it was changed to the same irrigation amount as the FULL treatments. Correspondingly, RDI vines were more stressed from DOY 218 to DOY 239, and exhibited the same variation pattern as FULL and PRD vines after DOY 239. Identical trends were not noticed between the CWSIe and T_c curves, because T_wet and T_dry were used to calculate CWSIe for each measurement to reduce the disturbance due to environmental conditions.

Generally, CWSIe showed an inverse pattern from Ψ_leaf. CWSIe of all three regimes reached the lowest values on DOY 225 and DOY 267, while Ψ_leaf had the highest values on the same days. However, T_c was not at the lowest point on those days, confirming that the wet and dry references obtained along with the imaging acquisition were appropriate for crop water stress detection. Similar findings were also reported by Idso et al. [26], who stated that different references should be employed for different plant growth stages. The need for better correlation with multiple wet and dry references is because different references are more efficient at reducing environmental disturbances on CWSI calculations under different weather conditions and growing stages.

3.3. Relationship between Ψ_leaf and CWSI

Correlations between T_c and Ψ_leaf for both the shaded and sunlit sides of the canopy are shown in Figure 7. T_c and Ψ_leaf were negatively correlated for both sides, indicating a higher T_c with more severe water stress. This result is expected, as the stomatal closure induced by water stress reduces the transpiration rate, thus lowering evaporative cooling and increasing the leaf temperature [27]. The coefficient of determination (R²) for the shaded and sunlit sides was 0.55 and 0.22, respectively. In addition, the slope of the linear correlation equations y = ax + b for the shaded and sunlit sides was 0.24 and −0.17, respectively. Consequently, a stronger correlation was obtained from the shaded side of the canopy than from the sunlit side.

Correlations between CWSIe and Ψ_leaf for the shaded and sunlit sides of the canopy are shown in Figure 8. To reduce the impact of weather conditions (e.g., windy, slightly cloudy) and growth stages, different artificial references were used for the CWSI calculation. The shaded and sunlit sides showed similar relationships between CWSIe and Ψ_leaf. The R² for the shaded and sunlit sides was 0.67 and 0.48, respectively, and the linear equations shared similar slopes a (−3.8466 and −4.3973, respectively). Similar to T_c, the CWSIe for the shaded side of the canopy correlated better with Ψ_leaf than did that for the sunlit side. Figure 9 shows the correlation plots between CWSIs and Ψ_leaf for the shaded and sunlit sides of the canopy. Neither correlation was significant, demonstrating that CWSIs is not an accurate representative of grapevine water status in this study.

The results of the correlation analysis demonstrated that CWSI and T_c were significantly correlated with grapevine water status as represented by Ψ_leaf. These results are in line with previous studies on grapevine subjected to deficit irrigation [1,28]. A higher R² was obtained between thermal indices and Ψ_leaf measured near solar noon on the shaded side of the canopy than on the sunlit side. These results are in agreement with those presented by Gutiérrez et al. [18]. Concerning the vineyard with north–south row orientation, the east side was gradually shaded after a whole morning of direct sunlight, resulting in larger acclimation to sun exposure for leaves in the shaded side. Hence, the shaded (east)-side leaves showed more sensitivity to differences in crop water stress than the sunlit (west) side at solar noon.

It was found that CWSIs was not significantly correlated with Ψ_leaf for measurements on both sides of the canopy. Previous studies using the highest and lowest parts of the temperature histogram to calculate CWSI were conducted with images collected from UAVs at the plot level [6,10]. Plots of full irrigation and deficit irrigation treatments were included in UAV-based thermal imaging; therefore, the means of the lowest and highest temperatures in the histogram might simulate the maximal and minimal leaf transpiration. The transpiration of grapevine leaves has been found to increase linearly or even exponentially with increasing leaf temperature [2]. In this study, infrared thermal images were collected at the canopy level. T_wet and T_dry for CWSIs calculation were extracted from the thermal images of the vines (FULL, RDI and PRD treatments), and these reference surfaces were greatly dependent on the T_c of an individual canopy, and therefore varied significantly among vines. The main reason for the weak correlation between CWSIs and Ψ_leaf could be that wet and dry references obtained at the canopy level cannot precisely represent a non-transpiring leaf and a fully transpiring leaf. T_dry extracted from fully irrigated vines was lower than the actual temperature of the dry reference, while T_wet derived from vines with no to minimal water stress was higher than the actual temperature of the wet reference.

4. Conclusions

Water deficit estimation at the individual canopy level could provide a powerful tool for precision water management in grape production. This study created and tested a non-invasive strategy to assess crop water status for grapevine using thermal-RGB images. It included the elimination of non-green vegetation pixels (including sky, soil, artificial objects, trunks, and shoots) from thermal imagery by multiplying a binary mask with the corresponding RGB image to determine the average temperature of the pure canopy pixels using CWSI calculations. Both calculated CWSIe and CWSIs from shaded and sunlit sides were reasonably correlated with Ψ_leaf. A correlation (R² = 0.67) was found between CWSIe from the shaded side and Ψ_leaf. The obtained verification results indicated that thermal imagery could provide a non-invasive tool for assessing crop water status at the canopy level. Despite relating to the canopy temperature, crop water status was found to be affected by many other environmental parameters, such as air temperature, wind speed, relative humidity, etc. Further studies would be needed to improve the performance of this sensing technology using advanced data processing methods. As deep learning has gained much attention for capturing the nonlinear relationship between environmental and crop physiological parameters, the authors would like to suggest exploring the possibility of creating deep learning-based methods for analyzing thermal imagery data to obtain faster and more reliable detection of crop water status in natural field environments.