1.

Introduction

An increasing problem in crop production systems across the U.S. Great Plains is the emergence of kochia (Kochia scoparia [L.] Schrad.) biotypes (i.e., strains of a species) resistant to the herbicides glyphosate and dicamba.^1,2 Since 1969, kochia has been recognized as one of the most economically significant weed problems in sugar beets, as kochia densities as low as 0.1 plants ${\mathrm{m}}^{-1}$ of sugar beet row can cause a 10% sugar beet yield reduction.³ Greater kochia densities can cause over 90% yield loss if left uncontrolled season long.⁴ Furthermore, kochia interference can reduce soybean, sorghum, wheat, and corn yields by up to 30%, 38%, 58%, and 40%, respectively.^5–7 Kochia control has become a greater concern for producers, and moreover, has increased cost of production since 2007, when the first instances of kochia populations resistant to the most widely used herbicide glyphosate were documented in Kansas.² Since then, glyphosate-resistant kochia has been reported in 10 states across the US Great Plains and Mountain West, including Montana.² Glyphosate-resistant kochia has become an increasing threat to the sustainability and profitability of dryland and irrigated cropping systems of this region.

Dicamba is an effective herbicide for controlling glyphosate-resistant kochia;⁸ however, occurrence of kochia biotypes with resistance to dicamba makes glyphosate-resistant kochia control even more challenging, and is a concern for cereal and corn producers in Montana and other Great Plains states.^2,9 First documented in wheat fields near Fort Benton, Montana, in 1994,^10,11 the recent spread of dicamba-resistant kochia in six states in the Great Plains region seriously limits the herbicide tools to manage kochia under reduced-tillage systems of this region.^2,9

The current methods for discriminating between herbicide-resistant and susceptible weed biotypes involve whole-plant dose-response assays, shoot bioassays, Petri dish bioassays, target enzyme bioassays, and molecular/genetic markers specific to the resistance trait.^2,12 Susceptible plants show high levels of herbicide-induced stress or death, while resistant plants tend to show only minimal damage. Although these currently used herbicide-resistance screening methods are accurate, they are laborious, cost intensive, and time consuming for confirmation of herbicide-resistant weeds in large production fields.¹² Furthermore, the delay in the process of screening makes it difficult to implement in-season remedial measures to mitigate or prevent further spread of herbicide-resistant weed populations in a field.

Our solution to the early detection of herbicide-resistant weed biotypes is to use hyperspectral remote sensing that will produce two-dimensional (2-D) images of crops and weeds with large numbers of wavelength bins (in our case 240). This is a site-specific weed control technique¹³ designed not only to locate weeds among crop fields, but to also determine the appropriate herbicide(s) to use on each individual plant. The process involves a detailed spectral analysis of different kochia plants in different crops (e.g., wheat, barley, and sugar beet), aimed at differentiating between the herbicide-resistant and susceptible biotypes of the weed. Previous work¹⁴ has shown that hyperspectral imagery can be used to distinguish between crop and weeds in field conditions and that glyphosate-resistant and glyphosate-susceptible Palmer amaranth (Amaranthus palmeri S. Wats.) biotypes can be distinguished using differences in reflectance across the near UV, visible, and near IR spectrum in laboratory conditions.¹⁵ Further work using hyperspectral reflectance profiles has been done to predict the herbicide resistance of Palmer amaranth biotypes,¹⁶ differentiate between glyphosate-susceptible and glyphosate-resistant Italian ryegrass,¹⁷ and to detect the injury on crops as a result of dicamba and glyphosate.¹⁸

Weeds also have been distinguished from crops using shape rather than spectral data,¹⁹ but in our application there are no systematic shape differences between herbicide-resistant and herbicide-susceptible kochia plants. Simple multispectral imaging methods have not yet been studied for this application.

Our research is applying hyperspectral imaging to distinguish dicamba-resistant versus susceptible and glyphosate-resistant versus susceptible kochia biotypes under field conditions among different crops (spring wheat and barley). Seeds were collected for the dicamba-resistant kochia biotype from Fort Benton, Chouteau County, Montana; for glyphosate-resistant kochia biotype from Joplin and Chester, Liberty County, Montana; and for susceptible kochia biotype (susceptible to both dicamba and glyphosate) from Huntley, Yellowstone County, Montana. These biotypes were confirmed resistant or susceptible, and homogenous lines for a resistance trait were developed by recurrent selection over several generations using a previously established method.^20,21

2.

Experimental Methods

2.1.

Data Collection

Data were collected at the Montana State University Southern Agricultural Research Center (SARC) in Huntley, Montana from December 2015 to July 2016. All kochia plants (homogenous for the resistance trait as described previously) used in the experiments were grown in pots containing a commercial potting mix (VermiSoil, Vermicrop Organics, 4265 Duluth Avenue, Rocklin, California) in the greenhouse under identical conditions [$25/23\pm 3°\mathrm{C}$ day/night temperatures, 80% relative humidity (RH), and 16 h photoperiod]. During December 2015, 150 kochia plants were imaged in the greenhouse using a halogen lamp as a light source when the plants attained 15- to 20-cm height. This test population included 50 each of susceptible, glyphosate-resistant, and dicamba-resistant plants. In the April 2016 experiment, we continued imaging plants in the greenhouse. This was also the first instance of imaging greenhouse-grown potted kochia plants (15- to 20-cm height) in the field, alongside crops under diffuse sunlight caused by an overcast sky. In all field experiments, the potted plants were moved outdoors 7 to 10 days before taking the hyperspectral measurements. During the July 2016 data collection in the field, we imaged potted kochia plants (at identical growth stages) among crops, this time in direct sunlight. Hyperspectral images were collected individually for crops, and herbicide-resistant versus herbicide-susceptible kochia, as well as for a combination of various crops and different kochia biotypes (Table 1).

For our experiments, we used a Pika IIg imaging system from Bozeman-based Resonon, Inc. with a Schneider Xenoplan $f/1.4$, 17-mm-focal-length lens. This hyperspectral imager had 240 spectral channels spread across the visible and near-infrared spectrum, from 394.6 to 885.219 nm in 2.053 nm increments. Each raster scan had a vertical height of 640 pixels and a horizontal width set by the scan parameters, with a 2500-pixel maximum under most conditions. The lens had an angular field of view of 0.88 mrad per pixel, giving us 0.56 rad (32 deg) by 2.20 rad (126 deg) angular field of view.

Table 1

The dates, number of images, and lighting conditions of our images taken at the SARC during the different phases of data collection.

Experiment date	No. of images	Location	Illumination type	Solar elevation angle (deg)
December 11, 2015	19	Greenhouse	Direct halogen lamp	n/a
April 13, 2016	31	Crop field	Diffuse solar	30.5 to 53.4
April 14, 2016	28	Greenhouse	Direct halogen lamp	n/a
April 14, 2016	2	Crop field	Diffuse solar	32.38 to 52.9
July 06, 2016	53	Crop field	Direct solar	45.2 to 65.1
July 07, 2016	19	Crop field	Direct solar	48.4 to 60.2

The imager provided us with a series of images with two spatial dimensions and one spectral dimension. Each spectrum was then extracted from its corresponding pixel in the spatial plane. For our outdoor experiments, it should be noted that the data collected were taken during moments when the wind was calm to minimize both spatial and spectral blurring (both occur, as each frame has one spatial and one spectral dimension).

For any image of an entire kochia plant, the camera was placed 2.25 m away from the base of the weeds (resulting in a pixel size of $1.98\times 1.98{\mathrm{mm}}^2$), a distance similar to having the camera look at weeds if mounted on the boom of a tractor-mounted herbicide sprayer. The crops in those images were placed at most 0.25 m from the weeds so both the crops and weeds could be contained within the same image. This resulted in the camera being between 2.0 and 2.5 m from the imaged crops (Figs. 1 and 2). For images of individual leaves, we placed the imager 10 cm away (resulting in a pixel size of $0.088\times 0.088{\mathrm{mm}}^2$) to capture more spatial features and more spectra of each type of plant. Both datasets also can be spatially averaged to reduce resolution to simulate being farther away or to simulate a low-altitude aerial platform. Further research needs to be done to determine the conditions that will aid in developing a viable method for detecting weeds mixed in with crops.

Fig. 1

The hyperspectral imager scanning across the three kochia biotypes (susceptible, glyphosate-resistant, and dicamba-resistant) in a field of wheat at the SARC in Huntley, Montana, on July 6, 2016. The white panel is our Spectralon calibration panel used for all calibration measurements in this study.

Fig. 2

An RGB rendering of an image taken by the hyperspectral camera of potted kochia biotypes among spring wheat (the white panel is for reflectance calibration).

2.2.

Image Processing

Plant species, crops, and weeds are distinguishable by spectral differences.^15,22 Among different biotypes of the same weed species, however, varying levels of enzymes [for instance, overproduction of the 5-enolpyruvylshikimate-3-phosphate (EPSP) synthase enzyme endowing resistance to glyphosate in glyphosate-resistant kochia and Palmer amaranth^20,23] produced by individuals (glyphosate-resistant versus susceptible biotype) can produce spectra that can be discriminated from one another.

We used a Spectralon panel to calibrate the digital numbers in the images to spectral reflectance. During each of the experiments, we began the imaging sequence by recording dark images of a tightly sealed lens cap and calibration images of the Spectralon panel alone, and then followed that with images of plants with the panel included in a small portion of the image. This allowed us to subtract the dark current (electronic sensor noise) from the raw image and then compensate for the spectral variation of the light source and spatial variations of pixel response using an algorithm we developed. This was done both under natural and artificial lighting conditions. This process resulted in reflectance spectra that express the percentage of light reflected as a function of wavelength.

Figure 3 is a sampling of the calibrated data. The spectra are an average and standard deviation of numerous spectra from an area of sunlit leaf of each plant, showing the variations in each plant’s reflectance pixel by pixel. The means and standard deviations shown in Figs. 3 and 4 were derived from 2482, 2509, 224, 160, 160, and 60 individual spectra for spring wheat, barley, susceptible kochia, dicamba-resistant kochia, glyphosate-resistant kochia from Chester, and glyphosate-resistant kochia from Joplin, respectively.

Fig. 3

Means and standard deviations of reflectance spectra for spring wheat and barley with susceptible, dicamba-resistant (Dicamba R), and glyphosate-resistant (Chester, Jopland) biotypes of kochia.

Fig. 4

Means and standard deviations of reflectance spectral gradients for spring wheat and barley with susceptible, dicamba-resistant (Dicamba R), and glyphosate-resistant (Chester, Joplin) biotypes of kochia.

This plot shows the differentiation in the spectral reflectance between different crops and kochia biotypes. Although the spectral differences are subtle and not always fully separable (as seen by the overlapping standard deviations), they are different enough under some conditions to allow differentiation, especially with more sophisticated processing than simple spectral comparison.

Figure 4 shows that spectral gradients (first derivatives) for the same species offer additional information that can provide improved species differentiation. The gradients were generated through Python’s built-in gradient function and were smoothed using the same Savitzky–Golay filter used to analyze the imager’s noise. Specifically, the locations of inflection points in the spectral gradients differed significantly between species, especially in the vicinity of 520-, 580-, 625-, 650-, and 720-nm wavelengths. We also considered using the second derivative profiles as well but rejected them because any signals were largely drowned out by the noise produced by taking the second derivative.

2.3.

Alternate Calibration

To remove the rigorous dependence on absolute radiometric calibration, which can be difficult to achieve in field applications, we also explored methods of processing spectra prior to reflectance calibration. First, a normalized difference vegetation index (NDVI) cutoff of 0.3 was applied to raw measured spectra (prior to reflectance calibration) to identify image pixels that contained vegetation. For this purpose, the NDVI was calculated using wavelengths of 650 and 840 nm and only pixels for which this value was $>0.3$ were classified as vegetation. The three spectra in Fig. 5 are for small regions of pixels that contained wheat, susceptible kochia, and dicamba-resistant kochia; each of these three spectra was divided by the median spectrum calculated from the vegetation-containing pixels. The median at each wavelength was calculated from all the pixels that were identified as vegetation by the NDVI filter. This is similar to the gray world technique, where it is assumed that on average everything is gray. This helps accent the differences in the spectra. Using the median, we assumed that the crop was the majority of these plant pixels. In this case, the crop was now the gray reference very close to 1, with other plants deviating from 1.

Fig. 5

Raw spectra measured for wheat, susceptible kochia (Kochia), and dicamba-resistant kochia (Kochia DR), divided by the median vegetation spectrum. The data between 803 and 842 nm were removed because of atmospheric water vapor absorption.

Next, the three curves shown in Fig. 6 were obtained by subtracting the median vegetation spectrum from the mean of all spectra in the selected regions of wheat, susceptible kochia, and dicamba-resistant kochia (without any assumptions regarding distribution). One of the more notable features shown in this figure is the different wavelengths for the peak near 720 nm in the susceptible and resistant biotypes of kochia. These two methods were used to calibrate the data from the December 2015 (controlled artificial illumination experiments) and April 2016 (outdoor experiments and controlled artificial illumination) datasets alongside the more standard calibration methods discussed previously.

Fig. 6

Difference spectrum obtained by subtracting median vegetation spectrum from raw spectra measured for wheat, susceptible kochia (Kochia), and dicamba-resistant kochia (Kochia DR). The data between 803 and 842 nm were removed due to contamination from atmospheric water vapor.

3.

Spectral Analysis Results

After calibration, we analyzed the spectra using machine-learning algorithms built to automate the process of detecting different weed strains. These required previously classified spectra to train various parameters that were then used to locate weed strains within images. These classifiers were generated using a multidimensional support vector machine (SVM) that used the radial basis function (RBF) kernel on 1000 individual spectra for each crop and weed that needed classification. The following image parameters were used by the SVM:

We also used the differences in each spectrum found using the alternate calibration methods, which sharply drew out the difference in the location of the red edge between resistant and dicamba-susceptible kochia when compared with wheat. The output produced by this algorithm was mapped to show locations of crops and the different biotypes of plants. Figure 7 shows an example of these results. These images were taken on April 14, 2016, under diffuse solar illumination, with seven potted kochia plants placed among spring wheat (15 to 20 cm in height and grown under identical conditions, as described previously). A visual examination of the RGB image reveals that five out of the seven kochia plants were identified properly. Significantly, the dicamba-resistant kochia biotype was differentiated automatically from the susceptible kochia biotype.

Fig. 7

Spectral slope difference spectrum obtained by subtracting median vegetation spectrum from raw spectra measured for wheat, susceptible kochia, and dicamba-resistant kochia. Five out of seven dicamba-resistant plants are correctly identified.

Confusion matrices were used to validate the model we developed to differentiate between the kochia biotypes. The results of these matrices are shown in Fig. 8, revealing that dicamba-resistant, glyphosate-resistant, and susceptible (susceptible to both glyphosate and dicamba) kochia were correctly identified with 67%, 76%, and 80% accuracy, respectively. The confusion matrix technique employed in this figure indicates the fraction of measurements the algorithm predicted correctly or incorrectly. Specifically, the correct prediction rates occur on the diagonal and various incorrect prediction options exist in the off-diagonal positions. Note that the true labels sum to 1.0 along each row. This matrix was created using 1000 spectra for each of dicamba-resistant kochia, glyphosate-resistant kochia, and susceptible kochia, respectively, the same spectra used to generate the SVM used in the analysis.

Fig. 8

(a) A normalized confusion matrix (for 1000 measurements) showing the performance of the algorithm under diffuse solar illumination to distinguish between susceptible (SUS), glyphosate-resistant (GR), and dicamba-resistant (DR) kochia. The darker the blue, the higher the percentage of correctly identified weeds. The percentages of correct identifications of each biotype in (a) match the results shown in Fig. 7. (a) Susceptible kochia versus glyphosate-resistant kochia versus dicamba-resistant kochia, (b) glyphosate-resistant kochia versus dicamba-resistant kochia, (c) susceptible kochia versus dicamba-resistant kochia, and (d) susceptible kochia versus glyphosate-resistant kochia.

Figures 8(b)–8(d) show the accuracies of distinguishing each kochia biotype from each other. As suggested by Fig. 8(a), we have a strong differentiation between dicamba-resistant and glyphosate-resistant kochia and a weaker differentiation between susceptible and glyphosate-resistant kochia. Interestingly, however, the algorithm is able to differentiate between susceptible and dicamba-resistant kochia to a high degree of accuracy.

4.

Discussion

We have data from greenhouse measurements of individual leaves of resistant and susceptible kochia biotypes illuminated by artificial lighting and from outdoor measurements under direct sunlight, but we are still working on algorithms for processing the direct sunlit measurements. Whether the method is more or less effective in direct sunlight needs to be determined later through systematic comparisons. However, anticipating that measurements in direct sunlight will be more difficult because of shadowing, one possibility would be to operate at night using broadband light sources to illuminate the measurement area from two or more directions. It may even be possible to modulate these light sources and use a lock-in detection approach for daytime measurements.

Of the previous studies we have noted, three have specifically applied hyperspectral imaging techniques in precision agriculture.^16–18 While we cannot directly compare the results because we studied kochia and they studied Palmer Amaranth, Italian ryegrass, and general weed identification, respectively, all of these studies suggest promise for using hyperspectral imaging in site-specific weed management. In particular, previous studies showed 91% classification accuracy for Palmer amaranth¹⁶ and an 80% classification accuracy for Italian ryegrass.¹⁷ While our accuracies are lower than those for the Italian ryegrass and for the Palmer amaranth classifications, the results indicate that hyperspectral imaging can be used to identify kochia biotypes, both in the greenhouse and in the field.

5.

Conclusion

This study showed successful identification of dicamba-resistant, glyphosate-resistant, and susceptible (to both glyphosate and dicamba) kochia biotypes with 67%, 76%, and 80% accuracy, respectively, from a dataset of 160 spectra for dicamba-resistant kochia, 220 spectra for glyphosate-resistant kochia, and 224 spectra for susceptible kochia. The spectra were obtained from hyperspectral images measured outdoors with the imager viewing greenhouse-grown potted kochia plants set amid spring wheat. The kochia plants (grown under identical conditions) were imaged at the same growth stage (15- to 20-cm height). The data used in the results shown here were measured outdoors under diffuse lighting conditions. We note that both resistant and susceptible weed biotypes were grown under the same conditions in the greenhouse and moved outdoors in between the crop rows 7 days prior to the hyperspectral measurements.

One way that our identification accuracies may be improved is to first discriminate between dicamba-resistant kochia and dicamba-susceptible kochia, as shown in Fig. 7 and then classify the dicamba-susceptible kochia into the glyphosate-resistant and dicamba- and glyphosate-susceptible biotypes. This approach is also supported by the results of the confusion matrices in Fig. 8. Future studies will also need to carefully take into account the growth stages of the various crops imaged, as well as soil quality and different types of plant stress and analyze how these factors change the spectra of both the crops and weeds.

We also plan to take hyperspectral measurements over a larger selection of herbicide-resistant and herbicide-susceptible kochia biotypes or ecotypes to further quantify the spectral variations between different plants as well as imaging kochia at different growth stages and growing conditions to better understand the accuracies of the detection method. In addition, we plan on imaging kochia plants in the midst of a larger variety of crops to broaden the usefulness of this technique in detecting herbicide-resistant kochia in crop production fields.