Introduction

Corn (Zea mays L.) is a major cereal crop that accounts for more than 95% of total feed grain production in the United States (US) and provides food, feed, and fuel for people and livestock worldwide1. The importance of corn lies not only in its high yield potential but also in its diverse uses, including as a source of starch, sweeteners, biofuels, and industrial products2,3. Corn is a staple food for many people, especially in developing countries, where it provides a significant portion of the daily calorie intake. In addition, corn is a vital component of animal feed, providing essential nutrients for livestock such as cattle, pigs, and poultry4. Moreover, corn has a significant role in the biofuel industry, where it is converted to ethanol and other biofuels5. This renewable energy source contributes to reducing greenhouse gas emissions and decreasing dependence on fossil fuels6. Corn cultivation also has numerous environmental benefits, such as reducing soil erosion and improving soil quality7. Given the importance of corn in food, feed, fuel, and industrial production, the sustainable and efficient management of corn cultivation is crucial for ensuring global food security, reducing greenhouse gas emissions, and promoting sustainable agriculture.

The current corn production system uses excessive amount of chemical herbicide for weed control. About 1 billion pounds of conventional pesticides are used each year to control undesirable vegetation such as weeds in the agricultural production system of the United States (U.S.), which amount to a cost of almost $9 billion8,9. One of the major reasons for excessive use of chemicals in agriculture is the conventional method of spraying herbicides for weed control, often called a blanket application, which accounts for 59% of the major pesticide expenditures in economic crops in North America10. The blanket application implies spraying herbicide uniformly across the field without considering the spatial distribution of the weeds, which often causes overuse of chemical herbicides.

Chemical herbicides are widely used in modern agriculture to control weeds, protect crop yields, and improve farming efficiency. Herbicides reduce competition for resources such as water, nutrients, and sunlight, promoting crop growth and leading to higher yields11. Additionally, they can prevent the spread of diseases and pests that may harm crops, thus improving food quality and safety12. However, overreliance on herbicides can have negative environmental consequences, such as contamination of soil and water resources and the development of herbicide-resistant weeds. In addition to being harmful to the environment13,14, chemical herbicides can be harmful to human health as they have been linked to cancer15, attention deficit hyperactivity disorder (ADHD)16, alzheimer’s17, DNA damage, cardiovascular diseases, neurological disorders, and reproductive disorders18. Moreover, excessive chemical use in agriculture has the potential to harm nervous system, endocrine system, and reproductive system of human19. Furthermore, excessive use of chemicals in agriculture have become one of the major reason for surface and groundwater pollution20,21,22,23, becoming a threat to the ecosystem and public health. Although organic farming is seen by many as a potential solution for the excessive use of chemicals in agriculture, it is not a silver bullet to meet the United Nations sustainable development goals by 203024. Therefore, to maintain environmental sustainability and human health, agricultural production systems are in dire need of technologies that can significantly reduce the amount of chemicals currently being used, without compromising crop yields.

An alternative way to neglect chemical use for weed control in agricultural production system is through physically removing weeds by using robotics platforms. Some of the latest automated robots such as the autonomous weed robot by “ecoRobotix” (ecoRobotix, Vaud, Switzerland) and “Deepfield Robotics” (Deepfield Robotics, Renningen, Germany), use ground-based machine vision and image processing techniques to perform site-specific weed control (SSWC) in row crops25. However, these robotic solutions are not effective to be deployed in a large commercial agricultural field (hundreds or thousands of acres). To overcome that issue, some companies have developed and commercialized solutions such as “Weedseeker\(\circledR\)” (Trimble agriculture, California, USA), and “WEED-it” (WEED-IT, Steenderen, Netherlands), which use optoelectronic sensors to measure the reflection intensity of vegetation, allowing them to discriminate vegetation from soil background26. The shortfall of those products, at the time of this writing, is that they cannot discern weeds from crops early in growing to allow such technologies to be used for weed control. Recently, John Deere (Illinois, USA) launched See & Spray Ultimate\(\circledR\), a sprayer that enables detection and control of weeds during the growing season in corn, soybean, and cotton. More details regarding the field performance of that technology remain to be seen since the sprayers fitted with the system will be available only in 202327.

Another approach of performing SSWC is possible through the use of remote sensing technology to accurately map the weed distribution information across a field and integrate that with a spraying platform28. The main idea here is to spray herbicide only to those areas where weeds are present29. To accomplish that, one would need a high-resolution imagery to discern plants growing in the field as weeds and crop. Since satellite imagery would present some challenges to discern weeds from crops, due to its limitations in providing adequate spatial resolution30, the use of UAS with high-resolution cameras seems to be a promising platform, which can be processed and analyzed to obtain an accurate weed distribution map from a field31. For that approach to work, is imperative to detect weeds accurately early in the growing season. In most crop weed management programs, weed treatment is recommended at the early growth stages of both crops and weeds. Mapping weeds at that stage can be a challenging task because of four main reasons: (1) weeds are not uniformly distributed across the field, which necessitates working at a single pixel size on the image32, (2) crop and weeds have the same or similar reflectance properties, (3) interference of soil background33, and (4) very small size weeds which might not captured by camera34.

One of the earliest studies related to SSWC using UAS35 reported about the possibility of the use of UAS imagery and imagery analysis techniques to perform accurate discrimination between weeds and sunflower at early growth stages, which is key to implement any SSWC. More recently, the use of UAS in agriculture for SSWC have become a topic of several publications in specialized literature related to agriculture36,37,38,39,40,41,42,43,44. Around the same time, more sophisticated and powerful image analysis algorithms, such as machine learning (ML) and deep learning (DL) have being developed and adapted for agricultural applications, and they have gained space as effective ways to process large amounts of data in agriculture45,46.

Another study conducted by Peña et al.37 developed an object-based image analysis (OBIA) procedure to generate a weed map in an experimental maize field in Spain using a six-band multispectral camera. The OBIA procedure classified crop rows, discriminated crops and weeds based on their relative positions to crop rows, and generated a weed infestation map. The study achieved an accuracy of 86% in generating weed map, however, the study was limited to virtual simulation, and further field implementation is necessary to validate its effectiveness. Likewise, Hunter et al.47 investigated the efficiency and efficacy of an unmanned aerial vehicle integrated system (UAV-IS) for weed management compared to ground-based broadcast applications. The UAV-IS was found to be more efficient at identifying and treating target weedy areas while minimizing treatment on non-weedy areas. However, it treated 20–60% less area than the ground-based applications and missed up to 26% of the target weedy area. The study concludes that UAV-IS has the potential to improve integrated pest management programs and reduce herbicide resistance, but battery replacement issues need to be addressed for commercial farming use. Similarly, a recent study by Jensen et al.48 presented an approach for automated site-specific fallow weed management using unmanned aerial vehicles (UAVs) and precision spray technology. The system employs machine vision and GIS technology to detect and map green weeds, which are subsequently targeted for precision herbicide application. Although field evaluation of the system showed some herbicide savings, the accuracy varied for different weed species, and some species were even not identified with good accuracy. This suggests that the system may not be suitable for managing all weed species in fallow fields. Future work is needed to optimize the system’s efficiency and commercial viability.

Identification of crop rows in an UAS imagery using ML and DL have become another subject for several recent studies49,50,51,52,53,54,55,56,57,58. Although, most of these studies have been able to identify crop rows, and discriminate between weeds and crops during the early stage, they were limited to virtual simulations only. Most of these reports regarding the use of UAS imagery for SSWC fall short of providing a real word solution for SSWC, since most of them just report on the capability of different algorithms to separate weeds from crops. Moreover, most of these studies regarding crop row detection, were able to identify the rows as lines with great accuracy51,52,53,54,55,56,57,58, however, the authors could not provide any insights on how to use those detected crop rows in practical agriculture. Additionally, most of the reports are performed in a small area of agricultural land which would probably raise questions about the research validity in bigger agricultural area.

Corn is a major crop for the state of North Dakota (ND), US. In 2020, ND farmers grew corn in 1.5 million hectares of corn in ND59. Weed competition is a major cause of corn yield losses in ND, and chemical herbicide is the most widely used option for weed control60. In order to optimize weed control efficacy, the use of pre-and post-emergence chemical herbicide treatments has been proven to be the most effective weed control strategy61,62. At present, most of the farmers apply a blanket application of herbicides for post-emergence weed control in corn, with no consideration regarding the spatial distribution of weeds across the field. That application often results in an overuse of chemical herbicides. The cost of post-emergence herbicide application (product + application) in ND is usually around $8-12/ha (2020 growing season). If one could use UAS imagery to generate a weed distribution map, implement SSWC, and apply chemicals to only those areas where weeds are present in the field, that could lead to significant savings on chemicals for corn growers. Using the area grown with corn in 2020 in ND and the cost associated with the chemical application, those savings could reach as much as $5.5–7.5 million, with as little as a 10% reduction in applied acreage. In addition to the financial savings, there are environmental savings, which are harder to put a number to it, that can be realized using applying fewer chemicals to the land.

This research is embodying the following contributions, as no previous study has reported on the whole workflow, from image collection to weed control application in the field, to implement SSWC in corn by integrating UAS imagery with a commercial-size sprayer:

  • Development and validation of a Crop Row Identification (CRI) algorithm for UAS-based imagery to accurately identify corn rows.

  • Mapping of post-emergent weed infestation in the corn field and creation of a grid-based weed prescription map for site-specific weed control.

  • Integration and field implementation of the prescription map through a commercial-size sprayer.

  • To compare the amount of chemical herbicide applied using our proposed approach (SSWC) with a traditional herbicide application method.

Methods

This study can be divided into four phases (Fig. 1); (1) acquiring and preprocessing UAS data, (2) developing a computer vision algorithm for corn row classification, (3) creating a weed prescription map and (4) field implementation of the prescription map through a commercial-size sprayer.

Figure 1
figure 1

Workflow diagram followed to implement a site-specific weed control approach in a corn field.

Full size image

Study site

The study was carried out in a corn field of around 17 hectares (ha), located at Carrington, North Dakota \(\left( 47.51^{\circ }\,\textrm{N}, 99.12^{\circ }\, \textrm{W}\right)\). The experimental area for this study was provided by North Dakota State University (NDSU) Carrington Research Extension Center (REC). According to the Web Soil Survey (WSS), the soil in the experimental field was composed of Heimdall and similar soil (42%), Emrick and similar soil (37%), and minor components 21%. The field was planted with silage corn on May 12, 2021, with a 30-inch row spacing. Prior to planting, on May 7, the field received a pre-emergence herbicide treatment (Verdict63, (0.98 kg/ha), which was applied at a rate 140.3 L/ha.

Data collection

The UAS flights for image collection were carried out on June 14, 2021. A DJI Matrice 600 Pro (M600) (Fig. 2a)(DJI, Shenzhen, China), outfitted with a Sony Alpha 7R II 42 Megapixel RGB camera (Sony City, Tokyo, Japan) was used to capture the aerial images of the field (Fig. 2d). The camera has a 7952 \(\times\) 5304 pixels (42.4 megapixels) sensor resolution, and a focal length of 35mm. Integration unit for the camera on the drone was made by FieldofView LLC (Fargo, North Dakota, USA), which manufactures a device called GeoSnap PPK (Fig. 2b) that allows one to trigger the camera and geotag the images with PPK (post-processed kinematic) accuracy (2 cm). Since distance between the research field and nearest CORS (Continuously Operating Reference Stations; Cooperstown, ND) was too far, which would cause a degradation of the geotag accuracy, an iG4 GNSS (Global navigation satellite system) base station (iGage Mapping Corporation, Salt Lake City, USA) (Fig. 2c) was used to implement the PPK correction to the images’ geotags. The UAS was flown autonomously using flight missions created in Pix4DCapture app (IOS version) (PIX4D, Prilly, Switzerland). Flights were carried out at 107 meters approximately above ground level (AGL), with the camera at nadir position, with 75% overlap both front and side, and the UAS speed was adjusted (by the app) to allow 1.4 seconds interval between pictures. Altogether, three flights were carried to cover the experimental field, collecting a total of 2251 images.

Figure 2
figure 2

(a) DJI Matrice 600 Pro with a Sony Alpha 7R II, 42.4-megapixel RGB camera and Geosnap PPK system (top) mounted to it; (b) detailed view of Geosnap PPK system mounted on top of the aircraft; (c) iG4 GNSS base station that was kept in ground for stationary position; and (d) Corn field orthomosaic showing the placement of the SSWC (green) and no-SSWC (pink salmon) treatment plots in the experimental area.

Full size image

All the data processing in this study was performed in a desktop computer that have Windows 10 Pro, version 20H2, a 64-bit operating system with 132 gigabytes of RAM. The desktop was provided with two switchable GPUs. One was Intel(R) UHD Graphics 630, and the other was NVIDIA GeForce RTX 2080 SUPER.

Image preprocessing and stitching

In order to generate accurate (2 cm accuracy for latitude and longitude, and twice that for elevation) geotag information for each image, the information collected by the GNSS base station and GeoSnap device were processed using EzSurv software (Effigis, Montreal, Canada), which provided an output as “.csv” file with post-corrected geotag information for each picture. That file was then used during the images stitching process. Images were processed into an orthomosaic (Fig. 2d) using Pix4DMapper (Pix4D, Prilly, Switzerland). It took 6 h and 25 min for the software to generate the output as orthomosaic, which had an average ground sample distance (GSD) of 0.63 cm/pixel. That orthomosaic served as basis for all the subsequent analyses to implement our SSWC approach. Figure 3 shows the orthomosaic with the experimental units where both the SSWC and conventional (NO-SSWC) treatments were applied.

Figure 3
figure 3

Flow diagram of Corn Row Identification (CRI) algorithm to detect lines over corn rows in binary imagery generated from UAS imagery.

Full size image

Vegetation identification

In order to segment the vegetation fraction from the background on the orthomosaic, excess green index (ExG) was calculated using Eq. (1).

$$\begin{aligned} ExGI = 2g-r-b ; where\, (r=\frac{R}{R + G + B}\, g =\frac{G}{R + G + B}\, b =\frac{B}{R + G + B} ) \end{aligned}$$
(1)

where r, g, and b are the normalized values of the bands red, green, and blue respectively, and R, G, and B represent the pixel digital number values of the red, green, and blue color bands respectively.

Once the ExGI was calculated, a threshold value (0.08) was applied to distinguish all green vegetation from the background. Since the objective of threshold selection is to choose a threshold value that can effectively distinguish the object of interest (in this case, green vegetation or corn rows) from the background, the threshold value of 0.08 was selected after manually trying different values in ArcGIS Pro software (ESRI, Redlands, United States) to find the optimal threshold that covers the maximum portion of green vegetation in the field during the early corn growing season.

The optimal threshold value may vary depending on several factors such as image quality, lighting conditions, and the characteristics of the object being segmented. Therefore, it is common practice to manually or automatically test different threshold values and evaluate their performance based on a specific criterion, such as the percentage of pixels classified correctly or the accuracy of the segmentation result. Once the threshold value was applied to the ExGI values to segment the image into the binary foreground (vegetation) and background (non-vegetation) region, it created a binary imagery with pixel intensity value 1 for the vegetation and 0 for the remaining background as described in Eq. (2):

$$\begin{aligned} g(x,y) = {\left\{ \begin{array}{ll} 1, &{} \text {if } ExGI(x,y) > 0.08 \\ 0, &{} \text {if } ExGI(x,y) \le 0.08 \end{array}\right. } \end{aligned}$$
(2)

where ExGI(x,y) represents the pixel value of the Excess green index calculated from UAS imagery at position (x,y), and g(x,y) is the binary output image that results from applying the thresholding operation.

In this expression, if the ExGI value of a pixel (x,y) is greater than the threshold value of 0.08, then the corresponding output pixel g(x,y) is set to 1, indicating that it belongs to the green vegetation (corn rows). Conversely, if the ExGI value of a pixel (x,y) is less than or equal to the threshold value of 0.08, then the corresponding output pixel g(x,y) is set to 0, indicating that it belongs to the background (non-vegetation). This results in a binary image with pixel intensity value 1 for the vegetation (corn rows) and 0 for the remaining background. That binary image was further processed to identify the corn rows.

There are a few scientific reasons why the excess green index could be more useful than NDVI in this scenario. First, ExGI is less affected by soil reflectance than NDVI, which can be particularly important during the early growing season when crops are small and soil background is more prevalent in the imagery64. NDVI is known to be sensitive to variations in soil background, which can lead to false positives or negatives in vegetation segmentation. ExGI, on the other hand, is less sensitive to soil reflectance and can more accurately differentiate vegetation from the soil. Second, ExG can better differentiate between different types of vegetation than NDVI. This is because NDVI is sensitive only to the chlorophyll content in plants, which can be similar between different types of plants. ExGI, on the other hand, is more sensitive to differences in leaf structure and pigments, which can vary more between different plant species. In the case of small corn plants and weed vegetation, which may have similar chlorophyll content but different leaf structures, ExGI can more accurately separate the two.

Algorithm development for crop row detection

We are proposing the use of a crop row identification (CRI) algorithm for detection of corn rows in the binary orthomosaic imagery. In order to expedite the processing time, the CRI algorithm was applied, using Python version only to the six SSWC treatment plots, which was around 30,500 square meters in area. Figure 3 shows the flow diagram for the CRI algorithm that is being proposed for corn row identification.

The first step in the procedure for line detection over the corn rows was to prepare a zero-like array using NumPy, which returned a new array of a given shape and type, filled with zeros. Once the zero-like array was created, a function was defined as a “cropped image” which cropped the input imagery into a smaller size (3000 \(\times\) 2000 pixels). This function was set with a combination of NumPy-SciPy libraries. The NumPy variable “sum y” sums up all non-zero-pixel values into one value for each cornrow. The SciPy variable “peaks” calculated the local maximum values (peaks) in the array of Y-axis summed values. Pixel intensity along X-axis (columns), as shown in Fig. 4a, were summed and projected towards Y-axis (rows) in order to compute the local maximums (peaks) in Y-axis as shown in Fig. 4b. Once the local maximums were computed for each row, straight lines perpendicular to Y-axis were drawn as shown in Fig. 4c. The same process was repeated for all section that was inside the boundary of the SSWC treatment plots.

Figure 4
figure 4

(a) A section of the binary ExGI imagery (3000 \(\times\) 2000 pixels) showing the horizontal orientation of rows, (b) local maximums (or peaks) for each corn row on the section of imagery, (c) lines drawn over the each computed local maximum, and (d) Comparison of actual corn rows with the identified lines created by the CRI algorithm on a corn field imagery.

Full size image

Once the lines were drawn over the computed local maximums in each row, those lines were visually compared with the original corn rows (ground truth) on the imagery using ArcGIS Pro to verify the performance of CRI algorithm for accurately identifying the corn rows from the UAS imagery. Figure 4d shows the visual comparison of the generated outputs as true positive (TP), false positive (FP) and false negative (FN). In this study, TPs are the total number of predicted lines that exactly laid over the actual corn rows. Likewise, there were some places where actual corn rows were present, but the algorithm did not generate line over that row. These kinds of cases were considered FNs. Similarly, there were some lines generated where there were not any corn rows, and the total number of those kinds of lines were considered as FPs. Also, since the background in the imagery was negative and the CRI algorithm did not detect any background as a background value, TN is zero.

Weed mapping across SSWC treatment plots

Our approach of mapping weeds is quite simple, and that is by design. All the vegetation growing within a certain distance (buffer) from the identified rows are considered corn plants, and the remaining of the vegetation, growing mainly between the rows, are considered weeds. Hence, once the corn rows were identified, the next step was to identify the vegetation fraction that would be classified as weeds. Our approach was to identify and delete all corn plants from the treatment plots, and the remaining vegetation was then classified as weeds. In order to implement that approach, we created a 3.5 inches buffer on both sides of the corn row lines as shown in Fig. 5a. This 3.5-inch buffer was wide enough to cover most of the corn plants, which were then deleted from the imagery, leaving behind the weeds present between the corn rows as shown in Fig. 5b. Since the weed pressure in the field was unusually high in the 2021 growing season, in order to implement the full workflow of the proposed approach for SSWC, from image collection to field spraying with a commercial size sprayer, the research team opted to overlay a grid of cells of 0.5 m \(\times\) 3.04 m on the imagery. That resulted in an average value of 35% of the cells free of weeds across the SSWC treatment. The decision of spray or no spray on a given cell was based on the presence of weeds in a given grid cell. Grid cells that contained weed were assigned a rate of 140.3 L/ha herbicide mix solution, while the cells free of weeds were assigned at a rate of 0 L/ha, which is no-spray. The idea is that sprayer would just shut-off the nozzles while travelling over the no-spray cells. Figure 5c shows the prescription shapefile created in ArcGIS Pro, which was converted to a prescription map in a later step. The red cells among the replications of SSWC treatment were free of weeds, whereas the green cells contained weeds.

Figure 5
figure 5

Buffered (3.5-inch on both sides) corn rows (a) that were removed from the imagery to create the weed map across the field (b) and the (c) variable rate prescription map (green cells = 140.3 L/ha and red cells= 0 L/ha) generated based on UAS imagery collected over a corn field using a Sony Alpha 7R II camera.

Full size image

Field area experimental design

To avoid an impact of irrigation on the results, the experiment was placed within the footprint of the center pivot irrigation system available for that field. A set of 24 plots, each measuring 121.9 m long by 41.6 m wide (the same width of the sprayer boom), were arranged in a 6 (North-South direction) by 4 (East-West direction) format placed over the experimental area. Next, every other row of plots were deleted (12 plots total), and the remaining plots were used for the experiment. For each to experimental units, treatments (SSWC and NO-SSWC) were randomly assigned, resulting in a complete randomized experimental design, with six replications across the field.

Integration of weed map into a commercial sprayer

A Case IH Patriot 4440 self-propelled sprayer (Racine, Wisconsin, USA), model year 2021, (Fig. 6a) was used to spray the experimental area. The sprayer was provided with an AIM command FLEX system, which was the latest technology from Case IH that enables consistent, flexible, and accurate spray application in commercial agriculture, regardless of speed and terrain. In addition, the system enhances control of liquid product flow and pressure more accurately than conventional rate controller and enables instant on/off of individual nozzles, with a nozzle valve diagnostic system.

Figure 6
figure 6

(a) Case IH Patriot 4440 series sprayer used to implement site-specific weed control in a corn field. The sprayer was equipped with an AIM command FLEX system, RTK GPS receiver, Viper 4+ cab computer, and 41.68 meter wide boom; (b) Overlap of Rx map, generated by AgSMS, where the regions enclosed by dark black lines where the no-spray regions (0 L/ha), and the as-applied map. Purple color regions are the area where the sprayer turned its nozzles off while operating at 10.5 km/hr speed (approximate), while all the remaining area was sprayed (140.3 L/ha).

Full size image

The sprayer was set up with a Raven RX1 real time kinematic (RTK) receiver (Raven Technology, Sioux Falls USA), which operates at a frequency of 10 Hz. Following Case IH engineer’s advice, the application speed for this study was kept below 11 km/hr (10.5 km/hr actual speed application) to allow for the sprayer to maintain position accuracy resolution of at least 0.3 m (one US foot) per 1/10th of a second or 3 m (10 US feet) per second. That cab computer requires a certain folder structure, so it can read prescription maps (Rx) from an external storage device, such as a thumb drive. AgSMS Advanced software (Ag Leader, Iowa, USA) was used to read the shapefile created in ArcGIS Pro and to convert that file into a prescription map (Rx map) following the format required by the Viper 4+ computer cab. Once the Rx map was brought into the Viper 4+ display, the regions indicated by green color was set to be sprayed with a 140.3 L/ha, which is the same rate that farmers would use for a conventional chemical application.

Assessing the spraying performance in terms of spray accuracy and chemical savings

Once the spraying operation was completed, an as-applied map was downloaded from the sprayer’s cab computer to compare with the prescription map in terms of spray and no-spray area. The spatial area recorded on the 6-SSWC treatment plots (Fig. 6b) of the as-applied map where the sprayer did not spray any chemical herbicide was calculated and compared with the other 6 plots (no-SSWC) where conventional method was applied.

Additionally, application accuracy of the sprayer was evaluated based on it’s ability to perform individual nozzle shut-off over the land where there was no presence of weeds. Based on the sprayed and not sprayed spatial area in the treatment zones between the prescription map and the as-applied map, the sprayer’s performance for chemical saving was evaluated. For that, the total area which was recorded as not-sprayed in the as-applied map was considered as Measured Value, and the total area which was set to be not-sprayed in the prescription map was considered as Expected Value. The accuracy of application in terms of spraying was calculated using Eq. (3):

$$\begin{aligned} Accuracy (\%) = \frac{Sum\ of\ Measured \ Value}{Sum\ of\ Expected\ Value \ n} \times 100\% \end{aligned}$$
(3)

Here, “n” represents the number of grid cells measured. The value of n is not explicitly present in this formula as it refers to the total number of grid cells measured, which is implicitly included in the Measured Value and Expected Value calculations for each treatment plots.

Post-harvest field analysis

In order to investigate the experimental plots in terms of weed growth in a post harvest season, we collected a post harvest imagery on September 21, 2021, where image collection, image stitching, georeferencing, ExGI calculation, segmentation, and thresholding was carried out in a similar way as described for early-stage corn field data collection. The surface area of leftover and newly germinated weeds in the SSWC and conventionally approached test plots in the post harvest imagery was calculated and the datasets were analyzed using SAS PROC MIXED (SAS Institute, Cary, USA) with a mixed procedure using REML (restricted maximum likelihood) estimation. The area covered by weeds in the SSWC test plots was compared to the area of weeds present in six no-SSWC test plots using a pair-wise T-test with a significance level of 0.05 (p = 0.05).

Results and discussion

CRI Algorithm performance for crop row detection

The CRI (corn row identification) algorithm developed in this study demonstrated great accuracy in identifying corn rows from UAS imagery. The algorithm produced 2313 true positives, 12 false positives, 0 true negatives, and 8 false negatives values. The precision, recall, F1-score, and accuracy metrics were all above 0.99, indicating excellent performance in identifying corn rows from a 61,000 square meter land area. The algorithm processing time was 8 seconds on the previously described desktop computer. Additional details on classification results from the CRI algorithm are shown in Table 1.

Table 1 Classification results for the CRI algorithm in identifying corn rows.
Full size table

Here, precision is the percentage of correctly identified corn rows out of all the rows identified as corn row, while recall is the percentage of correctly identified corn rows out of all the actual corn rows in the image. F1-score is the harmonic mean of precision and recall and provides a balanced measure of their performance. True positives (TP) are the correctly identified corn rows out of all the actual corn rows in the image, while false positives (FP) are the pixels identified as corn rows, but they are not actual corn rows. False negatives (FN), on the other hand, are actual corn rows that were not identified by the algorithm. The percentage of TPs and FPs provides insights into the accuracy of the algorithm in detecting corn rows. A higher percentage of TP indicates that the algorithm is correctly identifying corn rows, while a higher percentage of FP indicates the potential for over-detection, where non-corn pixels are falsely identified as corn rows. Over-detection can lead to unnecessary processing or misapplication of herbicides during weed management. In contrast, a higher percentage of FN indicates under-detection, where actual corn pixels are not identified by the algorithm. Under-detection can lead to missed opportunities for weed control or misapplication of herbicides leading to weed resistance.

Because of the very small size of some corn plants as shown in Fig. 7a, the algorithm failed to identify line over those regions. The row of corn plants that could not be identified in the figure is due to the effect of the threshold value (0.08) that was applied during image segmentation. Use of higher resolution camera that can map even the smallest size of corn plant could possibly solve this issue. Also, in very few regions, we have noticed two lines representing a single corn row as shown in Fig. 7b. This is because the corn rows are not perfectly straight inside the region of 3000 \(\times\) 2000 pixels, specified in the CRI algorithm. Since the corn row is not straight over the part of image, two small peaks(local maxima) have been identified in this region and hence, the algorithm displayed 2 lines to represent a single corn row. Setting up smaller region of interest during image processing could possibly solve this issue.

Figure 7
figure 7

(a) Sample image where the proposed CRI algorithm failed to detect an existent corn row as a line on the ExGI imagery of a corn field, (b) Sample image where the proposed CRI algorithm detected two lines over a single corn row on the ExGI imagery of a corn field.

Full size image

Application accuracy and chemical savings using our SSWC method

Overall, the study was able to achieve an application accuracy of 78.4% in terms of not-spraying in the regions free of weeds. Figure 8 shows the no-spray area distribution information based on the prescription map (actual value) and on the as-applied map (measured value). As one take in consideration the results showing on Fig. 6b, the application accuracy seems to be largely affected by the start and stop of the spraying in each spraying area. One key factor that can be playing a role on those results is the speed of the sprayer, which was 10.5 km/h on this study. Additional studies should be conduct to evaluate the impact of the sprayer speed on the spraying accuracy.

Figure 8
figure 8

Comparison of the land area, in m2, between the prescription map uploaded to the sprayer (Actual) and area covered a during application (Measured) across the six replications for the site-specific weed control treatment.

Full size image

Likewise, our approach for SSWC achieved significant chemical savings. Using the prescription map to turn off the sprayer’s nozzles on the grids cells free of weed resulted in savings of 26% of the area to be sprayed. Figure 9 shows the consumption of chemical herbicides, across all 6 replications, on the basis of land area using our SSWC approach and the conventional (no-SSWC) approach. Although the weed pressure at the season we performed this study was unusually high, the study proved that it is possible to spray chemicals only to the areas with weeds in the field. As an average, the SSWC treatment save around 26% of chemical herbicide in comparison with a blanket application (No-SSWC).

Figure 9
figure 9

Comparison of the land area, in m2, not sprayed across replications for site-specific (SSWC) and non site-specific (NO-SSWC) weed control treatments.

Full size image

However, to achieve more efficient and effective herbicide application, reducing costs and environmental impacts while increasing crop yields, there are still opportunities for further improvement in the performance of the sprayer through the integration of remote sensing tools. One potential area for improvement is the integration of real-time sensors on the sprayer itself. This would enable the sprayer to adjust the rate of herbicide application in real-time based on the density of weeds detected by the remote sensing tools. For example, if a dense patch of weeds is detected, the sprayer could increase the rate of herbicide application in that specific area to ensure effective weed control. Another potential improvement is the use of more advanced remote sensing technologies, such as hyperspectral imaging or LIDAR. Hyperspectral imaging can provide more detailed spectral information about the crop and weeds, allowing for more accurate classification and mapping65,66. LIDAR can provide 3D imaging, allowing for more accurate detection of the height and density of vegetation, which can help in distinguishing between crops and weeds67. Furthermore, integrating machine learning algorithms into the remote sensing analysis can improve the accuracy of weed detection and classification. By training machine learning models on a large dataset of remote sensing data and ground-truthed weed maps, the algorithms can improve the accuracy of weed identification and provide more detailed information about the distribution of weeds in the field.

Likewise, in this study, we focused on mapping weeds between the corn rows using a simple buffer-based approach. However, we acknowledge that there may be some weeds present within the rows that are not captured by our method. To address this limitation, our original idea for the grid cell size is either 1.5 × 3.0 m (5 × 10 US feet) or 3.0 × 3.0 m (10 × 10 US feet). The first one would cover two rows of corn, while the bigger cell size would cover four rows. Since we make the spray or not spray decision based on the cell grid and not on the individual weeds, we believe the approach would control most of the weeds withing the rows of corn as well, since the approach would result in spraying two or four corn rows. Due to the high weed infestation in the experimental area, we use a smaller cell size to implement the whole workflow and to generate some saving in chemical. If we had used either a 1.5 × 3.0 m or a 3.0 × 3.0 m grid cell size for this study, approximately 100% would need to be sprayed, and not saving would be achieved.

Although some leading agricultural machinery manufacturing companies, such as John Deere and CNH Industrial, have announced about their future release of an automated sprayer which could perform site-specific application in some crops including corn, the adoption rate of the new technologies by farmers could be slow due to the expensive cost of agricultural spray machines. Our approach provide farmers that already a sprayer with individual nozzle control an option for site-specific weed control.

In a post harvest imagery based field evaluation of the SSWC and no-SSWC treatments plots, we found that the treatment were significantly different in terms of weed presence \(\left( \textrm{F}_{1,5}=11.41,\; p<0.0197\right)\). The amount of weeds present in SSWC (3.4 m2) treatment plots in terms of surface area was 3.4 times higher than the amount of weeds that were present in no-SSWC plots (1 m2). The findings from this study could be a foundation to pursue various research questions that our study could not address, because of time and other limiting factors. Although the imagery obtained from the field had high resolution (GSD= 0.63 cm per pixel), we might have missed smaller weeds than the camera resolution during the data collection of the early growing season. Those weed detection skips might be one of the reasons why we observed weeds in the SSWC plots after the corn harvest. One could argue that image resolution should be increased to avoid those skips, but there are other challenges to achieve that, such as cost of better hardware (UASs and sensors), time to collect data (by flying lower with the same equipment used on this study), and time to process the data (lower altitude flight would lead to many more pictures captured, which increases processing time). The reasons to why the SSWC had more weeds than no-SSWC could likely be a combination of factors: (1) weeds were sprayed too late (rate not adequate to the size of the weeds); (2) lack of overlap when individual nozzles were applying chemical (applying even less chemical to the already too big/tall weeds); and (3) missing small weeds that germinated after we mapped the area.

The presence of weeds in agricultural fields can significantly impact crop yields, and farmers often rely on chemical herbicides to manage weed populations. However, weed resistance to herbicides is becoming increasingly common, and the effectiveness of chemical treatments can vary based on a variety of factors, such as weed species, growth stage, and environmental conditions68. Historical weed field conditions can also play a role, as previous management practices may have allowed for the buildup of weed seed banks in the soil. In the case of the SSWC and no-SSWC treatment plots, the higher weed presence in the SSWC plots could be attributed to a combination of factors, including spraying the herbicide too late for effective control, lack of overlap in herbicide application, and missing small weeds that germinated after the initial mapping. These findings highlight the importance of ongoing research and development of weed management strategies to address these challenges and improve crop yield and quality.

We do acknowledge that implementing the pipeline for SSWC presented on this paper requires specialized equipment, software, and knowledge in remote sensing and precision agriculture. It may not be feasible for all farmers to run this pipeline on their own, as it involves high initial investment in equipment such as drones and cameras, and requires technical expertise to operate the equipment and analyze the data. Additionally, the cost of acquiring and processing the high-resolution imagery may also be a limiting factor for some farmers. However, farmers could potentially hire service providers or companies that offer remote sensing and precision agriculture services to run the pipeline for them. This could be a more cost-effective and efficient solution for farmers looking for more sustainable solutions for weed control in their corn fields.

Conclusion and future suggestions

In this study, we proposed a new approach of fusing unmanned aerial system (UAS) data with a spraying platform to minimize chemical use in weed control application for corn production. The results showed that our site-specific weed control (SSWC) approach has the potential to reduce financial expenditure in chemicals to apply a blanket application of herbicide in corn production. Moreover, our approach could produce healthier corn by using fewer chemicals. By identifying corn rows and removing them from the UAS imagery, we demonstrated an effective way to map weeds during the early growing season of corn. However, the human intervention required for weed mapping in this study would take approximately a day to collect and process data for making a weed prescription map, which may not be considered real-time prescription. Because of the significant reduction of chemical usage, our approach of spraying chemical herbicide only to those area where there are weeds could help to improve sustainability on corn production systems. Since 75% of all planted acres of cropland across the US are row crops, the approach we adopted in this study could be a foundational study for performing SSWC in the vast majority of US row crops.

To make our SSWC approach more user-friendly and consumer grade for commercial agricultural production, we suggest automating real-time prescription map generation from the information collected using UAS to reduce data collection and processing time. One potential solution for this is the use of embedded platforms to integrate UAS data with sprayer platform, which could be a field to explore in the near future. In addition, the overlap between individual nozzles of the sprayer should be set in a way that the spray-pattern covers the entire region set in the prescription map. Furthermore, the sprayer’s GPS refreshing rate needs improvement to operate at a higher speed for commercial farming practices. While the current study required a day to collect and process data to generate the prescription map, future research should aim to reduce the time required for real-time prescription mapping. By doing so, we could provide farmers and researchers with an efficient and effective way to manage weeds in corn production systems. Overall, the SSWC approach offers a promising alternative to conventional weed control methods in corn production, and we believe that our future suggestions could make it more accessible and useful for commercial agricultural production.

While investigating the potential effects of SSWC on grain yield is an important issue, it should be noted that this study was performed on “silage” corn, which is primarily used for livestock feed. As such, the impact of SSWC on grain yield was not be the primary focus of this study, but rather on reducing the use of chemical herbicides and improving the precision of weed control to achieve more sustainable and efficient agricultural practices. It is also important to consider the overall sustainability of agricultural production, which takes into account economic, environmental, and societal factors, rather than just focusing on grain yield alone.