Insight into the kinematics of blue whale surface foraging through drone observations and prey data

Data collection

Fieldwork was conducted in February 2017 on a blue whale foraging ground in the South Taranaki Bight (STB) region of New Zealand (Fig. 1; Barlow et al., 2018) where whales target aggregations of N. australis. Blue whale occurrence data was collected during standardized survey effort (Barlow et al., 2018) aboard a 19.2 m vessel at speeds of 8 to 12 knots in suitable weather conditions (Beaufort Sea State < 5). Survey observation data was collected at 4 m above the waterline by two observers scanning either the port or starboard sides of the trackline independently. Survey tracklines did not follow a standardized layout (i.e., sawtooth or grid pattern lines), but rather aimed to maximize encounter rates with blue whales and therefore had an irregular trackline pattern directed toward productive (based on remotely sensed images of chlorophyll-a and sea surface temperature) or previously unsurveyed areas. Survey effort stopped at all sightings of blue whales and the whale(s) was slowly approached for behavioral observation. Whale behavior states were classified as travel, forage, social, rest, or unknown based on observations, with further details, such as observations of SLF events, noted. Travel was defined as directional movement and regular surfacing. Indications of foraging included surface lunges and staying in one area for a prolonged period with irregular surfacings or fluke-out dives. Social behaviors included mother-calf nursing, prolonged coordinated surfacing such as racing, and tactile contact between individuals. Resting behavior consisted of logging near the surface with minimal forward movement. All behaviors that did not fit within these classifications were considered unknown. Dive times of individual whales at all sightings were recorded as the interval of time between the whale’s surfacings. The mean dive time was attributed to the sighting. If multiple individuals were present, dive times were recorded for each individual whale, with an average dive time calculated per whale, and then a group mean was determined for that sighting. During survey effort and at many whale sightings, krill distribution data was collected by a Simrad EK60 echosounder. Field research on blue whales was conducted under research permit approval from the New Zealand Department of Conservation (45780-MAR) and Institutional Animal Care and Use Committee exemption from Oregon State University (16-1083).

A DJI Phantom 4 Advanced (non-Pro) UAS was flown over blue whales using manual flight control and real time camera output for the primary purpose of body morphometric assessment through photogrammetry (Burnett et al., 2019). The UAS camera had a 3.61 mm focal length and a 0.0015 mm pixel size. On 20 February 2017 at 19:21 local time the UAS was hand-launched off the research vessel to fly over a single blue whale. Conditions were exceptional, with no wind or swell, and bright off-angle lighting, which enhanced water clarity and contrast so that multiple surface patches of krill were identifiable. The UAS was navigated toward the whale’s location after collecting calibration data over a board of known length at multiple altitudes for the purpose of later correcting the barometric altimeter (Burnett et al., 2019). Just after the whale was localized in the UAS video with the camera pointing nadir, the whale began a SLF event (Table 1, Event 1 at 19:23:38). The UAS operator navigated the drone to film the event as best as possible given the rapid sequence of events, yet unfortunately the expanded buccal cavity is out of frame for ∼1.5 s during engulfment of the krill. Although the whale dove out of sight after this feeding event, we later filmed its surface activity for another 5:47 min using the UAS. The UAS maintained an altitude between 29 and 38 m above the whale during these observations while we recorded three other instances of this same blue whale recognizing surface krill patches but not feeding (Table 1). Low UAS battery power and the setting sun forced retrieval of the UAS. A small tissue biopsy sample was then collected from the whale using a lightweight biopsy dart (Krützen et al., 2002) for genetic analysis to determine sex (Barlow et al., 2018).

Krill data analysis

Hydroacoustic data were collected using a Simrad ES120 splitbeam transducer with a 120 kHz receiver, 250 W, 1.024 ms pulse length, and 0.5 s ping rate. The transducer was mounted on a pole and deployed 1.45 m below the surface. Raw data were processed and extracted using the MATLAB-based program ESP3 (Ladroit, 2017), developed for fisheries hydroacoustic analysis. Volume backscattering (Sv) measurements were binned vertically into 1 m depth bins and horizontally into 5-ping bins. The upper 2 m of the water column were excluded due to noise from disturbances at the surface such as wind and swell, and sections with recognizable interference from CTD casts or missed pings were removed. Following complete visual inspection of the echograms for quality control, measurements made at 2 m or deeper were retained for analysis. The echosounder was not calibrated, and therefore all values for backscatter strength represent a relative characterization of prey availability within this ecosystem.

Zooplankton-like schools were identified by excluding ping data with volume backscattering strength below the –90 dB threshold at 120 kHz. We are confident that this threshold reliably identified and captured aggregations visible on the echosounder, thereby effectively characterizing schools within this ecosystem. The threshold value applied is more conservative than those used in a similar analysis for larger krill species with established cutoff frequencies (e.g., Euphausia superba, Bernard & Steinberg, 2013). Recognizing that we were not able to definitively exclude other ensonified species (i.e., fish) and in the absence of a known target strength for the krill species of interest (N. australis), we assume that the aggregations identified acoustically were predominately comprised of krill. Krill aggregations were identified following the methods described by Bernard et al. (2017). Each element in the acoustic matrix that qualified as krill was considered to be part of an “aggregation” if one of its eight neighboring elements also qualified as krill (Lawson et al., 2008; Bernard & Steinberg, 2013).

The mean Sv, depth, and vertical thickness were calculated for each krill aggregation (Supplementary Information, Table S1, Fig. S1). Mean depth was defined as the depth at the midpoint of each aggregation. Mean Sv of each aggregation was used to represent patch density. Aggregations were defined as being at a blue whale sighting if the center of the aggregation was within a 2 km radius of the blue whale sighting, and all other krill aggregations were considered not to be at a blue whale sighting. A threshold of 2 km was applied because blue whale sightings ranged up to 2 km from the initial sighting location; hence, this distance was considered to best characterize the availability of krill aggregations to the whale(s) at a sighting.

Krill patch density observed during the four UAS filmed events could not be quantitatively determined because the underwater portion of the patches are indiscernible. Yet, density of surface swarms of N. australis can range between 3,000 and >450,000 ind m⁻³, with individual krill lengths 15–18 mm and the wet weight biomass ranging between 40 g m⁻³ and 7 kg m⁻³ (O’Brien, 1988). Furthermore, O’Brien (1988) estimated the wet weight of a 15 m cigar-shaped patch of N. australis to be 100 kg. The length of the krill patch consumed during the filmed SLF event was 16.2 m (excluding the long tail on the bottom-right: Fig. 2H). Therefore, although we cannot quantify the density or biomass of the patch targeted during the SLF event, both metrics may be relatively high.

UAS video of analysis of blue whale

Despite not having a laser altimeter on-board our UAS, accurate scaling between pixels and metric units of images captured in nadir was achieved using our calibration board (Dawson et al., 2017; Burnett et al., 2019). Barometric altitude was corrected for bias using pixel length (from video) and metric length measurements of the board at a range of altitudes to create a correction model (Burnett et al., 2019). Measurements were scaled from pixels to metric units by multiplying pixel measurements by the ground sampling distance (GSD) where GSD was calculated by multiplying model corrected barometric altitude by camera sensor pixel size in mm and dividing by camera focal length in mm (Comer et al., 1998).

In all four events, estimates of the whale’s position relative to the krill patch were reconstructed from the UAS video sequence using Adobe Photoshop CC (V 2015.5; hereafter Photoshop). At 0.5 s intervals, the following metrics were estimated: horizontal distance between the whale’s rostrum tip and the krill patch, instantaneous speed of the whale, instantaneous acceleration of the whale, and the whale’s body roll and head inclination angle. Also estimated were the distance at which the krill responded to the whale, krill patch area pre-strike, and krill patch area post-strike for the SLF event. The lack of reference points in the relatively homogenous ocean scene was a challenge for data analysis, however we attempted to mitigate this issue by using a consistent point on the leading edge of the krill patch as a reference marker (anchor point) and limiting data analysis to when the whale and krill were simultaneously visible. We therefore assume that the krill patch did not move significantly relative to the whale’s larger magnitude movements until its flee response from the whale at close range, which was also measured using the same anchor point to ensure consistency. We also assume no impact of water current on data analysis, which is considered a valid assumption as minimal lateral movement of the krill was observed during periods when the aircraft was stationary. Our estimated values of relative acceleration and speed should not be considered absolutes due to the many sources of minor uncertainty in UAS imagery that are difficult to quantify given the lack of reference points, in situ water current measurements, and ability to estimate error. However, our methods provide reasonable estimates of whale kinematics as confirmed by similarities to tag-derived metrics of blue whale lunge biomechanics, provide rare estimates of krill prey flee response, and are consistent across the four UAS filmed foraging events thus providing reliable relative comparisons between events. For future applications of UAS to film whale surface behavior we suggest the use of a laser altimeter to improve precision and accuracy of altitude estimates, and thus speed and acceleration measurements.

During the SLF event, time-stamps of fluke beats, pectoral fin movements, and mouth opening were also noted, and maximum gape was calculated using the online tool at https://www.ginifab.com/feeds/angle_measurement/. The times determined as when the whale first responds to the krill patch, krill begin flee response from the whale, and widest gape during the SLF event are recognized as subjective choices, yet all authors estimated these values independently and were in agreement.

The interval of interest in Event 1 when the SLF event was filmed was isolated using the video timeline functionality in Photoshop to the time frame beginning immediately before the whale changes directions in response to the krill patch and ending at the completion of the dive following the SLF event. This interval limited the analysis to frames between 135.37 s to 166.87 s and corresponds to 12 seconds pre-strike through 19.5 s post-strike. Video of the lunge feeding event was segmented into 3,840 ×2,160 pixel images at 0.5 s intervals using the video rendering function in Photoshop. Images were distortion corrected in MATLAB™ (Burnett et al., 2019) and mosaicked in Photoshop. To preserve image scale of the mosaic, a new 15,000 by 15,000 pixel blank image was generated with the same dots per inch (DPI) as the distortion-corrected images. Estimation of the whale’s relative speed and acceleration during the SLF event was limited to the time interval where the krill patch and whale were in the frame simultaneously from 9 s pre-strike to 19.5 s post-strike. These 0.5 s interval images were imported and spatially arranged using the krill strike point as the anchor position. Whale position with respect to time was estimated by recording the x, y coordinates of the tip of the whale’s rostrum position in each of the constituent images within the mosaic.

Distance at 0.5 second intervals was estimated using the x, y position at the beginning and end of each interval as inputs to the Euclidian distance equation. Distances were scaled to units of meters using the GSD equation (Burnett et al., 2019). The 3.61 mm focal length camera on a Phantom 4 Advanced at 29.5 m mean corrected altitude above-sea level resulted in a 0.0119 m GSD per pixel scaling constant. A spline function of distance with respect to time was developed by accumulating distance over time using the ‘Spline’ function in R (V 3.4.3) (R Development Core Team, 2018). The spline function allowed for the estimation of instantaneous speed and instantaneous acceleration. A spline was selected instead of using raw numbers to estimate average speed and acceleration over small time intervals in an effort to reduce the inherent noisiness of the data due to small errors associated with scaling and imprecision associated with visual interpretation. Speed was estimated instead of velocity because we were interested in accumulated distance instead of vector-dependent total displacement. Distance between the whale and krill patch was estimated by calculating the scaled pixel distance between the tip of the whale’s rostrum position and initial krill strike point at each time interval.

Following the mosaic, images associated with the undisturbed pre-strike krill patch were merged to facilitate a contiguous area estimate (Fig. 2H). Post-strike residual krill patch area was estimated using the frame at 151.87 s that displays the entire residual krill patch (Fig. 2I). Scaling of pixel areas to square meters occurs using the 0.0119 m scalar.

Individual images at 0.5 s intervals during Event 1 were visually assessed by all four co-authors independently to subjectively assess roll and head inclination throughout the SLF event. Body roll analysis began at 12 seconds pre-strike and was evaluated using the right-hand rotation convention where rotation to the left is negative and rotation to the right is positive (Fig. S2). 0° body roll occurs when the whale is swimming straight with the body level in the water. The position of the whale’s spine, pectoral fins, jaw line, and body flanks relative to the overhead perspective of the UAS camera in nadir were used to estimate the whale’s body roll. Head inclination is the measure of angular head position deviation from neutral where neutral (0°) occurs when the body is fully elongated with a neutral spine and an imaginary straight line extends from the tip of the rostrum to the tail. Head inclination ranges from 90° to −90° where positive is a position where the head is above the imaginary line connecting the rostrum and tail, and negative is the position below the imaginary line connecting the rostrum and tail. Change in head inclination is best observed at roll angles with absolute values between 45° and 110°, so estimations were limited to between 4 s pre- to 7.5 post-strike (unless unobservable between 2.5 and 3.5 s post-strike when the whale was off-screen; Figs. 2A, 2J–2L).

Subjective visual estimation of head inclination and roll angles by each co-author is imprecise but indicative of relative body attitude changes with respect to time. Interpretation of head inclination and roll was limited to 5° and 10° increments, respectively, to reflect the imprecision of the estimates (Fig. S2). Moreover, we calculated the Interrater Correlation Coefficient (ICC; Koo & Li, 2016) between the four independent evaluations of head inclination and roll by co-authors to evaluate consistency and absolute-agreement as metrics of confidence in estimations. ICC estimates and their 95% confident intervals were calculated using the ‘PSYCH’ statistical package in R based on a mean-rating (k = 4 raters), estimating consistency and absolute-agreement, using a 2-way mixed-effects model. The mean and CV values of the four head inclination and roll estimates for each 0.5 s interval image were used in Fig. 2A to illustrate the blue whale’s body positioning throughout the SLF event, and any variation in estimations.

Distribution of blue whales and krill

During 887 km of survey effort in the South Taranaki Bight (STB) of New Zealand, 32 blue whale sightings were recorded of 68 individuals (Fig. 1). Foraging behavior was apparent at 15 sightings, with SLF observed at six of those sightings. Echosounder data were recorded for 90.6 h during the field season and 2,911 krill aggregations were identified from analysis of these data. The highest krill density occurred in the upper 20 m of the water column, and blue whale sightings with perceived foraging behavior occurred in habitats with greater krill density relative to background availability (Fig. 3). Furthermore, the mean depth of krill aggregations was shallower at all blue whale sightings as compared to background availability, and depth became progressively shallower when compared to sightings where foraging behavior was detected or SLF was observed (Table 2, Fig. 3). The dive times of New Zealand blue whales followed this same trend, with relatively short dive times recorded for all sightings (2.83 ± 0.27 SE min), which became even shorter at apparent foraging sightings and where SLF was observed (Table 2). Krill aggregation depth and density during the sighting of the UAS filmed whale are representative of the general pattern of krill availability across the study region (Fig. 3), thus supporting the interpretation of these filmed surface behaviors relative to the study area and whale population.

Kinematics of the UAS filmed surface lunge feeding event

An 18.69 m female blue whale—determined through photogrammetry (Burnett et al., 2019) and genetics (Barlow et al., 2018) respectively—was filmed via UAS engaged in four surface foraging events (Table 1), including a successful SLF event. Kinematic analysis of this SLF event (Event 1, Table 1) reveals dynamic maneuvers by the whale in speed, acceleration, head inclination and roll to line up the prey for maximum engulfment (Fig. 2). The ICC and reliability ratings of the four independent estimations by each co-author of the whale’s head inclination and roll in 0.5 s interval images found that roll estimates were estimated with excellent reliability, and head inclination estimates were estimated with moderate to excellent reliability (Table S2). These results are also reflected in the low CV in Fig. 2A, all of which indicate high confidence in head inclination and roll estimates.

Initially the whale is travelling fast (2.8 m s⁻¹) 9.0 s before the strike, then slows down when she recognizes the krill patch at a distance of 20.74 m and begins her pre-lunge roll to her left side (min roll angle −27.5°) to line-up the patch. Then she performs one full fluke beat cycle while rolling onto her right side (max roll angle 110°), and reaches maximum relative acceleration of 5.8 m s⁻² 1 s before strike, followed by maximum downward head inclination (−15°) 0.5 s before strike just as her mouth is beginning to open. At strike, her mouth begins to open while at a maximum relative speed of 3.33 m s⁻¹, and reaches its widest gape angle of 33° 1.5 s after strike (Fig. 2G) when maximum upward head inclination (18.75°) and maximum deceleration (−3.7 ms⁻²) occurs. The duration of the lunge event, from mouth opening to mouth closure, is 4.14 s. The whale then rolls back to center and descends slowly (mean relative speed 1.22 m s⁻¹) without taking another breath until she surfaces 2:26 min after her previous breath. The whale’s surface series prior to the SLF event included breaths 54, 38 and 18 s before strike.

The UAS footage also enables observation of fluke and pectoral fin movements relative to the foraging mechanics of a SLF event. The observed whale gets up to speed with two complete fluke cycles (4 beats): the first cycle is used during pre-lunge roll maneuvering for target patch line-up; the second fluke cycle accelerates the whale toward the patch; she stops fluking at strike, and then glides down with small fluke beats starting 9.5 s after strike. The whale flares her left pectoral fin upwards 0.5 s before strike, and then lays the fin against her enlarged buccal cavity after strike; both pectoral fins remain extended to her sides as she descends like an airplane.

Predator and prey behavior

While we are unable to calculate prey patch volume in cubic meters, the estimated area of the krill patch before and after the SLF event is 45 and 32 m² respectively, although some krill may also be under the whale after the lunge event (Figs. 2H–2I). Reduced krill patch density is visually evident after the SLF event based on differences in color and saturation (Figs. 2H–2I). Surface krill in the target patch begin to respond to the SLF predation event when the whale is 2 m away, at 0.8 s before strike, perhaps due to the pressure wave from the whale’s rostrum (krill at depth not evaluated; Fig. 2E). At this time, the whale is moving fast (∼3 m s⁻¹), likely to overcome the rapid escape maneuvers of krill (O’Brien, 1987). The UAS footage allows observation of individual krill ‘uncoordinated tail-flips’, group ‘flash expansion’, and propagation of escape response (Fig. 2F) through the krill swarm as described by O’Brien (1987). Although the krill move in response to the predatory whale, the distance traveled is small relative to the speed and size of the whale’s gape. Indeed, the krill do not appear to flee in an outward direction from the patch and the general shape of the patch, including the long “tail”, is maintained throughout the lunge (UAS videos available in the Supplemental Information). This lack of prey dispersion likely allows a high capture percentage for the lunging whale.

During all four UAS filmed events, the blue whale appears to use her right eye to gain visual information about krill patch size, density and orientation to inform her decisions (Fig. 4; UAS videos available in the Supplemental Information). For example, prior to the whale’s right side roll for the SLF event, she performs a pre-lunge roll to her left allowing her right eye to observe the krill patch at a distance of 18.46 m 6.5 s before strike (Fig. 2D).