Exciting (S10) L pod calls as the sun sets over Orcasound Lab

Humans and machines both detected the sounds of Southern Resident killer whales (SRKWs) this evening (Wed 25 Aug 2021). For about a half hour (~19:25-20:05) the OrcaHello AI and attentive listeners heard diverse calls interspersed with many clicks as they monitored the Orcasound Lab hydrophones in Haro Strait (5km south of Roche Harbor on the west side of San Juan). This is a short (~3 minute) sample of what these orcas sounded like:

(ogg format; download via 3-vertical-dots at right)
(mp3 format; download via 3-vertical-dots at right)

And here’s a spectrogram of that highlight clip:

A few listeners had heard them previously on the Lime Kiln hydrophone (5 km to the south) and some sightings had been reported as they moved northward. The Center for Whale Research reported that the pod might include the L54 matriline which seems mostly consistent with the call types that were recognized during the live listening event and near real-time review of the automated detections. As the sun set, the residents continued north past Henry Island. Thanks to Jim Maya who was close to his boat at the time, we have this image of the scene — a rare one in that SRKWs have been seen and heard so sporadically and rarely this year within the Salish Sea.

A resident orca travels towards Canada in the last light of day. Photo by James Mead Maya.

Acoustic data

Since Orcasound hydrophones stream their data to listeners (human and machine) via Amazon’s S3 storage service, it’s straight-forward to retrieve a recording from the open-access data “buckets.” If you’d like to try it yourself, here’s some guidance on how to access Orcasound data. It’s still a little tricky, though, so for ease of access, here are some ~45-minute clips that encompass the acoustic event (aka “bout) in .ogg and .mp3 formats:

See the end of this post for more info about these acoustic bouts and accessing Orcasound’s free, open data.

Thanks to an awesome team of Microsoft hackathon volunteers collaborating with Orcasound, our open data, and Drs. David Bain, Val Veirs, and Scott Veirs over the last couple years, you can also access the automated detections from the OrcaHello live inference system (including via an API):

All automated detection events (18 x 1-minute candidates):

Bioacoustic analysis (preliminary!)

The acoustic bout is contained in a 6-hour “folder” which began accumulating streamed audio segments at Wednesday, August 25, 2021 6:30:19 PM GMT-07:00 DST (Unix datetime stamp = 1629941419). When I downloaded all the data in that folder, about 4..2 hours of data had accumulated. Viewing this long period as a spectrogram in Audacity, you can see that we got lucky: the SRKWs passed by the Orcasound Lab at a time when there wasn’t much vessel noise (making for better listening to Nature)!

Audacity waveform and spectrogram shows ~4 periods of ship noise (magenta) in a 4.25-hour period with otherwise low ambient noise.
The green section contains SRKW signals for about 40-minutes — luckily right between two bouts of vessel noise!

Zooming in on this period, we can define the start and end and duration of the acoustic event (aka “bout). From the first faint calls (at 40:30 in to the recording, or about 19:11 Pacific time) until the last calls (at 1:27 into the recording, or 19:57), the duration is roughly 40 minutes.

The acoustic bout (highlighted in green). You can see a few labels about 10 minutes after the bout, one of which sounds like an SRKW buzz, but without any other clicks, calls, or whistles audible, I’m attributing it to some other source… and not extending the stop time of the bout that far out.

A note about absolute timing of Orcasound data:

Relative to the timestamps of the human and machine detections, these start/end times for the bout seem off a bit. If you trust the datetime stamp conversion underlying the OrcaHello AI, then we can establish some of the earliest detections as having occurred at 19:25:45 (=19:25:07+00:00:38). By the way that was just 3 minutes before the first evidence of a human detection manifested (a post by Gayle on the Whale Sightings in the San Juan Islands Facebook group managed by the Orca Behavior Institute).

The first machine detection candidate had a time stamp of 19:25:07. The white lines indicate the approximate location of the periods when the model had high confidence that it had detected the call of a SRKW.

I was able to find the same sequence of faint S19 calls near the beginning of the bout through manual processing of the raw data in Audacity:

A series of faint S19 calls near the beginning of the bout.

The corresponding “S19 faint” call occurs about 44:39 into the raw recording, which was built from HLS segments that began streaming at 18:30:19. So, the absolute time of that faint call from Audacity’s perspective is 19:14:58 — about 10 minutes earlier than the OrcaHello time of 19:25:07. This presents a bit of a mystery, but one possible explanation is that some bits are getting dropped from the HLS segments (probably on the Raspberry Pi). If we lost the last ~1-2 seconds off the end of each 10-sec HLS segment during those first ~45 minutes, that could account for ~450-900s of data, or potentially about ~10 minutes.

Bout data details

For the purposes of naming this particular bout, I’m going with 210825_1922-2007_OS_SRKW_L, corresponding to 19:22-20:07 — due to me adding 10 minutes to the previously-determined bout boundaries 19:11-19:57. See the start of this post for direct links to .ogg and .mp3 versions of the bouts. (Later this fall, we’ll likely start providing each bout in FLAC format to preserve as much information as possible for research and AI development.)

Highlight clip details:

  • 2-minute 45-second highlight
  • 00:56:37-00:59:22 in the original downloaded 4.25-hour concatenated recording…
  • …so, approximately 19:26:56-19:29:41 in absolute time (or maybe if you add ~10 minutes, about 19:37-1940)?

Human and machine detections

The OrcaHello AI did a remarkably good job of being “first” to detect SRKWs this time. You can see, though, that the machine did miss a couple of similarly faint calls in the first 30 seconds of the first 60-sec detection candidate.

If you zoom out a bit from the above sequence of faint S19s, you can see that the annotation labels start some 3.5 minutes earlier. If a human had been listening very carefully — ideally a young human with good high-frequency hearing who was also able to watch for the tell-tale spikes of echolocation clicks in the waveform — they may have beaten the machine by a few minutes!

Of course the human listeners were only a few minutes behind and ultimately did a record-setting job of tagging the audio data in real time. The table of human detections — grouped temporally into candidate bouts — shows that one candidate had 29 annotations (a new record!). Looking within that candidate (below) you can see that a label was submitted every minute or so, and that there were more than 100 humans listening simultaneously a little after 8pm (also a new record!). I think the high we saw overall was ~107 listeners.

Keep up the good work, community scientists! And please comment with additional bioacoustic insights or other thoughts. Also, if you ever want to blog about Orcasound data (like this), don’t hesitate to ask for an account by emailing info@orcasound.net or just join our Slack Workspace to start collaborating.

For the geeks out there, here’s the code for downloading and transcoding the raw data for this event (~250MB):

ts2mp3.sh orcasound_lab

ts2mp3 orcasound_lab 1629941419

Vis this (hacky!) shell script:

ts2mp3.sh 
# ts2mp3.sh expects two arguments:
#1 node name (one string with underscores, e.g. bush_point (NOTE: no leading rpi_ !)
#2 UNIX timestamp of desired S3 folder within the nodes hls folder
#3 Start time in hours after the UNIX timestamp
#4 Stop time in hours after the UNIX timestamp

# Add some logic to skip data that isn't between the desired start and stop times
# i.e. within aws sync call or in the for loop (delete some .ts segments; rename others)

echo "You provided $# arguments: $1, $2, $3, and $4"
aws s3 sync s3://streaming-orcasound-net/rpi_$1/hls/$2/ .
for file in live*; do mv "$file" "${file#live}"; done;
for i in *.ts ; do
    mv $i `printf '%04d' ${i%.ts}`.ts
done
printf "file '%s'\n" ./*.ts > mylist.txt
ffmpeg -f concat -safe 0 -i mylist.txt -c copy all.ts
ffmpeg -i all.ts -c:v libx264 -c:a copy -bsf:a aac_adtstoasc output.mp4
ffmpeg -i output.mp4 output.mp3

You can find more details about how to access free Orcasound data here — https://github.com/orcasound/orcadata/blob/master/access.md

Just remember to adhere to our Creative Commons NC-SA-BY license, giving attribution to “Orcasound — orcasound.net”

One thought on “Exciting (S10) L pod calls as the sun sets over Orcasound Lab

Leave a Reply

Your email address will not be published. Required fields are marked *