https://www.twz.com/8778/the-u-s-navy-tried-to-turn-whale-songs-into-secret-code

Maybe they'll just learn to speak? 🤔

Interactive bioacoustic playback as a tool for detecting and exploring nonhuman intelligence: “conversing” with an Alaskan humpback whale https://peerj.com/articles/16349/

Progress!

https://osf.io/preprints/osf/285cs

• Does it have to be a biological whale?

• Are there limitations on how long the translation process takes?

• Would you accept it if it's possible in-principle?

• Does the whale have to understand you?

  • What if whale talk can be converted to human talk but human talk to whale talk is a compression of a very high dimensional space to a very low dimensional space such that most significant data is lost or very hard to recover?

@firstuserhere yes, yes, no, yes

whale email does not count as talking. idk what the minimum bits/sec should be. i am open to suggestions

it has to understand me at least a little. like a human toddler. i’d be happy to talk with an extremely limited words set

This is such a cool market. I am less ambitious on the timeline but I’m so glad that other people want this 🥲

More whale hopes and dreams: /Gen/will-ai-or-humans-decode-and-transl

INFO: Are you in a position to actually have a conversation with a whale (through a career or life circumstance) if such a translator actually existed? Basically, am I betting on simply whether or not the translator exists, or am I betting on that and whether or not you can locate a whale friend to have a conversation with?

@ChrisMills I am in no particular position to talk to whales at the moment.
I am not particularly qualified to be part of a "first contact" team. I do not know much about marine biology or linguistics.

I am in general very excited about technology and would probably do it if given the chance!

I hope so, but I would bet against.

New market with slightly longer timeline: https://manifold.markets/AdamK/meaningful-twoway-conversation-betw

Meaningful two-way conversation between a dolphin/whale and a human before 2040

50% chance. Inspired by: https://twitter.com/hamandcheese/status/1707097197714489550 Will we build an audio system before 2040 that facilitates a nontrivial two-way exchange of information between humans and a dolphin or sperm whale. This market aims to have a high bar for what would qualify as a meaningful exchange. Some examples that would resolve YES: We build a system that can communicate the steps of a complicated trick or maneuver using an audio system alone to a dolphin. The dolphin attempts the trick, and we provide audio feedback about what it might have done wrong. In less than 3 rounds of feedback, the dolphin does the trick correctly We negotiate a trade between ourselves and a dolphin, asking what we must offer in order for it to complete an unfamiliar and nontrivial task. The dolphin responds vocally with a request that we can confidently interpret, then performs the task after the request is met. On an I-know-it-when-I-see-it basis, a conversation more conceptually broad and substantive than one can have with a typical two-year-old human

I think having lots of whale audio is a bottleneck that's hard to get around.

Maybe we can finetune giant models on whale clicks? https://arxiv.org/abs/2102.01293

Would help w sample efficiency and probably w aligned representations

Scaling Laws for Transfer

We study empirical scaling laws for transfer learning between distributions in an unsupervised, fine-tuning setting. When we train increasingly large neural networks from-scratch on a fixed-size dataset, they eventually become data-limited and stop improving in performance (cross-entropy loss). When we do the same for models pre-trained on a large language dataset, the slope in performance gains is merely reduced rather than going to zero. We calculate the effective data "transferred" from pre-training by determining how much data a transformer of the same size would have required to achieve the same loss when training from scratch. In other words, we focus on units of data while holding everything else fixed. We find that the effective data transferred is described well in the low data regime by a power-law of parameter count and fine-tuning dataset size. We believe the exponents in these power-laws correspond to measures of the generality of a model and proximity of distributions (in a directed rather than symmetric sense). We find that pre-training effectively multiplies the fine-tuning dataset size. Transfer, like overall performance, scales predictably in terms of parameters, data, and compute.

@NoaNabeshima https://aclanthology.org/2022.emnlp-main.9.pdf

@NoaNabeshima It might be tricky to get a praire dog call autoencoder to break them down into the right symbols. Similar things might slow down dolphin vocalization autoencoder progress?

https://youtu.be/y1kXCh496U0?si=Io-9Z5SfIaxTsu5S

curious