Cape Eleuthera Institute

Recent advances in recurrent neural network architectures, such as Mamba and RWKV, have enabled RNNs to match or exceed the performance of equal-size transformers in terms of language modeling perplexity and downstream evaluations, suggesting that future systems may be built on completely new architectures. In this paper, we examine if selected interpretability methods originally designed for transformer language models will transfer to these up-and-coming recurrent architectures. Specifically, we focus on steering model outputs via contrastive activation addition, on eliciting latent predictions via the tuned lens, and eliciting latent knowledge from models fine-tuned to produce false outputs under certain conditions. Our results show that most of these techniques are effective when applied to RNNs, and we show that it is possible to improve some of them by taking advantage of RNNs' compressed state.

Abstract Climate-driven shifts in herbivores, temperature and nutrient runoff threaten coastal ecosystem resilience. However, our understanding of ecological resilience, particularly for foundation species, remains limited due to a rarity of field experiments that are conducted across appropriate spatial and temporal scales and that investigate multiple stressors. This study aimed to evaluate the resilience of a widespread tropical marine plant (turtlegrass) to disturbances across its geographic range and how this is impacted by environmental gradients in (a)biotic factors. We assessed the resilience (i.e. recovery) of turtlegrass to a simulated disturbance (complete above- and belowground biomass removal) over a year. Contrary to temperate studies, higher temperature generally enhanced seagrass recovery. While nutrients and light availability had minimal impact, combined high levels of nutrients and herbivore grazing (meso and megaherbivore) reduced aboveground recovery. Our results suggest that the resilience of some tropical species, especially in cooler subtropical waters, may initially increase with warming.

Oceanic whitetip sharks Carcharhinus longimanus aggregate at Columbus Point, Cat Island, The Bahamas, in at least April and May. We show that signs of putative mating activity (i.e. bite wounds) on mature females, which dominate the aggregation, are exceedingly rare at that time, but may be more common in July. Male plasma testosterone concentrations also suggest that spermatogenesis occurs in April and May and copulation is underway or has just ended by July. We support these preliminary data with direct observations of putative precopulatory behavior made at Columbus Point in July 2019. Using 25 min of video footage from the event, we identify common shark mating behaviors such as echelon or parallel swimming, following, shielding, nosing, and biting. We also describe the ‘circle-back’, a newly observed behavior where the male orients to the trail of the female, possibly to investigate olfactory cues. Based on these 3 parallel lines of evidence, we hypothesize that mating could occur in the eastern Bahamas as early as May, but is probably concentrated in midsummer (i.e. July), and that Columbus Point, Cat Island, may be a mating habitat for the oceanic whitetip shark.

Zheng Xin Yong, Ruochen Zhang, Jessica Forde, Skyler Wang, Arjun Subramonian, Holy Lovenia, Samuel Cahyawijaya, Genta Winata, Lintang Sutawika, Jan Christian Blaise Cruz, Yin Lin Tan, Long Phan, Long Phan, Rowena Garcia, Thamar Solorio, Alham Aji. Proceedings of the 6th Workshop on Computational Approaches to Linguistic Code-Switching. 2023.