Abstract:
Agency is a system's capacity to steer outcomes toward a goal, and is a central topic of study across biology, philosophy, cognitive science, and artificial intelligence. Determining if a system exhibits agency is a notoriously difficult question: Dennett (1989), for instance, highlights the puzzle of determining which principles can decide whether a rock, a thermostat, or a robot each possess agency. We here address this puzzle from the viewpoint of reinforcement learning by arguing that agency is fundamentally frame-dependent: Any measurement of a system's agency must be made relative to a reference frame. We support this claim by presenting a philosophical argument that each of the essential properties of agency proposed by Barandiaran et al. (2009) and Moreno (2018) are themselves frame-dependent. We conclude that any basic science of agency requires frame-dependence, and discuss the implications of this claim for reinforcement learning.
Agency is a system's capacity to steer outcomes toward a goal, and is a central topic of study across biology, philosophy, cognitive science, and artificial intelligence. Determining if a system exhibits agency is a notoriously difficult question: Dennett (1989), for instance, highlights the puzzle of determining which principles can decide whether a rock, a thermostat, or a robot each possess agency. We here address this puzzle from the viewpoint of reinforcement learning by arguing that agency is fundamentally frame-dependent: Any measurement of a system's agency must be made relative to a reference frame. We support this claim by presenting a philosophical argument that each of the essential properties of agency proposed by Barandiaran et al. (2009) and Moreno (2018) are themselves frame-dependent. We conclude that any basic science of agency requires frame-dependence, and discuss the implications of this claim for reinforcement learning.
"Agency: It's Not What You Are, But How You're Framed!"
Headline: Agency Under the Microscope: New Research Challenges Our Fundamental Assumptions in AI
Exciting news for AI thinkers! A thought-provoking paper has just emerged that promises to spark crucial conversations and potentially reshape how we understand agency in artificial intelligence. "Agency Is Frame-Dependent" isn't just an incremental advancement; it's a bold and insightful exploration that argues the very nature of agency, that essential quality we attribute to intelligent systems, isn't a fixed, inherent trait. Instead, it proposes a more nuanced and perhaps more profound idea – agency is relative to the observer's chosen perspective!
We often have an intuitive grasp of what constitutes an agent: think of a robot navigating complex environments, an AI making strategic decisions, or even a sophisticated program optimizing a process. But this compelling new work, authored by a distinguished team from Google DeepMind and the University of Alberta, introduces a significant conceptual shift. They argue that whether we recognize agency in a system is not determined by a simple, objective checklist, but fundamentally depends on the "frame" through which we analyze it.
What exactly is this "frame of reference"? Imagine it as the set of underlying assumptions and analytical choices we make when examining a system. The authors, drawing on established definitions of agency from fields like biology, philosophy, and cognitive science, distill it into four key components:
Individuality: Does the system possess a well-defined boundary, clearly distinguishing "it" from its surrounding environment?
Source of Action: Does the system's behavior originate from its internal processes, or is it merely a passive recipient of external forces?
Goal-Directedness: Does the system exhibit behavior that is directed towards specific aims or objectives?
Adaptivity: Does the system modify its behavior in response to environmental feedback in pursuit of its goals?
This framework appears straightforward and logical. However, the paper's central argument is that each of these seemingly clear-cut conditions becomes inherently subjective and dependent on our analytical choices – our "frame of reference." Consider their insightful example of a thermostat. Where do we draw its boundary? Is it solely the device itself, or does it encompass the room it regulates? Is it genuinely acting as a "source" of action, or simply reacting mechanically to temperature fluctuations? And what truly constitutes its "goal"? Maintaining a set temperature? Or merely functioning according to its design?
The authors meticulously demonstrate that by adjusting our frame – by subtly altering how we define the system's boundaries, its causal relationships, its goals, and even our criteria for "adaptation" – we can fundamentally change whether we perceive agency. Suddenly, even the seemingly simple act of a rock rolling downhill could, under a specific frame, be interpreted as pursuing a "goal" of reaching a lower elevation. (This is, of course, an illustrative extreme to highlight the point.)
Headline: Agency Under the Microscope: New Research Challenges Our Fundamental Assumptions in AI
Exciting news for AI thinkers! A thought-provoking paper has just emerged that promises to spark crucial conversations and potentially reshape how we understand agency in artificial intelligence. "Agency Is Frame-Dependent" isn't just an incremental advancement; it's a bold and insightful exploration that argues the very nature of agency, that essential quality we attribute to intelligent systems, isn't a fixed, inherent trait. Instead, it proposes a more nuanced and perhaps more profound idea – agency is relative to the observer's chosen perspective!
We often have an intuitive grasp of what constitutes an agent: think of a robot navigating complex environments, an AI making strategic decisions, or even a sophisticated program optimizing a process. But this compelling new work, authored by a distinguished team from Google DeepMind and the University of Alberta, introduces a significant conceptual shift. They argue that whether we recognize agency in a system is not determined by a simple, objective checklist, but fundamentally depends on the "frame" through which we analyze it.
What exactly is this "frame of reference"? Imagine it as the set of underlying assumptions and analytical choices we make when examining a system. The authors, drawing on established definitions of agency from fields like biology, philosophy, and cognitive science, distill it into four key components:
Individuality: Does the system possess a well-defined boundary, clearly distinguishing "it" from its surrounding environment?
Source of Action: Does the system's behavior originate from its internal processes, or is it merely a passive recipient of external forces?
Goal-Directedness: Does the system exhibit behavior that is directed towards specific aims or objectives?
Adaptivity: Does the system modify its behavior in response to environmental feedback in pursuit of its goals?
This framework appears straightforward and logical. However, the paper's central argument is that each of these seemingly clear-cut conditions becomes inherently subjective and dependent on our analytical choices – our "frame of reference." Consider their insightful example of a thermostat. Where do we draw its boundary? Is it solely the device itself, or does it encompass the room it regulates? Is it genuinely acting as a "source" of action, or simply reacting mechanically to temperature fluctuations? And what truly constitutes its "goal"? Maintaining a set temperature? Or merely functioning according to its design?
The authors meticulously demonstrate that by adjusting our frame – by subtly altering how we define the system's boundaries, its causal relationships, its goals, and even our criteria for "adaptation" – we can fundamentally change whether we perceive agency. Suddenly, even the seemingly simple act of a rock rolling downhill could, under a specific frame, be interpreted as pursuing a "goal" of reaching a lower elevation. (This is, of course, an illustrative extreme to highlight the point.)
This is not merely abstract philosophical debate. The paper persuasively argues that this frame-dependence has profound implications for reinforcement learning (RL), a core methodology in contemporary AI development. If agency is not an objective, measurable property, how do we effectively design and evaluate genuinely agentic AI systems? How do we define progress in creating artificial agents? Are we potentially building systems that merely appear agentic within our chosen frame, while lacking a deeper, more fundamental form of agency?
This paper presents a compelling challenge, urging us to critically examine the inherent subjectivity embedded in our understanding of agency. It encourages us to move beyond simplistic, binary classifications of "agent" versus "non-agent" and to embrace a more nuanced, frame-relative perspective.
Let's open the discussion! Does this frame-dependent view of agency resonate with your understanding of AI and intelligent systems? Does it challenge your prior assumptions? What are the significant practical implications for the future development and assessment of AI? Share your thoughts in the comments below and let's collectively explore the nuances of this fascinating concept. Is agency truly "frame-dependent," or are there objective aspects that transcend our chosen perspectives?
This paper presents a compelling challenge, urging us to critically examine the inherent subjectivity embedded in our understanding of agency. It encourages us to move beyond simplistic, binary classifications of "agent" versus "non-agent" and to embrace a more nuanced, frame-relative perspective.
Let's open the discussion! Does this frame-dependent view of agency resonate with your understanding of AI and intelligent systems? Does it challenge your prior assumptions? What are the significant practical implications for the future development and assessment of AI? Share your thoughts in the comments below and let's collectively explore the nuances of this fascinating concept. Is agency truly "frame-dependent," or are there objective aspects that transcend our chosen perspectives?
5 Key Takeaways from "Why AI Is A Philosophical Rupture"
1. AI Challenges the Human-Machine Dichotomy
For centuries, humans have distinguished themselves from machines through concepts like intelligence, creativity, and agency. AI disrupts this binary by demonstrating intelligence and agency without being alive or conscious. This philosophical rupture forces us to rethink what it means to be human and challenges the scaffolding of modern thought that has persisted for 400 years.
2. AI as a New Form of Intelligence
AI is not merely a tool but a form of intelligence that complements human cognition. It operates on scales beyond human comprehension, identifying patterns and logical structures in data that we cannot perceive. This opens up new possibilities for collaboration, where human and AI intelligences can work symbiotically to achieve insights neither could accomplish alone.
3. The Potential for a New "AIxial Age"
Just as the invention of writing transformed human thought and gave rise to abstract reasoning and self-reflection, AI has the potential to redefine our understanding of mind, creativity, and reality. This "AIxial Age" could lead to new philosophical frameworks and ways of experiencing the world, much like the Axial Age did millennia ago.
4. AI and the Concept of Interiority
AI can make the self visible to itself in unprecedented ways. By analyzing personal data, an AI system could reveal patterns of thought and behavior, enabling individuals to reflect on and transform themselves. This aligns with ancient philosophical practices of self-examination but extends them through the unique capabilities of AI.
5. Symbiosis Between Humans and AI
The future may hold a deep symbiosis between humans and AI, akin to natural symbiotic relationships like fungi and trees. This partnership could allow humans to think in ways previously unimaginable, while AI benefits from human guidance. Such a relationship blurs the boundaries between human and machine, creating a new, indistinct cognitive space.
AI is not just a technological advancement; it is a profound philosophical event that invites us to reimagine our place in the world. Let’s embrace this rupture as an opportunity to explore new ways of thinking and being.
1. AI Challenges the Human-Machine Dichotomy
For centuries, humans have distinguished themselves from machines through concepts like intelligence, creativity, and agency. AI disrupts this binary by demonstrating intelligence and agency without being alive or conscious. This philosophical rupture forces us to rethink what it means to be human and challenges the scaffolding of modern thought that has persisted for 400 years.
2. AI as a New Form of Intelligence
AI is not merely a tool but a form of intelligence that complements human cognition. It operates on scales beyond human comprehension, identifying patterns and logical structures in data that we cannot perceive. This opens up new possibilities for collaboration, where human and AI intelligences can work symbiotically to achieve insights neither could accomplish alone.
3. The Potential for a New "AIxial Age"
Just as the invention of writing transformed human thought and gave rise to abstract reasoning and self-reflection, AI has the potential to redefine our understanding of mind, creativity, and reality. This "AIxial Age" could lead to new philosophical frameworks and ways of experiencing the world, much like the Axial Age did millennia ago.
4. AI and the Concept of Interiority
AI can make the self visible to itself in unprecedented ways. By analyzing personal data, an AI system could reveal patterns of thought and behavior, enabling individuals to reflect on and transform themselves. This aligns with ancient philosophical practices of self-examination but extends them through the unique capabilities of AI.
5. Symbiosis Between Humans and AI
The future may hold a deep symbiosis between humans and AI, akin to natural symbiotic relationships like fungi and trees. This partnership could allow humans to think in ways previously unimaginable, while AI benefits from human guidance. Such a relationship blurs the boundaries between human and machine, creating a new, indistinct cognitive space.
AI is not just a technological advancement; it is a profound philosophical event that invites us to reimagine our place in the world. Let’s embrace this rupture as an opportunity to explore new ways of thinking and being.
As awesome as it is, it shows how innovation in AI targets almost exclusively the sweet spot between LLM pattern matching and social norms. Predicting expected social behavior, financial behavior, work behavior, etc.
We need AI innovation wandering uncharted territories.
We need AI innovation wandering uncharted territories.
"Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approach"
The research paper introduces a novel language model architecture that enables scaling test-time computation through latent reasoning. This approach contrasts with conventional methods that increase computation by generating more tokens or relying on chain-of-thought verbalization. Instead, the proposed model, named Huginn-0125, employs a recurrent depth block which iterates in latent space, effectively "thinking" for an arbitrary number of steps at test-time.
Problem Addressed:
Traditional language models primarily scale computation by increasing model size and training data, which is resource-intensive. More recent methods employ chain-of-thought reasoning, but these often require specialized training data and can be limited by context window size. The authors argue that forcing internal reasoning to be verbalized into a single next token is inefficient and that models could be more capable if they could reason natively in a continuous latent space.
Proposed Solution: Recurrent Depth Architecture:
The paper proposes a depth-recurrent architecture composed of three main parts:
Prelude (P): Embeds input tokens into a latent space using multiple transformer layers.
Recurrent Block (R): The core of the architecture, iteratively processes and updates a hidden state in latent space. This block is repeated multiple times at test-time to increase computation.
Coda (C): Un-embeds the final latent state and contains the prediction head to generate output tokens.
The model is trained by randomly sampling the number of recurrent iterations during training using a log-normal Poisson distribution. To manage computational cost during training, truncated backpropagation is used, limiting backpropagation to only the last few iterations of the recurrent unit.
Key Advantages of Latent Reasoning Approach:
No Bespoke Training Data: Unlike chain-of-thought, it doesn't require specialized training data with reasoning demonstrations. Standard training data can be used, and the model learns to enhance reasoning at test time.
Memory Efficiency: Requires less memory than chain-of-thought models because it doesn't necessitate extremely long context windows.
Communication Efficiency: Recurrent-depth networks are compute-heavy but parameter-light, reducing communication costs between accelerators during training, especially beneficial with slower interconnects.
Prioritization of "Thinking": By design, the architecture favors models that solve problems through "thinking" (meta-strategies, logic, abstraction) rather than just memorization.
Capturing Non-Verbal Reasoning: Latent reasoning can potentially capture facets of human reasoning that are difficult to verbalize, like spatial thinking or intuition.
Experimental Results:
The authors scaled a proof-of-concept model to 3.5 billion parameters and 800 billion tokens. Experiments on reasoning benchmarks (ARC challenge, GSM8K, OpenBookQA) show significant performance improvements with increased test-time recurrence, sometimes reaching performance levels equivalent to models with much larger parameter counts. The model outperforms older open-source models like Pythia and is comparable to early OLMo models on standard benchmarks. Notably, the recurrent model demonstrates strong performance in mathematical and coding benchmarks. Trajectory analysis in latent space reveals complex computational behaviors emerging with scale, such as orbital patterns and "sliders" for numerical computations.
Implications and Future Work:
The paper argues that latent reasoning, enabled by recurrent depth architectures, offers a promising third axis for scaling language model performance, complementing parameter scaling and verbalized inference scaling. It opens avenues for future research, including:
Exploring post-training schemes to enhance model capabilities (fine-tuning for recurrence compression, reinforcement learning for hardness levels).
The research paper introduces a novel language model architecture that enables scaling test-time computation through latent reasoning. This approach contrasts with conventional methods that increase computation by generating more tokens or relying on chain-of-thought verbalization. Instead, the proposed model, named Huginn-0125, employs a recurrent depth block which iterates in latent space, effectively "thinking" for an arbitrary number of steps at test-time.
Problem Addressed:
Traditional language models primarily scale computation by increasing model size and training data, which is resource-intensive. More recent methods employ chain-of-thought reasoning, but these often require specialized training data and can be limited by context window size. The authors argue that forcing internal reasoning to be verbalized into a single next token is inefficient and that models could be more capable if they could reason natively in a continuous latent space.
Proposed Solution: Recurrent Depth Architecture:
The paper proposes a depth-recurrent architecture composed of three main parts:
Prelude (P): Embeds input tokens into a latent space using multiple transformer layers.
Recurrent Block (R): The core of the architecture, iteratively processes and updates a hidden state in latent space. This block is repeated multiple times at test-time to increase computation.
Coda (C): Un-embeds the final latent state and contains the prediction head to generate output tokens.
The model is trained by randomly sampling the number of recurrent iterations during training using a log-normal Poisson distribution. To manage computational cost during training, truncated backpropagation is used, limiting backpropagation to only the last few iterations of the recurrent unit.
Key Advantages of Latent Reasoning Approach:
No Bespoke Training Data: Unlike chain-of-thought, it doesn't require specialized training data with reasoning demonstrations. Standard training data can be used, and the model learns to enhance reasoning at test time.
Memory Efficiency: Requires less memory than chain-of-thought models because it doesn't necessitate extremely long context windows.
Communication Efficiency: Recurrent-depth networks are compute-heavy but parameter-light, reducing communication costs between accelerators during training, especially beneficial with slower interconnects.
Prioritization of "Thinking": By design, the architecture favors models that solve problems through "thinking" (meta-strategies, logic, abstraction) rather than just memorization.
Capturing Non-Verbal Reasoning: Latent reasoning can potentially capture facets of human reasoning that are difficult to verbalize, like spatial thinking or intuition.
Experimental Results:
The authors scaled a proof-of-concept model to 3.5 billion parameters and 800 billion tokens. Experiments on reasoning benchmarks (ARC challenge, GSM8K, OpenBookQA) show significant performance improvements with increased test-time recurrence, sometimes reaching performance levels equivalent to models with much larger parameter counts. The model outperforms older open-source models like Pythia and is comparable to early OLMo models on standard benchmarks. Notably, the recurrent model demonstrates strong performance in mathematical and coding benchmarks. Trajectory analysis in latent space reveals complex computational behaviors emerging with scale, such as orbital patterns and "sliders" for numerical computations.
Implications and Future Work:
The paper argues that latent reasoning, enabled by recurrent depth architectures, offers a promising third axis for scaling language model performance, complementing parameter scaling and verbalized inference scaling. It opens avenues for future research, including:
Exploring post-training schemes to enhance model capabilities (fine-tuning for recurrence compression, reinforcement learning for hardness levels).