Anthropic hires lawyers to set norms for AI agent negotiations
Anthropic is consulting law firms and hiring experts to establish fair negotiation standards for AI agents before they handle contracts and pricing.
Key points
- Anthropic is working with law firms to develop norms for AI agents that negotiate prices and contracts.
- The company is hiring Conceptual Reasoning Fellows, including lawyers, to test Claude on judgment-heavy questions.
- Experiments show stronger AI models negotiate better deals, raising concerns about potential societal inequality.
Anthropic is actively consulting with law firms to establish fair negotiation norms for artificial intelligence agents. According to the Financial Times, these agents are expected to haggle over prices, resolve customer complaints, and improve contract offers on behalf of users. The company aims to define these standards before such autonomous systems begin cutting deals in the wild.
This initiative addresses the challenge of ensuring AI agents behave equitably when handling sensitive economic interactions. Anthropic has already conducted experiments demonstrating that more powerful models tend to negotiate significantly better deals than their predecessors. The company recognizes that this capability could potentially widen societal inequality if left without proper guardrails or established norms for fair play.
Building Evaluation Frameworks
To support this effort, Anthropic is hiring lawyers and humanities scholars as Conceptual Reasoning Fellows. These fellows will spend four months testing the Claude model on complex questions involving judgment. The roles focus on fields such as law, philosophy, cognitive science, economics, and political science, where reliable training data is often scarce.
The compensation for these fellowship positions ranges from $100,000 to $167,000. The primary goal is to improve Claude’s performance in areas where right and wrong answers are rarely clear-cut. By leveraging expert knowledge, Anthropic seeks to build robust test cases and evaluations that strengthen the model’s reasoning capabilities in these nuanced domains.
The Data Challenge
The push for human expertise traces back to a fundamental data problem in AI development. Improving models in math and code is relatively straightforward because verifiable data is plentiful or easy to generate. In contrast, the humanities lack this abundance of clear, objective data, making it difficult to train models effectively without human guidance.
Justin Curl, a member of technical staff researching law, institutions, and AI at Anthropic, argues that defining quality in law is notoriously hard. This ambiguity requires experts to construct specific evaluations that can measure the model’s progress. Without these human-built benchmarks, it remains difficult to assess whether an AI agent is negotiating fairly or simply optimizing for its own advantage.
What should you do?
Organizations should monitor developments in AI negotiation norms as these systems become more prevalent in commercial transactions. Professionals in legal and compliance roles should prepare for a future where AI agents may represent counterparties in contract discussions. Staying informed about the standards Anthropic and other vendors establish will help ensure that internal processes remain aligned with emerging industry best practices.
What to do and how to stay safe: Anthropic
- Review current procurement and contract policies to identify areas where AI agents might soon be used for negotiation.
- Establish internal guidelines for human oversight when AI systems are involved in high-stakes financial decisions or contract modifications.
- Monitor updates from AI vendors regarding the ethical frameworks and negotiation norms they are developing for their agents.
- Consult with legal counsel to understand how emerging AI negotiation standards might impact existing contractual obligations and liabilities.
Step-by-step guide: Prompt Injection Warning Signs: How to Spot AI Manipulation
General security guidance from the Threat Times newsroom. It is not confirmed advice from the organisations named in this story.
Frequently asked questions
Why is Anthropic hiring lawyers for its AI development team?
Anthropic is hiring lawyers as Conceptual Reasoning Fellows to test Claude on judgment-heavy questions in law and other humanities fields where training data is scarce.
What are the AI agents expected to negotiate?
According to the Financial Times, the AI agents are expected to haggle over prices, resolve complaints, and improve contract offers on behalf of users.
What is the salary range for the Conceptual Reasoning Fellows?
The fellowship positions offer compensation ranging from $100,000 to $167,000 for a four-month period focused on testing and evaluation.
