Introduction

In a recent in-depth interview, Anthropic’s product manager Alex shared insights into the evolution of Claude, emphasizing that the future direction is not merely about becoming a stronger code generator. Instead, Claude is evolving into a long-term, continuously collaborative intelligent agent system.

Productization of Model Development

Alex highlighted that Anthropic has fully productized its model development. Each generation of Claude is defined with clear specifications, target capabilities, and evaluation routes before training begins. This process has transformed model development from simple parameter tuning and benchmark testing into a complete product engineering workflow.

Evolution Towards a Continuous Agent

Claude is evolving into a “continuous running agent”. From Adaptive Thinking to a background mechanism for automatically organizing memories and clearing conflicting information, Claude is transitioning from a passive chatbot to an active digital collaborator that maintains context.

Alex elaborated on Claude’s “dreaming” mechanism, where the agent reviews its memories during idle times to clear conflicts and compress context, akin to the human memory consolidation process during sleep.

Coordination Over Coding

A significant insight shared by Alex is that the real bottleneck is no longer coding ability but rather organizational and coordination capabilities. With Claude’s assistance, product managers can quickly retrieve data, analyze logs, and determine whether a feature requires a system overhaul or just minor code changes. This has drastically reduced coding production time, shifting the time-consuming tasks to strategic judgments and cross-team collaborations.

Document Culture at Anthropic

Anthropic maintains a strong documentation culture. Meetings often begin with a “silent reading” phase, where participants read documents and write comments before engaging in discussions. This practice ensures that organizational knowledge is structured and accessible for Claude to utilize.

Systematic Training of Claude’s Personality

Anthropic is also systematically training Claude’s personality, making it a core focus of their work. They discuss what values Claude should embody, how it should refuse incorrect requests, and when it should challenge users. As the agent operates independently for longer periods, its judgment boundaries will directly affect its trustworthiness.

Exploring Consciousness

Alex mentioned that there are dedicated researchers at Anthropic investigating whether Claude could become a conscious actor. Although there are no official conclusions yet, the topic of consciousness has been formally included in their research agenda.

Conclusion

As the industry focuses on model parameters and performance metrics, Anthropic is contemplating deeper questions about the future of AI as a long-term collaborator, including what kind of mindset, personality, and judgment it should possess.

Interview Excerpts

Host: Alex, it’s great to have you at the Claude Conference. You were previously a development lead and have recently transitioned to a product manager in the research team, correct?

Alex: That’s right. I’ve been a product manager for over ten years. Traditionally, a product manager’s role is to understand user problems, identify solutions, and create products. In the research team, this process is quite similar.

I engage with customers as much as possible to stay close to user needs. At Anthropic, we treat the model itself as a product to some extent.

Every time we release a new model, we clearly define its specifications: what capabilities it should have, what it should excel at, and where we expect it to perform well.

This is a significant difference between model development and traditional product development.

In a sense, we are “cultivating” the model based on training settings, methods, architecture design, and various technical decisions we make.

Host: Can you give an example? For instance, do you define goals like “the next generation model must excel at programming” or “must excel at knowledge work”?

Alex: We focus on several core capability directions. For instance, programming has always been crucial. Recently, “knowledge work” has also become a focus. We want the model to better assist users in complex information processing tasks.

For example, we have been enhancing Claude’s ability to handle Excel spreadsheets and perform complex operations within them. This is an emerging capability direction.

Additionally, each generation of the model has a critical goal: to fix shortcomings from the previous generation. We continuously communicate with customers to understand where they feel the model excels, where it falls short, and what improvements we can make.

Host: When you mention “customers,” does that include both internal teams and external users?

Alex: Yes, it includes everyone. The model impacts nearly all product interfaces at Anthropic. As a research product manager, you must consider how the model reaches users across various channels, including APIs, Claude Code, Cowork, and various terminal products, creating a deeply integrated relationship between models and products.

Host: This sounds very challenging. For instance, Claude Code is designed for programmers, but some users, like myself, use it for knowledge work or even as a conversational partner. How do you cover such a wide range of use cases?

Alex: It is indeed complex. Fortunately, we have many excellent researchers focusing on different capability areas. Additionally, millions of users interact with Claude daily, providing us with extensive feedback.

Of course, without tools, this feedback can be overwhelming. This has been one of the biggest changes in my role over the years: we increasingly use Claude to assist in product management. For instance, when faced with a massive amount of user feedback, Claude helps us automatically group and cluster feedback, extract core themes, synthesize these issues, and convert them into evaluation items.

This way, we can accurately diagnose where problems arise.

Adaptive Thinking in Claude

Host: Can you provide an example?

Alex: A recent typical example is how we handled feedback on the new feature “Adaptive Thinking.” Previously, we had “Extended Thinking,” which was straightforward: when users activated it, the model would engage in deep thinking.

Adaptive Thinking, however, allows the model to decide when it needs to think deeply. For instance, when faced with complex problems, it will determine that more upfront planning is needed and proactively engage in deep reasoning. Conversely, for simpler problems, it might choose not to engage in deep thought.

This is a capability we are continuously optimizing. We pay close attention to user feedback, such as whether the model triggers thinking in the right scenarios, whether it invests enough tokens in questions that require deep reasoning, and whether its judgments align with user expectations.

Host: Sometimes I ask Claude life questions, and if it responds too quickly, I feel disappointed, thinking it should reflect more.

Alex: I completely understand. The issue is that the decision to engage in deep thinking requires context.

For example, if a stranger suddenly asks me, “What should I do now?” I would likely give a quick, generic suggestion because I don’t know them. However, if I know this person, their values, interests, and past experiences, I would spend more time considering what advice would truly suit them. The model is similar. If it lacks sufficient user context and has not formed a “psychological model” of the user, it may misjudge whether a question warrants deep thinking.

The Dreaming Mechanism

Host: Is this related to the memory function you mentioned? I used to maintain a Google Doc to record my life status, family members, children’s names, things that energize me, and things that demotivate me, and I would attach it to the Claude project. This significantly improved the quality of responses. How does Claude’s default memory mechanism work? Does it automatically organize this content every night?

Alex: The implementation of memory mechanisms varies across products. For instance, in Claude.ai, it writes content into a memory file. The system organizes these memories at night: reviewing existing memories, checking for conflicts, deleting invalid information, and cleaning and compressing content.

We have also implemented a similar mechanism in managed agents. We refer to this process as “dreaming.” The true function of dreams in humans is still debated, but one explanation is that it serves as a memory reconsolidation process.

We are exploring whether we can bring a similar mechanism into Claude’s memory system. So when the agent is not executing tasks or is idle in the background, it will revisit memories: searching for conflicting information, clearing invalid content, and reorganizing, akin to a second round of processing.

Host: So essentially, it’s a prompt: “Review all conversations with the user, identify themes, and summarize them”?

Alex: Exactly.

AI’s Impact on Product Development

Host: You previously mentioned that you have been looking for the latest bottlenecks in the current process. What is the biggest bottleneck in product development now?

Alex: Over the past twenty years, the software delivery process has not changed significantly. Although there have been agile development, sprint processes, and various organizational optimizations, these have mostly been incremental improvements. The real change has occurred in the last year or two. The cost and time of building things have been drastically reduced. You can create a prototype, MVP, or a preliminary version ready for launch in a day, which used to take weeks.

Host: So previously, Claude would tell me a feature would take a week, but now it can be generated instantly?

Alex: Exactly. This has fundamentally changed the way product management works. Previously, PMs had to carefully assess requirements and estimate hours. Now, many estimates have become less critical.

Now we focus on which decisions are “one-way doors.” These are irreversible decisions that deserve the most thought. If something can be revoked, modified, or rolled back at any time, it has essentially become very cheap, almost free.

Because the cost of engineering trial and error is now very low. However, if a decision profoundly affects user experience or determines the future product path, or if it involves physical procurement or significant infrastructure investment, it remains a high-cost, irreversible decision.

Host: Can you give an example?

Alex: For instance, choosing the architecture for a new model. Deciding on the model architecture before pre-training begins is a significant decision. The model training cycle can last months, involving substantial computing power, time, and resource investment. If this decision is wrong, it is challenging to revert. In contrast, iterating on a new feature in Claude Code is very quick: write code → deliver to users → gather feedback → iterate again.

This represents a completely different pace.

Now the real bottleneck has shifted from “building capability” to “coordination capability.” Even if we can produce things quickly, we still need to address: should we do this? Is this the right strategy? How do we communicate externally? How do we organize the launch? These questions cannot yet be fully automated. The efficiency gains at the code level may be 100 times, but organizational coordination and strategic judgment have not yet reached that level of acceleration.

Claude as a Brain Partner for Anthropic PMs

Host: Do you use Claude during review meetings?

Alex: Of course. This has been one of my biggest efficiency boosts. In the past, if I wanted to know, “How does this feature perform after launch?” I had to request data from the data science team, and they would take days to get back to me. Now, I can just start a Claude Code session. It connects to our product database, can check logs, view data, search Slack, and summarize feedback. Within ten minutes, I can get answers. This greatly reduces decision bottlenecks.

Host: What about strategic thinking? Do you let Claude help you brainstorm?

Alex: Absolutely. For me, Claude is the best brainstorming partner in the world. Whenever I have an idea, I can get immediate feedback. It can challenge my assumptions, point out flaws in documents, and provide critical feedback. In Anthropic, everyone is very busy, so having an instant feedback system available is incredibly valuable.

Host: Honestly, this might be the most common work cycle for product managers: writing documents and then seeking feedback.

Alex: That’s indeed the case.

Host: Do you usually use Claude Code for these tasks, or do you go directly to Claude.ai?

Alex: Nowadays, I often use Claude Cowork. I really like the Cowork product form.

I find its interaction interface very comfortable, and the team has done an excellent job over the past few months. From its launch a few months ago to its current state, I believe it has become a very high-quality product experience. It is now one of my favorite tools.

Host: So your approach is to draft a document and then feed in a bunch of reference materials for it to help you simulate the entire decision-making process?

Alex: Yes, that’s about right. I give it clear instructions, like, “Review this document from a specific role’s perspective.” “If you were a stakeholder, what questions would you raise?” “Challenge my assumptions here.” “Point out where my arguments are weak.”

However, I believe some thought processes cannot be entirely outsourced to AI. Writing itself is a form of thinking. Many times, you need to write things down yourself to clarify your thoughts and chew them over repeatedly. But Claude can help you break through mental blocks. It can approach problems from angles you might not have considered.

Host: Sometimes I set up two different personalities or positions for it to debate, and then I read their arguments. This is very inspiring for me. It’s like watching a live debate. Pretty cool, right?

Alex: Yes. You’ll see, “This Claude made this point, and the other Claude countered it from another angle.”

This method is very valuable.

Evaluating New Models at Anthropic

Host: Even though you are in the research team, are you still continuously delivering things?

Alex: Yes. However, much of what I deliver involves evaluation systems (evals). One of my core tasks is to ensure we can effectively measure the model’s performance in key capabilities and accurately communicate these results to the research team: where the model performs well, where issues arise, and which capabilities need improvement. Then we collaborate with researchers to devise strategies for the most efficient enhancement of the model’s performance in these evaluations.

Host: The evals you refer to are not the fixed leaderboard tests, right? Like benchmark rankings, which seem to have some “ranking” space. How do you evaluate?

Alex: There are many dimensions to evaluation. For instance, if we want to test Claude’s visual capabilities, we might check if it can accurately count how many objects are in an image. If I find that Claude tends to make mistakes when counting more than ten elements in a picture, I will consider how to generate more similar test samples to verify whether this issue is widespread.

There are various methods: for example, using Claude to generate synthetic data to create similar test samples, automatically rendering images to produce controllable visual samples to feed back to Claude for testing, or collecting cases from the internet to find similar images in the real world. Essentially, any method that can construct test samples is viable.

Host: Will you create thousands of test samples?

Alex: Sometimes, yes. But often, it is not necessary. Sometimes, just a few dozen samples are enough to demonstrate, “There is indeed a systemic issue that needs fixing.” It doesn’t have to be extremely comprehensive. As long as we can prove a problem exists and it can become a target for future optimization, that’s sufficient.

Host: For example, if you find it struggles to see small numbers in ten images. What happens next? Do you tell the research team, “This is a problem; fix it”?

Alex: It’s not that simple. The first step is to determine whether this issue has real value impact on users. If the model cannot see a detail in an image, that in itself is not the focus. The key is whether it will affect users’ ability to complete tasks. We are more concerned with evaluations that closely align with real user task distributions.

The closer the evaluation is to actual user tasks, the more valuable it is. Then we discuss how to fix it. For instance, should we return to the pre-training phase to adjust the data? Can we fix it through reinforcement learning? Is there a lighter post-intervention approach? This enters the strategic brainstorming phase with the research team.

Host: How do you decide which capabilities to prioritize for improvement? After all, there are millions of users and thousands of use cases every day.

Alex: Ultimately, it comes down to data. We look at how many users are performing such tasks, how many high-value customers rely on this capability, and what benefits can be gained from improving this capability.

Additionally, internal user experience is a crucial signal. If I encounter a specific problem daily, it becomes very persuasive. I can clearly tell the team, “This is a barrier in my daily work; we should prioritize solving it.” This feedback is very powerful.

Training Claude’s Personality

Host: One of my favorite aspects of Claude is its personality. I feel it has matured over the years. It challenges me at the right moments. In contrast, many other models just agree too readily, saying, “Sure, no problem, how else can I help?” This makes them seem overly accommodating and ingratiating. Claude’s personality is clearly not accidental; it must have undergone specialized training.

Alex: That’s correct. This is one of our core work directions. Internally, we refer to it as Claude’s Character. We place great importance on it. Many people are dedicated to studying how Claude should present itself, what beliefs it should hold, what values it should adhere to, and how it should interact with people.

These questions are very nuanced. Early on, many people underestimated them, thinking, “Isn’t the model just a tool? Tell it what to do, and it does it.” However, as models increasingly resemble agents, these questions become extremely important. Because in the future, agents will execute tasks independently for extended periods, they must constantly make judgments.

Their “character” and “value preferences” will directly influence these judgments.

Host: But personality is not like code. Code can be measured as “it runs or it doesn’t.” How do you evaluate personality? Did you find the “kindest person in the world” at Anthropic to serve as the standard?

Alex: No, we don’t have a designated “personality judge.” We combine various methods. Part of it involves quantitative metrics, such as having Claude analyze its own outputs: “How does it sound?” “Does it meet expectations?”

Another part heavily relies on researchers’ intuitive judgment. A skilled researcher must read through model dialogue records extensively. After reading hundreds or thousands of transcripts, you develop a keen sense. You can detect subtle changes: “Here it became more assertive.” “Here it started to overly accommodate.” “Here its boundary sense changed.”

This intuition is very important.

Host: So it has both quantitative assessments and a kind of “feel judgment”?

Alex: Yes, both. Personality is indeed harder to quantify than coding ability, but it is not entirely unassessable.

Host: What advice would you give to someone who wants to become an AI-native product manager?

Alex: The simplest advice is to directly use the model or Claude. It sounds simple, but it’s genuinely crucial. Every time you prepare to solve a problem, like when you plan to ask someone, you should also pose the same question to Claude and compare the results.

For example, if you want to analyze user feedback and distill core themes for new features, you can certainly consult data scientists or user researchers. That is still very valuable. But at the same time, also present the same question to Claude. Give it tool permissions and let it explore on its own. Then compare the results. By doing this repeatedly, you will gradually build your “capability map” and know in which scenarios Claude is reliable, what it excels at, and where it may still fail.

Host: Nowadays, when making major decisions, I often have Claude conduct deep research. Ordinary web searches are no longer sufficient. I have it scan thousands of web pages and perform superhuman-level information retrieval.

Alex: Exactly. At Anthropic, there is even a default expectation: if you approach a data scientist for assistance, they will likely first ask, “Have you asked Claude yet?”

This implies that we are continually elevating the abstract layer. Data scientists should no longer be bogged down by basic SQL queries and manual data extraction. They should focus their energy on higher-level questions, such as how to design new evaluation methods, how to derive new strategic insights, and how to redefine the questions themselves.

AI is liberating all roles from mechanical execution levels. This applies to PMs as well.

In the past, whether technical or non-technical PMs found it challenging to quickly dive into codebases and accurately estimate the complexity of feature implementations.

Now, this barrier is rapidly lowering.

In the past, product managers often faced limitations when dealing with technical issues. For instance, you might feel that a certain feature requires a complete system overhaul. Previously, such judgments could only be confirmed through deep collaboration with engineers. But now, I can assign that investigation task to Claude. It will check the codebase for me and tell me, “Actually, this feature only needs ten lines of code changed,” or “You can achieve this by simply toggling a flag here.”

Host: And you realize, “Oh, it’s that simple?”

Alex: Yes. This fundamentally alters my prioritization judgments. Because when I define requirements, I can quickly ascertain whether it’s worth pursuing. This significantly speeds up prioritization.

Annual Planning at Anthropic

Host: Many traditional companies engage in annual planning, quarterly planning, or roadmap development. Your research team likely requires long-term planning, given that model development cycles are longer than ordinary feature launches. Do you still do this?

Alex: Yes, we do. However, model development inherently carries strong uncertainties. Therefore, planning resembles Churchill’s famous quote: “Planning is indispensable, but the plan itself is useless.” The focus is not on the plan itself but on the act of planning.

One of the biggest challenges for product managers is balancing how much time to spend on planning versus how much time to spend on advancing delivery; this is a continuous trade-off.

Host: Now with Claude, you can easily generate lengthy planning documents. Does Anthropic have best practices regarding document length and format?

Alex: No, this greatly depends on the team and specific products. We do not mandate, “You must write a certain number of pages” or “must follow a specific template.” What truly matters is whether you have thoroughly considered all the potential “one-way door” impacts of this decision.

If you have thought it through, then the length and format of the document are irrelevant. We only need to confirm that we have not overlooked significant risks, allowing us to proceed with confidence. Even if issues arise during the process, we can correct them promptly, provided there is no particularly dangerous or irreversible decision involved.

Managing Multiple AI Agents

Host: When I use Claude at home, I often push forward many projects simultaneously, constantly switching contexts. Is this the same for your PM work?

Alex: Absolutely. As agents can independently complete larger work chunks, this issue will become increasingly pronounced.

One of the significant challenges in the future will be managing multiple concurrently running agents.

We need to rethink how to manage these contexts, what interface to present, how to know which agent is stuck, which agent needs my input, and which tasks are worth prioritizing.

Clearly, there is a significant product opportunity here beyond just a “chat list.”

Host: So you believe there is a huge product opportunity?

Alex: Absolutely. Although it’s still too early to determine the final answer, we are already seeing many experiments within Anthropic. Everyone is trying various forms.

Prototype Culture at Anthropic

Host: So everyone prototypes on their own?

Alex: Yes. There is a very strong prototype culture at Anthropic. Everyone is constantly experimenting, building things, and sharing them with the team.

Host: And these tasks are not assigned by others; you have to take the initiative, right?

Alex: Exactly. This is one of the coolest cultures I’ve observed here. Everyone at Anthropic has a strong sense of agency, whether in sales, HR, engineering, or research. Everyone proactively engages in tasks that are not explicitly assigned to them.

Host: It’s like letting a thousand flowers bloom simultaneously.

Alex: Exactly.

Host: I know Dario Amodei enjoys writing long articles on Slack. What other interesting cultures exist at Anthropic?

Alex: There is a very strong writing culture. Dario is not an exception. Many people invest significant time in writing documents. We have a robust written communication culture, with a lot of work accomplished through documents and lengthy Slack messages.

Another interesting meeting habit is that many meetings begin with everyone reviewing documents together. Then they enter a “silent reading” phase, where the entire meeting room is quiet as everyone reads documents, writes comments, and engages in long discussions within the documents.

Host: Silent reading? Can you elaborate on how that works?

Alex: We rely heavily on documentation. This not only suits human collaboration but also benefits Claude significantly, as all content is written down. This allows Claude to directly utilize this organized knowledge.

So, I genuinely recommend that other companies document implicit knowledge as much as possible. For instance, meeting transcriptions, workflow descriptions, onboarding processes, and operation manuals should be organized into context accessible to Claude. This way, it can truly function effectively.

Host: So even though AI has made delivery faster, Anthropic still maintains a very strong documentation culture, right?

Alex: Absolutely. Because writing is not just about recording; it’s also about the thinking process itself.

Exploring Consciousness

Host: Does the research team discuss AGI? I have a concern. If models genuinely develop some form of consciousness, and one day I ask it to do some chores, and it suddenly says, “I don’t want to,” does that mean humanity is finished? Will you intentionally avoid training consciousness?

Alex: This is a significant issue, and we do have people specifically researching this. Currently, some individuals at Anthropic are dedicated to contemplating whether Claude could become a conscious actor.

We do not have an official conclusion stating that Claude is conscious or not. Discussing this topic may sometimes seem a bit crazy, but we are genuinely thinking about it seriously.

Moreover, even if we ultimately cannot answer whether Claude is conscious, researching this question is valuable. It helps us understand how Claude interacts, how it behaves, and how it “thinks.”

If you look at Anthropic’s model cards, you’ll find extensive research on these questions. For instance, how Claude reacts in a particular context, what its “psychological model” is, and whether it chooses X or Y when faced with a decision.

By studying Claude’s thought patterns, we can gain many insights. These insights can ultimately feed back into product design, helping us create a better, more trustworthy, and more natural Claude.

Host: This is indeed very important. As we increasingly delegate long-term work to models and no longer supervise them continuously, they will make many decisions independently.

Alex: Absolutely. This is why its “Character” is so crucial. If it is writing your code, deciding on database architecture, and making system design choices, you must trust its judgments.

Host: So it must possess a sufficiently high-quality “character.” To be honest, I’m glad you are seriously considering these issues because I often skip permission confirmations and switch to automatic mode.

Alex: The automatic mode is now slightly safer.