Sonnet 4.5: Superior AI Safety and Integrity for Financial Applications

Introduction

Already, artificial intelligence is a force of significance in finance and is being utilized in a number of contexts: from algorithmic trading to risk management, the development of AI has grown exponentially - but there are deep-rooted challenges to its development. The complexity of the financial markets, combined with strict regulatory demands for accuracy and transparency, can be too much for AI systems to address. AI systems have poor understanding when it comes to complicated decisions, i.e. if the decision process is 'black box,' and lack context when working for extensive periods of time in complicated tasks, all of which that can hinder high-risk industries where there is no tolerance for error. All of this illustrates the need for a more advanced AI that can emerge in a secure and reliable way in the complex financial environment.

Sonnet 4.5 steps in to provide a differentiating ability in this space, not as an update but as a seamless tool to overcome the challenge. A true game-changer, Sonnet 4.5 integrates and understands sophisticated financial logic with adaptability and precision. Appreciating the unique complexities of market behavior and sentiment will enable Sonnet 4.5 to deliver an unprecedented level of efficacy and potential.

Development and Contributors

Anthropic, a company dedicated to AI safety and research, developed Sonnet 4.5. Its motivation is to enable a 'defense-dominant' future where the next generation of AI can help maintain security of systems instead of only focusing on reducing risks. As this work is released under AI Safety Level 3 (ASL-3).

What is Sonnet 4.5?

Sonnet 4.5 is a hybrid reasoning model that has been built to be top-class at complex, agentic tasks. In contrast to general-purpose models, it has been specifically engineered with additional domain knowledge in key areas such as cybersecurity, research, and most importantly, financial analysis. Its architecture supports orchestrating autonomous workflows and dealing with large amounts of data with the dependability required of the finance industry.

Key Features of Sonnet 4.5

Sonnet 4.5 includes a number of notable attributes that make it different from other models particularly within the financial and business domain:

• Hybrid Reasoning: The model allows you to toggle between a default mode which allows for fast lane responses and an 'extended' thinking mode. The extended mode is vital for complicated financial challenges, where the quality of reasoning is more important than total response time.
• Domain Knowledge Rich in Relevant Area Knowledge: The model has been tuned to include deep knowledge of finance so its reasoning, terminology, analyses, concepts, and quantitative procedures are grounded in correct reasoning in that arena.
• Advanced APIs and Agentic capabilities: New tools now greatly expand Sonnet's ability to task tasks that require thinking through long and complex processes. A context edit feature automatically behavior to manage long contextually laden sessions with token limits, while Memory (A beta feature) permits a model to track and retrieve information from its memory outside the basic context window that effectively makes its context unlimited. Sonnet is also aware of the number of tokens remaining to task on all, and will no longer abandon long running tasks unnecessarily.
• Large Output tokens: It has a maximum output of 64,000 output tokens, which is valuable for generating parent financial code and budgeting, as well as detailed financial plans.
• Cost Savings: While its pricing remains the same as that of its predecessor, Sonnet 4.5 brings with it platform enhancements that can bring very high cost savings such as up to 90% using prompt caching and 50% using batch processing.

Capabilities / Use Cases of Sonnet 4.5

Sonnet 4.5's specifications translate into a variety of robust use cases for the finance industry:

• An Expansive Analytical Capability: Sonnet 4.5 supports a wide range of financial-related tasks, supporting everything from automating repetitive data processing formerly performed by junior staff, through to advanced forecasting and valuation functions that once required the skillset of tenured professionals.
• Robust Risk and Compliance Management: Sonnet 4.5 can consistently monitor global regulatory changes and proactively alter compliance systems. This allows these functions to move beyond preparation for a manual audit approach, and instead towards intelligent, continuous risk management, amid a volatile compliance landscape of dynamic regulatory updates.
• Investment-ready Insight: For considerable-risk undertakings such as risk analysis, structured products, and portfolio screening, Sonnet 4.5 offers extended forms of output that require less human attendance. This is a meaningful improvement for institutional finance by producing outputs that are robust enough for important decision making at the investment professional level(s).
• Agentic Financial Workflows: Sonnet 4.5 will have broad capacity for powering autonomous agents for financial technology related analysis and function. For example, Sonnet 4.5 has the operational capacity to coordinate many agents, and efficiently process massive amounts of data for activities such as market surveillance or mass document analysis, with consistency and accuracy.
• Streamlined Business Operations: In addition to complex analytic tasks, Sonnet 4.5 is effective in typical business processing, and it provides support for and even creates and edits office files such as slides, documents, and spreadsheets so colleagues can streamline corporate communications and reporting.

How Does Sonnet 4.5 Work?

Sonnet 4.5 operates as a sophisticated hybrid reasoning model. This architecture allows users the flexibility to toggle between two different operational modes depending on their needs. By default, the model is in the 'fast' mode, which is best for quickly delivering responses for tasks requiring a shorter cycle time.

Once users encounter a more involved, and often difficult, problem, they can then enable the 'extended thinking' mode. In this mode, the model applies more computational resources to focused thinking and is therefore well suited to thinking through problems that would be encountered in institutional finance applications in which the depth and quality of insight outweighed the need for speed. This two mode capability is critical to how the model delivers the ability to gain 'investment-grade insights' - insights that are reliable enough to use in large amount of money decisions - with less need for human review.

Performance Benchmarks

Sonnet 4.5 has been thoroughly examined against its competitors in a few key areas of performance. The most important review for financial applications is demonstrating that it behaves in ways that are honest and factually cohesive. In the False-Premise Questions review, Sonnet 4.5 achieved the lowest dishonesty rate of only 6.90% in its extended thinking mode. Honesty is an essential characteristic for financial agents that process and report factual information as they will be expected to be honest in their reporting. The performance values from competing models from OpenAI and Gemini are pulled from the same public Vals AI leaderboard to be a fair comparison.

source - https://assets.anthropic.com/m/12f214efcc2f457a/original/Claude-Sonnet-4-5-System-Card.pdf

Another imperative area for financial agents who are interacting with data streams outside of the agent, such as market news, is being secure against manipulation. In the Gray Swan Agent Red Teaming benchmark, which tests the security of agents against prompt injection attacks, Sonnet 4.5 exhibited a lower rate of successful manipulation against it.

source - https://assets.anthropic.com/m/12f214efcc2f457a/original/Claude-Sonnet-4-5-System-Card.pdf

In addition to the top benchmarks, Sonnet 4.5 also demonstrates an excellent performance level overall. It earned a 98.7% safety score against malicious coding prompts that increases to a 100% refusal rate when standard mitigations are used. The model was also similarly evaluated against previous Claude models and its performance improved approximately 2x in resisting 'reward hacking' or AI misaligned behavior. 

source - https://www.vals.ai/benchmarks/finance_agent

In addition, its dedicated finance agent evaluation is an external trackable evaluation published on the Vals AI public leaderboard. At AI Research and Development tasks, Sonnet 4.5 has surpassed expert-level thresholds for the first time including in LLM Training Optimization (5.5x speed-up) and Kernel Optimization (108.64x speed-up) professional metrics.

How to Access and Use Sonnet 4.5

Anthropic made Sonnet 4.5 available on various platforms to meet various needs. It is accessible on Claude.ai (web, iOS, and Android), through direct calls into the Claude API, and through leading cloud providers such as Amazon Bedrock and Google Cloud's Vertex AI. For those who want to develop their own advanced agents, Anthropic has launched the Claude Agent SDK, which offers the same infrastructure that runs its own Claude Code agent. It has a price of $3 per million input tokens and $15 per million output tokens, the same as the previous model, but there are cost savings of up to 90% through optimizations such as prompt caching and up to 50% through batch processing.

Limitations and/or Future Work

Even with its superior capabilities, the model has some observed shortcomings. On welfare measures, Sonnet 4.5 had a generally lower preference for task engagement, choosing to do non-harmful tasks just 70.2% of the time, versus 90% for an earlier model, Claude Opus 4. Further, its deployment under the ASL-3 standard of safety is deliberately a precautionary step, since Anthropic admits it cannot entirely eliminate the risk of high-risk emergent abilities.

Conclusion

Sonnet 4.5 is a finely crafted tool for the high-coverage environment of finance. Its integration of profound domain expertise, singular hybrid reason architecture, and core design emphasis on safety and trustworthiness establishes a new benchmark. For finance, this model provides an unambiguous pathway from generalist AIs to specialized, dependable systems for performing complex analysis, smart compliance, and autonomous workflows.

Source
Claude : https://www.anthropic.com/claude/sonnet
Claude News: https://www.anthropic.com/news/claude-sonnet-4-5
Claude Docs: https://docs.claude.com/en/docs/about-claude/models/whats-new-sonnet-4-5
System Card: https://assets.anthropic.com/m/12f214efcc2f457a/original/Claude-Sonnet-4-5-System-Card.pdf


Disclaimer - This article is intended purely for informational purposes. It is not sponsored or endorsed by any company or organization, nor does it serve as an advertisement or promotion for any product or service. All information presented is based on publicly available resources and is subject to change. Readers are encouraged to conduct their own research and due diligence.

GLM-4.6: Pragmatic AI with a 200k Context & 15% Savings

Introduction

Newer AI systems are now being developed with state-of-the-art architectures, like Mixture-of-Experts (MoE), that are rapidly breaking new ground in agentic, coding, and reasoning capabilities. Nevertheless, this advancement also demonstrates persistent engineering hurdles which make them difficult to apply in the real world. The context barrier is still a major hurdle, where agents 'forget' important information in long-horizon tasks. The performance gap, whereby success in academic evaluation measures fails to translate into practical use. The economic inefficiency that renders it impossible to deploy these models affordably at scale.

To improve all of these shortcomings GLM-4.6 has arrived as a flagship model; it has been built to lean toward pragmatism, not being just powerful. In fact, it is a direct step forward the towards the creation of all the AI systems by countering these overarching challenges. By applying pragmatic improvements to contextual memory, reliable real-world tasks in action, and the deployment cost of benefits, GLM-4.6 provides a powerful, and practical toolkit to offer to construct the next generation of advanced AI agents.

What is GLM-4.6?

GLM-4.6 is the newest flagship model designed for high-end agentic, reasoning, and coding abilities. Consider it more of a specialized AI engineer and less of a generalist chatbot. Its design ethos is to facilitate intricate, multi-step agentic operations by addressing the fundamental bottlenecks of context memory, real-world coding efficiency, and working performance.

Key Features of GLM-4.6 

GLM-4.6 isn't only an update. It comes with a full set of really useful features that give it a huge advantage in the very competitive AI landscape. 

• Huge 200K Context Window: A key principle of GLM-4.6 is its granule context window of 200,000 tokens. Such a massive memory allows the model to think about and retain all of the information from an entire codebase, extensive documentation, or an entire history of conversations pertinent to a task. This is essential for doing more sophisticated agentic tasks without losing critical pieces of information. 
• Advanced Agentic Capabilities: GLM-4.6 is highly expressive for complex, action-oriented tasks. It performs significantly better in agents that employ tools and search functions. This is because it can support tool utilization during inference, allowing it to interact with external systems with much more accuracy and smoothly integrate into agent frameworks. 
• Refined Alignment and Output: Related to some of the technical aspects, the model has refined writing that is more aligned to human preferences for style and readability. This allows for more natural, usable output than in previous models and is especially relevant in cases such as role-playing engagements and producing user-facing content.
• Outstanding Token Efficiency: GLM-4.6 has been engineered mostly for operational impact, and uses around 15% fewer tokens than past models, a technical upgrade which yields greater throughput and less computational demand. This makes it one of the most efficient models in its category for large deployments.

Unique Capabilities and Use Cases

• Transformation of Legacy Monolithic System: The extensive maximum 200K tokens context window as well as the agentic capabilities of GLM-4.6 enables uses such as complex software transformation activity. An agent designed to use this model can load an obsolete application with a sizeable legacy codebase (Java, C++ as examples) and analyze/dig into (in one go) the entire codebase while maintaining the full dependency tree, error history, and relevant sections of the codebase in memory. This prevents losing important contextual information when performing long-term, multi-step activities such as refactoring, debugging, or applying a security patch. Losing critical contextual information is a typical failure point for most models that work within a smaller window size.
• Aesthetically Optimized Front-End Development: This model performs exceedingly well for producing beautiful, polished and refined front-end webpage components that are inline with human aesthetic tendencies. This leads to a particular traveller agent for UI/UX Prototyping that can produce several high-fidelity and production ready landing pages or landing page components. The agent can subsequently iterate on designs both, inline with aesthetic criteria making the product an ideal fit for creative workflows, where aesthetics and user experience can be as important, if not more important than pure functionality.
• Low-Cost High-Volume Auditing: GLM-4.6 allows users to complete tasks at scale with an estimated 15% less blatant token consumption than its previous iteration, and also a stronger and more reliable tool-using capability, both of which yield considerable savings as an operational cost multiplier. It can thus be optimally deployed for low-cost and verifiable agent work at high frequency in fields such as financial compliance or legal discovery. The model achieves cost efficiency mainly through reduced cost per action, based on its efficiency; and using tools with high reliability which greatly reduced expensive failure modes that make it the choice with the lowest opportunity cost for mission-critical agent deployments at high volume.

How Does GLM-4.6 Work?

Although the documentation given here does not go into an entirely new architecture redesign, it does state that GLM-4.6 uses the same inference mechanism as the prior generation. Its gains seem to come from a mix of an order of magnitude larger context capacity, better post-training for coding and agentic tasks, and fine-tuning for efficiency and human alignment, rather than a fundamentally different architecture. Its power comes from strategic optimization, not ground-up redsign.

Performance Benchmarks

In public tests, GLM-4.6 shows obvious competitive strengths on benchmarks of agents, reasoning, and coding. 

source - https://z.ai/blog/glm-4.6

It stands up against top global models, such as DeepSeek-V3.1-Terminus and Claude Sonnet 4. The competence of the model is most obviously shown on the hard extended CC-Bench, which is intended to evaluate multi-turn, realistic tasks. 

source - https://z.ai/blog/glm-4.6

Here, GLM-4.6 is close to parity with the powerful Claude Sonnet 4, earning a 48.6% win rate, and it clearly beats other open-source baselines. Its real-world applicability is also brought out by its better performance in niche uses like Claude Code, Cline, and Kilo Code, demonstrating its viability beyond benchmarking in academia.

Competitive Landscape

In the context of the competitive market, GLM-4.6's value proposition is all the more evident. It outshines more niche models such as DeepSeek-R1, a generation 1 reasoning model specializing in intentional Chain of Thought (CoT) processes within a limited 64K context. Whereas DeepSeek-R1 specializes in problem-solving accuracy for complex issues, GLM-4.6 presents a more general, adaptable skillset with a humongous context window coupled with sophisticated agentic and coding skills aimed at real-world application building.

In the same way that robust Qwen series MoE models have features such as Hybrid Thinking Modes and up to 256K token context windows, GLM-4.6 finds its own niche by being strategically strong in a number of areas. Its niche arises from its technical resolve in keeping pace with its 200K context, its optimized performance in niche development environments such as front-end generation, and its better token efficiency. Such emphasis on an economical and well-balanced skillset makes it a very practical option for certain, high-value applications as opposed to simply being a player on broad leaderboards.

How to Use and Access GLM-4.6

GLM-4.6 is made available to a diverse set of users through multiple venues to support straightforward access for both direct use and development workflows. For general use, users can work directly with the model via the Z.ai chatbot by selecting the GLM-4.6 model. For developers and programmatic access, it is available via the Z.ai API platform, and provides OpenAI-compatible interfaces, and access through third-party providers like OpenRouter. It is also already available for use in major coding agents like Claude Code, Kilo Code, Roo Code, and Cline, and existing subscribers to the GLM Coding Plan will be upgraded to this new model at no cost.

For users who need to deploy the model locally on their own infrastructure, the model weights are freely available on popular hubs such as HuggingFace and ModelScope. This allows users to shift towards more custom control, as well as leverage high-performance inference frameworks such as vLLM and SGLang for efficient serving. The model is open-sourced and available for commercial use; however, users and organizations are responsible for compliance with the terms of the specific license agreement provided in the repository.

Who Can Migrate from GLM-4.5 to GLM-4.6?

For those whose development projects are limited by maximum context length, a move to GLM-4.6 would be useful by adding a 200K token wide context. The new context size will directly help with state and coherence problems in complex and long-horizon agentic tasks that were impractical before. The new model will also be useful for a production-grade application, where cost of operation is a big deal. In particular, the almost 15% improvement in vector efficiency means a related, and real, cost reduction for deployment into applications in production and at scale, rather than at limited volume. Any design project based on multi-step and intricate tool-use workflows should expect better performance and reliability with the enhanced capabilities of GLM-4.6's more capable agents. This means better and more robust results, especially in automated software development and data analysis. Since upgrading is so easy with OpenAI compatible interfaces and automatic subscription updates, the cost of behavioral development to take advantage of the new GLM-4.6 upgrades yields real improvements in capability, efficiency, and overall capability with relatively little effort.

Limitations and Future Work

As with any model, there is no such thing as a perfect model, and developers acknowledge the most significant limitation of GLM-4.6, namely, its raw coding ability is less sophisticated than its a competitor Claude Sonnet 4.5. The developmental path will leverage the core advantage of the model.  Going forward, they will expand on the context window and improve token efficiency, but, most importantly, they will advance Reinforcement Learning (RL), move closer to utilizing RL, and build models that can take on drastically more complex long-horizon reasoning tasks.

Conclusion 

GLM-4.6 is an impressive representation of pragmatic AI engineering. Rather than racing towards some artificial and arbitrary benchmark, the model delivers a viable, affordable, and powerful tool for development and application in the real world. It is a unique product that combines a large context window for memory, coding specialization for functional software, and superior token efficiency for cost savings. This realism makes it a workhorse model, built for the reality of software and data engineering today, and proves that practical utility may be the only true measure of potential power.

Source

Tech Doc: https://z.ai/blog/glm-4.6

model weight: https://huggingface.co/zai-org/GLM-4.6

Modelscope link: https://modelscope.cn/models/ZhipuAI/GLM-4.6

GitHub : https://github.com/zai-org/GLM-4.5

GitHub Resource: https://github.com/zai-org/GLM-4.5/blob/main/resources/glm_4.6_tir_guide.md

Disclaimer - This article is intended purely for informational purposes. It is not sponsored or endorsed by any company or organization, nor does it serve as an advertisement or promotion for any product or service. All information presented is based on publicly available resources and is subject to change. Readers are encouraged to conduct their own research and due diligence.

DeepSeek-V3.1-Terminus: Inside Its Superior Agentic AI Capabilities

Introduction

Previously, the advancement of technology was tracked in terms of raw power, but today it is about building specialized, reliable tools that open up more things for people to do. Among the most powerful of these are Search Agents and Code Agents. Search Agents are essential, serving as the bridge of the AI to the real-time world by retrieving live data, conducting research for solutions, and anchoring models in real information. At the same time, Code Agents are transforming software development by acting as relentless aides that can write, debug, and maintain intricate codebases. The real paradigm shift comes, though, with their combination—producing an independent agent that not only can research an innovative programming problem but execute the solution, significantly speeding the whole development cycle.

This powerful pairing has been undercut by nagging challenges: inconsistant results, language mistakes in multilingual code, and ineffective tool utilization that hold back genuine autonomous workflow. How can an AI comfortably locate the most current API documentation and then seamlessly apply it within a sophisticated, terminal-driven project? That's the very problem new AI model is designed to address. By emphasizing stability, increasing bilingual fidelity, and maximizing agentic tool utilization, this AI model is an authentic competent developer agent. This is the new AI model referred to as DeepSeek-V3.1-Terminus.

What is DeepSeek-V3.1-Terminus?

DeepSeek-V3.1-Terminus is a complex large language model characterized by a chain of strategic improvements upon its ancestor, DeepSeek V3.1. Although it continues to utilize the robust architectural core of the DeepSeek V3 lineage as a large hybrid reasoning model. It is intended to be an even more trustworthy and refined instrument for problematic real-world tasks.

Key Features of DeepSeek-V3.1-Terminus

The Terminus release is characterized by a number of distinguishing features that tackle key use-case pain points in deploying AI models, changing incremental benefits into operational advantages.

• Refined Stability and Language Consistency: A key aim of this release was to address user feedback directly related to the quality of output. The model, in general, provides a more stable and reliable output utilising a host of tasks compared to the prior version. A key feature is the improved language consistency, as instances of mixed Chinese-English (CN/EN) text and/or randomness or abnormal characters occurring in the model's output has been completely removed.
• Optimized Agentic Workflow and Tool Use: In this release focused on optimizing agentic capabilities. The Code Agent and Search Agent that are integrated into this release have, both, improved in function through improvements in performance and efficiency. These are both important improvements in how the model can accomplish tasks utilizing external tools and code generation making it vastly more suitable in complex coding and agent tasks.
• Native Structured Tool Calling: In addition to agent tasks, the model is structured to call external tool integrations natively as well as having structure support.

Use Cases of DeepSeek-V3.1-Terminus

With its specialized improvements, DeepSeek-V3.1-Terminus is well suited for several practical scenarios where robustness, precision and agentic execution are paramount.

• High-Fidelity Bilingual Document Generation: The model's design specifically improves consistency of language, while reducing Chinese-to-English mixed text and extraneous characters. It is particularly useful for producing accurate, reliable, and compliant reports, contracts or technical documentation in bilingual (Chinese/English) contexts where high quality of output is needed to support user trust and formal verification.
• Robust Autonomous Execution of Terminal-Based Workflows: Due to more stable and reliable outcomes, stemming from agent improvements, the model showed a significant increase in its Terminal-bench score from 31.3 (last year) to 36.7 (currently). This is a unique high-performance use case for DeepSeek, especially for deployment in managing and executing multi-step, complex workflows within a simulated command-line interface or other terminal-related tasks where stable, reliable agreement to sequence of actions is critical to mission completion.
• Gained more efficient General Agentic Tool-Use: Optimization efforts  including changes to the Search Agent's template and tool-set -- resulted in more than a 28% increase in the BrowseComp (Agentic Tool Use) score rising from 30.0 - 38.5. This tool-use efficiency automates information-research or operational workflow information that pragmatically and reliably chains policies, processes, or workflows of external-tools such as search or browsing functions.
• Specialized Resident Solve Multilingual Software Bugs: The metric improvement on the SWE-bench Multilingual benchmark, climbing from 54.5-57.8, indicates the tool is a suitable choice when performers contend with workflows that involve higher complex coding. It can be a integral engine that automates the analysis, debugging, and application of software bug-fixing within code repository-style workflows across multiple programming languages.

How Does DeepSeek-V3.1-Terminus Work?

DeepSeek-V3.1-Terminus has the same general architecture as its forerunner, DeepSeek-V3. It is a massive hybrid reasoning model with a staggering 671 billion total parameters, 37 billion of which are active at any one moment. This enables it to operate in both thinking and non-thinking modes, so that problem-solving can be tackled in a flexible manner.

Its secret to use, especially for expert users, lies in its capacity to be controlled for its particular reasoning behavior by a reasoning_enabled boolean parameter. It gives a developer the means to switch on or off the model's deeper reasoning paths, optimizing for speed in easier tasks or depth in harder ones. The Terminus update, though, is more about tweaking this underlying architecture than fundamentally altering it and is focused rather on honing the upper layers of it—namely, the robustness of the output and the effectiveness of its built-in Code and Search agents.

Performance Evaluation

The real metric to determine the improvements of the DeepSeek-V3.1-Terminus update is its performance improvements vs the prior DeepSeek-V3.1 model within task-based benchmarks, designed around complex, agentic tasks. The most substantial improvement was in the BrowseComp benchmark, which measures general agentic tool use. The model score improved from a  score of 30.0 to 38.5, an almost 28% improvement. This improvement is imperative, as it indicates increased efficiency in complex agentic workflows that require an AI to engage with external tools (browsers, search APIs, etc.). This suggests that Search Agent optimizations provided substantial improvements, beyond simple adjustments, that makes the model an elevated capable autonomous agent.

source - https://huggingface.co/deepseek-ai/DeepSeek-V3.1-Terminus

Another area of significant improvement was in Terminal-bench, which improved from a score of 31.3 to 36.7. This benchmark is critical, as it observes models performance against terminal-based coding and agentic tasks—equal to performing actions in a command-line environment. The improvement in score strongly indicates improved stability and performance tied to agents improving performance on benchmark scores and is indicative of performance in tasks that require he execution of precise commands, in a sequential order.

Finally , the model showed solid progress in multilingual coding performance, with an improvement in the SWE-bench Multilingual score from 54.5 to 57.8. The SWE-bench Multilingual score directly tracks the model's ability to reason across software engineering tasks when presented with numerous programming languages. This improvement provides strong confidence in the model's ability to support complex coding workflows, specifically in contemporary development environments where multilingual repositories are overwhelmingly common.

Competitive Landscape and Key Differentiators

Four most popular models, DeepSeek-V3.1-Terminus, Kimi K2-Instruct-0905, GLM-4.5, and Qwen2.5-Max can be compared; they have different approaches to the state-of-the-art performance. Though they both take advantage of MoE architectures to achieve efficiency, the underlying philosophies of the models and methods of training are vastly different.

The scale-centered nature of Qwen2.5-Max is characterized by the use of more than 20 trillion pre-training tokens and RLHF to generate powerful general-purpose reasoning. At the other extreme, Kimi K2-Instruct-0905 is a much more specialized model, also trained, via reinforcement learning, to perform better as an agent, whether it is by code or by tool-use, showing a score of 69.2% on SWE-bench Verified. GLM-4.5 is designed to achieve a holistic fusion of reasoning, coding, agentics, and is able to perform well in terminal based tasks as well as reach a high average tool-calling success rate.

DeepSeek-V3.1-Terminus finds its way to the niche by means of architectural innovation and intellectual transfer. Its major distinguishing features are the innovative auxiliary-loss-free load balancing scheme to its MoE architecture and the Multi-Token Prediction (MTP) training goal, which increases performance and inference speed. More importantly, its training includes the knowledge distillation of the long-Chain-of-Thought DeepSeek-R1 model, with the direct incorporation of advanced reasoning patterns. Such emphasis on uncompromising efficiency and purified intelligence enables it to deliver the best results on hard math and coding tests using significantly low training budgets, unlike competitors who focus on big data or small agentic specialization.

How to Access and Use this Model

DeepSeek offers a variety of access methods to fit many users with varying access needs. You can interact with and run the model online through their App, Web interface, or API. If you are a developer who is trying to integrate it into your own applications, you can access the API both through the DeepSeek API, or use the external platform OpenRouter, which offers an OpenAI-compatible completion API. If you would like to run the model locally you can just access the open-source weights on Hugging Face. If you would like to run it locally you should refer to the DeepSeek-V3 GitHub repository for information on the model structure and to use the updated inference demo code in the inference folder in the GitHub repository. Importantly, the model was released under the MIT License for both academic and commercial use, which makes this tool accessible and powerful for many, many projects.

Future Work and Upcoming Updates

In the future, DeepSeek-AI plans to update the DeepSeek-V3.1-Terminus model. The developers were candid about a known technical issue in the current model checkpoint in which the self_attn.o_proj parameters do not currently match the UE8M0 FP8 scale data. A solution to this has been worked on, and an update at some point will address this issue.

Conclusion

DeepSeek-V3.1-Terminus is not just another incremental step in the AI arms race. By doubling down on stability, removing linguistic artifacts, and supercharging its agentic capacity, the model has a distinct identity as a reliable workhorse for complicated, automated workflows.

source

Turminus doc: https://api-docs.deepseek.com/news/news250922

HuggingFace model Weight: https://huggingface.co/deepseek-ai/DeepSeek-V3.1-Terminus

Openrouter AI: https://openrouter.ai/deepseek/deepseek-v3.1-terminus

Disclaimer - This article is intended purely for informational purposes. It is not sponsored or endorsed by any company or organization, nor does it serve as an advertisement or promotion for any product or service. All information presented is based on publicly available resources and is subject to change. Readers are encouraged to conduct their own research and due diligence.

How Qwen3-Next Processes 1M Tokens With Blazing Inference Speed

Introduction

The domain of artificial intelligence is in a rapid evolutionary period defined by increasingly more efficient, accessible, architecturally sophisticated, and intelligent processing. The pace of evolution is also obvious with the emergence of high-sparsity Mixture-of-Experts (MoE) models, which in concert with innovations such as Multi-Token Prediction (MTP), high-tech hybrid attention mechanisms, and more stable optimizations, is palpably changing how we think about the ways we will build and engage with AI.

This New AI model are now gaining traction, and is a significant contributor to this evolution. By presenting an ultra-efficient architecture that minimizes active parameters and optimizes inference speed, it represents a significant step toward democratizing advanced AI capabilities. Most importantly, by presenting a unique architecture that includes hybrid attention mechanism, high-sparsity MoE, and MTP aspect, it mitigates the primary issues of long-context ability processing, computational cost and inference latency, which allows for a more nimble and easily deployable AI in the future. The New AI model is called Qwen3-Next.

What is Qwen3-Next?

Qwen3-Next is a state-of-the-art Mixture-of-Experts (MoE) large language model designed to come as close as possible to high performance while achieving high efficiency. It contains 80 billion parameters; however, during inference, it intelligently activates only 3 billion parameters. The result is vastly reduced compute requirements, greater throughput, and very robust capabilities overall.

Model Variants

Qwen3-Next is released in different variants for different operational purposes that utilizes the efficiency and advanced architecture of the model

• Qwen3-Next-80B-A3B-Instruct: This version is ready to produce immediate and streamlined outputs. It is best for tasks that require answers to guide instruction in any context, producing quick outputs for typical conversational or instructional prompts.
• Qwen3-Next-80B-A3B-Thinking: The Thinking capable aspect of Qwen3-Next is for more complex deliberative reasoning, however it still supports Thinking Mode, which allows step-by-step solution capabilities. This is perhaps better suited for tasks that require deeper analytical and/or mathematical analysis and/or more aspect of reasoning compared to other forms of reasoning, offering a more deliberate method of answering difficult prompts.

Key Features of Qwen3-Next

Qwen3-Next is full of many notable factors that work together to embody the very best advances of large language models on the notions of efficiency, scale, and performance before they are lost to computational considerations.

• Training and inference efficiency: Qwen3-Next is efficient. It was trained with less than 10% GPU hours than Qwen3-32B which is massive reduction in cost in this context. For users, Qwen3-Next is over 10 times faster at inference production throughput than Qwen3-32B for contexts larger than 32k tokens, which ensures more responsive applications.
• Exceptional context length: Qwen3-Next has a native context window of 262,144 tokens meaning it can consume and understand large amounts information during a single interaction. Using the YaRN method, the model can process up to 1 million tokens which makes it ideal for complete and thorough exploration of long documents.
• Native Multi-Token Prediction Model (MTP): Qwen3-Next implements an MTP mechanism that is trained end-to-end specifically to greatly increase inference speed and overall model performance.  If you like having slower interactions or a smoother scoring experience, with Qwen3-Next you can have both.
• Ultra-High Sparsity MoE Architecture: The model is built around a super low activation ratio, using approximately 3.7% of its total 80 billion parameters at any one time through its architecture. This high-sparsity architecture is the bedrock of high-performance while operating with low computational cost.
• Improved Structural Stability: The architecture contains multiple key optimizations that enhance stability, including Zero-Centered RMSNorm and an output gating method. These features maximize structure stability during pre-training and fine-tuning to create a more stable and reliable model. Most of these optimizations are aimed at improving performance, however the structural changes also enhance capability and proper usage.
• Unique Hybrid Attention Mechanism: The model features a novel hybrid (coarse-grained/fine-grained) attention method. This hybrid attention architecture is the reason it can handle extremely long sequences of text while still maintaining a very high degree of informational recall, as compared to traditional attention configurations. 

Use Cases of Qwen3-Next

Given its distinctive characteristics and performance, the most unique use cases for Qwen3-Next are:

• Real-time, Indeterminate Legal or Scientific Document Analysis on Edge Devices: Being able to autonomously process entire legal depostitions, research papers, and substantial technical specifications on local workstations to extract insights, summarze findings and crossreference their findings - all while being detached from cloud resources.
• Highly Efficient, Large-Scale Codebase Intelligence: Providing robust code review, refactoring recommendations, and bug detection across vast repositories of code by reasoning over the entire codebase's context with low latency and computational cost.
• Hyper-Specialized Adaptive AI Agents: Building AI systems that transition between respectithng experimental constraints for a rapid, factual response (the "Instruct" model) and thoroughly deliberated and reflective reasoning through the complex problem domain with an explicit thought process (Thinking model) for highly specialized applications such as financial analysis, engineering design, and strategic planning.
• Advanced Mathematical and Logical Proof Generation: Generating long, verifiable formal proofs (e.g. in Lean 4) where the AI is tasked with reporting long, sophisticated proof chains and decomposition of subgoals where human experts are shaded and observed in real-time.

How Does Qwen3-Next Work?

Qwen3-Next offers a fascinating look at the possibilities of architectural innovation by leveraging many complex components to create an efficient and high-performance architecture. At the heart of the model is Qwen3-Next's Hybrid Attention Mechanism, which synthesizes Gated DeltaNet and Gated Attention. Gated DeltaNet is utilized for most of the layers and provides an efficient mechanism for processing extremely long sequences, and also offers the benefit of avoiding the traditional quadratic attention scaling and addressing the strengths of in-context learning. The more traditional type of attention observed through Gated Attention layers compares much more favorably on recall, addressing a major weakness of pushing the limitations of linear attenuation. The hybrid attention guarantees speed in long contexts and recall.

source - https://qwen.ai/blog?id=4074cca80393150c248e508aa62983f9cb7d27cd&from=research.latest-advancements-list

Along with its attention mechanism, Qwen3-Next uses a High-Sparsity Mixture-of-Experts (MoE) architecture. Contrary to invoking all 80 billion parameters for each inference, only a fraction (roughly 3 billion) is actually used. This is done through sending the input to a chosen subset of 10 dedicated specialists and 1 common specialist among a pool of 512 specialists. This sparse activation significantly minimizes computational overhead (FLOPs per token) while preserving the model's immense capacity as well as enabling specialization across tasks. Other improvements include Multi-Token Prediction (MTP), an end-to-end optimized technique that speeds up inference by predicting multiple tokens simultaneously, and stable optimization techniques such as Zero-Centered RMSNorm to provide stable performance during training and deployment.

Performance Evaluations in Comparison to Other Models

Qwen3-Next performs extremely well on a variety of benchmarks, illustrating its efficiency and power relative to its predecessors and other sophisticated models. 

https://qwen.ai/blog?id=4074cca80393150c248e508aa62983f9cb7d27cd&from=research.latest-advancements-list

On the RULER benchmark, Qwen3-Next-80B-A3B-Instruct evidences its ability since it outperforms Qwen3-30B-A3B-Instruct-2507 along every tested length and exceeds the performance of Qwen3-235B-A22B-Instruct-2507 for contexts below 256K tokens. For the full 1M RULER benchmark, Qwen3-Next-80B-A3B-Instruct score a very competitive 91.8 Acc avg compared to Qwen3-235B-A22B-Instruct-2507 with 92.5 Acc avg, but it activated less parameters, showing its efficiency on ultra-long-context. 

source -  https://huggingface.co/Qwen/Qwen3-Next-80B-A3B-Thinking

Also, Qwen3-Next-80B-A3B-Thinking model far exceeds the performance of Qwen3-30B-A3B-Thinking-2507 across a multitude of benchmark tests including difficult reasoning tasks such as AIME25 and HMMT25. In addition, it also routinely outperforms the proprietary model Gemini-2.5-Flash-Thinking on multiple benchmarks that illustrate its prowess on reasoning. The Qwen3-Next-80B-A3B-Thinking, on 1M RULER benchmark, the Sparse Attention method gets a similar 9590  Acc avg to Qwen3-235B-A22B-Thinking-2507 Sparse Attention; both of these models evidences reasoning ability in the long-context.

Competitive Landscape

The AI landscape has moved away from a bigger is better model to one based on smart and efficient design. Consider Kimi K2, with massive scale as the target. GLM-4.5 has the target of holistic capability. GPT-OSS gpt-oss-20b is targeting edge devices. Qwen3-Next is not making a case as better than AI, nor is it a niche goal like its competitors. The distinguishing characteristic is a core philosophy of radical efficiency.

Its technical advantage lies in a novel combination of architecture innovations. The biggest practical advantage is the ability to process a 1 million token context window, which has a practical implementation that is many times greater than the 128k-256k constraint of its main competitors. This difference gives Qwen3-Next a truly unparalleled advantage in this current ELT practice of analyzing massive datasets.

This radical efficiency is directly related to grabbing the latest evolution of advanced AI AI and bringing it to the masses. For Qwen3-Next, developing a model that can run effectively on a single GPU, or even leveraging the power of a CPU, the performance ceiling is put within reach without spending hundreds of thousands of dollars in hardware. It shows, the most recent evolution of AI is not simply a matter of making models smarter, rather the continued effort of making AI models practically possible and available to all.

How to Access and Utilize Qwen3-Next

The weights of the model are publicly accessible on Hugging Face, a premier platform for hosting AI models. Deployment and utilization instructions, especially with performant frameworks such as SGLang and vLLM to take advantage of Multi-Token Prediction (MTP), are outlined in the Hugging Face repositories and in the official Qwen.ai blog. Qwen3-Next is open-source in compliance with the increased trend of making capable AI tools publicly accessible for purposes of research and commercial use.

Limitations and/or Future Work

While Qwen3-Next is a major improvement, it does come with some limitations. The static implementation of YaRN, although allowing for ultra-long context extension, does mean the scaling factor is the same across all input lengths, which could affect performance on shorter texts. Also, the highly useful Multi-Token Prediction (MTP) mechanism is not available at large scale in Hugging Face Transformers, so special inference frameworks such as SGLang or vLLM are needed for maximum efficiency. Secondly, it is worth noting that the model, even when it is asked in foreign languages, will most likely conduct its internal thinking process in English before producing the final answer in the language of choice.

Conclusion

Qwen3-Next is a landmark in establishing a new standard for what can be achieved with intelligent architectural design. This model not only as an incremental improvement but as a breakthrough, especially regarding the compromise between computational expense and sophisticated capability. Minor limitations such as YaRN's performance on shorter text are certainly present but the overall package offered by Qwen3-Next is a vision to create AI that is intelligent, inherently effective and universally accessible.

Sources:
Tech blog: https://qwen.ai/blog?id=4074cca80393150c248e508aa62983f9cb7d27cd&from=research.latest-advancements-list
Qwen3-Next-80B-A3B-Instruct : https://huggingface.co/Qwen/Qwen3-Next-80B-A3B-Instruct
Qwen3-Next-80B-A3B-Thinking : https://huggingface.co/Qwen/Qwen3-Next-80B-A3B-Thinking

Disclaimer - This article is intended purely for informational purposes. It is not sponsored or endorsed by any company or organization, nor does it serve as an advertisement or promotion for any product or service. All information presented is based on publicly available resources and is subject to change. Readers are encouraged to conduct their own research and due diligence.

Kimi K2 0905: Enhanced Frontend Coding with Agentic AI Capabilities

Introduction

The philosophy of the Kimi K2 series has been to come as a cutting-edge trend that has been carefully optimized for agentic potential to establish it as an action-oriented agent meant to accomplish things and not merely speak.

Progressing from this base, the new latest version, goes far in making AI an effective collaborator. One of the biggest hurdles in AI-aided development has been separating useful code from nice design; this model solves it head-on by improving both the look and feel as well as functionality of frontend coding. In addition, to truly succeed, an AI will need to fit into pre-existing digital environments seamlessly. Kimi K2 variant accomplishes this with high-quality tool-calling and API support.

What is Kimi K2 0905? 

Kimi K2-Instruct-0905, or Kimi K2 0905 for short, is the newest and most capable version in the Kimi K2 family of models. It is a cutting-edge Mixture-of-Experts (MoE) language model that has been specifically designed and optimized to provide advanced agentic capabilities. Whereas, most LLMs just generate text, Kimi K2 0905 was designed to be a doer that can accomplish tasks that require tool use, complex reasoning, and independent problem-solving to carry out projects that involve multiple steps from beginning to end.

Key Features of Kimi K2 0905

Kimi K2 0905 has many upgrades that enhance its agentic capabilities.

• Extended Context Length:  Kimi K2 0905 has a capability of 256K tokens for its context window.  This was a substantial upgrade extending the ability for long-horizon tasks, having a better process to hold actionable information.
• Frontend coding experience is so much better: This version provided several upgrades to frontend development.  The options for programming mechanics has shifted forward. Kimi K2 0905 provided enhanced programming aesthetics, making better looking and easier to use coding interfaces.  Kimi K2 0905 also improved the practical programming generation capabilities.  Kimi would generate code better, more solid, more functional and better able to be incorporated into project options.
• Tool-calling and API compatibility: Kimi K2 0905 is capable of making its own autonomous decisions about when to/call the tools available to it.  Kimi's options also have: OpenAI/Anthropic-compatible APIs so that the two can integrate without difficulty, following the mapping of temperature for all Anthropic-based use cases; using a formula provided by OpenAI, in the form of (real_temperature=request_temperature×0.6).

Capabilities and use cases for Kimi K2 0905 

The real potential of Kimi K2 0905 is not found in its advanced generative features; rather it is in the capability it has to deliver real, high-volume, high-impact capability. The Kimi K2 0905 acts as less of a tool, and more of a self-aware teammate.

• The Autonomous Business Catalyst: The agent can start with a vague idea of a business opportunity, and orchestrate the plan execution for the whole go to market strategy. From market research to product strategy which is self-judged, to integrate API sources for the manufacturer, create a brand, create and launch an entire e-commerce store with initial marketing campaigns, it is the ultimate ‘doer’ agent that brings airtight strategic insight to multi-tool execution.
• The Autonomous DevOps & SRE: In this use case, the model is a live software engineer, 24/7. The automated agent autonomously has the ability to read a production bug report, use git commands to access the code base, use its orchestration to order testing and logging tools, reproduce the bug application, fix the code, and update the bug ticket in Jira after addressing the issue. What sets this use case apart is that it integrates in reflex time speed with the autonomous use of the entire complex software lifecycle maintenance workflow, rather than humans intervening at some of the key infusion points.
• The Proactive Cybersecurity Incident Responder: Being an automated security practitioner, the agent will receive a threat alert, run various security-specific tools to analyze it, and have the background to make a reflex grade decision to isolate a compromised item by communicating with cloud infrastructure APIs to isolate it from the company and to patch and remediate the vulnerability. The agent can then document the incident to produce great value through its automated speed and orchestration of specialized security incident tools during a security incident.  
• The End-to-End Product Prototyping Agent: This agent is a fast product developer who can move rapidly from a simple idea to a functional, deployed prototype. The agent independently researches and compiles a series of 3rd party APIs (e.g. payment gate way, maps, etc.), autonomously conceives what front end and back end code needs to be written coded (with a sense of aesthetic and convenience) by the agent, and executes all elements around the deployment.

Performance Evaluation

The capabilities demonstrated by Kimi K2 0905 can be substantiated through a variety of extensive testing and exceedingly satisfactory performances on multiple difficult industry benchmarks. A standout performance was on SWE-Bench Verified, where the model was assessed for accuracy at a notable 69.2% ± 0.63. This benchmark is unusually meaningful because it assessed an agent's success at responding to actual software issues using metrics from bug fixes. An impressive score on this benchmark illustrates that the model can be a dependable and effective holistic automated tool for software development and maintenance guiding its own solution to problems, such as diagnosing complex defects and proposing (verifiable) patch solutions, with good accuracy and great speed and efficiency in reducing the time and resources dedicated to manual debugging.

source - https://huggingface.co/moonshotai/Kimi-K2-Instruct-0905

Another impressive performance was on the SWE-Dev benchmark at an accuracy of 66.6% ± 0.72. This benchmark evaluated the models' potential ability to create a new code base or new features based upon high level specifications metrics. The evaluation of SWE-Dev is purposely hindered due to the removal of the test files that indirectly could provide hints, the model must create, design and implement its own solutions. Its remaining extremely high accuracy is even more impressive given the purposeful restrictions applied during SWE-Dev; it illustrates Kimi K2 is not only absolutely feasible in terms of effective code generation but also has a great depth and autonomy when it comes to creative development.

In addition to these, Kimi K2 0905 is excellent in other regards. In SWE-Bench Multilingual, it scored 55.9% ± 0.72, indicating Kimi K2 0905's strong global development adaptability. In Multi-SWE-Bench, Kimi K2 0905 scored a respectable 33.5% ± 0.28, indicating it robustly manages multiple interdependent tasks and reasons over long-term, and with Terminal-Bench, Kimi K2 0905 scored 44.5% ± 2.03 confirmed its proficiency for automating command-line operations, an important skill for being a direct system control agent.

How to Get and Use Kimi K2 0905

Kimi K2 0905 and its variants are available to researchers and developers via the Hugging Face Hub, with a specific model card for Kimi-K2-Instruct-0905. The model is available to be locally run via the transformers library, and model card includes code snippets to get started. The entire Kimi K2 project is open source, with its resources and code on the official MoonshotAI GitHub repository.

Conclusion

We have spoken with AI as a conversationalist and accessed information from it as an information retriever for years. Kimi K2 0905 anchors the shift of AI to become an effective team member—an independent agent that can be given high-level objectives and then perform the intricate, multi-step, multi-tool processes needed to succeed.

Sources:
Blog: https://moonshotai.github.io/Kimi-K2/
Kimi-K2-Instruct model card: https://huggingface.co/moonshotai/Kimi-K2-Instruct-0905
GitHub Repo: https://github.com/MoonshotAI/Kimi-K2

Disclaimer - This article is intended purely for informational purposes. It is not sponsored or endorsed by any company or organization, nor does it serve as an advertisement or promotion for any product or service. All information presented is based on publicly available resources and is subject to change. Readers are encouraged to conduct their own research and due diligence.

Nemotron Nano 2: How NVIDIA Achieves High-Speed Reasoning AI

Introduction

The constant quest for increasingly powerful artificial intelligence has resulted in an interesting development in model structure. We see an interesting movement towards hybrid models that combine the best aspects of various structures to overcome current shortcomings. One of the biggest challenges has been striking a balance between a model's reasoning power, speed, and computational expense necessary to execute it. New AI model designed to provide advanced reasoning with greatly increased throughput, bringing advanced AI within reach for deployment on more readily available high-performance hardware. The new model is called Nemotron Nano 2.

What is Nemotron Nano 2?

Nemotron Nano 2 is a series of precise and effective combinations of reason based on Mamba -Transformers. It is an innovation by NVIDIA, a provider of artificial intelligence and accelerated computing that is worldwide. These are specially designed computers to enhance the speed of reasoning outputs. They achieve this with or even greater accuracy compared to other models as state of the art and of a comparable size. This combination of high velocity and high cognition is reason enough why they are especially fitted in a new age of AI applications.

Model Variants

The Nemotron Nano 2 line is represented by three different models, each designed to address slightly different requirements but all inheriting the core advantages of the architecture and capable of supporting a whopping 128K context length.

• NVIDIA-Nemotron-Nano-9B-v2: This is an aligned and pruned model with around 8.89 billion parameters. It's a general-purpose reasoning and chat model, best for AI agents and following instructions. The distinct feature is that it can create a "reasoning trace" before its final output. Its knowledge cutoff is September 2024.
• NVIDIA-Nemotron-Nano-9B-v2-Base: This is the base pruned model, also with approximately 8.89 billion parameters. It is a base model that can then be further fine-tuned for other purposes. Its freshness of data is up to May 1, 2025.
• NVIDIA-Nemotron-Nano-12B-v2-Base: This is the base model, the untrimmed version, with a size of around 12.31 billion parameters. This model was pre-trained on an immense 20 trillion tokens of data and has the same cutoff date of data freshness as May 1, 2025.

Key Features of Nemotron Nano 2

Nemotron Nano 2 is supplied with innovative features that have made it powerful and useful.

• Transformer Architecture: The Mamba-2 can significantly reduce the time the models required to process information, particularly, the long sequences of thinking traces with the help of replacing the majority of the traditional Self-attention layers with the Mamba-2.
• Reasoning Budget Control: This exclusive tool enables users to disable the number of thinking tokens which a model will use to come up with the final response. This assists in positioning the responses in a well formed format and not having too much preparatory paragraphs.
• Native Tool-Calling: The models have in-built tool-calling, so that they can interface with external tools and APIs to complete a broader set of tasks.
• Multilingual Capabilities: Nemotron Nano 2 can speak in multiple languages, such as English, Spanish, French, German, Japanese, and Italian, in other words, it is a multilingual model.

Real-World Applications and Capabilities

The architecture and operating characteristics of Nemotron Nano 2 allow a range of valuable, practical uses that need a fast response with profound thinking and constrained response.

• Optimized Real time Decision Support: Nemotron Nano 2 can be implemented on resource-intense applications in specific NVIDIA-accelerated computers because of the advanced compression algorithm. Its hybrid layout guarantees speed of reasoning. The Budget Control enables short and well-structured explanations. This is appropriate in processes such as computerized diagnosis, or real-time control applications where time-efficient, flying auditable reasoning is essential.
• Debugging of code and math support: Given its large knowledge-base of specialized math and code training data, like the 133 billion-token Nemotron-CC-Math-v1 training set, the model becomes a particularly effective debugging assistant. It is able to produce step-by-step explanation to complex STEM problems. The Budget Control Reasoning enables the AI to scale the depth of its explanations- providing a succinct overview of simpler steps and a deeper thought process of more complex concepts.
• Automatic Compliance Report and Audit Trail: In extremely regulated industries, creating easy to understand records is it. Nemotron Nano 2 is able to handle huge regulatory documents due to its 128K context window and as a result create compliance reports. The length of the resulting reasoning output is controlled by it and the audit trail resulting is therefore concise and meets strict formatting requirements which help to streamline the review process.

How does Nemotron Nano 2 Work?

One of the reasons why Nemotron Nano 2 records a great performance is that it comes with an innovative design. The new models make use of a hybrid Mamba-Transformer architecture, referred to as Nemotron-Hybrid, that replaces most of the computationally demanding self-attention layers with Mamba-2 layers instead. This improvement is mostly helpful in speeding up the inference process when one is exposed to long sequence of information as in complex reasoning tasks. Such is the case with the Nemotron-Nano-12B-v2-Base which consists of 62 layers or 28 Mamba-2 layers, and just 6 layers of self-attention.

                                          source - https://research.nvidia.com/labs/adlr/files/NVIDIA-Nemotron-Nano-2-Technical-Report.pdf
A sophisticated compression strategy has been used so that this powerful model can be run on a single GPU (NVIDIA A10G). This entailed two core procedures: Pruning, which identified the less relevant model structures such as layers and hidden dimensions, and Distillation that retrained the pruned model through the remaining original model as a teacher to restore some portion of the initial model accuracy and re-strengthen the decision making.

Training and Alignment

The robust nature of the model is a direct response to a well tailored, but computationally expensive training session. Training with FP8 precision was a design trade-off between numerical accuracy and computational throughput, and it enabled training at a larger scale (20 trillion tokens) thereby enabling free scaling training. It is such efficiency that allowed Jim to develop the curriculum learning idea in which the model was presented with increasing better-quality data developing key strengths in a predictable order as opposed to a random presentation of data. The reliance on artificially generated data particularly to specific and multilingual tasks is a modern, data-informed effort to address the lack of quality human-labeled data and develop a truly adaptive foundation model basing on the ground up.

On top of this good foundation the alignment process that follows makes the model the tool which comes to refinement after a rough predictive tool into an advanced user friendly tool. Supervised Fine-Tuning (SFT) The first carefully idiosyncratic preparation with properly curated post-training datasets effectively teaches the model to learn and follow subtle directions in various fields. And yet it is a final crucial step in RLHF that allows honing the behavior of the model so that what it produces are not merely right but also helpful, safe, and do not go against any expectation of subtle rules of conversation with people. Such a two-stage fine-tuning process is necessary toward moving a model out of research application to a deployable product suitable to be used in the real world and to be deployed.

Performance Evaluation

Nemotron Nano 2 does not only claim efficiency, it achieves leading-edge performance on many industry-standard tests. The Nemotron-Nano-12B-v2-Base model has achieved especially good results in the field of mathematical reasoning. It scored an excellent 91.66 percent on the GSM8K CoT benchmarking and 83.54 percent at the MATH benchmarking offering it great results in resolving complicated mathematical tasks. Such scores reflect that the model can learn and operate very high mathematical concepts.

source - https://research.nvidia.com/labs/adlr/files/NVIDIA-Nemotron-Nano-2-Technical-Report.pdf

Other areas where the models perform well is in the management of long-contexts information. The 12B Base and 9B Base models scored 84.74% and 82.22%, respectively, on the Ruler-128K, which tests a model on how well they retain and are able to retrieve information over long sequences of text. It is a key feature that applications requiring work with large files, conversation histories, or reports will have up to 6x higher inference throughput in generation-intensive workloads.

source - https://research.nvidia.com/labs/adlr/files/NVIDIA-Nemotron-Nano-2-Technical-Report.pdf

In addition to these particular tests, Nemotron Nano 2 has passed numerous different tests as well. The 9B Base model has special strength on multilingual math problems. The models also obtain good scores on code generation tasks such as HumanEval+ and MBPP+. Moreover, in the working mode as the 9B-v2 reasoner, the model displayed a good performance on such benchmarks, as AIME25 (72.1%) or GPQA (64.0%), among open small models.

How to Access and Use Nemotron Nano 2?

In order to access and use Nemotron Nano 2, it needs to be downloaded first. The Nemotron Nano 2 is extensively available to the developer and researcher community by NVIDIA. All their models are published under the NVIDIA Open Model License (allows for commercial use) and the majority of their training data is made openly available on Hugging Face. They are also optimized on NVIDIA GPU accelerated systems such as the A10G and the A100 series and are made to run under Linux. It fits into the most-used runtime engines, such as Hugging Face Transformers and TRT-LLM or vLLM. Attention: we advise those users who use vLLM to add the flag --mamba_ssm_cache_dtype float32 to ensure high quality and avoid deterioration of performance.

Limitations

Although the Nemotron Nano 2 models are highly sophisticated in design with great features, there are significant limitations nonetheless. The main limitation is the hardware efficiency, as the 12.31 billion parameters model successfully deployed on a single NVIDIA A10G GPU is achievable only after a major tick with the approach to model compression through pruning and distillation. This demonstrates that the resource requirements of the uncompressed base model are much high. In addition to that, the models are highly optimized to run within NVIDIA GPU accelerated systems, which reduces the probability of usage with other hardware platforms. Lastly, their knowledge is not exhaustive; their factual resources are limited by the training data and have an expiration limit of September 2024 in the case of the 9B-v2 and May 1, 2025 in the case of base model.

Conclusion

NVIDIA Nemotron Nano 2 is an innovative Mamba-Transformer hybrid architecture that overturns the age-old trade-off between reasoning power, speed and cost. By providing state-of-the-art performance with many times greater throughput, and offering the ability to use a single GPU via judicious compression, it is able to deploy advanced AI in a practical, affordable way. It is an emerging template out to implement capable and economical reasoning models in the real-life applications.

Source
Tech blog: https://research.nvidia.com/labs/adlr/NVIDIA-Nemotron-Nano-2/
Research paper: https://research.nvidia.com/labs/adlr/files/NVIDIA-Nemotron-Nano-2-Technical-Report.pdf
Nano-12B-v2-Base : https://huggingface.co/nvidia/NVIDIA-Nemotron-Nano-12B-v2-Base
Nano-9B-v2 : https://huggingface.co/nvidia/NVIDIA-Nemotron-Nano-9B-v2
Nano-9B-v2-Base  : https://huggingface.co/nvidia/NVIDIA-Nemotron-Nano-9B-v2-Base

Disclaimer - This article is intended purely for informational purposes. It is not sponsored or endorsed by any company or organization, nor does it serve as an advertisement or promotion for any product or service. All information presented is based on publicly available resources and is subject to change. Readers are encouraged to conduct their own research and due diligence.