UI-Mate: Converting Screen Views into Platform-Agnostic Actions

Introduction

The main problem plaguing traditional desktop automation has been one of persistent structural weakness: the fragility of pixel-based coordinates or inflexible API wrappers that fall apart the instant there is an update to the GUI of the application. The GUI agents for foundations often find themselves struggling because of prompt ambiguity, oddities of local setups, and procedural subtleties, which cannot be captured in plain language.

The combination of environment-based reinforcement learning with visual demonstration learning solves this problem. With the introduction of screen pixels as the sole point of truth in real-time, the agent gains the ability to interpret the state of the interface and adapt to the drift of UI and even unusual interface layouts. This is precisely what UI-Mate does by allowing users to record a simple video of the task once and transform the video into coordinate-free subgoals, while constantly analyzing screenshots and providing an open-weight foundation model for executing multi-application computer tasks without using fragile scripts on Windows, macOS, and Linux.

What is UI-Mate?

UI-Mate is a  GUI agent framework which provides automation for long-term and multiple turn tasks in a native manner on any operating system. This project is maintained by the HY Frontier Team of Tencent. Being an agent which operates on raw pixel level, it converts language input and visual instructions into Platform agnostic mouse and keyboard actions on local software.

Model Variants

It comes in three different open-weight checkpoints:

• UI-Mate-9B: Developed from Qwen3.5-9B, used as a parameter-efficient GUI agent for generic computer manipulation.
• UI-Mate-27B: Developed from Qwen3.6-27B, used for long-horizon, application-specific software manipulation.
• UI-Mate-democua-27B: Fine-tuned using a special combination of generic computer usage data and demonstrations augmented trajectories. The model is capable of completing the task by following the demonstrations workflows while still maintaining the capability to follow instructions in the absence of any demonstrations.

Key Features of UI-Mate

• In-Context Demonstration Guided Execution (DemoCUA): Extracts a screen recording into a sequence of subtasks which are coordinate-free procedural checklists. It retains visual authority in real-time to dynamically omit repeated steps, add low-level actions (like scrolling and popups closure) and re-plan on the spot in case the screen deviates from the recorded one.
• Sharp Visual Reference and Cross-Platform Actuation: Processes the visual input in a normalized visual coordinate space of size 1000 × 1000 units, outputs tool calls for click actions, drag-and-drop, scroll, type text and key combination actions that can be resolved to physical monitor resolution.
• Long-Horizon Stability in Information: Retains very high stability of intent and coreference throughout a sequence of 100 interaction moves or more without any problems with context loss or drift of goal.
• Coordinate-Free Procedural Transfer: Evaluates spatial elements of the UI based on live screen view instead of copy-pasting of absolute coordinate values from recordings; hence, allows transferring of workflows across varying sizes and resolutions and visual themes of windows.

Use Cases of UI-Mate

• Zero-Shot Procedural Migration for Legacy Software Applications: Automates native, offline desktop applications (legacy ERPs, local database clients, or proprietary engineering tools) with inaccessible HTML DOM tree or API. Engineer records the workflow on an older OS, and UI-Mate locates the visual elements to execute the very same process in a new design layout of applications without any code refactoring.
• Visual Verification of Closed-Loop OS Sandbox Training: Facilitates RL agent training in the context of native OS sandboxing (Ubuntu, Windows, macOS). Data engine automatically performs randomization of desktop wallpapers, positioning of windows, and theme settings to avoid spatial shortcut learning. Step-by-step visual result verification through screenshotting validates milestones through Process Credit Model (PCM).
• Locally Hosted Automation of Processes in Secure Air-Gapped Environment: Distributes quantized FP8/INT4 models directly to locally hosted enterprise workstations either through vLLM or SGLang. Finance or government entities can automate multi-application processes such as querying of SQL GUI locally and processing of spreadsheets without going to cloud services.
• Environment Realism Benchmarking & Latency Auditing: Measures friction empirically on real-world desktop environment. Through testing, we found that processing instructions with noisy real-world spreadsheet data leads to 51.7% increase in trajectory length from 38.5 to 58.4 steps than when working with synthetic data. Profiling of latency shows that model prefill (1,597 ms) and decoding (314 ms) take up 91% of overall latency per step, setting benchmarks for hardware optimization of VLM serving.
• Screen-Observation-Only Security Control & Shell Defense: Secures enterprise network by limiting agents' interactions only to visual screen analysis and simulation of input actions, skipping direct execution of bash command-line or system shell API. It allows security specialists to constantly monitor trajectories recorded by the agent visually while human-in-the-loop control lets them to stop trajectory recording, analyze reasonings or even add their text instructions midway.

How does UI-Mate work?

UI-Mate has been engineered with an automated training data flywheel (see below img#1), an asynchronous reinforcement learning stack (illustrated in img#2), and runtime context injection. Its data engine is able to construct executable sandboxed executions on Ubuntu, Windows, and macOS, while randomizing file structures and environment visuals to evade synthetic bias. Coverage on the three-tiered capability tree (application domain, coarse capability, and fine-grained operation) is tracked to overcome training deficiencies, filtering trajectories through a multimodal judge and visual milestone check.

https://arxiv.org/pdf/2608.15930

Training is done using the Asynchronous Group-Relative Policy Optimization (GRPO) to reduce GPU downtime from OS rollout of variable duration. Rollout updates are made asynchronously as trajectory buffer becomes filled. To prevent policy staleness due to asynchronous rollout updates, IcePop discards isolated tokens having extreme ratio values, whereas SeqClip ensures that geometric means of token likelihoods do not drift. Decision-Turn Centering has been used to assign credits to actions to get rid of length biases due to failed trial-and-error approach, alongside with Token-Level Normalization which is meant to eliminate length bias in chain-of-thought output. There is an option to use Process Credit Model (PCM) for aligning active rollouts to milestone trees.

Source - https://arxiv.org/pdf/2608.15930

During inference, DemoCUA processes recordings to create subtask JSON files. These get injected as context blocks by the runtime engine. Intermediate low-level actions are intentionally excluded from training rollouts so that the model has to make those inferences visually using live screen shots. On macOS, the model uses an Accessibility (AX) API bridge for passing coordinates and resorts to PyAutoGUI click simulation if UI elements are not exposed directly.

Performance Evaluation with Other Models

In the core OSWorld-Verified testbed, the larger dense version of UI-Mate achieves a new open-weight state-of-the-art success rate average, whereas the parameter-efficient version performs exceptionally well. In this performance comparison, it has been established that UI-Mate performs way better than the dense foundation model, specialist open-weight agents, and large parameter scale Mixture-of-Experts architectures, and performs competitively against proprietary frontier flagships. The main implication of this finding is that dense and environment-informed reinforcement learning with pixel-wise actuation can do better computer control than parameter scaling and sparse mixture of experts routing.

source - https://ui-mate.github.io/

In the realistic OSWorkerBench dataset, the larger and denser model holds a prominent strict success rate and progress score relative to its smaller competitor. UI-Mate is designed to evaluate more complicated information flow and long-horizon problems involving many normalized applications running on the desktop, and UI-Mate performs significantly better than its own dense model and large general models in Multi-App and Long-Memory subsets. This result indicates the importance of late information and decision credit allocation of UI-Mate since keeping the visual state of applications over long time periods avoids goal degradation in complicated multi-application enterprise tasks.

source - https://ui-mate.github.io/

When considering Other evaluations, in which the larger dense model performs better than both the base model and large Mixture-of-Experts baselines and paired demonstration evaluation on the dedicated demonstration-guided checkpoint, the model demonstrates remarkable adaptability. Same task demonstration guidance helps to solve previously unsolvable tasks in public benchmarks and achieves higher strict success in office workflow subtasks. In special application scenarios such as Godot and QGIS, demonstration guidance helps to increase completion scores and reduce trajectory lengths. The importance of all of these benchmarks together is that the scaling of parameters of the model is the major factor for the complexity of multi-application sub-domains like Office, Daily, and Workflow tasks, while visual demonstrations help in eliminating exploratory paths.

How to Access and Use UI-Mate?

All model weights in Bfloat16 and FP8 quantized format are available on Hugging Face under the  license. Local execution scripts, service recipes and a Mac OS client for Apple Silicon computers are provided through GitHub and project website. This model works locally via vLLM or SGLang using standard OpenAI endpoints and parameters.

Limitations

Technical limitations include the current context window format which places the workflow injection at the beginning of the prompt, thus any update to the subtasks invalidates the shared KV-cache. Future work plans to place the workflow blocks at the end of the context in order to be able to utilize an append-only KV-cache. Furthermore, a limitation lies in the gap between specialized professional software compared to flagship products (75.5% vs 81.6% for Kimi-K2.6), as well as in late stage dropouts in multi-step chains. Improving on policy entropy collapse in RL training when using reasoning tracebacks and generalization of non-identical variant tasks are still works in progress.

Conclusion

UI-Mate represents the departure point for more flexible solutions than brittle coordinate-based scripting or prompt engineering, showing that the native control of computers visually is most effectively done via pixel-based reinforcement learning. By divorcing instruction of tasks from fixed positions on the screen, it gives open weight foundation models the flexibility necessary to perform complex multi-application workflows.

Sources:
https://ui-mate.github.io/
https://huggingface.co/papers/2608.15930
https://arxiv.org/pdf/2608.15930
https://huggingface.co/collections/tencent/ui-mate
https://github.com/Tencent/UI-Mate
https://huggingface.co/tencent/UI-Mate-27B
https://huggingface.co/tencent/UI-Mate-9B
https://huggingface.co/tencent/UI-Mate-democua-27B

Disclaimer - This article is intended purely for informational purposes. It is not sponsored or endorsed by any company or organization, nor does it serve as an advertisement or promotion for any product or service. All information presented is based on publicly available resources and is subject to change. Readers are encouraged to conduct their own research and due diligence.

GLM-5.3: Open Architecture Tailored for Advanced Cybersecurity Auditing

Introduction

Constant need to identify weak points within code and evaluate the state of underlying infrastructure has dramatically impacted the development of computational systems used for digital protection. Existing computational solutions usually face the problem of execution limitations while trying to make sense of huge volumes of code. With new generation of computational models, it is possible to develop multi-step reasoning without using human-made annotation of telemetry by creating artificial sandboxes for training. It is especially important when dealing with a large number of engineering pipelines and interrelated tasks.

GLM-5.3 can be called one of the flagships in the domain of cybersecurity precisely because it meets these demands. As opposed to being just a simple code assistant, GLM-5.3 serves as an autonomous engineering tool that allows identifying hidden architectural weaknesses and plotting whole chain of attacks. It is confirmed by the recent updates that can be found online.

What is GLM-5.3?

GLM-5.3 is a state-of-the-art flagship model that aims to facilitate the shift of artificial intelligence technology from producing standalone pieces of code to performing agentic engineering over the whole project lifecycle. Preserving the core structure of parameters from its predecessor, GLM-5.3 achieves its outstanding functionality thanks to the unprecedented scale of post-training computations and complicated synthetic environment design, allowing it to independently control enterprise-level projects with the complexity level of a few days of senior engineer’s work.

Key Features of GLM-5.3

• Mandatory Always-On Cognitive Processing: While previous versions could turn computational reasoning off, GLM-5.3 incorporates reasoning as a mandatory part of processing which happens in three explicit levels of low, high, and max and ensures that each result produced by the system is a result of an intensive analysis and not simple pattern recognition.
• Frontier Coding Advantage: Compared to its previous version, GLM-5.3 performs 50% better on strict internal coding tests and sets a new standard for open weights programming models working in complicated software structures.
• Complete Exploitation Reasoning Engine: Going beyond basic bug finding process, GLM-5.3 is equipped with state-of-the-art (SOTA) capabilities of vulnerability exploitation. Using logic chains, it significantly surpasses all the previous versions of reasoning by more than 2x times, turning vulnerabilities into exploits.
• Autonomous Complexity Management: In order to function in real conditions and not just demos, the model is capable of autonomously processing tens of thousands of lines of code in very interdependent multi-service systems without any intermediate prompting from a human.
• Integration of High-Velocity Goal Mode: High integration of the ZCode Graphical User Interface (GUI) helps create a seamless plan-test-verifiy cycle. This Goal Mode is able to provide an astounding cache hit ratio of 98%+, which greatly reduces computational overhead in long-running tasks.
• Token Economy for Agentic Actions: At its greatest computational capacity, the model is capable of solving a very difficult task using just 75,000 tokens (at a success rate of 34.5%) while previous models needed 96,000 tokens at a much smaller success rate.
• Remote Task Orchestration on Mobile: The model also allows the unique ability to conduct complex agentic actions from a mobile phone in WeChat and Feishu.

Use Cases of GLM-5.3

• Deep Archeological Analysis of Critical Legacy Infrastructure: The model is very proficient at performing security audits of the most ancient repositories. It can analyze the 40 years old code bases of kernel or browser engines, connecting legacy architecture assumptions with modern exploitation methods. As a result, it was able to discover the critical flaw, which appeared back in 1981 and stayed undetected until today.
• Black-Box Reference-Free RL Environment Generation: In case of scaling capabilities in some proprietary, classified, or brand new technology domain, for which there is no human reference available, the model will learn very fast by itself. It creates highly reliable reinforcement learning reward signals within artificial environments. Thus, capability deployment in highly restricted air-gapped or novel edge environments will be much faster.
• End-to-end delivery of the senior engineer project for multi-system software overhauling: Team may assign full project cycle of multi-system software overhauling to the model. It will automatically move through the process from problem recognition and deep analysis of systems to architectural design and production verification.
• Offensive/Defensive Proactive Cyber Security Chains Reasoning: The chain reasoning framework automatically conducts advanced Red Team enterprise operations. It does not only identify and report potential vulnerabilities but rather autonomously devises and tests multiple attack chains to prove the actual impact of the vulnerability through cryptographic means.

How does GLM-5.3 Work?

At the backend, GLM-5.3 is based on an extremely specialized 744B MoE architecture that makes use of a proprietary High-Throughput Slime MLOps pipeline to deal with the processing of large long-horizon reasoning tasks. These pipelines are generated by Z.ai itself and involve dynamic synthesis of task environments, hidden state, and dependencies. The parameter settings of the model's training, by drawing inspiration from the real-world professional systems, make sure that the model has integrated access to simulated computing clusters, localized storage, documentation, and repository information. In order to keep the computational overhead in check for such high-context models, a unique Hierarchical Caching mechanism is used by the system. This uses localized storage as an extension of model's state and data information, greatly minimizing host memory usage.

This training alignment is additionally enhanced by a refined Multi-Teacher Outcome-based Preference Distillation (OPD) framework. This framework allows for dynamic teacher selection and prefetching to enable the main model to learn and distill the logic from multiple experts in parallel without any delays due to multiple separate inference requests. The result of such design in the Slime framework is an incredible 99.99% decrease in the gap between training and rollout trajectories. Through the precise control of log-probability discrepancies on the 1e-7 scale, this architecture gains 2.3x performance improvements in the end-to-end Reinforcement Learning throughput, delivering exceptional mathematical stability when performing reasoning about complex security chains.

Performance Evaluation with Other Models

In performance evaluations that concern offensive security and infrastructure auditing tasks, GLM-5.3 introduces a novel paradigm on the CyberGym benchmark. Scoring 84.5% on the SOTA scale, the model significantly outperforms highly specialized frontier models such as Mythos 5 and GPT-5.6 Sol. This is because the benchmark, being naturally designed to measure a model's capability to operate in a live and strongly defended network topology, emphasizes the unique ability of GLM-5.3 to exploit isolated vulnerabilities in order to create complex multistage chains – something essential for top vulnerability hunters assessing the resilience of enterprises.

source - https://z.ai/blog/glm-5.3

In autonomous infrastructure management tasks, GLM-5.3 dramatically outperforms all other models on Terminal-Bench 3.0 by achieving a remarkable score of 28.3. This demonstrates a significant leap in performance compared to GLM-5.2, which scored 4.6, as well as compared to Claude Opus 4.8 with a score of 21.1. Terminal-Bench 3.0 benchmark evaluates a model's capability to natively work with CLI, to handle the complexity of filesystems, and to fix broken dependencies. This demonstrates its superior capability to operate in a raw and unstructured server environment without any GUI safety nets.

source - https://z.ai/blog/glm-5.3

In a wider range of tests, however, the performance gaps prove equally impressive. On ExploitBench, the system achieved a success rate of 54.4%, which is almost double the 24.4% achieved by its predecessor. For tests carried out in high-speed environments on ExploitGym Productivity, the system completed 105 tasks in two hours, whereas previous versions completed 29 tasks. Furthermore, its wide applicability in professional settings was validated through the GDPval-AA v2 tests, where it scored 1769 points in 44 different professions, purely from its 75K-token efficiency.

Opportunities for Offensive Architecture Evolution

Whereas GLM-5.3 has shown impressive ability in infrastructure auditing, can we possibly make it more advanced in deep offensive cybersecurity similar to other systems like GPT-5.6 Sol, Claude Fable 5, and Opus 4.8? In studying benchmarks on overall offensive capacity where GLM-5.3 obtains a success rate of 54.4% against GPT-5.6 Sol's 76.5% in case of ExploitBench, we are left with the following questions: how do we improve its 'Cyber Chain Depth' through long-term multi-step attack processes? As the current architectures of Fable 5 are able to maintain better coherence in logic in prolonged temporal periods through executing the most vague tasks, can we possibly train GLM-5.3 to independently design and implement full-fledged multi-level offensive packages after discovering a vulnerability?

In order to actualize the potential offered by such an approach, what specific technical enhancements might be incorporated within future iterations of the framework? Might we be able to transcend our current state of assisted environment synthesis and fully automate our process to include a fully autonomous dynamic and adversarial sandbox pipeline? In doing so, we could create many more robust RL reward signals specifically designed for end-to-end exploit chains instead of one-off exploits. Moreover, can we overcome the problem of temporal degradation of memory in long-term operations through improving the caching hierarchy system or implementing stateful memory pipes specifically designed for persistent red teaming? By constantly pondering how we can link together disparate past weaknesses into a more structured framework of persistence, we will be able to make the specific improvements needed to compete with frontier model systems.

How to Access and Use GLM-5.3?

Access to GLM-5.3 is currently limited to active GLM Coding Plan subscribers through the API with the points-based quota system. As for the local and structural integration, the ZCode GUI provides you with access to the continuous Goal mode planning. Besides, those developers who want to implement the model into their hardware or pipeline can download the open-weight version of the model from the GitHub/Hugging Face Weights repositories, which will be available to the public by the end of August 2026.

Limitations

Despite the powerful frontier capabilities of the design, there are some limitations on Cyber Chain Depth, where while the model is SOTA on the vulnerability analysis and initial exploitation mechanics, there are cases where the prolonged exploitation chains become inconsistent at the extremely large time scales. Besides, there are some limitations regarding the Pipeline Autonomy, which is dependent on human intervention for the creation of the final environment and is the key focus of future design developments.

Conclusion

The development of GLM-5.3 means the ultimate conclusion of the era when the LLMs were used just as assistants for conversations and the rise of the fully independent entities. The inclusion of such a model, controlled by the publicly available Security Disclosure Ledger, is an unprecedented chance for enterprise architects and security experts to solve the problem of decades-long technical debt.

Sources:
Blog: https://z.ai/blog/glm-5.3
Guide: https://docs.z.ai/guides/llm/glm-5.3

Disclaimer - This article is intended purely for informational purposes. It is not sponsored or endorsed by any company or organization, nor does it serve as an advertisement or promotion for any product or service. All information presented is based on publicly available resources and is subject to change. Readers are encouraged to conduct their own research and due diligence.

Muse Glimmer: Open Multimodal Private Fast Agentic Workflows

Introduction

There is an increasing requirement for custom-made solutions that can sustain continuous execution, maintaining the state over long-horizon workflows, and able to restart in case of failure, all without manual intervention. Moreover, there is an ever-increasing need for native support of multimodal reasoning, able to process the complex structure of real-world documents, screenshots, and images along with the text, without using additional performance-reducing wrapping software. The ability to use local processing of such a nature requires, in turn, the use of sophisticated techniques of speculative decoding, which allows one to avoid common inference speed limits.

This is precisely what Muse Glimmer provides, being  a product created specifically to be used in consumer devices in order to provide privacy-first local AI usage. Native visual comprehension together with high-speed block speculative decoding makes local agent execution not only possible but interactive and quick. As a custom tool for always-on agentic workflows, it allows developers, businesses, and researchers to call tools and maintain deep task memory offline. In this way, using Muse Glimmer gives total freedom of operations for enterprises, as their proprietary information will remain on local devices and be accessible due to logic and speed of its operation.

What is Muse Glimmer?

Muse Glimmer is a 30-billion-parameter foundation model engineered specifically for Optimised Local Agency, allowing it to execute frontier-level reasoning entirely on consumer-grade hardware like a Mac or PC. Distilled directly from its massive teacher model, Muse Spark, it shrinks super intelligent capabilities into a robust package that functions fully offline without an active internet connection.

Key Features of Muse Glimmer

• Controllable Reasoning Power: Developers are able to set the level of analysis power of the model through Low, Medium, High, and XHigh modes to ensure a perfect balance between the response time and needed reasoning power.
• Agentic Robustness (Failure Recovery): When the tool encounters any error or software timeout, it will diagnose the failure and repeat the execution process rather than stopping the whole process.
• End-to-End Task Resilience: By using self-managed memory and session state persistence, the model ensures coherence throughout hours of administrative tasks.
• Appropriate Information Flow (AIF): Data protection and privacy of the model are built into the core features of the model.
• Quantization Advantage: This model is specially designed for 4-bit quantization to make the language model fit within less than 20GB (and 17GB for K-Quant) while having just a 1.0% drop in performance on a standard VRAM of 24GB/32GB.
  

Use Cases of Muse Glimmer

• Zero-Dark Autonomous Factory Floor Maintenance: For AI Systems Engineer and Edge AI Architects tasked with maintaining zero-connectivity environments including nuclear power plants or highly secured factories. This is an autonomous system running through multi-hour shifts taking care of the maintenance and diagnostics of the factory floor machinery, recovering from software or sensor connectivity issues in the regular maintenance cycles, without the need for human input or data exfiltration.
• Exhaustive Private Corporate Archive Auditing: Useful for AI Privacy and Security Engineers and Technical Product Managers responsible for conducting audits of legal or governmental agencies. This system allows for a complete audit of large corpora consisting of text heavy documents, complex financial graphs, or high resolution images stored locally inside of secure vaults.
• Real Time Scientific Debugging and Simulation Feedback Loop: Customized to Scientific Computing AI Analysts and Hardware Acceleration Specialists who develop complex physics, biology or engineering models. Works as a live interactive lab assistant which monitors the long running simulation code and validates the mathematical theory of code execution in order to identify mistakes for scientists.
• Persistent Confidential Legal Strategy & Discovery System: Designed specifically for GenAI Application Developers and Legal Tech Executives building lasting solutions for intricate litigation cases. This system is capable of aiding attorneys in their week-long discovery work by compiling thousands of confidential documents into one, maintaining all context of the task even after a system reboot without having to reprocess any of the documents in question.
• Efficient Academic and Applied AI Research: Used by Chief Technology Officers and Academic Research Directors who work within restricted funding or grants. By doing this, it ensures that researchers at universities and independent research facilities are capable of conducting elaborate and multi-stage scientific investigations and document examinations from their local hardware systems without spending money on expensive cloud computing services or millions on server systems.

How Does Muse Glimmer Work?

The system uses a Dense Causal Transformer architecture that includes 29.6 billion parameters in 52 layers. In order to maximize computational efficiency while maintaining a large context window, it uses an extremely precise hybrid attention scheme where each iteration consists of three local layers followed by a global layer with the help of a sliding window size of 2048. Moreover, it has a frozen ~1.8B parameter ViT-G/14 perception encoder with 50 layers, which makes it possible to create up to 4,096 visual tokens per image, thus deep analysis of screenshots and dense documents.

source - https://research.meta.ai/blog/introducing-muse-glimmer-open-agentic-model

Its secret lies in the special Logit Distillation formula which helps it to extract the high-level reasoning of Muse Spark and stabilize it through a unique Mid-Training phase based on detailed, agent-heavy reasoning traces. Inference is powered by DFlash Speculative Decoding where a block-diffusion drafter is uniformly applied to the hidden layers 1, 13, 25, 37, and 49. This drafter suggests 16-token blocks at once while the main model validates them in parallel. The significantly faster generation achieved through this speculative decoding process can be seen in the screenshot above.

Performance Evaluation with Other Models

Mathematical Capability: Muse Glimmer sets a new milestone by obtaining an unprecedented performance of 94.7 on the AIME 2026 benchmark. It currently holds the highest possible score in its category, decisively beating much larger models. Handling complex logical mathematics with such an accuracy means that heavyweight computational reasoning is possible on edge devices.

source - https://research.meta.ai/blog/introducing-muse-glimmer-open-agentic-model

Agentic Reliability :  In terms of agentic reliability, it obtained 75.5 on the MCP-Atlas (Tool-Use) benchmark, thereby creating a significant margin over its direct open-weights competitor such as Qwen 3.6-27B (62.5) and Gemma 4-31B (54.2). In other words, this indicates its superiority in independently calling tools, analyzing the outcome, and making a sequence of decisions throughout the process.

source - https://research.meta.ai/blog/introducing-muse-glimmer-open-agentic-model

On the rest of the board, it was absolutely dominant, obtaining 43.6 in SciCode to beat Qwen, and beating Gemma and Qwen in DeepSearch QA (74.6), SWE-Bench Pro (51.2), and AA-LCR (80.0). Thus, it is evident that a specialized mid-size model can easily outperform much larger baseline models in coding, web browsing, and long-document reasoning.

Areas for Improvement: Muse Glimmer vs. Qwen 3.6-27B and Gemma 4-31B

Is Muse Glimmer the absolute silver bullet for all AI architects? When you compare it to models such as Qwen 3.6-27B or Gemma 4-31B, a few areas for improvement become obvious. If your aim is to shove all of your enterprise codebase into the prompt, then you'll reach its context limit much faster than when using the enormous 256K context of Gemma or 128K span of Qwen. What's more, it doesn't yet take advantage of the MLA architecture of Qwen, which means that it needs extra memory to work properly. Want to process native audio or video? Gemma will do it without any issues, whereas Glimmer currently deals with silent static frames and accepts a minor 1.0% performance trade-off when compressed to 4 bits.

Well, how could optimization engineers really address these areas? The process could be quite simple, but may require some experimenting. With the help of MLA, the engineers can reduce the memory footprint to such an extent that the context window will be able to accommodate 256K thanks to YaRN. As far as media senses go, there will be no need for an encoder; the engineers can easily add the audio to the system without increasing the size of the model, while the NaFlex vision encoder will help add video tracking at high resolution. Finally, QAT will take care of the 4-bit performance penalty.

How to Access and Use Muse Glimmer?

This model is quite easily accessible and deployable locally through frameworks such as MLX, ExecuTorch, and llama.cpp. Its main source of distribution is Hugging Face where it is available officially at the meta-models/Muse-Glimmer-30B repository. It is completely open-sourced and licensed under the very permissive Apache 2.0 license. In order to use it off-the-shelf without configuration, it can be simply plugged into local applications such as Ollama and LM Studio (for instance by running ollama run muse-glimmer:30b-mlx).

Limitations

Even though the model has state-of-the-art agentic ability, it still fails at multi-stage reasoning on unfamiliar tasks, which are significantly different from the ones used in its training process. It is not optimized for processing videos smoothly, considering video as just a series of images, and does not have audio input/output at all. Moreover, its efficiency reduces while it works in languages other than the main ones, and it is warned not to be used by people under the age of 18.

Conclusion

The release of the Muse Glimmer model is the last chapter of the era of clouds-only solutions. Showing that frontier-level multimodal analysis and failure recovery work perfectly well on a regular 24 GB GPU, the Muse Glimmer provides organizations with a permanent way to overcome the problem of data privacy limitations. If you are working on developing tools for security-sensitive applications, this is not just another model – it is a whole blueprint for creating localized systems.

Sources:
Blog: https://research.meta.ai/blog/introducing-muse-glimmer-open-agentic-model
AI Developer Center: https://developer.meta.com/ai/models/muse-glimmer/
Model Weights: https://huggingface.co/meta-models/Muse-Glimmer-30B
Developer Documentation: https://dev.meta.ai/docs/muse-glimmer
Evaluations Report: https://research.meta.ai/static/muse-glimmer-methodology

Disclaimer - This article is intended purely for informational purposes. It is not sponsored or endorsed by any company or organization, nor does it serve as an advertisement or promotion for any product or service. All information presented is based on publicly available resources and is subject to change. Readers are encouraged to conduct their own research and due diligence.

Qwen3.8-Max: A Self-Evolving AI For Complex Enterprise Workflows

Introduction

The journey of contemporary machine intelligence is more often than not traced by architectural designs that are able to maintain long-term, autonomous operation, thereby eliminating the necessity of consistent manual prodding. In terms of engineering, the real value of such systems rests in the ability of these systems to guide the whole cycle of any project from start to finish. This is facilitated through the built-in mechanism of dynamic self-correction, where the system is able to find the faults in logic, mismatched interfaces, and revise its strategy accordingly.

Professional engineers seeking a very capable digital workforce can make use of Qwen3.8-Max due to how it practically puts to use the concept of systemic self-reliance. It combines great parameter efficiency with its inherent self-verifying process to set a new precedent in solving complex problems when it comes to software and hardware designs.

What is Qwen3.8-Max?

Qwen3.8-Max is the premier multimodal foundation model within the Qwen family designed specifically to enable a new standard for coding, deep reasoning, and professional collaboration execution. Built on the sparse Mixture-of-Experts (MoE) architecture, the model reaches an immense scale of 2.4 trillion total parameters with a 1 million-token context window, deploying around 95 billion parameters in inference mode. Completely moving beyond static point generation, Qwen3.8-Max is designed as an autonomous self-evolving agentic system that can take any open-ended goal and turn it into a dependable asset with minimal external assistance.

Key Features of Qwen3.8-Max

• Ability to Work as Hybrid Agent: Has the unique capability of merging powerful code generation skills with real-time GUI activity such that it can actually sense the status of a live system and reconstruct applications from completely black-box status.
• Autonomous Evolution through Feedback Loop: Equipped with the unique capability of evolving by itself where it builds its own testing harnesses and these continuously evolve iteratively.
• Vision-based Intelligence at all Stages: Makes use of vision as the intrinsic feedback loop in order to observe and judge its own intermediate results like graphics misalignment and wrong orientation of objects physically.
• Long-Term Strategic Alignment Capability: Remains strategically aligned over thousands of rounds of interactions without forgetting the initial goal, thus allowing strategic reorientation even mid-task.
• Multimodal Large-Scale Ingestion and Processing: It Can ingest and process continuous video data up to more than 100 hours along with financial reports having more than 200 pages of content.

Use Cases of Qwen3.8-Max

• Black-box application reconstruction: Capable of autonomously reconstructing software applications completely from a black box without any access to the source code. It combines intense logic generation capability with real-time GUI sensing to observe the results and correct them till the perfect reverse engineering and reconstruction of the target system logic occurs.
• High-Performance Hardware Accelerator Design: It helps in designing and verifying high performance hardware such as GCD/RSA accelerator in area reduction constrained environments. The model maintains scaling of compute required for multi-day 500 MHz timing closure, making it possible to achieve the aggressive physical die area reduction not possible through static AI models.
• Self-Improving Autonomous Research and Methodology Optimization: Runs fully autonomous multi-week cycles of research that extend well beyond mere reproduction of previous results. It continuously optimizes its testing rigs and methodology used to make further gains over the original academic benchmarks.
• Hundred Hours of Multimodal Strategic Intelligence Visualization: Takes into account and correlates massive amounts of unstructured data, including 100+ hours of videos and dozens of pages of documentation, deciding between the generated images and agent inspection to build a highly-searchable, cohesive video memory database that is robust to the context decay.
• Highly-Efficient Strategic E-Commerce Asset Management: Operates the whole process of digital commerce, concentrating on achieving maximum profitability. It changes its strategic plans mid-task depending on the market noise.

How Does Qwen3.8-Max Work?

Technical architecture of Qwen3.8-Max is different from traditional sequential inference as it makes use of Loop Engineering Setup. This setup consists of an internal state machine for issues, task dispatcher, active monitor and watchdog. The task dispatcher, active monitor, and watchdog take charge of task management, moving task states from 'ready' to 'leased' and finally 'active.' In doing so, the model treats the tasks as stateful processes and not just text completion processes. In addition, to ensure stability of Reinforcement Learning in this diverse array of tasks, the model utilizes Online Data Balancer. This tool shapes training data batches dynamically in order to reduce gradient variance.

In the very heart of this reinforcement learning lies a concept of a Universal Reward System which brilliantly incorporates a range of different validation approaches into one consistent whole. Such a consistent reward system assesses the code check based on the principle of execution, the rubric-based assessment of rendered visual interfaces and the agentic validation at once. Also, the design incorporates an official parameter for reasoning effort in the API (with options for xhigh, medium, and low). It allows the end user to have full control over the level of the computational process inside the model itself.

Performance Evaluation with Other Models

In the high-stakes WWW2025 Multimodal Dialogue Intent Recognition Challenge, Qwen3.8-Max proved its capabilities of beating humans with its specialized performance. Being challenged to resolve the ambiguity of multimodal intentions in a dialogue, the model managed to create a winning solution in less than 24 hours independently. In total, this rapidly created solution outperformed 87% of 526 human teams competing in the challenge, securing its supremacy in rapid multimodal system generation among other proprietary models and talented human developers.

source - https://qwen.ai/blog?id=qwen3.8

On the E-Commerce Bench, which tests the ability of making long-term strategy for managing assets, the model managed to get an unheard-of financial result of 4.16x return (¥416,252). This result exceeds the performance of the flagship GLM-5.2 model by 38%, and it is more than 152% better than that of its immediate predecessor, Qwen3.7-Max. What makes this benchmark even more important is the demonstration of Qwen3.8-Max’s ability to switch strategies during the process of completing the task.

source - https://qwen.ai/blog?id=qwen3.8

Apart from commerce and conversations, the model demonstrates dominance in areas such as software engineering, mathematical research, and semiconductor design. Through a 16 day unaided operation on oh-my-cli project, it was able to commit 265 times and make 127 pull requests without any issue with feedback management and self-healing of states that went wrong. In hardware design advancements, the model autonomously optimized GCD/RSA acceleration by reducing the number of gates from 8,298 to 678 (resulting in 81% reduction in die area) and achieving timing closure of 500 MHz consistently. Additionally, it has shown tremendous research superiority by replicating a reasoning paper and autonomously improving the mathematics for a +2.7 point improvement on the AIME24 benchmark over the original paper.

How to Access and Use Qwen3.8-Max?

Qwen3.8-Max is currently available through cloud access via QwenCloud and Model Studio and provides an integrated interface with built-in support for both OpenAI-compliant and Anthropic-compliant API protocols for easy integration within the toolchain. It is highly optimized for local pairing with specialized developer tools such as the Qoder CLI and Qwen Code. Alibaba plans to release the Qwen3.8-Max model weights open source one week after its official launch as a milestone for massive-scale collaborative intelligence via Hugging Face Weights repositories.

Limitations

Whereas it has been designed to have an official capacity of one million tokens, Qwen3.8-Max has been found to be less efficient by five percent when processing huge volumes of information resulting in a 95 percent effective context window. Furthermore, solving complex problems in very complicated sectors such as the design of silicon requires high computation costs. The design calls for a high computational load of 500 turns of feedback loop and 71 distinct evaluations to balance the tradeoff of physical area and accuracy required in order not to make its reasonings stagnant at the stage of achieving basic optimization.

Potential Architectural Enhancements

The problem of computational overhead and context efficiency while performing ultra-long computations can be addressed by using a dynamic KV-cache compression system and introducing stateful linear attention hybrid layers. With such an improvement, the number of inference FLOPs could be decreased significantly to make the process of context preservation smooth with huge interaction horizons. Also, would it be possible to introduce an always-on micro-verifier that will ensure the critical bypass of reasoning is prevented despite the computational effort reduction being chosen?

In order to address the remaining ceilings in extreme STEM environments and in formal verification scenarios, it might be possible to introduce the integration of symbolic theorem provers within the framework of the reinforcement learning rewards. The fusion of informal reasoning techniques along with automated formal verification would result in achieving perfect proof accuracy. At the same time, the use of epistemic adapter networks might increase the level of rigour in domains such as life sciences and legal reasoning.

Conclusion

Instead of considering Qwen3.8-Max as a simple database of knowledge for inquiries, it ought to be used as an intent compiler, whereby the user dictates the intended design goal, and it takes care of all the processes involved in achieving that goal.

Sources:
Blog: https://qwen.ai/blog?id=qwen3.8

Disclaimer - This article is intended purely for informational purposes. It is not sponsored or endorsed by any company or organization, nor does it serve as an advertisement or promotion for any product or service. All information presented is based on publicly available resources and is subject to change. Readers are encouraged to conduct their own research and due diligence.

How Poolside's Laguna S 2.1 Runs 1M Context Code Agents Locally

Introduction

The world of software development today relies more and more on systems which are capable of conducting long multi-step development sequences. Instead of just being able to offer simple autocompletion of static code, modern intelligence has to be good at inference-time deliberation and dynamic verification. This implies that the models should be able to plan, execute, assess and fix their logic throughout the huge amounts of code without becoming inconsistent. In this environment, using a system architecture which will combine deep reasoning with limited memory resources becomes a very important strategic decision.

It is for this exact reason that Laguna S 2.1 has come to be such an exciting new coding agent model. It contains the level of reasoning that is required to coordinate autonomous, multi-file codebase actions in a manner that is fully implementable in the memory space of mid-tier equipment. This is accomplished by breaking away from conventional monolithic brute force and using an extremely optimized sparse structure that lets developers carry out ultra-long context periods, fast speculative writing, and recursive infrastructure optimization all on the edge node. For those who need air-gapped security, open trajectory validation, and fast local inference without the latency overhead of proprietary cloud endpoints, Laguna S 2.1 represents an uncommon fusion of frontier autonomy and unparalleled hardware accessibility.

What is Laguna S 2.1?

Laguna S 2.1 is an open-weight MoE model, featuring an astonishing amount of 118 billion parameters. Designed specially by Poolside to be employed for autonomous and long-horizon software development, Laguna S 2.1 works in the unprecedented 1-million-token context window. This model diverges dramatically from the common monolithic architecture in the sense that Laguna S 2.1 activates a small portion of its parameters in the process of inference. Rather than predicting the next token, Laguna S 2.1 is designed specifically to uncover the thought processes of human developers, thus becoming highly persistent.

Key Features of Laguna S 2.1

• Ultra-Effective MoE Model Structure: Even with its 118-billion-parameter structure, Laguna S 2.1 engages only 8 billion parameters per token. This creates the deep reasoning capabilities of an extremely powerful frontier model in an extremely efficient way.
• Natively Supported Quantization Variants: Laguna S 2.1 comes with natively supported 4-bit NVFP4 quantization format (along with FP8, INT4, and GGUF formats). This means no third-party tools for the process are required, and the model plus its context headroom fits easily into 128 GB of unified memory.
• Speculative Decoding Engine (DFlash): It features a special draft checkpoint called DFlash, providing huge boost to generation speed locally from 15 tokens per second to more than 80 to 117 tokens per second.
• Purpose-Built Pool Harness: The model is accompanied by a special native agentic coding harness called Pool, purpose-built to handle complex multi-stage engineering tasks on its own.
• Transparent Design and Evaluation Process: Poolside was developed in an extremely tight time frame of only 60 days . Unlike conventional black box benchmark scores, Poolside chose to reveal its full trajectory during the design process.

Use Cases of Laguna S 2.1

• Takeover of Air-Gapped Monolithic Codebase: Due to the ability to store the model with 1M context window and 118B parameters on one 128GB MacBook, NVIDIA RTX 3090, or DGX Spark node, it is exceptionally suitable for in-depth analysis of the legacy code monoliths. Enterprises may run numerous refactoring iterations locally without resorting to cloud-powered 64 accelerators clusters while keeping all proprietary data on-premises.
• Headless Proxying for Vision-Less Visual Verification: In very secure CLI-only environment when any multimodal inputs are limited, the reinforcement learning-powered resourcefulness of Laguna shines through. As an example, when developing a rendering engine, the model independently used headless Chromium for comparison of the generated canvas against the reference and created a code-based visual verification cycle without having a vision encoder itself.
• Self-Profile and Optimize Agentic Codebase Recursively: The model can be used for profiling and refactoring of its own execution stack. Acting as a senior architectural manager, Laguna S 2.1 independently discovered inefficient $O(n^2)$ and redundant memory copying within its own Pool harness and optimized it, decreasing memory allocations by ~70% and increasing speed by 5.2%.
• Scientific Discovery in Limited Sandboxes: The model exhibits remarkable logical persistence when dealing with scientific questions in outdated setups. It was capable of rediscovering the solution to the Erdős problem #397 (unsolved for half a century) by brute-force factorization and pattern analysis using only Perl in a limited sandbox.
• Fast Speculative Drafting in Long-Context Sessions: With this model being used as a co-pilot during extended hours of work, the user is capable of handling a large volume of context while getting suggestions in excess of 80 tokens per second. With the combination of NVFP4 quantization and DFlash draft model, long-context interaction becomes very plausible.

How Does Laguna S 2.1 Work?

Laguna S 2.1 is powered by a high-tech mixed attention design that is carefully optimized for handling the expansive 1-million-token window of Laguna S 2.1 without reducing the speed of the inference process. The design incorporates 12 global attention layers alongside 36 sliding window layers. This is done to ensure that any local dependencies in the code are handled quickly through the sliding window layers whereas the global attention layers handle the overall repository structure. In addition to this, the learning capabilities of the model are based on a unique reinforcement learning (RL) training technique where instead of teaching the model to output the correct code blocks, the RL pipeline makes the model go through the same planning and verification processes as a human engineer.

The key to Laguna S 2.1’s high speeds of local inference lies in the block diffusion speculator approach through its DFlash draft model. DFlash is a lightweight five-layer llama-style model that works based on predicting 15 consecutive tokens in advance. Because DFlash is lightweight, these tokens are predicted immediately. Then, the main 118B model checks this whole block of predicted tokens at once in one parallel check. If the tokens match the main model’s logic, then they are considered valid, and otherwise, they are ignored. This change from predicting one token at a time to checking an entire block in parallel shifts the economic focus of the model, thus increasing throughput from around 15 tokens per second to more than 80 tokens per second without decreasing the reasoning power of the 118B model.

Performance Evaluation with Other Models

Through rigorous benchmarking, Laguna S 2.1 has managed to prove itself to be the top-performing agentic coding model at its weight class. For instance, in the Terminal-Bench 2.1 which involves evaluating long horizon tasks whereby an agent is interacting through a terminal environment, Laguna scored 70.2%. This is very impressive since it beats the score by DeepSeek-V4-Pro-Max by 6.2%, even though DeepSeek activates 49 billion parameters while Laguna only activates 8 billion. The capability to score this high shows that the focus by Poolside of recovering the human planning process works better than simply scaling parameter numbers.

source - https://huggingface.co/poolside/Laguna-S-2.1

Apart from scoring high on the Terminal-Bench, Laguna also scored a respectable 40.4% on the DeepSWE test. This reinforces its superiority over other large frontier models. Though models such as the Claude Opus 4.8 may have performed better when evaluated as a whole, the hardware context in which Laguna performs makes it very impressive. For example, DeepSWE is very demanding and requires repository scale understanding and multi-file patching. This shows the validity of Laguna's 12 global and 36 sliding-window layer structure.

In addition to benchmark tests, the ability of the model was verified using very complex functional engineering tests. Running locally, Laguna S 2.1 created an entirely playable FPS game, designed a working macOS with authentic SVG icons, and even created an HTML/CSS rendering engine starting from an empty directory in just 50 minutes. 

How to Access and use Laguna S 2.1?

The model is made available in the form of open-weights and can be downloaded directly from its HuggingFace page. In the case of a local environment, it comes with seamless support for vLLM, SGLang, Transformers, TRT-LLM, Ollama, LM Studio, and a fork of llama.cpp adapted for the DFlash checkpoint. Alternatively, users who want to run the model in the cloud without having to set anything up can take advantage of the service of OpenRouter  and use it as an OpenAI replacement. The cloud version has built-in support for prompt caching, which can save up to 60-80% on costs if running the same prompts over large code bases repeatedly. Furthermore, users have access to a reasoning_details array through the API.

Limitations

despite its outstanding performance, Laguna S 2.1 exhibits several significant shortcomings. The lack of native vision functionality is perhaps the most pronounced of them, making it impossible to ingest images or UI screenshots and forcing it to use sophisticated workarounds to check the output of visual frontend through proxying canvas renders in headless Chromium instances. Additionally, although the model is known for the possibility to run on affordable hardware with NVFP4 quantization, using the unquantized version of the model with higher precision BF16 requires a colossal amount of VRAM – around 236 GB.

Future Directions and Technical Improvements

Would it be possible for the subsequent version of Laguna to overcome its limitations of a single agent and its dependence on vision through modularization of its execution substrate? The inclusion of lightweight pluggable vision adapters into its MoE routing process would make it possible for the model to inherently analyze terminal outputs and UI layouts without the use of external headless browser proxy or increasing its active parameter count. In addition to that, developing its agent harness into a multi-agent routing platform wherein specialized local agents operate concurrently and share a unified KV-cache memory could enable full stack swarm orchestration using local hardware, making multi-file backend, frontend, and database refactoring accessible to a single development computer.

In terms of infrastructure improvement, developing its hybrid attention system through memory-efficient advances such as Compressed Sparse Attention or KV-cache indexing with the style of IndexShare could drastically reduce prefill latency of long context sessions. Combining this sparsity technique along with the speculative decoding pipeline of Laguna could further reduce the memory overhead in the subsequent architecture. This would help facilitate autonomous software engineering of true project scale at high throughput on edge hardware.

Conclusion

By cramming the reasoning abilities of an MoE model with 118 billion parameters into 8 billion parameters, Laguna S 2.1 opens up a whole new realm of possibilities on mid-range hardware. For engineers, it allows performing ultrafast speculative decoding of 1 million token context window offline, making data security and cloud economics irrelevant barriers when employing state-of-the-art coding assistants. It allows the industry to get away from expensive cloud monoliths and opaque services in favor of transparent, highly recursive tool that not only can read and analyze codebases but also optimize its infrastructure.

Sources:

Blog: https://poolside.ai/blog/introducing-laguna-s-2-1

HuggingFace: https://huggingface.co/poolside/Laguna-S-2.1

openrouter: https://openrouter.ai/poolside/laguna-s-2.1

Disclaimer - This article is intended purely for informational purposes. It is not sponsored or endorsed by any company or organization, nor does it serve as an advertisement or promotion for any product or service. All information presented is based on publicly available resources and is subject to change. Readers are encouraged to conduct their own research and due diligence.