25 papers across AI, ML, NLP, and CV from the last 24 hours.
Three themes dominate today's batch. First, the agent memory problem is getting serious attention. Multiple papers tackle how agents store, update, and retrieve experience across sessions — not just short-context recall but the kind of accumulated workspace knowledge that separates a one-off demo from a reliable tool. LongMemEval-V2 evaluates whether agents internalize environment-specific failure modes, while MEME maps the full space of multi-entity and evolving memory with tasks no prior benchmark covers. Pair this with ToolCUA's work on GUI-tool path orchestration and it becomes clear: the community recognizes that raw reasoning is no longer the bottleneck; persistent context is.
Second, multimodal unification continues to accelerate. SenseNova-U1 argues that the understanding/generation split is a structural limitation rather than an engineering convenience, and builds a native unified architecture accordingly. AlphaGRPO extends GRPO to unified multimodal models without cold-start, enabling self-reflective refinement where the model diagnoses and corrects its own outputs. OmniNFT applies reinforcement learning to joint audio-video generation — a first for multi-objective, multi-modal alignment.
Finally, post-training reliability is getting harder scrutiny. Reward hacking in rubric-based RL gets formalized with cross-family judge panels, and a sparse-to-dense reward allocation principle challenges the standard practice of running GRPO directly on deployment models. The implicit message: we've been training on the wrong signals.
One paper stands out for reframing the problem entirely. "From Web to Pixels" introduces Perception Deep Research — the idea that visual perception sometimes requires external fact resolution before pixel-level localization is even possible. A visible object whose identity depends on recent events or multi-hop relations can't be recognized by a frozen model; it needs to search first, then see. This collapses the boundary between perception and research.
The batch collectively suggests the field is moving from isolated model improvements to systems-level questions: how agents remember, how modalities share representations, and how training signals match real-world reliability needs.
Miaosen Zhang, Xiaohan Zhao, Zhihong Tan · 2026-05-12
Computer-use agents automate on-screen work, yet their reliability on complex, low-frequency interactions remains poor. Analysis of failure cases from advanced models reveals a long-tail pattern in GUI operations, where a small fraction of complex interactions accounts for a disproportionate share of task failures, largely due to scarcity of training data for complex interactions.
Haiwen Diao, Penghao Wu, Hanming Deng · 2026-05-12
Large vision-language models remain constrained by a persistent dichotomy between understanding and generation, leading to fragmented architectures and misaligned representation spaces. This paper introduces SenseNova-U1, a native unified multimodal paradigm that treats understanding and generation as a single problem rather than distinct tasks.
Christen Millerdurai, Shaoxiang Wang, Yaxu Xie · 2026-05-12
Reconstructing absolute 3D hand pose from a single head-mounted camera is crucial for AR/VR and telepresence. Monocular RGB methods struggle with depth-scale ambiguity and generalization across diverse head-mounted optical configurations. EgoForce addresses this using forearm guidance to constrain camera-space hand reconstruction.
Bokang Yang, Xinyi Sun, Kaituo Feng · 2026-05-12
Visual perception typically assumes decisive evidence is already in the image or frozen model knowledge. This paper studies open-world cases where a visible object must first be resolved from external facts, recent events, or multi-hop relations before localization. Formalized as Perception Deep Research with a new benchmark and dataset.
Yihao Meng, Zichen Liu, Hao Ouyang · 2026-05-12
Autoregressive video generation aims for real-time, open-ended synthesis, but cinematic storytelling requires progressing through evolving events, viewpoint shifts, and discrete shot boundaries. Existing models treat long sequences as extended single shots, suffering from motion stagnation and semantic drift during long rollouts.
Runhui Huang, Jie Wu, Rui Yang · 2026-05-12
Applies Group Relative Policy Optimization to AR-Diffusion Unified Multimodal Models to enhance multimodal generation without a cold-start stage. Enables reasoning text-to-image generation where the model infers implicit user intents, and self-reflective refinement where it autonomously diagnoses and corrects its own outputs.
Jiahe Li, Jiawei Zhang, Xiao Bai · 2026-05-12
Presents AmbiSuR, a framework that explores an intrinsic solution upon Gaussian Splatting for photometric ambiguity-robust surface 3D reconstruction. Investigates two built-in primitive-wise ambiguities in Gaussian Splatting representations and proposes methods to resolve them for accurate surface reconstruction.
Nicholas Ikechukwu, Keanu Nichols, Deepti Ghadiyaram · 2026-05-12
Continual Panoptic Segmentation requires adapting to new categories over time. Existing methods group all unlabeled pixels into a single background class, conflating diverse unseen categories. FuTCR introduces future-targeted contrast and repulsion to better handle this dense prediction challenge.
Wufei Ma, Chloe Wang, Siyi Chen · 2026-05-12
Modern simulation platforms present steep technical barriers requiring extensive graphics expertise. LychSim provides a highly controllable and interactive simulation framework built on Unreal Engine 5 to bridge this gap for closed-loop optimization and out-of-distribution evaluation in vision research.
Huiran Duan, Qian Zhou, Zhongliang Guo · 2026-05-12
Conventional gait de-identification methods face a trade-off between insufficient identity suppression and spatiotemporal distortions. GaitProtector formulates privacy protection as a unified objective with obfuscation (repelling from source identity) and impersonation (adopting a target identity) without requiring retraining.
Di Wu, Zixiang Ji, Asmi Kawatkar · 2026-05-12
Long-term memory is crucial for agents in specialized web environments. Existing benchmarks focus on user histories or downstream task success. LongMemEval-V2 directly evaluates whether memory systems effectively internalize environment-specific experience, interface affordances, state dynamics, workflows, and recurring failure modes.
Ariel Gera, Shir Ashury-Tahan, Gal Bloch · 2026-05-12
Explores LLM-guided query refinement for extending embedding models to zero-shot search and classification tasks. Refines embedding representations using feedback from a generative LLM on a small set of documents, enabling embeddings to adapt in real time to the target task across diverse search and classification benchmarks.
Chen Li, Xiaoling Hu, Songzhu Zheng · 2026-05-12
LLMs often produce answers with high certainty even when incorrect. Verbalized confidence provides a flexible uncertainty signal applicable when token logits are unavailable. ORCE introduces order-aware alignment to improve confidence estimation by addressing the misalignment between answer generation and confidence generation.
Rian Touchent, Eric de la Clergerie · 2026-05-12
When adapting an encoder to a new domain, standard practice is continued MLM training. This paper shows that temporarily switching to Causal Language Modeling followed by a short MLM decay improves downstream performance by +1.2-2.8pp on biomedical texts with ModernBERT across French and English tasks.
Shauli Ravfogel, Gilad Yehudai, Joan Bruna · 2026-05-12
How do transformer language models memorize factual associations? This paper develops a theoretical and empirical account of geometric memorization where learned embeddings encode relational structure directly and the MLP plays a qualitatively different role from the associative memory view requiring linear parameter scaling.
Kexuan Shi, Hanxuan Li, Zeju Qiu · 2026-05-12
Introduces Pion, a spectrum-preserving optimizer for LLM training based on orthogonal equivalence transformation. Unlike additive optimizers like Adam and Muon, Pion updates weight matrices through left and right orthogonal transformations, preserving singular values throughout training while modulating weight geometry.
Rishabh Tiwari, Kusha Sareen, Lakshya A Agrawal · 2026-05-12
Parameter updates for downstream tasks can cause catastrophic forgetting and loss of plasticity, while in-context learning with fixed parameters adapts rapidly but can't match parameter update gains. This paper explores combining both approaches toward continually adapting LLMs.
Yuanda Xu, Hejian Sang, Zhengze Zhou · 2026-05-12
When verifiable training data is the binding constraint, each checked example should be allocated carefully. Proposes a reward-density principle: sparse sequence-level reward should train models where exploration is productive, while dense token-level teacher reward should train deployment-optimized models.
Alireza Nadali, Patrick Cooper, Ashutosh Trivedi · 2026-05-12
A simple, training-free long-context inference protocol treating the KV cache as an accumulator in a left fold over sequence chunks. At each step the model processes the next chunk conditioned on accumulated cache, appends new keys and values, and passes the enlarged cache forward.
Guinan Su, Yanwu Yang, Xueyan Li · 2026-05-12
Even advanced AI agents function on message exchange formats in a single stream of computation — a bottleneck. This paper explores Multi-Stream LLMs that enable parallel streams of thoughts, inputs, and outputs to unblock the single-stream architecture that has persisted since early instruction-tuned models.
Tom Sander, Hongyan Chang, Tomáš Souček · 2026-05-12
A state-of-the-art watermark for LLMs building on Gumbel-max sampling with dual-key generation to restore output diversity, entropy-weighted scoring, and multi-region localization for improved detection. Supports speculative decoding without inference overhead and dominates SynthID-text in detection strength.
Jacob Fein-Ashley, Paria Rashidinejad · 2026-05-12
Looped Transformers iteratively refine latent representations, improving language modeling and reasoning, but remain unstable to train. Attractor Models propose a backbone module that first proposes output embeddings, then an attractor module refines them by solving for the fixed point with gradients obtained through implicit differentiation.
Xuhao Hu, Xi Zhang, Haiyang Xu · 2026-05-12
Computer Use Agents act through both atomic GUI actions and high-level tool calls, but hybrid action spaces leave them uncertain about when to continue with GUI actions or switch to tools. ToolCUA addresses this through optimal GUI-Tool path orchestration with interleaved trajectory learning.
Seokwan Jung, Alexander Rubinstein, Arnas Uselis · 2026-05-12
LLM-based agents operate in persistent environments requiring storage, updates, and reasoning across sessions. MEME defines six tasks spanning the multi-entity and evolving axes, including three not scored by prior benchmarks: Cascade and Absence (dependency reasoning) and Deletion (post-removal state).
Anas Mahmoud, MohammadHossein Rezaei, Zihao Wang · 2026-05-12
Studies reward hacking in rubric-based RL where a policy is optimized against a training verifier but evaluated against a cross-family panel of three frontier judges. Separates verifier failure from reward divergence, reducing dependence on any single evaluator for measuring true reward hacking.
This digest is generated automatically from arXiv submissions. Not affiliated with arXiv or Cornell University.