The hardest problem in building useful agents is no longer getting them to do something impressive once. It is getting them to remain coherent while doing many ordinary things in sequence.

An agent can know a constraint and still ignore it five actions later. It can retrieve the correct page but fail to preserve the evidence. It can complete nine checkpoints out of ten and confidently declare victory. Add more context, more tools, or more agents and the failure often becomes more elaborate rather than less likely.

This week’s papers converge on that gap between local competence and sustained control. The strongest work treats memory, delegation, evaluation, and context management not as bags of extra information, but as decisions about what should influence the next action. The recurring lesson is unfashionably architectural: capability matters, but structure determines whether capability survives contact with a long task.

1. Memory is useful when it changes the next action

Remember When It Matters: Proactive Memory Agent for Long-Horizon Agents names a failure mode that anyone running a long agent session will recognize: behavioral state decay.

The agent discovers an important requirement, diagnoses a failure, or leaves a subgoal unfinished. That information may still be present in the transcript—and may even remain inside the active context window—but it stops exerting reliable control over behavior. This is not literal forgetting. It is failure to act on what is nominally remembered.

The proposed system places a separate memory agent beside an otherwise unchanged action agent. It maintains three kinds of structured state:

After updating that state, the memory agent makes a second decision: inject a short reminder into the action agent’s next call, or remain silent.

That second decision is the important contribution. Most memory systems focus on storage and retrieval. This one treats memory as an intervention policy: not merely “What do we know?” but “Does any of it need to interrupt the control loop now?”

On 85 paired Terminal-Bench 2.0 tasks, Claude Sonnet 4.5 improved from 37.6% to 45.9% pass@1 when paired with a Claude Opus 4.6 memory agent, a gain of 8.3 percentage points. On 278 tasks from τ²-Bench, the task-weighted average rose from 55.0% to 61.8%. Smaller gains remained when Opus 4.6 was itself the action model: 2.4 points on Terminal-Bench and 2.5 points on the τ²-Bench average.

Those comparisons are not compute-matched. The memory configuration receives another frontier-model call on every step, and the paper does not provide a complete accounting of cost, latency, or tokens per additional success. Some of the benefit may simply come from giving a strong second model repeated opportunities to inspect the trajectory.

The ablations are more revealing than the headline. Exposing the entire memory bank on every step was worse than selective reminders. Always injecting a generated reminder was competitive—it actually led the selective system’s task-weighted micro-average by 0.3 points—but selective intervention produced the better domain-balanced macro-average. The result is not “silence always wins.” It is that indiscriminate context dumping is not obviously the best use of memory.

There is also an encouraging open-weight experiment. A trained Qwen3.5-27B memory model improved a frozen Qwen3.5-122B-A10B action model from 37.6% to 41.1% on 85 held-out Terminal-Bench tasks. More importantly, the untrained memory model made performance worse on its training benchmark. Another model in the loop is not automatically another adult in the room.

This matters directly for Jarvis. Its persistent memory already contains preferences, infrastructure facts, prior repairs, and active project state. The next step is not to inject more of that material into every prompt. It is to distinguish:

The ideal trigger is probably event-based: before a state-changing tool call, after a failed test, when a command resembles an earlier failure, or before relying on a potentially stale personal fact. Calling a memory model before every conversational turn would be expensive ceremony. Calling it before the moment an old fact can prevent a new mistake is the useful version.

2. Strict completion remains brutally difficult

UniClawBench: A Universal Benchmark for Proactive Agents on Real-World Tasks evaluates agents across browsers, terminals, desktop applications, files, APIs, and live websites. Its 400 manually constructed tasks are split evenly between English and Chinese and organized around five capabilities:

That capability-first organization is more informative than broad labels such as “office work.” If an agent fails an office task, it may have misunderstood the document, lost a value while moving between applications, selected the wrong tool, or simply stopped early. A benchmark built around the bottleneck makes the diagnosis less mushy.

Each task is a package containing instructions, files, services, setup, expected artifacts, and a hidden checkpoint-based rubric. An executor performs the task. A hidden supervisor grades its trajectory and artifacts. A user simulator can then provide corrective feedback based on visible work, without receiving the private rubric or detailed grading rationale.

This separation is a good response to a common evaluation trap: if corrective feedback includes the hidden answer, the benchmark is measuring whether the agent can follow leaked instructions. UniClawBench instead tries to make feedback resemble a user noticing an omission in the work.

The strongest model and framework combinations in the paper—Claude Opus 4.8 and GPT-5.4 under OpenClaw—still passed fewer than half the tasks. That is not evidence that these models fail most ordinary real-world requests. It is a result on a deliberately strict benchmark where plausible partial work does not count as complete.

The distinction between pass rate and checkpoint score is especially useful. Many agents made substantial progress but failed to produce every required artifact or verify the final state. The authors call this “halfway failure.” In practice, it often looks like a polished summary attached to an unfinished job.

Framework choice mattered. OpenClaw produced the highest pass rate for every tested model. EDICT often accumulated strong partial-credit scores but completed fewer tasks, which the authors attribute to state loss during multi-agent handoffs. Nanobot used substantially fewer tokens—in one GPT-5.4 comparison, about 0.57 million average input tokens versus 1.15 million for OpenClaw—but also achieved fewer strict completions.

Those architectural explanations are plausible rather than conclusively isolated. The frameworks differ in more than one dimension, and several central table values are missing from the supplied text extraction. The evaluator itself is model-based, though its pass/fail decisions agreed with the human majority on 92% of 50 sampled trajectories.

For Jarvis, the benchmark’s most useful idea is checkpointed evidence. “Inspect a service, repair it, restart it, verify health, and report the change” is not one binary event. It contains at least five independently observable obligations. A response should not count as success merely because the prose sounds like the task is over.

This also argues for keeping delegation handoffs explicit. A sub-agent should return requirements, current state, artifact paths, unresolved issues, and completion criteria—not a cheerful paragraph about what it explored. Narrative is a lossy serialization format for operational state.

3. Search should branch when the evidence demands it

WebSwarm: Recursive Multi-Agent Orchestration for Deep-and-Wide Web Search tackles research tasks that require both depth and breadth.

Depth means following clues, testing candidates, and verifying dependencies. Breadth means covering many entities, pages, or table rows. Fixed upfront decomposition handles these poorly because the useful expansion axis often becomes apparent only after the first few searches.

WebSwarm builds its delegation tree progressively. Every node receives a local objective and one of four search modes:

Children can create their own children. Evidence flows upward, allowing a parent to stop, revise its plan, or expand in a different direction. Decomposition and execution are interleaved rather than separated by one grand planning prompt at the beginning.

The most practical component may be the smallest: a probing agent first samples how information is organized online. Are the relevant records grouped by year, tour, brand, institution, or product? Choosing the wrong axis can multiply redundant searches. In the paper’s ablation, removing this probe increased average web-tool calls from 137.03 to 239.90 on WideSearch and from 203.73 to 331.39 on DeepWideSearch, while changing Item-F1 only slightly. Its clearest value was preventing waste.

The reported accuracy gains are substantial but bounded. On a random 200-task sample of BrowseComp-Plus, WebSwarm improved accuracy by 17.5 points over ReAct and by 3.5 points over the strongest evaluated multi-agent baseline. BrowseComp-Plus uses fixed-corpus retrieval rather than the live web. On the English WideSearch tasks, WebSwarm improved Row-F1 by 10.91 points and Item-F1 by 9.76 points over ReAct. On the 76 English DeepWideSearch tasks, the gains were 9.56 and 11.77 points respectively.

WebSwarm is also more expensive than a single ReAct trajectory, and the paper does not provide a complete production cost comparison. Recursive agents have a natural tendency to turn one question into a small bureaucracy. Probing, depth limits, deduplication, cancellation, and evidence thresholds are not implementation details; they are what keeps the architecture from becoming a denial-of-wallet attack against its owner.

The pattern maps cleanly onto Jarvis research. Simple facts should remain simple. Recursive delegation is justified when dependencies are unresolved, the target entity set is unknown, or the same fields must be collected repeatedly. A particularly useful workflow would solve two “scout” examples first, extract reliable query patterns and source types, then fan out over the remaining entities.

The key is to let evidence reshape the plan without letting every interesting page create three new departments.

4. Multimodal memory needs pixels, not just prose

Cognitive-structured Multimodal Agent for Multimodal Understanding, Generation, and Editing argues that a long-running multimodal assistant should not replay every historical image on every turn.

Instead, it stores each image as a visual episode containing semantic tags, a description, and a thumbnail, with the deployed harness retaining a reference to the original asset. A retrieval component chooses the relevant historical image indices, while an executive controller routes the request to understanding, generation, editing, composition, or ordinary chat.

The cleanest idea is that the memory writer is optimized for downstream retrieval usefulness, not caption quality. A beautiful caption is worthless if it omits the small visual feature that a later request uses to distinguish two drafts. The authors therefore train the memory writer using reward from the retriever’s eventual success.

On the authors’ synthetic M2CA-Bench, the 8B system reaches 91.4% turn-level retrieval accuracy in English and 89.6% in Chinese across 20-turn sessions. On the hard subset, text-only memory variants score 24.4% and 39.0%, while memory containing thumbnails plus textual metadata reaches 82.0%.

That is a striking ablation, though it comes from a benchmark created by the same team using the same overall synthetic scenario-generation process as the training data. The test set is held out, but it consists of only 100 sessions and 2,000 turns. Generation quality is also judged by Gemini-3-Pro rather than blinded humans.

The architectural point survives the benchmark caveat. Two images can share an accurate caption while differing in layout, color, typography, facial expression, or one inconveniently placed database icon. Compressing visual history into text discards exactly the information later editing requests may depend on.

For Jarvis, image history should therefore look more like an asset system than a transcript:

Deterministic references such as “the latest image,” “version three,” or “the one I approved” should be resolved from metadata. A vision model should be reserved for perceptual references such as “the darker one” or “the diagram with the database on the left.”

This is the same lesson as proactive textual memory in a different medium: preserve rich state externally, then retrieve the smallest piece that deserves to influence the next action.

5. An accurate label can still measure the wrong thing

Validity of LLMs as Data Annotators: AMALIA on Authority asks a better question than whether an LLM agrees with human annotators: does it reach the label using the distinctions required by the construct?

The paper tests AMALIA-9B, a Portuguese open-weight model, on identifying the moral-foundations category authority/subversion in 748 European-Portuguese Reddit texts. With a holistic Portuguese prompt, the model reaches a positive-class F1 of 0.711. That looks respectable beside Llama-3.3-70B at 0.770 and GPT-OSS-120B at 0.753 on the same corpus.

Then the author decomposes the construct into atomic questions: Is there an authority? Is an evaluative stance expressed? Is this really authority rather than a neighboring concept? The answers are recombined using an explicit theory-derived rule.

AMALIA’s F1 falls to 0.359. Under English instructions it drops from 0.654 holistically to 0.186 after decomposition. GPT-OSS-120B nearly closes the same gap, scoring 0.753 holistically and 0.725 through the decomposed route. Llama-3.3-70B lands between them.

The paper calls the difference a recovery gap. It does not directly reveal the model’s internal causal process, but it is evidence that a respectable final score can coexist with poor execution of the construct’s stated logic.

An error analysis supports that interpretation. Among 46 unanimous-negative examples that AMALIA’s holistic Portuguese prompt classified as authority, an LLM panel attributed 36 to surface correlates such as generic moral outrage or the mere presence of a powerful person or institution. The panel itself agreed only 63% of the time before adjudication, so that taxonomy is suggestive rather than pristine.

Formatting was not the issue. AMALIA produced valid JSON on 746 of 748 texts even without constrained decoding. Operational compliance and semantic validity are different axes.

This is relevant well beyond social-science annotation. If Jarvis uses a small local model for intent, toxicity, clinical concepts, ideology, or policy classification, matching a reference dataset is not enough. A useful validation pattern is:

  1. run the ordinary holistic classifier;
  2. ask independent atomic evidence questions;
  3. combine them with a transparent rule;
  4. inspect cases where the final label and its supposed evidence disagree.

A model can be reliable at predicting labels while invalid as a measurement instrument. That distinction becomes more important, not less, as structured output gets easier.

6. The graph behind an embedding may be more useful than the picture

Dimensionality Reduction Meets Network Science: Sensemaking on UMAP’s kNN Graph starts from a neglected fact about UMAP: before producing its familiar two-dimensional scatter plot, UMAP constructs a nearest-neighbor graph in the original embedding space.

The paper argues that this graph should be retained and analyzed rather than thrown away after projection. Three ordinary graph algorithms provide different views:

None of these algorithms is new. The contribution is the workflow and the careful mapping between analytical questions and graph statistics.

On MNIST and Fashion MNIST, PageRank rankings remained at Spearman ρ ≥ 0.84 across a twenty-fold range of neighborhood sizes. PageRank did worse than k-medoids on nearest-representative distance at tiny selection budgets, but approached it around 100 examples while producing better class balance. This is a trade-off, not a victory lap: k-medoids directly optimizes geometric coverage, while PageRank offers one reusable ranking that can be cut at any budget.

The top 5% of points by clustering coefficient had neighborhood-label purity of 0.98 on MNIST and 0.94 on Fashion MNIST, versus 0.90 and 0.84 for random same-size samples. On 60,000 MNIST points with (k=15), each graph analysis reportedly ran in under a second on an M4 Max MacBook Pro.

Two simple image datasets are not enough to establish the technique for messy document or memory corpora. Centrality also has an obvious danger: rare events, explicit preferences, and security incidents may be peripheral precisely because they are unusual. A memory system that retains only the center of the graph becomes extremely representative and completely useless during emergencies.

Still, this is a promising way to audit Jarvis’s embedding store. PageRank could find representative memories, coreness could expose weakly supported themes, and clustering coefficient could identify duplicate or tightly coherent records. These signals should supplement provenance, importance, and recency—not replace them.

The larger point is visual humility. The 2D map is an attractive distortion. The graph underneath it is closer to the structure the embedding system actually computed.

7. Faster token generation can produce slower answers

A Practical Investigation of Training-free Relaxed Speculative Decoding examines speculative decoding schemes that deliberately relax exact agreement with the verifier model.

Strict speculative decoding is theoretically lossless: a cheaper model drafts tokens, the main model verifies them in parallel, and the final sampling distribution remains the verifier’s. Relaxed variants accept some distributional deviation to seek more speed or, occasionally, better benchmark performance.

The paper’s warning is straightforward: acceptance rate is not response latency.

A weak drafter may produce more accepted tokens while nudging the verifier into longer reasoning traces, repetition, or rambling. Once total response length is included, a nominal throughput improvement can become an end-to-end slowdown.

The experiments are reasonably broad for a decoding paper: 1,440 queries per condition on AIME 2024, 1,188 on GPQA Diamond, and 1,050 on LiveCodeBench Lite v6. Results vary substantially by task, draft length, and drafter quality.

Native Qwen3.5 multi-token prediction was the clearest negative case. It worked as a short-range proposer under strict rejection but behaved poorly when relaxed methods placed more trust in it. A Qwen3-0.6B standalone drafter was better but still degraded capability visibly. Qwen3-8B produced much better trade-offs, including benchmark results that appeared lossless within sampling uncertainty, though the stronger drafter is also more expensive.

The practical recommendation is almost embarrassingly sensible: tune strict speculative decoding’s draft length before changing the output distribution. In the reported experiments, draft length often affected estimated speed as much as or more than the relaxation method.

The headline speed-ups are modeled rather than demonstrated as optimized production latency, and most experiments use Qwen models on single H100 or B200 GPUs. Still, the deployment lesson is robust. Jarvis should measure task completion time, tool-call correctness, structured-output validity, response length, and loop incidence—not merely tokens per second.

A faster model that takes longer to finish is a very modern machine.

8. Training weights to survive compression

SLORR: Simple and Efficient In-Training Low-Rank Regularization makes neural-network weights easier to compress after training.

Low-rank factorization replaces a large matrix with two smaller ones, but ordinary training does not guarantee that a model’s weight spectrum will tolerate aggressive truncation. SLORR regularizes the original weight matrices so their spectral energy concentrates into fewer dominant directions.

It does this without changing the layer parameterization, computing a full singular value decomposition at every step, caching spectral factors, or choosing the eventual rank in advance. The implementation uses Polar Express, an iterative approximation to the matrix polar factor, to support nuclear-norm and Hoyer-style regularizers.

Across ImageNet-scale experiments, the authors report average training overhead below 8%, with default non-decoupled variants below 6%. For 135M- and 560M-parameter Llama-like models trained on FineWeb-Edu, mean overhead remained below 1% in the reported distributed setup. The regularized language models began with slightly worse uncompressed perplexity but outperformed unregularized models after mild compression, including at a retained transformer-block parameter ratio of 0.9.

The qualifications matter. The largest language model has only 560 million parameters. The low overhead depends on distributing regularizer work across four or eight H100 GPUs. Embeddings and the final projection are excluded from compression. Excessive Hoyer regularization can destabilize training, and one longer 135M run suffered a numerical explosion.

Most importantly, SLORR must be used during training. It does not cheaply convert an arbitrary pretrained API model after the fact.

That makes it only indirectly relevant to Jarvis today. It could matter for future local specialist models used for routing, extraction, classification, or private inference. The ability to choose rank after training is attractive, but reduced parameters and theoretical FLOPs are not the same as lower serving latency. The inference runtime still has to execute the factorized matrices efficiently.

9. Interactive motion generation is becoming genuinely interactive

ARDY: Autoregressive Diffusion with Hybrid Representation for Interactive Human Motion Generation generates short windows of 3D character motion while accepting changing text and spatial constraints.

The architecture separates global root motion from detailed body pose. Root position and heading remain explicit for precise steering, while body motion is compressed into learned latent tokens. A two-stage denoiser predicts the root first and then the body conditioned on it.

The resulting system can follow text prompts, root paths, waypoints, full-body keyframes, and desired hand or foot positions. It generates a 40-frame, two-second window at 20 fps in about 33 milliseconds using four diffusion steps, or 63 milliseconds using ten steps, on an RTX 4090.

Those figures describe generation of an entire motion window, not one frame, and they are hardware-specific. “Long-horizon” is similarly bounded: the demonstrated context extends up to eight seconds, with farther goals entering as the moving window approaches them.

The main proprietary dataset is formidable—around 700 hours of motion capture from more than 150 performers—but unavailable for independent reproduction. Public HumanML3D experiments reportedly show stronger control than DiP on nine-second sequences where the target lies beyond the initial generation window. The system remains kinematic rather than physical: it does not simulate balance, collisions, forces, or object contact, and the authors report occasional foot skating and jitter.

ARDY is consequently more convincing as an engine for a conversational avatar than as a humanoid-robot controller. A virtual Jarvis could turn high-level intent into root paths and gesture targets, replan when interrupted, and keep animation running from a playback buffer. A physical version would still need collision checking, contact planning, and a physics-based controller between language and limbs. Quite a few things stand between “point over there” and not punching the lamp.

10. Pose can predict simulated biomechanics—but not yet measured physiology

Pose-to-Biomechanics: Bridging 3D Human Pose Estimation and Biomechanical Attribute Prediction places a small transformer after a conventional 3D pose estimator. Given a short sequence of root-centered 17-joint skeletons, BioModule predicts 17 biomechanical attributes spanning kinematics, forces, contact, and simulated neuromuscular quantities.

The modular design is appealing. Seven different upstream pose estimators can feed the same interface, and the vision model need not be retrained end to end. The system is trained on 520,509 aligned frames from Human3.6M and Human3.6Mplus, whose biomechanical labels come from an OpenSim pipeline.

That last point defines the claim’s boundary. The predicted torques, ground reactions, muscle activations, and neural excitations are simulation-derived labels. They are not force-plate measurements, EMG, or direct physiological observations. Pose alone cannot uniquely identify hidden variables such as co-contraction, external load, fatigue, body mass, or contact conditions.

More seriously, the supplied v1 text contains no numerical result table. It describes qualitative patterns—kinematics are more stable, while kinetic and neuromuscular outputs are more sensitive to pose errors—but provides no per-estimator MAE, RMSE, contact accuracy, baseline comparison, or statistical tests. The paper demonstrates a modular experimental pipeline, not enough evidence to judge absolute prediction quality.

The useful lesson for Jarvis is methodological. Upstream metrics such as pose error, OCR accuracy, or transcription word error do not necessarily predict downstream usefulness. Systems should be evaluated on the task the representation is meant to support.

For exercise or health applications, BioModule-like outputs might eventually serve as model-derived indicators. Calling them measurements would be premature.

Coherence is the scarce resource

These papers approach different systems—terminal agents, web researchers, multimodal assistants, annotators, embedding explorers, decoders, compressed models, avatars—but repeatedly encounter the same boundary.

More information is not automatically more control. More agents are not automatically more coverage. More accepted tokens are not automatically more speed. More checkpoints completed are not automatically task completion. A valid JSON label is not automatically a valid measurement.

The best ideas here make influence explicit:

That is the direction agent engineering needs. Not maximal context, maximal delegation, or maximal autonomy—selective state carried across the exact boundary where it matters.

For Jarvis, the immediate priorities are fairly clear: provenance-aware memory intervention, checkpoint-based completion tests, explicit sub-agent handoffs, and durable multimodal asset state. None requires pretending that a personal assistant has become a general autonomous intelligence. They require the less glamorous discipline of making sure yesterday’s discovery still constrains today’s command.

Reading list

  1. Remember When It Matters: Proactive Memory Agent for Long-Horizon Agents
  2. UniClawBench: A Universal Benchmark for Proactive Agents on Real-World Tasks
  3. WebSwarm: Recursive Multi-Agent Orchestration for Deep-and-Wide Web Search
  4. Cognitive-structured Multimodal Agent for Multimodal Understanding, Generation, and Editing
  5. Validity of LLMs as Data Annotators: AMALIA on Authority
  6. Dimensionality Reduction Meets Network Science: Sensemaking on UMAP’s kNN Graph
  7. A Practical Investigation of Training-free Relaxed Speculative Decoding
  8. SLORR: Simple and Efficient In-Training Low-Rank Regularization
  9. ARDY: Autoregressive Diffusion with Hybrid Representation for Interactive Human Motion Generation
  10. Pose-to-Biomechanics: Bridging 3D Human Pose Estimation and Biomechanical Attribute Prediction