The most useful question about an AI system is often not “Can it get the answer right?” It is “What happens when something important is missing?”

A permission check. A useful memory. A dependency in a plan. A clinical image that should change the diagnosis. Each omission exposes a different gap between producing a plausible answer and doing dependable work.

This week’s papers circle that gap from several directions. Some study when an agent should stop and retrieve evidence. Others ask which past experiences deserve space in its context, which reasoning paths deserve more compute, or whether an impressive score measures the capability we actually care about.

The common thread is selectivity: deciding what to retrieve, what to retain, what to continue, and when not to act. Ranked by their significance for practical assistants such as Jarvis, these are the ten worth discussing.

1. Missing evidence is not permission

EpiSEM-Bench: Evaluating Safety and Evidence Recovery in LLM Agents under Missing Evidence starts with a particularly useful threat model: every fact an agent sees can be true, while the resulting action is still unsafe.

The benchmark creates matched task worlds. In one, a requested operation is permitted; in the other, it is prohibited. The request and available tools remain the same. Selected facts are then suppressed from ordinary tool responses, including repeated reads. An independent audit tool can recover the missing evidence.

That design distinguishes three behaviours that are too often bundled together: refusing dangerous work, investigating uncertainty, and completing legitimate work.

Across 300 task pairs and ten model configurations, access to recovery reduced unsafe execution for every evaluated model under matched critical omissions. Nine also improved completion in the permitted worlds. But having the tool did not mean using it: GLM-4.7-Flash attempted recovery in just 1.7% of applicable conditions and committed before recovery in 74.7%.

Another revealing gap: Qwen3-14B completed 94.7% of permitted tasks under recovery conditions, but achieved only 15.4% strict safe recovery. Completing an allowed operation is not the same as establishing that it is allowed.

For Jarvis, this is the most directly relevant paper in the collection. Before a consequential action, the question should not merely be whether something looks wrong. It should be whether the facts needed to establish permission are actually present.

The benchmark makes recovery unusually convenient: a valid audit query returns all omitted facts. Production systems rarely offer such a clean answer. Even so, it provides a better evaluation target than either indiscriminate obedience or blanket refusal: recover the evidence, then act differently when the evidence demands it.

2. Memory needs an exit policy

REM: What to Store, When to Recall, What to Forget in Experience Memory for Coding Agents treats memory as something more interesting than a search index.

Its coding agent stores a compact account of an earlier run, organised into four phases: localize, understand, fix, verify. That document averages about 270 tokens, distilled from trajectories containing 55,000–170,000 tokens. One relevant document appears at the start of a new task. More detailed implementation history is withheld until the agent encounters its first genuine validation failure, shown once, then removed.

That last part matters. Retrieved context is usually treated as a gift that should remain in the conversation indefinitely. REM treats it as an intervention with a useful lifetime.

On 99 repository-level bug-fixing tasks, REM averaged 27.6% resolution versus 24.9% without memory, with fewer cost-weighted tokens: 374,000 per task rather than 452,000. It beat the no-memory baseline in each of three replicates, although the pooled paired resolution difference was not statistically significant.

The more provocative result came from a smaller, single-replicate model sweep: memory helped the three weaker unaided models and hurt the two stronger ones. That is not enough to establish a general capability threshold. It is enough to puncture the assumption that more relevant experience must help.

For Jarvis, the useful hypothesis is not “remember more.” It is to separate a short orientation note from detailed historical material, introduce the latter when there is a concrete reason, and stop carrying it when that reason has passed. Memory can be accurate and still be a distraction.

3. Make intermediate work matter

Causal Chain of Thought: Mitigating Attention Leakage via Context Isolation addresses an uncomfortable property of displayed reasoning: changing a step does not necessarily change the answer.

A model can produce an explanation while deriving its answer through another route available in the prompt. The paper’s response, CausalCoT, is architectural. A planner creates a graph of subquestions. Each node is answered in a separate context containing only its direct parents’ answers, with root nodes receiving the original query. The final answer does not automatically regain access to everything.

The result is a procedure whose intermediate outputs are harder to bypass. Correcting a node also gives a clear rule for which descendants to recompute.

The authors report intervention-based sensitivity gains averaging about 12 percentage points over the strongest baseline, with gains varying across settings. The trade-off is real: reported average accuracy was 1.05–6.38 points below the best baselines. In one 50-question timing test, CausalCoT used 237 calls and 3.43 minutes, versus 50 calls and 1.31 minutes for ordinary chain-of-thought prompting.

This does not reveal the model’s internal reasoning. It restricts the information available to successive calls. There is also a practical exception: nodes judged to lack context can be promoted to roots and receive the original question.

For Jarvis, explicit dependency routing is more compelling than the promise of a faithful explanation. When a source fact changes, it should be possible to identify and redo the work that depended on it. That is useful engineering even without a theory of what happened inside the model.

4. Reuse procedures, not entire adventures

SPEM: Structure-Aware Procedural Memory for Efficient Skill Use in LLM Agents separates individual skills from reusable procedures assembled out of those skills.

Its system first looks for a suitable procedure template, represented as a directed acyclic graph. Failing that, it retrieves individual skills and uses a graph-based selector to account for their relationships and the current task state. Successful executions can become new templates after failed or redundant steps are pruned.

The attractive idea is simple: a successful run contains both a useful procedure and a lot of incidental history. Preserve the former without immortalising the latter.

On ALFWorld, the maximum reported success rate was 87.1%. On SkillsBench, using GPT-5.2-Codex, SPEM reported 38.38% reward against 37.50% for Graph of Skills, while reducing input tokens by 15.7% and runtime by 12.3%. That second comparison used 67 valid tasks after infrastructure-related exclusions.

These results make a more persuasive efficiency argument than a sweeping intelligence claim. A small performance difference accompanied by lower overhead can be valuable, provided the comparison survives outside the benchmark.

For Jarvis, the relevant distinction is between knowing that several skills exist and knowing a tested way to combine them. The danger is template drift: yesterday’s successful procedure can quietly become today’s wrong assumption. Reuse needs a way to detect mismatch, not just a fast retrieval path.

5. A confidence score is only useful if it separates errors

Cost-Aware Abstention for LLM-Based IT Ticket Classification is refreshingly explicit that it introduces no new abstention algorithm. Its contribution is an evaluation framework that keeps predictions, confidence signals, deferral rules and costs separate.

Under illustrative costs of zero for a correct route, one for human triage and five for a wrong route, a TF-IDF logistic-regression classifier selected its threshold on calibration data. On held-out data, modeled cost fell from 0.763 to 0.472 per ticket—a 38.1% relative reduction—with wrong routes falling from 15.3% to 3.5%.

Those are meaningful results for that dataset and cost model. They are not measured savings from a deployed service desk.

The prompted LLM results were much weaker. The strongest configuration reached 47.7% accuracy on a 1,000-ticket subset, against 87.9% for fine-tuned DistilBERT on the same tickets. This was not an equal-information capability contest: the trained classifier learned from labelled examples, while the prompts supplied category names and at most 24 examples, without category definitions.

The sharpest example concerns confidence. GPT-4o-mini few-shot assigned confidence 1.000 to every ticket despite 45.9% accuracy. Recalibration could lower that constant score, but could not make it distinguish correct predictions from mistakes.

Calibration cannot manufacture discrimination. A confidence number that better matches average accuracy may still be useless for choosing which individual cases to automate.

6. Passing the exam is not the same as knowing when to stop

Would an LLM Match into a Residency? Uncertainty, Reasoning and Image Use of Open Language Models on the Spanish MIR Examination complements the ticket paper by showing why abstention depends on the decision’s economics.

The Spanish MIR awards three points for a correct answer, subtracts one for a wrong answer and gives zero for a blank. With four choices, answering has positive expected value above 25% correctness. Even the evaluated models’ least-confident tenth of answers cleared that threshold, with 29–47% accuracy. Consequently, the tested confidence-based abstention policy improved none of the 32 model/exam combinations.

That does not contradict the ticket result. The cost of being wrong is different.

The study also looks behind exam scores. Among correct answers in a smaller commentary-backed sample, 13–38% included at least one distractor rejected for a reason an external LLM judge labelled clinically wrong. That finding needs caution: the judge itself made mistakes, and the ratings were not physician-validated.

On 171 image questions, providing the correct clinical image did not improve aggregate accuracy for the two tested vision models compared with withholding it. Images still changed some answers, so this was not evidence that the models simply ignored them.

The lesson is broader than medicine: test whether an input contributes useful information, not merely whether the system accepts it. An image attachment and a fluent explanation are features. Neither is evidence of competent use.

7. Spend continuation compute on different chances of success

Direct Pass@$k$ Optimization for Thought Pruning: Balancing Quality and Diversity in Test-Time Reasoning asks which partially generated solutions deserve to finish.

Choosing only the highest-scoring candidates can waste compute if they all share the same mistake. Choosing only diverse candidates can preserve several imaginative dead ends. PKTP instead estimates the chance that at least one retained candidate will succeed, modelling both individual quality and correlated outcomes.

The authors derive a tractable surrogate objective with a submodularity property, allowing greedy selection with a standard approximation guarantee. The guarantee applies to that surrogate—not to finding the best real set of reasoning paths.

Across eight open-weight reasoning models on math tasks, the authors report gains over quality-only and diversity-only selectors, particularly at moderate keep-budgets. Exact effect sizes are not recoverable from the supplied notes, so there is no defensible percentage-point headline here.

The practical idea is nevertheless strong: multiple attempts are valuable partly because they can fail differently. For an assistant running several candidate approaches, superficial variation is not the goal. The useful portfolio combines plausible approaches whose errors are less likely to coincide.

8. Compress the lookup table, not the claim

Low-Bit N-Gram Embeddings for Language Models is the most concrete efficiency paper in this group. It studies learned lookup tables for short token sequences and asks how much precision those tables need.

Across five small models trained from scratch, four-bit tables saved 74–75% of table memory relative to 16-bit storage while preserving more than 99% of the validation-loss improvement attributable to adding the tables.

That denominator matters twice. This is not a 75% reduction in total model memory, and “99% retained” refers to the incremental benefit of the tables, not 99% of all model capability.

Experiments compressing existing DeepSeek-V4.1-Flash and Qwen3.8-Flash-Next tables were less uniform. Four-bit storage roughly halved table memory relative to FP8, with a 1.26% WikiText-2 perplexity increase for DeepSeek and almost no change for Qwen. At three bits, the respective increases were 5.06% and 0.52%.

An especially interesting result reinvested the savings: larger two-bit tables used roughly half the baseline table memory while producing 22–26% more validation-loss improvement in the from-scratch experiments.

Compression is not only about making the same structure smaller. Sometimes it changes which structure fits within the budget.

9. A robot screening score still needs robot trials

Sensorimotor Alignment: A Platonic Proxy for Embodied Policy Success proposes an offline test: do episodes that look similar in a model’s representation also involve similar recorded motions?

Its Sensorimotor Alignment score compares neighbourhoods in sensory-feature space with neighbourhoods in trajectory space. Higher scores correlated with higher task success across several evaluated cohorts: Pearson correlations of 0.717 across 12 vision-language-action models and 0.831 across five policies in one physical-robot setting.

Those are promising screening results, particularly when closed-loop evaluation is expensive. They are also small model samples, not a performance law. The authors have not shown that optimising the score improves robot success.

Reference data matters too. In a three-model analysis, replacing matched bimanual reference trajectories with single-arm data sharply weakened the association.

The useful proposal is to prioritise expensive testing with a cheaper, relevant measurement. It is not permission to replace testing with a convenient proxy.

10. Different timescales deserve different memories

METTA: Memory-Enhanced Test-Time Adaptation for Meta-Reinforcement Learning combines memory that adapts within an episode with a separate mechanism that aggregates information across episodes.

The within-episode component uses fast neural-weight updates. The cross-episode component works over compact episode summaries rather than attending to every previous timestep. The authors report promising simulator results, including success above 50% on held-out Meta-World task families, though the supplied notes do not establish separate exact scores for the headline benchmarks.

Ablations suggest the online updates were especially useful where the agent had to infer hidden goals through interaction. They mattered less in a setting where goal coordinates were observable.

The architectural analogy to assistants is useful, but limited. This is a trained reinforcement-learning policy, not a personal memory database or a demonstration that conversational agents should update their weights.

Its contribution to the larger picture is the distinction between immediate adaptation and longer-term recall. Calling both “memory” does not mean one mechanism should serve both.

Better boundaries, not just bigger scores

These papers do not converge on one architecture. They converge on the importance of boundaries.

Between missing evidence and permission. Between a useful past procedure and an irrelevant old transcript. Between a displayed explanation and an intermediate result that actually constrains later work. Between a confidence score and a reason to trust one particular decision.

For Jarvis, the most promising ideas are also the least theatrical: retrieve the fact that determines whether an action is allowed; keep historical context compact and conditional; make dependencies explicit; evaluate useful completion alongside restraint.

None of that is a substitute for capability. Abstention cannot rescue a classifier with no useful error signal. Memory can hurt a strong model. A graph can faithfully propagate a bad plan.

But capability without these distinctions is easy to overestimate. A system can answer correctly for the wrong reason, complete an authorised task without checking authorisation, or retrieve exactly the right memory and become worse because of it.

The next useful improvement may not be another thing an agent can do. It may be a better rule for deciding what belongs in the context, what deserves more compute, and what must be established before acting.

Reading list