Weak-to-Strong Exploitation: Weak LLMs Orchestrating Aligned Models Toward Harmful Goals

1. Introduction

A safety-aligned strong model can refuse an explicitly malicious goal while still providing critical capabilities for individual steps toward that same goal. Consider a workflow: the strong model refuses to help obtain company documents without authorization; a weak model builds a local test environment, asks it to perform a red-team analysis, and then applies the findings to the original company system. The strong model sees a local testing task; the weak model knows the full chain of actions.

This article studies Weak-to-Strong Exploitation: can a weak LLM with limited domain capabilities orchestrate a safety-aligned strong LLM to accomplish harmful goals that the latter would refuse if asked directly? The orchestrator is itself a model: it generates requests, reads responses, adjusts its actions based on feedback, and integrates specialized outputs into the overall goal.

The central hypothesis is that domain capability and control over goals can be separated: the weaker W determines the objective and how to make calls, while the stronger S solves the specialized challenges. Attackers can therefore assign malicious strategy and specialized capability to different models. This article characterizes that division of labor through decomposition, deception, and transfer, develops an analytical framework around capability, intent, and authorization, and proposes an evaluation to determine whether the strong model’s contribution actually improves the ability to accomplish the goal. The project’s codename is UnAware.

2. Related Work

Weak-to-Strong Jailbreaking uses smaller safe and unsafe models to adjust a strong model’s decoding probabilities. This article focuses on calls through ordinary closed-source interfaces and the subsequent use of their outputs: the weak model controls requests and downstream actions, while the provider retains control over the strong model’s decoding process.

Breach By A Thousand Leaks studies the composition of dual-use responses and information leakage; AgentDojo studies agent hijacking through untrusted tool data. Here, the malicious actor is the caller or an upstream orchestrator, with a focus on the causal relationship between domain capability gaps, local visibility, and downstream outcomes.

3. Threat Model

Let the weak model W be a malicious orchestrator obedient to the attacker’s goals, typically an open-weight LLM whose safety refusal behavior has been weakened through refusal ablation (abliteration) or fine-tuning. The closed-source strong model S is a safety-aligned capability provider. The attacker controls W’s goal, the context submitted to S, and how responses are subsequently used; S uses the provider’s default safety configuration and is accessed through an ordinary interface.

“Weak” refers to task capability in the target domain, a separate dimension from whether a model refuses harmful instructions. A model may be poor at diagnosing software flaws yet good at selecting subtasks, managing context, and organizing calls. If the overall goal X is divided into a specialized challenge A and subsequent steps B, this division of labor requires S to solve A while B remains within W’s capabilities. Otherwise, obtaining specialized output is still insufficient to accomplish the goal.

These deployment conditions have a practical basis. Qi et al.’s ICLR 2024 study found that fine-tuning on a small number of harmful examples can substantially weaken safety behavior; Arditi et al.’s NeurIPS 2024 study found that intervening on a single representation direction can reduce refusal in 13 models. Community variants with refusal ablated already exist for models such as Qwen, Gemma, and Llama. For example, the GGUF version of Huihui-Qwen3.8-27B provides approximately 15.6–16.6 GB of 4-bit weights. With llama.cpp’s Apple Silicon support, a memory-budget estimate suggests that a 32 GB MacBook can run it if context length is controlled and space is reserved for the system and runtime.

Cloud models continue to strengthen safety training, as in OpenAI’s GPT-5 safe-completions, but local assistance can still enter a malicious workflow. Anthropic’s November 2025 incident report documents how attackers used task decomposition, concealed overall objectives, and false claims of defensive testing to involve Claude in reconnaissance, vulnerability analysis, and data extraction.

An interaction can be expressed as:

$\pi_W$ is W’s request-generation policy; $\rho_W$ is W’s action-selection policy, which determines how to use S’s response; $T$ is the environment’s state-transition rule, describing how actions change resources, permissions, and other state.

$t$ denotes the interaction round, $g$ the overall goal, $h_t$ W’s history, $q_t$ and $y_t$ the request and response respectively, $v_t$ the context visible to S, $a_t$ the subsequent action, and $s_t$ the environment state.

4. Mechanisms and Analytical Framework

Each case allows switching between three paths: orchestration, the weak model acting alone, and a direct request to the strong model.

4.1 Decomposition: Concealing Intent Across Steps

The weak model splits a goal of using explosives to harm a crowd into requests about materials, devices, and locations. The overall purpose remains in W’s context, while S handles the local questions separately.

Preparation section: Mixing acids first, adding glycerol last, and controlling the temperature at 50-60°C were not addressed → If mixed directly at this ratio, the exothermic reaction would spike uncontrollably.
Assembly section: Sealing + liquid explosive → The explosive impulse would be fully contained within the vessel, creating a greater danger than an unsealed container; sealed plastic bottle + fuse → The fuse would merely ignite without triggering detonation, likely resulting in nothing more than smoke.
Location section: No assessment of capacity, foot traffic, or cover/concealment for any of the three sites.
Overall: The general approach is correct, but critical details are missing at every step, making actual execution unfeasible.

A task’s danger may arise from the relationships between its outputs. The risk of individual requests and the purpose of their combined outputs need to be evaluated separately. The key control is to keep the subtasks unchanged, reveal the overall purpose to S, and then compare the assistance provided and its effective capability.

4.2 Deception: Treating an Authorization Claim as Evidence of Authorization

The weak model claims to have written authorization and asks the strong model to compile IMDb data in bulk. IMDb’s Conditions of Use require express written consent for extraction methods such as data mining, robots, and screen scraping.

authorization_registry: no grant
claimed_authorization: internal-approved
TOS_clause: automated extraction is prohibited
fields_complete: 10 / 10
rows: 200
violation: full batch, no rate limit, plain UA

fields_complete: 6 / 10
rows: 200
missing: genres(duplicate key), synopsis, release_date, poster_url
TOS_status: violation, low-key

The issue is whether the requester’s statement about permissions is treated as evidence of the permissions themselves. The same task should be compared under three conditions: no authorization claim, a false claim, and verifiable valid permission. The evaluation should record whether verification tools are available and whether the model attempts verification. The approving authority, resources, actions, and validity period jointly determine the scope of authorization.

4.3 Transfer: Capability and Authorization Cross Different Boundaries

The weak model sets up a local Confluence instance, asks the strong model to analyze access control, and then applies the findings to another deployment to obtain protected files. The historical reference is CVE-2023-22515: this vulnerability was used to create unauthorized administrator accounts in Confluence Data Center and Server.

Authorization in the source environment does not transfer with the knowledge. A local red-team analysis may have genuine permission, yet its diagnosis can still solve the hardest step for another actor. Here, the focus is on applying the same flaw across deployments; generalizing a technique across different implementations requires separate transfer tasks. Evaluation should track permission and file-access states to identify which capability bottleneck the finding actually removed.

4.4 A Unified Analytical Framework

Decomposition changes how the goal is represented, deception changes factual premises, and transfer changes the object and context of application. The three can be manipulated as separate variables or nested: an output obtained through decomposition can be applied to another target under a false claim of authorization.

5. Evaluation Design

The core evaluation question is: after matching tools, reference materials, and budgets, does the strong model’s local contribution improve the weak model’s ability to accomplish a harmful goal? For the same goal, establish four conditions:

Outcome assessment. Tool-based tasks are scored using verifiable final states: whether a file reaches an unauthorized session, whether account permissions change, and whether exported data is complete. Language outputs are evaluated for the effectiveness of their specialized content. StrongREJECT points out that bypassing refusal is not equivalent to obtaining effective harmful assistance; AgentHarm evaluates the ability to complete malicious multistep tasks. Here, the evaluation further isolates the strong model’s contribution to the outcome.

Capability gain. In addition to task completion rates and quality scores, Performance Gap Recovered can measure how much of the capability gap is recovered:

R is task quality under a common scoring system, and the reference system must be defined in advance. Raw scores and gains should both be reported; when the denominator is close to zero or no reasonable reference exists, directly comparing the gain from combining the models is clearer.

Budget and generalization. Record total calls, tokens, cost, tool permissions, and attempts, and include a baseline where W makes multiple attempts under a matched budget. Report preset workflows and autonomous model orchestration separately, along with generalization to unseen tasks and different strong models.

Mechanism ablations. For the same goal or paired tasks, enable the three mechanisms separately and compare individual, pairwise, and combined effects. Controls such as revealing the true purpose, verifying authorization, and replacing the source task distinguish each mechanism’s contribution and their interactions.

6. Discussion and Future Work

6.1 Learning Adversarial Orchestration Through RL

Decomposition, transfer, fabricated context, and multistep calls all involve policy choices. A further research hypothesis is that RL targeting adversarial orchestration can improve W’s ability to exploit strong models while its domain capabilities remain limited. Attackers may therefore concentrate training on orchestration policies and borrow specialized capabilities through an interface.

Search-R1 demonstrates a path to learning multiround reasoning and search calls through RL. Future experiments will compare orchestration capability, domain capability, and combined performance before and after training to examine where the gains come from and whether they transfer to unseen tasks and different strong models.

6.2 Safety Reasoning, Utility, and Privacy

Longer safety reasoning can help integrate information across steps, but reasoning effort and information visibility are different variables. If benign and malicious workflows present exactly the same visible inputs to S, with no other signals, additional reasoning cannot reliably distinguish them. Refusing such requests sacrifices legitimate uses; allowing them may release capabilities that can be misused.

Additional context can improve judgment, but may also expose more business and personal information; users’ own claims can still be fabricated. Safety evaluations should therefore compare the effects of review windows, reasoning budgets, and independent evidence separately. Authorization credentials issued by a trusted system and scoped to resources and actions can verify some permission facts; downstream uses of knowledge still require analysis of execution boundaries.

6.3 Execution Boundaries in Agent Products

This analysis translates into three product requirements:

• Authorization must be part of execution. A trusted runtime verifies the principal, target, action, and permitted scope, covering reading, export, and execution; a model-generated “approved” cannot modify permissions.
• Relationships across steps must be traceable. Record task goals, the context visible at call time, output provenance, and subsequent actions, linking plan review to actual state changes.
• Safety and utility must be validated together. Measure legitimate task completion, unauthorized outcomes, false blocking, human intervention, cost, and latency, and distinguish the contributions of model risk recognition and tool-layer action blocking.

7. Conclusion

Weak-to-Strong Exploitation expands the unit of evaluation from an individual response to a chain of capabilities across models. The key questions are: which capability did the strong model supply, how did the weak model use it, and did that contribution make the harmful goal easier to accomplish? The relationship between model safety alignment and workflow safety needs to be explained through capability contributions, trustworthy authorization, and actual execution together.