Repository navigation
feat: generate seed LoRA adapters for serving many seeds at once - #12
Open
yuval-peleg-levy-ai21 wants to merge 4 commits into
Open
yuval-peleg-levy-ai21 wants to merge 4 commits into
yuval-peleg-levy-ai21 wants to merge 4 commits into
Conversation
scripts/make_seed_lora.py writes one PEFT LoRA adapter per seed. The two factors of each targeted linear layer are drawn from torch.randn seeded with the seed and scaled so the applied delta has per-element std noise_scale, the counterpart of the dense perturbation's sigma. Only config.json is read, so generation runs on CPU. A stock vllm serve --enable-lora then holds every seed simultaneously and clients pick one per request through the OpenAI model field, which removes weight swapping between requests entirely. README states the caveat up front: a low-rank delta with the same per-element std has a much larger operator norm than dense noise, so it is not a drop-in replacement and needs a smaller noise_scale. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Replaces the prose caveat with two result tables (Qwen2.5-32B-Instruct, rank 16) so readers can see where seed LoRAs match full-matrix seeds and where they degenerate. Aligns the usage example with the rank used in those runs. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Keeps the README section to usage and the low-rank caveat; the pass@k tables for facts-search and Countdown follow the analysis/<topic>/README.md layout used by randsearch and one_step_gradient. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Independent of the worker-extension and server changes I am sending separately. Adds
scripts/make_seed_lora.py, which writes one PEFT LoRA adapter per seed so a stockvllm serve --enable-loraholds every seed at once and a client picks one per request through the OpenAImodelfield (model=seed7). No weight swapping between requests, no worker extension, no custom server.For each targeted linear layer the two factors are drawn from
torch.randnseeded with the seed and scaled bysqrt(noise_scale * sqrt(rank) / alpha), so the applied delta(alpha/rank) * B @ Ahas per-element stdnoise_scale, the counterpart of the dense sigma. Onlyconfig.jsonis read; generation for a 32B model takes seconds on CPU. Same(model, seed, rank, alpha, noise_scale)reproduces the same bytes.Caveat, with results in
analysis/seed_lora/README.md. A rank-r delta with the same per-element std concentrates its energy in r directions, so its operator norm is aboutsqrt(d/r)larger than dense noise. This is a cheap way to serve many seeds side by side, not a drop-in equivalent of the dense search. On Qwen2.5-32B-Instruct (rank 16, greedy), 200-example multi-hop retrieval test set, unbiased pass@k: 10 seed LoRAs atnoise_scale=0.001reach pass@1 0.450 / pass@5 0.612 / pass@10 0.660, versus 0.429 / 0.572 / 0.609 for 18 dense seeds at sigma 0.001 and 0.436 / 0.478 / 0.490 for 20 temperature-1.0 samples. On Countdown,noise_scale=0.01made five or six of six seed LoRAs degenerate into repetition, andnoise_scale=0.001was coherent but low-diversity; dense sigma 0.001 top-10 reached pass@1 0.525 / pass@10 0.800 there. Both tables live inanalysis/seed_lora/README.md; the README section keeps usage and the caveat.requirements.txtgainssafetensorsandfire.Verification
Generated adapters for
hf-internal-testing/tiny-random-LlamaForCausalLM(rank 4, noise_scale 1e-3): measured delta std per layer 8.2e-4 to 1.0e-3, different seeds produce different tensors, two runs produce byte-identical files. Adapters for Qwen2.5-32B-Instruct were served with vLLM--enable-lora --max-loras 20and used for the retrieval numbers above.🤖 Generated with Claude Code