Repository navigation
Add MicroHack 10: Offline Industrial Copilot (Foundry Local) - #594
Draft
Digvijay (Digvijay) wants to merge 9 commits into
Draft
Digvijay (Digvijay) wants to merge 9 commits into
Digvijay (Digvijay) wants to merge 9 commits into
Conversation
New lab 10_Offline_Industrial_Copilot: customer developers build a maintenance-technician copilot that runs entirely offline on Foundry Local. - 6 challenges with solutions: set up, choose an edge model, first local agent (function calling), agentic retrieval over SOPs with citations, train in the cloud and deploy at the edge (ONNX, with a shortcut), and cut the cord. - Reference code, SOPs, machine data, evaluation sets and a pre-built ONNX fault classifier with a read-through notebook. - labautomation: Azure Machine Learning workspace and an optional Windows 11 VM (Bicep validated with az deployment group validate). - Reference code validated on Foundry Local 0.11.0 (Windows ARM64, NPU/GPU). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Deployed the lab Bicep and ran the lab on a Windows 11 VM, an offline run with the internet blocked, a pre-cached model folder and the Azure Machine Learning training on the lab's workspace. Temporary resources were deleted. Fixes from the findings: - main.bicep: install the Visual C++ runtime. Without it the Foundry Local service stops at start-up on a fresh Windows image. - agent.py: the write tool is only offered when the technician asked for it, the app (not the model) reports the ticket result, and an answer that claims a ticket exists when no ticket tool ran is replaced. The model had claimed a ticket was opened when it never called the tool. - evaluate_retrieval.py: recognise refusals worded differently. - check_routing.py: new, counts how often the right tool is chosen first. Docs now carry the measured results and the open items: offline restart breaks the model catalog, pre-cache steps (config set --force), Azure ML timings, and the fragile tool routing. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Routing: a small model picked the wrong tool for 39% of held-out requests (14/23 correct, fault prediction 0/5) when it saw all four tools. ToolRouter matches the request to a short card per tool with the embedding model that is already loaded, drops tools that need a machine ID when none is named, and shows the model only the best match. Held-out requests: 23/23 on the laptop GPU variant and 23/23 on the CPU variant. Tuning set: 31/32. Showing two tools was worse (26/32), so the router picks and the model fills in the arguments. Offline start: the foundry service cannot resolve any model after a restart without the internet. serve_offline.py starts the service from the Foundry Local SDK with a catalog that fails at once, finds the models in the local cache and serves them; fl.py reads what it serves when FOUNDRY_LOCAL_URL is set. Also adds routing_eval.json and routing_holdout.json and rewrites check_routing.py to score them. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…sults - solution-05: a new task on making tool choice reliable. Held-out requests, no router 14/23 (fault prediction 0/5), router 23/23 on the GPU variant and 23/23 on the CPU variant. Showing two tools was worse than one. - solution-06: serve_offline.py verified on a fresh VM, restarted while the internet was blocked (ready in 30 s, the foundry service could not do this). Offline timings with the final code, and the GPU/NPU limits we could not verify. The Windows Firewall rule was verified on a laptop with elevation, scoped to the service and Python, including the venv launcher pitfall. - solution-03 and the challenge pages point to the router and the new stretch. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Root cause, found with the network blocked for the process (administrator firewall rule): the SDK downloads the WebGPU execution provider again at every start, even though it is stored in ~/.foundry/ep. Offline the download fails, the provider is not registered and every GPU model refuses to load, while the registration call still reports success. The NPU provider is installed with Foundry Local and was never affected. The native library reads FOUNDRY_LOCAL_WEBGPU_EP_LIBRARY (undocumented), which points it at a provider on the disk. serve_offline.py --accelerate now sets it from ~/.foundry/ep. Verified with the network blocked: GPU variants ready in 12 s, CPU variants in 18 s, and the copilot answered correctly in both. Also fixes an inverted sort in serve_offline.py that would have preferred the GPU variants by default, and corrects the Challenge 6 coach notes. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The lab deliberately leaves out Azure Arc and Kubernetes, but customers who run Azure Local will ask how this relates to Agents and Tools with Foundry Local. Add a short, neutral pointer to that page, and say that it is a cluster platform in preview and not the on-device runtime used in the lab. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…lic IPs, keep the VM password on re-runs - shared-deploy-lab.ps1 registers Microsoft.Network/AllowBringYourOwnPublicIpAddress and re-registers Microsoft.Network, as other labs do. Without it the subscription refuses the VM's public IP and the platform stops the lab. The SecurityControl: Ignore tag does not help: it is a subscription feature, not a policy. - deploy-lab.ps1 uses New-MhhStablePassword -Purpose 'vm-admin'. A redeploy keeps the first password on an existing VM, so a new random password made the dashboard wrong. - Tested unchanged with the platform helper module from the adminpwsh image: feature registered in 1.4 min, lab provisioned in swedencentral in 6.3 min. Challenges 1 to 6 re-run on the VM with Foundry Local 0.12.1, including offline. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Marco Weber (qxsch)
approved these changes
Nov 2, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
New MicroHack in
03-Azure/01-04-AI/10_Offline_Industrial_Copilot: Offline Industrial Copilot.Participants build a maintenance technician copilot for a fictional manufacturer that runs entirely offline on a laptop or VM with Foundry Local: local model, local knowledge and local tools. At the end they pull the network cable and prove every step still works. Basic Python; no Kubernetes or Azure Arc needed.
Contents
ToolRouter: tool shortlisting with the embedding model already loaded, so a small model picks the right tool.serve_offline.py: starts Foundry Local from the SDK with the local model cache, so the copilot starts with no internet.labautomation/main.bicepanddeploy-lab.ps1for the Azure Machine Learning workspace and an optional Windows 11 VM per participant (for those without a suitable laptop). No devcontainer: Foundry Local has no Linux command-line tool, so the lab runs on Windows 11, macOS on Apple silicon or that VM.03-Azure/01-04-AI/Readme.md.Validation
ghcr.io/qxsch/adminpwsh):shared-deploy-lab.ps1registeredMicrosoft.Network/AllowBringYourOwnPublicIpAddress(needed where the subscription refuses public IPs; theSecurityControl: Ignoretag does not cover this), thendeploy-lab.ps1provisioned the lab in swedencentral on the first attempt in 6.3 min.New-MhhStablePassword, so it stays valid when the platform re-runs the script.labadminon the lab VM with Foundry Local 0.12.1: challenges 1 to 6 work, including the offline run; challenge 5 notebook on the workspace: accuracy 0.94, ONNX matches 100%.check_file_structure.py) passes locally; no broken links.Known limits, documented in the walkthroughs:
serve_offline.py --accelerate, which points Foundry Local at the WebGPU provider on the disk.vmSizecan be set inmain.bicep.