macos-foundation-models

Node bindings for Apple’s Foundation Models, with AI SDK support and an OpenAI-compatible server

Install

pnpm add macos-foundation-models

Requires Apple-silicon Mac on macOS 26.0+ with Apple Intelligence and its model assets installed.

API

import { createSession, getAvailability } from 'macos-foundation-models'

const availability = getAvailability()
if (availability.state !== 'available') {
  console.error(`Cannot use Foundation Models: ${availability.reason}`)
  process.exit(1)
}

// `using` releases the native session when this scope ends
using session = createSession({ instructions: 'Be direct and factual.' })
const response = await session.respond({
  prompt: 'Summarize why local models can be useful.',
})

console.log(response.content)

Streaming and cancellation

const controller = new AbortController()
const stream = session.stream({
  prompt: 'Give three reading-focus tips.',
  signal: controller.signal,
})
for await (const snapshot of stream) {
  replaceDisplayedText(snapshot.content) // cumulative snapshots, never token deltas
  if (shouldStop()) controller.abort()
}
const finalResponse = await stream.completed

Image attachments (macOS 27 preview)

Image attachments are lazy prompt descriptors, not native resources the caller has to manage:

import { imageAttachment } from 'macos-foundation-models'

const file = imageAttachment.file('/Users/me/Pictures/cat.jpg')
const inMemory = imageAttachment.bytes(screenshotBytes) // Uint8Array; Node Buffer also works

const response = await session.respond({
  prompt: ['Compare these images:', file, inMemory],
})

Tools, adapters, feedback, and refusals

import { writeFile } from 'node:fs/promises'
import { createSession, defineTool, FoundationModelsError } from 'macos-foundation-models'

const weather = defineTool({
  name: 'weather',
  description: 'Gets the current weather for one city.',
  schema: {
    type: 'object',
    properties: [{ name: 'city', schema: { type: 'string' } }],
  },
  async execute({ city }, { signal }) {
    if (signal.aborted) throw new Error('Weather lookup was cancelled.')

    // Call only a service this host is authorized to use.
    return { city, forecast: 'sunny' }
  },
})

using session = createSession({ tools: [weather] })
try {
  const response = await session.respond({
    prompt: 'What is the weather in Auckland? Use the weather tool.',
  })
  const feedback = session.createFeedbackAttachment({ sentiment: 'positive' })
  await writeFile('foundation-model-feedback.bin', feedback)
  console.log(response.content)
} catch (error) {
  if (error instanceof FoundationModelsError && error.code === 'refusal') {
    console.log((await session.getRefusalExplanation()).content)
  } else {
    throw error
  }
}

AI SDK

Use the /ai-sdk subpath when the application already uses Vercel AI SDK. It is an in-process provider: it does not start an HTTP server or require an API key.

import { generateText, streamText } from 'ai'
import { appleFoundationModels } from 'macos-foundation-models/ai-sdk'

const model = appleFoundationModels()

const result = await generateText({
  model,
  prompt: 'Explain local inference in one sentence.',
})

const stream = streamText({ model, prompt: 'Give three focus tips.' })
for await (const delta of stream.textStream) process.stdout.write(delta)

Compatibility and limitations

AI SDK feature	Behavior
stopSequences with plain text	Applied locally; streams cancel once a sequence is observed.
stopSequences with JSON output or function-tool selection	Ignored with an AI SDK warning, preserving valid JSON and tool arguments.
presencePenalty / frequencyPenalty	Unsupported by Foundation Models; an AI SDK warning is returned.
topP and topK	Both map natively, but topK takes precedence when both are supplied; a warning explains this.
Tool history	Passed to the next model step as compatible text, not native Foundation Models tool state.
AI SDK dynamic tools	Supported; AI SDK normalizes them to function tools before calling the adapter.
Provider-defined or provider-executed tools	Not available: Foundation Models exposes no Apple-provided AI SDK tool identities or services.
Usage accounting	Token counts are unavailable from Foundation Models, so AI SDK usage fields are undefined.

OpenAI-compatible local server

Use the /openai subpath when an existing application expects OpenAI’s language-model HTTP API. The server is local only by default: it listens on 127.0.0.1, does not send prompts to a remote service, and does not need an API key. It is built with Hono.

For the common case, starting it is one call—there is no separate app construction or listen() step:

import OpenAI from 'openai'
import { serveOpenAICompat } from 'macos-foundation-models/openai'

const server = await serveOpenAICompat({ port: 4141 })
const client = new OpenAI({ apiKey: 'not-needed', baseURL: server.baseURL })

try {
  const completion = await client.chat.completions.create({
    model: 'apple-foundation-models',
    messages: [{ role: 'user', content: 'Give one concise local-inference benefit.' }],
  })
  console.log(completion.choices[0]?.message.content)
} finally {
  await server.close()
}

For a long-running process, the package executable provides the same server without writing any Node code:

macos-foundation-models openai serve --port 4141
# Point an OpenAI client at http://127.0.0.1:4141/v1

Pass --api-key "$LOCAL_OPENAI_KEY" (or { apiKey }) to require a bearer token. Binding to any non-loopback host requires an API key. --cors https://app.example.com enables CORS for one browser origin; CORS is otherwise disabled. The lower-level createOpenAIApp(options) is available only when an application needs to mount app.fetch in its own Node server.

Implemented endpoints

Endpoint	Support
GET /v1/models, GET /v1/models/:id	Advertises the single apple-foundation-models model
POST /v1/chat/completions	Text, JSON Schema output, json_object, function calls, stop sequences, and SSE
POST /v1/responses	Text and JSON Schema output, function calls, previous_response_id, local store, and Responses SSE events
GET / DELETE /v1/responses/:id	Bounded, TTL-backed local response retrieval and deletion
POST /v1/completions	Single-prompt legacy compatibility, including SSE
GET /health	Local availability diagnostic; not an OpenAI endpoint

Function tools are returned in the usual OpenAI response shape. As with OpenAI’s own API, the client is responsible for executing the function and submitting its result in the next Chat Completions or Responses request. A JSON Schema final answer and function-call decision share one native constrained schema, so tools and structured output can be used together. Parallel function calls are supported (up to eight per model decision); set parallel_tool_calls: false to allow one.

Responses are kept only in the server process (100 records for 30 minutes by default), so previous_response_id, retrieval, and deletion do not survive a restart. Configure responseStoreSize and responseTtlMs when constructing the app or one-call server.

Packaging Vite and Electron applications

Tip: For Vite or Electron packaging, use [vite-plugin-native-modules](https://github.com/biw/vite-plugin-native-modules)

import { defineConfig } from 'vite'
import nativeModules from 'vite-plugin-native-modules'

export default defineConfig({
  plugins: [nativeModules()],
})

License

MIT