Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
bdd713c
feat(migrate): review generated migrations for data loss; verify --se…
cultron Oct 26, 2026
0dd4434
migrate verify: follow data moved into new tables, --check assertions…
cultron Oct 27, 2026
9164257
feat(migrate): apso migrate plan writes questions for ambiguous schem…
cultron Oct 27, 2026
e578eff
feat(ddl): Apso's own DDL writer for a whole .apsorc schema
cultron Oct 27, 2026
bbae7dd
fix(ddl): boolean defaults follow the fixed template (#161)
cultron Oct 28, 2026
39ac253
feat(ddl): render an answered migration plan as change DDL
cultron Oct 28, 2026
4152a0f
feat(migrate): apso migrate generate writes the migration from the an…
cultron Oct 28, 2026
f4c2fc2
feat(migrate): plan, generate and verify find the deployed schema on …
cultron Oct 28, 2026
b678fe7
feat(migrate): one-step apso migrate (generate, plan, ask, write, ver…
cultron Oct 28, 2026
04dd1b3
feat(migrate): JSON questions carry option labels, SQL hints and spli…
cultron Oct 28, 2026
a10423d
feat(migrate): Alembic and goose output for planned migrations; fix P…
cultron Oct 28, 2026
8d8a13a
docs: apso migrate is the guided flow; the old preview is apso migrat…
cultron Oct 28, 2026
8567f16
Merge pull request #163 from apsoai/feat/migration-plan-pygo
cultron Oct 28, 2026
5c59156
Merge remote-tracking branch 'origin/main' into feat/migration-plan
cultron Oct 28, 2026
bbfc9fa
test(generate): 30 s timeout for the stale-file test (two generates p…
cultron Oct 28, 2026
0e516b2
test: 30 s default timeout; PGlite suites are slow on CI runners
cultron Oct 28, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 5 additions & 3 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,9 +25,11 @@ apso generate # Generate code from .apsorc schema
apso dev # Start local dev server via Docker Compose

# Schema
apso migrate # Test schema migrations locally (PGlite sandbox)
apso migrate generate # Migration from a deployed .apsorc to this project's entities (TypeScript, PGlite)
apso migrate verify # Run a migration on the deployed schema, then check for drift
apso migrate # The guided flow: generate code, plan against the deployed schema, ask about anything ambiguous (or print questions as JSON and exit 2), write the migration, verify it
apso migrate sandbox # The older snapshot preview on an empty PGlite database (--sql prints DDL for an .apsorc)
apso migrate plan # Compare the deployed schema (found automatically) with .apsorc; ambiguous changes become questions in .apso/migration-plan.json; JSON when piped; exits 2 while any are open
apso migrate generate # Write the TypeORM migration from the answered plan: up() with renames, data moves and backfills, plus a reversing down(); --typeorm uses TypeORM's generator
apso migrate verify # Run the newest migration on a local PGlite copy of the schema it starts from (optionally --seed rows, --check assertions), then check data loss and drift
apso test smoke # Start the service; call /health, list, and create/read/delete each entity (TS on PGlite, Python/Go on DATABASE_URL)
apso schema validate # Validate local .apsorc (includes lint)
apso schema lint # Lint .apsorc with @apso/schema-tools (--fix, --json)
Expand Down
18 changes: 7 additions & 11 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -185,22 +185,18 @@ Requires Docker and Docker Compose. Looks for `docker-compose.yml` in the curren

### `apso migrate`

Detect schema changes, generate migration SQL, and test against a local PGlite sandbox. No Docker or external database required.
Write the migration for a schema change without losing data. `apso migrate` regenerates the code, compares `.apsorc` with the deployed schema, and asks about every change whose intent the schema can't show: was a missing field renamed or dropped, how does a field split, what replaces a removed enum value, what do existing rows get for a new required field. It then writes the migration from your answers (a TypeORM migration for TypeScript, an Alembic revision for Python, a goose file for Go), saves the answered plan beside it, and tests it on a local PGlite copy of the deployed schema.

```bash
apso migrate # Detect changes, generate and test SQL
apso migrate --apply # Update snapshot after successful test
apso migrate --reset # Clear sandbox and start fresh
apso migrate --sql # Output raw SQL only (for piping)
apso migrate --baseline deployed.apsorc # first time: the .apsorc that is deployed
apso migrate # later runs find the baseline from the last migration
apso migrate --answer field:Contact.phone=rename:mobilePhone
apso migrate --seed seed.sql --check checks.sql
```

| Option | Description | Default |
|--------|-------------|---------|
| `--apply` | Update local schema snapshot after verified migration | `false` |
| `--reset` | Reset the sandbox (clear snapshot and PGlite data) | `false` |
| `--sql` | Output raw SQL statements only | `false` |
In a terminal it asks each question. Without one (or with `--json`) it prints the open questions as JSON and exits 2, so a coding agent can answer them with `--answer` or in `.apso/migration-plan.json` and run it again. The same answers always produce the same SQL.

The sandbox works by comparing your current `.apsorc` against the last-known snapshot, generating the migration SQL, and executing it against an in-process Postgres instance (PGlite). If the migration fails locally, you know before it reaches any real database.
The steps are also separate commands: `apso migrate plan`, `apso migrate generate` and `apso migrate verify`. `apso migrate sandbox` is the older snapshot preview on an empty database (`--apply`, `--reset`, `--sql`).

### `apso deploy`

Expand Down
2 changes: 2 additions & 0 deletions jest.config.js
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@
module.exports = {
preset: 'ts-jest',
testEnvironment: 'node',
// PGlite-backed suites (migration render, verify, generate) take seconds per test on CI runners.
testTimeout: 30000,
testMatch: [
"**/src/**/*.spec.ts",
"**/src/**/*.test.ts",
Expand Down
2 changes: 1 addition & 1 deletion src/commands/deploy.ts
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@
}),
};

async run(): Promise<void> {

Check warning on line 56 in src/commands/deploy.ts

View workflow job for this annotation

GitHub Actions / build-and-publish (22)

Async method 'run' has a complexity of 22. Maximum allowed is 20

Check warning on line 56 in src/commands/deploy.ts

View workflow job for this annotation

GitHub Actions / build-and-publish (24)

Async method 'run' has a complexity of 22. Maximum allowed is 20
const { flags } = await this.parse(Deploy);

// Auth guard
Expand Down Expand Up @@ -95,7 +95,7 @@
this.log(`Error: ${result.error}`);
}
this.log("Fix the schema issue before deploying.");
this.log("Run 'apso migrate' for details.");
this.log("Run 'apso migrate sandbox' for details.");
this.exit(1);
}

Expand Down Expand Up @@ -193,7 +193,7 @@
}
}

/**

Check warning on line 196 in src/commands/deploy.ts

View workflow job for this annotation

GitHub Actions / build-and-publish (22)

Missing JSDoc for parameter 'useLocalGit'

Check warning on line 196 in src/commands/deploy.ts

View workflow job for this annotation

GitHub Actions / build-and-publish (22)

Missing JSDoc for parameter 'branch'

Check warning on line 196 in src/commands/deploy.ts

View workflow job for this annotation

GitHub Actions / build-and-publish (22)

Missing JSDoc for parameter 'repoFlag'

Check warning on line 196 in src/commands/deploy.ts

View workflow job for this annotation

GitHub Actions / build-and-publish (22)

Missing JSDoc for parameter 'service'

Check warning on line 196 in src/commands/deploy.ts

View workflow job for this annotation

GitHub Actions / build-and-publish (22)

Missing JSDoc @returns for function

Check warning on line 196 in src/commands/deploy.ts

View workflow job for this annotation

GitHub Actions / build-and-publish (24)

Missing JSDoc for parameter 'useLocalGit'

Check warning on line 196 in src/commands/deploy.ts

View workflow job for this annotation

GitHub Actions / build-and-publish (24)

Missing JSDoc for parameter 'branch'

Check warning on line 196 in src/commands/deploy.ts

View workflow job for this annotation

GitHub Actions / build-and-publish (24)

Missing JSDoc for parameter 'repoFlag'

Check warning on line 196 in src/commands/deploy.ts

View workflow job for this annotation

GitHub Actions / build-and-publish (24)

Missing JSDoc for parameter 'service'

Check warning on line 196 in src/commands/deploy.ts

View workflow job for this annotation

GitHub Actions / build-and-publish (24)

Missing JSDoc @returns for function
* Push the local scaffold to the service's GitHub repo with the user's own
* git, and make sure the repo is connected to the service so the provisioner
* can clone it at deploy time. No code passes through Apso — we read the local
Expand Down Expand Up @@ -310,7 +310,7 @@
}
}

/**

Check warning on line 313 in src/commands/deploy.ts

View workflow job for this annotation

GitHub Actions / build-and-publish (22)

Missing JSDoc for parameter 'connectionId'

Check warning on line 313 in src/commands/deploy.ts

View workflow job for this annotation

GitHub Actions / build-and-publish (22)

Missing JSDoc @returns for function

Check warning on line 313 in src/commands/deploy.ts

View workflow job for this annotation

GitHub Actions / build-and-publish (24)

Missing JSDoc for parameter 'connectionId'

Check warning on line 313 in src/commands/deploy.ts

View workflow job for this annotation

GitHub Actions / build-and-publish (24)

Missing JSDoc @returns for function
* Decide which GitHub repo to deploy to. No local git or `gh` needed to make
* one — we create it through the user's Apso GitHub connection. Headless (or
* when the account has no repos) auto-creates a repo named after the service;
Expand Down
283 changes: 176 additions & 107 deletions src/commands/migrate.ts
Original file line number Diff line number Diff line change
@@ -1,149 +1,218 @@
import * as fs from "fs";
import * as path from "path";
import { Flags } from "@oclif/core";
import * as inquirer from "inquirer";
import BaseCommand from "../lib/base-command";
import { parseApsorc } from "../lib/apsorc-parser";
import { readApsorcFile } from "../lib/apsorc-parser";
import { writePlannedMigration } from "../lib/ddl/migration-file";
import { openQuestions, Plan, planWithAnswers, Question } from "../lib/plan";
import { Baseline, baselineFile, resolveBaseline } from "../lib/plan/baseline";
import { describeQuestion, optionLabel, splitDefaults, sqlHint } from "../lib/plan/prompts";
import {
runMigrationSandbox,
applyMigration,
resetSandbox,
} from "../lib/migrate/sandbox";
import { createGenerator } from "../lib";
import { TargetLanguage, GeneratorConfig } from "../lib/types";
import { createFile } from "../lib/utils/file-system";
cliBinPath,
report,
run,
supportsPglite,
verifyMigration,
} from "../lib/verify";

const required = (v: string) => v.trim() !== "" || "Enter an expression";

export default class Migrate extends BaseCommand {
static description =
"Detect schema changes, generate migration SQL, and test against a local PGlite sandbox";
"Migrate the database to this .apsorc in one step: regenerate the code, compare with the deployed schema, ask about anything ambiguous (a rename or a drop, how a field splits, what replaces a removed value), write the migration, and test it on a local copy. In a terminal it asks; for an agent it prints the open questions as JSON and exits 2, and the agent reruns it with --answer. The steps are also separate commands: apso migrate plan, generate and verify.";

static examples = [
`$ apso migrate`,
`$ apso migrate --apply`,
`$ apso migrate --reset`,
`$ apso migrate --sql`,
"$ apso migrate",
"$ apso migrate --answer field:Contact.phone=rename:mobilePhone",
"$ apso migrate --seed seed.sql --check checks.sql",
];

static flags = {
apply: Flags.boolean({
baseline: Flags.string({
description:
"Apply the migration: update the local snapshot after a successful test",
"The deployed .apsorc, the first time only. Later runs find it from the last migration",
}),
answer: Flags.string({
description: "Answer a question: <question id>=<option>. Repeatable.",
multiple: true,
default: [],
}),
seed: Flags.string({
description: "SQL file of rows to load first; the test then fails if the migration loses any",
}),
check: Flags.string({
description: "SQL file of SELECTs with a boolean ok column, run after the migration",
}),
"accept-loss": Flags.string({
description: "A table or table.column whose seeded data may be dropped on purpose (repeatable)",
multiple: true,
default: [],
}),
name: Flags.string({ description: "Migration name", default: "SchemaUpdate" }),
"skip-generate": Flags.boolean({
description: "Don't regenerate code from .apsorc first",
default: false,
}),
reset: Flags.boolean({
description: "Reset the sandbox (clear snapshot and PGlite data)",
"skip-verify": Flags.boolean({
description: "Write the migration without testing it",
default: false,
}),
sql: Flags.boolean({
description: "Output raw SQL only (for piping to files)",
json: Flags.boolean({
description: "Print the result as JSON (the default when output is not a terminal)",
default: false,
}),
};

private json = false;

/** Progress for people; agents get only the final JSON. */
private say(s: string): void {
if (!this.json) this.log(s);
}

/** Print the outcome: one JSON object for agents, nothing extra for people. */
private out(status: string, extra: Record<string, unknown>): void {
if (this.json) this.log(JSON.stringify({ status, ...extra }, null, 2));
}

private fail(message: string, extra: Record<string, unknown> = {}): never {
if (this.json) {
this.out("failed", { error: message, ...extra });
this.exit(1);
}
this.error(message, { exit: 1 });
}

async run(): Promise<void> {
const { flags } = await this.parse(Migrate);
const cwd = process.cwd();
this.json = flags.json || !process.stdout.isTTY;
const interactive = !this.json && Boolean(process.stdin.isTTY);
// Keep stdout pure JSON for agents: parser notices go to stderr.
if (this.json) console.info = (...m: unknown[]) => process.stderr.write(m.join(" ") + "\n");

// Handle reset
if (flags.reset) {
resetSandbox();
this.log("Sandbox reset. Snapshot and PGlite data cleared.");
return;
const { apsorc } = readApsorcFile();
const cliBin = cliBinPath(this.config.root);

if (!flags["skip-generate"]) {
this.say("Generating code from .apsorc...");
// Agents run without a terminal, where generate needs the language spelled out.
const language = apsorc.language ?? "typescript";
const gen = await run(process.execPath, [cliBin, "generate", "--language", language], cwd);
if (!gen.ok) this.fail(`apso generate failed:\n${gen.output.trim().split("\n").slice(-15).join("\n")}`);
}

// Parse .apsorc
let parsed;
const planPath = path.resolve(".apso/migration-plan.json");
let baseline: Baseline;
try {
parsed = parseApsorc();
} catch (error: unknown) {
const msg = error instanceof Error ? error.message : String(error);
this.error(`Failed to parse .apsorc: ${msg}`);
baseline = resolveBaseline(cwd, apsorc, flags.baseline, planPath);
} catch (error) {
this.fail((error as Error).message);
}
this.say(`Comparing with ${baseline.source}.`);

if (!flags.sql) {
this.log("Schema changes detected. Running migration sandbox...");
const previous: Plan | undefined = fs.existsSync(planPath)
? JSON.parse(fs.readFileSync(planPath).toString())
: undefined;
let plan: Plan;
try {
plan = planWithAnswers(baseline.schema, apsorc, previous, flags.answer);
} catch (error) {
this.fail((error as Error).message);
}
const save = () => {
fs.mkdirSync(path.dirname(planPath), { recursive: true });
fs.writeFileSync(planPath, JSON.stringify({ ...plan, before: baseline.schema }, null, 2) + "\n");
};

// Run the sandbox
const result = await runMigrationSandbox(
{ entities: parsed.entities, relationshipMap: parsed.relationshipMap },
);

if (!result.needed) {
if (!flags.sql) {
this.log("No database-level changes detected.");
// Ask until nothing is open. Answers can raise new questions (dropping a
// field leaves a required one to backfill), so re-plan after each round.
for (let open = openQuestions(plan); open.length > 0; open = openQuestions(plan)) {
save();
if (!interactive) {
const next =
'Rerun apso migrate --answer <id>=<option> for each. For "sql" and "split:" answers, also set the question\'s "sql" in the plan file: a string, or one expression per split field.';
if (this.json) {
this.out("questions", { plan: path.relative(cwd, planPath), baseline: baseline.source, open: open.map((q) => describeQuestion(q)), next });
} else {
for (const q of open) this.log(`\n${q.id}\n ${q.subject}\n options: ${q.options.join(" | ")}`);
this.log(`\n${next}`);
}
this.exit(2);
}
return;
for (const q of open) {
// eslint-disable-next-line no-await-in-loop
await this.ask(q);
}
plan = planWithAnswers(baseline.schema, apsorc, plan, []);
}
save();

// Output results
if (flags.sql) {
// Raw SQL output mode
for (const sql of result.upSql) {
this.log(sql + ";");
}
const written = writePlannedMigration(cwd, baseline.schema, flags.name, planPath);
if (!written.ok) this.fail(written.steps.map((s) => s.output).filter(Boolean).join("\n"));
if (!written.needed) {
this.say("The database already matches .apsorc. Nothing to migrate.");
this.out("up-to-date", {});
return;
}
this.say(`Wrote ${written.file}`);
for (const f of written.findings ?? []) this.say(` warning ${f.code}: ${f.message}`);

if (result.success) {
this.log("");
this.log("Migration SQL (up):");
this.log("---");
for (const sql of result.upSql) {
this.log(` ${sql};`);
}
this.log("---");
this.log("");
this.log(`${result.upSql.length} statement(s) generated and tested.`);
this.log("Migration validated against local PGlite.");

// Generate language-native migration files for Python and Go
const language: TargetLanguage = parsed.language || "typescript";
if (language !== "typescript") {
const generatorConfig: GeneratorConfig = {
language,
rootFolder: parsed.rootFolder || "src",
apiType: parsed.apiType || "rest",
entities: parsed.entities,
relationshipMap: parsed.relationshipMap,
};

const generator = createGenerator(generatorConfig);
const migrationFiles = await generator.generateMigration({
entities: parsed.entities,
relationshipMap: parsed.relationshipMap,
upSql: result.upSql,
downSql: result.downSql,
});

for (const file of migrationFiles) {
const fullPath = path.join(process.cwd(), file.path);
// eslint-disable-next-line no-await-in-loop
await createFile(fullPath, file.content);
this.log(`Generated migration file: ${file.path}`);
}
}
let verified: boolean | "skipped" = "skipped";
// Python and Go verify without the project's own database driver.
const canVerify = (apsorc.language ?? "typescript") !== "typescript" || supportsPglite(cwd);
if (!flags["skip-verify"] && canVerify) {
this.say("Testing the migration on a local copy of the deployed schema...");
const result = await verifyMigration(cwd, baselineFile(baseline), written.file!, cliBin, {
seedFile: flags.seed,
checkFile: flags.check,
acceptLoss: flags["accept-loss"],
});
verified = result.ok;
if (!this.json) report((s) => this.log(s), { ...result, findings: [] }, false);
if (!result.ok)
this.fail(
`The migration in ${written.file} failed its test. Fix the answers in ${written.planFile} or the .apsorc, then run apso migrate again.`,
{ file: written.file, planFile: written.planFile, steps: result.steps }
);
}

if (flags.apply) {
applyMigration({
entities: parsed.entities,
relationshipMap: parsed.relationshipMap,
});
this.log("Snapshot updated.");
} else {
this.log("Run with --apply to update the snapshot.");
}
} else {
this.log("");
this.log("Migration FAILED.");
if (result.error) {
this.log(`Error: ${result.error}`);
}
if (result.upSql.length > 0) {
this.log("");
this.log("Attempted SQL:");
for (const sql of result.upSql) {
this.log(` ${sql};`);
}
this.say(`\nReview and commit ${written.file} and ${written.planFile}, then deploy.`);
if (verified === true && !flags.seed)
this.say("Tip: --seed <rows.sql> also checks that existing rows survive the migration.");
this.out("written", { file: written.file, planFile: written.planFile, verified });
}

private async ask(q: Question): Promise<void> {
const { choice } = await inquirer.prompt<{ choice: string }>([
{
type: "list",
name: "choice",
message: q.subject,
choices: q.options.map((o) => ({ name: optionLabel(q, o), value: o })),
default: q.suggested,
},
]);
q.answer = choice;
if (choice === "sql") {
const { sql } = await inquirer.prompt<{ sql: string }>([
{ type: "input", name: "sql", message: sqlHint(q), validate: required },
]);
q.sql = sql;
}
if (choice.startsWith("split:")) {
const targets = choice.slice("split:".length).split(",");
const defaults = splitDefaults(q, targets);
const sql: Record<string, string> = {};
for (const t of targets) {
// eslint-disable-next-line no-await-in-loop
const { expr } = await inquirer.prompt<{ expr: string }>([
{ type: "input", name: "expr", message: `SQL for ${t}`, default: defaults[t], validate: required },
]);
sql[t] = expr;
}
this.exit(1);
q.sql = sql;
}
}
}
Loading
Loading