Skip to content

Improve migration copy in CLI - #109

Merged
jumski merged 12 commits into
mainfrom
improve-cli-migration-copy
May 8, 2025
Merged

jumski merged 12 commits into
mainfrom
improve-cli-migration-copy

Conversation

@jumski

@jumski jumski commented May 7, 2025 •

Copy link
Copy Markdown
Contributor

Summary by CodeRabbit

  • New Features
    • Migration copy now generates new timestamp-based IDs and prefixes filenames to maintain order.
    • Installer prevents duplicate migrations on repeated runs, ensuring clean migration history.
  • Bug Fixes
    • Improved detection and prevention of duplicate migration files, even with altered filenames.
  • Tests
    • Added automated scripts to validate migration installation and duplication prevention.
  • Documentation
    • Updated installer docs to explain timestamped migration filenames and duplication prevention.

jumski added 4 commits May 7, 2025 21:41
- Added helper functions to extract and generate incremental timestamps for migrations
- Updated migration copying to assign new timestamps while preserving order
- Modified existing migration detection to prevent duplicates based on filename content
- Enhanced logging to display styled filenames and migration status more clearly
- Removed redundant code and streamlined the migration installation process
…ntion

Update project.json to include new test:e2e:install:duplicates target.
Add test-install and test-install-duplicates scripts for automated testing of
installation process and migration duplication prevention, including optional
timestamp-based duplicate detection.
These scripts facilitate end-to-end testing
of the CLI's install functionality and ensure no duplicate migrations are created.
Enhance user feedback when copying migrations by displaying detailed information
about already installed migrations, including their installation method and timestamps.
…mpatibility

Enhance migration copying by always generating a new timestamp-based ID, prepending it to the
original filename, and improving source migration matching to ensure backward compatibility
with installed migrations.
@changeset-bot

changeset-bot Bot commented May 7, 2025 •

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: ba8fe1f

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 6 packages
Name Type
pgflow Patch
@pgflow/core Patch
@pgflow/dsl Patch
@pgflow/edge-worker Patch
@pgflow/example-flows Patch
@pgflow/website Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@netlify

netlify Bot commented May 7, 2025 •

Copy link
Copy Markdown

✅ Deploy Preview for pgflow-demo canceled.

Name Link
🔨 Latest commit ba8fe1f
🔍 Latest deploy log https://app.netlify.com/sites/pgflow-demo/deploys/681c3a2b4ea2e700098400bb

@coderabbitai

coderabbitai Bot commented May 7, 2025 •

Copy link
Copy Markdown
Contributor

Walkthrough

This update enhances the migration copy functionality in the CLI tool by introducing timestamp-based prefixes to migration filenames when copying them. The copy logic now ensures new migrations are uniquely identified by incrementally increasing timestamps. New test scripts and project configuration changes are included to automate and verify the prevention of duplicate migrations during installation.

Changes

File(s) Change Summary
pkgs/cli/src/commands/install/copy-migrations.ts Migration copy logic updated to generate new timestamp-based filenames for copied migrations, track installed migrations by original filenames, and prompt users with a detailed summary before copying. Helper functions for timestamp extraction and generation were added.
pkgs/cli/scripts/test-install
pkgs/cli/scripts/test-install-duplicates
New executable Bash scripts added to automate testing of CLI installation and to specifically verify that duplicate migration files are not created during repeated installations. The duplicate test script also simulates user modifications by renaming migration files and ensures no duplicates are created.
pkgs/cli/project.json The "test" target was updated to depend on a new "test:e2e:install:duplicates" target. The "test:e2e:install" target was refactored to use two scripts (test-install and test-install-duplicates). The "test" target now disables parallel execution.
.changeset/short-zebras-wish.md Changeset added describing the migration copy improvements and the logic for handling migration filenames and backward compatibility.
pkgs/website/src/content/docs/getting-started/install-pgflow.mdx Documentation updated to clarify migration copying behavior, emphasizing timestamp prefixes to maintain execution order and prevent duplicate migrations on repeated installs or upgrades. A note with an example was added to explain the timestamp prefix usage.

Sequence Diagram(s)

sequenceDiagram
    participant User
    participant CLI
    participant FS as File System

    User->>CLI: Run migration install/copy command
    CLI->>FS: Read existing migrations in target directory
    CLI->>FS: Read source migrations
    CLI->>CLI: Identify new migrations (by original filename)
    CLI->>CLI: Generate new timestamped filenames for new migrations
    CLI->>User: Prompt summary and request confirmation
    User-->>CLI: Confirm
    CLI->>FS: Copy new migrations with new timestamped names
    CLI->>User: Show success message with installed migrations
Loading

Poem

🕰️
A hop and a skip, the migrations now grow,
With timestamps that tick as the versions all flow.
No duplicates found, just order and cheer,
The scripts check it twice—your schema is clear!
A bunny with code and a clock in its paw,
Keeps migrations in line, without a flaw.
🐇✨

Tip

⚡️ Faster reviews with caching
  • CodeRabbit now supports caching for code and dependencies, helping speed up reviews. This means quicker feedback, reduced wait times, and a smoother review experience overall. Cached data is encrypted and stored securely. This feature will be automatically enabled for all accounts on May 16th. To opt out, configure Review - Disable Cache at either the organization or repository level. If you prefer to disable all data retention across your organization, simply turn off the Data Retention setting under your Organization Settings.

Enjoy the performance boost—your workflow just got faster.

✨ Finishing Touches
  • 📝 Generate Docstrings

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Support

Need help? Create a ticket on our support page for assistance with any issues or questions.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai generate docstrings to generate docstrings for this PR.
  • @coderabbitai generate sequence diagram to generate a sequence diagram of the changes in this PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented May 7, 2025 •

Copy link
Copy Markdown

Deploying pgflow with  Cloudflare Pages  Cloudflare Pages

Latest commit: ba8fe1f
Status: ✅  Deploy successful!
Preview URL: https://e33de094.pgflow.pages.dev
Branch Preview URL: https://improve-cli-migration-copy.pgflow.pages.dev

View logs

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🧹 Nitpick comments (2)
.changeset/short-zebras-wish.md (1)

11-12: Minor grammatical improvements needed

There are a couple of small grammatical issues in this changeset:

  1. Remove the comma before "because" since it introduces an essential clause.
  2. Add "the" before "destination folder" for clarity.
-This change should be backwards compatible with already installed migrations,
-because the matching logic will just match the source filename in destination folder.
+This change should be backwards compatible with already installed migrations
+because the matching logic will just match the source filename in the destination folder.
🧰 Tools
🪛 LanguageTool

[formatting] ~11-~11: If the ‘because’ clause is essential to the meaning, do not use a comma before the clause.
Context: ...atible with already installed migrations, because the matching logic will just match the ...

(COMMA_BEFORE_BECAUSE)


[uncategorized] ~12-~12: Possible missing article found.
Context: ... will just match the source filename in destination folder.

(AI_HYDRA_LEO_MISSING_THE)

pkgs/cli/src/commands/install/copy-migrations.ts (1)

96-103: Ensure timestamp doesn't exceed current time

Good implementation to cap the generated timestamp at the current time. However, the timestamp string generation could be simplified.

You could extract the timestamp string generation to a helper function since it's used in two places (lines 84-90 and 98-99):

+// Helper function to generate timestamp string from a Date object
+function formatDateAsTimestamp(date: Date): string {
+  return `${date.getFullYear()}${String(date.getMonth() + 1).padStart(2, '0')}${String(date.getDate()).padStart(2, '0')}${String(date.getHours()).padStart(2, '0')}${String(date.getMinutes()).padStart(2, '0')}${String(date.getSeconds()).padStart(2, '0')}`;
+}

 function generateNewTimestamp(referenceTimestamp: string, increment: number = 10): string {
   // ... existing code ...
   
   // Use current time if invalid timestamp
   if (!referenceTimestamp || referenceTimestamp.length !== 14) {
     const now = new Date();
-    const year = now.getFullYear();
-    const month = String(now.getMonth() + 1).padStart(2, '0');
-    const day = String(now.getDate()).padStart(2, '0');
-    const hours = String(now.getHours()).padStart(2, '0');
-    const minutes = String(now.getMinutes()).padStart(2, '0');
-    const seconds = String(now.getSeconds()).padStart(2, '0');
-    return `${year}${month}${day}${hours}${minutes}${seconds}`;
+    return formatDateAsTimestamp(now);
   }

   // ... existing code ...
   
   // Ensure we don't exceed current time
   const now = new Date();
-  const currentTimestampStr = 
-    `${now.getFullYear()}${String(now.getMonth() + 1).padStart(2, '0')}${String(now.getDate()).padStart(2, '0')}${String(now.getHours()).padStart(2, '0')}${String(now.getMinutes()).padStart(2, '0')}${String(now.getSeconds()).padStart(2, '0')}`;
+  const currentTimestampStr = formatDateAsTimestamp(now);

This would improve maintainability and reduce the chance of inconsistencies.

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 5c39600 and abe02ec.

📒 Files selected for processing (5)
  • .changeset/short-zebras-wish.md (1 hunks)
  • pkgs/cli/project.json (1 hunks)
  • pkgs/cli/scripts/test-install (1 hunks)
  • pkgs/cli/scripts/test-install-duplicates (1 hunks)
  • pkgs/cli/src/commands/install/copy-migrations.ts (3 hunks)
🧰 Additional context used
🪛 LanguageTool
.changeset/short-zebras-wish.md

[formatting] ~11-~11: If the ‘because’ clause is essential to the meaning, do not use a comma before the clause.
Context: ...atible with already installed migrations, because the matching logic will just match the ...

(COMMA_BEFORE_BECAUSE)


[uncategorized] ~12-~12: Possible missing article found.
Context: ... will just match the source filename in destination folder.

(AI_HYDRA_LEO_MISSING_THE)

⏰ Context from checks skipped due to timeout of 90000ms (2)
  • GitHub Check: main
  • GitHub Check: Cloudflare Pages
🔇 Additional comments (9)
pkgs/cli/scripts/test-install-duplicates (1)

1-66: Clean and thorough test implementation

This script thoroughly tests the duplicate prevention functionality, including the basic case and an edge case with manually modified timestamps. The error handling, logging, and verification are well-implemented.

pkgs/cli/scripts/test-install (1)

1-35: Well-structured installation test script

This test script effectively validates the installation process with good error handling and clear logging. The optional duplicate testing functionality provides a convenient way to test that aspect as well.

pkgs/cli/project.json (1)

34-57: Good integration of new tests into the build pipeline

The changes properly integrate the new test scripts into the build system. Adding the duplicate prevention test as a dependency of the main test target ensures it will be run as part of the regular test suite, which is a good practice.

pkgs/cli/src/commands/install/copy-migrations.ts (6)

71-75: Well-structured helper function for timestamp extraction

The getTimestampFromFilename function uses a clear regex pattern to extract timestamp prefixes from migration filenames. It properly handles both cases - filenames with timestamps and those without.


137-149: Good implementation of finding latest timestamp

The code correctly identifies the latest timestamp from existing migration files. The approach of iterating through all files and comparing timestamps is appropriate for this use case.


180-202: Great user experience with detailed migration status

The enhanced output showing already installed migrations with their original and timestamped filenames significantly improves user experience. The use of chalk for styling adds visual clarity to help users understand the state of their migrations.


207-214: Timestamp sequence generation ensures proper ordering

The implementation correctly generates incremental timestamps for new migrations, maintaining proper ordering. Using a fixed increment of 10 provides room for potential future manual insertions between migrations if needed.


221-228: Clear visualization of new migrations

The formatted output clearly shows what will be installed and how the filenames will be transformed, which helps users understand the changes that will be made.


264-273: Excellent user feedback on successful installation

The detailed success message clearly shows what was installed with appropriate styling. This enhances user experience by providing confirmation and clarity about the changes made.

Comment on lines 257 to 262
for (const file of filesToCopy) {
const source = path.join(sourcePath, file);
const destination = path.join(migrationsPath, file);
const sourcePath1 = path.join(sourcePath, file.source);
const destinationPath = path.join(migrationsPath, file.destination);

fs.copyFileSync(source, destination);
fs.copyFileSync(sourcePath1, destinationPath);
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Consider handling potential copy errors

The file copying implementation works, but lacks error handling which could be important when dealing with file system operations.

Add error handling to the file copy operations:

 for (const file of filesToCopy) {
   const sourcePath1 = path.join(sourcePath, file.source);
   const destinationPath = path.join(migrationsPath, file.destination);

-  fs.copyFileSync(sourcePath1, destinationPath);
+  try {
+    fs.copyFileSync(sourcePath1, destinationPath);
+  } catch (error) {
+    log.error(`Failed to copy ${file.source}: ${error.message}`);
+    throw new Error(`Migration installation failed: Could not copy ${file.source}`);
+  }
 }

This would provide clearer feedback if a file copy operation fails due to permissions, disk space, or other issues.

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
for (const file of filesToCopy) {
const source = path.join(sourcePath, file);
const destination = path.join(migrationsPath, file);
const sourcePath1 = path.join(sourcePath, file.source);
const destinationPath = path.join(migrationsPath, file.destination);
fs.copyFileSync(source, destination);
fs.copyFileSync(sourcePath1, destinationPath);
}
for (const file of filesToCopy) {
const sourcePath1 = path.join(sourcePath, file.source);
const destinationPath = path.join(migrationsPath, file.destination);
try {
fs.copyFileSync(sourcePath1, destinationPath);
} catch (error) {
log.error(`Failed to copy ${file.source}: ${error.message}`);
throw new Error(`Migration installation failed: Could not copy ${file.source}`);
}
}

Comment on lines +158 to 173
for (const sourceFile of sourceFiles) {
// Check if this migration is already installed (by checking if the original filename
// appears in any existing migration filename)
const isAlreadyInstalled = existingFiles.some(existingFile =>
existingFile.includes(sourceFile)
);

if (fs.existsSync(destination)) {
skippedFiles.push(file);
if (isAlreadyInstalled) {
skippedFiles.push(sourceFile);
} else {
filesToCopy.push(file);
filesToCopy.push({
source: sourceFile,
destination: sourceFile // Will be updated later with new timestamp
});
}
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Verification agent

🧩 Analysis chain

Novel approach for detecting already installed migrations

The implementation for checking if a migration is already installed by looking for its filename within any existing migration is clever and provides backward compatibility with both timestamped and non-timestamped migrations.

However, there's a subtle edge case to consider. If you have two migrations like "create_table.sql" and "recreate_table.sql", the current approach would match "recreate_table.sql" when checking for "create_table.sql" because it's a substring match.

Verify if this edge case could occur in your migrations:


🏁 Script executed:

#!/bin/bash
# Look for potential partial-name collisions in existing migrations
sourcePath=$(find . -type d -path "*/migrations" | head -1)
if [ -z "$sourcePath" ]; then
  echo "No migrations directory found"
  exit 0
fi

echo "Analyzing migrations in $sourcePath"
sourceFiles=$(find "$sourcePath" -name "*.sql" -type f -exec basename {} \;)

# Remove timestamps from filenames for comparison
for file in $sourceFiles; do
  # Strip timestamp prefix if present
  baseFile=$(echo "$file" | sed -E 's/^[0-9]+_//g')
  
  # Check if this filename is contained in other filenames
  for otherFile in $sourceFiles; do
    otherBaseFile=$(echo "$otherFile" | sed -E 's/^[0-9]+_//g')
    if [ "$baseFile" != "$otherBaseFile" ] && [[ "$otherBaseFile" == *"$baseFile"* ]]; then
      echo "Potential collision: '$baseFile' is contained in '$otherBaseFile'"
    fi
  done
done

Length of output: 11128


No substring collisions found in current migrations, but matching logic could be hardened

We ran the provided script against all .sql files in examples/playground/supabase/migrations and saw no “create_table” vs. “recreate_table”–style collisions. The existing .includes(sourceFile) check works today, but it could yield false positives if one migration’s basename is a strict substring of another.

• File: pkgs/cli/src/commands/install/copy-migrations.ts (lines 158–173)
• Current check:

const isAlreadyInstalled = existingFiles.some(existingFile =>
  existingFile.includes(sourceFile)
);

• Suggested improvement:
– Strip timestamps and compare the full basename for exact equality
– Or match with delimiters (e.g. /(^|_)${basename}(_|$)/) instead of simple substring

[fine-tuning the match will prevent potential edge-case collisions]

@nx-cloud

nx-cloud Bot commented May 7, 2025 •

Copy link
Copy Markdown

View your CI Pipeline Execution ↗ for commit ba8fe1f.

Command Status Duration Result
nx affected -t lint typecheck test build ✅ Succeeded 2m 23s View ↗

☁️ Nx Cloud last updated this comment at 2025-05-08 05:02:26 UTC

jumski added 2 commits May 7, 2025 22:14
…uring pgflow setup

- Updated documentation to clarify how migration files are copied with timestamps
- Added notes on preventing duplicate migrations when rerunning the installer
- Enhanced explanation of migration timestamp prefixes and their purpose
- Minor formatting adjustments for clarity in the installation guide
- Simplify function parameter default assignment
- Enhance timestamp parsing robustness for migration file handling

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

♻️ Duplicate comments (3)
pkgs/cli/src/commands/install/copy-migrations.ts (3)

161-162: Consider improving the migration detection logic

The current approach using includes() could potentially have false positives if one migration's basename is a substring of another (e.g., "create_table.sql" vs "recreate_table.sql").

-const isAlreadyInstalled = existingFiles.some(existingFile => 
-  existingFile.includes(sourceFile)
-);
+const isAlreadyInstalled = existingFiles.some(existingFile => {
+  // Extract the base name (removing timestamp prefix if present)
+  const existingBaseName = existingFile.replace(/^\d+_/, '');
+  // Check if it's an exact match to the source file
+  return existingBaseName === sourceFile || existingFile === sourceFile;
+});

81-81: 🛠️ Refactor suggestion

Add validation for the timestamp format

The function doesn't validate if referenceTimestamp contains only digits when it's exactly 14 characters long. This could lead to NaN issues during conversion.

-if (!referenceTimestamp || referenceTimestamp.length !== 14) {
+if (!referenceTimestamp || referenceTimestamp.length !== 14 || !/^\d{14}$/.test(referenceTimestamp)) {

261-261: 🛠️ Refactor suggestion

Add error handling for file copying operations

File system operations can fail due to various reasons (permissions, disk space, etc.), but there's no error handling for the copy operation.

-fs.copyFileSync(sourcePath1, destinationPath);
+try {
+  fs.copyFileSync(sourcePath1, destinationPath);
+} catch (error) {
+  log.error(`Failed to copy ${file.source}: ${error.message}`);
+  throw new Error(`Migration installation failed: Could not copy ${file.source}`);
+}
🧹 Nitpick comments (1)
pkgs/website/src/content/docs/getting-started/install-pgflow.mdx (1)

40-40: Fix grammar issue in bullet point

There's a small grammar error in this line. The bullet point is missing the preposition "to" before the verb "Prevent".

-- Prevent duplicate migrations if you run the installer multiple times
+- To prevent duplicate migrations if you run the installer multiple times
🧰 Tools
🪛 LanguageTool

[grammar] ~40-~40: Did you mean “to Prevent”?
Context: ... functions environment file if needed - Prevent duplicate migrations if you run the ins...

(MISSING_TO_BEFORE_A_VERB)

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between abe02ec and 5a256c6.

📒 Files selected for processing (2)
  • pkgs/cli/src/commands/install/copy-migrations.ts (3 hunks)
  • pkgs/website/src/content/docs/getting-started/install-pgflow.mdx (2 hunks)
🧰 Additional context used
🪛 LanguageTool
pkgs/website/src/content/docs/getting-started/install-pgflow.mdx

[grammar] ~40-~40: Did you mean “to Prevent”?
Context: ... functions environment file if needed - Prevent duplicate migrations if you run the ins...

(MISSING_TO_BEFORE_A_VERB)

⏰ Context from checks skipped due to timeout of 90000ms (2)
  • GitHub Check: main
  • GitHub Check: Cloudflare Pages
🔇 Additional comments (4)
pkgs/website/src/content/docs/getting-started/install-pgflow.mdx (1)

69-76: Great documentation of the migration timestamp system!

This explanation clearly describes how the timestamp system works and provides a helpful concrete example. This will make it much easier for users to understand why migration files appear with timestamp prefixes and why this approach prevents duplicates.

pkgs/cli/src/commands/install/copy-migrations.ts (3)

77-104: Well-designed timestamp generation with good safeguards

The timestamp generation function is well-implemented with multiple safeguards:

  1. Falls back to current time for invalid timestamps
  2. Ensures generated timestamps don't exceed current time
  3. Maintains consistent 14-digit format with padding

This robust approach will prevent ordering issues and ensure migrations are properly sequenced.


207-214: Good implementation of incremental timestamps

The implementation uses a base timestamp and increments it by 10 for each migration, which provides room for potential intermediate migrations while maintaining order. The new filename format clearly preserves the original filename while adding the timestamp prefix.


264-273: Clear and informative success messaging

The detailed success message provides good visibility into what migrations were installed and how they were renamed. The use of chalk for styling makes the output more readable and helps users distinguish between the timestamp prefix and the original filename.

jumski added 2 commits May 7, 2025 22:50
- Added --force flag to supabase init commands in test-install and test-install-duplicates
scripts to ensure clean initialization
- Minor formatting adjustments for consistency in test scripts
…cript

Refactors test task dependencies to include compile step and updates test-install commands to
run both install and install-duplicates scripts.
@jumski

jumski commented May 8, 2025

Copy link
Copy Markdown
Contributor Author

Solves #108

jumski added 4 commits May 8, 2025 06:35
Enhance the timestamp validation to ensure it matches a 14-digit numeric pattern, preventing
invalid timestamps from being processed.
This improves robustness when generating new timestamps based on filenames.
- Adjusted generateNewTimestamp to ensure new timestamps are always at least one greater than
the reference
- Corrected existing files reading to handle undefined or missing files safely
- Enhanced migration filename handling to accurately identify already installed migrations
- Updated progress and summary messages for clarity and consistency
- Minor formatting and code style improvements for better maintainability
…ation scripts

- Added helper functions to format Date objects into timestamp strings
- Implemented robust parsing of timestamp strings into Date objects with validation
- Updated generateNewTimestamp to produce incremented timestamps that are always forward-moving
- Improved handling of invalid or unparseable timestamps to default to current time
- Overall, these changes improve reliability and correctness of migration timestamp management
Refactors getTimestampFromFilename to ensure correct length and format validation.
Enhances timestamp comparison logic to consider only validated timestamps when updating
latestTimestamp.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

♻️ Duplicate comments (2)
pkgs/cli/src/commands/install/copy-migrations.ts (2)

217-232: Existing migration detection logic works but has edge cases

The implementation for checking if a migration is already installed looks for the source filename as a substring of any existing migration filename. This approach provides backward compatibility but could have false positives if one migration name is a substring of another.

For example, if you have "create_users.sql" and "create_users_roles.sql", the current approach would consider "create_users.sql" already installed if "create_users_roles.sql" exists.

#!/bin/bash
# Look for potential substring collisions in migration filenames
echo "Analyzing for potential substring collisions..."

# Get all SQL files in the migrations directory
sourcePath=$(find . -path "*/migrations" -type d | head -1)
if [ -z "$sourcePath" ]; then
  echo "No migrations directory found"
  exit 0
fi

echo "Found migrations directory: $sourcePath"
sourceFiles=$(find "$sourcePath" -name "*.sql" -type f -exec basename {} \; | sort)

echo "Found migration files:"
echo "$sourceFiles"
echo ""

echo "Checking for potential collisions..."
hasCollision=false

for file1 in $sourceFiles; do
  # Remove timestamp prefix if present
  baseFile1=$(echo "$file1" | sed -E 's/^[0-9]+_//g')
  
  for file2 in $sourceFiles; do
    # Skip comparing to itself
    if [ "$file1" = "$file2" ]; then
      continue
    fi
    
    # Remove timestamp prefix if present
    baseFile2=$(echo "$file2" | sed -E 's/^[0-9]+_//g')
    
    # Skip if files are the same after timestamp removal
    if [ "$baseFile1" = "$baseFile2" ]; then
      continue
    fi
    
    # Check if baseFile1 is a substring of baseFile2
    if [[ "$baseFile2" == *"$baseFile1"* ]]; then
      echo "Potential collision detected:"
      echo "  '$baseFile1' is a substring of '$baseFile2'"
      echo "  This could cause '$baseFile1' to be incorrectly detected as already installed"
      hasCollision=true
    fi
  done
done

if [ "$hasCollision" = false ]; then
  echo "No potential collisions found."
fi

332-337: Consider handling potential copy errors

The file copying implementation works, but lacks error handling which could be important when dealing with file system operations.

Add error handling to the file copy operations:

 for (const file of filesToCopy) {
   const sourcePath1 = path.join(sourcePath, file.source);
   const destinationPath = path.join(migrationsPath, file.destination);

-  fs.copyFileSync(sourcePath1, destinationPath);
+  try {
+    fs.copyFileSync(sourcePath1, destinationPath);
+  } catch (error) {
+    log.error(`Failed to copy ${file.source}: ${error.message}`);
+    throw new Error(`Migration installation failed: Could not copy ${file.source}`);
+  }
 }
🧹 Nitpick comments (1)
pkgs/cli/src/commands/install/copy-migrations.ts (1)

186-206: Reliable implementation for finding the latest timestamp

The code effectively finds the latest timestamp across existing migrations with proper validation at each step. The multi-stage validation ensures only properly formatted timestamps are considered.

One small optimization could be replacing numeric string comparison with direct string comparison, since these are fixed-length strings of digits:

-if (parsedDate && parseInt(timestamp, 10) > parseInt(latestTimestamp, 10)) {
+if (parsedDate && timestamp > latestTimestamp) {

This would work equally well and might be slightly more efficient, but the current implementation is already sound.

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between c47e1e7 and ba8fe1f.

📒 Files selected for processing (1)
  • pkgs/cli/src/commands/install/copy-migrations.ts (3 hunks)
🧰 Additional context used
🪛 Biome (1.9.4)
pkgs/cli/src/commands/install/copy-migrations.ts

[error] 75-75: Change to an optional chain.

Unsafe fix: Change to an optional chain.

(lint/complexity/useOptionalChain)

⏰ Context from checks skipped due to timeout of 90000ms (2)
  • GitHub Check: main
  • GitHub Check: Cloudflare Pages
🔇 Additional comments (3)
pkgs/cli/src/commands/install/copy-migrations.ts (3)

71-153: Well-structured timestamp handling with comprehensive validation

The implementation of four helper functions for timestamp extraction, formatting, parsing, and generation is clean and thorough. I particularly appreciate the robust validation in parseTimestampToDate which checks both the format and logical validity of dates, preventing edge cases like invalid dates passing through.

The fallback mechanism in generateNewTimestamp ensures we always have a valid timestamp even if the reference is invalid, and the comparison with current time prevents going backwards in time - both excellent defensive programming practices.

🧰 Tools
🪛 Biome (1.9.4)

[error] 75-75: Change to an optional chain.

Unsafe fix: Change to an optional chain.

(lint/complexity/useOptionalChain)


269-278: Well-designed incremental timestamp generation

The implementation for generating new unique timestamps for migrations is well-designed:

  1. It maintains a base timestamp that increments with each migration
  2. It ensures the timestamps preserve the original migration order
  3. It uses the previous result as the base for the next timestamp

This approach ensures each migration gets a unique, strictly increasing timestamp, which is excellent for maintaining order and preventing conflicts.


339-351: Enhanced user feedback with clear migration details

The detailed success message with styled filenames and clear separation between new and existing migrations provides excellent user feedback. The approach of showing both the timestamp prefix (dimmed) and the original filename (bold) makes it easy for users to identify migrations while emphasizing their original names.

This level of detail in the output significantly improves the user experience when installing migrations.

@jumski
jumski merged commit 5e82c04 into main May 8, 2025
@jumski
jumski deleted the improve-cli-migration-copy branch May 8, 2025 05:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant