Mass Migration Automation: Porting 524 Domains in Less Than 4 Hours

The Overhead

Migrating hundreds of active domains across different servers is traditionally a massive operational bottleneck. Doing it manually introduces severe human error, unpredictable downtime, and immense overhead. The objective was to build a highly scalable, automated pipeline to handle DNS synchronization across multiple providers, server provisioning, file transfers, and SSL issuance with zero dropped payloads.

The Architecture

To handle the scale and avoid API rate limits or server I/O bottlenecks, I designed a 4-stage pipeline in Node.js utilizing p-limit for controlled concurrency. The system is strictly idempotent—powered by a custom StateManager that tracks the migration lifecycle, ensuring that if a process fails, it resumes exactly where it left off without duplicate executions.

Pipeline Design

The execution is split into 4 discrete, idempotent stages:

The Migration Lifecycle

The execution is heavily optimized for speed while respecting the limits of the target infrastructure. DNS propagation (which has a higher threshold) is run at a concurrency of 20, while file uploads (which are I/O intensive) are strictly capped at 5 concurrent streams to prevent server overload.

Execution Sequence

flowchart LR
    Pipeline["Node.js Pipeline"] --> StateManager["State Manager"]
    StateManager --> Read["Read Pending Tasks"]
    
    Read --> S1["Stage 1: DNS Sync<br>(Max 20)"]
    Read --> S2["Stage 2: Provisioning<br>(Max 10)"]
    Read --> S3["Stage 3: File Transfer<br>(Max 5)"]
    Read --> S4["Stage 4: SSL Issuance<br>(Max 10)"]
    
    S1 --> APIs["Cloudflare / Namecheap / Spaceship"]
    S2 --> Hestia["HestiaCP API"]
    S3 --> SFTP["SFTP Stream"]
    S4 --> ACME["Let's Encrypt ACME"]

Each stage writes its completion to the StateManager before advancing, ensuring the pipeline is resumable at any point.

Key Engineering Decisions

• Idempotency first: The StateManager writes to disk atomically, meaning network failures or unexpected timeouts during massive batch operations won't corrupt the target server state.
• Granular Concurrency Control: By decoupling the concurrency limits based on the operation type (Network vs I/O vs CPU), the script maximizes throughput without triggering API rate limits (HTTP 429) from providers like Namecheap or bringing down the target HestiaCP node.
• Fail-safe Execution: Designed to be run either as a full pipeline or triggered in isolated granular steps (pnpm step:dns, pnpm step:ssl) for debugging and strict control.

The Impact

• Time Reduction: Condensed a multi-day manual migration process into less than 4 hours of autonomous execution.
• Scalability: Successfully ported 524 live domains with zero configuration mismatch.
• Observability: Complete audit trails via state mapping for every single domain migrated.
• Zero downtime: All domains remained accessible throughout the migration window.