Skip to content
/ nmig Public
forked from AnatolyUss/nmig

NMIG is a database migration tool, written in Node.js and highly inspired by FromMySqlToPostgreSql.

License

Notifications You must be signed in to change notification settings

atlassian/nmig

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

NMIG - the database migration tool.

WHAT IS IT ALL ABOUT?

NMIG is an app, intended to make a process of migration from MySQL to PostgreSQL as easy and smooth as possible.

KEY FEATURES

  • Ease of use - the only thing needed to run this app is the Node.js runtime.
  • Accuracy of migration the database structure - NMIG converts MySQL data types to corresponding PostgreSQL data types, creates constraints, indexes, primary and foreign keys exactly as they were before migration.
  • Ability to migrate big databases - in order to reduce RAM consumption NMIG will split each table's data into several chunks.
    Chunk size can be adjusted easily via configuration file.
  • Speed of data transfer - in order to migrate data fast NMIG uses PostgreSQL COPY protocol.
  • Ease of monitoring - NMIG will provide detailed output about every step, it takes during the execution.
  • Ease of configuration - all the parameters required for migration should be put in one single JSON document.

SYSTEM REQUIREMENTS

  • Node.js 5.x.x

USAGE

1. Create a new database.
Sample:  CREATE DATABASE my_postgresql_database;

2. Download NMIG package and put it on the machine running your PostgreSQL.
Sample: /path/to/nmig

3. Edit configuration file located at /path/to/nmig/config.json with correct details.

Remarks:

4. Go to nmig directory, install dependencies, and run the app with --expose-gc flag
    Sample:

$ cd /path/to/nmig

$ npm install

$ node --expose-gc nmig.js

   Remark: you can increase node.js memory limit (RAM usage) using --max-old-space-size flag

   Following command will increase memory limit to ~2GB and run nmig
  $ node --max-old-space-size=2048 --expose-gc nmig.js

5. At the end of migration check log files, if necessary.
    Log files will be located under "logs_directory" folder in the root of the package.
    Note: "logs_directory" will be created during script execution.

6. In case of any remarks, misunderstandings or errors during migration,
    please feel free to email me [email protected]

VERSION

Current version is 1.2.0
(major version . improvements . bug fixes)

TEST

Tested using MySQL Community Server (5.6.21) and PostgreSQL (9.3).
The entire process of migration 59.6 MB database (52 tables, 570754 rows),
which includes data types mapping, creation of tables, constraints, indexes,
PKs, FKs, migration of data, garbage-collection (VACUUM) and analyzing the newly created
PostgreSQL database took 1 minute 18 seconds.

Remark:  it is highly recommended to VACUUM newly created database!
Just keep in mind, that VACUUM is a very time-consuming procedure.
So if you are short in time - disable VACUUM via config.json ("no_vacuum" parameter).
Such step will save you ~25% of migration time.
The migration process described above without VACUUM took 58 seconds only.

LICENSE

NMIG is available under "GNU GENERAL PUBLIC LICENSE" (v. 3)
http://www.gnu.org/licenses/gpl.txt.

REMARKS

Errors/Exceptions are not passed silently.
Any error will be immediately written into the error log file.

About

NMIG is a database migration tool, written in Node.js and highly inspired by FromMySqlToPostgreSql.

Resources

License

Stars

Watchers

Forks

Packages

No packages published

Languages

  • JavaScript 100.0%