Table of Contents
项目地址
功能
设计
InitCollection
CopyData
OplogSync
问题
目前进度 :
Home Database Mysql Tutorial mongodb迁移工具

mongodb迁移工具

Jun 07, 2016 pm 04:41 PM
github mongodb address tool migrate project

项目地址 [github]地址戳这里.(https://github.com/xbsura/mongo-migrate) 使用go语言编写,与m-tools保持一致,除了go语言自带的库,还需要mgo库支持. 功能 mongodb实例之间的数据迁移,带有根据源与目的的节点性质,进行自动分片,预split与movechunk,创建索引,

项目地址

[github]地址戳这里.(https://github.com/xbsura/mongo-migrate)

使用go语言编写,与m-tools保持一致,除了go语言自带的库,还需要mgo库支持.

功能

mongodb实例之间的数据迁移,带有根据源与目的的节点性质,进行自动分片,预split与movechunk,创建索引,拷贝数据与同步oplog几个过程.

设计

迁移工具分为三个阶段,初始化阶段,数据拷贝与oplog同步,分别使用InitCollection类,CopyData类与OplogSync类实现.

InitCollection

过程:

  1. 建立与源,目标的连接,若失败,raise.
  2. 初始化两个标识位,s(shard),o(oplog),为0, 0.
  3. 检测源类型,若为mongod,检测是否为复制集,若无法检测,raise;若为复制集,令o=1;若为mongos,查看要迁移的ns是否分片,如果分片,检测数据分片类型,若都为复制集,令o=1.
  4. 检测目标类型,若为mongod,令s=0.
  5. 若s=1,在目标进行分片,按照源的chunk进行split并随机movechunk.

CopyData

过程:
1. 建立源与目的的连接.
2. 获取要迁移的ns.
4. 如果o=1,记录源每个数据分片oplog最新值.
5. 如果源ns分片,按照chunk边界从源并发读,并向目标并发写;如果源未分片,使用一个query查询数据并向目标写.

OplogSync

过程:
1. 如果o=0,退出.
2. 建立源各个数据节点,目标节点(单节点)的连接.
3. 在每个源分片查找oplog ts大于等于CopyData记录时间点的记录,并逐条应用到目标节点.

问题

  1. 日志貌似打印有一点点问题.
  2. 性能没测试.
  3. 代码有点烂.

目前进度 :

完成可运行的版本,没有完整测试.

Statement of this Website
The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn

Hot AI Tools

Undresser.AI Undress

Undresser.AI Undress

AI-powered app for creating realistic nude photos

AI Clothes Remover

AI Clothes Remover

Online AI tool for removing clothes from photos.

Undress AI Tool

Undress AI Tool

Undress images for free

Clothoff.io

Clothoff.io

AI clothes remover

Video Face Swap

Video Face Swap

Swap faces in any video effortlessly with our completely free AI face swap tool!

Hot Tools

Notepad++7.3.1

Notepad++7.3.1

Easy-to-use and free code editor

SublimeText3 Chinese version

SublimeText3 Chinese version

Chinese version, very easy to use

Zend Studio 13.0.1

Zend Studio 13.0.1

Powerful PHP integrated development environment

Dreamweaver CS6

Dreamweaver CS6

Visual web development tools

SublimeText3 Mac version

SublimeText3 Mac version

God-level code editing software (SublimeText3)

Bitcoin price today Bitcoin price today Apr 28, 2025 pm 07:39 PM

Bitcoin’s price fluctuations today are affected by many factors such as macroeconomics, policies, and market sentiment. Investors need to pay attention to technical and fundamental analysis to make informed decisions.

Free coins trading market software recommendations The top ten easy-to-use coins trading apps Free coins trading market software recommendations The top ten easy-to-use coins trading apps Apr 28, 2025 pm 04:33 PM

The top ten recommended cryptocurrency trading software are: 1. OKX, 2. Binance, 3. Coinbase, 4. KuCoin, 5. Huobi, 6. Crypto.com, 7. Kraken, 8. Bitfinex, 9. Bybit, 10. Gate.io. These apps all provide real-time market data and trading tools, suitable for users at different levels.

Recommended reliable digital currency trading platforms. Top 10 digital currency exchanges in the world. 2025 Recommended reliable digital currency trading platforms. Top 10 digital currency exchanges in the world. 2025 Apr 28, 2025 pm 04:30 PM

Recommended reliable digital currency trading platforms: 1. OKX, 2. Binance, 3. Coinbase, 4. Kraken, 5. Huobi, 6. KuCoin, 7. Bitfinex, 8. Gemini, 9. Bitstamp, 10. Poloniex, these platforms are known for their security, user experience and diverse functions, suitable for users at different levels of digital currency transactions

Download the official website of Ouyi Exchange app for Apple mobile phone Download the official website of Ouyi Exchange app for Apple mobile phone Apr 28, 2025 pm 06:57 PM

The Ouyi Exchange app supports downloading of Apple mobile phones, visit the official website, click the "Apple Mobile" option, obtain and install it in the App Store, register or log in to conduct cryptocurrency trading.

How much is Bitcoin worth How much is Bitcoin worth Apr 28, 2025 pm 07:42 PM

Bitcoin’s price ranges from $20,000 to $30,000. 1. Bitcoin’s price has fluctuated dramatically since 2009, reaching nearly $20,000 in 2017 and nearly $60,000 in 2021. 2. Prices are affected by factors such as market demand, supply, and macroeconomic environment. 3. Get real-time prices through exchanges, mobile apps and websites. 4. Bitcoin price is highly volatile, driven by market sentiment and external factors. 5. It has a certain relationship with traditional financial markets and is affected by global stock markets, the strength of the US dollar, etc. 6. The long-term trend is bullish, but risks need to be assessed with caution.

Spot King Transformation Note: How to layout the next generation of on-chain ecosystem with Gate.io MeMebox 2.0? Spot King Transformation Note: How to layout the next generation of on-chain ecosystem with Gate.io MeMebox 2.0? Apr 28, 2025 pm 03:36 PM

Gate.io has achieved the transformation from spot trading to on-chain ecosystem through MeMebox 2.0. 1) Build a cross-chain infrastructure and support the interoperability of 12 main chains; 2) Create a DeFi application ecosystem and provide one-stop services; 3) Implement incentive mechanisms and reconstruct value allocation.

How to choose a compliant and secure Bitcoin trading platform How to choose a compliant and secure Bitcoin trading platform Apr 28, 2025 pm 05:42 PM

When choosing a compliant and secure Bitcoin trading platform, you need to evaluate its regulatory license, KYC/AML policies and security measures, and recommend three major platforms: Binance, OKX and gate.io.

How to use the chrono library in C? How to use the chrono library in C? Apr 28, 2025 pm 10:18 PM

Using the chrono library in C can allow you to control time and time intervals more accurately. Let's explore the charm of this library. C's chrono library is part of the standard library, which provides a modern way to deal with time and time intervals. For programmers who have suffered from time.h and ctime, chrono is undoubtedly a boon. It not only improves the readability and maintainability of the code, but also provides higher accuracy and flexibility. Let's start with the basics. The chrono library mainly includes the following key components: std::chrono::system_clock: represents the system clock, used to obtain the current time. std::chron

See all articles