Home Database Mysql Tutorial hbase权威指南: store file合并(compaction)

hbase权威指南: store file合并(compaction)

Jun 07, 2016 pm 04:30 PM
COMP file hbase store merge guide

????????? hbase为了防止小文件(被刷到磁盘的menstore)过多,以保证保证查询效率,hbase需要在必要的时候将这些小的store file合并成相对较大的store file,这个过程就称之为compaction。在hbase中,主要存在两种类型的compaction:minor? compaction和maj

????????? hbase为了防止小文件(被刷到磁盘的menstore)过多,以保证保证查询效率,hbase需要在必要的时候将这些小的store file合并成相对较大的store file,这个过程就称之为compaction。在hbase中,主要存在两种类型的compaction:minor? compaction和major compaction。

????????? major compaction 的功能是将所有的store file合并成一个,触发major compaction的可能条件有:major_compact 命令、majorCompact() API、region server自动运行(相关参数:hbase.hregion.majoucompaction 默认为24 小时、hbase.hregion.majorcompaction.jetter 默认值为0.2 防止region server 在同一时间进行major compaction)。hbase.hregion.majorcompaction.jetter参数的作用是:对参数hbase.hregion.majoucompaction 规定的值起到浮动的作用,假如两个参数都为默认值24和0,2,那么major compact最终使用的数值为:19.2~28.8 这个范围。

???????? minor compaction的运行机制要复杂一些,它由一下几个参数共同决定:

?????????????????? hbase.hstore.compaction.min :默认值为 3,表示至少需要三个满足条件的store file时,minor compaction才会启动

?????????????????? hbase.hstore.compaction.max 默认值为10,表示一次minor compaction中最多选取10个store file

?????????????????? hbase.hstore.compaction.min.size 表示文件大小小于该值的store file 一定会加入到minor compaction的store file中

?????????????????? hbase.hstore.compaction.max.size 表示文件大小大于该值的store file 一定会被minor compaction排除

?????????????????? hbase.hstore.compaction.ratio 将store file 按照文件年龄排序(older to younger),minor compaction总是从older store file开始选择,如果该文件的size 小于它后面hbase.hstore.compaction.max 个store file size 之和乘以 该ratio,则该store file 也将加入到minor compaction 中。

??????? 如果对minor compaction过程还是不了解,可以去看hbase中关于minor compaction 的源码,或者: http://blog.csdn.net/macyang/article/details/6423078

作者:azhao_dn 发表于2013-4-29 16:05:52 原文链接

阅读:25 评论:0 查看评论

hbase权威指南: store file合并(compaction)

Statement of this Website
The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn

Hot AI Tools

Undresser.AI Undress

Undresser.AI Undress

AI-powered app for creating realistic nude photos

AI Clothes Remover

AI Clothes Remover

Online AI tool for removing clothes from photos.

Undress AI Tool

Undress AI Tool

Undress images for free

Clothoff.io

Clothoff.io

AI clothes remover

Video Face Swap

Video Face Swap

Swap faces in any video effortlessly with our completely free AI face swap tool!

Hot Tools

Notepad++7.3.1

Notepad++7.3.1

Easy-to-use and free code editor

SublimeText3 Chinese version

SublimeText3 Chinese version

Chinese version, very easy to use

Zend Studio 13.0.1

Zend Studio 13.0.1

Powerful PHP integrated development environment

Dreamweaver CS6

Dreamweaver CS6

Visual web development tools

SublimeText3 Mac version

SublimeText3 Mac version

God-level code editing software (SublimeText3)

Guide to turning off VBS in Windows 11 Guide to turning off VBS in Windows 11 Mar 08, 2024 pm 01:03 PM

With the launch of Windows 11, Microsoft has introduced some new features and updates, including a security feature called VBS (Virtualization-basedSecurity). VBS utilizes virtualization technology to protect the operating system and sensitive data, thereby improving system security. However, for some users, VBS is not a necessary feature and may even affect system performance. Therefore, this article will introduce how to turn off VBS in Windows 11 to help

Setting up Chinese with VSCode: The Complete Guide Setting up Chinese with VSCode: The Complete Guide Mar 25, 2024 am 11:18 AM

VSCode Setup in Chinese: A Complete Guide In software development, Visual Studio Code (VSCode for short) is a commonly used integrated development environment. For developers who use Chinese, setting VSCode to the Chinese interface can improve work efficiency. This article will provide you with a complete guide, detailing how to set VSCode to a Chinese interface and providing specific code examples. Step 1: Download and install the language pack. After opening VSCode, click on the left

Hongmeng native application random poetry Hongmeng native application random poetry Feb 19, 2024 pm 01:36 PM

To learn more about open source, please visit: 51CTO Hongmeng Developer Community https://ost.51cto.com Running environment DAYU200:4.0.10.16SDK: 4.0.10.15IDE: 4.0.600 1. To create an application, click File- >newFile->CreateProgect. Select template: [OpenHarmony] EmptyAbility: Fill in the project name, shici, application package name com.nut.shici, and application storage location XXX (no Chinese, special characters, or spaces). CompileSDK10, Model: Stage. Device

Detailed explanation of jQuery reference methods: Quick start guide Detailed explanation of jQuery reference methods: Quick start guide Feb 27, 2024 pm 06:45 PM

Detailed explanation of jQuery reference method: Quick start guide jQuery is a popular JavaScript library that is widely used in website development. It simplifies JavaScript programming and provides developers with rich functions and features. This article will introduce jQuery's reference method in detail and provide specific code examples to help readers get started quickly. Introducing jQuery First, we need to introduce the jQuery library into the HTML file. It can be introduced through a CDN link or downloaded

Conda usage guide: easily upgrade Python version Conda usage guide: easily upgrade Python version Feb 22, 2024 pm 01:00 PM

Conda Usage Guide: Easily upgrade the Python version, specific code examples are required Introduction: During the development process of Python, we often need to upgrade the Python version to obtain new features or fix known bugs. However, manually upgrading the Python version can be troublesome, especially when our projects and dependent packages are relatively complex. Fortunately, Conda, as an excellent package manager and environment management tool, can help us easily upgrade the Python version. This article will introduce how to use

Install Deepin Linux on tablet: Install Deepin Linux on tablet: Feb 13, 2024 pm 11:18 PM

With the continuous development of technology, Linux operating systems have been widely used in various fields. Installing Deepin Linux system on tablets allows us to experience the charm of Linux more conveniently. Let’s discuss the installation of Deepin Linux on tablets. Specific steps for Linux. Preparation work Before installing Deepin Linux on the tablet, we need to make some preparations. We need to back up important data in the tablet to avoid data loss during the installation process. We need to download the image file of Deepin Linux and write it to to a USB flash drive or SD card for use during the installation process. Next, we can start the installation process. We need to set the tablet to start from the U disk or SD

Detailed explanation of Word document operation: merge two pages into one Detailed explanation of Word document operation: merge two pages into one Mar 26, 2024 am 08:18 AM

Word documents are one of the most frequently used applications in our daily work and study. When working with documents, you may sometimes encounter a situation where you need to merge two pages into one. This article will introduce in detail how to merge two pages into one page in a Word document to help readers handle document layout more efficiently. In Word documents, the operation of merging two pages into one is usually used to save paper and printing costs, or to make the document more compact and neat. The following are the specific steps to merge two pages into one: Step 1: Open the Word that needs to be operated

PHP7 installation directory configuration guide PHP7 installation directory configuration guide Mar 11, 2024 pm 12:18 PM

PHP7 Installation Directory Configuration Guide PHP is a popular server-side scripting language used to develop dynamic web pages. Currently, the latest version of PHP is PHP7, which introduces many new features and performance optimizations and is the preferred version for many websites and applications. When installing PHP7, it is very important to correctly configure the installation directory. This article will provide you with a detailed guide to configuring the PHP7 installation directory, with specific code examples. To download PHP7 first, you need to download it from the PHP official website (https://www.

See all articles