Table of Contents
回复内容:
Home Backend Development PHP Tutorial 请问如何对大数据进行排序?

请问如何对大数据进行排序?

Jun 06, 2016 pm 08:16 PM
mysql php Big Data big data processing

我没处理过大数据,假设一种情况,有10亿条数据,这么大的数据,请问一般有什么样的算法能对这样大的数据进行排序?我希望的是能给出解决方案或者是解决思路。在此先感谢各位。

回复内容:

我没处理过大数据,假设一种情况,有10亿条数据,这么大的数据,请问一般有什么样的算法能对这样大的数据进行排序?我希望的是能给出解决方案或者是解决思路。在此先感谢各位。

纯属探讨,我也没有实践经验。
我觉得大数据的唯一处理就是怎么写入大数据到数据库,至于排序和查询...这件事情根本就不应该发生在大数据上。如你所说的10E条,水平拆分掉就不是大数据了,所以你的问题也就不存在了。
如果现在的情况已经是10E条存在一个库里面了,我觉得应该根据你的需求决定如何拆分才是关键的。
比如商城系统订单数据就很多,但是最明显的一个特征就是用户总是先看到 “最近的订单” 这种东西,在我看来这就是一种明显的拆分。

在我的看来,大数据的主体只有一个索引,就是主键,因为太多的索引会影响高并发的插入速度,那么如果需要搜索大数据怎么办?当然是空间换时间,比如淘宝那么多商品,想搜索特定的商品怎么才能快?从主数据中搜索?那不是疯了,我觉得肯定是从派生表上查询的,比如 日用品商品表, 鱼竿商品表 等等,根据搜索关键字去找符合条件的表,然后从这个表中再读数据,这才能快起来。

以上纯属个人臆测,有错误的地方恭请指正,不胜感激。

Statement of this Website
The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn

Hot AI Tools

Undresser.AI Undress

Undresser.AI Undress

AI-powered app for creating realistic nude photos

AI Clothes Remover

AI Clothes Remover

Online AI tool for removing clothes from photos.

Undress AI Tool

Undress AI Tool

Undress images for free

Clothoff.io

Clothoff.io

AI clothes remover

Video Face Swap

Video Face Swap

Swap faces in any video effortlessly with our completely free AI face swap tool!

Hot Article

Roblox: Bubble Gum Simulator Infinity - How To Get And Use Royal Keys
4 weeks ago By 尊渡假赌尊渡假赌尊渡假赌
Nordhold: Fusion System, Explained
4 weeks ago By 尊渡假赌尊渡假赌尊渡假赌
Mandragora: Whispers Of The Witch Tree - How To Unlock The Grappling Hook
3 weeks ago By 尊渡假赌尊渡假赌尊渡假赌

Hot Tools

Notepad++7.3.1

Notepad++7.3.1

Easy-to-use and free code editor

SublimeText3 Chinese version

SublimeText3 Chinese version

Chinese version, very easy to use

Zend Studio 13.0.1

Zend Studio 13.0.1

Powerful PHP integrated development environment

Dreamweaver CS6

Dreamweaver CS6

Visual web development tools

SublimeText3 Mac version

SublimeText3 Mac version

God-level code editing software (SublimeText3)

Hot Topics

Java Tutorial
1673
14
PHP Tutorial
1278
29
C# Tutorial
1257
24
MySQL: The Database, phpMyAdmin: The Management Interface MySQL: The Database, phpMyAdmin: The Management Interface Apr 29, 2025 am 12:44 AM

MySQL and phpMyAdmin can be effectively managed through the following steps: 1. Create and delete database: Just click in phpMyAdmin to complete. 2. Manage tables: You can create tables, modify structures, and add indexes. 3. Data operation: Supports inserting, updating, deleting data and executing SQL queries. 4. Import and export data: Supports SQL, CSV, XML and other formats. 5. Optimization and monitoring: Use the OPTIMIZETABLE command to optimize tables and use query analyzers and monitoring tools to solve performance problems.

What is the significance of the session_start() function? What is the significance of the session_start() function? May 03, 2025 am 12:18 AM

session_start()iscrucialinPHPformanagingusersessions.1)Itinitiatesanewsessionifnoneexists,2)resumesanexistingsession,and3)setsasessioncookieforcontinuityacrossrequests,enablingapplicationslikeuserauthenticationandpersonalizedcontent.

Steps to add and delete fields to MySQL tables Steps to add and delete fields to MySQL tables Apr 29, 2025 pm 04:15 PM

In MySQL, add fields using ALTERTABLEtable_nameADDCOLUMNnew_columnVARCHAR(255)AFTERexisting_column, delete fields using ALTERTABLEtable_nameDROPCOLUMNcolumn_to_drop. When adding fields, you need to specify a location to optimize query performance and data structure; before deleting fields, you need to confirm that the operation is irreversible; modifying table structure using online DDL, backup data, test environment, and low-load time periods is performance optimization and best practice.

Detailed explanation of the installation steps of MySQL on macOS system Detailed explanation of the installation steps of MySQL on macOS system Apr 29, 2025 pm 03:36 PM

Installing MySQL on macOS can be achieved through the following steps: 1. Install Homebrew, using the command /bin/bash-c"$(curl-fsSLhttps://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)". 2. Update Homebrew and use brewupdate. 3. Install MySQL and use brewinstallmysql. 4. Start MySQL service and use brewservicesstartmysql. After installation, you can use mysql-u

How to use MySQL functions for data processing and calculation How to use MySQL functions for data processing and calculation Apr 29, 2025 pm 04:21 PM

MySQL functions can be used for data processing and calculation. 1. Basic usage includes string processing, date calculation and mathematical operations. 2. Advanced usage involves combining multiple functions to implement complex operations. 3. Performance optimization requires avoiding the use of functions in the WHERE clause and using GROUPBY and temporary tables.

How to uninstall MySQL and clean residual files How to uninstall MySQL and clean residual files Apr 29, 2025 pm 04:03 PM

To safely and thoroughly uninstall MySQL and clean all residual files, follow the following steps: 1. Stop MySQL service; 2. Uninstall MySQL packages; 3. Clean configuration files and data directories; 4. Verify that the uninstallation is thorough.

An efficient way to batch insert data in MySQL An efficient way to batch insert data in MySQL Apr 29, 2025 pm 04:18 PM

Efficient methods for batch inserting data in MySQL include: 1. Using INSERTINTO...VALUES syntax, 2. Using LOADDATAINFILE command, 3. Using transaction processing, 4. Adjust batch size, 5. Disable indexing, 6. Using INSERTIGNORE or INSERT...ONDUPLICATEKEYUPDATE, these methods can significantly improve database operation efficiency.

Composer: The Package Manager for PHP Developers Composer: The Package Manager for PHP Developers May 02, 2025 am 12:23 AM

Composer is a dependency management tool for PHP, and manages project dependencies through composer.json file. 1) parse composer.json to obtain dependency information; 2) parse dependencies to form a dependency tree; 3) download and install dependencies from Packagist to the vendor directory; 4) generate composer.lock file to lock the dependency version to ensure team consistency and project maintainability.

See all articles