Backend Development
PHP Tutorial
How to determine if the data is greater than N when querying data? Is it recursive?
How to determine if the data is greater than N when querying data? Is it recursive?
For example, there are 300,000 pieces of data in the database. Now there are 30,000 pieces of data that match the query according to the conditions. Fetching too many at one time may be slow or other problems. I want to fetch 10,000 each time and fetch it three times. This is just an example. How should it be implemented? Use recursion?
Let me know. Thank you! ! !
Also, if the data from two tables are merged together, is there any way to sort them according to a certain field?
Reply content:
For example, there are 300,000 pieces of data in the database. Now there are 30,000 pieces of data that match the query according to the conditions. Fetching too many at one time may be slow or other problems. I want to fetch 10,000 each time and fetch it three times. This is just an example. How should it be implemented? Use recursion?
Let me know. Thank you! ! !
Also, if the data from two tables are merged together, is there any way to sort them according to a certain field?
Thanks for the invitation. For result sets with large amounts of data, I currently generally use the following two methods to process them. The purpose of using these two methods of processing is: first, to reduce the occupation of bandwidth resources when obtaining the data set, and second, to reduce the memory usage of the program when processing the result set.
Blocking is the idea of the question
<code>$sql = 'SELECT xxx FROM table WHERE xxx LIMIT limit ：limit offset :offset';
$limit = 10000;
$offset = 0;
do{
$stmt = $dbh->prepare($sql, array(PDO::ATTR_CURSOR => PDO::CURSOR_FWDONLY));
$stmt->execute(array('：limit' => $limit, ':offset' => $offset));
$data = $stmt->fetchAll();
// 处理逻辑
$offset += $limit;
}while(count($data));</code>Cursor, read and return by row
<code>$sql = 'SELECT xxx FROM table WHERE xxx';
$stmt = $dbh->prepare($sql, array(PDO::ATTR_CURSOR => PDO::CURSOR_SCROLL));
$stmt->execute();
$row = $stmt->fetch(PDO::FETCH_NUM, PDO::FETCH_ORI_LAST);
do {
// 处理逻辑 $row
} while ($row = $stmt->fetch(PDO::FETCH_NUM, PDO::FETCH_ORI_PRIOR));</code>
Personally suggest that when you want to find the subsequent blocks, use the where condition instead of directly limit 100000, 100000:
The difference between the two is as follows:
<code>//这个是第一种，取第20w开始的10w条数据 select * from table limit 100000 , 100000 order by id //这是第二种取法 select * from table where id > 100000 limit 100000 order by id 因为mysql用limit越往后，排序之后取后面的越慢</code>
It can be done directly through sql statements, no background processing is required; it can be sorted according to the fields you specifyorder by
Joint table query
For example: SELECT a.* from tableA as a, tableB as b where a.id = b.id order by a.id desc limit 0,10000";
You can first select count(1) from table where condition. See how much data there is in total. When there are more than N items,
you can select * from table limit m,n to query
2 tables in paging, select * from table1 union all select * form table2 order by a certain field. Note that the number of columns in the query must be the same. The same goes for the best type
Assuming that table IDs are evenly distributed, you can use the ID interval to query data in segments
//每次计划读取的数据条数
$max_per_size = 10000;
//找到符合条件的最小ID和最大ID
$sql = "SELECT min(id),max(id) AS max_id AS num FROM `table` WHERE xx='xx'";
$stmt = $pdo->prepare($sql);
$stmt->execute();
$res = $stmt->fetch(PDO::FETCH_ASSOC);
//符合条件的最大ID
$max_id = $res['max_id'];
//符合条件的最小ID
$min_id = $res['min_id'];
$times = ceil(($max_id - $min_id + 1) / $max_per_size);
//第一次的ID区间
$current_min_id = $min_id;
$current_max_id = $min_id + $max_per_page;
for($i = 0; $i<$times; $i++) {
//生成SQL语句
$sql = "SELECT * FROM `table` WHERE xx='xx' AND (id>= {$current_min_id} AND id <= {$current_max_id})";
//得到数据之后的操作
//.........
//重新计算下一次的ID区间
$current_min_id = $current_max_id + 1;
$current_max_id = $current_max_id + $max_per_page;
if ($current_max_id > $max_id) {
$current_max_id = $max_id;
}
}
Hot AI Tools
Undresser.AI Undress
AI-powered app for creating realistic nude photos
AI Clothes Remover
Online AI tool for removing clothes from photos.
Undress AI Tool
Undress images for free
Clothoff.io
AI clothes remover
Video Face Swap
Swap faces in any video effortlessly with our completely free AI face swap tool!
Hot Article
Hot Tools
Notepad++7.3.1
Easy-to-use and free code editor
SublimeText3 Chinese version
Chinese version, very easy to use
Zend Studio 13.0.1
Powerful PHP integrated development environment
Dreamweaver CS6
Visual web development tools
SublimeText3 Mac version
God-level code editing software (SublimeText3)
Hot Topics
1664
14
1423
52
1318
25
1269
29
1248
24
MySQL and phpMyAdmin: Core Features and Functions
Apr 22, 2025 am 12:12 AM
MySQL and phpMyAdmin are powerful database management tools. 1) MySQL is used to create databases and tables, and to execute DML and SQL queries. 2) phpMyAdmin provides an intuitive interface for database management, table structure management, data operations and user permission management.
Explain the purpose of foreign keys in MySQL.
Apr 25, 2025 am 12:17 AM
In MySQL, the function of foreign keys is to establish the relationship between tables and ensure the consistency and integrity of the data. Foreign keys maintain the effectiveness of data through reference integrity checks and cascading operations. Pay attention to performance optimization and avoid common errors when using them.
The Compatibility of IIS and PHP: A Deep Dive
Apr 22, 2025 am 12:01 AM
IIS and PHP are compatible and are implemented through FastCGI. 1.IIS forwards the .php file request to the FastCGI module through the configuration file. 2. The FastCGI module starts the PHP process to process requests to improve performance and stability. 3. In actual applications, you need to pay attention to configuration details, error debugging and performance optimization.
Compare and contrast MySQL and MariaDB.
Apr 26, 2025 am 12:08 AM
The main difference between MySQL and MariaDB is performance, functionality and license: 1. MySQL is developed by Oracle, and MariaDB is its fork. 2. MariaDB may perform better in high load environments. 3.MariaDB provides more storage engines and functions. 4.MySQL adopts a dual license, and MariaDB is completely open source. The existing infrastructure, performance requirements, functional requirements and license costs should be taken into account when choosing.
What happens if session_start() is called multiple times?
Apr 25, 2025 am 12:06 AM
Multiple calls to session_start() will result in warning messages and possible data overwrites. 1) PHP will issue a warning, prompting that the session has been started. 2) It may cause unexpected overwriting of session data. 3) Use session_status() to check the session status to avoid repeated calls.
SQL vs. MySQL: Clarifying the Relationship Between the Two
Apr 24, 2025 am 12:02 AM
SQL is a standard language for managing relational databases, while MySQL is a database management system that uses SQL. SQL defines ways to interact with a database, including CRUD operations, while MySQL implements the SQL standard and provides additional features such as stored procedures and triggers.
Composer: Aiding PHP Development Through AI
Apr 29, 2025 am 12:27 AM
AI can help optimize the use of Composer. Specific methods include: 1. Dependency management optimization: AI analyzes dependencies, recommends the best version combination, and reduces conflicts. 2. Automated code generation: AI generates composer.json files that conform to best practices. 3. Improve code quality: AI detects potential problems, provides optimization suggestions, and improves code quality. These methods are implemented through machine learning and natural language processing technologies to help developers improve efficiency and code quality.
MySQL: The Database, phpMyAdmin: The Management Interface
Apr 29, 2025 am 12:44 AM
MySQL and phpMyAdmin can be effectively managed through the following steps: 1. Create and delete database: Just click in phpMyAdmin to complete. 2. Manage tables: You can create tables, modify structures, and add indexes. 3. Data operation: Supports inserting, updating, deleting data and executing SQL queries. 4. Import and export data: Supports SQL, CSV, XML and other formats. 5. Optimization and monitoring: Use the OPTIMIZETABLE command to optimize tables and use query analyzers and monitoring tools to solve performance problems.
