Web Front-end
JS Tutorial
Introduction to the method of Node batch downloading files to the local (with code)
Introduction to the method of Node batch downloading files to the local (with code)
This article brings you an introduction to the method of Node batch downloading files to the local (with code). It has certain reference value. Friends in need can refer to it. I hope it will be helpful to you.
Process multiple files separately
If the number of files is controllable and there is no requirement for the downloaded file format, you can use the simplest method to directly traverse the files and process them separately. Each download link creates a single-file download or iframe download link.
zip package batch download
Although it is possible to traverse all files and then download individual files in batches, this experience is not very good after all. The most common approach is Download and package batch files into zip.
So the first implementation idea is: in the proxy service, first traverse all files to request file data, then compress it into a zip package, and then return the zip package to the client.
This is OK if the download volume data is relatively small, but if the batch file is very large and large, the user has to wait for the background to request all the data and package them into a compressed package before the front end can give feedback. This can take a long time and the user experience can be poor.
During the preliminary research of colleagues, it was said that there can be a streaming compression and downloading capability here. The general idea is to chunk back the package and add streaming compression.
......
let fileCounter = 0;
const zippedFilename = encodeURIComponent(downloadData.name);
const list = downloadData.list || [];
const header = {
'Content-Type': 'application/x-zip',
'Pragma': 'public',
'Expires': '0',
'Cache-Control': 'private, must-revalidate, post-check=0, pre-check=0',
'Content-disposition': 'attachment; filename="' + zippedFilename + '"',
'Transfer-Encoding': 'chunked',
'Content-Transfer-Encoding': 'binary'
};
res.writeHead(200, header);
archive.store = true;
archive.pipe(res);
list.map(item => {
fileCounter++;
let inStream = request.get(item.downLoadUrl);
let name = item.fileName;
let length = 0;
inStream.on('response', function(awsData) {
archive.append(inStream, {
name: name
});
}).on('data', function(data) {
length += data.length;
}).on('error', function(e) {
console.error(name + '-error', e);
}).on('end', function(endData) {
fileCounter--;
if (fileCounter < 1) {
archive.finalize();
}
});
});
archive.on('error', function(err) {
throw err;
});
archive.on('finish', function(err) {
return res.end();
});
......Of course, there are still some details that need to be dealt with: such as the problem of Chinese file names, whether the total size of the downloaded file needs to be limited, whether the file does not exist, etc.
The above is the detailed content of Introduction to the method of Node batch downloading files to the local (with code). For more information, please follow other related articles on the PHP Chinese website!
Hot AI Tools
Undresser.AI Undress
AI-powered app for creating realistic nude photos
AI Clothes Remover
Online AI tool for removing clothes from photos.
Undress AI Tool
Undress images for free
Clothoff.io
AI clothes remover
Video Face Swap
Swap faces in any video effortlessly with our completely free AI face swap tool!
Hot Article
Hot Tools
Notepad++7.3.1
Easy-to-use and free code editor
SublimeText3 Chinese version
Chinese version, very easy to use
Zend Studio 13.0.1
Powerful PHP integrated development environment
Dreamweaver CS6
Visual web development tools
SublimeText3 Mac version
God-level code editing software (SublimeText3)
Hot Topics
1667
14
1426
52
1328
25
1273
29
1255
24
How to download episodes of Hongguo short drama
Mar 11, 2024 pm 09:16 PM
Hongguo Short Play is not only a platform for watching short plays, but also a treasure trove of rich content, including novels and other exciting content. This is undoubtedly a huge surprise for many users who love reading. However, many users still don’t know how to download and watch these novels in Hongguo Short Play. In the following, the editor of this website will provide you with detailed downloading steps. I hope it can help everyone in need. Partners. How to download and watch the Hongguo short play? The answer: [Hongguo short play] - [Audio book] - [Article] - [Download]. Specific steps: 1. First open the Hongguo Short Drama software, enter the homepage and click the [Listen to Books] button at the top of the page; 2. Then on the novel page we can see a lot of article content, here
What should I do if I download other people's wallpapers after logging into another account on wallpaperengine?
Mar 19, 2024 pm 02:00 PM
When you log in to someone else's steam account on your computer, and that other person's account happens to have wallpaper software, steam will automatically download the wallpapers subscribed to the other person's account after switching back to your own account. Users can solve this problem by turning off steam cloud synchronization. What to do if wallpaperengine downloads other people's wallpapers after logging into another account 1. Log in to your own steam account, find cloud synchronization in settings, and turn off steam cloud synchronization. 2. Log in to someone else's Steam account you logged in before, open the Wallpaper Creative Workshop, find the subscription content, and then cancel all subscriptions. (In case you cannot find the wallpaper in the future, you can collect it first and then cancel the subscription) 3. Switch back to your own steam
How to download links starting with 115://? Download method introduction
Mar 14, 2024 am 11:58 AM
Recently, many users have been asking the editor, how to download links starting with 115://? If you want to download links starting with 115://, you need to use the 115 browser. After you download the 115 browser, let's take a look at the download tutorial compiled by the editor below. Introduction to how to download links starting with 115:// 1. Log in to 115.com, download and install the 115 browser. 2. Enter: chrome://extensions/ in the 115 browser address bar, enter the extension center, search for Tampermonkey, and install the corresponding plug-in. 3. Enter in the address bar of 115 browser: Grease Monkey Script: https://greasyfork.org/en/
Introduction to how to download and install the superpeople game
Mar 30, 2024 pm 04:01 PM
The superpeople game can be downloaded through the steam client. The size of this game is about 28G. It usually takes one and a half hours to download and install. Here is a specific download and installation tutorial for you! New method to apply for global closed testing 1) Search for "SUPERPEOPLE" in the Steam store (steam client download) 2) Click "Request access to SUPERPEOPLE closed testing" at the bottom of the "SUPERPEOPLE" store page 3) After clicking the request access button, The "SUPERPEOPLECBT" game can be confirmed in the Steam library 4) Click the install button in "SUPERPEOPLECBT" and download
How to download foobar2000? -How to use foobar2000
Mar 18, 2024 am 10:58 AM
foobar2000 is a software that can listen to music resources at any time. It brings you all kinds of music with lossless sound quality. The enhanced version of the music player allows you to get a more comprehensive and comfortable music experience. Its design concept is to play the advanced audio on the computer The device is transplanted to mobile phones to provide a more convenient and efficient music playback experience. The interface design is simple, clear and easy to use. It adopts a minimalist design style without too many decorations and cumbersome operations to get started quickly. It also supports a variety of skins and Theme, personalize settings according to your own preferences, and create an exclusive music player that supports the playback of multiple audio formats. It also supports the audio gain function to adjust the volume according to your own hearing conditions to avoid hearing damage caused by excessive volume. Next, let me help you
How to download Quark network disk to local? How to save files downloaded from Quark Network Disk back to the local computer
Mar 13, 2024 pm 08:31 PM
Many users need to download files when using Quark Network Disk, but we want to save them locally, so how to set this up? Let this site introduce to users in detail how to save files downloaded from Quark Network Disk back to the local computer. How to save files downloaded from Quark network disk back to your local computer 1. Open Quark, log in to your account, and click the list icon. 2. After clicking the icon, select the network disk. 3. After entering Quark Network Disk, click My Files. 4. After entering My Files, select the file you want to download and click the three-dot icon. 5. Check the file you want to download and click Download.
How to download Beilehu children's songs
Mar 28, 2024 am 11:10 AM
As an indispensable accompaniment to children's growth, Beilehu's children's songs have won the love of countless parents and children with their cheerful melody, vivid pictures and entertaining and educational content. In order to allow babies to enjoy the joy brought by children's songs anytime and anywhere, many parents hope to download Beilehu's children's songs to their mobile phones or tablets so that they can listen to their children at any time, but how to save Beilehu's children's songs? On your mobile phone, this tutorial will bring you a detailed introduction. Users who don’t understand it yet can come and read along with this article to learn more. Beilehu Nursery Rhymes Download Children's Songs Multi-Picture Tutorial: Open the software and select a children's song you want to download. The editor takes "Classic Children's Songs" as an example. 2. Click the "Download" button below the children's song star.
Solve the problem of downloading PDF files in PHP7
Feb 29, 2024 am 11:12 AM
Solve the problems encountered in downloading PDF files with PHP7 In web development, we often encounter the need to use PHP to download files. Especially downloading PDF files can help users obtain necessary information or files. However, sometimes you will encounter some problems when downloading PDF files in PHP7, such as garbled characters and incomplete downloads. This article will detail how to solve problems you may encounter when downloading PDF files in PHP7 and provide some specific code examples. Problem analysis: In PHP7, due to character encoding and H
