How to optimize the performance of large number of image link replacements using Golang?

This article discusses how to optimize Golang code and efficiently handle large-scale image link replacement, especially for scenarios where images are uploaded to Alibaba Cloud OSS and replaced the original link. Performance bottlenecks are often frequent network requests and resource creation.

Assume that the initial string contains multiple image links:

 <code>content := `<img  class="wscnph lazy" src="/static/imghw/default1.png" data-src="https://img.php.cn/upload/article/001/246/273/174278558460021.jpg" alt="How to optimize the performance of large number of image link replacements using Golang?" ><img  class="wscnph lazy" src="/static/imghw/default1.png" data-src="https://img.php.cn/upload/article/001/246/273/174278558769141.jpg" alt="How to optimize the performance of large number of image link replacements using Golang?" > `</code>

We need to match the link, upload it to OSS, and replace the original link. The original code has performance problems, which are mainly reflected in the repeated creation of OSS clients and blocking uploads.

For optimization, we will adopt the following strategies:

1. Singleton OSS client: Use sync.Once to ensure that oss.New is executed only once, avoiding repeated creation of OSS client instances.

2. Concurrent upload: Use goroutine to upload images concurrently to OSS to avoid blocking the main thread.

3. Caching mechanism (optional): Introduce caches (such as sync.Map ) to store old and new link mappings of uploaded images to avoid repeated uploads of the same images.

The improved code is as follows:

 package main

import (
    "bytes"
    "fmt"
    "regexp"
    "strconv"
    "strings"
    "sync"
    "time"

    "github.com/aliyun/aliyun-oss-go-sdk/oss"
)

// ... (Aliyun OSS configuration: Endpoint, key, AccessKeySecret, BucketName) ...

var (
    ossClient *oss.Client
    once sync.Once
    imgCache sync.Map // key: old URL, value: new URL
)

func getOSSClient() *oss.Client {
    once.Do(func() {
        client, err := oss.New(Endpoint, key, AccessKeySecret)
        if err != nil {
            panic(err) // Handle error appropriately in production
        }
        ossClient = client
    })
    return ossClient
}


func Upload(url string) (string, error) {
    client := getOSSClient()
    bucket, err := client.Bucket(BucketName)
    if err != nil {
        return "", err
    }

    t1 := time.Now().Unix()
    long := 0 // Simplified counter, consider a more robust solution
    obj := fmt.Sprintf("article/%d%d.png", t1, long)

    err = bucket.PutObject(obj, bytes.NewReader(ReadImgData(url))) // ReadImgData function assumed to exist
    if err != nil {
        return "", err
    }
    return "https://" BucketName "." Endpoint "/" obj, nil
}

func RepImages(htmls string) string {
    var imgRE = regexp.MustCompile(`<img  alt="How to optimize the performance of large number of image link replacements using Golang?" > ]*src=["']([^"'] )["'][^>]*>`) // Improved regex
    imgs := imgRE.FindAllStringSubmatch(htmls, -1)
    var wg sync.WaitGroup
    resultChan := make(chan string, len(imgs))

    for _, img := range imgs {
        if len(img) > 1 {
            oldURL := img[1]
            if newURL, ok := imgCache.Load(oldURL); ok {
                resultChan <img  class="wscnph lazy" src="/static/imghw/default1.png" data-src="https://img.php.cn/upload/article/001/246/273/174278558769141.jpg" alt="How to optimize the performance of large number of image link replacements using Golang?" > `
    fmt.Println(RepImages(content))
}

// Placeholder for the actual image reading function
func ReadImgData(url string) []byte {
    // Replace this with your actual image reading logic
    // ...
    return []byte("image data")
}

Remember to replace Endpoint , key , AccessKeySecret , and BucketName for your Alibaba Cloud OSS configuration. ReadImgData function needs to be implemented according to your actual situation. This improved version utilizes concurrency and caching, significantly improving the efficiency of handling large number of image links. Error handling has also been strengthened to avoid program crashes. The caching mechanism can be further optimized, such as using LRU caching policies to manage memory usage.

The above is the detailed content of How to optimize the performance of large number of image link replacements using Golang?. For more information, please follow other related articles on the PHP Chinese website!