What are the serialization methods in golang?

Golang serialization methods include: 1. Use the Gob package to manage the gob stream. Gob is bound to the type. If it is found that there is more or less, it will be filled or truncated according to the order. 2. Using the json package, you can implement JSON encoding and decoding defined in RFC 7159; during the serialization process, if the members in the structure are lowercase, an error will occur. 3. Using the Binary package, simple conversion between numbers and byte sequences and encoding and decoding of varint can be achieved. 4. Use protobuf protocol.

The operating environment of this tutorial: Windows 7 system, GO version 1.18, Dell G3 computer.

In the programming process, we always encounter the problem of transmitting our data objects over the network or saving them to files, which requires encoding and decoding actions.

There are currently many encoding formats: json, XML, Gob, Google Protocol Buffer, etc. In Go language, how to encode and decode data in this way?

Serialization and deserialization definition

Serialization (Serialization) is to convert the state information of an object into a form that can be stored or transmitted the process of. During serialization, an object writes its current state to temporary or persistent storage.

In turn, re-reading the variable from the storage area and re-creating the object is deserialization.

In the Go language, the encoding package is specifically designed to handle this type of serialization encoding and decoding issues.

Serialization method – Gob

gob Package management gob stream – encoder (sender) and decoder The binary value exchanged between (receivers). A typical use is to transport parameters and results of remote procedure calls (RPCs), such as the gobs stream used in the "net/rpc" package.

For details, please refer to the documentation: https://docs.studygolang.com/pkg/encoding/gob/

His official website gives an example:

package main

import (
	"bytes"
	"encoding/gob"
	"fmt"
	"log"
)

type P struct {
	X, Y, Z int
	Name    string
}

type Q struct {
	X, Y *int32
	Name string
}

// This example shows the basic usage of the package: Create an encoder,
// transmit some values, receive them with a decoder.
func main() {
	// Initialize the encoder and decoder. Normally enc and dec would be
	// bound to network connections and the encoder and decoder would
	// run in different processes.
	var network bytes.Buffer        // Stand-in for a network connection  //Buffer是具有Read和Write方法的可变大小的字节缓冲区。
	enc := gob.NewEncoder(&network) // Will write to network.
	dec := gob.NewDecoder(&network) // Will read from network.

	// Encode (send) some values.
	err := enc.Encode(P{3, 4, 5, "Pythagoras"})
	if err != nil {
		log.Fatal("encode error:", err)
	}
	err = enc.Encode(P{1782, 1841, 1922, "Treehouse"})
	if err != nil {
		log.Fatal("encode error:", err)
	}

	// Decode (receive) and print the values.
	var q Q
	err = dec.Decode(&q)
	if err != nil {
		log.Fatal("decode error 1:", err)
	}
	fmt.Printf("%q: {%d, %d}\n", q.Name, *q.X, *q.Y)
	err = dec.Decode(&q)
	if err != nil {
		log.Fatal("decode error 2:", err)
	}
	fmt.Printf("%q: {%d, %d}\n", q.Name, *q.X, *q.Y)

}

Run The result is:

"Pythagoras": {3, 4}
"Treehouse": {1782, 1841}

Personally, I think this example is really good. We see that structures P and Q are different. We see that Q is missing a Z variable.

However, it can still be parsed when decoding. This shows that when using gob, it is bound according to the type. If it is found that there is more or less, it will be based on the order. Pad or truncate.

Next, let’s talk about how to encode in detail:

1. bytes.Buffer Type

First, we need to define a bytes.Buffer type to accept the structure that needs to be serialized. This type is like this:

// A Buffer is a variable-sized buffer of bytes with Read and Write methods.(Buffer是具有Read和Write方法的可变大小的字节缓冲区)
// The zero value for Buffer is an empty buffer ready to use.
type Buffer struct {
	buf      []byte // contents are the bytes buf[off : len(buf)]
	off      int    // read at &buf[off], write at &buf[len(buf)]
	lastRead readOp // last read operation, so that Unread* can work correctly.
}

Using the above example, you can see The output is:

"Pythagoras": {3, 4} ==>
{[42 255 129 3 1 1 1 80 1 255 130 0 1 4 1 1 88 1 4 0 1 1 89 1 4 0 1 1 90 1 4 0 1 4 78 97 109 101 1 12 0 0 0 21 255 130 1 6 1 8 1 10 1 10 80 121 116 104 97 103 111 114 97 115 0] 0 0}

You can see that in Buffer, it is a binary number (one byte has 8 bits, up to 255)

2. Encode After encoding

, encode the structure that needs to be encoded and serialized:

enc := gob.NewEncoder(&network) // Will write to network.
// Encode (send) some values.
if err := enc.Encode(P{3, 4, 5, "Pythagoras"}); err != nil {
	log.Fatal("encode error:", err)
}

Here, the first thing is to obtain*Encoder Object, after obtaining the object, use the method Encode of the *Encoder object for encoding.

Here, it should be noted that Encode If it is network programming, you can actually send messages directly to the other party without performing the socket send operation.

For example: There is code on the srever side:

func main() {
	l, err := net.Listen("tcp", "127.0.0.1:8000")  //监听端口
	if err != nil {
		log.Fatal("net Listen() error is ", err)
	}

	p := P{
		1, 2, 3,
		"name"}

	conn, err := l.Accept()
	if err != nil {
		log.Fatal("net Accept() error is ", err)
	}
	defer func() { _ = conn.Close() }()
	//参数是conn 时，即可发出
	enc := gob.NewEncoder(conn)
	if err = enc.Encode(p); err != nil {  //发生结构体数据
		log.Fatal("enc Encode() error is ", err)
	}
}

There is code on the client side:

func main() {
	conn,err := net.Dial("tcp","127.0.0.1:8000")
	if err != nil {
		log.Fatal("net Dial() error is ", err)
	}
	defer func() { _ = conn.Close() }()
	/**
	type Q struct {
		X, Y int
		Name string
	}
	 */
	var q Q
	dec := gob.NewDecoder(conn)
	if err = dec.Decode(&q); err != nil {
		log.Fatal("enc Encode() error is ", err)
	}
	fmt.Println(q)
}

Output:

{1 2 name}

3. Decode Decoding

Finally, the steps to decode it are:

dec := gob.NewDecoder(&network) // Will read from network.
if err = dec.Decode(&q);err != nil {
	log.Fatal("decode error 2:", err)
}

sequence The -json

json package implements the JSON encoding and decoding defined in RFC 7159. The mapping between JSON and Go values ​​is described in the documentation for the Marshal and Unmarshal functions.

For an introduction to this package, see "JSON and Go": https://www.php.cn/link/241200d15bc67211b50bd10815259e58json/

The example is as follows:

type Message struct {
	QQ      string
	Address string
}

type Student struct {
	Id   uint64 `json:"id"` //可以保证json字段按照规定的字段转义，而不是输出 Id
	Age  uint64 `json:"age"`
	Data []Message
}

func main() {
	m1 := Message{QQ: "123", Address: "beijing"}
	m2 := Message{QQ: "456", Address: "beijing"}
	s1 := Student{3, 19, append([]Message{}, m1, m2)}
	var buf []byte
	var err error

	if buf, err = json.Marshal(s1); err != nil {
		log.Fatal("json marshal error:", err)
	}

	fmt.Println(string(buf))

	var s2 Student
	if err = json.Unmarshal(buf, &s2); err != nil {
		log.Fatal("json unmarshal error:", err)
	}
	fmt.Println(s2)
}
//输出：
//{"id":3,"age":19,"Data":[{"QQ":"123","Address":"beijing"},{"QQ":"456","Address":"beijing"}]}
//{3 19 [{123 beijing} {456 beijing}]}

Note

During the serialization process, if the members in the structure are in lowercase, an error will occur. The above two methods will produce such results

Let’s take json serialization as an example to see what the result will be if it is lowercase:

package main

import (
	"encoding/json"
	"fmt"
	"log"
)

type Message struct {
	qq      string
	address string
}

type Student struct {
	Id   uint64 `json:"id"` //可以保证json字段按照规定的字段转义，而不是输出 Id
	Age  uint64 `json:"age"`
	Data []Message
}

func main() {
	m1 := Message{"123", "beijing"}
	m2 := Message{"456", "beijing"}
	s1 := Student{3, 19, append([]Message{}, m1, m2)}
	var buf []byte
	var err error

	if buf, err = json.Marshal(s1); err != nil {
		log.Fatal("json marshal error:", err)
	}

	fmt.Println(string(buf))

	var s2 Student
	if err = json.Unmarshal(buf, &s2); err != nil {
		log.Fatal("json unmarshal error:", err)
	}
	fmt.Println(s2)
}

Output:

{"id":3,"age":19,"Data":[{},{}]}
{3 19 [{ } { }]}

We see that the lowercase part will not be serialized, that is, it will be a null value.

Although this will not report an error, it is obviously not the result we want to see.

Error reporting: gob: type xxx has no exported fields

Let’s look at an example of an error reporting:

type Message struct {
	qq      string
	address string
}

type Student struct {
	Id   uint64 `json:"id"` //可以保证json字段按照规定的字段转义，而不是输出 Id
	Age  uint64 `json:"age"`
	Data []Message
}

func main() {
	m1 := Message{"123", "beijing"}
	m2 := Message{"456", "beijing"}
	s1 := Student{3, 19, append([]Message{}, m1, m2)}

	var buf bytes.Buffer
	enc := gob.NewEncoder(&buf)
	if err := enc.Encode(s1); err != nil {
		log.Fatal("encode error:", err) //报错
	}
	fmt.Println(string(buf.Bytes()))
}

This code will report an error:

2020/12/30 16:44:47 encode error:gob: type main.Message has no exported fields

reminds us that the structure is case-sensitive! ! !

序列化方式–Binary

Binary 包实现 数字 和 字节 序列之间的简单转换以及varint的编码和解码。

通过读取和写入固定大小的值来转换数字。 固定大小的值可以是固定大小的算术类型（bool，int8，uint8，int16，float32，complex64等），也可以是仅包含固定大小值的数组或结构体。详情可参考：https://www.php.cn/link/241200d15bc67211b50bd10815259e58binary/#Write

示例：

package main

import (
	"bytes"
	"encoding/binary"
	"fmt"
)

func main() {
	buf := new(bytes.Buffer)
	var pi int64 = 255

	err := binary.Write(buf, binary.LittleEndian, pi)
	if err != nil {
		fmt.Println("binary.Write failed:", err)
	}
	fmt.Println( buf.Bytes())
}
//输出：
[255 0 0 0 0 0 0 0]

这里需要注意：如果序列化的类型是 int 类型的话，将会报错：

binary.Write failed: binary.Write: invalid type int

而且，序列化的值是空的。

这是由于，他在前面已经解释清楚了，只能序列化固定大小的类型（bool，int8，uint8，int16，float32，complex64…），或者是结构体和固定大小的数组。

其他序列化方法

当然，go语言还有其他的序列化方法，如 protobuf 协议，参考：https://geektutu.com/post/quick-go-protobuf.html

【相关推荐：Go视频教程、编程教学】

The above is the detailed content of What are the serialization methods in golang?. For more information, please follow other related articles on the PHP Chinese website!