Backend Development
Python Tutorial
A brief discussion on saving and restoring loading of Tensorflow models
A brief discussion on saving and restoring loading of Tensorflow models
This article mainly introduces the saving and restoring of Tensorflow model. Now I share it with you and give it as a reference. Let’s come and take a look
Recently we have done some anti-spam work. In addition to using commonly used rule matching and filtering methods, we also used some machine learning methods for classification prediction. We use TensorFlow to train the model. The trained model needs to be saved. In the prediction phase, we need to load and restore the model for use, which involves saving and restoring the TensorFlow model.
Summarize the commonly used model saving methods of Tensorflow.
Save checkpoint model file (.ckpt)
First of all, TensorFlow provides a very convenient API, tf.train.Saver() to save and restore a machine learning model.
Model Saving
It is very convenient to use tf.train.Saver() to save model files. Here is a simple example:
import tensorflow as tf
import os
def save_model_ckpt(ckpt_file_path):
x = tf.placeholder(tf.int32, name='x')
y = tf.placeholder(tf.int32, name='y')
b = tf.Variable(1, name='b')
xy = tf.multiply(x, y)
op = tf.add(xy, b, name='op_to_store')
sess = tf.Session()
sess.run(tf.global_variables_initializer())
path = os.path.dirname(os.path.abspath(ckpt_file_path))
if os.path.isdir(path) is False:
os.makedirs(path)
tf.train.Saver().save(sess, ckpt_file_path)
# test
feed_dict = {x: 2, y: 3}
print(sess.run(op, feed_dict))The program generates and saves four files (before version 0.11, only three files were generated: checkpoint, model.ckpt , model.ckpt.meta)
checkpoint text file, recording the path information list of the model file
model.ckpt.data-00000 -of-00001 Network weight information
model.ckpt.index The two files .data and .index are binary files that save the variable parameter (weight) information in the model
model.ckpt.meta binary file, which saves the calculation graph structure information of the model (the network structure of the model) protobuf
The above is tf.train The basic usage of .Saver().save(), the save() method also has many configurable parameters:
tf.train.Saver().save(sess, ckpt_file_path, global_step=1000)
Adding the global_step parameter means saving the model after every 1000 iterations. "-1000" will be added after the model file, model.ckpt-1000.index, model.ckpt-1000.meta, model.ckpt.data- 1000-00000-of-00001
Save the model every 1000 iterations, but the structural information file of the model will not change. It will only be saved every 1000 iterations, not every 1000 times. Save once, so when we don’t need to save the meta file, we can add the write_meta_graph=False parameter, as follows:
Copy code Code As follows:
tf.train.Saver().save(sess, ckpt_file_path, global_step=1000, write_meta_graph=False)
If you want to save the model every two hours and only save the latest 4 models, you can add max_to_keep (the default value is 5. If you want to save it every epoch of training, you can It is set to None or 0, but it is useless and not recommended), keep_checkpoint_every_n_hours parameter, as follows:
##Copy code The code is as follows:
tf.train.Saver().save(sess, ckpt_file_path, max_to_keep=4, keep_checkpoint_every_n_hours=2)
tf.train.Saver([x, y]).save(sess, ckpt_file_path)
##ps. During the model training process, the variable or parameter name attribute name that needs to be obtained after saving cannot be lost, otherwise the model cannot be obtained through get_tensor_by_name() after restoration.
For the above model saving example, the process of restoring the model is as follows:
import tensorflow as tf
def restore_model_ckpt(ckpt_file_path):
sess = tf.Session()
saver = tf.train.import_meta_graph('./ckpt/model.ckpt.meta') # 加载模型结构
saver.restore(sess, tf.train.latest_checkpoint('./ckpt')) # 只需要指定目录就可以恢复所有变量信息
# 直接获取保存的变量
print(sess.run('b:0'))
# 获取placeholder变量
input_x = sess.graph.get_tensor_by_name('x:0')
input_y = sess.graph.get_tensor_by_name('y:0')
# 获取需要进行计算的operator
op = sess.graph.get_tensor_by_name('op_to_store:0')
# 加入新的操作
add_on_op = tf.multiply(op, 2)
ret = sess.run(add_on_op, {input_x: 5, input_y: 5})
print(ret)
First restore the model structure, then restore the variable (parameter) information, and finally we can obtain various information in the trained model (saved variables , placeholder variables, operators, etc.), and various new operations can be added to the obtained variables (see the above code comments).
Regarding the saving and restoration of ckpt model files, there is an answer on stackoverflow with a clear explanation, which you can refer to.
At the same time, the tutorial on saving and restoring TensorFlow models on cv-tricks.com is also very good, you can refer to it.
"Tensorflow 1.0 Learning: Model Saving and Restoration (Saver)" has some Saver usage tips.
Save a single model file (.pb)
I have run the demo of Tensorflow's inception-v3 myself and found that a .pb will be generated after the run is completed. Model file, this file is used for subsequent prediction or transfer learning. It is just one file, very cool and very convenient.
The main idea of this process is that the graph_def file does not contain the Variable value in the network (usually the weight is stored), but it does contain the constant value, so if we can convert the Variable to constant ( Using the graph_util.convert_variables_to_constants() function), you can achieve the goal of using one file to store both the network architecture and weights.
ps: Here .pb is the suffix name of the model file. Of course, we can also use other suffixes (use .pb to be consistent with Google ╮(╯▽╰)╭)
Similarly based on the above example, a simple demo:
import tensorflow as tf
import os
from tensorflow.python.framework import graph_util
def save_mode_pb(pb_file_path):
x = tf.placeholder(tf.int32, name='x')
y = tf.placeholder(tf.int32, name='y')
b = tf.Variable(1, name='b')
xy = tf.multiply(x, y)
# 这里的输出需要加上name属性
op = tf.add(xy, b, name='op_to_store')
sess = tf.Session()
sess.run(tf.global_variables_initializer())
path = os.path.dirname(os.path.abspath(pb_file_path))
if os.path.isdir(path) is False:
os.makedirs(path)
# convert_variables_to_constants 需要指定output_node_names，list()，可以多个
constant_graph = graph_util.convert_variables_to_constants(sess, sess.graph_def, ['op_to_store'])
with tf.gfile.FastGFile(pb_file_path, mode='wb') as f:
f.write(constant_graph.SerializeToString())
# test
feed_dict = {x: 2, y: 3}
print(sess.run(op, feed_dict))程序生成并保存一个文件
model.pb 二进制文件，同时保存了模型网络结构和参数（权重）信息
模型加载还原
针对上面的模型保存例子，还原模型的过程如下：
import tensorflow as tf
from tensorflow.python.platform import gfile
def restore_mode_pb(pb_file_path):
sess = tf.Session()
with gfile.FastGFile(pb_file_path, 'rb') as f:
graph_def = tf.GraphDef()
graph_def.ParseFromString(f.read())
sess.graph.as_default()
tf.import_graph_def(graph_def, name='')
print(sess.run('b:0'))
input_x = sess.graph.get_tensor_by_name('x:0')
input_y = sess.graph.get_tensor_by_name('y:0')
op = sess.graph.get_tensor_by_name('op_to_store:0')
ret = sess.run(op, {input_x: 5, input_y: 5})
print(ret)模型的还原过程与checkpoint差不多一样。
《将TensorFlow的网络导出为单个文件》上介绍了TensorFlow保存单个模型文件的方式，大同小异，可以看看。
思考
模型的保存与加载只是TensorFlow中最基础的部分之一，虽然简单但是也必不可少，在实际运用中还需要注意模型何时保存，哪些变量需要保存，如何设计加载实现迁移学习等等问题。
同时TensorFlow的函数和类都在一直变化更新，以后也有可能出现更丰富的模型保存和还原的方法。
选择保存为checkpoint或单个pb文件视业务情况而定，没有特别大的差别。checkpoint保存感觉会更加灵活一些，pb文件更适合线上部署吧（个人看法）。
以上完整代码：github https://github.com/liuyan731/tf_demo
相关推荐：
The above is the detailed content of A brief discussion on saving and restoring loading of Tensorflow models. For more information, please follow other related articles on the PHP Chinese website!
Hot AI Tools
Undresser.AI Undress
AI-powered app for creating realistic nude photos
AI Clothes Remover
Online AI tool for removing clothes from photos.
Undress AI Tool
Undress images for free
Clothoff.io
AI clothes remover
Video Face Swap
Swap faces in any video effortlessly with our completely free AI face swap tool!
Hot Article
Hot Tools
Notepad++7.3.1
Easy-to-use and free code editor
SublimeText3 Chinese version
Chinese version, very easy to use
Zend Studio 13.0.1
Powerful PHP integrated development environment
Dreamweaver CS6
Visual web development tools
SublimeText3 Mac version
God-level code editing software (SublimeText3)
Hot Topics
How to recover expired WeChat files? Can expired WeChat files be recovered?
Feb 22, 2024 pm 02:46 PM
Open WeChat, select Settings in Me, select General and then select Storage Space, select Management in Storage Space, select the conversation in which you want to restore files and select the exclamation mark icon. Tutorial Applicable Model: iPhone13 System: iOS15.3 Version: WeChat 8.0.24 Analysis 1 First open WeChat and click the Settings option on the My page. 2 Then find and click General Options on the settings page. 3Then click Storage Space on the general page. 4 Next, click Manage on the storage space page. 5Finally, select the conversation in which you want to recover files and click the exclamation mark icon on the right. Supplement: WeChat files generally expire in a few days. If the file received by WeChat has not been clicked, the WeChat system will clear it after 72 hours. If the WeChat file has been viewed,
How to recover browsing history in incognito mode
Feb 19, 2024 pm 04:22 PM
Private browsing is a very convenient way to browse and protect your privacy when surfing the Internet on your computer or mobile device. Private browsing mode usually prevents the browser from recording your visit history, saving cookies and cache files, and preventing the website you are browsing from leaving any traces in the browser. However, for some special cases, we may need to restore the browsing history of Incognito Browsing. First of all, we need to make it clear: the purpose of private browsing mode is to protect privacy and prevent others from obtaining the user’s online history from the browser. Therefore, incognito browsing
How to restore chat spark on TikTok
Mar 16, 2024 pm 01:25 PM
On Douyin, a short video platform full of creativity and vitality, we can not only enjoy a variety of exciting content, but also have in-depth communications with like-minded friends. Among them, chat sparks are an important indicator of the intensity of interaction between the two parties, and they often inadvertently ignite the emotional bonds between us and our friends. However, sometimes due to some reasons, the chat spark may be disconnected. So what should we do if we want to restore the chat spark? This tutorial guide will bring you a detailed introduction to the content strategy, hoping to help everyone. How to restore the spark of Douyin chat? 1. Open the Douyin message page and select a friend to chat. 2. Send messages and chat to each other. 3. If you send messages continuously for 3 days, you can get the spark logo. On a 3-day basis, send pictures or videos to each other
How to save pictures without watermark in Xiaohongshu How to save pictures without watermark in Xiaohongshu
Mar 22, 2024 pm 03:40 PM
Xiaohongshu has rich content that everyone can view freely here, so that you can use this software to relieve boredom every day and help yourself. In the process of using this software, you will sometimes see various beautiful things. Many people want to save pictures, but the saved pictures have watermarks, which is very influential. Everyone wants to know how to save pictures without watermarks here. The editor provides you with a method for those in need. Everyone can understand and use it immediately! 1. Click the "..." in the upper right corner of the picture to copy the link 2. Open the WeChat applet 3. Search the sweet potato library in the WeChat applet 4. Enter the sweet potato library and confirm to get the link 5. Get the picture and save it to the mobile phone album
How to restore Xiaomi Cloud photo album to local
Feb 24, 2024 pm 03:28 PM
How to restore Xiaomi Cloud Photo Album to local? You can restore Xiaomi Cloud Photo Album to local in Xiaomi Cloud Photo Album APP, but most friends don’t know how to restore Xiaomi Cloud Photo Album to local. The next step is to restore Xiaomi Cloud Photo Album to local. Local method graphic tutorials, interested users come and take a look! How to restore Xiaomi cloud photo album to local 1. First open the settings function in Xiaomi phone and select [Personal Avatar] on the main interface; 2. Then enter the Xiaomi account interface and click the [Cloud Service] function; 3. Then jump to Xiaomi For the function of cloud service, select [Cloud Backup]; 4. Finally, in the interface as shown below, click [Cloud Album] to restore the album to local.
Tutorial to restore win11 default avatar
Jan 02, 2024 pm 12:43 PM
If we change our system account avatar but don’t want it anymore, we can’t find how to change the default avatar in win11. In fact, we only need to find the folder of the default avatar to restore it. Restore the default avatar in win11 1. First click on the "Windows Logo" on the bottom taskbar 2. Then find and open "Settings" 3. Then enter "Account" on the left column 4. Then click on "Account Information" on the right 5. After opening, click "Browse Files" in the selected photo. 6. Finally, enter the "C:\ProgramData\Microsoft\UserAccountPictures" path to find the system default avatar picture.
How to restore default wallpaper in win10
Feb 10, 2024 pm 10:51 PM
Windows 10's May 2019 Update features a new, brighter default desktop background. It looks great - with the new light theme. If you use Windows 10’s dark theme, you may want a darker background. Strangely, the original Windows 10 desktop background has been removed from the latest version of Windows 10. You have to download it from the web or copy its files from an old Windows 10 PC. Although we were unable to find this wallpaper image on Microsoft's official website, you can download it from other sources. We found a copy of the original Windows 10 desktop wallpaper in 4K resolution on Imgur. Additionally, there are other sizes and more default walls
How to recover cleared browsing history
Feb 18, 2024 pm 10:05 PM
How to restore web page history after it has been cleared Date: June 10, 2022 Introduction: When we use computers or mobile phone browsers daily, we often use the browser's history to find web pages we have visited before. However, sometimes we may accidentally clear our browser history, causing us to be unable to retrieve a specific web page. In this article, I will tell you some ways to recover cleared web history. Method 1: Use the browser recovery function. Most common browsers provide the function of restoring history, such as Google
