MNIST handwritten numerical classification: the accuracy rate after pixel normalization is low. What is the problem?

TensorFlow MNIST classification: Causes and solutions for low accuracy after pixel normalization

When using TensorFlow for MNIST handwritten numerical classification, many developers may encounter a problem: after pixel normalization of the dataset, the accuracy of model training is extremely low. This article will analyze this problem in depth and provide solutions in conjunction with code examples.

The root cause of the problem lies in the misuse of the tf.nn.softmax_cross_entropy_with_logits function. In the original code, the predicted value y_pred uses the tf.nn.softmax function to calculate the softmax probability:

 y_pred = tf.nn.softmax(tf.matmul(x, w) b)

However, tf.nn.softmax_cross_entropy_with_logits function expects the input of the linear output (logits), not the softmax probability. Pass y_pred that has been converted into softmax to calculate the loss, resulting in errors in the calculation of the loss function, which in turn affects the model training effect.

Solution:

The key is to modify the calculation method of y_pred and remove tf.nn.softmax function:

 y_pred = tf.matmul(x, w) b

At the same time, when calculating the accuracy, the tf.nn.softmax function needs to be applied to y_pred to obtain the probability distribution for comparison with the real tag:

 correct_prediction = tf.equal(tf.argmax(y, 1), tf.argmax(tf.nn.softmax(y_pred), 1))

The modified code snippet is as follows (assuming some code already exists):

 # ... (The code part of importing packages and setting hyperparameters remains unchanged)

# Download the dataset mnist = input_data.read_data_sets('original_data/', one_hot=True)

train_img = mnist.train.images
train_label = mnist.train.labels
test_img = mnist.test.images
test_label = mnist.test.labels
train_img /= 255.0
test_img /= 255.0

X = tf.compat.v1.placeholder(tf.float32, shape=[None, inputSize])
y = tf.compat.v1.placeholder(tf.float32, shape=[None, numClasses])
W = tf.Variable(tf.random_normal([inputSize, numClasses], stddev=0.1))
B = tf.Variable(tf.constant(0.1), [numClasses])
y_pred = tf.matmul(X, W) B # Modify: Remove softmax

loss = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(labels=y, logits=y_pred)) 0.01 * tf.nn.l2_loss(W)
opt = tf.train.GradientDescentOptimizer(learning_rate).minimize(loss)

correct_prediction = tf.equal(tf.argmax(y, 1), tf.argmax(tf.nn.softmax(y_pred), 1)) # Modify: Apply softmax when calculating accuracy
accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))

saver = tf.train.Saver()
multiclass_parameters = {}

# ... (The running code part remains unchanged)

Through the above adjustments, tf.nn.softmax_cross_entropy_with_logits function will receive the correct linear output and calculate the correct loss value, so that the model can be effectively trained and obtained higher accuracy. This once again emphasizes the importance of correctly understanding and using TensorFlow functions for building efficient deep learning models.

The above is the detailed content of MNIST handwritten numerical classification: the accuracy rate after pixel normalization is low. What is the problem?. For more information, please follow other related articles on the PHP Chinese website!