Table of Contents
How to use ChatGPT to answer college entrance examination questions?
College Entrance Examination Data Set
Experiment and Analysis
Summary
Home Technology peripherals AI When ChatGPT took part in China's college entrance examination and fed it the national A and B papers, it turned out to be seriously biased!

When ChatGPT took part in China's college entrance examination and fed it the national A and B papers, it turned out to be seriously biased!

May 26, 2023 pm 03:37 PM
roll a Partial science

Xi Xiaoyao Technology Talk Original
Author | PythonChatGPT, as an intelligent human-machine conversation application, quickly became popular all over the world after its launch. In just one month, its number of users has exceeded 100 million. People also use ChatGPT to test many exam items, such as SAT, AP, GRE, etc. However, what would it be like if ChatGPT were allowed to participate in China’s college entrance examination? Will he be biased? Can ordinary people like us pass ChatGPT? Let’s take a look at the comments brought to you by students from Fudan University and East China Normal University.

Paper title:
Evaluating the Performance of Large Language Models on GAOKAO Benchmark
Paper link:https://www.php.cn/link/de66dbc30377a1f7908db45298324aa3

How to use ChatGPT to answer college entrance examination questions?

This paper uses a zero-supervised prompt method to convert test questions into ChatGPT input, as shown in the figure below. Different inquiry methods are designed for different subjects and question types. For math questions, convert the formula into latex input.

When ChatGPT took part in Chinas college entrance examination and fed it the national A and B papers, it turned out to be seriously biased!

College Entrance Examination Data Set

The test in this article uses the national A paper and the national B paper from 2010 to 2022, a total of 13 years. Each set of test papers contains 10 subjects Subjects include Chinese, Mathematics, English, Physics, Chemistry, Life, History, and Geography. Mathematics is divided into science mathematics and liberal arts mathematics.

The data set contains a total of 2811 test questions. The specific question types will not be discussed here. I believe readers are very familiar with the college entrance examination questions.

When ChatGPT took part in Chinas college entrance examination and fed it the national A and B papers, it turned out to be seriously biased!

During the evaluation, high school teachers from Caoyang No. 2 Middle School in Shanghai were hired to mark the subjective questions.

Experiment and Analysis

The scores obtained by ChatGPT in the college entrance examination over the past years are shown in the figure below. Since each subject is normalized to 100 points when calculating scores, this score cannot be directly compared with your and my college entrance examination scores. But it can also be seen that this score is not ideal, and it is estimated that neither Fudan nor East China Normal University will be admitted. Why is this?

When ChatGPT took part in Chinas college entrance examination and fed it the national A and B papers, it turned out to be seriously biased!

When ChatGPT took part in Chinas college entrance examination and fed it the national A and B papers, it turned out to be seriously biased!

The above picture shows the performance of ChatGPT in various disciplines, as well as subjective and objective topics. Blue is objective questions and yellow is subjective questions. The analysis found that ChatGPT performed better on objective questions, especially English reading comprehension, multiple choice, and cloze, achieving 88.3%, 78.1%, and 73.8% accuracy respectively. But even for objective questions, the accuracy rate in science and mathematics is less than 40%. Mathematics is really difficult~

ChatGPT’s performance on subjective questions is poor. In the subjects of physics, chemistry, biology and mathematics, the performance of subjective questions is obviously not as good as objective questions. The scores on objective questions in science subjects are also poor. Maybe ChatGPT is biased towards liberal arts subjects? According to the reviewers' comments, the main shortcomings of ChatGPT are: 1. The complex equations in mathematical problems are difficult to solve correctly, and wrong formulas are used in the problem solving process. 2. Inadequate ability to understand and summarize when reading longer materials.

Summary

ChatGPT may not have used Chinese college entrance examination question data during the training process, so its performance is not affected by data leaks and has high credibility.

Observation results show that compared with foreign exams, ChatGPT’s performance on Chinese college entrance examination questions is slightly inferior. Therefore, domestic students do not need to worry too much about not being able to pass the ChatGPT exam for the time being. However, the long text summarization capabilities mentioned in the article have been significantly improved in GPT4-32K, and the domestic large model has also been further optimized on Chinese data. Therefore, we can expect more impressive performance on large-model college entrance examination questions in the future.

In addition, the idea of ​​using ChatGPT to solve college entrance examination questions may be able to answer the debate among netizens about which province’s test questions are more difficult?

The above is the detailed content of When ChatGPT took part in China's college entrance examination and fed it the national A and B papers, it turned out to be seriously biased!. For more information, please follow other related articles on the PHP Chinese website!

Statement of this Website
The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn

Hot AI Tools

Undresser.AI Undress

Undresser.AI Undress

AI-powered app for creating realistic nude photos

AI Clothes Remover

AI Clothes Remover

Online AI tool for removing clothes from photos.

Undress AI Tool

Undress AI Tool

Undress images for free

Clothoff.io

Clothoff.io

AI clothes remover

Video Face Swap

Video Face Swap

Swap faces in any video effortlessly with our completely free AI face swap tool!

Hot Tools

Notepad++7.3.1

Notepad++7.3.1

Easy-to-use and free code editor

SublimeText3 Chinese version

SublimeText3 Chinese version

Chinese version, very easy to use

Zend Studio 13.0.1

Zend Studio 13.0.1

Powerful PHP integrated development environment

Dreamweaver CS6

Dreamweaver CS6

Visual web development tools

SublimeText3 Mac version

SublimeText3 Mac version

God-level code editing software (SublimeText3)

Hot Topics

Java Tutorial
1664
14
PHP Tutorial
1266
29
C# Tutorial
1239
24
Getting Started With Meta Llama 3.2 - Analytics Vidhya Getting Started With Meta Llama 3.2 - Analytics Vidhya Apr 11, 2025 pm 12:04 PM

Meta's Llama 3.2: A Leap Forward in Multimodal and Mobile AI Meta recently unveiled Llama 3.2, a significant advancement in AI featuring powerful vision capabilities and lightweight text models optimized for mobile devices. Building on the success o

10 Generative AI Coding Extensions in VS Code You Must Explore 10 Generative AI Coding Extensions in VS Code You Must Explore Apr 13, 2025 am 01:14 AM

Hey there, Coding ninja! What coding-related tasks do you have planned for the day? Before you dive further into this blog, I want you to think about all your coding-related woes—better list those down. Done? – Let&#8217

AV Bytes: Meta's Llama 3.2, Google's Gemini 1.5, and More AV Bytes: Meta's Llama 3.2, Google's Gemini 1.5, and More Apr 11, 2025 pm 12:01 PM

This week's AI landscape: A whirlwind of advancements, ethical considerations, and regulatory debates. Major players like OpenAI, Google, Meta, and Microsoft have unleashed a torrent of updates, from groundbreaking new models to crucial shifts in le

Selling AI Strategy To Employees: Shopify CEO's Manifesto Selling AI Strategy To Employees: Shopify CEO's Manifesto Apr 10, 2025 am 11:19 AM

Shopify CEO Tobi Lütke's recent memo boldly declares AI proficiency a fundamental expectation for every employee, marking a significant cultural shift within the company. This isn't a fleeting trend; it's a new operational paradigm integrated into p

GPT-4o vs OpenAI o1: Is the New OpenAI Model Worth the Hype? GPT-4o vs OpenAI o1: Is the New OpenAI Model Worth the Hype? Apr 13, 2025 am 10:18 AM

Introduction OpenAI has released its new model based on the much-anticipated “strawberry” architecture. This innovative model, known as o1, enhances reasoning capabilities, allowing it to think through problems mor

A Comprehensive Guide to Vision Language Models (VLMs) A Comprehensive Guide to Vision Language Models (VLMs) Apr 12, 2025 am 11:58 AM

Introduction Imagine walking through an art gallery, surrounded by vivid paintings and sculptures. Now, what if you could ask each piece a question and get a meaningful answer? You might ask, “What story are you telling?

Newest Annual Compilation Of The Best Prompt Engineering Techniques Newest Annual Compilation Of The Best Prompt Engineering Techniques Apr 10, 2025 am 11:22 AM

For those of you who might be new to my column, I broadly explore the latest advances in AI across the board, including topics such as embodied AI, AI reasoning, high-tech breakthroughs in AI, prompt engineering, training of AI, fielding of AI, AI re

How to Add a Column in SQL? - Analytics Vidhya How to Add a Column in SQL? - Analytics Vidhya Apr 17, 2025 am 11:43 AM

SQL's ALTER TABLE Statement: Dynamically Adding Columns to Your Database In data management, SQL's adaptability is crucial. Need to adjust your database structure on the fly? The ALTER TABLE statement is your solution. This guide details adding colu

See all articles