


Capable of interdisciplinary understanding and multi-scale modeling, MIT LAMM releases fine-tuned large language model MechGPT
Editor | Radish Skin
Over the past few centuries, researchers have been looking for ways to connect different fields of knowledge. With the advent of artificial intelligence, we now have the opportunity to explore relationships across fields (e.g., mechanics and biology) or between different fields (e.g., failure mechanics and art)
To achieve this goal, MIT Researchers at MIT's Laboratory for Atomistic and Molecular Mechanics (LAMM) use fine-tuned large language models (LLM) to obtain a subset of relevant knowledge about multi-scale material failure
The steps of this method are to first use a general LLM to extract question and answer pairs from the original source, and then fine-tune the LLM. A series of computational experiments were conducted using this fine-tuned MechGPT LLM base model to explore its capabilities in knowledge retrieval, various language tasks, hypothesis generation, and connecting knowledge across different domains
Although the model has certain ability to recall knowledge from training, but researchers found that LLM is more meaningful for extracting structural insights through ontology knowledge graphs. These interpretable graph structures provide interpretive insights, frameworks for new research questions, and visual representations of knowledge that can also be used for retrieval-enhanced generation.
The research is titled "MechGPT, a Language-Based Strategy for Mechanics and Materials Modeling That Connects Knowledge Across Scales, Disciplines and Modalities" and was published on October 19, 2023. "Applied Mechanics Reviews".
Modeling physical, biological, and metaphysical concepts has been a focus of researchers in many disciplines. Early scientists and engineers were often deeply rooted in fields ranging from science to philosophy, physics to mathematics, and the arts (e.g., Galileo Galilei, Leonardo da Vinci, Johann Wolfgang von · Goethe), but with the development of science, specialization has become dominant today. Part of the reason is that a large amount of knowledge has been accumulated across fields, which requires humans to spend a lot of energy in research and practice.
Nowadays, the emergence of large language models (LLM) has challenged the paradigm of scientific research. It not only brings new modeling strategies based on artificial intelligence/machine learning, but also provides opportunities to connect knowledge, ideas, and concepts across domains. These models can complement traditional multiscale modeling for the analysis and design of layered materials and many other applications in mechanics
Figure: Schematic workflow. (Source: Paper)
Here, LAMM researchers build on the recently proposed use of LLM in mechanics and materials research and development, and a general LLM based on Llama-2 based OpenOrca-Platypus2-13B , developed a fine-tuned MechGPT model that focuses on modeling material failure, multi-scale modeling, and related disciplines.
The OpenOrca-Platypus2-13B model was chosen because it performs well on key tasks such as reasoning, logic, math/science, and other disciplines, providing rich, applicable Subject knowledge and general concepts, and efficient computing capabilities
LLM has powerful applications in the scientific field. In addition to being able to analyze large amounts of data and complex systems, in the fields of mechanics and materials science, LLM is used to simulate and predict the behavior of materials under different conditions, such as mechanical stress, temperature, and chemical interactions. As shown in earlier work, by training LLMs on large data sets from molecular dynamics simulations, researchers can develop models capable of predicting material behavior in new situations, thus accelerating the discovery process and reducing the need for experimental testing.
Such models are also very effective for analyzing scientific texts such as books and publications, allowing researchers to quickly extract key information and insights from large amounts of data. This can help scientists identify trends, patterns, and relationships between different concepts and ideas, and generate new hypotheses and ideas for further research.
Please take a look at the figure below, which is an overview of the autoregressive decoder transformer architecture used to build MechGPT. (Source: paper)
Here, the team focuses on the development of the latter and explores the use of MechGPT, a generative artificial intelligence tool from the Transformer-based LLM family, specifically targeting material failure and related multi-scale methods Training was conducted to evaluate the potential of these strategies.
The strategy proposed in this study includes several steps. First is the distillation step, where researchers use LLM to generate question-answer pairs from text extracted from raw data chunks (such as one or more PDF files). Next, use this data to fine-tune the model in the second step. This study also specifically trained the initial MechGPT model, demonstrating its usefulness in knowledge retrieval, general language tasks, and hypothesis generation in the field of atomistic modeling of material failure
Figure: Overview of the modeling strategy used. (Source: Paper)
This paper introduces an overarching modeling strategy in which researchers employ specific language modeling strategies to generate datasets to extract knowledge from sources and leverage novel mechanics and Material dataset to train the model. The researchers analyzed and discussed three versions of MechGPT, with parameter sizes ranging from 13 billion to 70 billion and context lengths exceeding 10,000 tokens. Following the general remarks, the researchers applied the model and tested its performance in a variety of settings, including using LLMs for ontology graph generation and developing insights on complex topics across disciplines, and agent modeling, where multiple LLMs work collaboratively or against each other. Interact in a way that generates deeper insights into a topic area or answer to a question.
Graph: Developing ontological knowledge graph representations to relate hyperelasticity in the context of supersonic fracture and protein unfolding mechanisms. (Source: paper)
At the same time, the team further provides a conceptual comparison between language models and multi-particle systems at different levels of abstraction, and explains how the new framework can be viewed as extracting universal relationships that govern complex systems. s method.
Rewritten content: The above image shows the conceptual analogy between LLM and multi-particle simulation. (Source: paper)
Overall, the work presented in this study contributes to the development of more powerful and general artificial intelligence models that can help advance scientific research and solve complex problems in specific application areas , allowing for an in-depth evaluation of the model’s performance. Like all models, they must be carefully validated, and their usefulness lies in the context of the questions asked, their strengths and weaknesses, and the broader tools that help scientists advance science and engineering.
Moreover, artificial intelligence tools, as tools of scientific inquiry, must be viewed as a collection of tools for understanding, modeling, and designing the world around us. As artificial intelligence tools develop rapidly, their application in scientific contexts is only beginning to bring new opportunities
Paper link:
https://arxiv.org/ftp/arxiv/papers/ 2310/2310.10445.pdfRelated reports:
https://twitter.com/llama_index/status/1723379654550245719The above is the detailed content of Capable of interdisciplinary understanding and multi-scale modeling, MIT LAMM releases fine-tuned large language model MechGPT. For more information, please follow other related articles on the PHP Chinese website!

Hot AI Tools

Undresser.AI Undress
AI-powered app for creating realistic nude photos

AI Clothes Remover
Online AI tool for removing clothes from photos.

Undress AI Tool
Undress images for free

Clothoff.io
AI clothes remover

Video Face Swap
Swap faces in any video effortlessly with our completely free AI face swap tool!

Hot Article

Hot Tools

Notepad++7.3.1
Easy-to-use and free code editor

SublimeText3 Chinese version
Chinese version, very easy to use

Zend Studio 13.0.1
Powerful PHP integrated development environment

Dreamweaver CS6
Visual web development tools

SublimeText3 Mac version
God-level code editing software (SublimeText3)

Hot Topics

But maybe he can’t defeat the old man in the park? The Paris Olympic Games are in full swing, and table tennis has attracted much attention. At the same time, robots have also made new breakthroughs in playing table tennis. Just now, DeepMind proposed the first learning robot agent that can reach the level of human amateur players in competitive table tennis. Paper address: https://arxiv.org/pdf/2408.03906 How good is the DeepMind robot at playing table tennis? Probably on par with human amateur players: both forehand and backhand: the opponent uses a variety of playing styles, and the robot can also withstand: receiving serves with different spins: However, the intensity of the game does not seem to be as intense as the old man in the park. For robots, table tennis

On August 21, the 2024 World Robot Conference was grandly held in Beijing. SenseTime's home robot brand "Yuanluobot SenseRobot" has unveiled its entire family of products, and recently released the Yuanluobot AI chess-playing robot - Chess Professional Edition (hereinafter referred to as "Yuanluobot SenseRobot"), becoming the world's first A chess robot for the home. As the third chess-playing robot product of Yuanluobo, the new Guoxiang robot has undergone a large number of special technical upgrades and innovations in AI and engineering machinery. For the first time, it has realized the ability to pick up three-dimensional chess pieces through mechanical claws on a home robot, and perform human-machine Functions such as chess playing, everyone playing chess, notation review, etc.

The start of school is about to begin, and it’s not just the students who are about to start the new semester who should take care of themselves, but also the large AI models. Some time ago, Reddit was filled with netizens complaining that Claude was getting lazy. "Its level has dropped a lot, it often pauses, and even the output becomes very short. In the first week of release, it could translate a full 4-page document at once, but now it can't even output half a page!" https:// www.reddit.com/r/ClaudeAI/comments/1by8rw8/something_just_feels_wrong_with_claude_in_the/ in a post titled "Totally disappointed with Claude", full of

At the World Robot Conference being held in Beijing, the display of humanoid robots has become the absolute focus of the scene. At the Stardust Intelligent booth, the AI robot assistant S1 performed three major performances of dulcimer, martial arts, and calligraphy in one exhibition area, capable of both literary and martial arts. , attracted a large number of professional audiences and media. The elegant playing on the elastic strings allows the S1 to demonstrate fine operation and absolute control with speed, strength and precision. CCTV News conducted a special report on the imitation learning and intelligent control behind "Calligraphy". Company founder Lai Jie explained that behind the silky movements, the hardware side pursues the best force control and the most human-like body indicators (speed, load) etc.), but on the AI side, the real movement data of people is collected, allowing the robot to become stronger when it encounters a strong situation and learn to evolve quickly. And agile

At this ACL conference, contributors have gained a lot. The six-day ACL2024 is being held in Bangkok, Thailand. ACL is the top international conference in the field of computational linguistics and natural language processing. It is organized by the International Association for Computational Linguistics and is held annually. ACL has always ranked first in academic influence in the field of NLP, and it is also a CCF-A recommended conference. This year's ACL conference is the 62nd and has received more than 400 cutting-edge works in the field of NLP. Yesterday afternoon, the conference announced the best paper and other awards. This time, there are 7 Best Paper Awards (two unpublished), 1 Best Theme Paper Award, and 35 Outstanding Paper Awards. The conference also awarded 3 Resource Paper Awards (ResourceAward) and Social Impact Award (

This afternoon, Hongmeng Zhixing officially welcomed new brands and new cars. On August 6, Huawei held the Hongmeng Smart Xingxing S9 and Huawei full-scenario new product launch conference, bringing the panoramic smart flagship sedan Xiangjie S9, the new M7Pro and Huawei novaFlip, MatePad Pro 12.2 inches, the new MatePad Air, Huawei Bisheng With many new all-scenario smart products including the laser printer X1 series, FreeBuds6i, WATCHFIT3 and smart screen S5Pro, from smart travel, smart office to smart wear, Huawei continues to build a full-scenario smart ecosystem to bring consumers a smart experience of the Internet of Everything. Hongmeng Zhixing: In-depth empowerment to promote the upgrading of the smart car industry Huawei joins hands with Chinese automotive industry partners to provide

Deep integration of vision and robot learning. When two robot hands work together smoothly to fold clothes, pour tea, and pack shoes, coupled with the 1X humanoid robot NEO that has been making headlines recently, you may have a feeling: we seem to be entering the age of robots. In fact, these silky movements are the product of advanced robotic technology + exquisite frame design + multi-modal large models. We know that useful robots often require complex and exquisite interactions with the environment, and the environment can be represented as constraints in the spatial and temporal domains. For example, if you want a robot to pour tea, the robot first needs to grasp the handle of the teapot and keep it upright without spilling the tea, then move it smoothly until the mouth of the pot is aligned with the mouth of the cup, and then tilt the teapot at a certain angle. . this

Conference Introduction With the rapid development of science and technology, artificial intelligence has become an important force in promoting social progress. In this era, we are fortunate to witness and participate in the innovation and application of Distributed Artificial Intelligence (DAI). Distributed artificial intelligence is an important branch of the field of artificial intelligence, which has attracted more and more attention in recent years. Agents based on large language models (LLM) have suddenly emerged. By combining the powerful language understanding and generation capabilities of large models, they have shown great potential in natural language interaction, knowledge reasoning, task planning, etc. AIAgent is taking over the big language model and has become a hot topic in the current AI circle. Au
