Mysql large website technical architecture core case analysis
7. On demand: scalable architecture of the website
Extensibility (Extensibility):refers to the system with minimal impact on the existing system. The ability to continuously expand or improve functionality. It is the opening and closing principle at the system architecture design level. The architecture design takes into account future functional expansion. When new functions are added to the system, there is no need to modify the structure and code of the existing system.
Scalability (Scalability): refers to the system's ability to enhance (reduce) its own computing and processing capabilities by increasing (decreasing) the scale of its own resources.
A. Build a scalable website architecture
1. The greatest value of a software architect does not lie in how many advanced technologies he has mastered, but in his ability to cut a large system into pieces. The ability to divide it into N low-coupling sub-modules. These sub-modules include horizontal business modules and vertical basic technology modules.
2. The core idea is modularization. On this basis, the coupling between modules is reduced and the reusability of modules is improved.
B. Use distributed message queues to reduce system coupling
1. Event-driven architecture
Event-driven architecture ( Event Driven Architecture): By transmitting event messages between low-coupled modules to maintain loose coupling of modules, and completing inter-module cooperation with the help of event message communication, distributed message queues are commonly used.
#The message queue works using the publish-subscribe model. The message sender publishes the message and one or more message receivers subscribe to the message.
2. Distributed message queue
The queue is a first-in-first-out structure, and the application can access the interface through the remote Use distributed message queues to perform message access operations to achieve distributed asynchronous calls.
The message producer application pushes the message to the message queue server through the remote access interface. The message queue server writes the message to the local memory queue and immediately returns a successful response to the message producer. By. The message queue server searches for the message consumer application that subscribes to the message based on the message subscription list, and sends the messages in the message queue to the message consumer program through the remote communication interface according to the first-in-first-out (FIFO) principle.
Distributed message queues can be very complex. For example, they can support ESB (Enterprise Service Bus) and SOA (Service-Oriented Architecture), or they can also be very simple using MySQL records: The message producer program writes messages into the database as data records, and the message consumer program queries the database and sorts them by the record writing timestamp, thus realizing a de facto distributed message queue.
C. Use distributed services to create a reusable business platform
1. Distributed services decompose system coupling through interfaces , different subsystems make service calls through the interface description of Desert Rose.
2. Problems with the Big Mac system: difficulty in compilation and deployment; difficulty in code branch management; exhaustion of database connections; difficulty in adding new services;
3. Solution
Vertical split: split a large application into multiple small applications
Horizontal split: split the reused business, Deployed independently as distributed services, new businesses only need to call these distributed services and do not need to rely on specific module codes
4. Web Service and enterprise-level distributed services
Disadvantages: bloated registration and discovery mechanism; inefficient XML serialization means; relatively high overhead HTTP remote communication; complex deployment and maintenance means;
5. Large website distribution Requirements and characteristics of distributed services
Load balancing, failover, efficient remote communication, integration of heterogeneous systems, minimal intrusion into applications, version management, real-time monitoring
6. Distributed service framework Design: Thrift, Dubbo
D. Extensible data structure
Designed using ColumnFamily (column family) used in NoSQL databases.
E. Use an open platform to build a website ecosystem
1. The open platform is the interface for internal and external interaction of the website, and the external side needs to face the large number of third-party developers Or, we need to face many business services within the website internally.
2. Architecture: API interface, protocol conversion, security, auditing, routing, process
8. Impregnable: Security architecture of the website
A. Website application attack and defense
1.XSS attack
An attack method is to tamper with web pages, inject malicious HTML scripts, and control the user's browser to perform malicious operations when the user browses the web.
One type of attack is the reflection type. The attacker induces the user to click on a link embedded with a malicious script to achieve the purpose of the attack
Another type of attack is a persistent XSS attack. The hacker submits a request containing a malicious script and saves it in the database of the attacked website. When the user browses the web page, the malicious script is included in the normal page, achieving the purpose of the attack. Purpose. It is often used in web applications such as forums and blogs.
Preventative measures include disinfecting and filtering dangerous characters, while prohibiting page JS from accessing Cookies with HttpOnly attributes
2. Injection attack
It is divided into SQL injection and OS injection
- ##SQL injection to obtain the database structure: using open source software programs, error echo, blind injection
- SQL injection prevention: disinfection; parameter binding, use pre-compilation method, bind parameters;
- CSRF (Cross Site Request Forgery), the attacker performs illegal operations as a legitimate user through cross-site requests. The main method is to use cross-site requests to forge requests with the user's identity without the user's knowledge, and use browser cookies or server session policies to steal the user's identity.
- Prevention: form token, verification code, Referer check (check the request source recorded in the Referer field of the HTTP request header)
- Error Code: error echo, HTML comments, file upload, path traversal
6. Website security vulnerability scanning
B. Information encryption technology and key security management1. One-way hash encryption: md5, sha, etc., add salt
2. Symmetric encryption: DES algorithm, RC algorithm, etc., use the same key for encryption
3. Asymmetric encryption: RSA Algorithm
4. Key security management
- By placing the key and algorithm in an independent server or dedicated hardware device, and realizing data encryption and decryption through service calls .
1. Text matching: solving the problem of sensitive word filtering
- Just use regular replacement for a small amount of content
3. Blacklist: Hash table, Bloom filter
D. E-commerce risk control1. Risks: account risk, buyer risk, seller risk, transaction risk
2.Risk control
- The machine automatically identifies high-risk transactions and information and sends it to risk control auditors for manual review. The technology and methods of machine risk control are constantly being gradually improved through new risk types discovered manually.
1.LAMP->JAVA/ORACLE->MySQL/ NoSQL
2. Business drives the continuous progress of technology
10. Wikipedia’s high-performance architecture design analysisA.Wikipedia website as a whole Architecture:
LAMP open source products, GeoDNS, LVS, Squid, Lighttpd, PHP, Memcached, Lucene, MySQL
B.Wikipedia performance optimization strategy1. Front-end performance optimization
- The core of the front-end architecture is the reverse proxy server Squid cluster, which is load balanced by LVS and returned through CDN before the reverse proxy.
3. Back-end performance optimization:
Cache
- Data in particularly concentrated hot spots is cached directly into the local memory of the application server
- Use larger server memory
11. High-availability architecture design analysis of Doris, a massive distributed storage system
For a data storage system, high availability means: high availability Available services, highly reliable data
A. High-availability architecture of distributed storage system
1. Redundancy: server hot backup, multiple data storage
2. Overall system division:
Application server: The client of the storage system initiates a data operation request to the system
-
Data storage server: The core of the storage system, stores data and responds to data operation requests from the application server
Management center server: Main-master composed of two machines A hot backup small-scale server cluster is responsible for cluster management, health heartbeat detection of data storage clusters; cluster expansion and fault recovery management; providing cluster address configuration information services to application servers, etc.
B. High availability solutions under different fault conditions
1. Fault classification of distributed storage systems: instantaneous fault, temporary fault, permanent fault
2. Transient fault resolution: multiple retries
3. Temporary fault resolution: manual intervention is required, the problematic server uses a temporary storage server
4. Permanent fault resolution: enable backup server replacement Permanently invalid server
12. Online shopping flash sale system architecture design case analysis
A. Technical challenges of flash sale activities:To the existing website business Impact, high concurrency applications, database load, sudden increase in network and server bandwidth, direct order placement
B. Response strategy for the flash sale system
Independent deployment of flash sale system
Static flash sale product page
Rent network bandwidth for flash sale activities
Dynamicly generate random order page URL
C. Flash sale system architecture design
1. How to control the lighting of the purchase button on the flash sale product page: use a JS file, modify the content at the beginning, request it every time, not be cached by CDN, etc., and use a random version number.
2. How to only allow the first submitted order to be sent to the order subsystem: control the entrance to the order page, so that only a few users can enter, and other users directly enter the flash sale end page. For example, there are 10 servers, each processing 10 requests. When the number of requests exceeds 10, the others will return errors, and then request the global cache record. If it is the first one, it will enter the order page, and the others will return failures.
13. Analysis of typical failure cases of large websites
A. Writing logs can also cause failures
-
The application's own log output configuration and the log output of third-party components must be configured separately
Check the log configuration file, let's play with the log. Michelle considers at least Warn
Need to turn off too many Error logs that some third-party components may output
B. High Faults caused by concurrent access to the database
The homepage should not access the database
The homepage should be static
C. Faults caused by locks under high concurrency conditions
Be careful when using lock operations
D. Cache Faults caused
The cache server is already an integral part of the website architecture and needs to be managed at the same level as the database
E. Faults caused by out-of-sync application startup
F. Faults caused by exclusive disk reading and writing of large files
Do not share storage for small files and large files
G .Faults caused by abusing the production environment
Be extra careful when accessing the production environment, and please have a dedicated DBA maintain the database
H.Faults caused by non-standard processes
Use the diff command to compare the code before submitting it to confirm that no code that should not be submitted is submitted; strengthen code review, have at least one other engineer conduct a code review before submission and share the responsibility for failures caused by the code
I. Failures caused by bad programming habits
Pay attention to the handling of empty objects, null values, etc.
14. Architect The art of leadership
A. Focus on people rather than products
1. A group of excellent people doing something they love will definitely achieve success
2. The best software management is to explore the outstanding potential of each member of the project team
3. Find a goal worth striving for together and create a job where everyone can maximize their self-worth. Atmosphere
B. Discover the excellence of people
1. Things make people, not people make things
2.Most people , including ourselves, we are all better than we think. Some excellence needs to be stimulated in the right environment, such as doing something challenging, cooperating with better people, or having the courage to surpass ourselves
3. Discovering the excellence of people is far more meaningful than discovering outstanding people
C. Sharing a beautiful blueprint
1. The blueprint should be clearly stated of: what the product should do, what it should not do, and what business goals it should achieve
2. The blueprint should be visual: what value can the product create for users, what market goals can it achieve, and what will the product look like in the end?
3. The blueprint should be simple: in one sentence Understand: What are we doing?
4. Architects should maintain focus on the target blueprint and be alert to any designs and decisions that deviate from the blueprint. Wrong deviations must be corrected in a timely manner. Necessary changes must be discussed by everyone, and Need to regain everyone's approval.
D. Jointly participate in the architecture
1. Don’t be the only architect who owns the architecture
2. Let others maintain the framework and architecture documentation
E. Learn to compromise
Opinions against architectural and technical solutions are essentially paying attention to, trying to understand and accept these solutions. Architects should not be too sensitive and should share their opinions frankly and seek common ground while reserving differences
2. Arguments about technical details should be verified immediately instead of continuing to discuss
3. When everyone is not discussing the architecture, make it clear Architecture has been integrated into projects, systems and developers. The sooner the architect is forgotten, the more successful the architecture is.
F. Achieve others
1. Our work Not only to produce products, but also to achieve people, and ultimately ourselves
2. To complete a project, we must not only create value for customers and make profits for the company, but also allow project members to grow
3. As the technical leader of the team, the architect should not try to control anything during the project process. Advance with a flexible plan and blueprint, and the team will take care of themselves
15. Website Architect Career Guide
#The purpose of developing software is to solve real-world problems, but many times people don’t know what the real problems are.
Many problems will also be encountered in the software development process. It is necessary to coordinate the interests of all parties to obtain the greatest possible support. It is necessary to balance customer needs, software output, and development. The relationship between resources requires many things to be done to realize the original blueprint of software design.
A. Discover problems and find breakthroughs
When expectations cannot be met, people will feel that something is wrong, because there is a problem It is the gap between experience and expectation. There are two ways to eliminate problems: improve the experience or lower expectations. Just lowering expectations will not solve the problem. On the contrary, facing the difference between expectations and actual experience can identify the problem and find a breakthrough.
2. The first thing new employees need to do is to integrate into the team
3. The last thing new employees need to do is to prove their abilities.
B. Ask questions and seek support
1. The problem is found, it is only the problem of the problem finder, not the problem owner, if you want To solve a problem, you must raise the issue and let the owner of the problem know that the problem exists.
2. Tips for raising questions:
Represent "my problem" as "our problem"
Ask closed-ended questions to your boss (give AB options for the boss to choose which one is better), and ask open-ended questions to your subordinates
Point out the problem instead of criticizing the person
Ask questions in an agreeable way
#3. The so-called outspokenness means that the intention you want to express must be stated directly. Understand, don’t talk in circles, but be careful in the way of expression, and take into account the feelings of the parties involved
C. Solve the problem and achieve performance
1. Solve my problem Before you solve the problem, solve your problem first
If you help others solve their problem, others will also help you solve their problem
In In the process of helping others solve problems, I became familiar with the situation
You use your solution to solve other people's problems, and this solution is under your control
2. Proper escape from problems
16. Manhua website architect
A. Divided by function Architect
Design architect, fire-fighting architect, evangelist architect, Geek architect
B.Divide architects by effect
Sherpa architect: usually develops the most technically difficult and challenging modules in the project, Spartan architect, dignitary architect
C. Press Division of Responsibilities and Roles Architect
Product Architect: Participate in the entire life cycle of the product, basic service architect (platform architect), infrastructure architect
D .Divide architects by level of concern
Architects who only focus on functions, architects who focus on non-functions, architects who focus on team organization and management, architects who focus on product operations, and architects who focus on the future of the product Architect
E. Divide architects by word of mouth
The best architect, a good architect, an average architect, a poor architect, and the worst Architects
F. Non-mainstream ways to divide architects
Ordinary architects, literary architects, 1 1 architects
Appendix A: Overview of large-scale website technologies
A. Front-end architecture
Browser optimization technology, CDN, static and dynamic separation, independent deployment of static resources, image service, Reflection proxy, DNS
B. Application layer architecture
Development framework, page rendering, load balancing, Session management, dynamic page staticization, business splitting, virtualized server
C. Service layer architecture
Distributed messaging, distributed services, distributed cache, distributed configuration
D. Storage layer architecture
Distributed files, relational databases, NoSQL databases, data synchronization
E.Backend architecture
Search engine, data warehouse, recommendation system
F.Data collection (log) and monitoring
Browser data collection, server business collection, server performance data collection, system monitoring, system alarm
G. Security architecture
Web attack , Data protection
H. Data center computer room architecture
Computer room, cabinet, server
The above is the detailed content of Mysql large website technical architecture core case analysis. For more information, please follow other related articles on the PHP Chinese website!

Hot AI Tools

Undresser.AI Undress
AI-powered app for creating realistic nude photos

AI Clothes Remover
Online AI tool for removing clothes from photos.

Undress AI Tool
Undress images for free

Clothoff.io
AI clothes remover

Video Face Swap
Swap faces in any video effortlessly with our completely free AI face swap tool!

Hot Article

Hot Tools

Notepad++7.3.1
Easy-to-use and free code editor

SublimeText3 Chinese version
Chinese version, very easy to use

Zend Studio 13.0.1
Powerful PHP integrated development environment

Dreamweaver CS6
Visual web development tools

SublimeText3 Mac version
God-level code editing software (SublimeText3)

Hot Topics











The main role of MySQL in web applications is to store and manage data. 1.MySQL efficiently processes user information, product catalogs, transaction records and other data. 2. Through SQL query, developers can extract information from the database to generate dynamic content. 3.MySQL works based on the client-server model to ensure acceptable query speed.

Laravel is a PHP framework for easy building of web applications. It provides a range of powerful features including: Installation: Install the Laravel CLI globally with Composer and create applications in the project directory. Routing: Define the relationship between the URL and the handler in routes/web.php. View: Create a view in resources/views to render the application's interface. Database Integration: Provides out-of-the-box integration with databases such as MySQL and uses migration to create and modify tables. Model and Controller: The model represents the database entity and the controller processes HTTP requests.

MySQL and phpMyAdmin are powerful database management tools. 1) MySQL is used to create databases and tables, and to execute DML and SQL queries. 2) phpMyAdmin provides an intuitive interface for database management, table structure management, data operations and user permission management.

Compared with other programming languages, MySQL is mainly used to store and manage data, while other languages such as Python, Java, and C are used for logical processing and application development. MySQL is known for its high performance, scalability and cross-platform support, suitable for data management needs, while other languages have advantages in their respective fields such as data analytics, enterprise applications, and system programming.

I encountered a tricky problem when developing a small application: the need to quickly integrate a lightweight database operation library. After trying multiple libraries, I found that they either have too much functionality or are not very compatible. Eventually, I found minii/db, a simplified version based on Yii2 that solved my problem perfectly.

Article summary: This article provides detailed step-by-step instructions to guide readers on how to easily install the Laravel framework. Laravel is a powerful PHP framework that speeds up the development process of web applications. This tutorial covers the installation process from system requirements to configuring databases and setting up routing. By following these steps, readers can quickly and efficiently lay a solid foundation for their Laravel project.

The basic operations of MySQL include creating databases, tables, and using SQL to perform CRUD operations on data. 1. Create a database: CREATEDATABASEmy_first_db; 2. Create a table: CREATETABLEbooks(idINTAUTO_INCREMENTPRIMARYKEY, titleVARCHAR(100)NOTNULL, authorVARCHAR(100)NOTNULL, published_yearINT); 3. Insert data: INSERTINTObooks(title, author, published_year)VA

When developing an e-commerce website using Thelia, I encountered a tricky problem: MySQL mode is not set properly, causing some features to not function properly. After some exploration, I found a module called TheliaMySQLModesChecker, which is able to automatically fix the MySQL pattern required by Thelia, completely solving my troubles.
