news

한어Русский языкEnglishFrançaisIndonesianSanskrit日本語DeutschPortuguêsΕλληνικάespañolItalianoSuomalainenLatina

text/observer.com columnist fu jiewen high-speed emu operation and maintenance engineer

during the national day holiday, have you grabbed a ticket on 12306?

although complaints such as "it's so hard to buy high-speed rail tickets" and "why do you have to wait for a ticket when it's issued?" are common during holidays, the china railway customer service center 12306 platform has been unknowingly gone since it went online on june 12, 2011. after 13 years, it has become the preferred ticket purchasing platform for most passengers traveling by high-speed rail.

looking back at the development history of 12306, it can be said that it is an "epic" that involves a battle of wits and courage with ticket-grabbing software.

on national day 2024, railways usher in peak passenger traffic icphoto

everything has to start from the time when 12306 was established. with the sale of the first beijing-tianjin intercity railway e-ticket on june 12, 2011, china officially kicked off the ticket sales on the railway network. in just a few months, the scope of online ticket sales has rapidly expanded to beijing, shanghai, wuhan and wuhan. guangzhou, zhengxi and other high-speed railway lines. as of september 30, high-speed rail tickets (prefixed with g and d) nationwide can be purchased on 12306. by the end of 2011, nationwide train tickets (prefixed with z, t, k) can be purchased on 12306. in just half a year, ticket purchasing on china's railway network has made a leap from "from scratch" to "comprehensive popularization"!

but the novelty of buying tickets on the railway network has not faded away, and the spring festival transport of 2012 has quietly arrived. the spring festival transport, a rare mass migration event in the world, brings unprecedented pressure to china's transportation industry every year. it can be said that the “big test” in the transportation industry. and the first system crash of 12306 came beyond everyone’s expectations.

starting from january 5, 2012, the 12306 website has received more than 1 billion hits for 5 consecutive days. the number of visits has increased by more than 10 times compared with the previous month. on january 9, the number of hits exceeded 1.4 billion, making it one of the busiest websites in the world. 1 - you must know that at the end of 2011, there were only 500 million internet users in china.

such access pressure was not taken into consideration at the beginning of the construction of the platform. at the beginning of the construction of the 12306 system, based on the core system architecture of passenger tickets and based on the characteristics of internet applications, the developers designed several relatively independent systems for the system such as caching services, user management, ticket query, order and electronic ticket processing. business partitions, and three-level network security domains, namely the external network, the internal network and the ticketing network. in the stress test before going online, a process includes user login, ticket inquiry, order placement and payment and other business operations. the system's maximum transaction capacity is 34 tickets/second. calculated based on the peak period of 10 hours, the ticket sales volume can reach 1.2 million tickets/day. design capabilities.

however, the stress test missed an "uninvited guest" - ticket-grabbing software. the ticket grabbing software is a browser-based plug-in that uses the machine's quick response to help users complete the actions of constantly querying and refreshing the page. it uses high-frequency query of remaining tickets for trains, automatically fills in personal information, and operates multiple accounts at the same time. complete the ticket purchase process faster than users who purchase tickets manually.

passengers who suffer from "no tickets" on the platform can only spend money to seek help from ticket grabbing software in order to ensure everything is safe.

due to the lack of sufficient estimation of the huge traffic caused by ticket-grabbing software, 12306 was vulnerable to a large number of ticket-grabbing software - the 12306 website was paralyzed, a large number of users were unable to log in, and the page refresh time was as long as 30 minutes. i paid for the ticket but could not purchase the ticket. many ticket buyers helplessly denounced the 12306 website as “in name only”.

however, there was no turning back, so the 12306 ticketing team had no choice but to fight: in response to the problem of insufficient bandwidth, the 12306 ticketing team made a prompt decision and quickly increased the bandwidth from 600mbit to 1000mbit and then to 1500mbit. after monitoring and analyzing the data, it was found that a very few users purchased a large number of tickets online, with the total number of tickets purchased in a single day exceeding 100. in order to ensure fair ticket purchasing, starting from january 5, the 12306 ticketing team has adjusted the online ticket purchasing process. after the user successfully purchases the ticket, the system will force the user to log out, and they need to log in again when purchasing tickets again.

the former ministry of railways also thought of inviting "foreign aid". among all the websites at that time, the only ones with rich experience in "not collapsing despite a huge increase in visits in a short period of time" were tmall and taobao, which were in full swing during the "double eleven". therefore, alibaba group sent 17 technical elites to form a project team to assist the 12306 website in optimizing and improving it. the two teams worked side by side to greatly improve the user experience of the 12306 website. the system's daily ticket sales increased from 650,000 in early january to more than 1 million, and set a record of 1.192 million daily ticket sales on january 20.

tmall taobao mall's "double 11 global shopping carnival" advertising dominates the shanghai xujiahui subway station to compete for user popularity and traffic ic photo

after this "extreme test" of the spring festival travel less than a year after it was launched, discussions and controversies about the 12306 architecture also appeared on the internet. the ministry of railways at that time carefully listened to the opinions of all parties and also carefully sorted out the causes of the problem and summarized the main this is due to the insufficient processing capacity of the ticket inquiry and order/e-ticket business partitions, resulting in long response times under high concurrent access requests during peak periods. in addition, each business partition is not well isolated, resulting in an "avalanche" effect from the inside to the outside of the system. , causing website congestion and affecting users’ ticket purchasing experience.

in response to the above problems and reasons, the developers focused on architecture optimization and reconstruction ideas to improve the response speed of ticket inquiry and transaction processing, improve the scalability of the backend system, change the online queuing method, and reduce the bandwidth pressure caused by centralized ticket release during peak hours. . at the same time, separate core businesses as much as possible to reduce strong correlations between business links. specific contents include:

first of all, the concurrent query capability must be greatly improved. 12306 uses an in-memory computing database to replace the traditional database, increasing the response speed of ticket queries from less than 1,000 times/second to more than 20,000 times/second, and reducing the response time from the original 1s to 10ms, allowing users to you can quickly get the train number and remaining tickets.

secondly, it is necessary for users to queue up and place orders without congestion during peak periods. to this end, 12306 has built a transaction processing queuing system. the queue's order request reception capacity exceeds 100,000 orders/second, and users can quickly complete orders during peak ticket sales periods. operation, wait for the system to process it in sequence. during the waiting process, you can check the queuing status (time waiting for processing). in-memory computing databases are also used in queuing systems.

third, the order/e-ticket was transformed into nodes, databases and tables, and the original 1 node, 1 database, and 1 table were split into 3 nodes, 30 databases, and 30 tables. online related operations are dispersed to various nodes and database tables, so that users' ticket booking requests on the website can be responded to and processed faster.

finally, the business separation of ticket booking and ticket collection operations is carried out, and different business nodes (ticket sales nodes and ticket collection nodes) carry the online ticket sales and offline ticket collection services; the read and write separation of order/e-ticket generation and query is carried out , using an in-memory computing database to centrally store orders/e-tickets, the response speed of order queries has been increased from about 200 times/second to more than 5,000 times/second, which greatly improves the query efficiency of orders/e-tickets.

in the stress test before going online, the system after the optimized architecture achieved a maximum transaction capacity of 300 tickets/second, which can meet the business needs of a daily ticket sales volume of 5 million. during the spring festival in 2013, the highest daily ticket sales of the optimized 12306 website reached 3.64 million, accounting for 40% of the total ticket sales. the ticket sales were more than three times the peak of the spring festival in 2012 (1.19 million).

on february 14, 2013, parents and their children were waiting for their return journey at huaibei railway station in anhui province. ic photo

however, the demand for the 12306 platform is still growing rapidly, and the original improvements are quickly reaching their limits. during the national day golden week in 2013, 12306 internet ticket sales reached 4.6 million, which was once again close to the upper limit of system processing. due to the limitations of the 3g network (the 4g network was first year but not yet popular) and the rapid growth of the number of internet users, the external network entrance was blocked during the peak period. bandwidth is tight and cannot meet the needs of further increase in internet ticket sales. in addition, as the main channel for railway ticket sales, the single-center operation model of the internet ticketing system can no longer meet the needs of business security and reliability.

to this end, the second round of structural optimization of the 12306 website has been launched since the end of 2013:

migrate services such as user login and frequently used contact inquiries to the memory database, improving the processing performance and reliability of related services.

the second production center of the academy of railway sciences was built to achieve "double-active" with the existing first production center of china state railway group co., ltd. to improve the security and reliability of the website and integrate the processing capabilities of the order/electronic ticket cluster. increased by 1x. the order/e-ticket cluster has been expanded to 10 groups of nodes, 100 libraries, and 100 tables.

deploy the ticket inquiry service on the public cloud. through policy configuration, the ticket inquiry traffic can be diverted to the public cloud at any time to alleviate the pressure on the processing resources and bandwidth of the website during peak ticket sales periods.

the stress test before going online verified that the system could meet the designed ticket sales capacity of 10 million tickets per day. during the peak period of spring festival travel in 2015, the actual ticket sales speed exceeded 1,000 tickets per second (approximately 3.6 million tickets per hour). the public cloud diverted up to 75% of query requests during the 2015 spring festival travel period, and the website's external ticket query service capacity increased threefold. the 12306 website processed more than 18 billion ticket inquiries during the peak period of spring festival travel in 2015, with an average response speed of more than 300,000 times/second.

as 12306's carrying capacity for the "extreme pressure" of spring festival travel has been doubled, the "ticket-grabbing plug-in" that once caused headaches for the design and operation teams has become the biggest factor of instability. although the 12306 team has repeatedly protested against the use of ticket grabbing software when purchasing tickets, there are still endless small software on the internet that rely on automatically and frequently refreshing ticket information and automatically filling in reports to grab tickets. this type of software improves the success rate of ticket grabbing by quickly refreshing ticket information, aiming to help users grab tickets as soon as possible. although it meets the market demand, it also seriously interferes with the public who buy tickets manually, affecting the fairness of ticket purchasing; and in order to improve the success rate, users often need to enter the passenger's personal information into the system in advance before "grabbing tickets" in the midst of anxiety, he personally leaked his personal information. ‌

against this background, the 12306 team must also make targeted changes - in addition to continuously optimizing the system's peak processing capacity, it also introduced real-name systems, complex verification codes, mobile phone verification codes and other functions, but it still cannot curb ticket grabbing software. the rampant situation has led to the phenomenon of poor network ticket purchasing during the spring festival and the national day golden week.

until the 2019 spring festival travel period, 12306 piloted the wait-list ticket purchase service function. this function is to put the manual user orders and the user orders of the ticket grabbing software into the same "prize pool", so that everyone can compete fairly, and the system randomly selects the "winner" to complete the next ticket purchase process. in this way, even if the ticket-grabbing software refreshes the screen quickly, it has no advantage at all. this has gradually curbed the popularity of ticket-grabbing software.

as the saying goes, "the magic is higher than the road is higher." the standby ticket model is a heavy blow for 12306 against the ticket-grabbing software. however, driven by interests, the ticket-grabbing software will definitely continue to unfairly target this model again. competition, such as increasing the "winning probability" of ticket grabbing software orders in the waiting ticket prize pool, etc. i believe that our 12306 team can overcome difficulties and maintain the ticketing system with the largest number of visits and transactions in the world. the battle of wits and courage between the two parties is still evolving along with china's ever-developing railway network and residents' travel needs.