the first national standard for big data batch-stream integration was officially released, with alibaba cloud as the lead drafting unit
2024-09-03
한어Русский языкEnglishFrançaisIndonesianSanskrit日本語DeutschPortuguêsΕλληνικάespañolItalianoSuomalainenLatina
recently, the state administration for market regulation and the national standards administration officially released the first national standard for batch-stream integration in the big data field, gb/t 44216-2024 "technical requirements for batch-stream fusion computing of information technology big data". the lead drafting unit of the standard is alibaba cloud computing co., ltd., and it will be officially implemented on february 1, 2025.
the "technical requirements for batch-stream fusion computing of big data in information technology" clearly defines batch-stream fusion technology, stipulates that the batch-stream fusion system must be able to support both real-time and offline computing models, and should have key capabilities such as unified resource management, unified computing framework, unified api, unified sql interface, unified job management, unified permission management and elastic expansion.
batch-stream fusion is an important emerging technology trend in the field of big data. with the development of the internet, enterprises have increasing demands for large-scale real-time data computing in many scenarios such as e-commerce recommendations, transportation logistics, and financial transaction monitoring. the traditional batch computing paradigm is difficult to meet the requirements in terms of latency. the integration of streaming computing and batch computing for real-time data processing under a unified technical framework has become a must for the industry.
apache flink, an open source computing engine that uses stream computing and batch computing, is a rising star in big data computing. it allows users to perform stream processing or batch processing with a set of code, saving a lot of development costs. in 2019, alibaba acquired the parent company of apache flink. combining its own e-commerce scenario practices, it invested a lot of r&d efforts to optimize the apache flink product to achieve enterprise-level application standards with high throughput, low latency, high availability, and high accuracy. it continues to contribute to the global open source community and become a leading force in promoting community and technological innovation. at present, apache flink has become the de facto standard for batch-stream fusion computing in the industry. alibaba, ant group, bytedance, xiaomi, kuaishou, xiaohongshu, bilibili, didi, china unicom and hundreds of other companies are using apache flink for real-time computing.
as a leader in the domestic big data field, alibaba cloud initiated the establishment and compilation of the national standard for batch-stream fusion computing technology in 2019, and worked with various units in the industry to draft and improve it. integrating alibaba cloud's experience in building the flink version of real-time computing and rich industry practices, the national standard "technical requirements for batch-stream fusion computing of information technology big data" will better promote the technical development of batch-stream fusion computing, guide the design, development and deployment of domestic batch-stream fusion computing systems, and further help batch-stream fusion technology to be applied in various industries.
it is understood that alibaba cloud has accumulated profound technical strength in the field of big data. alibaba cloud has full-stack technical capabilities from big data processing and analysis, data warehouse to ai engineering services. its independently developed flagship products such as maxcompute, dataworks, and hologres can provide the industry's highest level of data processing performance, scalability, and stability; at the same time, alibaba cloud actively contributes and maintains a series of top open source projects, including apache flink, apache paimon, etc., providing developers with a rich selection of tools and providing fully managed cloud services.
