On April 16, 2024 (Beijing time), the Apache Software Foundation (hereinafter referred to as ASF) officially announced that Apache Paimon graduated as the top project of Apache (TLP, Top Level Project)。 Through the joint efforts of the community and

Abstract: This article is compiled from the Alibaba Cloud open-source big data platform Xu Bangjiang (snowstorm). The Flink CDC project donated by Alibaba has officially joined the Apache Foundation. The content is mainly divided into the following four parts: 1. Flink CDC new warehouse, new process, 2. Flink CDC new definition

Apache Flink PMC (project management committee) is pleased to announce the release of Apache Flink 1.19.0. As always, this is a substantial version, including a wide range of improvements and new features. A total of 162 people contributed to this version, and completed 33 FLs

Abstract: This article shares with Wang Gang, Fan Wen and Li Qianxuan from the Auto Home. This paper introduces some practices based on Paimon of Auto Home and some backgrounds. The content is mainly composed of the following four parts: 1. Background 2. Business practice 3. Paymon optimization practice 4. Future planning

Abstract: This paper was submitted from the Shell Home Decoration Digital Warehouse Team, which explored a sort scheme based on Flink+Paimon in combination with the home decoration business scenario. This scheme can accurately group and sort the full amount of data in a real-time environment, while reducing the consumption of memory resources. On

Abstract: This article is based on the sharing of Apache Paimon PMC Chair Li Jinsong (Zhixin) on Streaming Lakehouse Meetup · Online on May 16, mainly sharing the evolution, goals and applications of Apache Paimon. The content is mainly divided into

Abstract: This article was written by Su Xuannan, a teacher from Alibaba Cloud Flink team, to introduce the application of Flink streaming and batching in several common scenarios. The content is mainly divided into the following four parts: main scenarios landing situation, future outlook summary Part I: Introduction to streaming batch technology on

Abstract: This article is based on the sharing on Streaming Lakehouse Meetup · Online on May 16 by teacher Wang Feng (Mo Wen), the head of Alibaba Cloud open source big data platform, and mainly introduces how to conduct real-time big data analysis on the new generation of Hucang architecture. The content is mainly divided into

Abstract: This article is a compilation of the practice guide for the optimization of large-scale homework written by Yu Hangxiang, Chen Jingmin and Huang Pengcheng. Due to the rich content, this article shares the final status error report and start stop slow chapter, which is mainly divided into the following four parts: the diagnosis and tuning of checkpoint and snapshot timeout is fast

Abstract: This article is a compilation of the practice guide for the optimization of large-scale homework written by Yu Hangxiang, Chen Jingmin and Huang Pengcheng. Due to the rich content, the second part of this article shares the principle and method of tuning Flink SQL jobs that lead to backpressure, which is mainly divided into the following three parts: status

Abstract: This article is a compilation of the practice guide for the optimization of large-scale homework written by Yu Hangxiang, Chen Jingmin and Huang Pengcheng. Due to its rich content, this article shares the Datastream assignment, which is mainly divided into the following four parts: Flink state (State) introduction The big state assignment causes

June 13 | Hong Kong | The wind of offline Apache Flink Meetup blows to the bank of Xiangjiang River, and Apache Flink Hong Kong Meetup comes! In this event, we invited top experts from Alibaba Cloud to help developers fully understand the flow of Apache Flink

Abstract: This article was written by Su Xuannan, a teacher from Alibaba Cloud Flink team, to introduce Flink users to the technology and challenges of Flink streaming and batching as a whole. The content is mainly divided into the following three parts: Introduction to streaming and batching technology Challenges faced Summary of Flink's streaming and batching overview

The Apache Flink community is pleased to announce the release of Flink CDC 3.1.0! This is the first version after the community accepted Flink CDC as a subproject of Apache Flink, bringing exciting new features, such as transformation support and sub database and sub table consolidation

Abstract: This article is based on the sharing of Fu Dawei, a senior technical expert of Cloud Granule Intelligence, at the 2024 OceanBase developer conference on April 20, which described a series of efforts made by its data center under the traditional digital warehouse technology framework, and stepped into FlinkCDC and OceanBase

With the continuous maturity and development of Apache Flink technology community, more and more enterprises begin to use Flink for streaming data processing, so as to improve the value of data timeliness and obtain real-time business effects. At the same time, the data lake architecture in the field of big data is becoming increasingly new

Abstract: This article was written by Guo Weijie, a teacher from Alibaba Cloud Flink team, to introduce Flink Batch community users to the evolution of Flink DataStream API batch processing capabilities. The content is mainly divided into the following three parts: batch processing semantics and performance optimization Batch AP

Abstract: I sorted out the sharing of the platform construction special session in Flink Forward Asia 2023 by Wang Liuju, a technical expert of Alibaba Cloud Intelligent Group. The content mainly includes the following four parts: Alibaba Lingyang's platform evolution based on Flink real-time computing Flink capability optimization and

1. Youmeng+Introduction Youmeng+with the mission of "data intelligence, driving business growth", provides mobile application developers and enterprises with one-stop solutions including statistical analysis, performance monitoring, message push, intelligent authentication, etc. As of June 2023, it has accumulated 2.7 million moves

Abstract: This article is based on the sharing of data integration session in Flink Forward Asia 2023 by Wang Mingya (Yunshi), a senior technical expert of Alibaba Cloud DataWorks data integration team. The content mainly includes the following four parts: AliCloud DataWorks Data Integration Media