7 posts tagged with "FlinkSQL"

Wuxin Technology was founded in January 2018. The current main business includes the research and development, design, manufacturing and sales of RELX brand products. With core technologies and capabilities covering the entire industry chain, RELX is committed to providing users with products that are both high quality and safe.

Although the Hadoop system is widely used today, its architecture is complicated, it has a high maintenance complexity, version upgrades are challenging, and due to departmental reasons, data center scheduling is prolonged. We urgently need to explore agile data platform models. With the current popularization of cloud-native architecture and the integration between lake and warehous, we have decided to use Doris as an offline data warehouse and TiDB (which is already in production) as a real-time data platform. Furthermore, because Doris has ODBC capabilities on MySQL, it can integrate external database resources and uniformly output reports.

Abstract: This article is compiled from the sharing of Mu Chunjin, the head of China Union Data Science's real-time computing team and Apache StreamPark Committer, at the Flink Forward Asia 2022 platform construction session. The content of this article is mainly divided into four parts:

• Introduction to the Real-Time Computing Platform Background
• Operational Challenges of Flink Real-Time Jobs
• Integrated Management Based on StreamPark
• Future Planning and Evolution

Preface: This article primarily discusses the challenges encountered by Shunwang Technology in using the Flink computation engine, and how StreamPark is leveraged as a real-time data platform to address these challenges, thus supporting the company's business on a large scale.

• Introduction to the company's business
• Challenges encountered
• Why choose StreamPark
• Implementation in practice
• Benefits Brought
• Future planning

Abstract: This article originates from the production practices of StreamPark at Dustess Information, written by the senior data development engineer, Gump. The main content includes:

1. Technology selection
2. Practical implementation
3. Business support & capability opening
4. Future planning
5. Closing remarks

Dustess Information is a one-stop private domain operation management solution provider based on the WeChat Work ecosystem. It is committed to becoming the leading expert in private domain operation and management across all industries, helping enterprises build a new model of private domain operation management in the digital age, and promoting high-quality development for businesses.

Currently, Dustess has established 13 city centers nationwide, covering five major regions: North China, Central China, East China, South China, and Southwest China, providing digital marketing services to over 10,000 enterprises across more than 30 industries.

Abstract: This article presents the production practices of StreamPark at Joyme, written by Qin Jiyong, a big data engineer at Joyme. The main contents include:

• Encountering StreamPark
• Flink SQL job development
• Custom code job development
• Monitoring and alerting
• Common issues
• Community impressions
• Summary