Related products

Customer Stories

Related recommendations

Recommend full link deep customization development platform

The recommended full link in-depth customization development platform PAIREC is a platform level service suitable for enterprise developers to independently build, develop, iterate, operate and maintain a complete set of recommendation systems. According to offline, near line and real-time data links of the recommendation system, engineering architecture, and mainstream recall, sorting and exploration model applications in the recommendation technology field, all modules and links involved rely on Alibaba Cloud Apsara big data architecture. Developers can flexibly select models based on enterprise technology stacks and development habits, and flexibly develop, debug Release, etc., and combine the capabilities of the experimental platform to significantly improve the iteration efficiency of the recommendation system. The pure white box development mode will bring developers a more transparent, controllable and flexible development experience. In addition, if the enterprise recommendation algorithm and engineering team construction are relatively young, we suggest that at the initial stage of docking, we use the algorithm model customized by the Alibaba algorithm team based on the industry to start the service. On the one hand, it can help enterprises complete the deployment of the complete recommendation system in a short period of time, and on the other hand, it can help enterprise developers quickly start and independently complete model training and effect evaluation. If Alibaba engineers are required to provide in-depth tuning customization and experience sharing, they can also carry out in-depth cooperation through business negotiation and communication. It is recommended that the full link in-depth customization development platform PAIREC deeply integrates the following services/frameworks of Alibaba Cloud to build a recommendation system with stable performance and high-quality effects.

 Beijing Taizi Mobile Technology Co., Ltd
Beijing Taizi Mobile Technology Co., Ltd

Beijing Taizi Mobile Technology Co., Ltd. is a video software developer. Its products are mainly ticket circle videos. It is a full category, all-purpose video recording, creation and expression platform based on mobile Internet that integrates multiple powerful functions such as life recording, creative expression, video production, AI processing, sharing and interaction. Four core products: ticket circle video for PC, ticket circle video APP, ticket circle video applet and ticket circle Vlog applet complement each other to build a ticket circle video matrix. Over the past two years, it ranked top 3 on the Aladdin video list, accumulated 400 million+platform users, served millions of content creation producers, and generated millions of revenue.

 Beijing Milian Technology Co., Ltd
Beijing Milian Technology Co., Ltd

Yidui is a brand of Beijing Milian Technology Co., Ltd., which was founded in 2015 and is a national high-tech enterprise and Beijing Zhongguancun high-tech enterprise. In 2019, the company's revenue has reached nearly 1 billion yuan. In 2020, the company will complete the B round of financing. The investors include: Xiaomi, Yunjiu Capital, People's Network Cultural Industry Fund, Shunwei Capital, Light Source Capital, Lanchi Venture Capital, and XVC Venture Capital. Yipai App focuses on making friends and dating on the mobile end, creatively integrates video, live broadcast and online matchmaking, opens up an independent track of video dating community, and provides a new social experience for singles. At present, there are 100 million registered users, more than 40000 active matchmakers, and about 10 million online matchmaking activities every month. Find someone, go to Iraq Yes! More real love community, meet your exciting TA!

 Shanghai Feirui Network Technology Co., Ltd
Shanghai Feirui Network Technology Co., Ltd

Shanghai Ferry Network Technology Co., Ltd. (verystar) was founded in 2011 by a senior and experienced international management team. The company is located in Shanghai. As the official certification acceptor of WeChat payment and Alipay payment, Ferry focuses on innovation and R&D in the fields of mobile marketing, O2O, social media and mobile e-commerce, It is in a leading position in China's rapidly expanding mobile marketing and online commerce market.

Explore Alibaba Cloud products and build cloud applications and services

Free trial

Related products

Customer Stories

Related recommendations

Recommend full link deep customization development platform

The recommended full link in-depth customization development platform PAIREC is a platform level service suitable for enterprise developers to independently build, develop, iterate, operate and maintain a complete set of recommendation systems. According to the offline, near line and real-time data links of the recommendation system, the engineering architecture, and the mainstream recall, sorting, and exploration model applications in the recommendation technology field, all the modules and links involved rely on the Alibaba Cloud Apsara big data architecture. Developers can flexibly select models based on enterprise technology stacks and development habits, and flexibly develop, debug Release, etc., and combine the capabilities of the experimental platform to significantly improve the iteration efficiency of the recommendation system. The pure white box development mode will bring developers a more transparent, controllable and flexible development experience. In addition, if the enterprise recommendation algorithm and engineering team construction are relatively young, we suggest that at the initial stage of docking, we use the algorithm model customized by the Alibaba algorithm team based on the industry to start the service. On the one hand, it can help enterprises complete the deployment of the complete recommendation system in a short period of time, and on the other hand, it can help enterprise developers quickly start and independently complete model training and effect evaluation. If Alibaba engineers are required to provide in-depth tuning customization and experience sharing, they can also carry out in-depth cooperation through business negotiation and communication. It is recommended that the full link in-depth customization development platform PAIREC deeply integrates the following services/frameworks of Alibaba Cloud to build a recommendation system with stable performance and high-quality effects.

 Beijing Taizi Mobile Technology Co., Ltd
Beijing Taizi Mobile Technology Co., Ltd

Beijing Taizi Mobile Technology Co., Ltd. is a video software developer. Its products are mainly ticket circle videos. It is a full category, all-purpose video recording, creation and expression platform based on mobile Internet that integrates multiple powerful functions such as life recording, creative expression, video production, AI processing, sharing and interaction. Four core products: ticket circle video for PC, ticket circle video APP, ticket circle video applet and ticket circle Vlog applet complement each other to build a ticket circle video matrix. Over the past two years, it ranked top 3 on the Aladdin video list, accumulated 400 million+platform users, served millions of content creation producers, and generated millions of revenue.

 Beijing Milian Technology Co., Ltd
Beijing Milian Technology Co., Ltd

Yidui is a brand of Beijing Milian Technology Co., Ltd., which was founded in 2015 and is a national high-tech enterprise and Beijing Zhongguancun high-tech enterprise. In 2019, the company's revenue has reached nearly 1 billion yuan. In 2020, the company will complete the B round of financing. The investors include: Xiaomi, Yunjiu Capital, People's Network Cultural Industry Fund, Shunwei Capital, Light Source Capital, Lanchi Venture Capital, and XVC Venture Capital. Yipai App focuses on making friends and dating on the mobile end, creatively integrates video, live broadcast and online matchmaking, opens up an independent track of video dating community, and provides a new social experience for singles. At present, there are 100 million registered users, more than 40000 active matchmakers, and about 10 million online matchmaking activities every month. Find someone, go to Iraq Yes! More real love community, meet your exciting TA!

 Shanghai Feirui Network Technology Co., Ltd
Shanghai Feirui Network Technology Co., Ltd

Shanghai Ferry Network Technology Co., Ltd. (verystar) was founded in 2011 by a senior and experienced international management team. The company is located in Shanghai. As the official certification acceptor of WeChat payment and Alipay payment, Ferry focuses on innovation and R&D in the fields of mobile marketing, O2O, social media and mobile e-commerce, It is in a leading position in China's rapidly expanding mobile marketing and online commerce market.

Explore Alibaba Cloud products and build cloud applications and services

Free trial

reminder

Sorry, you can only see 8 pages before login. Log in now and browse all the best practice cases of technical solutions!

If you are an enterprise account, you can generate a sub account to authorize access.

BpFile (id=263, bpId=140, name=Build enterprise level personalized recommendation system based on PAI, author=null, keyword=PAI, machine learning, recommendation, training, reasoning, CTR, description=Quickly implement a set of efficient, accurate, easy to use, and scalable enterprise level personalized recommendation system from 0 to 1. , position=null, ossUrl=bp-MDHX281XTP3M134R.pdf, tags=null, level=null, tagList=null, products=null, productList=null, hotspot=null, oneClick=1, createTime=null, modifiedTime=null, timeConsuming=null, status=1, pdfDescription=

This scheme takes the basic algorithm scheme preset by Alibaba Cloud's PAI team as an example to demonstrate how to use the data and AI products provided by Alibaba Cloud as the basis. The offline part uses Maxcompute&Dataworks&PAI's big data&AI system, and the online service uses the recommendation engine PAI-REC, A/B test system PAI-A/B, online model service PAI-EAS, and online data service Hologres, Through the PAI-REC operation and maintenance and experimental operation platform, Alibaba Cloud can quickly build a set of efficient, accurate, easy to use and scalable intelligent recommendation system from 0 to 1.

, templateId=OJ9RC5SK2P6M5VOQ, freetry=null, visitTime=null, visitCount=null, video_url=, buttonName=null, buttonUrl=null, targetId=OJ9RC5SK2P6M5VOQ, partner=null, partnerUrl=null, partnerLogo=null, cooperation=, cooperationList=null)
one Best Practice of Building Enterprise Personalized Recommendation System on PAI Platform Business Flow Chart Scenario description PAI is an AI platform launched by Alibaba Cloud, providing one-stop Machine learning solutions. This best practice uses the PAI platform to combine AliCloud RDS for MySQL, OSS and cloud number According to the database, Redis version and other products build an efficient offline training+ Line reasoning recommendation business system. Applicable customers: 1. Internet industry customers, MAU8 million - 15 million. 2. The business is classic recommendation scenarios such as information flow and advertising recommendation. 3. The data has used MaxCompute or is ready for use. 4. Have 1-5 engineering personnel with algorithm background. 5. The original service is based on the self built system of open source algorithm, but it is limited Recommended business architecture Personnel and costs cannot be further improved. solve the problem 1. Use PAI platform to build offline training system. 2. Use PAI platform to build online reasoning system. Product List  Machine learning PAI  MaxCompute  DataWorks  RDS MySQL  Object storage OSS  ApsaraDB for Redis  Table Store
two ECS (product name) document template (manual name)/document version information Alibaba Cloud Enterprise cloud practice Building enterprise level personalized recommendation on PAI platform System Best Practices Document version: 20210624 (release date) Document version: 20150122 (release date) 2
three PAI platform builds enterprise level personalized recommendation system document version information Document version information Text information Attribute Content Document name Best practice of building enterprise level personalized recommendation system on PAI platform Document No. 140 Document Version V1.6 Version date 2021-06-24 Document Status External Publishing Producer Jing Hai Reviewers You Sheng, Yun Kui Document change record Version No. Date Author Reviewer Description Jing Hai, Bo Yan, Aohai, Ding Jing V1.0 2020-02-06 - Create Zhongde V1.1 2020-02-27 Modification and update of Jinghai Yunkui and Yousheng V1.2 2020-03-01 Jinghai - modified and simplified V1.3 2020-03-03 Xiaohui - document optimization V1.4 2020-03-05 Jinghai - change name V1.5 2020-12-05 Add CADT to Jinghai V1.6 2021-06-24 Jinghai content update Document version: 20210624 (release date) I
four Preface to building enterprise level personalized recommendation system on PAI platform preface summary PAI is an artificial intelligence platform launched by Alibaba Cloud, which provides a one-stop machine learning solution. This best practice benefits Use the PAI platform to combine Alibaba Cloud RDS for MySQL, OSS for object storage, and ApsaraDB for Redis To build an efficient offline training+online reasoning recommendation business system. There are many steps in this best practice, and it takes about 6~8 hours to complete the full operation. Scope of application Use PAI platform to build a recommendation system of offline training+online reasoning. Terminology  Machine learning: machine learning refers to the learning of a large number of historical data by a machine through statistical algorithms to generate Experience model is used to guide business. In short, machine learning has two core steps: One is training, the other is reasoning. Training refers to the process of constantly modifying the model by using a large number of training samples. PUSH Rationality refers to the process of using trained models to predict.  Machine learning PAI (): It is an artificial intelligence platform launched by Alibaba Cloud, Providing one-stop machine learning solutions is a machine learning platform positioned to serve Alibaba Group, It is committed to making AI technology more efficient, concise and standard to be used by internal developers of the company. See https://help.aliyun.com/document_detail/69223.html  CTR (Click Through Rate): click through rate refers to the click through rate of online advertising, and the actual amount of advertising The number of clicks divided by the total number of ads displayed. CTR is an important indicator to measure the whole recommendation business, that is It means that the focus of recommendation business is how to improve CTR.  MaxCompute (formerly ODPS): it is a big data computing service that can provide fast and fully hosted PB level data warehouse solution enables you to analyze and process massive data economically and efficiently. See https://www.aliyun.com/product/odps  Dataworks: It provides big data OS capabilities and provides professional high quality services in the way of allonebox Efficient, safe and reliable one-stop big data intelligent cloud research and development platform. At the same time, it can meet users' requirements for data governance and quality Volume management requirements, giving users the ability to provide external data services. See https://data.aliyun.com/product/ide  RDS MySQL: Alibaba Cloud Relational DatabaseService, RDS for short) is a stable, reliable and elastic online database service. AliCloud based distributed Document version: 20210624 (release date) III
five Preface to building enterprise level personalized recommendation system on PAI platform High performance storage of file system and SSD disk. RDS supports MySQL, SQLServer, PostgreSQL PPAS (highly compatible with Oracle) and MariaDB engine, and provides disaster recovery, backup, recovery, and monitoring A complete set of solutions in terms of control, migration, etc., completely solve the problems of database operation and maintenance. RDSMySQL Based on the MySQL source code branch of Alibaba, it has passed the test of high concurrency and large amount of data during the Double 11 Festival Excellent performance. RDSMySQL supports instance management, account management, database management, backup recovery Basic functions such as white list, transparent data encryption and data migration. See https://www.aliyun.com/product/rds/mysql  ApsaraDB for Redis: ApsaraDB for Redis is compatible with open source Redis Protocol standard, database service with hybrid storage mode of memory and hard disk, based on high reliability dual machine hot standby The architecture and smoothly scalable cluster architecture meet the business requirements of high read/write performance scenarios and elastic configuration. detailed see https://www.aliyun.com/product/kvstore  Table store (original OTS): Alibaba Cloud self-developed table store for massive structured data storage ServerlessNoSQL multi model database, widely used in social networking, Internet of Things, artificial intelligence, metadata And big data. Provide the WideColumn model and message model Timeline compatible with HBase And the time-space model Timestream, which can provide petabyte level storage, 10 million TPS and millisecond level delay services Ability. See https://cn.aliyun.com/product/ots  Object storage OSS: massive, secure, low-cost, highly reliable cloud storage service, providing 99.9999999999999% Data reliability. RESTful API can be used to store and access anywhere on the Internet Resilient expansion of management capacity, multiple storage types for selection, and comprehensive optimization of storage costs. See https://www.aliyun.com/product/oss  The cloud architecture design tool CADT is a product that provides self-service cloud architecture management for cloud applications, which significantly Reduce the difficulty and time cost of application cloud management. This product provides a wealth of prefabricated application framework templates, the same as Self service drag and drop is also supported to define the application cloud architecture; Support the configuration and management of more AliCloud services. Users can easily manage the cost, deployment, operation and maintenance, and recovery of the cloud architecture solution throughout its life cycle Li. See https://www.aliyun.com/product/developerservices/cadt Document version: 20210624 (release date) IV
six Building enterprise level personalized recommendation system directory on PAI platform catalog Document version information I ......................................................................................................................................................................... Legal Statement II preface................................................................................................................................................................................ III ................................................................................................................................................................................. Catalog V Overview of Best Practices one Preconditions four ................................................................................................................................................... 1. Introduction to recommended business background 5 1.1 Scope of application five ........................................................................................................................................................ 1.2. General Principle 5 1.3. Disclaimer five 1.4 Recommended system five ........................................................................................................................................................... 2. Introduction to Pre dependency 9 2.1. Model production tools nine .............................................................................................................................................. 2.2. Cloud Tools 10 2.3. Job scheduling tool ten 2.4 Online service tools eleven ......................................................................................................................................................... 3. Introduction to Business Architecture 13 3.1. Recommendation process thirteen ...................................................................................................................................................... 3.2. Data Description14 3.3. Processing logic nineteen 3.4. Deployment Architecture twenty-one ......................................................................................................................................... 3.5. Code package file description 22 4. Deploying the Basic Environment twenty-three ................................................................................................................. 4.1. Creating a Basic Environment Using the CADT Template23 5. Basic environment configuration thirty 5.1 RDS Instance Configuration thirty .......................................................................................................................................... 5.1.1. Creating an Account 30 5.1.2. Creating a Database thirty-one .............................................................................................................................. 5.1.3. Setting the intranet whitelist 32 5.1.4. Application for Internet address thirty-five 5.1.5. Add Internet white list thirty-five .................................................................................................................................. 5.2. TableStore Instance Configuration37 5.3. Inspection of OSSBucket forty-one ........................................................................................................................................... 5.4. Redis Instance Configuration42 5.4.1. Setting the Internet white list forty-two 5.4.2. Application for Internet address forty-three ............................................................................................................................................ 5.5. Checking ECS Instance44 5.6. Importing Sample Data to RDS forty-four ............................................................................................................................................. 5.7. Opening PAI Service46 5.7.1. Creating AccessKey forty-six 5.7.2. Opening MaxCompute, a big data computing service forty-eight .................................................................................................................................. 5.7.3. Opening PAI Service50 Document version: 20210624 (release date) V
seven Building enterprise level personalized recommendation system directory on PAI platform 5.8. Opening DataWorks Service fifty-three ................................................................................................................................................. 6. Deploy offline training system57 6.1. Creating a workspace fifty-seven .............................................................................................................................................. 6.2. Creating Data Integration 60 6.2.1. Enter data integration sixty 6.2.2. Add RDS data source sixty .......................................................................................................................... 6.2.3. Adding Redis Data Source63 6.2.4. Adding a TableStore Data Source sixty-five ........................................................................................................................... 6.3. Import rec work project source code67 6.4. Data development seventy-five 6.4.1. Importing Data into MaxCompute seventy-five .................................................................................................................................. 6.4.2. Introduction of PAI experiment 84 6.4.2.1. Experiment of importing two PAIs eighty-four .................................................................................................................... 6.4.2.2. Configuring OSS Path89 6.4.3. Reload PSSMART refined scheduling model training node ninety-one 6.4.4. Reload 01 Collaborative Filtering Recall Module Node ninety-three .............................................................................................................................. 6.4.5. Modifying Data Source Configuration95 6.4.6. Publish Custom Computing Resources and Functions ninety-nine ........................................................................................................................ 6.4.7. Modifying Project Cycle Schedule102 6.4.8. Publish Offline Workflow one hundred and five 6.4.9. Validate online workflow and supplement data one hundred and six ................................................................................................. 6.4.10. Run the task generation model of the latest day 113 6.5 Description of offline training system one hundred and fifteen ........................................................................................................................................ 6.5.1. Data Description115 6.5.2. CF Recall one hundred and fifteen 6.5.3. Data feature engineering one hundred and sixteen ........................................................................................................................................ 6.5.4. Model training 116 6.6. Ranking model training (PS-SMARTRANK) one hundred and nineteen ........................................................................................................................................ 6.6.1. Sample generation 119 6.6.2. Characteristic engineering one hundred and twenty 6.6.3. Model training one hundred and twenty-seven ........................................................................................................................................ 6.6.4. Model evaluation 129 6.6.5 Data synchronization one hundred and thirty ........................................................................................................................................... 6.7. Description of code structure131 7. Deploy Online Recall System one hundred and thirty-two 7.1 Data description one hundred and thirty-two ....................................................................................................................... 7.1.1.User item data description132 7.1.2. Item item data description one hundred and thirty-three .................................................................................................................. 7.2. Purchase online service of PAIEAS model134 7.3. Configuration recommendation recall one hundred and thirty-five 8. Online Reasoning one hundred and forty-five ........................................................................................................................................... 8.1. Deployment sorting model145 8.2. Building Online Services one hundred and fifty .................................................................................................................................................... 9. Demo system demo152 9.1. Setting up the demo system one hundred and fifty-two 9.2. Demo System Description one hundred and fifty-five Document version: 20210624 (release date) VI
eight Building enterprise level personalized recommendation system directory on PAI platform 10. Appendix one hundred and fifty-eight ................................................................................................................ 10.1. Example of OSS model file packaging steps 158 Document version: 20210624 (release date) VII