Open source Chinese community team broadcast the first show, telling the story behind the open source Chinese community in the name of sharing "

Private clouds are built for a single customer to use, thus providing the most effective control over data, security, and quality of service. The precondition is that the customer has the infrastructure and can use it to deploy applications on it. Its core attribute is proprietary resources. This article will combine Netease Yunxin And take you to understand the overall architecture of the on-demand privatization platform in the form of a global overview.

Recommended reading

《 How to design the server architecture when hundreds of thousands of people chat online in the live broadcast room at the same time? 》

《 High concurrency IM system architecture optimization practice 》

The emergence of cloud computing abstracts a large number of server hardware into a huge resource pool through hardware virtualization, which can dynamically provide users with services in three forms: infrastructure, platform and application. At present, enterprises use public cloud and private cloud. Under the public cloud, enterprises can abandon complex infrastructure construction and maintenance and purchase computing resources and application services on demand. However, considering the sensitivity of some data and the limitations of network interconnection, enterprises have great concerns about completely hosting their core businesses to the public cloud. Therefore, based on the reliability, security and controllability of business, many enterprises choose to build private clouds.

Private cloud is built for a customer to use alone, thus providing effective control over data, security and quality of service. The precondition is that the customer has the infrastructure and can use it to deploy applications on it. Its core attribute is proprietary resources.

Architecture

Module design of on-demand private cloud platform

Basic services include:

1. Cache, database, message queue and other services deployed on the PaaS layer provide data storage and access.
2. Container management is based on Docker and Kubernetes to manage the life cycle of each component of the on-demand service.

The capability management cluster includes:

1. Uploading service cluster: upload breakpoints distributed on different nodes based on S3 devices.
2. Streaming media service cluster, supporting video playback features such as playing while playing.
3. Transcoding cluster, an engine for processing video transcoding.

By providing basic services and capability management cluster building platform services, users only need to access business applications on this basis, integrate and play SDKs and upload SDKs, and quickly build on-demand services.

Deployment and implementation design of on-demand private cloud platform

The figure above illustrates the minimum deployment cluster of the on-demand privatization platform. The control cluster includes hardware resource virtualization based on Openstack, container service management realized by Docker and Kubernetes, sentinel monitoring based on virtual resources and containers, and account management. The computing cluster includes the deployment of on-demand service components and the dependent storage and data management services.

Platform composition

The whole privatization platform is built from the bottom up, including the virtualization of hardware resources, the construction of data storage services, and the deployment of on-demand component services.

Virtualization of hardware resources

The figure above illustrates the hierarchical abstract architecture of virtualization of hardware resources:

IaaS： Openstack based cloud computing basic services (including cloud computing, cloud network and local storage)

Virtualize hardware resources into virtual machines, support virtual machine management operations (create, start, stop, restart, delete, snapshot, modify specification, offline migration, modify virtual machine name, etc.), image snapshot management, security group management, network resource management (through managing intranet IP and internet IP floating pools, using acquisition, destroying and releasing to IP pools) Monitoring alarm (monitoring various indicators of virtual machine).

Pass service: Multi tenant PaaS services built on IaaS (including storage services and databases)

Kubernetes： Container service for multi tenant cluster orchestration

Kubernetes service provides container creation, orchestration, scheduling, service discovery, elastic scaling and other functions for distributed application services. The load balancing service based on Kubernetes' characteristics and integrating basic services can ensure high availability, high reliability, elastic capacity expansion and service isolation at different levels.

Management services: Provide user management and API operation related services

Container service platform that provides product development environment, test environment, online environment and other life cycles. Through the deployment of SOA service-oriented system, it supports static resource publishing, dynamic capacity expansion and publishing of back-end services, automatic online and offline service, etc.

Data storage service construction

Construction of database on PaaS layer

Based on MySQL, the master-slave deployment is performed on the computing node to isolate the network environment and provide private network instances. All instances are highly available, that is, each instance has the master and slave roles. When the slave goes down, it will not affect the service. When the master goes down, it will switch to the slave instance. At the same time, the service management will pull up the master instance. In order to provide stable and reliable database services, multiple security protection measures and professional backup, recovery and other functions.

Construction of storage service on PaaS layer

Based on S3 devices, it also provides multi node breakpoint upload and image and video processing cloud information acquisition services. It provides high availability, supports breakpoint resume, and obtains the storage characteristics of video file meta information according to the characteristics of video files. The upload service and cloud information acquisition service adopt Docker image deployment to ensure the automation of service management.

On Demand Component Service Deployment

The deployment of all on-demand components is based on Docker images, and container management services are used to ensure high availability and automated management of services. The component diagram is as follows:

1. Registry： A registry for service registration and discovery. Deploy the native zookeeper cluster as an independent registry, mainly using the consistency, synchronization and coordination capabilities and service activity detection capabilities provided by zookeeper. The deployment of Zookeeper uses Docker containers, and uses the container's service management capabilities to ensure the stability and high availability of services.
2. Consumer： The service consumer that invokes the remote service. It includes the API interface provided externally and the interface open for live recording video storage. Users upload, transcode and manage videos through the interface. The service deployment of the consumer uses Docker containers, and uses the container's service management capabilities to ensure the stability and high availability of services.
3. Provider： Call the service provider of the remote service. It includes video processing service, video detection service, recorded video processing service and statistics service. The provider's service deployment uses the Docker container, and uses the container's service management capabilities to ensure the stability and high availability of services.
4. Monitor： The monitoring center that counts the number and time of service calls.

Call relationship between components

1. The service provider starts to register its own services with the registry.
2. The service consumer starts to subscribe to the required services from the registration summary.
3. The registry pushes the list of service providers to consumers based on long connections.
4. The service consumer selects one from the list to call based on a certain load balancing algorithm. If it fails, select another to call.
5. Service consumers and providers accumulate the number and time of calls in memory, and regularly send statistical data to the monitoring center.

excellent     spot

Compared with public cloud, private cloud has outstanding advantages in data security, making full use of existing hardware and software resources, quality of service, and management process. Building on-demand private cloud platform based on Docker not only has the above advantages, but also has the characteristics of elastic resource management, perfect monitoring, simple deployment, automated maintenance, etc.

(1) Data security 。 Because the storage service is deployed in the user's hardware environment, built behind the firewall, and the high availability of the storage service can ensure the reliability and security of user data.

（2） Perfect monitoring 。 The sentinel system described above is involved in the construction process of the entire on-demand privatization platform, and can report the abnormal conditions of components in each process in a timely manner.

（3） Resource elastic management 。 Build IaaS platform based on Openstack, which can freely manage and create virtual machines. The container management service is built based on Docker and Kubernetes, which can freely create services based on the service image. At the same time, the container management service can achieve elastic expansion.

（4） Simple deployment and automated maintenance 。 After the basic IaaS platform is built through pre arranged scripts, the service image can be used to quickly deploy services. The service discovery capability of container management services makes service maintenance simple.

total     junction

On the whole, due to the characteristics of private cloud, the operation and maintenance cost is much higher than the development cost in the implementation process. Therefore, the process of user oriented delivery and implementation must be simple, and subsequent maintenance should be automated as far as possible. Minimize manual intervention. The technical characteristics of the architecture (Openstack, Docker, Kubernetes, Zookeeper) used in the construction process of this article are in line with these characteristics. This article attempts to introduce you to the overall architecture of the on-demand privatization platform in the form of a global overview. On this basis, the details of each process will be further explored to discuss the considerations and rationality of implementation. If you want to obtain more dry products and technical dry goods, please pay attention Netease Yunxin Blog 。