High performance, easy to use, safe and reliable
Data science platform

Visualization and process of data mining

The function of data processing is modularized, which greatly shortens the time for cleaning and organizing data;

Build a visual task flow by dragging and dropping to easily implement data operation logic;

The process can be reused when analysts trace back and verify logic;

Integrate more than 50 mainstream machine learning algorithms of 6 categories, and conduct real-time display of parameter adjustment evaluation;

Powerful user-defined integration module function, module reuse can directly play a role;

Build model intellectual property rights and comprehensively enhance the value of enterprise data assets;

Highly flexible and scalable system architecture

Support HDFS, S3, OSS and other distributed storage;

Alluxio distributed cache ensures high scalability of the whole system;

Distributed computing bottom layer based on SPARK, FLINK and MXNET;

Docker based underlying system architecture facilitates easy deployment and expansion;

Independent research and development of ACO scheduling layer, in-depth optimization of computing logic and resource allocation, and improvement of system performance;

Comprehensive big data management system

Unified management of files, databases and API data;

The data results can be displayed, exported and downloaded in charts;

Distributed storage greatly ensures data security and read/write efficiency;

Data governance, metadata management, data retrieval, permission management, etc;

There are multiple snapshot versions of each data, and the version can be selected for use or recovery at any time;

Efficient cross regional collaboration between teams

The roles and permissions of collaboration members can be defined. The permissions of personnel inside and outside the project are separated as follows:

Timed tasks and real-time message notifications can significantly reduce communication costs and achieve win-win cooperation;

The whole process visual data presentation and flexible data output enable analysts to focus on data applications;

Highly integrated functions of data application layer

Data mining results, one click data visualization through BI Factory components, and dynamic update of data sources;

Action Factory can easily connect data mining with the application system, so that the data results can directly serve the production system;

The data model can be separated from the function, docked with the existing system in API mode, and independently landed as an application product;

Multi directional data security

Encrypted storage of original files;

Encryption processing of communication between services to ensure transmission security;

Isolate the running environment of different users through the container;

The data cache adopts strict read and write permission control to isolate different user data;

Fine grained user permission control enables only authorized data and tasks to be accessed;

Amazon, Alibaba Cloud and other cloud security policies ensure secure encryption of cloud data;

Professional, reliable and all-round technical support

The Elemental Data Engine adopts the microservice architecture, has a fully asynchronous, highly available, elastic and scalable technology stack, integrates distributed computing engines of different styles such as batch, streaming, and deep learning, and carries out corresponding driver development. The system seamlessly integrates the management of computing engine and computing resources as service invocation into the overall product architecture, with multi-level separation and abstraction. The background engine optimizes the process and automatically generates code for the logical calculation process customized by users through page interaction, decomposes tasks into jobs of different sizes, intelligently distributes them to the computing resource nodes hosted by the cluster for calculation, and feeds back the execution results in real time.

The Scala language is used as the main development language. Compared with the Java language, it has excellent expressive ability. Its strong type checking and functional features make the service more stable

Self developed process management engine, which supports local operators and seamless access to a variety of open source computing engines, automatically optimizes the computing process, and realizes Reactive style complex topology computing management through full asynchronous non blocking mode

Self developed cluster resource management framework to dynamically adjust computing resources and distribute the operating environment; Match appropriate operation nodes according to algorithm and data characteristics, and support advanced functions such as manual adjustment

The notification service based on Websocket provides users with real-time messages and running result feedback, and supports collaborative communication between different clients

Support multiple underlying data storage (HDFS/S3/OSS), and use Alluxio to improve data read/write efficiency

Support a variety of underlying distributed computing engines Spark and Flink, and optimize a lot of algorithms and architectures on the basis of open source products

Based on MXNet and TensorFlow deep learning frameworks, it implements multiple algorithms and provides powerful modeling capabilities with GPU

The micro service architecture based on ServiceMesh, combined with Docker and Kubernetes, realizes the automatic discovery, registration and deployment of services

Elemental adopts microservice architecture, has a fully asynchronous, highly available, elastic and scalable technology stack, and integrates distributed computing engines of different styles such as batch, streaming, and deep learning.

High performance process management engine

Self developed cluster resource management framework

Multiple underlying data stores

Multiple underlying distributed computing engines

Deep learning modeling capability

Provide elastic scaling deployment based on K8s

The front end adopts the cutting-edge MVVM architecture, which separates data, interface and interactive behavior. React+Redux works with WebSocket to ensure the real-time reflection of data updates on the interface. The throughout data visualization technology makes the complex data processing process intuitive, and can be flexibly modified without a line of code or mouse drag, Big data processing has never been so simple.

It needs appearance, interaction, experience and performance. Based on Facebook's open source React, Ant.design is deeply customized to make the user interface fresh, simple and smooth

Not only knowing, but also seeing pictures. With D3.js, every step of big data processing can be seen and touched, and a large number of charts and visual results can be displayed in real time

Completely separate the front end from the rear end, making the deployment and upgrade of front-end code flexible and smooth

The front-end routing system based on HTML5 history provides strong support for cooperative operation and task processing

NodeJS+MongoDB's golden partner, preferred for lightweight data applications

Webpack+Babel enables users to enjoy ES6 from the future in advance

Realize seamless data sharing among subsystems with PostMessage

GraphQL, the next generation API query language, makes the cooperation between the front end and the rear end more flexible and free

The drag and drop interaction and data visualization technology throughout make the complex data processing process intuitive.

React based

D3.js supports visual interaction

Front end and rear end separation, asynchronous notification service

HTML5 history

Lightweight data application