Vector database

The MCP (Model Context Protocol) service based on the Tablestore supports two functions: document storage and hybrid retrieval tool. The Cherry Studio interface and the generic qwen max model are used to demonstrate the process of text data upload, vector embedding and query. In addition, it details the local running steps, environment configuration and secondary development methods of Python and Java versions, and provides application examples of integrated third-party tools such as Cherry Studio. Tablestore provides an efficient solution for MCP scenarios by virtue of hybrid query, low cost of Serverless, elastic expansion and other advantages.

Vector database is mainly used to support efficient vector retrieval scenarios (search for pictures by pictures, search for pictures by text, etc.). Through this training, you can master the core theory of vector database and the characteristics, scenarios and algorithm principles of the two vector indexing technologies, and master the application and performance optimization strategies of vector database through practical cases.

It's a good city to interact with Zero Run C10

Vector database summary

Analysis of milvus insert api process source code

Through a practical example, I will show you a comprehensive understanding of the vector retrieval service DashVector

On January 9, 2024, Nail will release the AI assistant product available to everyone. Users can click the magic wand in the upper right corner of the first nail screen to arouse the AI assistant to carry out conversational data AI, information summary, writing work summary, writing documents and other work. AnalyticDB for PostgreSQL (hereinafter referred to as ADB-PG) vector retrieval engine provides vector recall of core entities, helping AI assistants greatly improve the accuracy of model output in scenarios such as intelligent query.

The vector retrieval service is a vector engine Proxima kernel developed by Alibaba Cloud itself, which provides an efficient vector retrieval service with horizontal expansion, full hosting and cloud native. The vector retrieval service provides powerful vector management, query and other capabilities through the simple and easy to use SDK/API interface, which facilitates the integration of large model knowledge base building, multimodal AI search and other application scenarios.

The delete operation of milvus

Real time data analysis is required for business, and the cost of offline task transformation is high? When data changes, it is difficult to correct it like a batch task? AnalyticDB for PostgreSQL provides a real Stream warehouse solution with real-time ETL, mixed row and column storage, and a high-performance analysis engine to build databases in real time. In order to further improve ease of use, AlnayticDB for PostgreSQL has released an enterprise data intelligence platform, which provides visual real-time task development+real-time data insight, allowing you to easily translate offline tasks, and complete the construction of the entire real-time database using SQL and simple configuration. It also supports real-time data insight, and can conduct Ad hoc query, data exploration and chart analysis on any table of real-time database, so that you can debug real-time data and quickly conduct business insight.

Query of db and collection information in milvus

The balancer analysis of queryCoord

Comparison analysis of dataCoord 2

Milvus's compaction analysis (small files merged with large files)

Through this experiment, we can experience the whole process of building and applying enterprise specific knowledge base in Alibaba Cloud Bailian. At the same time, experience the use of ADB-PG vector retrieval engine to provide exclusive and secure storage to ensure the privacy and security of enterprise data.

Analysis of queryCoord's checkerController

Insert API execution process_milvus source code analysis

CreatePartition API execution process_syncNewCreatedPartitionStep_milvus source code analysis

CreatePartition API Execution Process_milvus Source Code Analysis

Be eager to apply large models, but why is the technology stack complex and difficult to start? The water has been tested, but the lack of optimization means cannot guarantee the recall rate and accuracy of Q&A? It is difficult to operate and maintain the self built large model, vector retrieval engine, service API and other basic components? There are many kinds of large-scale models, but lack of industry models and application templates? Alibaba Cloud Bailian x AnalyticDB vector engine launched a one-stop enterprise specific large-scale model development and application platform, which can easily complete the development of enterprise specific large-scale model applications like building blocks, provide application APIs, and provide external services with one click access to enterprises' own business applications.

CreateCollection API Execution Process_milvus Source Code Analysis

Install and deploy milvus stand-alone version (quick experience)

In the era of AIGC, many people understand that vector database is to add a vector index to the traditional database. However, as the application of large models gradually expands to the core business field, the large-scale replication will be hindered by splicing large models, vector indexes and structured data analysis results through complex code engineering. At the same time, concurrent query performance, data consistency, high reliability and elastic scaling will become more and more important. Alibaba Cloud AnalyticDB anchor point has developed an enterprise level vector database, which is the only vector engine recommended by ChatGPT and LangChain among domestic cloud manufacturers. This article will share the wonderful speech of QCon 2023, decrypt the core technology of AnalyticDB fully self-developed enterprise level vector database, and the technology evolution route of the new generation vector database on the separation of cloud native survival computing and AI native.

Build AI intelligent assistant based on LLM+RDS PostgreSQL

Problems encountered when FC+RDS creates an AI assistant

As ChatGPT ignited the big language model market at the beginning of the year and LLM broke out in a concentrated way, most enterprises have completed the research on AIGC products and entered the second stage, that is, to seek solutions for large-scale AIGC products. This article introduces how to implement large-scale large language models for enterprises and support the rapid use of multiple models, including Tongyi QianQ-7b, ChatGLM-6b, Llama2-7b and Llama2-13b.

How to build an enterprise specific Chatbot that understands you better based on vector database+LLM (big language model).

AnalyticDB Postgres vector version pure vector retrieval has twice the performance of the open source milvus HNSW algorithm and 10 times the performance of IVFSQ8 in the high-dimensional face retrieval scene.