Databricks open source LLM, training only costs three hours and 30 dollars

Source: contribution

2023-03-28 08:23:52

Databricks Inc, a big data analysis company, has recently joined the competition in the field of generative AI and released a Dolly The open source large-scale language model of Dolly, named Dolly, is a tribute to Dolly, the first cloned sheep.

Generative AI such as ChatGPT and Bard usually use data from thousands of different websites. The amount of data they use is amazing. In addition, thousands of powerful GPUs are needed to provide support for training AI with these data.

Databricks hopes that through open source Dolly and its training data, anyone can develop an AI that is truly human without investing millions of dollars, and that this kind of AI will no longer be affordable only for large technology companies, and millions of small companies will also be able to benefit from it.

In addition, let each small company build its own model without sending data to the service provider behind the proprietary model through API, which can also protect the sensitive data and proprietary intellectual property of small companies. In addition, some companies may have different trade-offs in model quality, cost and expected behavior, and these companies can also adjust the model according to needs.

Ali Ghodsi, CEO of Databricks, said Dolly only needs very little data and very short time to complete the training.

For only $30, a server and three hours, we can teach Dolly to start human level interaction.

Databricks did not create its own model from scratch, but based on the natural language processing model GPT-J, which EleutherAI will open source in 2021. GPT-J is a natural language processing AI model based on GPT-3 and composed of 6 billion parameters. The model is trained on an 800GB open source text dataset, and can be comparable to the GPT-3 model of similar scale.

Databricks said that although the model is much smaller, with only 6 billion parameters, and smaller data sets and training time (ChatGPT's parameters are 175 billion), Dolly still showed the same "magical human interaction ability" shown by ChatGPT.

Databricks is currently in GitHub A simple Databricks notebook has been opened on Databricks. Developers can use it to build Dolly on Databricks.

This wonderful review

Lifetime code 9527

Well done, the domestic independent research and development, independent innovation AI project will start to blowout

2023-03-30 09:32

fourteen fabulous

Research on Artificial Intelligence of Traditional Chinese Medicine

It is believed that small models can also work miracles, and LLM cannot be exclusive to oligarchs.

2023-03-30 10:43

four fabulous

A Jiang Jiang Jiang

This is very suitable for the company to deploy and use independently, and use professional knowledge for training, without worrying about the disclosure of the company's sensitive data

2023-03-30 09:37

two fabulous

Yokesily 2024-06-02 15:11

So designed

monkey_cici 2024-05-09 00:25

My I9 CPU, 64GB memory module and 3080Ti computer are inferior to the top configuration of 19999 on a tablet

Love to eat raw pears 2024-06-01 11:48

Why is this so-called "vulnerability" not a vulnerability? Spring, MyBatis and other frameworks can accept all kinds of CVE criticism, while MyBatisPlus has to dump the pot and accuse programmers of being too low-level# There is a difference. The premise is that you write XML, MyBatisPlus encapsulates Wrapper and claims to simplify code. Since it encapsulates and hides $#, it is not appropriate to do some necessary security checks? Instead of doubting the authority of CVE, you should know that SQL ->MyBatis ->MyBatisPlus ->various back-end scaffolds have multiple layers, each layer is simplifying, and each layer is throwing away the upper layer of the boiler. Who dares to use them. The programmers who use MyBatisPlus can't be expected to be at a high level. Every programmer wants to save effort. The front-end parameters can be directly obtained by HttpServletRequest from the back-end. Wrapper splicing can be found everywhere. If something goes wrong, is it the front-end or the framework? According to Qingmiao, can the injection vulnerability of the previous log4j and the deletion vulnerability of the Druid be used to eliminate low-level programmers?

GDWhisperer 2024-05-15 17:23

I transferred tens of thousands of yuan to my own account, which was under risk control. How did I do this? The bank should be responsible for this**

Small and beautiful software development 2024-06-01 05:06

Cheat one's job

oldpig 2024-04-28 09:59

”Huawei contributed all the source code "?, the title is completely inconsistent with the content.

All the way north GP 2024-04-25 14:55

America, the future of mankind

Shuimu Yi'an 2024-05-20 09:58

The news should be read continuously. I'm waiting for the third news besides rustdesk and teamviewer. Localized remote desktop software is far ahead.

Hakuna 2024-05-31 18:28

It is compatible with Oracle, but does not know "just" or "just". Those who can be compatible with Oracle and do well are real men and real warriors. You should know that compatibility means that even bugs must be compatible, and you have no other code that can not be copied. It's all based on real skills and understanding of oracle.

Chief taxi captain 2024-05-17 11:17

I suggest that 360 open source all its products, and then become the leading enterprise in the domestic open source industry through open source, leading everyone to compete with foreign enterprises

sweet potato chips 2024-05-31 22:08

Glue code consumes few resources

osc_92224065 2024-04-29 10:57

Long term oppressed outsourcing of state-owned enterprises

Bright 2024-05-19 23:25

What a fool! I killed myself. How can people deal with me later.

Xiao Xu Middle aged 2024-06-01 06:49

thank

gamedot 2024-05-17 11:14

Old Zhou is deeply concerned about Huawei's great cause of open source. He is not a Huawei person, but has Huawei's soul.

Xiao Xu Middle aged 2024-05-31 19:13

Very good

infoworld 2024-05-11 15:12

Universities should use open source free software instead of commercial ones. In this way, hands and feet will not be tied technically.

One code Yma 2024-05-09 09:58

Recently, I often go to interviews. People who hate Ali background most regard me as a fool, even though I am a fool

osc_25732934 2024-06-01 19:30

It seems that the current version of the Foreign Function&Memory API is not as fast as that of jni, or even worse. In addition, before vallhala comes out, all interactions between java and c have to get an additional memory. Even if it comes out, it may not be possible to directly throw a copy of binary data into memory as a structure. When the two apis are completely stable, the day lily is cold

One code Yma 2024-05-06 09:14

My technical article was moved by CSDN. Why didn't anyone step on the sewing machine? This kind of report is a joke to me. The monsters with background are fine, and the monsters without background fight to death

Rocket ship 2024-05-31 19:22

It's a ghost anyway.

kangaroo 2024-06-01 22:23

The next version focuses on improving existing functions * improving internal power and qi * and continues to move towards the goal of Grand Master.

MrChen89 2024-04-29 09:18

There are a group of people like this. I don't know what they have experienced. When it comes to HW, I can't say anything good, even if it's neutral

looly 2024-06-02 14:32

@Qingmiao Hutool has also been mentioned some loopholes that I think are relatively "low-level", or I think are not loopholes. At first, I was also very angry, but after thinking it through, I found that CVE's idea was that once you did not actively remind users that there was a pit, the user fell into the pit is your fault, that is, your vulnerability. For example, as a traffic policeman, you should remind everyone who crosses the road to pay attention to safety, and ask him to answer whether he knows. Once you don't remind someone and are hit by a car, you can't get away from it. Similarly, when using frameworks and tools, you should provide at least one parameter to remind users that there may be SQL injection vulnerabilities. Note that it is not in the comments, but in the method parameters, which is the user's responsibility. Therefore, it is not comprehensive to provide solutions in comments or documents.

Monkeys think of apes 2024-05-31 18:31

You can cheat your brother. Just don't cheat yourself

Single structure 2024-05-11 10:09

Selected as Open Source China's disgrace pillar

haol666 2024-05-31 18:56

This story is powerful, I take it seriously, until I see the end.

xiaoqibabby 2024-05-15 17:36

The bank is strongly required to be responsible for

Francesca 2024-05-19 18:00

Wine runs the Android emulator of Windows. Chrome OS is installed in the Android emulator. Linux environment is installed in chrome OS. Linux environment is installed in the Linux environment. Wine is installed in the Android emulator

Code craftsman 2024-06-01 11:22

I also said "user controllable parameters"

-SORA- 2024-04-30 17:07

When this happened in a foreign country, the comment area suddenly became very objective and rational**

Ma Nong Little Fatty Brother 2024-05-16 14:40

I give you six seconds. I give you six moves with the same effect in the martial arts contest, which shows the invincibility and confidence of the master

Love to eat raw pears 2024-06-01 19:18

Don't expect programmers to have a deep understanding of the document. I still think that since the tool hides the details of $#, some necessary security checks are necessary. Many people do not use MybatisPlus directly, but use various so-called rapid development platforms. The MyBatisPlus rapid development platform Snowy, Guns, etc., has an impression that many versions have the problem of using Wrapper directly to splice the Request parameter. I remember that JeecgBoot was opened a lot of CVEs last year or the year before last because of the Wrapper splicing problem. Do you know the author of ibeetl? Many CVE blaming holes have been opened before. The problem is similar. The lack of basic knowledge "script editing permission" is actively handed over to the front end. What a low-level error or even low-energy behavior. However, I accepted it with an open mind and added a white list check.

Shen Lang Panda 2024-06-01 08:16

You can directly ask questions in the project work order. The comment area is not suitable for answering such questions

young crops 2024-06-01 16:21

There is no tipping point. There are also many official documents stating that SQL fragments involving direct string splicing need to be controlled by the user, and specific solutions are also provided. If you say that the value part is injected, then we are also 100% free of any dispute. This obvious SQL fragment is unrealistic for ORM to explain without your control, Since SQL allows splicing fragments, there must be some scenarios that cannot be forced into non SQL strings. It is also very simple. Have you ever thought about why not force them???

hanf 2024-05-31 17:45

If only the design and architecture are similar, what's the point? Good things must be learned, and you can't prove that the design is not the same. As for the source code, you also said that neither Oracle nor Damon is open source, and you can't prove it. There are many people who question Dream, but so far, no one has come up with strong evidence. You should at least provide evidence to copy

sunday12345 2024-05-15 18:31

What does the bank do? It's blamed on the remote desktop. Persimmons really pick up soft pinches~?

Starry Night Destiny 2024-06-01 21:49

It feels like Mybatis. It's OK to provide users with optional security solutions. It's useless for users to complain about this problem

-SORA- 2024-06-01 09:30

American characters

zzeric 2024-04-28 20:01

Although France is the parent community, the core developers of OCCT on github are all Russians. Without Russians, the French parent community cannot continue to operate. So Huawei took over, moved to China, changed its name and resumed open source and community operations. What's the problem?

The seven in one little King Kong 2024-06-02 15:54

Those people only use resources, others are not developed by NPM...

Voice of God 2024-06-01 20:47

By default, injection ($) and splicing are turned off. If you want to use it, you need to sign the birth and death form and press the fingerprint.

Yeah, for 2024-05-17 13:42

That's too right. Old Zhou can't control Google, but he can control 360. Do not do to others what you do not want. All 360 products should be opened first.

Ning Jinnong 2024-06-01 21:04

Correct it. The example of loading the library is wrong. It should be # library=@ loading the dynamic library, "./yards to the treasurer. dll"

Xiao Xu Middle aged 2024-06-01 07:03

good

Brother Xiao Yang 2024-06-01 20:39

Isn't Ali developed? What are you afraid of? There's no need for every family to set up a set

jalena 2024-05-31 23:57

I can imagine that I will also receive the CVE repair request next week..... I don't use the key!!!!!!!!!

CodeDoger 2024-05-02 20:48

35 It's too old to go to work and too early to retire at 60

kakai 2024-05-10 10:21

The world only knows that Android was created by Google. Several people know that Android is only a product acquired by Google. Similarly, what is the problem with Huawei's contribution to the collection of OGG open source work and integration into its own proprietary product line?

Qin Liming 2024-05-11 09:12

be devoid of any sense of shame

Bright Stars 2 2024-05-31 23:28

Remove Unsafe? You don't want netty anymore?

Li Yinghui 2024-05-09 16:40

Buddhism has a good word, evil opinion. In dealing with the world, it is meaningless to draw conclusions from preset positions; It is also important to receive good logic training.

Yoona520 2024-05-17 16:34

Zhou Hongyi is now living more and more like a clown. If he stays behind the scenes, he has to become an online celebrity. Can you learn from Lei Jun?

Apizza 2024-06-01 17:52

You can switch from lodash to radash in 2024!!!

Happy LeapFrog 2024-05-18 09:18

But the question is: "What's the use of this for ordinary Android users?" Now the answer seems to be: "Almost nothing.".

zhy 2024-05-16 13:16

At the end of Shannon is Nong

People are addicted to food 2024-06-01 13:53

History history combination

Dogo_Little People 2024-06-02 12:24

Not everyone will go to see the document in full detail. As a general basic framework, the method naming should consider not only readability but also understandability. At least, it should also establish a cognition for developers. LambdaQueryWrapper is recommended. The official only briefly said that QueryWrapper may lead to SQL injection risks, There are no detailed examples (many people don't understand what SQL injection is). Now I met a jerk and submitted it to CVE to see who is the most powerful

osc_566335 2024-04-28 14:44

This is also called floor washing? Does it mean that Tesla will not wash the floor if it releases all the source code? Some people HWptds? That is to say, the language is ambiguous, which will also rise to the washing ground? Are some people too focused? Think the people he pays attention to must be staring at?

Databricks open source LLM, training only costs three hours and 30 dollars

Hot content

Popular comments of the whole site

Hot News

Excellent column

How do programmers get started with AI application development?

Talk about Unity and Native Bridging

How to Use Excelize to Process Excel Office Documents Efficiently

Source Code Analysis of Bookie Storage Architecture | Get Things Technology

KubeAI Large Model Reasoning Acceleration Practice | Get Things Technology

Maximizing the Effective Throughput of LLM Serving

Hot software

OSCHINA Community

Online tools

Introduction

QQ group

Public account

Video number

Databricks open source LLM, training only costs three hours and 30 dollars

Related Links

Hot content

Popular comments of the whole site

Hot News

Excellent column

How do programmers get started with AI application development?

Talk about Unity and Native Bridging

How to Use Excelize to Process Excel Office Documents Efficiently

Source Code Analysis of Bookie Storage Architecture | Get Things Technology

KubeAI Large Model Reasoning Acceleration Practice | Get Things Technology

Maximizing the Effective Throughput of LLM Serving

Recommended attention

Hot software

OSCHINA Community

Online tools

Introduction

QQ group

Public account

Video number