Record your understanding of tensorflow.js

TensorFlow.js This is an open source library that can use JavaScript and advanced layer APIs to define, train and run complete machine learning models in the browser. If you are new to ML, TensorFlow.js is a good way to start learning. Or, if you are a ML developer new to JavaScript, please read on to learn more about new opportunities for ML in browsers. Let's start with it.

Operating conditions

ML running in the browser means that from the user's point of view, no libraries or drivers need to be installed. Just open a web page and your program will run. In addition, it is ready to use GPU to accelerate operation. TensorFlow.js automatically supports WebGL and accelerates code in the background when GPU is available. Users can also open your web page through mobile devices, in which case your model can use sensor data, such as gyroscope or acceleration sensor. Finally, all data is retained on the client, making TensorFlow.js available for low latency scenarios and stronger privacy protection.

How to use it

If you use TensorFlow.js for development, you can consider the following three workflows.

You can import existing pre trained models for reasoning. If you have an existing TensorFlow or Keras model that has been trained offline before, you can convert it to TensorFlow.js format and load it into the browser for prediction.
You can retrain the imported model. You can use transfer learning to enhance the existing model, use the technology called "image retraining", and use a small amount of data collected in the browser for offline training. This is a way to quickly train accurate models, using only a small amount of data.
Create the model directly in the browser. You can also use TensorFlow.js to completely define, train and run models in the browser using Javascript and advanced layer APIs.

Code working process

Let's define a model through a set of layers

 import * as tf from ‘@tensorflow/tfjs’; const model = tf.sequential(); model.add(tf.layers.dense({inputShape: [4], units: 100})); model.add(tf.layers.dense({units: 4})); model.compile({loss: ‘categoricalCrossentropy’,  optimizer: ‘sgd’});

The layers API we use supports Keras layers, including Dense, CNN, LSTM, etc; We can use the same Keras compatible API to train our model

 await model.fit( xData, yData,  { batchSize: batchSize, epochs: epochs });

After the training, the model can be used for prediction

 // Get measurements for a new flower to generate a prediction // The first argument is the data,  and the second is the shape. const inputData = tf.tensor2d([[4.8, 3.0, 1.4, 0.1]], [1, 4]); // Get the highest confidence prediction from our model const result = model.predict(inputData); const winner = irisClasses[result.argMax().dataSync()[0]]; // Display the winner console.log(winner);

Architecture principle

First look at the following figure:

TensorFlow.js is mainly supported by WebGL, and TensorFlow.js provides a high-level API for defining models and a low-level API for linear algebra and automatic differentiation. TensorFlow.js supports importing TensorFlow SavedModels and Keras models.

FAQ

Does TensorFlow.js support Node.js now?
Not yet, but Google has started to write and call TensorFlow's C API for Node.js, which will allow JavaScript to run directly on Node.js and the browser. In addition, this matter has a high priority in TensorFlow.js.

Can I import TensorFlow or Keras models to the browser?
Of course, you can import the TensorFlow SavedModel tutorial: [here ], Tutorial for importing Keras HDF5 models: [here ]

Does it support exporting models?
It is not supported yet, but it will be supported soon, because the work priority of TensorFlow.js is also high.

The relationship between TensorFlow.js and TensorFlow?
The TensorFlow.js API is similar to the TensorFlow Python API. Of course, it is not fully supported yet. Later, it will provide the same API as the Python API

How about the current performance of TensorFlow.js?
Compared with TensorFlow Python with AVX, if it is a prediction task, TensorFlow.js with WebGL will be 1.5 to 2 times slower; If it is a training task, some small models TensorFlow.js with WebGL will be faster, but if it is a large model, it will be about 10 to 15 times slower.

What is the difference between TensorFlow.js and deeplearn.js?
TensorFlow.js is a system developed from deeplearn.js, which provides a complete set of JavaScript tools around deep learning. Deeplearn.js can be considered as the core of TensorFlow.js.

Operating conditions

How to use it

Code working process

Architecture principle

FAQ

Related Links 🔗