Core concepts of tensorflow.js

in Tutorials with 19 comments

If you don't know tensorflow.js, turn right: here Here are some of the core concepts of tensorflow. JS.

Since tensorflow.js is limited to official documents and there is no Chinese information, this article is compiled, collected and translated by linpx

Tensors (tensors)

Tensor is a multilinear function which can be used to express the linear relations among some vectors, scalars and other tensors. The basic examples of these linear relations are inner product, outer product, linear mapping and Cartesian product. In the   n   dimensional space, there are   n ^ R components of a quantity, in which each component is a function of coordinates, and in coordinate transformation, these components are also linearly transformed according to some rules. R is called the rank or order of the tensor (independent of the rank and order of the matrix).

Tensorflow. JS's data unit is a tensor: a set of values stored in a one-dimensional or multidimensional array. An instance of a tensor has the attribute shape, which is used to construct a multidimensional array. The main constructor of tensor is tf.tensor

 // 2x3 Tensor
const shape = [2, 3]; // 2 rows, 3 columns
const a = tf.tensor([1.0, 2.0, 3.0, 10.0, 20.0, 30.0], shape);
a.print(); // print Tensor values
// Output: [[1 , 2 , 3 ],
//          [10, 20, 30]]

// The shape can also be inferred:
const b = tf.tensor([[1.0, 2.0, 3.0], [10.0, 20.0, 30.0]]);
b.print();
// Output: [[1 , 2 , 3 ],
//           [10, 20, 30]]

But sometimes, to facilitate the construction of simpler tensors, it is recommended to use tf.scalar , tf.tensor1d , tf.tensor2d , tf.tensor3d and tf.tensor4d This will also make the code more readable. The following example uses the tf.tensor2d

 const c = tf.tensor2d([[1.0, 2.0, 3.0], [10.0, 20.0, 30.0]]);
c.print();
// Output: [[1 , 2 , 3 ],
//          [10, 20, 30]]

Tensorflow. JS also provides the tf.zeros Function and build value 1 tf.ones Function of

 // 3x5 Tensor with all values set to 0
const zeros = tf.zeros([3, 5]);
// Output: [[0, 0, 0, 0, 0],
//          [0, 0, 0, 0, 0],
//          [0, 0, 0, 0, 0]]

Once you create them in tensor, they can't be changed.

Variables (variables)

Variables are initialized by the value of tensor. Unlike tensors, variables are variable. You can go through assign Use existing variables to declare a new tensor.

 const initialValues = tf.zeros([5]);
const biases = tf.variable(initialValues); // initialize biases
biases.print(); // output: [0, 0, 0, 0, 0]

const updatedValues = tf.tensor1d([0, 1, 0, 1, 0]);
biases.assign(updatedValues); // update values of biases
biases.print(); // output: [0, 1, 0, 1, 0]

Variables are mainly used to store or update values during training.

Operations(Ops)

Use tensors to store data, and operations allows you to use that data. Tensorflow. JS provides a set of operation functions for linear algebra and machine learning. Because tensors are immutable, the OPS will not be modified to their values, only a new tensors will be returned.

The following is the square calculation of the value in the tensor:

 const d = tf.tensor2d([[1.0, 2.0], [3.0, 4.0]]);
const d_ squared = d.square();
d_ squared.print();
// Output: [[1, 4 ],
//          [9, 16]]

You can also add two tensors:

 const e = tf.tensor2d([[1.0, 2.0], [3.0, 4.0]]);
const f = tf.tensor2d([[5.0, 6.0], [7.0, 8.0]]);

const e_ plus_ f = e.add(f);
e_ plus_ f.print();
// Output: [[6 , 8 ],
//          [10, 12]]

Of course, tensorflow.js OPS supports Chaining:

 const sq_ sum = e.add(f).square();
sq_ sum.print();
// Output: [[36 , 64 ],
//          [100, 144]]

// All operations are also exposed as functions in the main namespace,
// so you could also do the following:
const sq_ sum = tf.square(tf.add(e, f));

Models and layers

In terms of the concept, models can be regarded as input and output functions.

In tensorflow. JS, there are two ways to create models. You can use OPS to create a usable model directly, as follows:

 // Define function
function predict(input) {
  // y = a * x ^ 2 + b * x + c
  // More on tf.tidy in the next section
  return tf.tidy(() => {
    const x = tf.scalar(input);

    const ax2 = a.mul(x.square());
    const bx = b.mul(x);
    const y = ax2.add(bx).add(c);

    return y;
  }); 
}

// Define constants: y = 2x^2 + 4x + 8
const a = tf.scalar(2);
const b = tf.scalar(4);
const c = tf.scalar(8);

// Predict output for input of 2
const result = predict(2);
result.print() // Output: 24

You can also use a higher level API tf.model From the current level of deep learning, this is the popular level of learning. use tf.sequential To build models:

 const model = tf.sequential();
model.add(
  tf.layers.simpleRNN({
    units: 20,
    recurrentInitializer: 'GlorotNormal',
    inputShape: [80, 4]
  })
);

const optimizer = tf.train.sgd(LEARNING_ RATE);
model.compile({optimizer, loss: 'categoricalCrossentropy'});
model.fit({x: data, y: labels)});

There are many different types of layers in tensorflow. JS, including tf.layers.simpleRNN , tf.layers.gru and tf.layers.lstm

Memory Management

Because tensorflow.js uses GPU to speed up mathematical operations, it needs to manage GPU memory at runtime.

Tensorflow.js provides two functions to help handle GPU memory: dispose and tf.tidy

dispose

You can call dispose To clear the occupied GPU memory:

 const x = tf.tensor2d([[0.0, 2.0], [4.0, 6.0]]);
const x_ squared = x.square();

x.dispose();
x_ squared.dispose();

tf.tidy

Call the dispose It's going to be a lot of trouble. That's when it's needed tf.tidy It is somewhat JavaScript like scope for, but only for GPU backed tensors.

tf.tidy After execution, all newly created tensors will be cleared to free GPU memory, but the internal return value will not be eliminated.

 // tf.tidy takes a function to tidy up after
const average = tf.tidy(() => {
  // tf.tidy will clean up all the GPU memory used by tensors inside
  // this function, other than the tensor that is returned.
  //
  // Even in a short sequence of operations like the one below,  a number
  // of intermediate tensors get created. So it is a good practice to
  // put your math ops in a tidy!
  const y = tf.tensor1d([1.0, 2.0, 3.0, 4.0]);
  const z = tf.ones([4]);

  return y.sub(z).square().mean();
});

average.print() // Output: 3.5

use tf.tidy Helps prevent memory leaks in applications. Of course, it can be used to precisely control when memory is recycled.

Two important tips

Other information

Responses
  1. wwaf

    Homeboy benefits: https://6wll.com

    Reply
    1. van
      @wwaf

      You just don't like it

      Reply
  2. Thanks for sharing!

    Reply
  3. No one wrote a personal blog, the previous attention is rarely updated

    Reply
  4. zich

    Website certificate prompt invalid

    Reply
  5. yanghb

    I like learning~~~

    Reply
  6. Hello blogger, how to set the QRcode at the bottom of the article page? I used your theme, but QR doesn't know how to set it.

    Reply