Python Deep Learning Libraries and Frameworks

In this chapter, we’ll connect deep learning with different libraries and frameworks.

Deep Learning and Theano

If we want to start coding a deep neural network, it’s best to understand how different frameworks, such as Theano, TensorFlow, Keras, and PyTorch, work.

Theano is a Python library that provides a range of functions for building deep networks and training them quickly on our machines.

Theano was developed at the University of Montreal, Canada, under the leadership of deep network pioneer Yoshua Bengio.

Theano allows us to define and evaluate mathematical expressions using vectors and matrices, which are rectangular arrays of numbers.

Technically, both neural networks and input data can be represented as matrices, and all standard network operations can be redefined as matrix operations. This is important because computers can perform matrix operations very quickly.

We can process multiple matrix values in parallel, and if we build a neural network using this underlying structure, we can train very large networks in a reasonable time window using a machine with a GPU.

However, if we use Theano, we must build the deep net from scratch. The library does not provide complete functionality for creating specific types of deep nets.

Instead, we must code every aspect of the deep net, such as the model, layers, activations, training methods, and any special methods to prevent overfitting.

However, the good news is that Theano allows us to build our implementation on top of vector functions, providing us with a highly optimized solution.

There are many other libraries that can extend Theano’s functionality. TensorFlow and Keras can be used as backends for Theano.

Deep Learning with TensorFlow

Google’s TensorFlow is a Python library. This library is a great choice for building commercial-grade deep learning applications.

TensorFlow evolved from another library, DistBelief V2, which is part of the Google Brain project. The library aims to expand the portability of machine learning, enabling research models to be applied to commercial applications.

Much like the Theano library, TensorFlow is based on computational graphs, where nodes represent persistent data or mathematical operations, and edges represent the flow of data between nodes, which are multidimensional arrays or tensors; hence the name TensorFlow.

The output of one operation or a set of operations is fed as input to the next operation.

Although TensorFlow is designed for neural networks, it also works well with other networks where computation can be modeled as data flow graphs.

TensorFlow also leverages some of Theano’s features, such as common and subexpression elimination, automatic differentiation, and shared and symbolic variables.

Different types of deep networks can be built using TensorFlow, including convolutional nets, autoencoders, RNTNs, RNNs, RBMs, DBMs/MLPs, and more.

However, TensorFlow does not support hyperparameter configuration. For this functionality, we can use Keras.

Deep Learning and Keras

Keras is a powerful, easy-to-use Python library for developing and evaluating deep learning models.

It has a minimalist design that allows us to build a network layer by layer, train it, and run it.

It encapsulates the efficient numerical computation libraries Theano and TensorFlow, allowing us to define and train neural network models in just a few lines of code.

It is a high-level neural network API that facilitates the widespread use of deep learning and artificial intelligence. It runs on top of several low-level libraries, including TensorFlow, Theano, and others. Keras code is portable; we can implement a neural network in Keras using Theano or TensorFlow as the backend without making any code changes.