{
"cells": [
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"# This cell is added by sphinx-gallery\n# It can be customized to whatever you like\n%matplotlib inline"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Turning quantum nodes into Keras Layers\n=======================================\n\n*Author: PennyLane dev team. Posted: 2 Nov 2020. Last updated: 28 Jan\n2021.*\n\nCreating neural networks in [Keras](https://keras.io/) is easy. Models\nare constructed from elementary *layers* and can be trained using a\nhigh-level API. For example, the following code defines a two-layer\nnetwork that could be used for binary classification:\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"import tensorflow as tf\n\ntf.keras.backend.set_floatx('float64')\n\nlayer_1 = tf.keras.layers.Dense(2)\nlayer_2 = tf.keras.layers.Dense(2, activation=\"softmax\")\n\nmodel = tf.keras.Sequential([layer_1, layer_2])\nmodel.compile(loss=\"mae\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The model can then be trained using\n[model.fit()](https://www.tensorflow.org/api_docs/python/tf/keras/Model#fit).\n\n**What if we want to add a quantum layer to our model?** This is\npossible in PennyLane: QNodes <../glossary/hybrid\\_computation>\ncan be converted into Keras layers and combined with the wide range of\nbuilt-in classical\n[layers](https://www.tensorflow.org/api_docs/python/tf/keras/layers) to\ncreate truly hybrid models. This tutorial will guide you through a\nsimple example to show you how it's done!\n\n> **note**\n>\n> A similar demo explaining how to\n> turn quantum nodes into Torch layers <tutorial\\_qnn\\_module\\_torch>\n> is also available.\n\nFixing the dataset and problem\n==============================\n\nLet us begin by choosing a simple dataset and problem to allow us to\nfocus on how the hybrid model is constructed. Our objective is to\nclassify points generated from scikit-learn's binary-class\n[make\\_moons()](https://scikit-learn.org/stable/modules/generated/sklearn.datasets.make_moons.html)\ndataset:\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"import matplotlib.pyplot as plt\nimport numpy as np\nfrom sklearn.datasets import make_moons\n\n# Set random seeds\nnp.random.seed(42)\ntf.random.set_seed(42)\n\nX, y = make_moons(n_samples=200, noise=0.1)\ny_hot = tf.keras.utils.to_categorical(y, num_classes=2) # one-hot encoded labels\n\nc = [\"#1f77b4\" if y_ == 0 else \"#ff7f0e\" for y_ in y] # colours for each class\nplt.axis(\"off\")\nplt.scatter(X[:, 0], X[:, 1], c=c)\nplt.show()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Defining a QNode\n================\n\nOur next step is to define the QNode that we want to interface with\nKeras. Any combination of device, operations and measurements that is\nvalid in PennyLane can be used to compose the QNode. However, the QNode\narguments must satisfy additional conditions\n<code/api/pennylane.qnn.KerasLayer> including having an argument\ncalled `inputs`. All other arguments must be arrays or tensors and are\ntreated as trainable weights in the model. We fix a two-qubit QNode\nusing the\ndefault.qubit <code/api/pennylane.devices.default\\_qubit.DefaultQubit>\nsimulator and operations from the\ntemplates <introduction/templates> module.\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"import pennylane as qml\n\nn_qubits = 2\ndev = qml.device(\"default.qubit\", wires=n_qubits)\n\n@qml.qnode(dev)\ndef qnode(inputs, weights):\n qml.templates.AngleEmbedding(inputs, wires=range(n_qubits))\n qml.templates.BasicEntanglerLayers(weights, wires=range(n_qubits))\n return [qml.expval(qml.PauliZ(wires=i)) for i in range(n_qubits)]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Interfacing with Keras\n======================\n\nWith the QNode defined, we are ready to interface with Keras. This is\nachieved using the \\~pennylane.qnn.KerasLayer class of the\n\\~pennylane.qnn module, which converts the QNode to the elementary\nbuilding block of Keras: a *layer*. We shall see in the following how\nthe resultant layer can be combined with other well-known neural network\nlayers to form a hybrid model.\n\nWe must first define the `weight_shapes` dictionary. Recall that all of\nthe arguments of the QNode (except the one named `inputs`) are treated\nas trainable weights. For the QNode to be successfully converted to a\nlayer in Keras, we need to provide the details of the shape of each\ntrainable weight for them to be initialized. The `weight_shapes`\ndictionary maps from the argument names of the QNode to corresponding\nshapes:\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"n_layers = 6\nweight_shapes = {\"weights\": (n_layers, n_qubits)}"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"In our example, the `weights` argument of the QNode is trainable and has\nshape given by `(n_layers, n_qubits)`, which is passed to\n\\~pennylane.templates.layers.BasicEntanglerLayers.\n\nNow that `weight_shapes` is defined, it is easy to then convert the\nQNode:\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"qlayer = qml.qnn.KerasLayer(qnode, weight_shapes, output_dim=n_qubits)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"With this done, the QNode can now be treated just like any other Keras\nlayer and we can proceed using the familiar Keras workflow.\n\nCreating a hybrid model\n=======================\n\nLet's create a basic three-layered hybrid model consisting of:\n\n1. a 2-neuron fully connected classical layer\n2. our 2-qubit QNode converted into a layer\n3. another 2-neuron fully connected classical layer\n4. a softmax activation to convert to a probability vector\n\nA diagram of the model can be seen in the figure below.\n\n![](/demonstrations/qnn_module/qnn_keras.png){width=\"100%\"}\n\nWe can construct the model using the\n[Sequential](https://www.tensorflow.org/api_docs/python/tf/keras/Sequential)\nAPI:\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"clayer_1 = tf.keras.layers.Dense(2)\nclayer_2 = tf.keras.layers.Dense(2, activation=\"softmax\")\nmodel = tf.keras.models.Sequential([clayer_1, qlayer, clayer_2])"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Training the model\n==================\n\nWe can now train our hybrid model on the classification dataset using\nthe usual Keras approach. We'll use the standard\n[SGD](https://www.tensorflow.org/api_docs/python/tf/keras/optimizers/SGD)\noptimizer and the mean absolute error loss function:\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"opt = tf.keras.optimizers.SGD(learning_rate=0.2)\nmodel.compile(opt, loss=\"mae\", metrics=[\"accuracy\"])"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Note that there are more advanced combinations of optimizer and loss\nfunction, but here we are focusing on the basics.\n\nThe model is now ready to be trained!\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"fitting = model.fit(X, y_hot, epochs=6, batch_size=5, validation_split=0.25, verbose=2)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"How did we do? The model looks to have successfully trained and the\naccuracy on both the training and validation datasets is reasonably\nhigh. In practice, we would aim to push the accuracy higher by thinking\ncarefully about the model design and the choice of hyperparameters such\nas the learning rate.\n\nCreating non-sequential models\n==============================\n\nThe model we created above was composed of a sequence of classical and\nquantum layers. This type of model is very common and is suitable in a\nlot of situations. However, in some cases we may want a greater degree\nof control over how the model is constructed, for example when we have\nmultiple inputs and outputs or when we want to distribute the output of\none layer into multiple subsequent layers.\n\nSuppose we want to make a hybrid model consisting of:\n\n1. a 4-neuron fully connected classical layer\n2. a 2-qubit quantum layer connected to the first two neurons of the\n previous classical layer\n3. a 2-qubit quantum layer connected to the second two neurons of the\n previous classical layer\n4. a 2-neuron fully connected classical layer which takes a\n 4-dimensional input from the combination of the previous quantum\n layers\n5. a softmax activation to convert to a probability vector\n\nA diagram of the model can be seen in the figure below.\n\n![](/demonstrations/qnn_module/qnn2_keras.png){width=\"100%\"}\n\nThis model can also be constructed using the [Functional\nAPI](https://keras.io/guides/functional_api/):\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"# re-define the layers\nclayer_1 = tf.keras.layers.Dense(4)\nqlayer_1 = qml.qnn.KerasLayer(qnode, weight_shapes, output_dim=n_qubits)\nqlayer_2 = qml.qnn.KerasLayer(qnode, weight_shapes, output_dim=n_qubits)\nclayer_2 = tf.keras.layers.Dense(2, activation=\"softmax\")\n\n# construct the model\ninputs = tf.keras.Input(shape=(2,))\nx = clayer_1(inputs)\nx_1, x_2 = tf.split(x, 2, axis=1)\nx_1 = qlayer_1(x_1)\nx_2 = qlayer_2(x_2)\nx = tf.concat([x_1, x_2], axis=1)\noutputs = clayer_2(x)\n\nmodel = tf.keras.Model(inputs=inputs, outputs=outputs)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"As a final step, let's train the model to check if it's working:\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"opt = tf.keras.optimizers.SGD(learning_rate=0.2)\nmodel.compile(opt, loss=\"mae\", metrics=[\"accuracy\"])\n\nfitting = model.fit(X, y_hot, epochs=6, batch_size=5, validation_split=0.25, verbose=2)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Great! We've mastered the basics of constructing hybrid\nclassical-quantum models using PennyLane and Keras. Can you think of any\ninteresting hybrid models to construct? How do they perform on realistic\ndatasets?\n"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.2"
}
},
"nbformat": 4,
"nbformat_minor": 0
}