{"id":154274,"date":"2019-08-27T07:59:10","date_gmt":"2019-08-27T11:59:10","guid":{"rendered":"https:\/\/www.countingpips.com\/?p=154274"},"modified":"2019-08-27T07:46:23","modified_gmt":"2019-08-27T11:46:23","slug":"introduction-to-artificial-neural-networks-in-python","status":"publish","type":"post","link":"https:\/\/www.investmacro.com\/forex\/2019\/08\/introduction-to-artificial-neural-networks-in-python\/","title":{"rendered":"Introduction to Artificial Neural Networks in Python"},"content":{"rendered":"<div id=\"inves-3653273084\" class=\"inves-below-title-posts inves-entity-placement\"><div id =\"posts_date_custom\"><div align=\"left\">August 27, 2019<\/div><hr style=\"border: none; border-bottom: 3px solid black;\">\r\n<\/div><\/div><p><strong>By Padmaja Bhagwat, <a href=\"http:\/\/kite.com\" target=\"_blank\" rel=\"noopener noreferrer\">Kite.com<\/a><\/strong><\/p>\n<div class=\"homepage__section\">\n<div class=\"homepage__section__content blog__content\">\n<div class=\"content-block\">\n<p>The Python implementation presented may be found in the Kite repository on <a href=\"https:\/\/github.com\/kiteco\/kite-python-blog-post-code\/tree\/master\/neural-networks-intro\">Github<\/a>.<\/p>\n<h2>Biology inspires the Artificial Neural Network<\/h2>\n<p><i>The Artificial Neural Network (ANN) is an attempt at modeling the information processing capabilities of the biological nervous system<\/i>. The human body is made up of trillions of cells, and the nervous system cells \u2013 called neurons \u2013 are specialized to carry \u201cmessages\u201d through an electrochemical process. The nodes in ANN are equivalent to those of our neurons, whose nodes are connected to each other by <i>Synaptic Weights (or simply weights)<\/i> \u00a0\u2013 equivalent to the synaptic connections between axons and dendrites of the biological neuron.<\/p>\n<\/div>\n<div class=\"image-block\"><img decoding=\"async\" class=\" ls-is-cached lazyloaded\" src=\"https:\/\/kite.com\/wp-content\/uploads\/2019\/05\/image8.png\" data-src=\"https:\/\/kite.com\/wp-content\/uploads\/2019\/05\/image8.png\" \/><\/div>\n<div class=\"content-block\">\n<p>Let\u2019s think of a scenario where you\u2019re teaching a toddler how to identify different kinds of animals. You know that they can\u2019t simply identify any animal using basic characteristics like a color range and a pattern: just because an animal is within a range of colors and has black vertical stripes and a slightly elliptical shape doesn\u2019t automatically make it a tiger.<\/p>\n<p>Instead, you should show them many different pictures, and then teach the toddler to identify those features in the picture on their own, hopefully without much of a conscious effort. This specific ability of the human brain to identify features and memorize associations is what inspired the emergence of ANNs.<\/p>\n<h2>What is an Artificial Neural Network?<\/h2>\n<p>In simple terms, an artificial neural network is a set of connected input and output units in which each connection has an associated weight. During the learning phase, the network learns by adjusting the weights in order to be able to predict the correct class label of the input tuples. Neural network learning is also referred to as connectionist learning, referencing the connections between the nodes. In order to fully understand how the artificial neural networks work, let\u2019s first look at some early design approaches.<\/p>\n<h3>What can an Artificial Neural Network do?<\/h3>\n<p>Today, instead of designing a standardized solutions to general problems, we focus on providing a personalized, customized solution to specific situations. For instance, when you log in to any e-commerce website, it\u2019ll provide you with personalized product recommendations based on your previous purchase, items on your wishlist, most frequently clicked items, and so on.<\/p>\n<p>The platform is essentially analyzing the user\u2019s behavior pattern and then recommending the solution accordingly; solutions like these can be effectively designed using Artificial Neural Networks.<\/p>\n<p>ANNs have been successfully applied in wide range of domains such as:<\/p>\n<ul>\n<li><b>Classification of data \u2013 Is this flower a rose or tulip?<\/b><\/li>\n<li><b>Anomaly detection \u2013 Is the particular user activity on the website a potential fraudulent behavior? <\/b><\/li>\n<li><strong>Speech recognition<\/strong> \u2013 Hey Siri! Can you tell me a joke?<\/li>\n<li><strong>Audio generation<\/strong> \u2013 Jukedeck, can you compose an uplifting folk song?<\/li>\n<li><strong>Time series analysis<\/strong> \u2013 Is it good time to start investing in stock market?<\/li>\n<\/ul>\n<p>And the list goes on\u2026<\/p>\n<h3>Early model of ANN<\/h3>\n<h4>The McCulloch-Pitts model of Neuron (1943 model)<\/h4>\n<p>This model is made up of a basic unit called Neuron. The main feature of their Neuron model is that a weighted sum of input signals is compared against a threshold to determine the neuron output. When the sum is greater than or equal to the threshold, the output is 1. When the sum is less than the threshold, the output is 0. \u00a0It can be put into the equations as such:<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-medium wp-image-1708 ls-is-cached lazyloaded\" src=\"https:\/\/kite.com\/wp-content\/uploads\/2019\/05\/Equation-300x73.png\" sizes=\"auto, (max-width: 300px) 100vw, 300px\" srcset=\"https:\/\/kite.com\/wp-content\/uploads\/2019\/05\/Equation-300x73.png 300w, https:\/\/kite.com\/wp-content\/uploads\/2019\/05\/Equation-352x85.png 352w, https:\/\/kite.com\/wp-content\/uploads\/2019\/05\/Equation.png 360w\" alt=\"McCulloch-Pitts Model of Neuron\" width=\"300\" height=\"73\" data-src=\"\/wp-content\/uploads\/2019\/05\/Equation-300x73.png\" data-srcset=\"https:\/\/kite.com\/wp-content\/uploads\/2019\/05\/Equation-300x73.png 300w, https:\/\/kite.com\/wp-content\/uploads\/2019\/05\/Equation-352x85.png 352w, https:\/\/kite.com\/wp-content\/uploads\/2019\/05\/Equation.png 360w\" data-sizes=\"(max-width: 300px) 100vw, 300px\" \/><\/p>\n<\/div>\n<div class=\"content-block\">\n<p>This function <code>f<\/code> which is also referred to as an <b>activation function or transfer function<\/b> is depicted in the figure below, where T stands for the threshold.<\/p>\n<\/div>\n<div class=\"image-block\"><img decoding=\"async\" class=\" ls-is-cached lazyloaded\" src=\"https:\/\/kite.com\/wp-content\/uploads\/2019\/05\/transfer-function.png\" data-src=\"https:\/\/kite.com\/wp-content\/uploads\/2019\/05\/transfer-function.png\" \/><\/div>\n<div class=\"content-block\">\n<p>The figure below depicts the overall McCulloch-Pitts Model of Neuron.<\/p>\n<\/div>\n<div class=\"image-block\"><img decoding=\"async\" class=\" ls-is-cached lazyloaded\" src=\"https:\/\/kite.com\/wp-content\/uploads\/2019\/05\/image3.a849ed72.png\" data-src=\"https:\/\/kite.com\/wp-content\/uploads\/2019\/05\/image3.a849ed72.png\" \/><\/div>\n<div class=\"content-block\">\n<p>Let\u2019s start by designing the simplest Artificial Neural Network that can mimic the basic logic gates. On the left side, you can see the mathematical implementation of a basic logic gate, and on the right-side, the same logic is implemented by allocating appropriate weights to the neural network.<\/p>\n<\/div>\n<div class=\"image-block\"><img decoding=\"async\" class=\" ls-is-cached lazyloaded\" src=\"https:\/\/kite.com\/wp-content\/uploads\/2019\/05\/neural-network.jpg\" data-src=\"https:\/\/kite.com\/wp-content\/uploads\/2019\/05\/neural-network.jpg\" \/><\/div>\n<div class=\"content-block\">\n<p>If you give the first set of inputs to the network i.e. (0, 0) it gets multiplied by the weights of the network to get the sum as follows: <code>(0*1) + (0*1) = 0 (refer eq. 1)<\/code>. Here, the sum, 0, is less than the threshold, 0.5, hence the output will be 0 (refer eq. 2).<\/p>\n<p>Whereas, for the second set of inputs (1,0), the sum <code>(1*1) + (0*1) = 1<\/code> is greater than the threshold, 0.5, hence the output will be 1.<\/p>\n<p>Similarly, you can try any different combination of weights and thresholds to design the neural network depicting <code>AND gate<\/code> and <code>NOT gate<\/code> as shown below.<\/p>\n<\/div>\n<div class=\"image-block\"><img decoding=\"async\" class=\" ls-is-cached lazyloaded\" src=\"https:\/\/kite.com\/wp-content\/uploads\/2019\/05\/neural-network-in-python.jpg\" data-src=\"https:\/\/kite.com\/wp-content\/uploads\/2019\/05\/neural-network-in-python.jpg\" \/><\/div>\n<div class=\"content-block\">\n<p>This way, the McCulloch-Pitts model demonstrates that networks of these neurons could, in principle, compute any arithmetic or logical function.<\/p>\n<\/div>\n<div class=\"banner animation active\">\n<div class=\"banner__title\">Want to Code <strong>Faster<\/strong>?<\/div>\n<div class=\"banner-content\">\n<div class=\"banner-content__info text__centered\">Kite is a plugin for PyCharm, Atom, Vim, VSCode, Sublime Text, and IntelliJ that uses machine learning to provide you with code completions in real time sorted by relevance.<\/div>\n<\/div>\n<\/div>\n<div class=\"content-block\">\n<h2>Perceptron model<\/h2>\n<p>This is the simplest type of neural network that helps with linear (or binary) classifications of data. The figure below shows the linearly separable data.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"size-full wp-image-1725 aligncenter ls-is-cached lazyloaded\" src=\"https:\/\/kite.com\/wp-content\/uploads\/2019\/05\/Perceptron-model.png\" alt=\"perceptron model artificial neural networks\" width=\"273\" height=\"241\" data-src=\"\/wp-content\/uploads\/2019\/05\/Perceptron-model.png\" \/><\/p>\n<p><b>The learning rule<\/b> for training the neural network was first introduced with this model. In addition to the variable weight values, the perceptron added an extra input that represents <i>bias<\/i>. Thus, the equation 1 was modified as follows:<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-medium wp-image-1724 ls-is-cached lazyloaded\" src=\"https:\/\/kite.com\/wp-content\/uploads\/2019\/05\/equation2-300x58.png\" sizes=\"auto, (max-width: 300px) 100vw, 300px\" srcset=\"https:\/\/kite.com\/wp-content\/uploads\/2019\/05\/equation2-300x58.png 300w, https:\/\/kite.com\/wp-content\/uploads\/2019\/05\/equation2.png 308w\" alt=\"artificial neural networks equation\" width=\"300\" height=\"58\" data-src=\"\/wp-content\/uploads\/2019\/05\/equation2-300x58.png\" data-srcset=\"https:\/\/kite.com\/wp-content\/uploads\/2019\/05\/equation2-300x58.png 300w, https:\/\/kite.com\/wp-content\/uploads\/2019\/05\/equation2.png 308w\" data-sizes=\"(max-width: 300px) 100vw, 300px\" \/><\/p>\n<p>Bias is used to adjust the output of the neuron along with the weighted sum of the inputs. It\u2019s just like the intercept added in a linear equation.<\/p>\n<\/div>\n<div class=\"image-block\"><img decoding=\"async\" class=\" ls-is-cached lazyloaded\" src=\"https:\/\/kite.com\/wp-content\/uploads\/2019\/05\/image8.9f349345.png\" data-src=\"https:\/\/kite.com\/wp-content\/uploads\/2019\/05\/image8.9f349345.png\" \/><\/div>\n<div class=\"content-block\">\n<h3>Multilayer perceptron model<\/h3>\n<p>A perceptron that as a single layer of weights can only help in linear or binary data classifications. What if the input data is not linearly separable, as shown in figure below?<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"size-full wp-image-1727 aligncenter ls-is-cached lazyloaded\" src=\"https:\/\/kite.com\/wp-content\/uploads\/2019\/05\/Multilayer-perceptron-model.png\" alt=\"Multilayer perceptron model\" width=\"264\" height=\"245\" data-src=\"\/wp-content\/uploads\/2019\/05\/Multilayer-perceptron-model.png\" \/><\/p>\n<p>This is when we use a multilayer perceptron with a non-linear activation function such as sigmoid.<\/p>\n<\/div>\n<div class=\"content-block\">\n<p>Multilayer perceptron has three main components:<\/p>\n<ul>\n<li><b>Input layer: This layer accepts the input features. Note that this layer does not perform any computation \u2013 it just passes on the input data (features) to the hidden layer.<\/b><\/li>\n<li><b>Hidden layer: This layer performs all sorts of computations on the input features and transfers the result to the output layer. There can be one or more hidden layers.<\/b><\/li>\n<li><b>Output layer: This layer is responsible for producing the final result of the model.<\/b><\/li>\n<\/ul>\n<\/div>\n<div class=\"image-block\"><img decoding=\"async\" class=\" ls-is-cached lazyloaded\" src=\"https:\/\/kite.com\/wp-content\/uploads\/2019\/05\/image10.49614d3c.png\" data-src=\"https:\/\/kite.com\/wp-content\/uploads\/2019\/05\/image10.49614d3c.png\" \/><\/div>\n<div class=\"content-block\">\n<p>Now that we\u2019ve discussed the basic architecture of a neural network, let\u2019s understand how these networks are trained.<\/p>\n<h2>Training phase of a neural network<\/h2>\n<p>Training a neural network is quite similar to teaching a toddler how to walk. In the beginning, when she is first trying to learn, she\u2019ll naturally make mistakes as she learns to stand on her feet and walk gracefully.<\/p>\n<p>Similarly, in the initial phase of training, neural networks tend to make a lot of mistakes. Initially, the predicted output could be stunningly different from the expected output. This difference in predicted and expected outputs is termed as an \u2018<i>error<\/i>\u2019.<\/p>\n<p>The entire goal of training a neural network is to minimize this error by adjusting its weights.<\/p>\n<p>This training process consists of three (broad) steps:<\/p>\n<p>1. <b>Initialize the weights <\/b><\/p>\n<p>The weights in the network are initialized to small random numbers (e.g., ranging from -1 to 1, or -0.5 to 0.5). Each unit has a bias associated with it, and the biases are similarly initialized to small random numbers.<\/p>\n<\/div>\n<div class=\"code-block python\">\n<pre><code class=\"Python hljs livecodeserver\"><span class=\"hljs-function\"><span class=\"hljs-keyword\">def<\/span> <span class=\"hljs-title\">initialize_weights<\/span><span class=\"hljs-params\">()<\/span>:<\/span>\r\n    <span class=\"hljs-comment\"># Generate random numbers<\/span>\r\n    random.seed(<span class=\"hljs-number\">1<\/span>)\r\n\r\n    <span class=\"hljs-comment\"># Assign random weights to a 3 x 1 matrix<\/span>\r\n    synaptic_weights = random.uniform(low=<span class=\"hljs-number\">-1<\/span>, high=<span class=\"hljs-number\">1<\/span>, size=(<span class=\"hljs-number\">3<\/span>, <span class=\"hljs-number\">1<\/span>))\r\n    <span class=\"hljs-keyword\">return<\/span> synaptic_weights<\/code><\/pre>\n<\/div>\n<div class=\"content-block\">\n<p>2. <b>Propagate the input forward<\/b><\/p>\n<p>In this step, the weighted sum of input values is calculated, and the result is passed to an activation function \u2013 say, a sigmoid activation function \u2013 which squeezes the sum value to a particular range (in this case, between 0 to 1), further adding bias with it. This decides whether a neuron should be activated or not.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-medium wp-image-1730 ls-is-cached lazyloaded\" src=\"https:\/\/kite.com\/wp-content\/uploads\/2019\/05\/equation3-300x73.png\" sizes=\"auto, (max-width: 300px) 100vw, 300px\" srcset=\"https:\/\/kite.com\/wp-content\/uploads\/2019\/05\/equation3-300x73.png 300w, https:\/\/kite.com\/wp-content\/uploads\/2019\/05\/equation3.png 349w\" alt=\"artificial neural networks \" width=\"300\" height=\"73\" data-src=\"\/wp-content\/uploads\/2019\/05\/equation3-300x73.png\" data-srcset=\"https:\/\/kite.com\/wp-content\/uploads\/2019\/05\/equation3-300x73.png 300w, https:\/\/kite.com\/wp-content\/uploads\/2019\/05\/equation3.png 349w\" data-sizes=\"(max-width: 300px) 100vw, 300px\" \/><\/p>\n<\/div>\n<div class=\"image-block\"><img decoding=\"async\" class=\" ls-is-cached lazyloaded\" src=\"https:\/\/kite.com\/wp-content\/uploads\/2019\/05\/sigmoid-function.png\" data-src=\"https:\/\/kite.com\/wp-content\/uploads\/2019\/05\/sigmoid-function.png\" \/><\/div>\n<div class=\"content-block\">\n<p>Our sigmoid utility functions are defined like so:<\/p>\n<\/div>\n<div class=\"code-block python\">\n<pre><code class=\"Python hljs livecodeserver\"><span class=\"hljs-function\"><span class=\"hljs-keyword\">def<\/span> <span class=\"hljs-title\">sigmoid<\/span><span class=\"hljs-params\">(x)<\/span>:<\/span>\r\n    <span class=\"hljs-keyword\">return<\/span> <span class=\"hljs-number\">1<\/span> \/ (<span class=\"hljs-number\">1<\/span> + exp(-x))\r\n\r\n\r\n<span class=\"hljs-function\"><span class=\"hljs-keyword\">def<\/span> <span class=\"hljs-title\">sigmoid_derivative<\/span><span class=\"hljs-params\">(x)<\/span>:<\/span>\r\n    <span class=\"hljs-keyword\">return<\/span> x * (<span class=\"hljs-number\">1<\/span> - x)<\/code><\/pre>\n<\/div>\n<div class=\"content-block\">\n<p>3.\u00a0 <b>Backpropagate the error<\/b><\/p>\n<p>In this step, we first calculate the error, i.e., the difference between our predicted output and expected output. Further, the weights of the network are adjusted in such a way that during the next pass, the predicted output is much closer to the expected output, thereby reducing the error.<\/p>\n<p>For neuron <code>j<\/code> (also referred to as <i>unit j<\/i>) of the output layer, the error is computed as follows:<\/p>\n<\/div>\n<div class=\"code-block python\">\n<pre><code class=\"Python hljs livecodeserver\">Errj = Oj*(<span class=\"hljs-number\">1<\/span> \u2013 Oj )*( Tj \u2013 Oj ) \u2026\u2026\u2026\u2026\u2026\u2026.. (<span class=\"hljs-number\">5<\/span>)<\/code><\/pre>\n<\/div>\n<div class=\"content-block\">\n<p>Where <code>Tj<\/code>is the expected output, <code>Oj<\/code>is the predicted output and <code>Oj *(1 \u2013 Oj)<\/code> is the derivative of sigmoid function.<\/p>\n<p>The weights and biases are updated to reflect the back propagated error.<\/p>\n<\/div>\n<div class=\"code-block python\">\n<pre><code class=\"Python hljs livecodeserver\">Wij = Wij + (l*Errij*Oj ) \u2026\u2026\u2026\u2026\u2026\u2026\u2026\u2026\u2026. (<span class=\"hljs-number\">6<\/span>)\r\nbi = bj + (l* Errij) \u2026\u2026\u2026\u2026\u2026\u2026\u2026\u2026\u2026\u2026\u2026\u2026.  (<span class=\"hljs-number\">7<\/span>)<\/code><\/pre>\n<\/div>\n<div class=\"content-block\">\n<p>Above, <code>l<\/code> is the learning rate, a constant typically varying between 0 to 1. It decides the rate at which the value of weights and bias should vary. If the learning rate is high, then the weights and bias will vary drastically with each epoch. If it\u2019s too low, then the change will be very slow.<\/p>\n<p>We terminate the training process when our model\u2019s predicted output is almost same as the expected output. Steps 2 and 3 are repeated until one of the following terminating conditions is met:<\/p>\n<ul>\n<li>The error is minimized to the least possible value<\/li>\n<li>The training has gone through the maximum number of iterations<\/li>\n<li>There is no further reduction in error value<\/li>\n<li>The training error is almost same as that of validation error<\/li>\n<\/ul>\n<p>So, let\u2019s create a simple interface that allows us to run the training process:<\/p>\n<\/div>\n<div class=\"code-block python\">\n<pre><code class=\"Python hljs livecodeserver\"><span class=\"hljs-function\"><span class=\"hljs-keyword\">def<\/span> <span class=\"hljs-title\">learn<\/span><span class=\"hljs-params\">(inputs, synaptic_weights, bias)<\/span>:<\/span>\r\n     <span class=\"hljs-keyword\">return<\/span> sigmoid(dot(inputs, synaptic_weights) + bias)\r\n\r\n<span class=\"hljs-function\"><span class=\"hljs-keyword\">def<\/span> <span class=\"hljs-title\">train<\/span><span class=\"hljs-params\">(inputs, expected_output, synaptic_weights, bias, learning_rate, training_iterations)<\/span>:<\/span>\r\n     <span class=\"hljs-keyword\">for<\/span> epoch <span class=\"hljs-keyword\">in<\/span> range(training_iterations):\r\n          <span class=\"hljs-comment\"># Forward pass -- Pass the training set through the network.<\/span>\r\n          predicted_output = learn(inputs, synaptic_weights, bias)\r\n\r\n        <span class=\"hljs-comment\"># Backaward pass<\/span>\r\n        <span class=\"hljs-comment\"># Calculate the error<\/span>\r\n        error = sigmoid_derivative(predicted_output) * (expected_output - predicted_output)\r\n\r\n        <span class=\"hljs-comment\"># Adjust the weights and bias by a factor<\/span>\r\n        weight_factor = dot(inputs.T, error) * learning_rate\r\n        bias_factor = error * learning_rate\r\n\r\n        <span class=\"hljs-comment\"># Update the synaptic weights<\/span>\r\n        synaptic_weights += weight_factor\r\n\r\n        <span class=\"hljs-comment\"># Update the bias<\/span>\r\n        bias += bias_factor\r\n\r\n        <span class=\"hljs-keyword\">if<\/span> ((epoch % <span class=\"hljs-number\">1000<\/span>) == <span class=\"hljs-number\">0<\/span>):\r\n            print(<span class=\"hljs-string\">\"Epoch\"<\/span>, epoch)\r\n            print(<span class=\"hljs-string\">\"Predicted Output = \"<\/span>, predicted_output.T)\r\n            print(<span class=\"hljs-string\">\"Expected Output = \"<\/span>, expected_output.T)\r\n            print()\r\n    <span class=\"hljs-keyword\">return<\/span> synaptic_weights<\/code><\/pre>\n<\/div>\n<div class=\"content-block\">\n<h2>Bringing it all together<\/h2>\n<p>Finally, we can train the network and see the results using the simple interface created above. You\u2019ll find the complete code in the <a href=\"https:\/\/github.com\/kiteco\/kite-python-blog-post-code\/tree\/master\/neural-networks-intro\">Kite repository<\/a>.<\/p>\n<\/div>\n<div class=\"code-block python\">\n<pre><code class=\"Python hljs livecodeserver\"><span class=\"hljs-comment\"># Initialize random weights for the network<\/span>\r\n    synaptic_weights = initialize_weights()\r\n\r\n    <span class=\"hljs-comment\"># The training set<\/span>\r\n    inputs = array([[<span class=\"hljs-number\">0<\/span>, <span class=\"hljs-number\">1<\/span>, <span class=\"hljs-number\">1<\/span>],\r\n                    [<span class=\"hljs-number\">1<\/span>, <span class=\"hljs-number\">0<\/span>, <span class=\"hljs-number\">0<\/span>],\r\n                    [<span class=\"hljs-number\">1<\/span>, <span class=\"hljs-number\">0<\/span>, <span class=\"hljs-number\">1<\/span>]])\r\n\r\n    <span class=\"hljs-comment\"># Target set<\/span>\r\n    expected_output = array([[<span class=\"hljs-number\">1<\/span>, <span class=\"hljs-number\">0<\/span>, <span class=\"hljs-number\">1<\/span>]]).T\r\n\r\n    <span class=\"hljs-comment\"># Test set<\/span>\r\n    test = array([<span class=\"hljs-number\">1<\/span>, <span class=\"hljs-number\">0<\/span>, <span class=\"hljs-number\">1<\/span>])\r\n\r\n    <span class=\"hljs-comment\"># Train the neural network<\/span>\r\n    trained_weights = train(inputs, expected_output, synaptic_weights, bias=<span class=\"hljs-number\">0.001<\/span>, learning_rate=<span class=\"hljs-number\">0.98<\/span>,\r\n                            training_iterations=<span class=\"hljs-number\">1000000<\/span>)\r\n\r\n    <span class=\"hljs-comment\"># Test the neural network with a test example<\/span>\r\n    accuracy = (learn(test, trained_weights, bias=<span class=\"hljs-number\">0.01<\/span>)) * <span class=\"hljs-number\">100<\/span>\r\n\r\n    print(<span class=\"hljs-string\">\"accuracy =\"<\/span>, accuracy[<span class=\"hljs-number\">0<\/span>], <span class=\"hljs-string\">\"%\"<\/span>)<\/code><\/pre>\n<\/div>\n<div class=\"content-block\">\n<h2>Conclusion<\/h2>\n<p>You now have seen a sneak peek into Artificial Neural Networks! Although the mathematics behind training a neural network might have seemed a little intimidating at the beginning, you can now see how easy it is to implement them using Python.<\/p>\n<p>In this post, we\u2019ve learned some of the fundamental correlations between the logic gates and the basic neural network. We\u2019ve also looked into the Perceptron model and the different components of a multilayer perceptron.<\/p>\n<p>In my upcoming post, I\u2019m going to talk about different types of artificial neural networks and how they can be used in your day-to-day applications. Python is well known for its rich set of libraries like Keras, Scikit-learn, and Pandas to name a few \u2013 which abstracts out the intricacies involved in data manipulation, model building, training the model, etc. We shall be seeing how to use these libraries to build some of the cool applications. This post is an introduction to some of the basic concepts involved in building these models before we dive into using libraries.<\/p>\n<h3>Try it yourself<\/h3>\n<p>The best way of learning is by trying it out on your own, so here are some questions you can try answering using the concepts we learned in this post:<\/p>\n<ol>\n<li>Can you build an XOR model by tweaking the weights and thresholds?<\/li>\n<li>Try adding more than one hidden layer to the neural network, and see how the training phase changes.<\/li>\n<\/ol>\n<p>See you in the next post!<\/p>\n<\/div>\n<\/div>\n<\/div>\n<p><em><strong>About the Author:<\/strong><\/em><\/p>\n<p><a href=\"https:\/\/kite.com\/blog\/python\/artificial-neural-networks\/\" target=\"_blank\" rel=\"noopener noreferrer\">This article<\/a> originally appeared on <a href=\"https:\/\/kite.com\" target=\"_blank\" rel=\"noopener noreferrer\">Kite.com<\/a>.<\/p>\n<p>(Reprinted with permission)<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n","protected":false},"excerpt":{"rendered":"<p>By Padmaja Bhagwat, Kite.com The Python implementation presented may be found in the Kite repository on Github. Biology inspires the Artificial Neural Network The Artificial Neural Network (ANN) is an attempt at modeling the information processing capabilities of the biological nervous system. The human body is made up of trillions of cells, and the nervous [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[],"tags":[],"class_list":["post-154274","post","type-post","status-publish","format-standard","hentry","no-post-thumbnail"],"_links":{"self":[{"href":"https:\/\/www.investmacro.com\/forex\/wp-json\/wp\/v2\/posts\/154274","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.investmacro.com\/forex\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.investmacro.com\/forex\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.investmacro.com\/forex\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.investmacro.com\/forex\/wp-json\/wp\/v2\/comments?post=154274"}],"version-history":[{"count":2,"href":"https:\/\/www.investmacro.com\/forex\/wp-json\/wp\/v2\/posts\/154274\/revisions"}],"predecessor-version":[{"id":154276,"href":"https:\/\/www.investmacro.com\/forex\/wp-json\/wp\/v2\/posts\/154274\/revisions\/154276"}],"wp:attachment":[{"href":"https:\/\/www.investmacro.com\/forex\/wp-json\/wp\/v2\/media?parent=154274"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.investmacro.com\/forex\/wp-json\/wp\/v2\/categories?post=154274"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.investmacro.com\/forex\/wp-json\/wp\/v2\/tags?post=154274"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}