What is a neural network?

Explaining terms relating to AI, machine learning and neural networks

What is a neural network? What is machine learning? These are terms that are often used interchangeably, but their definitions are quite different. Here we explain what they mean.

How do neural networks, the most popular form of artificial intelligence (AI), work? Defining terms: The terms AI, machine learning and neural networks often appear in articles on new technologies. While these terms share some similarities and are increasingly used synonymously, it is important to differentiate between them. AI is the overarching term for the attempt to teach computers human-like or complex thinking – in contrast to the simple processing of rigid preprogrammed functions. The computer programs itself, learns from experience and is capable of handling unfamiliar situations.

Many technologies fall under the term ‘artificial intelligence’ © bremeninvest/Jann Raveling

Machine learning defined as a subdiscipline of artificial intelligence

There are many ways of achieving this goal. Search engines like Google fall under AI, for example. One particular approach within AI is machine learning, which involves computers learning from experience rather than being explicitly programmed. The computer learns how to use basic data to derive a pattern or a rule that best suits this data and can be applied to new data. Being able to tell an apple from a pear, for example.

This always results in a probability value, as the computer is confronted with new and unfamiliar data. It is the same for humans – we could equally think that an Asian pear is an apple because it looks like one. The difference is that AI makes better decisions than humans do once it has been trained.

Within machine learning, there are a range of approaches to programming self-learning systems. These are divided into supervised and unsupervised learning. In supervised learning, a human tells the computer what it has to learn (“This photo of an apple is an apple. Learn to recognise apples in other images.”). In unsupervised learning, the computer has to find patterns in the data and sort the data itself – but only humans can interpret it.

What is a neural network?

Neural networks fall within supervised learning and are the most topical area of machine learning. They are responsible for the current buzz around AI, as they are particularly good at analysing and finding patterns in unstructured data. By unstructured data we mean images, videos or sound, i.e. the type of data that we produce in high volumes every day.

The term ‘neural’ comes from biology. In the human brain, a neuron is a nerve cell that is linked to other nerve cells and sends electric signals to them. The brain is made up of billions of neurons, and its activity is the foundation of our mind.

Just like a nerve cell, an artificial neuron processes signals and passes on an output © bremeninvest/Jann Raveling

An artificial neural network works in a similar way. Here, the neuron is a mathematical formula that processes an input and generates an output from it. The values in the formula are defined by the initial data. The many artificial neurons working together make up an artificial neural network (ANN).

An example of a neural network

Let’s take the example of a neural network that is designed to tell apples from pears. Images are input into the network where they are broken down pixel by pixel. Every pixel (translated into a colour value) feeds into a neuron at the initial level.

A neural network starts by breaking images down pixel by pixel into values that can be calculated by the neurons © bremeninvest/Jann Raveling

This neuron processes the information (calculates a formula) and passes the result on to the next level of neurons. The final level of neurons corresponds to the two possible outputs – apples or pears. The pointer indicates the side where the sum of the calculation is higher, in this case approaching a value of 1.

The randomly chosen parameters of the individual neurons initially return a random and therefore incorrect answer © bremeninvest/Jann Raveling

The result of the calculation is wrong as our network thinks the apple is a pear. That is because the paths that the information takes through the neural network are initially chosen randomly. Now comes the important step: the randomly chosen values of the individual neurons are adjusted based on the result so that they better match the correct result – but only in very small steps. The network does this automatically using a complex formula.

By comparing the result with the target value, the ANN is able to slowly train its neurons to return the correct answer © bremeninvest/Jann Raveling

This step is then repeated many thousands or millions of times, and not just with one image but with many images of apples and pears. Every time, the values of the neurons change a little. This is the ‘learning’ process and requires huge computing power. At the end of the process, the neurons have been refined to such an extent that they are able to tell images of apples from those of pears, even if they have never seen those images before. They have abstracted a general pattern from the training images.

The ANN has taught itself the distinguishing features, but they do not correspond with human standards. We would use features such as “is round and red” = apple and “is more elongated and green” = pear to tell them apart. Terms such as ‘red’ or ‘elongated’ mean nothing to a computer. It identifies its own distinguishing features, for example “has an average red value of X per cent” or something even more abstract. They are not something that can be looked up in a book; rather they are contained implicitly in the overall formula that is based on all neurons.

The paths that AI takes in reaching a decision, i.e. the reason why it chooses ‘apple’, are not directly transparent to us. That is why the term ‘black box’ is sometimes used in this context, as it is not clear to human intuition how an ANN has reached its decision – a computer does not think in terms of ‘elongated’ or ‘green’.

The path to the decision for every single image can be traced retrospectively, but it is represented by a highly complex mathematical formula that is determined by thousands or even millions of parameters.

Every neural network is only as good as the data it was trained with, so it is vital to use clearly categorised and uniformly structured data to train an ANN. Now it becomes clear why companies like Facebook and Google collect so much data. Having access to vast data pools makes it easier for them to train their ANNs and to analyse the data. That is the reason why companies place such a high value on collecting as much data as possible.

The limitations of neural networks

Neural networks cannot cope if data is fed into them that does not match the type used to train them © bremeninvest/Jann Raveling

Our apples and pears example also shows the limitations of ANNs: if networks are suddenly confronted with images of fire engines, they are unable to process them – or they incorrectly identify them as apples. ANNs must be retrained before they are able to process different data types. The data set needs to provide answers to a very narrow range of questions for neural networks to function properly.

A collection of neural networks

Neural networks can look and work very differently, and the one described above is just one of many. Recently, deep neural networks have come to prominence. These consist of several consecutive levels of neurons and are capable of greater accuracy, similar to our example. Other types include convolutional neural networks, which are used to analyse visual imagery and are already capable of identifying images more accurately than humans. Genetic neural networks take nature as their template. In these models, a number of ANNs compete with each other, instead of one constantly being adjusted using additional data. The best ones are retained, the worst are discarded.

Understanding neural networks better

It is not easy to understand exactly how artificial intelligence works, but there are many websites and projects that have taken on this task. We have made a list of them to help everyone who wants to find out more about the subject.

1. CGP Grey: YouTuber CGP Grey has created a short video that describes how genetic neural networks work in a simple and accessible way: How Machines Learn

2. 3Blue1Brown: This YouTuber explains in three parts how a deep neural network works. The explanation is more complex and involves some maths, but that is helpful to understanding how an image can be translated into a mathematical function, i.e. how AI ‘thinks’: What is a neural network?

3. Play around with neural networks: On http://playground.tensorflow.org/, anyone can have a go at neural networks in their browser. We recommend you read the instructions first as otherwise it is difficult to understand the purpose of each parameter.

4. Play around with neural networks II: Using your own photos, you can train an AI using the Teachable Machine Google app. Simply take three photos and the AI will learn to tell them apart and assign them to different outputs.

Want to find out more about AI? The Bremen.AI network is here to answer all your AI-related questions and provide an initial glimpse into the technology.

Andreas Gerber

Team Leader Bremeninvest

P +49 (0) 421 9600-123

!moc.tsevni-nemerb[AT]rebreg

Andreas Gerber

Team Leader Bremeninvest

P +49 (0) 421 9600-123

!moc.tsevni-nemerb[AT]rebreg

Startups

4 August 2021

What Happens In the Supermarket at Night

This robot's job doesn't start until the last member of staff leaves the store at the end of the working day: Roaming through the shelves, he helps physical stores to compete with online retailers.

Welcome our new robotic night manager

Startups

23 July 2021

The Commercial Cargo Bike is Here to Stay

Cargo bikes are truly multitalented – they are not just appreciated by private individuals, but also those in business. Bremen is supporting the cargo bike in business and trade with a wider range of initiatives. Here is an overview.

Learn more

Investing in Bremen

2 July 2021

Velo Lab: with new premises so it can keep pace with the boom in cargo bikes

The cargo bike business is booming, as is the Bremen bike manufacturer Velo Lab. With a new production facility in the New Harbour District, and innovative bike designs, its founder Stathis Stasinopoulos is truly in tune with the times. A "test drive".

Learn more