NEURAL NETWORKS
***Disclaimer: the subject of neural networks is quite extensive & complex. The overview & analyses below covers only a brief, simplistic model***
Overview: Neural networks are a machine learning model inspired by the structure and function of the human brain. Biological neurons are cells that translate electrical signals through complex networks throughout the body [9]. Similarly, the complex networks that make up ML neural networks consist of interconnected layers of artificial neurons, which are simple computational units that take in one or more inputs and produce an output. Each neuron applies a mathematical function to its inputs and passes the result on to the next layer of neurons.
Like many of the supervised models discussed, neural networks are typically used for tasks such as classification and regression, with a goal of predicting a target variable based on a set of input features. During training, the network adjusts the weights between the neurons to minimize the difference between its predictions and the true values. This is done using an algorithm called back propagation, which calculates the gradient of the loss function with respect to the weights and adjusts them accordingly.
Getty Images
Neural networks are known for their ability to learn complex patterns in data, and have been successfully applied in a wide (and ever-growing) range of fields, including image recognition, natural language processing, and artificial inteligence. While NN's offer extreme processing power, they can be computationally expensive to train, and can require large amounts of data and computational resources to achieve proper performance.
Neural networks play a significant role in the context of applied text data. This is particularly relevant in modern technologies with the recent boom in generative AI and large language models- both of which use natural language processing. In text data, NN's are used for a wide range of tasks, such as text classification, sentiment analysis, language translation, and text generation.
Predicting Article Stance on GMO's
Many of the explorations in this project provided somewhat limited distinction-ability, due to the significant amount of overlap between the terminology used in arguments for and against GMO’s. Many models found similar subject matter and topics of discussion in supporting and objecting text, and therefore, predicting sentiment, and classifying stance has proved difficult. However, neural networks delve, much deeper, and identify patterns well beyond simply matching terminology. For this reason, they are particularly powerful in identifying sentiment.
In the following exploration, simple neural networks will be used to determine the stance (i.e. pro GMO or anti-GMO) of articles and blog posts.
Data Prep: Test data was scraped from aticles and blogs from various online sources that tend to either support or object GMO's. Articles and blogs from these sources were curated into a 'PRO GMO' and an 'ANTI GMO' corpus. The sources listed below were selected based on their known leaning and their popularity among readers.
Sources With Generally PRO-GMO Stance:
Genetic Literacy Project: This website provides articles and resources on genetics, biotechnology, and GMOs, with a focus on science-based information.
GMO Answers: This is a website run by the Council for Biotechnology Information, which is a trade association representing biotech companies. The site provides information and answers to common questions about GMOs.
Biofortified: This is a non-profit organization that promotes evidence-based information about genetic engineering and biotechnology in agriculture.
Alliance for Science: This is a global initiative based at Cornell University that aims to promote access to scientific information about biotechnology and its potential to address global challenges.
Sources With Generally ANTI-GMO Stance:
Natural News: This website promotes alternative medicine and natural health, and often publishes articles that are critical of GMOs.
GMO Free USA: This organization advocates for GMO labeling and transparency, and provides information on the potential risks and negative impacts of GMOs.
Food Babe: This website is run by a popular blogger and food activist who is vocal about the potential risks associated with consuming GMOs and other processed foods.
Organic Consumers Association: This organization promotes organic farming and food production, and is critical of GMOs and industrial agriculture.
Non-GMO Project: A non-profit organization that is generally opposed to GMO's in food and agriculture, and advocates for increased transparency in labeling of GMOs in products
The text data consists of 70 articles (35 Pro and 35 anti). The text was cleaned in order to remove excess characters, symbols, and numbers. Then it was tokenized using count, vectorize, or in order to create a vocabulary. The vocabulary was generated by curating, a list of all the unique tokens in the full data set. These tokens were then transferred to a numerical representation via “embedding.” The original, text-containing data set, was then split into training, test, and validation subsets to be fed into neural networks. Because there were only 70 articles the original dataset, 80% were randomly selected for testing and the remaining 20% were split between test and validation (7 each).
- Artificial Neural Network -
First, a simple ANN was fit to the training and test data with an initial 10 epochs. When predicting the validation data, this model yielded a fairly poor accuracy so the model was smoother by applying dropout and increasing the epoch to 20. This yielded a slightly improved accuracy:
57.14% Accuracy
- Recursive Neural Network -
Due to the nature of the data (text data, where sentence structure/order) matters, the use case is better applied to a recursive neural network.* A simple RNN was fit to the training and test data with 20 epochs. It then predicted on the validation set with:
85.71% Accuracy
*RNNs are well-suited for sequential and time-dependent data, capturing temporal dependencies, and modeling sequences. They excel in tasks where the order of the data points is significant (such as sentences and the order of words). For this reason they are commonly used for natural language processing. For this reason, a recurrent neural network was effective in capturing the differences between pro-GMO and anti-GMO documents, that other, more simplistic models failed to see. This emphasizes the similar nature of the opposing sides of the GMO discussion in subject matter. In fact, many of the same impacts (ex: biodiversity, health, pesticides, etc) are heavily discussed by both sides with differences in the beliefs, opinions and implications of these factors.
