Daily Archives: 18 September 2020

Can robots write? Machine learning produces dazzling results, but some assembly is still required

by


Shutterstock

Alexandra Louise Uitdenbogerd, RMIT University

You might have seen a recent article from The Guardian written by “a robot”. Here’s a sample:

I know that my brain is not a “feeling brain”. But it is capable of making rational, logical decisions. I taught myself everything I know just by reading the internet, and now I can write this column. My brain is boiling with ideas!

Read the whole thing and you may be astonished at how coherent and stylistically consistent it is. The software used to produce it is called a “generative model”, and they have come a long way in the past year or two.

But exactly how was the article created? And is it really true that software “wrote this entire article”?

How machines learn to write

The text was generated using the latest neural network model for language, called GPT-3, released by the American artificial intelligence research company OpenAI. (GPT stands for Generative Pre-trained Transformer.)

OpenAI’s previous model, GPT-2, made waves last year. It produced a fairly plausible article about the discovery of a herd of unicorns, and the researchers initially withheld the release of the underlying code for fear it would be abused.

But let’s step back and look at what text generation software actually does.

Machine learning approaches fall into three main categories: heuristic models, statistical models, and models inspired by biology (such as neural networks and evolutionary algorithms).

Heuristic approaches are based on “rules of thumb”. For example, we learn rules about how to conjugate verbs: I run, you run, he runs, and so on. These approaches aren’t used much nowadays because they are inflexible.



Read more:
From Twitterbots to VR: 10 of the best examples of digital literature

Writing by numbers

Statistical approaches were the state of the art for language-related tasks for many years. At the most basic level, they involve counting words and guessing what comes next.

As a simple exercise, you could generate text by randomly selecting words based on how often they normally occur. About 7% of your words would be “the” – it’s the most common word in English. But if you did it without considering context, you might get nonsense like “the the is night aware”.

More sophisticated approaches use “bigrams”, which are pairs of consecutive words, and “trigrams”, which are three-word sequences. This allows a bit of context and lets the current piece of text inform the next. For example, if you have the words “out of”, the next guessed word might be “time”.

This happens with the auto-complete and auto-suggest features when we write text messages or emails. Based on what we have just typed, what we tend to type and a pre-trained background model, the system predicts what’s next.

While bigram- and trigram-based statistical models can produce good results in simple situations, the best recent models go to another level of sophistication: deep learning neural networks.

Imitating the brain

Neural networks work a bit like tiny brains made of several layers of virtual neurons.

A neuron receives some input and may or may not “fire” (produce an output) based on that input. The output feeds into neurons in the next layer, cascading through the network.

The first artificial neuron was proposed in 1943 by US neuroscientists Warren McCulloch and Walter Pitts, but they have only become useful for complex problems like generating text in the past five years.

To use neural networks for text, you put words into a kind of numbered index. You can use the number to represent a word, so for example 23,342 might represent “time”.

Neural networks do a series of calculations to go from sequences of numbers at the input layer, through the interconnected “hidden layers” inside, to the output layer. The output might be numbers representing the odds for each word in the index to be the next word of the text.

In our “out of” example, number 23,432 representing “time” would probably have much better odds than the number representing “do”.



Read more:
Friday essay: a real life experiment illuminates the future of books and reading

What’s so special about GPT-3?

GPT-3 is the latest and best of the text modelling systems, and it’s huge. The authors say it has 175 billion parameters, which makes it at least ten times larger than the previous biggest model. The neural network has 96 layers and, instead of mere trigrams, it keeps track of sequences of 2,048 words.

The most expensive and time-consuming part of making a model like this is training it – updating the weights on the connections between neurons and layers. Training GPT-3 would have used about 262 megawatt-hours of energy, or enough to run my house for 35 years.

GPT-3 can be applied to multiple tasks such as machine translation, auto-completion, answering general questions, and writing articles. While people can often tell its articles are not written by human authors, we are now likely to get it right only about half the time.

The robot writer

But back to how the article in The Guardian was created. GPT-3 needs a prompt of some kind to start it off. The Guardian’s staff gave the model instructions and some opening sentences.

This was done eight times, generating eight different articles. The Guardian’s editors then combined pieces from the eight generated articles, and “cut lines and paragraphs, and rearranged the order of them in some places”, saying “editing GPT-3’s op-ed was no different to editing a human op-ed”.

This sounds about right to me, based on my own experience with text-generating software. Earlier this year, my colleagues and I used GPT-2 to write the lyrics for a song we entered in the AI Song Contest, a kind of artificial intelligence Eurovision.

AI song Beautiful the World, by Uncanny Valley.

We fine-tuned the GPT-2 model using lyrics from Eurovision songs, provided it with seed words and phrases, then selected the final lyrics from the generated output.

For example, we gave Euro-GPT-2 the seed word “flying”, and then chose the output “flying from this world that has gone apart”, but not “flying like a trumpet”. By automatically matching the lyrics to generated melodies, generating synth sounds based on koala noises, and applying some great, very human, production work, we got a good result: our song, Beautiful the World, was voted the winner of the contest.

Co-creativity: humans and AI together

So can we really say an AI is an author? Is it the AI, the developers, the users or a combination?

A useful idea for thinking about this is “co-creativity”. This means using generative tools to spark new ideas, or to generate some components for our creative work.

Where an AI creates complete works, such as a complete article, the human becomes the curator or editor. We roll our very sophisticated dice until we get a result we’re happy with.



Read more:
Computing gives an artist new tools to be creative


The Conversation

Alexandra Louise Uitdenbogerd, Senior Lecturer in Computer Science, RMIT University

This article is republished from The Conversation under a Creative Commons license. Read the original article.

Guide to the Classics: The Secret Garden and the healing power of nature

by


StudioCanal

Emma Hayes, Deakin University

Frances Hodgson Burnett’s The Secret Garden has been described as “the most significant children’s book of the 20th century.”

First published in 1911, after being serialised in The American Magazine, it was dismissed by one critic at the time as simple and lacking “plenty of excitement”. The novel is, in fact, a sensitive and complex story, which explores how a relationship with nature can foster our emotional and physical well-being. It also reveals anxieties about national identity at a time of the British Empire, drawing on ideas of Christian Science.

The Secret Garden has been read by generations, remains a fixture on children’s publishing lists today and has inspired several film versions. A new film, starring Colin Firth, Dixie Egerickx and Amir Wilson, updates the story in some ways for modern audiences.

2020 movie still, a tree covered in pink flowers
A scene from the new movie version of the book.
Studiocanal

The book opens as nine-year-old Mary Lennox is discovered abandoned in an Indian bungalow following her parents’ deaths during a cholera outbreak. Burnett depicts India as a site of permissive behaviour, illness and lassitude:

[Mary’s] hair was yellow, and her face was yellow because she had been born in India and had always been ill in one way or another.

Mary is “disagreeable”, “contrary”, “selfish” and “cross”. She makes futile attempts at gardening, planting hibiscus blossoms into mounds of earth. The Ayah tasked with caring for Mary and the other “native servants … always obeyed Mary and gave her her own way in everything.”

On the death of her parents, Mary is sent to live with her reclusive uncle Archibald Craven at Misselthwaite Manor in Yorkshire.

Mary’s arrival in England proves a shock. The “blunt frankness” of the Yorkshire servants – in contrast to those in India – checks her behaviour. Martha Sowerby, an outspoken young housemaid, presents Mary with a skipping rope: Yorkshire good sense triumphing over Mary’s imperial malaise.

Also in the manor is Colin, her 10-year-old cousin. Hidden from Mary, she discovers him after hearing his cries at night.

Colin is unable to walk and believes he will not live to reach adulthood. Sequestered in his bedroom, Colin terrorises his servants with his tantrums: he performs “hysterics” in the model of Gothic femininity.



Read more:
The Yellow Wallpaper: a 19th-century short story of nervous exhaustion and the perils of women’s ‘rest cures’

Transformation

Perhaps the most famous image associated with Burnett’s text is the locked door leading to the eponymous garden.

The first edition of The Secret Garden, published in 1911.
Houghton Library, Harvard University

This walled garden had formerly belonged to Colin’s mother, Lilias Craven. When she died after an accident in the garden, her husband, Archibald, locked the door and buried the key.

After Mary unearths the key, she begins to work in this mysterious, overgrown garden along with Martha’s brother, Dickon. Eventually, she manages to draw Colin out of his room with the help of Dickon, and the garden helps him to recover his strength.

Burnett draws upon the cultural connection between childhood and nature, highlighting Edwardian beliefs about the importance of the garden. Like other Edwardian texts, such as Kenneth Grahame’s The Wind in the Willows (1908) and J. M. Barrie’s Peter Pan in Kensington Gardens (1906), The Secret Garden also explores an English turn-of-the-century interest in paganism and the occult, expressed through the book’s fascination with the Greek god Pan.

Dickon, who shares an affinity with animals and the natural world, is first introduced as he sits under a tree “playing on a rough wooden pipe” reminiscent of Pan’s flute.

Dickon and Mary bathe Colin in a stream.
The garden becomes a space of rejuvenation for the children.
IMDB/Studiocanal

Mary and Colin are both physically and psychically transformed through working in the garden. The stifling rooms and constricting passages of Misselthwaite Manor are contrasted to the freedom of the secret garden.

At first it seemed that green things would never cease pushing their way through the earth, in the grass, in the beds, even in the crevices of the walls. Then the green things began to show buds and the buds began to unfurl and show colour, every shade of blue, every shade of purple, every tint and hue of crimson.



Read more:
B&Bs for birds and bees: transform your garden or balcony into a wildlife haven

The children are healed by gardening in the “fresh wind from the moor”. Both gain weight and strength and lose their pallor. Colin’s gardening suggests mastery of the space as he plants a rose – the floral emblem of England.

Mary is subordinated as Colin’s healing becomes the text’s main focus; Colin gains the ability to walk and – importantly – to win a race against her.

Movie still from 1993, children play in the garden
By interacting with nature, the children grow in strength and in heart.
Warner Bros

‘Just mere thoughts’

The Secret Garden emphasises the power of positive thinking: “thoughts – just mere thoughts – are as powerful as electric batteries – as good for one as sunlight is, or as bad for one as poison”.

This focus on the power of positive thoughts highlights Burnett’s interest in New Thought and Christian Science. New Thought teaches that people can enhance their lives by altering their thought patterns. It was developed by Phineas Parkhurst Quimby in the 19th century, and one of Quimby’s students was Mary Baker Eddy, the founder of Christian Science. While Burnett did not join either religion, she acknowledged that they influenced her work. Both religions often reject mainstream medicine.



Read more:
Why you should know about the New Thought movement

1949 movie still
The garden, seen here in the 1949 movie, captures the New Thought ideals of the healing power of thoughts.
IMDB/MGM

Belief in the healing power of thoughts is reflected as Colin chants about the “magic” of the garden.

The sun is shining – the sun is shining. That is the Magic. The flowers are growing – the roots are stirring. That is the Magic. Being alive is the Magic – being strong is the Magic. The Magic is in me… It’s in every one of us.

The Secret Garden today

Written in a time of British imperial expansion, The Secret Garden’s anxieties around national identity are evident. It draws implicit (and explicit) distinctions between the sickliness and languor of India, and the health and vitality associated with life on the Yorkshire moors.

Yet The Secret Garden still resonates with contemporary audiences. This new adaptation elaborates upon the “magic” associated with the healing power of thoughts, introducing a fantastic element to the story as Mary, Colin and Dickon enter a secret garden filled with otherworldly plants.

Director Marc Munden’s new adaptation also appears to revisit the colonialist emphasis of Burnett’s text. The adaptation shifts the time period during which the film is set to 1947, the year of the Partition of India.

This temporal change suggests an alteration to the original text’s ideas about national identity. While Burnett’s 1911 text considered Britain’s relationship with India at the height of British imperialism, Munden’s adaptation situates the narrative in the period of India gaining independence from Britain.

This new film suggests a desire to ensure The Secret Garden’s continued relevance to today’s audiences, who may be attuned to the book’s colonialist ideologies.

The Secret Garden opens in select cinemas today.The Conversation

Emma Hayes, Academic, School of Communication and Creative Arts, Deakin University

This article is republished from The Conversation under a Creative Commons license. Read the original article.