It’s what every investor has dreamed of: a way of predicting stock market moves. And according to researchers in the UK, it’s easier than you’d think.
Changes in the US Dow Jones index are presaged by the ebb and flow of Google searches for certain financial keywords.
Better still, acting on these trends could make you a huge profit – at least, until others find the same keywords, of course.
Predictably, publication of this claim in the journal Nature Scientific Reports attracted a lot of attention.
It was also hailed as evidence of the power of so-called Big Data: the extraction of insight from datasets.
Now it looks set to become a classic example of what Big Data can do – but not in a good way.
An analysis of the claim published this month suggests it is an object lesson in how digging into Big Data can trigger an avalanche of dross.
With everyone from politicians to billion-dollar corporations looking to Big Data for answers to big questions, it holds vital lessons for all of us.
And topping the list is: never let go of common sense.
When it first began to make headlines around a decade ago, the exploitation of Big Data was hailed as nothing short of a revolution in how we can make sense of the world. In the face of so much information, there was simply no reason to bother dreaming up intricate theories and then testing them.
"Out with every theory of human behaviour, from linguistics to sociology. Forget taxonomy, ontology and psychology," wrote Chris Anderson, editor-in-chief of Wired. "With enough data, the numbers speak for themselves."
At last month’s World Government Summit in Dubai, the potential of Big Data was held up as the means to solve a host of problems, from delivering local public services to tackling global food security.
But now many of those who have looked to Big Data for insight have found that when left to “speak for themselves”, numbers often spout gibberish.
Take that claim that Google search data can predict stock market moves. The idea isn’t so crazy: market swings reflect changes in sentiment among investors, and these might well show up as changes in the volume of searches for certain financial terms.
But which ones? In their Nature Science Reports study, the researchers came up with a list of about 100 terms they thought might pop up, like "risk" and "investment". They then set computers digging into Google's databases, looking for links between search volumes for each term and changes in the Dow Jones index.
And it worked. The computer found correlations between the two.
But the researchers went further, showing that the best correlation could be turned into hard cash.
Their results showed that a trading strategy of buying when the word “debt” is trending upwards and vice-versa would have generated a profit of more than 300 per cent compared to a simple buy-and-hold strategy.
But the key phrase there is “would have”. The researchers did not make any money themselves - not because they were above such things but because they would have needed a time machine. That’s because all their results were retrospective, revealing only past correlations.
And that’s where the whole idea unravels.
By running a list of 100 terms against past records of the Dow Jones, the researchers risked falling into a classic Big Data trap: finding correlations that are meaningless flukes.
Such correlations are shockingly common, as former Harvard law student Tyler Vigen has shown through his now-celebrated eponymous website.
It contains countless silly but impressively strong correlations found by automatically scouring the web.
For example, Vigen’s computers found that the annual amount of honey produced in the US is very strongly correlated to the number of murders using blunt objects.
Of course, no one with any common sense would fall for such “links”. But that’s the problem with using computers to find them: common sense does not come as standard.
In theory, genuine links might exist between search terms like “debt” and stock market changes. But the only way to find out is to check their predictive power.
Dr Wai Mun Fong, associate professor of finance at the National University of Singapore, has now done precisely that, in research published in the current Journal of Index Investing.
Dr Fong took the same list of about 100 terms used in the original study but this time tested their predictive power in years not examined by the original researchers.
Search terms that proved most strongly correlated with the Dow Jones in one year were used to predict market behaviour in the following year.
Sure enough, Dr Fong confirmed that some Google search terms were correlated with market moves.
But they proved to be “society”, “cancer”, “home” and various others with pretty tenuous links to investor sentiment. Significantly, one search term that failed to predict the market over this new timescale was “debt”.
Dr Fong’s conclusions are blunt: using Big Data to predict stock prices that aren’t based on solid economic theory or properly tested “are doomed to be misguided and futile”.
That such basic warnings are still necessary says much about the spell Big Data is weaving over smart people.
Even Google, that titan of Big Data, has proved vulnerable. In 2008 it unveiled Google Flu Trends (GFT), with the aim of helping predict future flu epidemics through spikes in certain search terms.
To find these keywords, computers had looked for links between 50 million candidates and just 1,200 flu outbreak data points. So big a mismatch is a sure-fire way of finding spurious correlations.
And thus it proved. Like the stock market predictor, GFT made headlines, then mistakes and, ultimately, proved useless. Google threw in the towel in 2015.
Now others are going down the same path. The latest victims of Big Data are hedge funds, who have ploughed huge sums into finding patterns that give them an edge. One industry insider recently told Bloomberg that the “insights” they’re finding in Big Data have a failure rate of about 90 per cent.
Few can resist surveying the towering masses of global data without believing the prospector’s credo “There’s gold in them thar hills”. There is, but it’s buried deep in dross - and simply grabbing a shovel and digging is not the way to find it.
Robert Matthews is visiting professor of science at Aston University, Birmingham, UK
PROFILE OF SWVL
Started: April 2017
Founders: Mostafa Kandil, Ahmed Sabbah and Mahmoud Nouh
Based: Cairo, Egypt
Sector: transport
Size: 450 employees
Investment: approximately $80 million
Investors include: Dubai’s Beco Capital, US’s Endeavor Catalyst, China’s MSA, Egypt’s Sawari Ventures, Sweden’s Vostok New Ventures, Property Finder CEO Michael Lahyani
The Sand Castle
Director: Matty Brown
Stars: Nadine Labaki, Ziad Bakri, Zain Al Rafeea, Riman Al Rafeea
Rating: 2.5/5
Brief scoreline:
Tottenham 1
Son 78'
Manchester City 0
The specs
Engine: 1.5-litre turbo
Power: 181hp
Torque: 230Nm
Transmission: 6-speed automatic
Starting price: Dh79,000
On sale: Now
The biog
Born: Kuwait in 1986
Family: She is the youngest of seven siblings
Time in the UAE: 10 years
Hobbies: audiobooks and fitness: she works out every day, enjoying kickboxing and basketball
The specs
AT4 Ultimate, as tested
Engine: 6.2-litre V8
Power: 420hp
Torque: 623Nm
Transmission: 10-speed automatic
Price: From Dh330,800 (Elevation: Dh236,400; AT4: Dh286,800; Denali: Dh345,800)
On sale: Now
COMPANY%20PROFILE
%3Cp%3E%3Cstrong%3ECompany%20name%3A%3C%2Fstrong%3E%20Revibe%20%0D%3Cbr%3E%3Cstrong%3EStarted%3A%3C%2Fstrong%3E%202022%0D%3Cbr%3E%3Cstrong%3EFounders%3A%3C%2Fstrong%3E%20Hamza%20Iraqui%20and%20Abdessamad%20Ben%20Zakour%20%0D%3Cbr%3E%3Cstrong%3EBased%3A%3C%2Fstrong%3E%20UAE%20%0D%3Cbr%3E%3Cstrong%3EIndustry%3A%3C%2Fstrong%3E%20Refurbished%20electronics%20%0D%3Cbr%3E%3Cstrong%3EFunds%20raised%20so%20far%3A%3C%2Fstrong%3E%20%2410m%20%0D%3Cbr%3E%3Cstrong%3EInvestors%3A%20%3C%2Fstrong%3EFlat6Labs%2C%20Resonance%20and%20various%20others%0D%3C%2Fp%3E%0A
The%20specs
%3Cp%3E%3Cstrong%3EEngine%3A%3C%2Fstrong%3E%203.0-litre%20six-cylinder%20turbo%20(BMW%20B58)%3Cbr%3E%3Cstrong%3EPower%3A%3C%2Fstrong%3E%20340hp%20at%206%2C500rpm%3Cbr%3E%3Cstrong%3ETorque%3A%3C%2Fstrong%3E%20500Nm%20from%201%2C600-4%2C500rpm%3Cbr%3E%3Cstrong%3ETransmission%3A%3C%2Fstrong%3E%20ZF%208-speed%20auto%3Cbr%3E%3Cstrong%3E0-100kph%3A%3C%2Fstrong%3E%204.2sec%3Cbr%3E%3Cstrong%3ETop%20speed%3A%3C%2Fstrong%3E%20267kph%3C%2Fp%3E%0A%3Cp%3E%3Cstrong%3EOn%20sale%3A%3C%2Fstrong%3E%20Now%3Cbr%3E%3Cstrong%3EPrice%3A%3C%2Fstrong%3E%20From%20Dh462%2C189%3Cbr%3E%3Cstrong%3EWarranty%3A%3C%2Fstrong%3E%2030-month%2F48%2C000k%3C%2Fp%3E%0A
A new relationship with the old country
Treaty of Friendship between the United Kingdom of Great Britain and Northern Ireland and the United Arab Emirates
The United kingdom of Great Britain and Northern Ireland and the United Arab Emirates; Considering that the United Arab Emirates has assumed full responsibility as a sovereign and independent State; Determined that the long-standing and traditional relations of close friendship and cooperation between their peoples shall continue; Desiring to give expression to this intention in the form of a Treaty Friendship; Have agreed as follows:
ARTICLE 1 The relations between the United Kingdom of Great Britain and Northern Ireland and the United Arab Emirates shall be governed by a spirit of close friendship. In recognition of this, the Contracting Parties, conscious of their common interest in the peace and stability of the region, shall: (a) consult together on matters of mutual concern in time of need; (b) settle all their disputes by peaceful means in conformity with the provisions of the Charter of the United Nations.
ARTICLE 2 The Contracting Parties shall encourage education, scientific and cultural cooperation between the two States in accordance with arrangements to be agreed. Such arrangements shall cover among other things: (a) the promotion of mutual understanding of their respective cultures, civilisations and languages, the promotion of contacts among professional bodies, universities and cultural institutions; (c) the encouragement of technical, scientific and cultural exchanges.
ARTICLE 3 The Contracting Parties shall maintain the close relationship already existing between them in the field of trade and commerce. Representatives of the Contracting Parties shall meet from time to time to consider means by which such relations can be further developed and strengthened, including the possibility of concluding treaties or agreements on matters of mutual concern.
ARTICLE 4 This Treaty shall enter into force on today’s date and shall remain in force for a period of ten years. Unless twelve months before the expiry of the said period of ten years either Contracting Party shall have given notice to the other of its intention to terminate the Treaty, this Treaty shall remain in force thereafter until the expiry of twelve months from the date on which notice of such intention is given.
IN WITNESS WHEREOF the undersigned have signed this Treaty.
DONE in duplicate at Dubai the second day of December 1971AD, corresponding to the fifteenth day of Shawwal 1391H, in the English and Arabic languages, both texts being equally authoritative.
Signed
Geoffrey Arthur Sheikh Zayed
Europe’s rearming plan
- Suspend strict budget rules to allow member countries to step up defence spending
- Create new "instrument" providing €150 billion of loans to member countries for defence investment
- Use the existing EU budget to direct more funds towards defence-related investment
- Engage the bloc's European Investment Bank to drop limits on lending to defence firms
- Create a savings and investments union to help companies access capital
A State of Passion
Directors: Carol Mansour and Muna Khalidi
Stars: Dr Ghassan Abu-Sittah
Rating: 4/5
Rock in a Hard Place: Music and Mayhem in the Middle East
Orlando Crowcroft
Zed Books
2025 Fifa Club World Cup groups
Group A: Palmeiras, Porto, Al Ahly, Inter Miami.
Group B: Paris Saint-Germain, Atletico Madrid, Botafogo, Seattle.
Group C: Bayern Munich, Auckland City, Boca Juniors, Benfica.
Group D: Flamengo, ES Tunis, Chelsea, Leon.
Group E: River Plate, Urawa, Monterrey, Inter Milan.
Group F: Fluminense, Borussia Dortmund, Ulsan, Mamelodi Sundowns.
Group G: Manchester City, Wydad, Al Ain, Juventus.
Group H: Real Madrid, Al Hilal, Pachuca, Salzburg.
Formula Middle East Calendar (Formula Regional and Formula 4)
Round 1: January 17-19, Yas Marina Circuit – Abu Dhabi
Round 2: January 22-23, Yas Marina Circuit – Abu Dhabi
Round 3: February 7-9, Dubai Autodrome – Dubai
Round 4: February 14-16, Yas Marina Circuit – Abu Dhabi
Round 5: February 25-27, Jeddah Corniche Circuit – Saudi Arabia
LIVING IN...
This article is part of a guide on where to live in the UAE. Our reporters will profile some of the country’s most desirable districts, provide an estimate of rental prices and introduce you to some of the residents who call each area home.
More from Neighbourhood Watch