Robots taking our jobs

The figures below are from the recent paper Robots and Jobs: Evidence from US Labor Markets, by Acemoglu and Restrepo. VoxEU discussion: … Estimates suggest that an extra robot per 1000 workers reduces the employment to population ratio by 0.18-0.34 percentage points and wages by 0.25-0.5%. This effect is distinct from the impacts of imports, …

More

Don’t Touch the Computer

Under what circumstances should humans override algorithms? From what I have read I doubt that a hybrid team of human + AlphGo would perform much better than AlphaGo itself. Perhaps worse, depending on the epistemic sophistication and self-awareness of the human. In hybrid chess it seems that the ELO score of the human partner is …

More

Super-human Relational Reasoning (DeepMind)

These neural nets reached super-human (better than an average human) performance on tasks requiring relational reasoning. See the short video for examples. A simple neural network module for relational reasoning https://arxiv.org/abs/1706.01427 Adam Santoro, David Raposo, David G.T. Barrett, Mateusz Malinowski, Razvan Pascanu, Peter Battaglia, Timothy Lillicrap (Submitted on 5 Jun 2017) Relational reasoning is a …

More

How the brain does face recognition

  This is a beautiful result. IIUC, these neuroscientists use the terminology “face axis” instead of (machine learning terminology) variation along an eigenface vector or feature vector. Scientific American: …using a combination of brain imaging and single-neuron recording in macaques, biologist Doris Tsao and her colleagues at Caltech have finally cracked the neural code for …

More

Face Recognition applied at scale in China

The Chinese government is not the only entity that has access to millions of faces + identifying information. So do Google, Facebook, Instagram, and anyone who has scraped information from similar social networks (e.g., US security services, hackers, etc.). In light of such ML capabilities it seems clear that anti-ship ballistic missiles can easily target …

More

Rise of the Machines: Survey of AI Researchers

These predictions are from a recent survey of AI/ML researchers. See SSC and also here for more discussion of the results. When Will AI Exceed Human Performance? Evidence from AI Experts Katja Grace, John Salvatier, Allan Dafoe, Baobao Zhang, Owain Evans Advances in artificial intelligence (AI) will transform modern life by reshaping transportation, health, science, …

More

Yann LeCun on Unsupervised Learning

This is a recent Yann LeCun talk at CMU. Toward the end he discusses recent breakthroughs using GANs (Generative Adversarial Networks, see also Ian Goodfellow here and here). LeCun tells an anecdote about the discovery of backpropagation. The first implementation of the algorithm didn’t work, probably because of a bug in the program. But they convinced …

More

History of Bayesian Neural Networks

This talk gives the history of neural networks in the framework of Bayesian inference. Deep learning is (so far) quite empirical in nature: things work, but we lack a good theoretical framework for understanding why or even how. The Bayesian approach offers some progress in these directions, and also toward quantifying prediction uncertainty. I was …

More

Robots Proctor Online Exams

For background on this subject, see How to beat online exam proctoring. It is easy for clever students to beat existing security systems for online exams. Enterprising students could even set up “cheating rooms” that make it easy for test takers to cheat. Judging by the amount of traffic this old post gets, cheating on …

More

AlphaGo (BetaGo?) Returns

Rumors over the summer suggested that AlphaGo had some serious problems that needed to be fixed — i.e., whole lines of play that it pursued poorly, despite its thrashing of one of the world’s top players in a highly publicized match. But tuning a neural net is trickier than tuning, for example, an expert system …

More

Toward A Geometry of Thought

Apologies for the blogging hiatus — I’m in California now for the holidays :-) In case you are looking for something interesting to read, I can share what I have been thinking about lately. In Thought vectors and the dimensionality of the space of concepts (a post from last week) I discussed the dimensionality of the space of …

More

Machine Learning for Personalized Medicine: Heritability-based models for prediction of complex traits (David Balding)

Highly recommended talk by David Balding on modern approaches to heritability, relatedness, etc. in statistical genetics. (I listened at 1.5x normal speed, which worked for me.) MLPM (Machine Learning for Personalized Medicine) Summer School 2015 Monday 21st of September Heritability-based models for prediction of complex traits by David Balding Complex trait genetics has been revolutionised …

More

Over- and Underfitting

I just read a nice post by Jean-François Puget, suitable for readers not terribly familiar with the subject, on overfitting in machine learning. I was going to leave a comment mentioning a couple of things, and then decided that with minimal padding I could make it long enough to be a blog post. I agree …

More

Java “Deep Learning” Library

If you are a Java (or Scala) (or maybe Clojure?) programmer interested in analytics, and in particular machine learning, you should take a look at Deeplearning4j (DL4J). Quoting their web site: Deeplearning4j is the first commercial-grade, open-source, distributed deep-learning library written for Java and Scala. Integrated with Hadoop and Spark, DL4J is designed to be …

More

Theory, Money, and Learning

After 25+ years in theoretical physics research, the pattern has become familiar to me. Talented postdoc has difficulty finding a permanent position (professorship), and ends up leaving the field for finance or Silicon Valley. The final phase of the physics career entails study of entirely new subjects, such as finance theory or machine learning, and developing …

More

Genetic ancestry and brain morphology

Population structure — i.e., distribution of gene variants by ancestral group — is reflected in brain morphology, as measured using MRI. Brain morphology measurements can be used to predict ancestry. Strictly speaking, the data only show correlation, not genetic causation, but the most plausible interpretation is that genetic differences are causing morphological differences. One could …

More

Geoff Hinton on Deep Learning

This is a recent, and fairly non-technical, introduction to Deep Learning by Geoff Hinton. In the most interesting part of the talk (@25 min; see arxiv:1409.3215 and arxiv:1506.00019) he describes extracting “thought vectors” or semantic meaning relationships from plain text. This involves a deep net, human text, and resulting vectors of weights. The slide below summarizes some …

More

Moore’s Law and AI

By now you’ve probably heard that Moore’s Law is really dead. So dead that the semiconductor industry roadmap for keeping it on track has more or less been abandoned: see, e.g., here, here or here. (Reported on this blog 2 years ago!) What I have not yet seen discussed is how a significantly reduced rate …

More

DeepMind and Demis Hassabis

Recent profile in the Guardian; 15 facts about Hassabis. The mastery of Atari games through reinforcement learning deep neural nets is described here (Nature). See also Deep Neural Nets and Go: AlphaGo beats European champion. Guardian: … “We’re really lucky,” says Hassabis, who compares his company to the Apollo programme and Manhattan Project for both …

More

Contemplating the Future

A great profile of Nick Bostrom in the New Yorker. I often run into Nick at SciFoo and other similar meetings. When Nick is around I know there’s a much better chance the discussion will stay on a highbrow, constructive track. It’s surprising how often, even at these heavily screened elitist meetings, precious time gets …

More

Deep Learning in Nature

When I travel I often carry a stack of issues of Nature and Science to read (and then discard) on the plane.The article below is a nice review of the current state of the art in deep neural networks. See earlier posts Neural Networks and Deep Learning 1 and 2, and Back to the Deep. …

More

IQ prediction from structural MRI

These authors use machine learning techniques to build sparse predictors based on grey/white matter volumes of specific regions. Correlations obtained are ~ 0.7 (see figure). I predict that genomic estimators of this kind will be available once ~ 1 million genomes and cognitive scores are available for analysis. See also Myths, Sisyphus and g. MRI-Based …

More

Back to the deep

The Chronicle has a nice profile of Geoffrey Hinton, which details some of the history behind neural nets and deep learning. See also Neural networks and deep learning and its sequel. The recent flourishing of deep neural nets is not primarily due to theoretical advances, but rather the appearance of GPUs and large training data …

More

Neural Networks and Deep Learning

One of the SCI FOO sessions I enjoyed the most this year was a discussion of deep learning by AI researcher Juergen Schmidhuber. For an overview of recent progress, see this recent paper. Also of interest: Michael Nielsen’s pedagogical book project. An application which especially caught my attention is described by Schmidhuber here: Many traditional methods …

More

Minds and Machines

HLMI = ‘high–level machine intelligence’ = one that can carry out most human professions at least as well as a typical human. I’m more pessimistic than the average researcher in the poll. My 95 percent confidence interval has earliest HLMI about 50 years from now, putting me at ~ 80-90th percentile in this group as …

More

The Mystery of Go

Nice article on the progress of computer Go. See also The Laskers and the Go master: “While the baroque rules of Chess could only have been created by humans, the rules of Go are so elegant, organic, and rigorously logical that if intelligent life forms exist elsewhere in the universe, they almost certainly play Go.” …

More