ICML notes

It’s never been a better time to work on AI/ML. Vast resources are being deployed in this direction, by corporations and governments alike. In addition to the marvelous practical applications in development, a theoretical understanding of Deep Learning may emerge in the next few years. The notes below are to keep track of some interesting …

More

Usefulness of Computer Science: An Example

I thought I would follow up on my June 29 post, “Does Computer Science Help with OR?“, by giving a quick example of how exposure to fundamentals of computer science recently helped me. A current research project involves optimization models containing large numbers of what are basically set covering constraints, constraints of the form \(\displaystyle …

More

Does Computer Science Help with OR?

Fair warning: tl/dr. After reading a blog post yesterday by John D. Cook, “Does computer science help you program?“, I decided to throw in my two cents (convert to euros at your own risk) on a related topic: does computer science (which I will extend to including programming) help you as an OR/IE/management science/analytics professional? …

More

Evolution of a (data) visualization

Last summer, I taught the MAET Year 2 Summer Cohort with Danah Henriksen. After teaching the class, Danah realized she had taught five cohorts of (awesome) students and that we had some information available from pre- and post-course self-reported surveys to understand how students grew in terms of their confidence in using different educational (and …

More

Callback Cuts That Repeat

The following post is specific to the CPLEX integer programming solver. I have no idea whether it applies to other solvers, or even which other solver have cut callbacks. Every so often, a user will discover that a callback routine they wrote has “rediscovered” a cut it previously generated. This can be a bit concerning …

More

A Brief History of the (Near) Future: How AI and Genomics Will Change What It Means To Be Human

I’ll be giving the talk below to an audience of oligarchs in Los Angeles next week. This is a video version I made for fun. It cuts off at 17min even though the whole talk is ~25min, because my team noticed that I gave away some sensitive information :-( The slides are here. A Brief …

More

Big Tech compensation in 2018

I don’t work in Big Tech so I don’t know whether his numbers are realistic. If they are realistic, then I’d say careers in Big Tech (for someone with the ability to do high level software work) dominate all the other (risk-adjusted) options right now. This includes finance, startups, etc. No wonder the cost of …

More

How NSA Tracks You (Bill Binney)

Anyone who is paying attention knows that the Obama FBI/DOJ used massive government surveillance powers against the Trump team during and after the election. A FISA warrant on Carter Page (and Manafort and others?) was likely used to mine stored communications of other Trump team members. Hundreds of “mysterious” unmasking requests by Susan Rice, Samantha …

More

Genetic testing and embryo selection: current status and ethical issues

This is a conversation with two Stanford students about the current status of genetic testing of embryos in IVF, focusing on related ethical issues. Because there is a lot of interest in this topic I suggested we record the conversation and put it online. I was at Stanford last fall to give a #nofilter talk …

More

AI and Genomics, explained (2 videos)

This video is a nicely done short introduction to AI for non-specialists. It’s part of Shift Change, a six part series on automation and the future of work. I came across the video when creator Joss Fong (Vox) contacted me about her new project on human genomics and genomic prediction. As readers know I think the …

More

US Needs a National AI Strategy: A Sputnik Moment?

The US needs a national AI strategy. Many academic researchers that could contribute to AI research — including to fundamental new ideas and algorithms, mathematical frameworks for better understanding why some algorithms and architectures work better than others, etc. — are not able to get involved at the real frontier because they lack the kind …

More

Quantum Computing near a Tipping Point?

I received an email from a physicist colleague suggesting that we might be near a “tipping point” in quantum computation. I’ve sort of followed quantum computation and quantum information as an outsider for about 20 years now, but haven’t been paying close attention recently because it seems that practical general purpose quantum computers are still …

More

Nature, Nurture, and Invention: analysis of Finnish data

What is the dominant causal mechanism for the results shown above? Is it that better family environments experienced by affluent children make them more likely to invent later in life? Is it that higher income fathers tend to pass on better genes (e.g., for cognitive ability) to their children? Obviously the explanation has important implications …

More

Recursive Cortical Networks: data efficient computer vision

Will knowledge from neuroscience inform the design of better AIs (neural nets)? These results from startup Vicarious AI suggest that the answer is yes! (See also this company blog post describing the research.) It has often been remarked that evolved biological systems (e.g., a baby) can learn much faster and using much less data than existing artificial neural nets. …

More

How Europe lost its tech companies

Some perspectives from a Berlin tech guy who has also worked in China. To some extent Europe is like the Midwest of the US: a source of human capital for SV and other places. Europe and the Midwest have strong universities and produce talented individuals, but lack a mature tech ecosystem which includes access to venture …

More

Creating A New MIME Type

I struggled a bit this afternoon creating a new MIME type and associating it with a particular application, so I’m going to archive the solution here for future reference. This was on a Linux Mint system, but I found the key information in a GNOME documentation page, so I suspect it works for Ubuntu and …

More

CMSE (Computational Mathematics, Science and Engineering) at MSU

At Oregon I was part of an interdisciplinary institute that included theoretical physicists and chemists, mathematicians, and computer scientists. We tried to create a program (not even a new department, just an interdisciplinary program) in applied math and computation, but failed due to lack of support from higher administration. When I arrived at MSU as …

More

In a previous post Half of all jobs (> $60k/y) coding related? I wrote In the future there will be two kinds of jobs. Workers will either Tell computers what to do or Be told by computers what to do I’ve been pushing Michigan State University to offer a coding bootcamp experience to all undergraduates who want … More Behold, the Super Cow Hmm… how do they compute the Net Merit and GTPI? (But, but, what about all of that missing heritability?) See also Applied genomics: the genetic “super cow” Genomic prediction: no bull. Attention climate virtue signalers: more efficient cows produce less methane per liter of milk! Drink milk from genetically engineered cows :-) Tweet Benders Decomposition with Generic Callbacks Brace yourself. This post is a bit long-winded (and arguably geekier than usual, which is saying something). Also, it involves CPLEX 12.8, which will not ship until some time next month. I have an updated version of an old example, solving a fixed charge transportation problem using Benders decomposition. The example (using Java, naturally) is … More 23andme I’m in Mountain View to give a talk at 23andMe. Their latest funding round was$250M on a (reported) valuation of \$1.5B. If I just add up the Crunchbase numbers it looks like almost half a billion invested at this point… Slides: Genomic Prediction of Complex Traits Abstract: We apply methods from Compressed Sensing (L1-penalized regression; Donoho-Tanner …

More

As I noted in yesterday’s post, one of the major changes associated with the new “generic” callback structure in CPLEX is that users now bear the responsibility of making their callbacks thread-safe. As I also noted yesterday, this is pretty new stuff for me. So I’m going to try to share what I know about thread …

More

CPLEX 12.8: Generic Callbacks

IBM is getting ready to release CPLEX 12.8, and I had the opportunity to attend a presentation about by Xavier Nodet at the 2017 INFORMS annual meeting. Here are links to two presentations by Xavier: CPLEX Optimization Studio 12.8 – What’s New and CPLEX 12.8 – the Generic Callback. As with any new release, there …

More

The Future is Here: Genomic Prediction in MIT Technology Review

MIT Technology Review reports on our startup Genomic Prediction. Some basic points worth clarifying: 1. GP’s first product, announced at the annual ASRM (American Society of Reproductive Medicine) meeting this week, tests chromosomal abnormality. It is a less expensive but more accurate version of existing tests. 2. The polygenic product, to be launched in 2018, checks …

More

What “R” qualitative research methods?

I recently stumbled upon a post on R-bloggers entitled “Qualitative Research in R.” This title got me pretty excited, since I’m generally excited about most things R and since I recently helped teach a qualitative methods course, which has had me thinking about adding more ethnographic and other qualitative elements to my work. I’d also heard of …

More

The Physicist and the Neuroscientist: A Tale of Two Connectomes

This is video of an excellent talk on the human connectome by neuroscientist Bobby Kasthuri of Argonne National Lab and the University of Chicago. (You can see me sitting on the floor in the corner :-) The story below is for entertainment purposes only. No triggering of biologists is intended. The Physicist and the Neuroscientist: A …

More

AlphaGo Zero: algorithms over data and compute

AlphaGo Zero was trained entirely through self-play — no data from human play was used. The resulting program is the strongest Go player ever by a large margin, and is extremely efficient in its use of compute (running on only 4 TPUs). Previous versions of AlphaGo initially trained on thousands of human amateur and professional games …

More

Blade Runner 2049: Demis Hassabis (Deep Mind) interviews director Villeneuve

Hassabis refers to AI in the original Blade Runner, but it is apparent from the sequel that replicants are merely genetically engineered humans. AI appears in Blade Runner 2049 in the form of Joi. There seems to be widespread confusion, including in the movie itself, about whether to think about replicants as robots (i.e., hardware) …

More

Information Theory of Deep Neural Nets: “Information Bottleneck”

This talk discusses, in terms of information theory, how the hidden layers of a deep neural net (thought of as a Markov chain) create a compressed (coarse grained) representation of the input information. To date the success of neural networks has been a mainly empirical phenomenon, lacking a theoretical framework that explains how and why …

More

Where are participants in American and Canadian teacher hashtags?

My dissertation research is focused on Regional Educational Twitter Hashtags (RETHs), which are teacher-focused hashtags that are associated with particular geographic regions, such as American states or Canadian provinces or territories. This isn’t the first time that I’ve done a project on this phenomenon, and it’s rewarding to come back to RETHs to answer some questions that …

More

A Gentle Introduction to Neural Networks

“A gentle introduction to the principles behind neural networks, including backpropagation. Rated G for general audiences.” This very well done. If you have a quantitative background you can watch it at 1.5x or 2x speed, I think :-) A bit more on the history of backpropagation and convexity: why is the error function convex, or nearly …

More

Phase Transitions and Genomic Prediction of Cognitive Ability

James Thompson (University College London) recently blogged about my prediction that with sample size of order a million genotypes|phenotypes, one could construct a good genomic predictor for cognitive ability and identify most of the associated common SNPs. The Hsu Boundary … The “Hsu boundary” is Steve Hsu’s estimate that a sample size of roughly 1 …

More

Public data and digital research ethics

The Verge recently posted an article that highlights some of the ethical dilemmas involved in collecting publicly-available data for research purposes. The article begins by describing the work of a researcher working on facial recognition of people before and after hormone replacement therapy: On YouTube, he found a treasure trove. Individuals undergoing HRT often document their progress …

More

BENEFICIAL AI 2017 (Asilomar meeting)

AI researcher Yoshua Bengio gives a nice overview of recent progress in Deep Learning, and provides some perspective on challenges that must be overcome to achieve AGI (i.e., human-level general intelligence). I agree with Bengio that the goal is farther than the recent wave of excitement might lead one to believe. There were many other …

More

DeepMind and StarCraft II Learning Environment

This Learning Environment will enable researchers to attack the problem of building an AI that plays StarCraft II at a high level. As observed in the video, this infrastructure development required significant investment of resources by DeepMind / Alphabet. Now, researchers in academia and elsewhere have a platform from which to explore an important class …

More

Normies Lament

Ezra Klein talks to Angela Nagle. It’s still normie normative, but Nagle has at least done some homework. Click the link below to hear the podcast. From 4Chan to Charlottesville: where the alt-right came from, and where it’s going Angela Nagle spent the better part of the past decade in the darkest corners of the …

More

A couple of podcasts on screencasting

I’ve posted before about teaching CEP 813, a class on electronic assessment that features a unit on game-based assessment in Minecraft. This unit is by far the most intense in terms of technical support, and we had a major hiccup earlier this month that caused some frustration for the whole class (and instructional team). After …

More

Swiss accents and using the Internet as a French teacher

Last week, on August 1st, I popped over to Radio Télévision Suisse to spend a couple of minutes celebrating the Swiss national holiday. While I was there, I spotted an article containing five “spoken portraits” of Swiss Francophones from different regions. Each portrait highlighted a different accent (or two) from Francophone Switzerland, and it was …

More

Helpful resources for principal components analysis in R

I’m currently working on my dissertation proposal, which has meant exploring principal components analysis. I’ve worked with PCA before, but it’s been a couple of years, so I’m trying to refresh my memory, improve my understanding, and get this proposal moving! Along the way, I’ve found (and been recommended) some helpful resources that I thought …

More

China’s rise in Science and Engineering indicators (NSF)

Data from the 2016 NSF report on global Science & Engineering Indicators shows the rapid rise of China in both academic science and applied technology. Rapid growth in number of Chinese S&E articles, reaching parity with US in 2013, and well ahead of Japan and India. Fraction of high impact (top 1% most cited) papers highest for …

More

Robots taking our jobs

The figures below are from the recent paper Robots and Jobs: Evidence from US Labor Markets, by Acemoglu and Restrepo. VoxEU discussion: … Estimates suggest that an extra robot per 1000 workers reduces the employment to population ratio by 0.18-0.34 percentage points and wages by 0.25-0.5%. This effect is distinct from the impacts of imports, …

More

Like little monkeys: How the brain does face recognition

This is a Caltech TEDx talk from 2013, in which Doris Tsao discusses her work on the neuroscience of human face recognition. Recently I blogged about her breakthrough in identifying the face recognition algorithm used by monkey (and presumably human) brains. The algorithm seems similar to those used in machine face recognition: individual neurons perform …

More

First Human Embryos Edited in U.S. (MIT Technology Review)

It’s only a matter of time… Note this kind of work can be done very secretly and with very modest resources — it does not require banks of centrifuges, big reactors, or ICBM test launches. First Human Embryos Edited in U.S. (MIT Technology Review) Researchers have demonstrated they can efficiently improve the DNA of human …

More

Don’t Touch the Computer

Under what circumstances should humans override algorithms? From what I have read I doubt that a hybrid team of human + AlphGo would perform much better than AlphaGo itself. Perhaps worse, depending on the epistemic sophistication and self-awareness of the human. In hybrid chess it seems that the ELO score of the human partner is …

More

Super-human Relational Reasoning (DeepMind)

These neural nets reached super-human (better than an average human) performance on tasks requiring relational reasoning. See the short video for examples. A simple neural network module for relational reasoning https://arxiv.org/abs/1706.01427 Adam Santoro, David Raposo, David G.T. Barrett, Mateusz Malinowski, Razvan Pascanu, Peter Battaglia, Timothy Lillicrap (Submitted on 5 Jun 2017) Relational reasoning is a …

More

Machine intelligence threatens overpriced aircraft carriers

The excerpt below is from a recent comment thread, arguing that the US Navy should de-emphasize carrier groups in favor of subs and smaller surface ships. Technological trends such as rapid advancement in machine learning (ML) and sensors will render carriers increasingly vulnerable to missile attack in the coming decades. 1. US carriers are very …

More

Face Recognition applied at scale in China

The Chinese government is not the only entity that has access to millions of faces + identifying information. So do Google, Facebook, Instagram, and anyone who has scraped information from similar social networks (e.g., US security services, hackers, etc.). In light of such ML capabilities it seems clear that anti-ship ballistic missiles can easily target …

More

Premature Obituaries

[T]he report of my death was an exaggeration. (Mark Twain, 1897) In a recent blog post, “Data Science Is Not Dead“, Jean-Francois Puget discussed and dissented with a post by Jeroen ter Heerdt titled “Data Science is dead.” Barring the possibility that Schroedinger shoved data science into a box and sealed it, both assertions cannot …

More

Update Error: Wrong Architecture

Yesterday I ran into one of those mystifying glitches that, will infrequent, serve as a reminder that Linux is not for the faint of heart. When I booted my desktop system (Linux Mint 18.1 Serena), the system tray icon for the software updater was displaying a red “X” that indicates it tried and failed to …

More

Borges, blogging, and a vast circle of invisible friends

This blog gets about 100k page views per month. My sense is that there are a lot of additional views through RSS feeds and social media (FB, G+, etc.), but those are hard to track. Most of the hits are on the main landing page, with a smaller fraction going to a specific article. I’d …

More