Backpropagation in the Brain?

Ask and ye shall receive :-) In an earlier post I recommended a talk by Ilya Sutskever of OpenAI (part of an MIT AGI lecture series). In the Q&A someone asks about the status of backpropagation (used for training of artificial deep neural nets) in real neural nets, and Ilya answers that it’s currently not …

More

Pseudocode in LyX

Fair warning: This post is for LyX users only. When I’m writing a paper or presentation in LaTeX (using LyX, of course) and want to include a program chunk or algorithm in pseudocode, I favor the algorithmicx package (and specifically the algpseudocode style). There being no intrinsic support for the package in LyX, I have …

More

B.S.-ing Precisely

In a recent blog post titled “Excessive Precision“, John D. Cook points out the foolishness of articulating results to an arbitrarily high degree of precision when the inputs are themselves not that precise. To quote him: Excessive precision is not the mark of the expert. Nor is it the mark of the layman. It’s the …

More

Coordinating Variable Signs

Someone asked me today (or yesterday, depending on whose time zone you go by) how to force a group of variables in an optimization model to take the same sign (all nonpositive or all nonnegative). Assuming that all the variables are bounded, you just need one new binary variable and a few constraints. Assume that …

More

NP Confusion

I just finished reading a somewhat provocative article on the CIO website, titled “10 reasons to ignore computer science degrees” (when hiring programmers). While I’m not in the business of hiring coders (although I recent was hired as a “student programmer” on a grant — the Universe has a sense of humor), I find myself …

More

ICML notes

It’s never been a better time to work on AI/ML. Vast resources are being deployed in this direction, by corporations and governments alike. In addition to the marvelous practical applications in development, a theoretical understanding of Deep Learning may emerge in the next few years. The notes below are to keep track of some interesting …

More

Selecting Box Sizes

Someone posted an interesting question about box sizes on Mathematics Stack Exchange. He (well, his girlfriend to be precise) has a set of historical documents that need to be preserved in boxes (apparently using a separate box for each document). He wants to find a solution that minimizes the total surface area of the boxes …

More

Usefulness of Computer Science: An Example

I thought I would follow up on my June 29 post, “Does Computer Science Help with OR?“, by giving a quick example of how exposure to fundamentals of computer science recently helped me. A current research project involves optimization models containing large numbers of what are basically set covering constraints, constraints of the form \(\displaystyle …

More

Does Computer Science Help with OR?

Fair warning: tl/dr. After reading a blog post yesterday by John D. Cook, “Does computer science help you program?“, I decided to throw in my two cents (convert to euros at your own risk) on a related topic: does computer science (which I will extend to including programming) help you as an OR/IE/management science/analytics professional? …

More

Callback Cuts That Repeat

The following post is specific to the CPLEX integer programming solver. I have no idea whether it applies to other solvers, or even which other solver have cut callbacks. Every so often, a user will discover that a callback routine they wrote has “rediscovered” a cut it previously generated. This can be a bit concerning …

More

Grouping Rows of a Matrix

I spent a large chunk of yesterday afternoon doing something I thought would be simple (relatively speaking) in LaTeX. I wanted to group rows of a matrix (actually, in my case, a vector) with right braces, and label the groups. An example of what I wanted is in the image below. This seems to me …

More

Big M and Integrality Tolerance

A change I made to an answer I posted on OR-Exchange, based on a comment from a well-informed user of OR-X, might be worth repeating here on the blog. It has to do with issues that can occur when using “big M” type integer programming models, a topic I’ve covered here before. As I mentioned …

More

How NSA Tracks You (Bill Binney)

Anyone who is paying attention knows that the Obama FBI/DOJ used massive government surveillance powers against the Trump team during and after the election. A FISA warrant on Carter Page (and Manafort and others?) was likely used to mine stored communications of other Trump team members. Hundreds of “mysterious” unmasking requests by Susan Rice, Samantha …

More

More on “Core Points”

A few additions to yesterday’s post occurred to me belatedly. First, it may be a good idea to check whether your alleged core point \(y^0\) is actually in the relative interior of the integer hull \(\mathrm{conv}(Y)\). A sufficient condition is that, when you substitute \(y^0\) into the constraints, all inequality constraints including variable bounds have …

More

Finding a “Core Point”

In a famous (or at least relatively famous) paper [1], Magnanti and Wong suggest a method to accelerate the progress of Benders decomposition for certain mixed-integer programs by sharpening “optimality” cuts. Their approach requires the determination of what they call a core point. I’ll try to follow their notation as much as possible. Let \(Y\) …

More

Gork revisited, 2018

It’s been almost 10 years since I made the post Are you Gork? Over the last decade, both scientists and non-scientists have become more confident that we will someday create: A. AGI (= sentient AI, named “Gork” :-)  See Rise of the Machines: Survey of AI Researchers. B. Quantum Computers. See Quantum Computing at a Tipping …

More

Quantum Computing near a Tipping Point?

I received an email from a physicist colleague suggesting that we might be near a “tipping point” in quantum computation. I’ve sort of followed quantum computation and quantum information as an outsider for about 20 years now, but haven’t been paying close attention recently because it seems that practical general purpose quantum computers are still …

More

Creating A New MIME Type

I struggled a bit this afternoon creating a new MIME type and associating it with a particular application, so I’m going to archive the solution here for future reference. This was on a Linux Mint system, but I found the key information in a GNOME documentation page, so I suspect it works for Ubuntu and …

More

Thread Safety

As I noted in yesterday’s post, one of the major changes associated with the new “generic” callback structure in CPLEX is that users now bear the responsibility of making their callbacks thread-safe. As I also noted yesterday, this is pretty new stuff for me. So I’m going to try to share what I know about thread …

More

CPLEX 12.8: Generic Callbacks

IBM is getting ready to release CPLEX 12.8, and I had the opportunity to attend a presentation about by Xavier Nodet at the 2017 INFORMS annual meeting. Here are links to two presentations by Xavier: CPLEX Optimization Studio 12.8 – What’s New and CPLEX 12.8 – the Generic Callback. As with any new release, there …

More

Where are participants in American and Canadian teacher hashtags?

My dissertation research is focused on Regional Educational Twitter Hashtags (RETHs), which are teacher-focused hashtags that are associated with particular geographic regions, such as American states or Canadian provinces or territories. This isn’t the first time that I’ve done a project on this phenomenon, and it’s rewarding to come back to RETHs to answer some questions that …

More

Minimizing a Median

\( \def\xorder#1{x_{\left(#1\right)}} \def\xset{\mathbb{X}} \def\xvec{\mathbf{x}} \)A somewhat odd (to me) question was asked on a forum recently. Assume that you have continuous variables \(x_{1},\dots,x_{N}\) that are subject to some constraints. For simplicity, I’ll just write \(\xvec=(x_{1},\dots,x_{N})\in\xset\). I’m going to assume that \(\xset\) is compact, and so in particular the \(x_{i}\) are bounded. The questioner wanted to …

More

Rolling Horizons

I keep seeing questions posted by people looking for help as they struggle to optimize linear programs (or, worse, integer linear programs) with tens of millions of variables. In my conscious mind, I know that commercial optimizers such as CPLEX allow models that large (at least if you have enough memory) and can often solve …

More

Robots taking our jobs

The figures below are from the recent paper Robots and Jobs: Evidence from US Labor Markets, by Acemoglu and Restrepo. VoxEU discussion: … Estimates suggest that an extra robot per 1000 workers reduces the employment to population ratio by 0.18-0.34 percentage points and wages by 0.25-0.5%. This effect is distinct from the impacts of imports, …

More

Don’t Touch the Computer

Under what circumstances should humans override algorithms? From what I have read I doubt that a hybrid team of human + AlphGo would perform much better than AlphaGo itself. Perhaps worse, depending on the epistemic sophistication and self-awareness of the human. In hybrid chess it seems that the ELO score of the human partner is …

More

Memory Minimization

As I grow older, I’m starting to forget things (such as all the math I ever learned) … but that’s not the reason for the title of this post. A somewhat interesting question popped up on Mathematics StackExchange. It combines a basic sequencing problem (ordering the processing of computational tasks) with a single resource constraint …

More

Super-human Relational Reasoning (DeepMind)

These neural nets reached super-human (better than an average human) performance on tasks requiring relational reasoning. See the short video for examples. A simple neural network module for relational reasoning https://arxiv.org/abs/1706.01427 Adam Santoro, David Raposo, David G.T. Barrett, Mateusz Malinowski, Razvan Pascanu, Peter Battaglia, Timothy Lillicrap (Submitted on 5 Jun 2017) Relational reasoning is a …

More

Face Recognition applied at scale in China

The Chinese government is not the only entity that has access to millions of faces + identifying information. So do Google, Facebook, Instagram, and anyone who has scraped information from similar social networks (e.g., US security services, hackers, etc.). In light of such ML capabilities it seems clear that anti-ship ballistic missiles can easily target …

More

Premature Obituaries

[T]he report of my death was an exaggeration. (Mark Twain, 1897) In a recent blog post, “Data Science Is Not Dead“, Jean-Francois Puget discussed and dissented with a post by Jeroen ter Heerdt titled “Data Science is dead.” Barring the possibility that Schroedinger shoved data science into a box and sealed it, both assertions cannot …

More

Rise of the Machines: Survey of AI Researchers

These predictions are from a recent survey of AI/ML researchers. See SSC and also here for more discussion of the results. When Will AI Exceed Human Performance? Evidence from AI Experts Katja Grace, John Salvatier, Allan Dafoe, Baobao Zhang, Owain Evans Advances in artificial intelligence (AI) will transform modern life by reshaping transportation, health, science, …

More

Epistemic Caution and Climate Change

I have not, until recently, invested significant time in trying to understand climate modeling. These notes are primarily for my own use, however I welcome comments from readers who have studied this issue in more depth. I take a dim view of people who express strong opinions about complex phenomena without having understood the underlying …

More