OpenAI Computer Vision Models Are A Bit Too Literal

Don’t Believe Anything You Read

Several computer vision models, including OpenAI and Tesla, are proving the truth of this aphorism in some recent results.  You may recall the power of a couple of pieces of electrical tape to convince a Tesla’s autopilot that a 35 MPH zone is actually an 85MPH zone, however OpenAI’s computer vision model, CLIP, is a little more gullible.

If we were to write a noun, like pizza, on a piece of paper and then stick it on a disappointingly non-pizza object, the model will happily declare that thing is indeed a pizza.  These attacks have been dubbed typographical attacks, and seem to be very effective.  As long as the model can identify your handwriting as being text and can match the word to a noun in it’s database, it will take your word over any existing pattern which might match the actual object.

The reason that this kind of attack can succeed is due to the flexibility of OpenAI’s computer vision model which can “recognize Spider-Man when the superhero is depicted in a photo, a sketch, or described in text“, which was the example offered to The Register.  That flexibility is both the model’s greatest strength and weakness, at least for as long as text receives higher weight in these conditions; a very difficult balancing act if it is to continue to be able to associate text with images.

Try not to torment our prospective AI overlords with this too much, they may store it for later reference.