Last week, the saliency-based image cropping algorithm deployed by twitter came into scrutiny. Inspired by some of the conversations that unraveled on Twitter and the widely shared reported incidents of racial discrimination, we sought to investigate, experiment, and elucidate the workings of cropping algorithms. Following up from last week, here are the updates.

Democratizing the audit

In order to democratize the scrutiny of this technology, we have created an educational saliency based cropping app where you can upload images and see what a state-of-the-art machine learning model similar to the one deployed by twitter thinks are important parts of the…


TL;DR: Fcuck the algorithm
Table of Contents:
- Experiment details
- Feedback from twitter: The two camps of the Rashomon aisle
- Unbiased algorithmic saliency cropping is a pipedream
- Conclusion

This Saturday noon, I was made aware of an ongoing saliency based cropping bias fracas on my timeline. As I was literally rushing through the final experiments for a forthcoming paper that seeks to address Computer Vision’s growing fascination with physiognomy, I thought I’d run this little statistical experiment live on twitter with the exact same Psychonomic dataset I was currently using to investigate this.

Experiment details:

The CFD dataset


Credit: ‘ Striving for Disentanglement’ by Simon Greig — https://www.flickr.com/photos/xrrr/500039281/

TL;DR : Disentangling disentanglement. Via this blog-post, I intend to try and summarize all of the dozen papers presented on disentanglement in deep learning in this year’s NEURIPS-2019 Vancouver.

Companion Github repo replete with paper summaries and cheat sheets: https://github.com/vinayprabhu/Disentanglement_NEURIPS_2019/

Background: Disentanglement in Representation learning

On Thursday evening of the conference week, as I sauntered around the poster session in the massive east exhibition halls of the Vancouver convention center, I realized that I had chanced upon probably the 5th poster in the past couple of days entailing analysis of a disentanglement framework the authors had worked on.


TLDR: Google TTS -> Noise augment -> {wav files} ->SnowBoy ->{.pmdl models} -> Raspberry Pi

OK, so it’s that time of the year again. You know there’s *that* thing in the desert. Last time around, I rigged up a Google AIY vision kit and added espeak on Chip and Terra , the art installations of the motley bunch that is BŸTE: Burners for Ÿntelligent Technology Emancipation.

The result was this:

This time around, I decided to add an extra sensory ability: The ability to listen and respond, which in Machine Learning plain-speak translates to rigging up a hotword detection…


Class-wise mean images of the 10 handwritten digits in the Kannada MNIST dataset

TLDR:

I am disseminating 2 datasets:
Kannada-MNIST dataset: 28X 28 grayscale images: 60k Train | 10k Test
Dig-MNIST: 28X 28 grayscale images: 10240 (1024x10) {See pic below}


TLDR: They seeded their webscrape via REDDIT, the mother lode of all ideas tinderboxy and weaponizable. So, it will at the very least be a PR disaster if they release the bigger model. The smaller 117M is nasty as is sans the subtleties!

Table of Contents

Intro:

One paper that I recommend my interns to read is Unbiased Look at Dataset Bias that has these awesome introductory lines:

Torralba, Antonio, and Alexei A. Efros. “Unbiased look at dataset bias.” (2011): 1521–1528.

And if I may add, it is also a capital mistake to not curate that data with much care. Given the(rightful) focus these days on the issue of the biases and toxic idiosyncrasies that your dataset begets, you’d think it would be a given for large organizations to deliberate extensively about the specifics of dataset collection procedures before embarking on a venture…


A rant on collective amnesia and faux-ancientization of atrocities

As I sifted through the tweets and rants protesting the pitiful refusal by the NIPS board to inflict a name change, I was shocked to learn that the sole emphasis of protest was on the prevalence of nips as a double entendre in our culture with obvious misogynistic connotations.

It immediately elicited thoughts in me regarding one of the most fascinating dark strengths of Whiteness and privilege in general: the ability to manufacture, nourish and sustain collective amnesia as a tool to cover up crimes committed en masse targeting the weak and disenfranchised .

Few realize that not too…


Prelude:

Let’s begin with a simple introduction into the world of adversarial inputs. These are inputs into a machine learning classifier that have been shrewdly perturbed in such a way that these changes are near damn invisible to the naked eye but can fool the machine learning classifier into predicting either a arbitrary wrong class (Un-targeted) or a specific wrong class (targeted).

There are two defining images that come to my mind when I think of this field at large. The first one is the classic Panda-to-Nematode image from here.

The now iconic example of a panda’s image getting perturbed into a gibbon (Source: https://arxiv.org/pdf/1412.6572.pdf )

The second one, is this one below that provides a geometrical…

Vinay Prabhu

PhD, Carnegie Mellon University. Chief Scientist at UnifyID Inc

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store