In this episode of the Techdirt podcast there’s a particularly great metaphor Ravi Iyer uses to describe bad algorithms that reinforce content you don’t want to see more of, and yet draws you in:
I will eat donuts, and so I kind of “want” donuts, by one definition of “want.” I will eat them if you put them in front of me. But I don’t aspire to eat more donuts. I want to eat healthier food, right?
When my wife was pregnant a couple years ago, Instagram started serving her more and more posts from people who had traumatic birthing stories, or who had lost babies, either during childbirth or soon after. When something like that comes up, you can’t exactly look away. Especially for her (and for me) being a hardcore empath. I don’t think it’s a bad thing that people post about their real stories, in fact I encourage it. But for Instagram to identify that she was more “engaged” with these posts and then to say “I bet she wants more!” is pretty messed up. It’s a negative spiral that might have driven her to spend more time on the platform (thus increasing her value as a user, from Instagram’s view) but produced a very negative mental impact on her. That’s what people talk about when wanting to get away from the algorithm.
