It’s 2025 and the world is constantly talking about AI. Every major podcast seems to have had multiple guests on talking about the dangers of AI and the societal havoc it will wreak if it remains unchecked. There’s so much hype on this subject that many are warning that the bubble will soon burst. Some worry about what will happen to to the arts, and many worry about what will happen to jobs. They are all missing the most destructive things about it.
It didn’t really begin in 2025. If I had to guess, I’d say it was around 2008 or maybe 2009. I don’t remember the exact year, but the experience will forever be seared into my memory. I suppose it’s become my biggest regret.
I was working as a freelance software engineer and the contracts I was picking up were few and far between. Someone found me online somehow and emailed me about a project they wanted help with. I took a phone call and discussed the project and the payment but never met this customer in person. It was a man who spoke great English but with an accent I did not recognize, and he wanted me to scrape data from a webpage of Facebook and then comment on certain users posts. The users were all beautiful young American women, and I was intrigued. I was just a lonely awkward autistic nerd after all. This project looked interesting, at least at first.
He gave me a list of users and I wrote them down. I wrote the code to scrape the webpage and logic to “watch” the users that I was following through a fake account I setup. I used fake images and made it look like an average looking young American man who had a simple profile and only a few posts. Nothing too flashy. I didn’t want anyone to interact with the account after all. If they did, they would go ignored.
The instructions were simple. At first I was only to automate the posting of innocuous comments on the posts that these girls were making. The comments would come shortly after the post went up. The logic wasn’t even hard, I didn’t need to understand what their post was about, I just needed to put up a comment and tell them they were smart or looked good today or some other shallow nonsense. I did as I was requested and the job paid well, much more than the other contracts I had been taking up. At this time, those were often around $100/hour of actual time spent. This contract was $500/hour.
The list of users grew quickly and I had written a framework that allowed me to scale easily. I just had to make some configuration changes without changing any of my code and BAM, the new user would be receiving comments on their videos immediately. I had even configured the list of comments so that I could add new ones easily and the code would rotate through them so they wouldn’t feel repetitive.
About 6 months in, after we had scaled up to watching and interacting with over a thousand pre-selected users (I have no idea how they were chosen) things changed. I started to receive a different list of users, some girls and others were boys, and my instructions were to post negative comments. I didn’t like the direction of this, so my reluctance caused my customer to give me explicit instructions on the comments that I would make. Most of them were slightly insulting, but many of them were outright rude. I used my framework to scale up easily, yet again, and the automated-insult-framework was born. The initial users who were receiving nothing but compliments were now receiving a mixture of compliments and insults. At this time I had several fake user accounts in play, and they would all post comments from my framework on the same users.
I was curious about what the negative comments would do, so I opened another fake account but I isolated it away from the others so none of it would tie together. I also used a virtual private network to use a server in a different country just in case Facebook was logging my IP. I wanted to make sure the account would in no way be tied to the insult-framework accounts.
I watched some of the older posts on one especially beautiful girl’s page. Her name was Autumn and she was a college student in Iowa. She had long blonde hair and she was stunning, but in the girl next door sort of way. She was unassuming and adorable. I read the comments on those posts, and noticed that she was paying attention to the comments and she usually responded to many of them. In her videos she was glowing, so excited about life and happy to talk to her growing audience. She had replied to many of the automated comments with a “Thank you!”.
As I watched more of her videos, her demeanor began to shift. It was as if she was beginning to lose her normal bubbly personality and she was looking for feedback about how to be more entertaining. Some of her videos made me feel terrible. Occasionally she’d talk about her struggles trying to get more followers and trying to “remain relevant”. She wanted to be a better entertainer, and her own personality was disappearing. Her content was changing. Her pictures became more seductive. She wore makeup. She was becoming a caricature, a dancing monkey. Some of her followers would give her well intentioned advice, and she would take it. She’d make a video doing what these people would suggest, only to be met by a chorus of insults from guess who? That’s right, my automated-insult-framework.
I was really struggling with this. For several months things remained as they were, and I was being paid more than just my hourly development rate. I think my customer knew that I was not thrilled with what was occurring. He offered to shift how I was being paid, since I was maintaining this application I had built, running it on my own servers and making changes every time Facebook made an update that broke my code. My new contracted rate was $15K per month.
A few more years passed like this. I occasionally checked in on some of the users and was disheartened by what I saw. All of their personalities were shifting, and not in good ways.
One day, my customer called me up and told me to make some changes so that if a post was made that used certain words, the comment that would be triggered would be a suggestion for the user to kill themselves. I refused to do it, and then the offer came. The customer wanted to pay me $500k to hand over all the code I had written and walk away as if I had never been involved.
I took the offer.
I will always regret that decision and hope one day I will be able to reverse what I helped set in motion.
