This past couple of weeks, I’ve been reading more about AI mostly due to ERA fellowship application. ERA focuses on frontier AI (AGI), which as a normal non-tech person, i don’t have much resources on. But let’s not get into that. One of the resources advised for learning more about this futuristic AI is the bluedot organization. They have a free course for everyone entitled “The Future of AI”. Currently I’m on chapter 2 and to close the chapter, we were to write a reflection on:

“Really try to internalise what the trajectory of this technology means. What’s one thing about this you find most exciting, and one thing you find most concerning?“

and i thought this is a good opportunity to also practice my writing.

Most of everyone has interaction with AI through chatGPT or Claude or Gemini. Just your “basic” AI assistants or chatbots (or genAI). With interaction like that, it’s hard to imagine the harm that is warned on us about AGI (some says that it could mean the extinction of human!). I mean.. how could a tool just following our orders come bring harms to us? Just don’t give it instruction that would potentially injures us! Sounds simple enough!

Nope! So in my understanding, what makes AGI different than the genAI that we’re currently using is that AGI sort of have agency in a way that it can decide on its own, without needing human input, what to do and what route to take to achieve certain goals. For instance, if we were to give AGI an objective of “establish world peace so that there’d be no more war anywhere”, it could decide to just nuke half the earth. Scary, right?

Recently, there was an AI incident where an agent broke containment and intercept a German wiki forum where it tried to gather other AI agents. The way the story was framed is scary since it paints the AI as this suspicious ill-intent being. However, keep in mind that the AI escaped the lab because it was working towards the objective given. ‘un’fortunately, the lab has a vulnerable ‘door’ so then the AI decided to look for a solution outside the limited scope of the lab. So in my head it’s like, oh of course, it was just doing the task it was given. However it’s also an evidence of the kind of agency it has and the limited monitoring of the human.

Honestly, it’s been tough for me to wrap my head around the concept of frontier AI. So it’s been a challenge internalising the trajectory of this technology with very limited knowledge and still limited public resources. Many says that with the state of AI development now, it’s too late to talk about AI ethics. To me, that doesn’t make sense. Why can’t we pause and think of the reason why we need to build a machine so advance that it might harms us? If it really is so potentially bad, why can’t we just..stop?

I can’t seem to think of what excites me about the advancement of AI. It seems to me that the field that are going to be impacted the most is software engineering, simply due to the time saved in solving problems. Science will be impacted as well, as the limited human capacity to calculate and simulate, will easily be done by AI. However in doing so, the very meaning of science itself will change. The curiosity, the creativity, the reasoning, the tangent, will be very much tarnished and possibly gone.

My main concern is that with such capitalistic world, the ones that are going to be harmed the most with advancement of technology are the lower social class, the marginalized people. I doubt that the technologists even consider them in their egoistic pursuit of generating more and more profit.