How I made my 20 years old self redundant in under 60 seconds, 30 years after
30 years ago or so at The American University of Paris, I chose to do my Bachelor of Computer Science Senior Seminar on Neural Networks. Pretty obscure topic at the time! It took months to put everything together: gather the literature from actual books, learn some of the theory, code the feed-forward and back-propagation algorithms and the interface in Objective-C on a 32-bit 8MB RAM NeXT computer, create a dataset to represent digits in 5x7 grids, annotate, feed the data into the algorithms, and wait long hours to conclude that it wasn’t working, test additional layers with various number of artificial neurons, fail, try again, repeat… until the model or rather the effort was deemed acceptable enough. The goal was to investigate and write a paper about the technology, not achieve industrial strength accuracy. (Shout out to my co-author Bassel Haidar 👋)
Today, just for the heck of it, I asked ChatGPT this: “write a python script that loads the digits dataset from sklearn and then setup a pytorch deep learning model”.
30 seconds later, I had a decent piece of code. But it failed to run with a logical error somewhere in the middle of the code.
Nice try, but frustrating! My request was reasonable and a fairly trivial thing to ask a massive LLM in 2023… (!)
I thought that perhaps my python packages were incompatible with what ChatGPT had learnt and that I now needed to take over. Typically, I would hit the documentation online and start debugging… but no. I flexed my “lazy muscle” and challenged the bot’s response to give it the feedback it deserved for sweating it out in the first place. So I wrote back: “in the above, I get an error here: loss = criterion(outputs, labels)”
ChatGPT responded: “I apologize for the error in my previous code. The issue is that the PyTorch CrossEntropyLoss function expects the target label tensor to have a shape of (batch_size, ) rather than (1, ).” with the updated code.
Barely looking at the fix, I ran the updated code on my MacBook Pro M1. 30 seconds later, I had a fitted model with 97% accuracy to recognize digits.
Great, so the AI did meet my expectations after all (wait, 97% only ?). What I did not expect was the error in the middle of the code and the casual apology with a correct fix upon request!
Multiple take-aways here:
- a lot of the DL tech has matured over the past 30 years, thanks to massive contributions by Geoffrey Hinton, Yann LeCun, Yoshua Bengio, Andrew Ng, Fei-Fei Li, their colleagues and successors
- the tech is sufficiently well packaged to be harnessed by a junior computer scientist, a hobbyist or a tinkerer able to express complex ideas in simple terms
- even though there is a human in the loop here, twice (me, with the prompt & the error fix request), there is a little bit of meta action going on: LLM deep learning inferring deep learning code without breaking a sweat…
Yup, this changes everything.
The next decades are going to be interesting.
Unless things go much faster than we are allowed to know…
https://futureoflife.org/open-letter/pause-giant-ai-experiments/
