Betting on the wrong framework for fun

Early this year, I remember playing Mahjong with my partner and some relatives during CNY.

My style of playing was, to them, unconventional. They find it funny that I would keep hold of the 'four winds', because probability-wise, it is harder to get three-matches of the same tile. Also, if you hold on to it longer, and throw it later in the game, someone might '碰' or '杠' to get a nice set of three.

Yet I still play like that throughout the rounds. Sure, I do end up on the bottom most often, but it's not like we're betting big money, so I just play along casually for funsies.

When luck plays into my hand though, and I do win playing with the 'four winds' or 'three dragon' tiles...

It can result in a pretty hand that pays out a little more. By sheer chance, I won twice in one sitting with 'seven little pairs' 😄

Some days, I feel that I've led a conventional life. Go to school, then uni, work after graduating, buy a house, etc.

On other days, I think about the unconventional things I've chosen to do compared to my school peers. Taking geography instead of accounting as a subject, going to NZ instead of Australia or the UK, etc.

Recently I was in Singapore visiting my sister, and she and her husband took me to lunch with a financial advisor person, as they were just starting to plan out their insurance, investments and stuff. The guy asked me what I did out of curiousity, and even my sister mentioned that she could never describe what I did to other people in that part of the world.

Working with satellites? Taking photos of the land and seeing how it changes? That's how I usually describe it, because photos make sense to people, but it's more complicated than that 😜

My job title has 'engineer' in it, but I didn't 'get' what an engineer was as a craft until very recently. I do 'machine learning', which is the less fancy aspect of AI, yet I shun away from language models and prefer to focus on interesting algorithms. Paradoxically, I find myself wanting to learn more than have the machine do the learning; I'm required to engineer deterministic solutions as part of my job, yet deep inside, my motivation is to discover what's next like a scientist.

At a conference recently in Tāmaki Makaurau, I sat down for dinner with some colleagues at a Mamak restaurant, and they made a good point about how our company is really at the Pareto frontier when it comes to good pay and good work. Any higher pay, and it might be a soul-selling job; any more impactful NGO-style work, and the pay would drop significantly.

It's an interesting way to trap people who 'get' it. Personally, I would have kinda liked to stay in academia for all its freedom, but the paperwork would have bogged me down and the pay is sorta abysmal. There is enough practical, impactful work to do in this unique industry position, and a smidgen of time to innovate cool things. Scarcity does breed appreciation.

Harnessing innovative energy from bright minds while keeping to budget and time is near impossible.

But if it's fun, I'm all for it.

Programming started for me at a surprisingly young age at school, and it was always about fun.

I started with QBasic, and managed to program simple text games for class projects. I went about as far as almost programming a game called 'Dots' (akin to Go with different rules), only getting stuck when I couldn't break out of an inifinite loop when a 'capture' was made due to some logic error. There were countless other games coded in FreeBasic or similar I used to download from a website and play for hours during the holidays.

In later years, Flash games in the browser became a thing, and there were entire websites dedicated to it, my favourites being physics games with flying penguins and how to build a bridge or something. I got pretty proficient with ActionScript, programming a whole space shooter game in the really easy to use editor, that allowed you to draw stuff, and point and click at well organized dropdown menu items to select functions like 'gotoAndPlay'.

Impressive as it was, I wouldn't know how to start programming 'simple' 2000s-era games like those right now!

As I went through uni, even though I didn't take computer science, I got into GIS and ocassionally needed to run some code here and there. There was Matlab for some remote-sensing ish classes, R for my undergrad thesis that involved biology, and Python 2 in the form of ArcPy and eventually PyQGIS that I picked up during my first internship.

Python eventually become my goto scripting language as I started my first industry job in the mid 2010s, interspersed with some SQL and HTML/Javascript maps here and there. The fun went from games into 'how can I automate a very manual point and click process into a single button click', so that I can work on other cool stuff.

The automation aspect typically came about due to a desire to quantify 'changes'. In my undergrad uni class, we did supervised and unsupervised classification of proglacial lakes in the Southern Alps over time, as they grew from melting glaciers. At work, I coded up an algorithm based on Hausdorff distance to create 'diff' maps of how road networks have changed as updates were made.

Eventually, the algorithms evolved to handle more 'fuzzy' matching. The whole deep learning era had started, and got even hotter as I progressed through my PhD. It became more common to talk less about programming languages itself, but 'frameworks' in a sense.

Python wasn't a hard bet to be honest, even if I was the only one really using it at my first job in a company full of Java and SQL Server programmers. Deep learning frameworks on the other hand, shifted a lot!

I started with Keras with a Tensorflow backend, that had a really nice API design. Then I went all in on Chainer, because it was built on CuPy that mimicked NumPy's syntax, and I thought that was the way to go. The code for my first paper was in Chainer actually, so really went hard on that.

Eventually, I dabbed at more GPU-accelerated algorithms in the RAPIDS AI stack like cuspatial (now unmaintained). Pytorch was something I picked up only during my postdoc, a little heartbroken because it overshadowed Chainer. It seems to have won out so far, though for how long though?

I suppose that I've cycled between 'frameworks' and 'algorithm' development over the past decade. Recently, I'm less into the high-level abstractions, and more getting into the low-level weeds of how to make things go fast.

Rust is something I've mentioned in a previous blog post, which is a modern systems programming language to help me express nitty gritty details close to the hardware. A recent pattern I'm seeing is people wanting to go beyond CUDA, from paradigm-shifting stacks like Mojo/Modular, the announcement of a startup called VectorWare going GPU-native first, and crates like CubeCL that have been quietly chugging along.

Whether it's Rust or Mojo, the challenge is the same, rewrite GPU-accelerated algorithms in a new programming language for different hardware.

Away from all the flux that is agentic programming and language model development, which are the pinnacle of abstractions, I'm retreating to the near-atomic level on what makes an algorithm tick.

It's a nice place to be away from the crowd.

Lonely sometimes yes.

But like mahjong, with the right tiles in place, riches and beauty beholds when it works.