Knowing what you want is hard – expressing it is even harder

Everyone knows the genie trope, and most people read it as a story about greed. It is, but is also a good metaphor about the inherent difficulty of specification. How to be precise in expressing what you want, and don’t want. You get three wishes, and the structure of the tale is that the first one is spoken naively, and it comes out wrong. Not because the genie is not well-meaning, but because it does exactly what you said instead of what you meant. Then you spend the second wish correcting the first, and it overcorrects, or breaks something new. The third wish, you then are forced to spend on putting everything back. A reset: you end where you started, having learned only how bad you are at saying what you want.

I think that’s the most useful metaphor we have for having this much AI power available and still struggling to get real value out of it.

At the core, this is a very old delegation problem. People mostly can’t articulate what good looks like. So they (try to) hand off work, get back something wrong, and conclude the person they delegated to is incompetent. What they can’t see is that the hard part really was describing the requirements, the limits, the verification steps. That’s genuinely difficult. Most people struggle with it.

But unfortunately, this is where we are at now, in knowledge work and beyond. These LLMs conscript everyone into management, whether you want the job or not. There’s no more sitting at the bottom of the food chain just executing a task someone else specified. Everyone has to think about the workflow, the decision points, the parts that repeat. And if you can’t describe, or better: decompose, what you do, how and why, in a way that’s repeatable, if you can’t think in systems, you’re not going to be of that much use, I’m afraid.

I want to be precise here, because this isn’t a prediction that everyone gets automated out of their job. There is way too much talk about the future already. Most of it will turn out to be utter nonsense. We don’t have the economics right now. Most of it is fiction anyway, subsidized by venture capital, and the bounce hasn’t settled on what any of this is worth long term. The tension between the expectancy of U.S. venture capital and the undermining pricing from China is significant. But that is not the point here.

The interesting part is that a specific skill suddenly becomes scarce and valuable: decomposition. The ability to take the fuzzy blob of what you do all day, break it apart, and ask honestly which pieces actually need a human. A lot of what people have been paid for, the looking things up, the researching, the busy work, is going to fall away. Not the whole job. The automatable seams inside it.

This could be teachable, yes. But it is a mindset shift. Anyone who gets paid for what they ship, rather than for showing up, already thinks this way. They’re obsessed with getting the same result spending less time, or more result in the same amount of time. If you’re paid for attendance, reducing that is unattractive, because noticing that half your job is mechanical doesn’t give you anything. However, there’s no real refuge from this.

The exception is outside of white collar work: My son is in daycare, and I deeply value the people who care for him. You cannot ask them to deliver the same care in less time. Same with parenting, with teachers, with therapy, with a handful of other things. You can’t run scaling laws on those. They should be well compensated for the human time put into them. We might, hopefully, see compensation in these areas increase. That’s human time very well spent.

For everything else, it’s a stack.

1. You have to understand the work itself, that’s the core expertise.
2. You have to understand how the technology works — that these agents are instantiated into a world where they know nothing but what is presented to them or what they can discover from there. Unless you understand those mechanics, you get fooled into assuming they know more, remember more, or learn the way you’re used to from humans. They don’t.
3. You have to think in systems, so you can decompose the work and then build, maintain, and improve these new kinds of processes over time.

And then a fourth thing, which I suspect is badly underpriced: levelheadedness. It’s so easy to tip into either camp, the true believer or the doomer, and both play better than the truth. If you’re an influencer, or paid to push a side, a balanced perspective doesn’t sell. But anyone seriously looking into this has to be somewhat torn, struggling to find a position between the extremes. And you can spot the real thing fast. If you’ve experienced it in practice, you can recognize it in others based on a fragment of a sentence. It is a hard to hide attitude.

Example: In preparation for this article, I had the model mishear “genie in a bottle” as “genie in a model,” decide that nonsense was clever wordplay on my part, and cling to it, defending it across several corrections before it let go. That’s the whole thing in miniature, really. Worse, it could easily have gone either way. It is impossible to predict. It’s all just probabilities and weights. The smallest typo could change the coordinates the model finds itself in latent space.

Hence, it is similar, but also very different from managing people. If you hire someone with little training, their limitations are consistent. You hand them an SOP, you don’t rely on their judgment, you build in checks, and the unreliability has a shape you can plan around. Models aren’t like that. Working with them is isomorphic to building an organization in some ways, and in others you have to check things you’d never think to check with a human.

So what does someone who’s actually felt this do differently when they build?

It’s why these models have been so successful in software, and are making real strides in math, soon physics and engineering. When a domain is verifiable, you can run the optimization loop against the verification, and with enough time and tokens, the models get there.

The biggest leverage isn’t that the model is reliable. It isn’t. It’s that this unreliable, probabilistic thing can produce software that is deterministic, testable, and reliable. So you don’t sit there trusting prose instructions, going back and forth forever. You abstract your processes into write-ups, into skills, do this, then that. Then you notice that the skills, applied directly, are flaky. So you translate them into scripts, into command-line tools, into things that can be checked. The skill becomes a thin wrapper around a deterministic core. You compile your own judgment down into something you can test.

Which brings the genie all the way back around. Yes, with these models you effectively have unlimited wishes, not three. But this genie has no memory. The implementations try to bootstrap a memory-like set of files, but it doesn’t update the way humans intuitively expect, and it misses context in often very unexpected ways. Whatever you corrected yesterday is gone unless you encoded it somewhere the next attempt can find it. So unlimited wishes only save you if you spend them climbing toward precision instead of re-rolling the same fuzzy request. Start naive, then ask: how do I say this without ambiguity? That’s the deterministic code. How do I verify it? You run it, you hit edge cases, you fix them. The genie’s curse was never the number of wishes. It was that the character keeps wishing in “natural” language, an ambiguous medium. Wish in code instead, and your third wish doesn’t drop you back at square one. It locks in something you can build on.

And that’s the quiet joke at the end of all this. The scarce, future-proof skill everyone’s scrambling for turns out to be an old discipline: Specify. Test. Handle the edge cases. Repeat.