Designing Less Painful AI Production Lines

-




I overdid it on my AI-code production line.

So often, designing things is overdoing them and then bringing them back down… once the suffering has been noticed.

I wanted to solve the problem where:

I ask AI to do a thing → …bleep bloop… AI says the thing is done.

But it’s not done. And now I play AI whack-a-mole.

In this case I wanted AI to write tests, then fulfill them with code—which sounds great.

But it went wrong.

What went really right were the first 2 steps:

1 - where Claude interviewed me to create a ‘job spec’ following a template we had designed together.

2 - where Claude reviewed the spec against a set of ‘checks’ we also templated.

Amazing.

What went wrong?

All the test writing.

It went nowhere and made the AI dev dumber. The tests were overkill and the Dev was designing to pass tests instead of being smart and looking at greater context.

(There’s a parallel to our school systems here but I’ll leave you to parse it out.)

So after that suffering, what did I do?

I skipped the test writing. Updated the ‘implementor removing the test references, and just handed it the spec.

Boom. Now the production line is pumping.

I have to admit that I’ve sunk more hours than I want to count trying to avoid testing, since I dislike it so much.

I still think it’s a great challenge to solve.

But it was killing my production line.

When I started building the production line, my goal was to stop playing so much AI whack-a-mole and output quality code, more quickly.

You’ll notice there are a bunch of objectives rolled up in there:

1 - Quality code

2 - Faster

3 - Me not playing whack-a-mole

They are all related but not identical. Not noticing this up front was one cause of down-the-line suffering.

Going one level deeper: I took a while to notice that ‘creating better job specs’, ‘automating testing’ and ‘reducing loops between approval and re-work’ are all separate levers that help with different combinations of my objectives.

Separating these would have lead me to less pain.

What I did well?

  • I designed myself into the loop at key quality control moments and handovers so that I could know the system’s parts well enough to iterate.

  • When something didn’t go well I broke it into multiple steps, favoring consistency and control over each step over trying to automate end-to-end.

  • This came from the mental model that each step is akin to a workstation in a production line, and that any improvements to my main constraint, which is time lost to re-work, is a win.

Onward and upwards.

Location: Little Burgundy, Montreal, QC H3J, Canada

Comments

Post a Comment

Popular posts from this blog

Todoist & PARA - Effortless Personal Task Management; Setup and Routines

-
Image
How I use a few minutes each day + ~30 minutes each week to make sure nothing falls through the cracks Two or three weeks ago, I rebuilt my tasking setup. I had let it break and decided to be ok with that.  I chose to be comfortable with the broken state of things and simply observe the consequences. With that knowledge, I could rebuild my system with only what is valuable.  Being among the second brain community helped motivate me but what really hit me was this note: What it feels like to be disorganized There is always a low level background anxiety that there is something important undone Chaos, lost, frazzled, overwhelming, messy, reactive. It feels normal until you experience the clarity that organization offers. Without ever experiencing organization, a scatter brained, stressed, and tired state is just "normal" and a consequence of working. It was all true.  My daily routines were going well, but I felt like tasks and projects were starting to pile up...
Read more

Designing the Future of Work | What I learned as a Flow Manager

-
Image
For the better part of the past 2 years, I was the Flow Manager at a company that guided clients through digital transformation, helping each of them move into the future of work. Many people have asked me: "Mitch, what the F*** is a Flow Manager?" Good news: while there, I wrote a post: What my goals were What I learned while fulfilling them The good people there were kind enough to let me use that post for my post-employment posting, and now I'll be sharing it with you. Below is my short manifesto: what I learned about teams, people, systems, and overall management, while working as, essentially, the internal productivity manager for a team of productivity consultants.
Read more

How to Setup Todoist Tasks, Projects, and Areas for Maximum Flow

-
Image
Welcome, this post is m eant to help you streamline your setup for managing your tasks, projects, and life areas. To follow the video, here are some of the principles I build around: Invisible work is the enemy of our inner peace and flow. Work items we hold in our heads and don't write tend like to wait until we're engaged in some other activity before assaulting the backs of our brains. In extreme cases, this feels like a whirlwind in our minds and harms our ability to focus and stay present. Because of this, we need to commit to capturing every to-do . In order to keep that commitment, we need to be mindful of our internal resistance which tends to flare if we don't have: Easy and predetermined ways to capture these items a system to manage (or process) items easily after capturing them certainty that they will surface at the right time Because the goal is productivity but also flow, and because we aren't always at our best, we need to build systems t...
Read more