Hasnain says:

AirBnB recently did a large scale migration of code using LLMs. I've been trying to find good datapoints for things like this, so personally reading this blogpost (link in comments) was quite exciting.

I have done a crapton of these migrations by hand (aided with static analysis/codemod scripts in the past) and I'm excited to see where LLMs can go in this space.

Some key takeaways from this post (which resonated well for me, similar to how I'd do it as a human funnily enough):

* Break the migration into manageable chunks:
* Migrate each file independently
* Within that too, have a logical set of sequential steps.
* Don't try to do the whole thing in one shot
* Choosing the exact specific context matters a lot for the LLM
* Retries help -- if it fails, just try again
* It's easy to get "good enough" to e.g get 75% of the way there
* From there to 97% took a lot more work
* ... and it doesn't make sense to go to 100% (in terms of ROI)

Money quote from the article below

"Airbnb recently completed our first large-scale, LLM-driven code migration, updating nearly 3.5K React component test files from Enzyme to use React Testing Library (RTL) instead. We’d originally estimated this would take 1.5 years of engineering time to do by hand, but — using a combination of frontier models and robust automation — we finished the entire migration in just 6 weeks."

Posted on 2025-03-21T04:40:04+0000