Updating from deprecated to latest without getting fired — #2
If you’re a developer, you’re probably running an outdated software version, and the reason you don’t upgrade it is because you’re scared it will break everything. Let’s face it — that’s a pretty reasonable fear. Here’s how I updated a Python library from a decade ago to the latest version in a safe way, and without getting fired.
Once upon a time, there was a software developer named Kai who worked at an awesome company called Via.
Kai’s software application was widely used by many millions of people, but it had been around for a while, and some parts of it were starting to show their age.
One day, Kai received a bug report from a user, He found that the issue was caused by an old and deprecated version of some library. Kai knew he had to update the library, but it wouldn’t be easy.
Later that day he started to receive several more bug reports. When Kai’s evil manager found out, he ordered Kai to update to the latest version of that library or pack his boxes!
why should you update your Python libraries?
Before we continue with our dramatic tale of Kai’s heroism in the face of adversity, let’s talk about why you should update your software version.
First things first: Security.
Security is the №1 reason to update software.
You’ve probably heard about the Log4J vulnerability that panicked the software industry in 2021? That was a massive vulnerability that was fixed by a version update of a library called Log4j!
In addition to security, software updates can boost your performance, offer new features and bug fixes, and ensure compatibility with the latest technology.
Kai’s first round against the archaic library
Now back to our hero’s tale.
Kai opened his requirements.txt file and saw the problematic library, lets call it ujson, was pinned to version v1.34.
Kai opened the release notes in GitHub and started scrolling down, looking for ujson version v1.34. He scrolled own…and down…and down, clicking Next Page over and over again, until FINALLY he found it — ujson version v1.34. It was released in 2015. Almost a decade ago.
It was at that point that panic set in, and Kai, despondent and despairing, started walking towards the storage room to get a cardboard box and pack up his stuff.
But with nothing to lose, Kai decided to give it a shot. He removed the pin from the old version and updated it to the latest version. The tension in the air was palpable. Everyone in the room held their breath.
Kai clicked the button, ran some tests against the version update, and waited.
requirements.txt
# ujson==v1.34 # do not change this will break everything
ujson==5.7Disaster strikes
Several minutes passed by and the results were in.
Kai was hoping for the best, but prepared for the worst. Which is pretty much what he got.
1,000 tests had failed. That’s 25% of the tests.
Kai ran full speed towards the storage room to get a cardboard box.
Kai’s second round against the evil and the ultra-fast ujson
As Kai packed, he glanced at his screen and noticed that the ujson release notes were actually very informative. He had a thought: “What if I go over the release notes one at a time and fix everything that the update broke?”
Kai started reading and typing non-stop, like a madman possessed! He also added several flags and arguments to the ujson calls in his code.
Holding his breath, Kai ran all of his tests again. Surprise surprise — all tests passed!
Kai knew he was going in the right direction, but he wasn’t there yet. Like almost any code base, many of the older features weren’t comprehensively tested, and developers were even afraid to make revisions.
Kai decided that the time had come to deploy his changes to the Dev environment and see how it performed with additional traffic from the cavalry — the formidable QA team.
Several long minutes passed, when suddenly our villain, Kai’s manager, came storming in shouting “Are you TRYING to get fired? You’re breaking the Dev environment!”
Kai saves the day
Kai, desponded and broken, put his head in his hands, tired from the long day. Suddenly, he began to laugh…and laugh…and laugh.
He knew what had to be done to solve this problem for good.
He decided to get his code to use the new version of ujson, and if failed, he’d revert back to the old version. That would allow him to get all the new features of the latest version while allowing backward compatibility with the old version. Then, one error at a time, Kai would slowly fix all problems with the new ujson version, and eventually, delete the old version for good.
import ujson_new # ujson version 5.7.0
import ujson_old # ujson version v1.34
ujson_dumps(some_obj: Any) -> str:
try:
return ujson_new.dumps(some_obj)
except ValueError:
logging.error('Error parsing with new ujson version')
return ujson_old(some_obj)Since it’s not possible to install multiple Python versions with pip, Kai knew he had to get creative.
Kai cloned ujson to his local PC, checked out to the old version, changed the name of the ujson package to via_ujson and uploaded it to some package repository (Gemfury, PyPi, Helm, Conan, npm …)
The final code will looks something like this:
import ujson # ujson version 5.7.0
import via_ujson # ujson version v1.34
ujson_dumps(some_obj: Any) -> str:
try:
return ujson.dumps(some_obj, reject_bytes=False)
except ValueError:
logging.error('Error parsing with new ujson version')
return via_ujson(some_obj)A triumph against evil
Kai confidently deployed the change again to the Dev environment.
QA threw everything they could think of at it, but Kai was so confident that he already started unpacking his cardboard boxes. And Kai was right — it had worked!
When Kai’s evil boss popped his head into the office, he was surprised to see Kai at ease. Kai explained his creative solution, and his boss smiled — Kai had passed the tests. He shook Kai’s hand with a big smile on face, and said: “Well done. Now let’s discuss your sparkling future at Via.”
Note: While this is based on a true story, names and other details have been changed to protect the innocent. A few dramatic turns also have been added for effect.
A few technical details that need to be considered when updating your Python library versions
- Some Python libraries are written in C, like ujson, so to build the old ujson version I needed to compile the C source code.
- Check if there are other dependencies that are using ujson. For example, our logging client used ujson. Because it was an internal dependency, I was able to change it, but external dependencies require careful inspection.
- Because we use good coding practices, we had a json_utils.py that made it very simple to update the ujson library, since it was being called from a single place
- As a follow-up to this task, I’m thinking about a project that will be integrated into our CICD pipeline and will alert and open a Jira ticket (or GitHub issue, or task) for libraries that are getting too old. See this blog post for similar ideas.
