Saturday, July 6, 2024

An arc welder in the datacenter: what could go wrong?

Must read

Who, Me? Yet again Monday is upon us, bringing the prospect of another working week filled with joy, opportunity, new horizons, and a fresh dose of Who, Me? – The Reg‘s weekly confessional in which readers share stories of jobs that had promising beginnings, and … interesting ends.

This week, meet a reader we’ll Regomize as “Andrew” who told us about the fun times he spent in the 1960s and 1970s working as an IBM customer engineer – mostly for financial services clients of the sort for whom a few minutes’ downtime meant a few million in losses.

One of those clients was a stock exchange which called Andrew to ask what IBM intended to do about the large crack that had formed in the metal frame holding a large high-speed impact printer.

The printer was a monster that produced a then-astounding 1,100 lines a minute but made a horrible racket while doing so. A huge, powered cover was therefore in place to reduce noise.

The crack in the frame meant the cover would not function properly, making the printer a mighty irritant for the stock exchange.

As luck would have it, Andrew’s family ran a metalwork business. He had grown up in its factory, and become a handy wielder of an arc welder.

IBM and the stock exchange therefore hatched a plan.

A portable arc welder would be found and brought onsite. The stock exchange would open its doors on the weekend, power down its computers – and the fire alarm and sprinkler system – and Andrew would fix the frame without causing a meltdown.

“Come the day, all went as planned. I first cleaned all the oily paper dust that had accumulated, then duly fixed the frame. The cover worked to everyone’s satisfaction,” Andrew told Who, Me?

Years later, after Andrew had moved onto a different team, his phone rang.

“You just broke the stock exchange,” a fellow Big Blue employee said, angrily.

Andrew hadn’t set foot in the bourse for years, so was bemused as his colleague turned the air blue berating him for having caused the crash at a key client.

“After he had calmed down, he explained that another printer frame had cracked and one of the team who watched me weld the other one years ago thought he would do the same.”

Unlike Andrew, however, this chap had never welded anything before in his life.

He did at least repeat Andrew’s process of securing client permission, and having the sprinklers turned off.

But he forgot to have the alarms turned off.

And he did the job during business hours.

You can guess the rest: the fire alarms were tripped, all critical systems went offline, IT staff were evacuated, and the stock exchange took an expensive unplanned break.

And somehow Andrew was blamed for the mess – because years earlier he had left the portable welder onsite.

Really, how hard can welding possibly be, right? Have you ever assumed a skill would be easily acquired? If so, click here to send us an email and we may share your experience in a future instalment of Who, Me? ®

Latest article