Out of a fog, back to the past

It must be the fact that I’m putting pen to paper at all again. I can’t really imagine what else it could reasonably be.

This might get a little wacky, I don’t know. I’m all stream of consciousness all the time anyway lately so you get what you get.

Lately, and by that I mean within the last couple weeks to a month, I’ve felt myself coming out of a fog of sorts. It had been years since my brain felt like my own, since I progressed at all in thought or skill in any measurable way. This is not to say that I’ve been a window licking idiot for the last N years, but the difference is one in degree rather than kind.

There’s a thing I say to people when they get on my nerves, which also means I’m chanting it to myself ad nauseum: “If you were wrong, how would you know?” When I use it as an admonishment it’s usually, if I’m being honest, my way of saying “shut the fuck up.” But it has more value than that.

How DO you know if you’re wrong? You should be able to ask yourself that question about all things you think you know anything about at all. Now, the answer may very well be (and is, with frightening regularity) “I have no idea.” But at least be aware of that.

This comes up for a couple reasons: First, I’ve been listening to the narration of Nassim Nicholas Taleb’s book “The Black Swan” which, a couple hours in is exceptional. Go get it and read it. I’ll put a link in the sidebar. I’m not going to butcher his thesis here. Suffice it to say he touches on it enough that I was bouncing up and down in my truck saying “I do that! I do that!”

The other is that one of my favorite things in business happened to me today at about 4:30. This is not something that comes from my better nature as a human (indeed there are at least a couple people who would no doubt argue that I don’t HAVE one of those. But I’ll tell that story when I’m good and fucking ready and not a moment before.)

The most nightmarish place I’ve ever worked was a firm on Wall Street. It was a huge, german, backstabby financial institution with a 12-14 hour work day. I was a C++ programmer (oh THOSE were the days *le sigh*) working on a trading system. I feel compelled to add that it was one of the best few year stretches of my life. My boss and I locked horns frequently on the job, but left that shit in the office. It’s one of the only jobs where I made friends that I still keep in touch with. For all the misery, we build quite an esprit de corps. And holy SHIT did we drink.

Anyway I was on a long running project where I had to interact with a team of junior developers on a green field project. A more contentious team of fucknuts I’ve never had the displeasure of dealing with, before or since. Anyway I was providing (after 6-9 months of negotiation and spec tweaking) a feed file every night. There were constant complaints about data quality (I could only provide data that was as good as the data I had, Garbage In Garbage Out after all) and there were all kinds of other “I’ll tell your manager/VP/Director” horseshit problems I’ve forgotten within the last 14 (holy shit is it 14?) years.

We were running tests on a daily cycle. I give them a file, their overnight process ingests it, we sort through the problems. It should’ve been really straightforward. But it was a case of them being able to do no wrong and punting everything because they didn’t understand defensive coding (be liberal in what your program accepts, conservative in what it emits.)

One morning I got in (Seven Fucking Thirty) to a series of OMG THIS CAN’T KEEP HAPPENING emails firing back and forth. Yeah there was some really wacky problem with the file.

I went out to the drop location and found “filename.xls” and “Copy of filename.xls”, screen shotted the directory and copied the files locally, zipped them up, then started looking at copies I’d made of that, while keeping my eyes on the drop location.

Someone had opened the file to putter around with it before they imported it and had fucked with the formatting badly enough that the data wouldn’t load, then tried to pin the failure on me.

Now, we all like to tell stories wherein we are the hero. It’s WHY we tell stories most of the time. I wasn’t a hero. The reason this sequence of events were sufficiently powerful to make me sweat is that a lot of the time my feed was bad. I did forget things. I was eeking in on accuracy but it was taking a long time.

But not this time. THIS time higher managers were getting called in to deal with the situation. After some rounds of nonsense I called the developer (on the PHONE, which says something about how hot I was.) The “copy of…” file had disappeared in the interim.

“Dude, you edited the file and broke it.”
“I didn’t touch anything. You submitted a bad file.”
“Okay man, look. I have screen shots of the directory and local copies of the original and the backup copy you made. I have the real original I generated here so I can point to the exact changes you made that broke your import.”
“Here’s what’s going to fucking happen next. I’m going to give you the opportunity to send an email to me, my boss, HIS boss and whomever else you want, explaining exactly what happened. If you don’t, I will and I promise you it will be a much different email.”

The righteous fury with which I was barking at him by the end of that sentence was one of the most incredible things I’ve ever felt. I was RIGHT. It was the ultimate gotcha.

Petty? Sure. But I was fucking right.

I don’t go over that story often in my head; not like I used to.

The less interesting parts of my job today involve sending all kinds of data all over the place, including to a company who’s charging us a terrible amount of money to provide a service they can’t seem to get off the ground. They’ve got ENDLESS problems ingesting incoming data. They shift goalposts, slide new specs in as “quick changes” and play all kinds of underhanded games they think we can’t see. They’re not only assholes, they’re inept assholes. (For the record, we deal with many counterparties who are perfectly competent and pleasant. I don’t actually just hate everybody.)

Well, I got an email…Tuesday, “Hey man, the last two files you uploaded were both truncated and they’re corrupting our upload process. This keeps happening. Can you check the process on your end?” Fuck. (Also. “keeps happening?” I’d never heard about it.) I dodged the email because I didn’t want to deal with it and replied with a mea culpa email this afternoon, when I finally started looking at it.

Okay. They’re files coming out of two different processes completely. So something’s truly fucky.

I keep copies of these files for months (3-6 depending.) I find the files, one is in a zip file I send, and the other is just a csv I transfer natively. I crack them open and…they’re fine. I look at the end of file rowcount screen shots that were sent over and, sure enough, they were truncated. I puzzled around this for about an hour this afternoon.

An ftp transfer truncating files? That’s…weird. I run a test (though not a file size test) post-upload, to be sure they transfer was successful. So it’s not possible that the data connection bails during a partial upload.

I grab our systems guy and drag him over to show him and re-narrate the sequence of events, show him the email and the files. We bat about a couple theories when I go ashen.

“Wait..” I said. “THAT file came out of a zip archive right?”
“If the transfer had truncated the zip file, it would be corrupted. He wouldn’t be able to open it, much less retrieve a partial file.” Understanding dawned.

“Fucking lying.”

I can not WAIT to go in to work tomorrow.

Leave a Reply

Your email address will not be published. Required fields are marked *