I had a rule

I’m finding my brain, even after a four day work week, starting to organize itself around the structure of the week and, perhaps oddly, around structure in general. 

The fallout from yesterday’s day of frustration continues.  I just have to figure out what the right books are.  Those technologies change so damned fast that something from a few years ago might not be even out of date so much as straight-up wrong.

But it sets me on a program of trying to get this crap in my head.

On the other side, I’m starting to think in terms of what I want to get done by when and how to break that down into meaningful discrete pieces.

On another front, one of the big problems with working has always been lunch.  Doesn’t seem like it should be, but for 35 years I’ve, very soon after I’ve started someplace, come to the conclusion that “there’s nothing to eat in <fill in town>.”  Now that makes its own sense in little suburbs.  But I spent decades working in downtown Manhattan. 

It’s just not true.  But a certain kind of boredom, coupled with lunchtime executive function burnout makes every option dull within a couple weeks.  So I think what I’m going to have to start doing is both bringing lunch and setting aside days to eat at one place or another and just order the same damned thing, with the option to change it up but to always have a fallback.

Of course now that I’m thinking about it I remember that I used to do that.  I’d forgotten what my fallback was, but I had a rule that said “if I thought of three places and none of them hit me right, I’d head to…”  I think it might have been Panda Express.

Actually, fuck it.  Yeah.  That’s the rule for now. Until I come up with a better one.  It’s to easy to just eat fucking garbage then feel like shit about it by three in the afternoon.

It’s not quite Einstein’s closet full of the exact same suit so he didn’t have to make decisions about what to wear, but it’s on the right track.  Set up rules so that you don’t have to make decisions about stupid shit.  It hasn’t been so much of a problem lately since there wasn’t really any pressure.  But I’ve got other shit to do with my head.

What and when to eat.  What I want to learn.  Indeed, what to wear.  There will be a scheduling of the weeks and days, something to measure against.

I’m not sure what the final form will take.  Well…that’s a silly concept at its outset.  I’m not sure what form it’ll all settle in to first.

It’ll be fun.

A flame about this high

I can’t even begin to describe the level of blinding frustration I’ve been soaking in today.

I was thinking on to the page a couple days ago about using Microsoft OneNote’s API to go back and forth between my own PKMS data store. Frankly, the more I thought about it the better an idea it seemed to be. I could programmatically create a wiki notebook in OneNote, then detect new and changed data both from home and from the notebook itself and sync the data back and forth.

Assuming I can get all the formatting stuff right, links between pages and such so that I can get a solid round-trip going, it would really do most of the things I want the mobile interface for a PKMS to do. Most of the “advanced” functionality can just sit in the back end and create static files for OneNote while I just fiddle about with any of a few different wiki interface gateways into the engine in the background.

So I figured today I’d write a simple little “Hello World” script that connected to OneNote and got, I don’t know, a list of pages. Just to prove the API worked. Then I’d build on that.

Yeah.

That was the plan.

It’s 9 hours later and I’ve got more conflicting information about the fucking security model, OAuth, tokens, tenants, certificates, proxied permissions, roles, access modes and all KINDS of byzantine fucking horseshit.

We’ve gotten to a point in software complexity now where SOMEone has written something to do just about anything. But it’s sealed away between 724 steps and layers of access control. It’s gotten to the point where service configuration itself has become the programming task and actual functionality takes a back seat.

It reminds me a fair bit of the J2EE phenomenon back in the day. The idea being that not only should you be reusing source code or compiled libraries, but you should be able to reuse run-time instantiated components.

Given that little blocks of code did all kinds of stuff, it became a configuration job to knit all those blocks together for an application. But because it was “configuration” and not “coding” (even though it absofuckinglutely was) it wasn’t implemented in anything like a sane programming language. Instead everything was codified in some XML dialect from hell (but I repeat myself.)

J2EE eventually collapsed under its own load, though I imagine there’s a bunch of nonsense still kicking around out there using it. There always is.

But I’ve been through a bunch of tutorials, quick starts, Microsoft guides and all MANNER of resources.

None of it is QUITE clear. None of it seems to start in the right place.

I wrote my first program in the late 70s. I’m no idiot. There’s ZERO excuse for this crap to be like this. It’s just “security mindedness” run amok, full reducto ad absurdam.

And maybe that’s really it. No one wanting to say “no” to the “this will make us safer” question.

I don’t know. I don’t care.

What I am going to do is buy a couple few books on these security models that the big cloud providers seem to all be jerking off over and make one good honest attempt to grok this shit from the ground up.

Failing that? I’m out. Maybe what I’ll do is sub out the work that sets things up. Maybe I’ll just fucking bail on the projects. Who fucking knows.

So if you’ll excuse me I’m gonna go find some resources and come up with a study schedule to inject as much of this fucking shit into my head as is inhumanly possible.

Hell, maybe I’ll find out I’m just going full “I’m mad because I’ve had a pot of espresso and I don’t already know all this stuff.”

But I seriously fucking doubt it.

Phoning in Friday

I’ve got approximately zero brain left.  I’ve stared at this blank page for about a half hour, figuring I’d have something.  But not so much.

This is the close of my first week of work in a few years, likely related I’m sure.  I’ve got an awful lot of thoughts about that.  But to the eternal shock of many of you (not that there ARE many of you) I have enough of a sense of discretion to not have any desire to talk about it here.

A couple of the guys are looking forward to hanging out at Smokey this weekend, and I may very well do that.  It’ll be a good time, pretty much always is. 

But I also have to start working on other stuff, both programming projects and some more analog things.

As this month draws to a close I’m starting to think about next month’s daily pursuits. I’ve got a couple ideas kicking around in my head, but nothing concrete yet.  One thing I am considering is junking the word count requirement that I held to until a couple days ago but keeping the “post a day.”

Thing is, and I remember having written this before: When I scribble a few words down with the aim of putting them on the website, ALL other writing, including journaling ceases completely.  Now, multiply that with the level of constraint I have to endure in order to hit the “publish” button and it creates a bit of a problem. So I’ll have to sort that crap out as well.

I was…well, not ‘excited’ exactly about Black Friday deals.  But I did spend a (thankfully small) amount of energy looking for deals.  I’d had a tool bundle on my amazon wishlist for…hell, probably a couple years.  Bunch of stuff.  Back in the day it was $600.  Well it, like everything else had gone up considerably.   I happened to scroll through my wishlist and saw it was back down to $599.

I got all excited and pulled the trigger then shot out a couple text messages crowing about the great deal.  I went and had a cigar after work, thought about it a bit, realized that as much as I actually wanted it; it was a well thought out purchase, if that thinking was done some time ago, then picked up my phone and cancelled the order. 

A great deal does not a need create.

Gobble Gobble

I’ve got nothin’ today guys, sorry. 

Just got back from a spectacular Thanksgiving day at Frank’s place.

Truly spectacular food, great cigars, rather a lot of whiskey and above all, great company, for which I’m extraordinarily thankful.

For my part I made lemon bars and pignoli cookies20221124_101706

The lemon bars are bog standard.  I use this recipe. BUT I multiply the crust by 1.5, so it’s:

3 Cups of flour

3/4 Cups of sugar

1/3 Teaspoon of salt

3 sticks of butter.

Other than that, it’s this, by the book: https://www.thepioneerwoman.com/food-cooking/recipes/a12104/lemon-bars/

I find that when you use the zest AND the juice they’re super tart, which I adore.  But it justifies the 1.5x crust quite well. 

The pignoli cookies are straight off Giada’s recipe.

https://www.foodnetwork.com/fnk/recipes/pignoli-cookies-8869507

A note about those: If you’re unfamiliar with making them, the recipe reads almost like a meringue.  It’s not.  Put all the ingredients in a food processor and blend the absolute everloving shit out of them.  Make sure you cube the almond paste first because it’s really tough, having the texture of something between clay and old play-doh.

At 350 for 16-18 minutes (there’s no real  way to tell by feel if they’re done), the pine nuts are a little too darkly toasted which I think adversely affects the flavor, if subtly.  So I’m going to turn the heat down to maybe 300 and increase the cook time to 20-25 minutes next time.  But, like I said, it’s pretty much impossible to tell if they’re done by touching them.  So I’m not sure how they’ll go.

Also, I like using whites from extra large eggs rather than large.  It makes more of a difference than I expected.

I realized today that I’d never had a slice of my mother’s apparently famous pumpkin pie.

I just took a few advil and a couple electrolyte tablets with a couple tall glasses of water and am heading to bed as I’ve got to be in work in a little less than 10 hours.

o/

Angle of Attack

I’ve got some thoughts about codifying your approach to things; intentionally turning your attitude about situations around by focusing on changing your approach to the situation.

But it’s 11:30 and I’m going to completely bitch out and hit post on that teaser.

The aside that became the body:

Oh, I was thinking about something that I suspect might actually be possible:  Microsoft OneNote has a REST api.  It’s not outside the realm of possibility that I could round trip a OneNote notebook as a PKMS with at least nominal wiki functionality.

I know you can create links between pages at least within a notebook.  So it stands to reason that you’d be able to generate them easily enough, even if it was a matter of creating a link in the OneNote software, downloading the page and cracking the content open to see how it’s formatted.  Can’t be TOO tough to then regenerate page links, build the pages and feed them back up to the server, which would be reflected on the phone pretty damned quickly.

If I combined that with the Todoist api stuff then I’d never have to go NEAR mobile programming.  NOT that phone app programming is so inherently scary.  But I just have a wicked blind spot when it comes to designing user experiences, particularly for constrained form factors.

But I’m fucking beat and I have to get up in the morning and bake cookies, having just put lemon bars in the fridge.

Friday night I’m going to sleep 37 hours.

Show me the way to go home

I’m tired and I wanna go to bed.

Welp, I’m not going to say it’s over.  But I am going to say that when I made this commitment to publish a post a day of a thousand words or more I did NOT anticipate starting a new job this week. 

It’s only day two and, while I have some thoughts about it and the way it’s going I find it very unlikely that those thoughts are going to see the light of day, at least not in the sense at least a couple people are hoping for.

For the last couple days I’ve left the office, turned right, driven a half mile, then turned left into the parking lot at Smokey.

Tonight someone who doesn’t usually come out during the week “put out the call” to hang out there.

As I talked about yesterday, you’ve got to understand what your priorities are and why you have them, and while I didn’t think I’d be out until 10:30 tonight, I knew it was going to be a fair hang with a level of conversation I just don’t generally get very often anymore.

He wasn’t going to get up the hill until about 7.  So I sat there with a new book “Building A Second Brain” and a cigar.

And I passed the fuck out for about an hour.  I was well awake when he showed, but I realized I was really run down.

But again, priorities are priorities.

Had some cigars then, as Smokey was about to close we headed over to Sam’s and sat out front, smoked a couple more and knocked a couple back.

Tomorrow I’ve got a shitload of baking to do when I get back from work.  Thursday is Thanksgiving and I’ve got to start it by doing even more, then expect to be gone ‘til the wee hours.

I expect I’ve got a few days where “post a thousand words” is an iffy prospect at best.

Again; you’ve got to know what your rules are, why you have them and have the strength of will to know when they don’t make sense. 

I spend an awful lot of energy proving to myself that I’ve got some self-discipline by exercising that muscle and it serves me remarkably well. 

Dogmatism certainly has its place.  But being blind about it when it’s not proactively serving you…doesn’t.

And if I’m being honest, and perhaps inappropriately self-congratulatory, knowing when to and being able to go beyond those rules is its own skill as well.

I’m going to bed.

I’ve got yet another very long day ahead of me tomorrow.

Dezombification

We’ve all seen and some of us have commented overmuch on the affect having these hyper connected super computers in our pockets has had on us.

We’ve all seen twenty somethings (and some of us have noticed the seventy somethings as well, to say nothing of the rest of us) turning in to c-shaped people barely ever looking up from their devices, endlessly chasing those dopatonin hits (I can never remember which one it is, give me some fucking rope) at such a pace that defies, quite literally, all reason.

We’ve all seen value and values grow more and more shallow.

Over the last twenty years I’ve watched people going from condescending to put their cell phone and their blackberrys on the bar, annoyed when they’d get an email or a call from The Office in the middle of a session to living one-handed, the cell phone becoming an appendage.

And I’ve watched, in the last ten years, those very same people go from having a cell phone as an appendage to being little more than an appendage for their cell phone.

We’ve gone from looking upwards hoping towards some day, to looking across, anxious to soon. And now we look down, demanding Now.

From albums, to songs, to samples.

From paintings, to images, to memes.

From novels, to articles, to tweets.

From philosophies, to quotes.

We demand everything and sacrifice nothing.

Or do we.

We’ve all watched this and seeing some of what it’s costing us.

We’re turning into some kind of animal longing for comfort of laying in the sun of abdication of the self.

We know it’s costing us something, which causes us to chase it more desperately, not realizing that the complete lack of silence in our lives has become a complete silence to ourselves.

But I saw something fairly recently that made me think rather a lot in rather a short amount of time.

I’m not going to talk about the specifics. Crossover to the real world isn’t really something I’m super hot on after all. The odd name here and there, sure. But certainly nothing like this.

I watched something I fear isn’t anywhere near the end game of this pathlessness we’re fighting for with everything we’ve got.

It was the end of self worth. The final wisp of self dissolve in the mad race for not merely reinforcement, but…something worse I can’t even really put my finger on, as if someone was waiting not to be validated but to be created, utterly dependent on a force something beyond carelessness, to a prideful scorn.

I watched someone lost in their dependence on others for their own sense of self worth get whipped around like a dog toy, totally aware of what was going on and endeavoring no defense, a state that amounted to regret for the future.

Though the facts would mark it out as an all too common story, never in my life had I seen it so nakedly.

And I suppose it could be a lot of things. I’m no shrink and am disinclined at the moment to swim in those waters.

But seemed to me to be the downstream of this massive zeitgeist of hollowing out of the soul, giving up everything of any meaning as our all too enthusiastic sacrifice to these twisted flippant gods through hand held silicon altars.

What defense after that do we have then against situations like the above? What strengths? What will? Hell, a sense of identity seems almost a Wu concept at this point, a pollyannic myth form.

We have no sense of meaning any more at all. Real purpose has fled from us entirely.

So, having groped my a bit more than thoroughly around that particular elephant, what’s the alternative?

That’s easy, but fucking tough.

Put the damned phone down.

If you can’t leave it home for a day, leave it in the car, leave it in the bag. Turn off your notifications, every one you don’t absolutely need. Do it for a month, a day, an hour.

There’s no way but to put the breaks on the flood.

Take all of these things in their reverse order.

Read fewer tweets and more articles, then essays, then novels.

Dispense with posting and sharing memes back and forth all over the place and go to websites (if you must) like DeviantArt (which has a tremendous amount of non-deviant art.) Go to a gallery and read a book on an artist, or artistic movement. Hell, one of the strangest things I ever learned was a whole hell of a lot (read: not all that much) about the Viennese Reniassance to impress a girl. The topic was more interesting than the girl.

Take your favorite oversampled, overproduced songstress (of which I have many. I claim little in the way of high moral ground here) and listen to one of their albums front to back. Maybe hunt down some of the samples they used and go listen to the originals. That’s an awful lot of fun. Go back a couple levels of “my inspirations were” from interviews (they can’t resist asking that question in interviews.) Then do some research. See where your favorite music really comes from.

Take your favorite “wisdom quips” that are posted in images (because words no longer serve) and go find where they come from. Read the original works. I guarantee most of the world has the wrong idea about Nietzsche, including some very smart people.

And, if you’re from my world: Stop using fucking stack overflow as an information source. Instead of looking up the solution to a programming syntax problem, go hunt down the original design principles, articles and tutorials, books and long-form lectures.

In a social setting, let a question remain unanswered. Resist that immediate pull. You don’t need to know that piece of blindingly irrelevant trivia. No one needs to know. There’s no general principle in there. But if you must reach for your phone, leave yourself a note to go digging around later, even if all you do is batch your research time. Make it a half hour a day of “resolving the questions of the day.” It’ll turn in to an hour (unless you end up on tvtropes, in which case you’re just fucked.)

And, for the Master Class: Think about why you believe what you believe. Do you even know WHAT you believe? Do you just regurgitate quips from your favorite public figures? Make THAT an exploration. Discover then develop your core philosophies. What do you demand of yourself? Where do you fail? What else is there to fix? How do you strive (if you even do)? Do you like who that is? No? What are you going to do about it? Sit around and hate yourself, or what little of “yourself” there even is?

You don’t have time? Why? You too busy watching fucking tiktoks? Fuck out of here with tiktok.

The power of technology needs to be brought to fucking heel and used as a tool for us, rather than us for it. But it takes intentionality, will, and a degree of personal power. And like any muscle, using it more and more will strengthen it. You will be less and less able to be cast completely adrift by the need for immediate gratification of some impulse you can’t even name.

Which way forward, modern man?

Bespoke

It’s 10:00 at night and I’m fucking exhausted. NOT that I had a particularly strenuous day. I hit Books-A-Million looking for something to spend money on and couldn’t come up with anything. Well…I could, but I price checked them against Amazon and left without buying anything.

It’s a damn shame and I’d rather spend money at a brick and mortar store. But my limit on the “local spending tax” is about 20 percent. If the price difference is any more than that, I’m just not spending my money in person. The instant gratification bonus is just not worth more than that to me.

It makes me sad, it really does. I’m watching the world change and I just don’t think it’s for the better, at least in this way. Sure, it’s great to get stuff cheaper. With the truly incomprehensible selection of things available to be delivered to my doorstep, sometimes within a couple hours is really little short of a literal miracle.

But it’s costing us in ways I don’t think we really appreciate. And no, I’m not about to presume I’ve figured out a bunch of ways you haven’t and then proceed to lecture.

Not…I suppose, that I wouldn’t. I mean, in fairness it’s a pretty me thing to do. But not this time.

What made me think of this today in particular, aside from this morning’s little instant gratification frustration (sounds like a grunge album name) is the juxtaposition between this morning and a brief conversation I had on twitter a few days ago.

There are a couple accounts that just post objects d’arte. I’m particularly enamored of Art Deco stuff. And graphic art is one thing, but Deco and Nouveau stylings of things I’m much more interested in, door knobs and lamps, desk legs, doorways and staircases. Interestingly, now that I give it even a little more than a little thought, I like Art Deco anything, but I’m not much for Nouveau in two dimensional art, but adore it when it’s made a bit more concretely manifest, as if the nominally organic nature of the form lends itself more naturally to 3d manifestation. I’m gonna have to give that a think.

Someone responded with something like “It’s a shame people don’t appreciate this kind of craftsmanship anymore.” That launched into a discussion about where one would even go to FIND such things, supposing with perhaps naive optimism that if such things were more easily obtained they would be in demand. Markets work both ways, after all.

Which brings me back to the point (believe it or not.)

The incredible ubiquitous availability of…stuff has contributed mightily, as we all are painfully aware, of the not so slow downfall of “mom and pop” stores. Specialty brick and mortar establishments are just closing up shop, unable to compete with the commoditization of goods available from institutions that can afford to put in place the perhaps literally inconceivable logistical infrastructure required to get us “pretty much anything” from “pretty much anywhere.”

As such, those products have gravitated, though not I suspect through necessity so much as through gravitational convenience, to generic lowest common denominator “solutions to problems” with little or no thought given to design past notions of necessarily generic mass appeal.

For instance, no one is truly mass producing steampunk light fixtures.

Oh sure, you can find some on etsy.com here and there. Maybe something on ebay or something similar. But such things as are “artistically designed” have become distant and few.

This was clearly on the way as a result of mass production to begin with. But the as The Internet Age gave way to The Internet Shopping Age, it’s really exploded and exacerbated that divide between the custom and the mass produced.

So what then has happened to the craftsmen? Where are the woodworkers, sculptors, engravers, stained glass…err…well, glazier isn’t quite right. But whatever they’re called.

Is there even a place near where you live where you’d be able to go if you wanted something ornate even made? I’d love to be able to go someplace and look at representative examples of an artist or craftsman’s work. Sure, there’s the odd gallery in the more prideful neighborhoods, though they tend to be little more than consignment shops.

Someone responded to this conversation with some pictures of work done “by local graduates.” And it…really didn’t help the case at all. Local graduate of what?

Say you wanted to work with a designer to have…oh I don’t even know because it’s been driven so far away from cultural consciousness by this phenomenon, a lamp. Let’s say a lamp. You want all the lighting in a room (if not the house) to be of a type, nothing bananas, just something that has some style to it.

Right now you’d go to Lowe’s or Home Depot. Maybe you’d go to Amazon, Pottery Barn, or Wayfair or something like that and you’d find a “collection” that you liked and, presumably working with your subcontractor, arrange to buy the “generic sterile ass fucking white and steel house with vaulted ceilings built so it looks like a picture on a magazine cover of a house with no soul” collection. (Which means it’s matte black or bronze and very square, but I digress, if only slightly.)

That kind of thing is what really passes for “fancy” nowadays or as though you had some part in the design process, instead of just bolting together commoditized crap and convincing yourself the result had some personality and didn’t look like it was something out of a little town in Connecticut named Stepford. NOT that I have any opinions on that.

I don’t even really mean that you’d need to go so far as having something custom made for you, working with an artiste who’d get to know you, come up with an idea, and make it happen. Just…one of a kind pieces.

I may be wrong. But it really is all but gone.

Of course it COULD simply be that I’m missing something that never existed. To me it seems unlikely. Sure, maybe the downstream of the industrial revolution having iterated a few generations has bred out unique works across all disciplines.

As bad as it was twenty years ago, it just seems like the gulf between made and created is growing impossibly wide, and I don’t really have any good answers on how to create/facilitate/enable an alternative.

There ARE things like etsy…but…that’s the only one I can think of, where you can go to get handmade stuff. It’s got its own problems, to be sure. But I’ll take it as a starting place.

Perhaps we’ve moved beyond the age of the hand made, of the personal.

It’s a kind of cultural soullessness at least as frightening as any Cyberpunk dystopia.

Huge Honkin Baseline

So I’ve spent most of the afternoon working on my HugeHonkinConsole project that I’ve been crowing about on and off (and lately mostly on) about for years. Seems as an incoming change in lifestyle has caused my brain to refocus on software projects as a way of settling into the new configuration.

Hell, I was going to go down to Red Phonebooth tonight and celebrate with some fantastic scotch and cigars. But frankly I’d really rather just sit here and code, even if it is lightly.

The current state of this project is both much farther along and much closer to a pile of crap thrown against the wall than the average person might presume.

What I’ve currently got running is:

– A desktop wiki server that also works seamlessly with emacs, so I can edit, view, and search pages from either platform.

– A light todo list application that I wrote in order to start getting my hands around web development even a little. I only use it for features and bugs for the whole HHC project itself. So it’s really a dogfood sub-application.

– The cigar deal downloader: This thing goes to about half a dozen cigar websites and downloads their sale pages every day, parses them, integrates the data and displays it so I don’t have to schlep all over hell’s half acre looking for Melanios or whatever at a reasonable price. It’d be a lot more slick if the damned sites didn’t change their page layouts.

– A couple little agents that sync my Todoist data back and forth using their REST api. At this stage it’s just so that I can keep a local copy of all the data.

– Various data downloaders from things like reddit, a library of rss feeds.

– Pulling, aggregating, and storing all incoming (and outgoing) emails with their attachments.

– Pulling and aggregating all of my text messages (with attachments) and call logs (from a backup file my phone drops on OneDrive)

– backing up OneDrive nightly so that I just don’t have to worry about fatfingering something and losing shittons of data.

– Many other things that I’m totally not doing because that would be bad and wrong.

All of these things sit in independent little processes, humming away on their own time, collecting, parsing and sorting data.

But now it’s time to start bringing them all together.

There are a couple major missing components. But if I wait ‘til I get to those then this will never get integrated.

I’ve already got a basic website that’s an interface to the Wiki and the Todo system. But it’s time to build a front end that I can use to view and change configuration, to watch logs, and to add and schedule subprocesses.

It’s time to actually put the console on the front of the HugeHonkinConsole, at least a simple baseline thing I can use as a starting point to build on to later.

I’m currently using a wsgi library that was in heavy use at my last job (oh irony of ironies) called bottle. It’s…fine. But development on the thing seems to have stopped a couple years ago and I’ve never heard of anyone else using it.

So I’m going to take the opportunity to adapt to a more popular (and actively maintained) one called FastApi.

One of the fun things about taking an existing code base and switching underlying frameworks like that (not that a wsgi library is thick enough to call a framework really. But it does technically fit the definition) is that it gives you an opportunity to really go over your code architecture and clean it up. You figure out very quickly what decisions you’ve made that were predicated on the specific tool set you were using, where you may have over-adapted.

Refactoring legacy code (even if not especially my own) is one of my favorite programming tasks. Problem is I get a little obsessive and over-engineery about the whole affair. Well…it’s not as though it’s a real problem on something that’s essentially an extended toy project.

So I’ll be ripping the dependency on the Bottle framework out and replacing it with something flexible and generic enough to be portable between multiple such systems (which, as a side-effect decouples things for a truly delightful level of testability as well seeing as how I could just as easily put a “framework” in place that was really just test scaffolding and the code wouldn’t know any different.)

But along with that I’ll be doing something similar (though at a lower priority and on an admittedly slower burn) to each of those little bulleted items up top. If you glance back over them you’ll see that an awful lot of them are really centered around the exact same kinds of operations: Download data, homogenize it, ingest it and keep it stored and indexed. Perhaps even generate some kind of digested output, even if it is only some kind of status report showing what happened.

Those semantics are pretty simple to pull out of each of those little project-lets. And having more than one (or two, or three) implementations of the exact same kind of thing, differing specifics notwithstanding, lets me see what the actual commonalities are rather than trying to extrapolate out to what the generic form of those little projects might be. It’ll all be splayed out in front of me, emergent pattern based design.

It’ll be nice to get my chops back, though I’m not sure how atrophied I really am. That’s pretty tough to diagnose from the inside of one’s own head. I won’t really have a good read on it for a month or so. It’s one of those things I’ve found that’s quite like chess: Taking more time off than you’d think would be useful really clears your head, as if the time away actually lets that region of your brain relax and sort all the learning and practice you’d been doing so it really integrates more thoroughly in to your thinking, becoming a more easily accessible set of conceptual tools.

Off to. I’ve been coding and fiddlyfucking with software all day. I’m going to go kill some dragons or something.

Taxed on a me

So I’ve got all these posts, most of the word count density is over the last couple years. It’s really something goofy like a million and a half words in the last three years alone.

Over and over again I run in to this problem of not being sure if I’d ever written on a specific topic or not.

Between how absolutely all over the fucking place I am on a post-by-post basis (something I’d been improving on until I embarked on this fool “post a day for a month” silliness) and the fact that even if I wasn’t, there’s no really good way to run a topic search.

Categories and tag clouds fail me utterly because I can’t recall my own taxonomy, much less exhibit the discipline of assigning categories and tags with each post.

But it occurs to me that in the heart of a piece of code I wrote a VERY long time ago (FAR closer to 30 years ago than not) might have at least a hint of a solution to the issue.

I’m pretty sure I could kick the shit out of it, what with decades more coding and thinking about this specific kind of problem.

Imagine you’re an insurance company and companies are trying to figure out if they should use you for your corporate health insurance plan.

What they do (or did) was submit a giant document written in Microsoft Word. That document was really a questionnaire; a giant outline of topics, categories, subcategories, and lists of questions with in them.

The insurance company would take this document and hand it to some poor bastard whose job it was to go through the whole thing and look up questions in a giant database of questions that had “subject matter expert” answers that had been vetted by the legal department. If it found a match to the question in the database it would pull up the answer to the matched question and plug it in to the document, then move on.

If there wasn’t a match, then that question would be sent off to a SME, who would double-check it wasn’t just something else then, if necessary, write up an answer, submit it to legal and once approved, stuff it in the database for next time and send it back to Said Poor Bastard.

This was just stupid. But it had to be done with every one. There were teams of these people who’d split these giant documents up, which were frequently far in excess of 200-300 pages, and go to work trying to match them.

The naive solution would be to say “well get a computer to look up those blocks of text.” But that’s wrought with nightmarish problems. There are a thousand ways to say a thing, so the likelihood you’d find a match, even if there was a perfect one, was effectively zero.

So what I did as an experiment was build a piece of software that took each question, eliminated what I called “noise words” then ran a scoring search of the tuple of resulting words against each of the stock questions in the database. What would come back was a list of questions that contained the greatest number of matching words from the original.

We all know that as a plain old web search now. But that shit didn’t really exist in the mid 90s. Not like that.

But that’s not enough. Let’s say you have an outline and the bottom/deepest node contains a question like “How does this affect your pricing structure?” Well that’s no fucking good. You can’t search for that. But if you take every parent, grandparent, etc, item up the line and just append them all, THEN filter out the noise words and run the search?

Yeah you can automatically populate more than 80% of the answers with no human interaction (aside from the verification step to be sure everything matches up cleanly, which I insisted on because I didn’t want the legal team from an insurance company up my fucking ass.)

In fact the tests were SO good that I predicted we could indeed reduce headcount by a tremendous margin.

They heard that and bailed out on the project. Weren’t interested. “Nope. We can’t say that.”

Bunch of fucking cowards.

BUT

Take some of the ideas behind that tech. You can replace “removing noise words” with “search for statistically significant digrams and trigrams” and then look for clusters of occurrences of those across the document space.

Now what would that give you? The software itself wouldn’t “understand” shit. Software can’t. (Sorry Carmack, no AI for you.) But what it WOULD be able to do is say “Hey, these two documents talk about the same thing. Not only that, but talking about topic X (having a preponderance of commonly related statistically significant phrases) crosses over with talking about topic Y.”

So it would be able to figure out that Document A and Document B were topically related and keep that information indexed.

How the hell would that help?

At the end of the day, one of the things I’d be able to do is write a few sentence paragraph and submit the thing, looking for other instances of the same topic.

So no more mindless repetition!

(Lol. That one was just for us.)

Now, a couple other things:

Because the author is always one of three people (them being me, myself, and I) the style will be remarkably consistent, especially since I tend to codify the same topics over time using the same nomenclature. So such a thing should work particularly well.

Plus, it’s not inconceivable that I could take these pseudo-topics and use them to come up with a category or topic name. There can’t be THAT many of them, and the process would be purely additive, so I could just kinda chip away at it.

THAT way I’d be able to tell if I’d written about all this before having embarked on writing this post about it, possibly again.