HHC: Brainstorming Part 2

 

Unfortunately I’ve been running up against a wall in my previous todo list and have subsequently decided (however tentatively) that I’m going to be writing all this stuff from scratch, which irks me to no end.  RSS feed readers, on-screen notification systems, local content repositories, etc.  I just have to be prepared to write it all from scratch, then keep my eyes open for toolkits (at both the product and library levels) that will help me skip ahead a bit.  So while I’m sketching that out, let’s keep going with the process of breaking down the initial list into more concrete requirements and todo list items.

  1. A contact-management system that would make Harvey MacKay faint from information overload.

I’ve always wanted to tie together my flow of email with a real contact management database.  Partially for a simple address book that’s not hosted on some effervescent mobile device, partially for “business networking” (something that was important to me back then and is suddenly starting to come back around) and partially to keep an eye on who I know, what I know about them, etc.  Yeah, it probably sounds a little Big Brothery. But I think it would be a fascinating tool that wouldn’t require so much in the way of active maintenance effort.

Now, I could just use a CRM system.  Something like SugarCRM. Just pull the thing down and slap it in to a database.  So I’ll check around for something like that. (I have enough professional experience with Sugar to know it’s positively awful, so it’s definitely not a contender.)

  1. Notifications and ticklers (woohoo!) for the following:
    1. People who’ve emailed me who I’m “officially due” to contact
    2. Software projects and updates that I’m interested in but may not have explicitly subscribed to.
    3. The same thing with commercial products (books, music, movies, etc.)
    4. All of this to bubble up into the ticker application mentioned above as well as a “product wishlist” that’s automatically maintained and published to a well-known location so I never have to answer the question “What do you want for (insert holiday here)?” again.

Of course this feeds in to the contact management system. But here we start getting in to the interesting stuff, something I used to call the “Interest Engine”.  Let’s not put the cart before the horse.

  1. People who’ve emailed me who I’m “officially due” to contact

The unspoken component here is that I want to tie my email (and perhaps other forms of messaging) in to the contact database system so that I can see things like “you usually go a round of email with soandso every week, but you haven’t in a few weeks.  Do you mean to let that slide?” etc.

This isn’t a particularly easy problem, but it’s not exactly high-end computer science either.

The key to this, and how it relates to the other sub-items in #4 is that I want to be proactively notified not only of events (Frob sent you an email!) but of peculiar gaps between events (hey, you haven’t emailed Frob in a month!)

The other three things on that list are essentially “event sources” in that they’ll take the shape of a few search engines that’ll notify me if the results change.  So if I like Deadpool (and if you don’t you’re wrong) this thing should take a peek on the internet every “time interval” and see if something interesting pops up (what “interesting” means is an interesting question) and creates a similar alert.

Add to that the other items, software updates, youtube channel updates, movie releases, etc.

I have to confess, that #4 item I don’t really understand.  One problem might just be that back in 2003, Amazon wasn’t SUCH a big deal and my wish list just wasn’t that well developed.  But now? Now I really think that’s solidly covered.  Besides, what good is maintaining a wish list (which is ostensibly for sharing) on a personal internal closed network application?  So I’m officially striking #4.

So what this all comes down to for these two major-heading items is three things:

  1. contact management database
  2. Email repository
  3. An engine that creates custom notifications based on a variety of heterogeneous data sources, both internal and external

Fortunately the email database is something I already have and use.  I created a little custom database and a few scripts that parse mbox files (from Thunderbird), break them out into individual messages, then insert them in to a relatively logical database.  I’m encountering some interesting capacity issues now that it’s growing with email content bodies. But I’ll be able to manage that well enough.

The todo list:

  1. Roughly outline what I’m going to want in my contact management database.
  2. Use that outline to shop for off the shelf systems that have the flexibility, UX, and open architecture I need.
  3. Create a single sample of a custom rss feed off of a scraped website of some kind.
  4. Create a custom rss feed off of incoming emails from one or two email addresses.

Should be fun.  As for the last todo list, I’m waiting on a couple resources to start coding up the custom feed management system, so this will dovetail nicely with that.

 

 

More updatesauce

Well, it’s pushing 23:00, so I’m not doing a whole lot more coding or writing today.

Using Thunderbird’s feed reader has absolutely no applicability to what I’m trying to accomplish.  What I need is to use rss as something close to a message queuing format, so I’m probably going to write my own over the next few days (with the help of some canned code, no doubt.)

Work’s been driving me up a wall, but it’s fine.  Normal technology and people stress so I’ve been coming home and just shutting my brain off.  Nothing to be terribly proud of. but the last thing I’ve wanted to do at night has been more coding.

More posts from the vault coming in the next couple days and I’m working on a new one or two little vignettes from the wild.

o7

 

HHC: Prototyping frustration

I really, honestly and truly expected to be farther along with the quick little todo list from the other day by now.  But I’m thwarted by trying to do things the easy way.

Sure, it’s just an education problem on my part. I’m sure Growl for windows and/or Snarl are just fine.  But I can’t seem to get them to pull RSS feeds and I’ve wasted too much time on it already.  So I’m bailing on that approach for now while looking for another feed reader.

I didn’t want to get quite so hung up on this, but, for the purposes of a technology proof, a notification engine of some kind (even if it’s something that ends up being replaced or rewritten six or seven times) is a core piece of functionality.

Sure, I’m sorta reading ahead, because I’ve got a pretty good picture in my head of what I think I want this all to look like on the back end (a design that may not survive contact with the development, to be sure.)

So what I’m going to do right now is pick up a couple RSS Readers and get them running so I can at least have something scaffolded out.  Hell, I may write my own before this is all over. I’ve done it before, it’s not exactly a high-end piece of computer science.

So… RSSOwl, here we come.

And, as always, if you’ve got a suggestion for a better component/solution to some of this stuff, I’m all ears.

EDIT: So.  Fucking.  Frustrated.  I installed RSSOwl, but it wouldn’t start. So instead I’m using Thunderbird’s RSS support.  It’s definitely NOT what I want.  It won’t handle the style of “push notification” semantic I’m looking for. BUT I put all my reddit RSS feeds in there, so at least it will serve the purpose of behaving as a proxy so I don’t have to go monkeying around directly to every social media site.

“Whatever. It’s…fine.”  *twitch*

 

HHC. Brainstorming, part 1

What I’m going to do next is just go through that feature wishlist I came up with 13 years ago, and put in the earlier post, and roughly dissect the items and see if I can’t come up with a practical feature list of some kind.  I’ll extract a running todo list from these little brainstorming sessions and keep them in some kind of special post someplace (I’m sure I can do that here in wordpress somehow.  I’ll figure it out.)

Ordinarily my inclination would be to lay all of these pieces out on 3×5 cards or a huge whiteboard or something, so I could design it from scratch (top up and bottom down, sorta meet in the middle.)  But as an experiment I’m going to just try and get things going as soon as I reasonably can, looking for low cost inter-operable pieces that I don’t have to write myself.  Yeah, maybe I get half way through and realize the approach won’t work.  That’s absolutely fine.

One architectural note that’s going to figure largely in my designs:  I’m not giving up my data.  The master location for all of my data MUST be on the machine at my feet.  Perhaps I’ll publish some data to a web service, but strictly as a client, if at all.

Let’s begin at the beginning then, shall we? I’ll try taking two at a time.

  1. A real-time ticker application that will be updated using RSS. (Not the current “refresh to update” silliness in aggregators of today) Publishable schedule items (to the rest of the world or a specific subset at my option)

What does that mean?  It’s really a few different pieces of functionality.

  • “real-time ticker”  That’s a marquis or on-screen display system for events.  So, if I’m going to go with a monolithic application, a single scrolling window.  BUT let’s keep things separate and use a desktop push alert system. THAT means something like Growl or Snarl that will capture updates (whatever they are, we don’t really know yet) that will be published at will.  Since Growl seems to have the larger market presence, I’m going to start with that.  It pops up windows with notifications published from “someplace.”  I assume I’ll be able to digest RSS somehow and feed it to Growl periodically.
  • “Publishable schedule items” To be frank I have NO idea what I meant by this.  Publish to who?  I already have my schedule.  For now I’ll translate this to: “Pick a calendar application such as google calendar, a desktop PIM application, Mozilla Thunderbird’s ‘Lightning’ extension or similar that will allow me to sync items between devices, making it available to other HHC components.”  Fortunately I already use Thunderbird for email (it’s really useful for managing more than one email account, much less the dozen plus I’m juggling) and, while I find the Lightning extension a bit cumbersome, it might be a good candidate

2. To subscribe to lists of events that are occurring around my area geographically, and virtually and have those events appear in a calendaring application. To use a blog or blog-like publishing environment (I’m thinking Zope with  CMF) for a personal desktop heads-up-display console from which I work at all times.

  • Ah, the heads-up-display item.  I really wanted to have an application with a gui component that would live on the right side of my screen containing a little calendar and some other stuff (which we’ll get to later), such that if I maximized other application windows, it would still be, not “always on top” but would shrink the client window so that it was always there (but not obscuring anything.)  In older versions of Windows, you could create an application that would do that.  I’m not at all sure if you can in Windows 10 or not. This needs research.
  • Subscribing to local event lists is easy.  I just have to find and select some.  Things like meetup, etc.  There are a few sites, local subreddits, yadda yadda.

Todo

  1. Install Growl and publish a test event or two.
  2. Create test case for publishing an RSS feed of some kind to Growl.  (I may have to digest a feed and publish some other kind of event.  I’ve no idea.  Needs proof of concept work.)
  3. Put calendar items in Lightning (already installed) and see if it’s open enough that I can get to them.  Start looking for alternative if that’s not the case.
  4. Find out if I can create a client-diminishing sidebar application window that will stay up at all times.
  5. Dig up a half dozen feeds or subscription services (even if I have to screen scrape some html) with local events (specifically local to me.  Make it work first. Make it generic later.  Premature optimization is the root of all evil.)

There. So I have five discrete todo items to take care of.  Frankly this all shouldn’t take more than a few hours of poking around and testing stuff out in one coding platform or another.

And we’re off.

 

o7

 

HHC, Exposed

In a meager and perhaps completely futile attempt to keep myself honest I’ve tentatively decided that I’m going to chart the progress of actually putting the HHC system together here, complete with notes, design decisions, etc.

I’ve spent an awful lot of my life hoarding hard-won knowledge and while I don’t think there’s the slightest thing wrong with that (you fuckers still aren’t getting my cookie recipe) I’m not sure the tactic has done me any favors.

So in the interest of experimentation, let’s try this out and see what comes of it.

Huge Honkin’ Console…back from the dead

I’ve been working through my archives of writing, old blog posts, etc. and I came across the original post I made back in 2003 (really? 13 years?  ugh) talking about my fantasy “Huge Honkin’ Console” project.  Going through it I realized (cringeworthy editorializing aside) that I pretty much still want every single thing on this list.  I may load up the actual page somewhere here, as it’s a marvel of early 2000s Radio Userland blog formatting (past the age of spinning guitar bullet icons, but too early for magazine layout web pages, which suck just as much.)

So here’s the outline, for your amusement and my accountability, pasted right out of the original post text’s html.  I’ll even leave in the crappy links for now, to show how much the blogosphere (remember that word?) has changed in the last 13 years:


I want…

  1. A real-time ticker application that will be updated using RSS. (Not the current “refresh to update” silliness in aggregators of today) Publishable schedule items (to the rest of the world or a specific subset at my option)
  2. To subscribe to lists of events that are occurring around my area geographically, and virtually and have those events appear in a calendaring application. To use a blog or blog-like publishing environment (I’m thinking Zope with  CMF) for a personal desktop
    heads-up-display console from which I work at all times.
  3. A contact-management system that would make Harvey MacKay faint from information overload.
  4. Notifications and ticklers (woohoo!) for the following:
    1. People who’ve emailed me who I’m “officially due” to contact
    2. Software projects and updates that I’m interested in but may not have explicitly subscribed to.
    3. The same thing with commercial products (books, music, movies, etc.)
    4. All of this to bubble up into the ticker application mentioned above as well as a “product wishlist” that’s automatically maintained and published to a well-known location so I never have to answer the question “What do you want for (insert holiday here)?” again.
  5. A database of articles and references to articles (and other reference materials) on a huge honkin’ LOCAL hard drive that’s accessible through automated AND manual topic discovery as well as a full blown search engine.  (For instance:  All of Shakespeare, the “30 days to a more readable blog” article set. Comments on my own posts off of UCCU.)
  6. the built-in modem that comes with my computer to behave as my
    answering machine, saving audio transcripts of incoming messages as well as recording live conversations. AND:

    1. Trap caller-ids and match them against my address book, entering the call event PERMANENTLY in an ever-growing log system.
    2. A record to either automatically or at my option (selectable at run time) pull up references to the caller, and lists of communications over time. Including but not limited to:
    3. Incoming and outgoing phone calls with:
      • speech-to-text generated transcripts
      • instant messaging transcripts
      • lists of instantly accessible emails between both parties.
      • cross-references between other parties involved in all mentioned conversations.
  7. Intelligent Agents (ooh, blast from the past 😉 to be constantly indexing and searching the internet for other things that I might be interested in.
  8. Fully functional annotation engines to work in conjunction with my window-on-the-web environment so I can mark-up both cached and remote versions of web pages for future reference.
  9. The ability to publish these annotations (the way a few applications used to be able to do… uTok, etc.) so that other people can view them as well.
  10. Ubiquitous PGP (or equiv) system for communication across email, instant messaging, etc. People will need to be approved to be added to the list.
  11. Notifications when people read, comment on, or annotate anything I’ve published out in publicly-accessable land. (and all the other psycho TrackBack, Pingback features that have shown up in the blogosphere in the last year or so.)
  12. Published playlists from iTunes, WinAmp and any other damn thing. Note I don’t mean publishing the mp3s, just the playlists so I can compare with other people and see what music I might be missing in the world. (Hilary sit the hell back down and shut up.)
  13. Remote whiteboard chatting  (Jabber plug-in anyone?)
  14. horrifyingly high-performance rpc mechanisms (XmlRpc for now, but something a bit more svelt in the future) to distribute this work load on my local lan however I see fit (or auto-balanced.)
  15. Published blog entries (like this one) to auto-annotate with links to the right places (i.e. Jabber, uTok,(no longer available.) , Zope, CMF, RSS, etc should all be auto-linked without me having to “create shortcuts” in Radio (which just took about 20 minutes).)
  16. Integrated desktop:
    1. liveTopics
    2. Wiki
    3. Blogging
  17. Built-in mind-mapping and diagramming toolkits for charting ideas and representing them textually once the “virtual whiteboarding” session is done. (Not to mention the automated post-session analysis and discovery phase designed to extrapolate on behalf of the participants.)
  18. That cool little thing from the AT&T commercial a few years ago where the girl plays the first 3 notes of a song on her guitar and the Agent goes and grabs the song title.
  19. Live chatting and always on IRC with private subscription-only channels between circles of friends.
  20. Newsgroups as RSS feeds, distilled into threads automatically, with a signal-to-noise ratio rating accompanying each article and group.
  21. And I want it ALL in a unified source-accessible platform (I’ll pay.  Doesn’t have to be open source.  But I will need the source)
  22. A massively flexible api for developing plug ins for the 50,000,000 things I haven’t thought of and the 150 or so I’m gonna make money on and therefore am not mentioning here.

Adorable right?  But it highlights some really fascinating things to me. As contrasted with the internet back then, the web is a worse place now. None of this stuff seemed the least bit outlandish to me back then, and certainly it’s much more within reach now.

This is what the conversation was like back in the day (something something ‘damn kids’.)  People talked about the possibilities of technology and the web and how we were going to integrate it in to our lives.

There were all kinds of neat ways for wiring my website with your web site so we could have our own little domains but still keep abreast of what was going on with everyone else.

And I wonder still what really happened.  Certainly the blame can be laid largely at the door of social media and later at the rise of the smart phone.   There used to be cool applications for interacting with websites. Now it’s just “post this to twitter, share this on facebook” and that’s it.

Let’s take my favorite example from back in the day: uTok.

uTok was a horrifically insecure little piece of software.  It would attach itself like a cute little barnacle to your web browser and, for every web page you visited, it had a live chat room of other uTok users who were accessing that page (or site, I forget the granularity) at the same time, a comment section that was live.  It was great. And yeah, of course it worked by sharing your browsing habits to a centralized server and recording your every move to provide tracking information to advertisers (I presume.)

BUT IT WAS FUCKING COOL!

Remember ICQ? Yahoo Messenger? AIM? Little chat applications that allowed you to just have little IM conversations without bleating everything that comes out of your mouth in to public space. And yes, you can direct message on twitter, or message people on facehole.

Yes. On one hand, I’m That Old Guy now. I’m absolutely fine with that. (And once you hit your 40s, you’ll understand why, you little shit.)

What I don’t know is the answer to this:

Has the immediacy of bite sized instant gratification social media actually resulted in a dumbing down of discourse (across all subjects) or has it simply resulted in a democratization of use that’s made it appear that way by reducing the accessibility barrier to the unwashed masses of users?

Admittedly, you have to write stuff to have a blog. You’ve got to have at least a marginal facility at stringing words together, coupled with an intention to do so.  That right there shuts out most of the population of the Little Blue Dot.

And it could also be that I frequent things like Twitter, where bite-sized bleats are all you can really manage, or reddit, where messages are fire and forget, discussion groups with no real continuity.

But for now I’m going to start at the top of that list and see if I can’t chart my way through and estimate what it will take to get that done, maybe even take a bite out of it.

 

o7