The Evolution of Automation

After 5 lessons (well, 10 really), I thought I'd pause and take a look, not at specific automation code, but how automation evolves out of current processes. One of my first posts on this site was about the reasons to automate (I'M HAVING AN INDIGNATION MEETING). It gave four reasons why to automate and I've spent the last five weeks showing how to automate. In this article, I want to discuss how current processes evolve into an automated process in an office environment (can you use anymore Darwin buzz words without talking about 'Natural Selection'?!). There were a couple of surprisingly useful developments in a recent project that I hadn't actually planned out but just organically evolved when the project came to critical junctures.

The other day a project came across my desk that needed some updating. For some time, I've been retrieving some text files via a secured web server and adding some html to the contents to deliver to end users via a different secured web connection. Due to the security setup at my organisation, there had to be a lot of 'tweaking' done to make the automation server (let's call it 'Athens') connect with the data server (let's call it 'Carthage') as well as the machine that was to be the final resting place of the data where the end users will access the report (let's call that one 'Hellespont'). The process was executed by a 7 year old VB exe that frankly was a pain to deal with. Removing reports were a pain; adding reports were a pain; not fubar-ing the entire process doing either of the above was a pain...Not knowing what you were doing with VB was a pain. The whole thing was enough to make you ignore it till you could get your hands on an Impala outfitted with a military class JATO rocket, put the entire source-code and exe in the front seat, flip the switch, sit back and watch the glory. Well, as it turns out, the final security piece fell into place before I could get my hands on that JATO and I knew it was time to grab a large-caliber handgun, a tac-light, and crawl into that Damnbeast den (better make it machine pistol...um...actually two (John Woo style you know)....with extra ammo..all armour piercing of course...(the bullets not the pistols)...although....armour piercing pistols....hmmmmm...anyway). I then started to map out how I was going to make this thing work.

My first decision was to make it easy to add and subtract reports from the process (see above). I didn't want to code each report into the function separeatly because that would take a 10 line process and make it 'the-number-of-reports' times 10 lines. In case you weren't sure of what to call that, it's called 'Ugly and Kludgey'. I decided to create a text file that would contain the address of each report needed on the server Carthage, as well as, the final name of the report when I place it on the server Hellespont for distribution. These two pieces of info would be separated by a '|' character. It is just a very simple text document that I can step through one line at a time using 'foreach' to parse the file and and 'gett' to parse the line. The file is sort of a 'definitions' file. This method has several benefits:

1. to add a report, I just add one line to the end of a text document rather than opening up QM to add like 10 lines of code to the 180 that are already there (if there were say 18 reports). When I say 'I', I'm really saying 'any of my co-workers' who need to add a report later on because I got hit by a bus or won the lottery.
2. to remove a report, all I have to do is delete one line rather than opening up QM and deleting like 10 lines of code. When I say 'I', I really....well you get the point....It's called Agile baby!!
3. If reports are added or deleted, there's no need to make 'versions' of the program (like we do now) because I'm not actually changing the code.

So, now that I have my list of reports and can get info on each of them individually, I need to build the routine for Athens to get them from Carthage..... And here we have all this simple and wonderful code using cURL to retrieve the files from Carthage and load them each individually into a string variable . I'm not going to go into... (wow, that didn't work well at all)...I'm not going to go into what I'm doing to get them or how I'm manipulating them (maybe I'll go over that another time); I'll just say that for https retrieval, cURL is really, really, nice!

So, to sum up, I've retrieved the files from Carthage and modified them so they look better for the end user and then placed them out on Hellespont for the end user. When I ran it the 'first' time it worked great till I realized one of the reports is no longer used. No problem, all I have to do is delete that line and....wait....that is never a good idea when dealing with capricious users (or any for that matter). I'll tell you a little principle I've developed in my years of report design. The report that has not been looked at for over a year by a single living soul will never even be thought of until you delete it from your report delivery system. Then an admin assistant is on the phone saying they want it....oh, and they want it for a meeting....today....at 9:30.......um, it's 9:40 now....yep. And here it is in business speak:

The value of a report is not based on its actual use or even its usefulness but rather directly on its difficulty in recreating its process after it has been discontinued....for six months....by someone other than yourself.....who quit because they won the lottery.......you just wish they would get hit by a bus....while you were driving it.....which is what you'll be doing for a career if that admin assistant doesn't get that report printed within the next 10 minutes.

I don't have a catchy name for it yet but now that it's published I have a copy-write on it 80) ....BTW: I'm taking suggestions for the name.

So, as you can see, I'm a little reluctant to just delete the report entry. [Enter Stage Right: Comment Delimiter] ....Ooooh, excellent! Comment delimiter! I can actually use almost any character since all the report addresses have to start with the '/' character. I chose to use the '#' symbol because I'm used to it when I'm doing Korn shell work in Unix. My next step was to use 'find' in a simple 'if' statement to skip the entire process if the first character is a '#'. Done. Now I can keep that report ready in case someone wants it again.....wait a minute....I can also use a 'Comment Delimiter' to do things like 'make comments'...just in case I'm found guilty of a crime for hitting that guy with a bus and have to go to prison or something...yeah, I know, long shot; no jury is going to convict me of 'painting a new center-line' with that guy after deleting that report process without documentation. So without any extra code, I've also added the ability to place as many comments as I want anywhere inside the file as well as deactivating reports.

The 'definitions' file and the commenting capability were things that weren't specifically planned on, they just 'grew' out of the process as it went along. It solved problems I hadn't realized and became more and more useful as the project progressed. The comment capability really makes the whole project much more Agile since anyone who comes in behind me can easily find out what I'm doing from the comments in this definitions file. Which leads me to the next step...documentation.....uhgh.....talk about a Damnbeast.