Now, I know you all have
long wondered what a typical deploy cycle looks like. Who are the players in these
mysterious circles? What sorts of
science create the
magic of a (mostly) functioning web application? How do the various parties
work together to create these glorious examples of intelligent application of information systems technology?
WONDER. NO. MORE.
It pretty much goes down like this.
Our Players:
This is Princess. She is usually some combination of business-type person, analyst-type-person, and project manager.
: This is $DEITY. Which you don’t really need to know ahead of time because $DEITY will tell you so
every time he speaks. He is usually Princess’s boss (or her boss’s boss), and will occupy a fairly high rung on the food chain.
: This is Coffee Cup Guy. He probably has a title like Database Developer V. Probably also had a new nameplate drilled for himself that says “Coffee Cup Guy: DEV V” on it within
hours of the promotion. Tends to be dragged into things whenever Princess isn’t hearing what she wants to hear, agrees with her that whatever it is would be “very simple” to implement and then proceeds to actually
do…nothing.
: This is the new kid, Ms. QA. Wait. Ms. Analyst. Wait. She also does development work. Even though she has absolutely
zero dev experience. But that’s ok, because Coffee Cup Guy will tell her is it
very simple, point her at the first site he found on Google with at least
some matching keywords and then claims he ‘taught her everything she knows.’ VERY enthusiastic, somewhat light on experience and commonsense.
: Here we have Tech Lead. He might be a for-real manager, he might just be The Guy Formerly Known As Coffee Cup Guy, before he titled-out of the system and had to be promoted to a management position due to there being no ‘Developer VI’ in the HR book. Cool guy, but tends to not have time to get into the trenches with you.
: And this is our developer. Or, as we like to call him for the story being told here, the only sane person in the whole @^*&@ing organization. (Note that who has this ‘only sane person’ title varies from story to story…sometimes it is the developer being a complete idiot, but for this particular vignette, he is our one sane crew member.)
The Morning After The Last Disaster Deploy:
: OK, so, Developer, here are
your tasks for this release. There are nine of them total, but I want to bring your attention to
this item in particular as it is a Priority 1 task. I have attached the finalized requirements documentation for your review; I don’t think there are any surprises there, as we discussed this earlier. Keep in mind that as a priority 1 task, it is a
must go, so, the whole thing
must be 100% finished before 7/15. You sized it during our pre-sizing meeting last month at 15 hours, so this will not be a problem.
: Hmm. OK. Let’s see here. Oh yeah, I remember this one…land new data from
blah blah blah…whoa-whoa-whoa, time out! Princess, what we discussed and I estimated 15 hours for was one (1) new package to load three (3) tables from one source. This item has twenty-seven (27!!!!) new tables from FIVE (5!!!!!) sources?!
: Ahem.
We discussed this. Oh. Wait. You weren’t invited to that meeting. Because you did not need to be. Plus I think you were trying to catch all those snakes somebody released in the server room…did we ever figure out who
did that?! Anyway, don’t worry, we
did adjust your sizing to account for the additional tables.
: Oh, I see, OK, well…
heh heh…yeah, because, the
last time you did this,
heh-heh…
: We’re not
idiots, you know!
: Heh-heh, well, I wasn’t trying to imply…
: So, you have 20 hours in the plan for this.
: …
: What?
: …
: WHAT?!
: That’s a big old
negative, there, Princess. I’m gonna need more like 150 hours total. This is four
brand new sources. I have to set up application user ids, get connections strings, update Oracle linkages, get data review and governance, build four additional packages, not to mention creating all the objects themselves…
: Pffft. It’s cookie cutter. Once you’ve done the first one you can just copy/paste the others. 25 hours.
: It really is
very simple, you know. In my experience, Cntl+C and then Cntl+V will work for that.
: Exactly. It is
very simple so it really shouldn’t take
that much longer. You’re being ridiculous.
: Uh-huh. Very simple. Copy/paste. Gotcha. Tell you what: How about you
show me how simple it is, Princess…?
: I am $DEITY. Let’s keep the conversation
constructive.
:
Sigh. 70 hours.
: No.
: Do you want this to fail?
: No. It cannot fail. It is a
must go item.
: 70 hours.
: No. I can give you 30. Plus Coffee Cup Guy.
: No problem. Always glad to help the juniors out with stuff they don’t understand.
: …
: 600 hours.
:
Fine. No Coffee Cup Guy, and I’ll give you 35 hours.
: 60.
: 40
: 55
: 45
: 55
: No. You’re supposed to say 50.
: On this planet, we have an assortment of things called
laws. There are the laws of
time and also of
space, and…
: DADDDDDDYYYYYYY!
: I am $DEITY. Princess is right. You will have 45 hours.
: {thinks}
@^*&@ you, $DEITY…
: I AM $DEITY. I HEARD THAT.
: {thinks harder}
CAN YOU ALSO HEAR THI-
: {texts} Time outta, bro.
: {texts} Doughnut comment suey side on this.
: {texts} Dammit. Don’t. Comment. Suicide. LOL. Autocorrect.
: {texts} *commit
: {texts} I got ur back. Will dizcus w/Big gay l8r.
: {texts} Dammit.
: {texts} Big Guy. LOL. Autocorrect!
: {texts} agree w/Pins for now, will fix l8r, gtg letz bee done hair!
: {texts} *here. Argh. Autocorrect.
: {texts} OK, thanks. {says} OK,
fine, 45 hours for initial rough-out, but my lead is going to circle back with you…
: {texts} np, btw, am OOO for 6 wks lol but this 1st thing when back, kk?
: {texts} WUT?! out of office, WTF man?!?!
: {texts} in hq mtgs then PTO, sent email?
: {texts} @^&@!!
: {texts} all gud, I got this 4 u
: So, actually, my lead is out, maybe we should revisit…
: You said you were fine with 45 hours! It is carved in stone on these tablets! WE ALL HEARD YOU!
: I am $DEITY. It was spoken. It is now written.
So it shall be. 45 shall be the counting of the hours.
I am $DEITY.
: …I hate my life…
so much right now…oh well, maybe a miracle will happen…let’s just…moving on, where was I in this
unending nightmare…blah blah blah tables blah blah blah schedules blah bla-…wait…you want to land all this garbage directly into…um…hang on…you realize that’s the
application database, right? That’s our
transactional tier, we can’t start landing large tables there, they have to drop into staging first and then…
: YOU ARE BORING ME. I NEED THEM IN THE APP TIER SO THAT I CAN PLAY WITH THEM AS THEY ARE MY NEW SHINY TOYS. MAKE IT HAPPEN.
: Um, I really have to push back on this one, because see, if we do that, we will be using up resources that are needed for even super-basic functions within…
: PUH-PUH-PUH! Just stop right there. Someone wants to say something to you about this!
: I am $DEITY. Do what Princess says. My orders. It is necessary.
: …
: …
: Ooookayyyyy, this has Bad Idea written all over it, but, you’re the boss…lessee…you want to land
etc. etc. etc…lemme just take a look at these source tables HOOOOOOOOOLY CRAP-APPLES OF DIVINE RETRIBUTION +10 OF THE GODS!!!!
: {does bunch of calculations on scratch paper} {does them again} {pales} {re-reads requirements} {throws requirements across room} {picks requirements up again} {does more calculations} {picks up phone, has terse exchange with DBA team in re: storage space, buffer memory, transaction log size and so forth} {discusses what
kind of idiot would decide
this was a good idea with now equally irritable DBA} {rinses mouth out with soap} {returns to discussion}
: OK. Um. Let’s…just take a quick sanity check here…what are you going to be
using this data for again?
: Stuff!
: L’il more detail?
: Lotsa stuff!
: Not exactly what I need to know…maybe if I explain the problem…
seeeeeeeee, it’s going to take, 5-6 hours to pull
just this one 300+ column by 800 million row table, and, you’ve got
eleven others just like it listed here. That you want pulled
daily. Each table will eat up around 50 gig of storage space, plus of course using tables with
that big a footprint in update statements is, just,
wow…HOW ABOUT if we
just pull over the
eight or so columns we actually
use, and
only the source types we need…that’s 20 minutes start to finish and under 3 gig of data, which…
: BLAH BLAH BLAH,
just stop talking! SIIIIIIIIIIGH…look, I do not understand any of that mumbo-jumbo. And I don’t
know what I need. Or might need someday. Just pull all of it. Besides! We might want to do
analysis and some junk!
: …
: …you’re going to do “analysis”…directly on our
application tier… which runs our
web services…against tables that size…
lemme guess, you’re going to be joining them to every other table in the tier?
: Probably! We will need to see
all the stuff!
: …I am…it’s just…there are no
words for this feeling…
: Pffft! don’t worry! Coffee Cup Guy will handle that
analysis portion!
: SELECT * really is the fastest way to get
all the information from a table, in my experience.
:
Tee-hee! Oh, Coffee Cup Guy, you’re so
smart!
: Yes. Yes I am. Note how my coffee cup says “#1 Developer” on it.
Because it is true.
:
Uh-HUH. Well, I guess I’ll start working on the…
: Oh. Real quick, just a
couple little things…we kind of need it all done by Thursday afternoon at the latest. Because I hired a new QA person, well, actually, she’s more of a
BA but she has
super-crazy-MAD database skills!, and since she has
no idea what’s going on pretty much
at all yet, she’ll need the extra time for testing sooooooo we need it
done-done super-early so she can test it while it is still in the development environment.
: HI! LOL! I’M ENTHUSIASTIC! LOL! DO WE GET A KEYBOARD? ROFL!! HOW DO I CONNECT TO THE DATABASE DO I NEED A PASSWORD OR SOMETHING? ROFLMAO! I USE SELECT * EXCLUSIVELY BECAUSE SOMEONE SAID IT SHOWS ME EVERYTHING! I ONLY USE LEFT JOINS BECAUSE THAT ALSO SHOWS ME EVERYTHING! AND I DON’T KNOW WHAT A WHERE CLAUSE EVEN IS, LOL!! ALSO I KEEP TYPING ‘DATABASE’ INTO THE RUN COMMAND WHICH IS ALL I HAD TO DO AT MY LAST JOB WHERE I WAS THE SUPER HIGH RANKED LEAD DATABASE PERSON EXCEPT THAT MY JOB TITLE WAS ‘GOPHER’ BUT THAT WAS EVERYBODY’S TITLE PRETTY MUCH AND GOSH, THIS NEW BLACK SQUARE OBJECT THEY GAVE ME COMES WITH A CUPHOLDER AND EVERYTHING! SUPER FANCY! WAIT! WHAT ARE WE DOING AGAIN?! I LOVE IT! EXCEPT THAT YOUR STUFF IS WRONG, BECAUSE I COMPARED IT TO THE TABLE OF CONTENTS IN THIS BOOK I FOUND UNDER MY DESK AND I THINK THE DATA TYPES ARE DIFFERENT!
: OMG IT’S WRONG?!?! WTF, DEVELOPER?!?!
: …I haven’t even
started it yet, how can it be…?
: An
experienced developer would have known not to make that kind of rookie mistake…
: Does this mean the task is red? It’s red, isn’t it? We’d better have a meeting about this! The task is clearly red if the data types are wrong!
: THE DATA TYPES AREN’T WRONG! THERE ARE NO DATA TYPES YET! Geez. We haven’t even started the deploy cycle itself officially yet, so -
: YEAH LOL BECAUSE YOUR DATA TYPES ARE ALL WRONG IN THE TABLES I FOUND ROFL ABSOLUTELY NONE OF THEM ARE ‘PICKLE’ THEY’RE ALL THINGS LIKE ‘VARCHAR’ AND ‘NUMERIC’ ROFL I MEAN LIKE WHAT DOES THAT EVEN
MEAN?! RIGHT?!?! PLUS I’D PREFER IF THE QUERY WINDOW WERE PURPLE. CAN WE MAKE IT PURPLE? PURPLE IS GOOD. MORE GOOD THAN BLUE. AND DATETIME ISN’T EVEN A WORD, YOU KNOW, LOL? SO YOU TOTALLY NEED TO USE DATA TYPE = PICKLE THERE. IT ONLY MAKES SENSE. ROFL.
: …
: …
: …Kill it. Kill it with
fire…
: I do not appreciate your tone. If you had just done your coding right in the first place, you have nothing to fear from our new QA person.
: I’M NOT AFRAID OF HER! She’s just…
wrong. She
can’t be looking at the tables and finding issues,
because they do not exist yet. I don’t know what she
is looking at, but,
I’ll come in again!, whatever she just QA’d?
Was not THESE tables!
: I am $DEITY. The QA person is right. Your code must be wrong. The item will be pulled if you do not fix it per QA’s findings.
: With all due respect, on this one, QA is
wrong. Among many other things…“pickle” is not a valid data type.
: I am $DEITY. You will crate a new custom datatype called “pickle” and assign it to those columns.
: BRILLIANT PROBLEM SOLVING, SIR!
: That’s why they pay him the big bucks, for sure!
: JUST LIKE WOW ROFL!
: I
am $DEITY. Have a benevolent smile! {beams}
: I…work with
insane people…
: OO! MY ACCESS REQUEST CAME IN! WHAT DOES ‘SYSADMIN’ MEAN? IS THAT GOOD? DOES THAT MEAN I QUERY STUFF NOW?
: …
dear mother of god…
: I am $DEITY.
Watch it.
: I LIKE PICKLES!
: I will now quote an extremely long beat poem regarding pickles…a-chem!
Pickle…pickle…pickle-pickle-pickle…pickle on hamburger, pickle on rye…
: I’m just…gonna go over here and…see if I can’t somehow get a hundred hours of work done in the next two days…
: Is it done yet is it done yet is it done yet is it done yet is it done yet is it done yet is it done yet is it done yet is it done yet is it done yet is it done yet is it done yet is it done yet is it done yet is it done yet is it done yet is it done yet is it done yet is it done yet is it done yet is it done yet is it done yet is it done yet is it done yet is it done yet is it done yet is it done yet is it done yet is it done yet is it done yet is it done yet
: IS IT DONE YET IS IT DONE YET IS IT DONE YET IS IT DONE YET IS IT DONE YET IS IT DONE YET IS IT DONE YET IS IT DONE YET IS IT DONE YET IS IT DONE YET IS IT DONE YET IS IT DONE YET IS IT DONE YET IS IT DONE YET IS IT DONE YET IS IT DONE YET IS IT DONE YET IS IT DONE YET IS IT DONE YET IS IT DONE YET IS IT DONE YET IS IT DONE YET IS IT DONE YET IS IT DONE YET
: I am $DEITY. Is it done yet. I am $DEITY. Is it done yet. I am $DEITY. Is it done yet. I am $DEITY. Is it done yet. I am $DEITY. Is it done yet. I am $DEITY. Is it done yet. I am $DEITY. Is it done yet. I am $DEITY. Is it done yet. I am $DEITY. Is it done yet. I am $DEITY. Is it done yet. I am $DEITY. Is it done yet. I am $DEITY. Is it done yet. I am $DEITY. Is it done yet. I am $DEITY. Is it done yet. I am $DEITY. Is it done yet. I am $DEITY. Is it done yet. I am $DEITY. Is it done yet. I am $DEITY. Is it done yet.
: …
pickles on my eggs, pickles on my cheese…
: RU thr bro? lol, bahamas r gr8, hope L iz gud tlk l8r!
: Screw you, guys. Seriously. OK.
These are done and will import daily.
These five are too big to be pulled in daily. We don’t have enough disk space, plus it is causing some dangerous locking / blocking on the system in the test environment. Given the overall loads in production right now, trying to pull all of this in this way at that time of day might well crash the box. We need to rethink these and do them in a future release.
:
Nooooooo, it is a MUST GO. ALL the tables
must go!
: I am $DEITY. Make all the tables go.
: Oy!
Read my lips: Laws! Time! Space!
Cannot be done. PERIOD.
: It’s actually very simple, you know…
: Cool.
You do it.
: Ohhhhhh, yeahhhhhhhhh, would
love to take up the slack and get this done for you since you’re struggling with it, buuuuuuuuuut, I’m up to ‘here’ in pickle-related testing right now. Sorry.
: HEY! HEY! HEY! I CAN DO IT! I CAN TOTALLY DO IT! I HAVE MAD SKILZ! WATCH!
: Um, no, if you do that you’re going to lock down the whole app fo-
: I’ll mentor her. She can totally handle it. It really is
very simple. And mentoring is a major part of my job function. Developer
V stuff.
: I am $DEITY. She will do this work. This is a
must go item.
: You’re going to break the…!
: I FIXED IT! COFFEE CUP GUY SENT ME A WEBSITE LINK AND I MADE A PACKAGE LOL! AND IT WORKED AND NOW THE TABLES ARE LOADING YAY ME! ROFLMAO!
: YAY! You’ve saved the day! Gosh, I guess
you learned something today,
huh, Developer?!
: My pleasure, Princess. After all, I taught her everything she knows. Mentoring
is a major part of my day-to-day.
: I IZ A NATURAL TALENT ROFL!
: It’s so great working with such smart,
positive people!
Everyone should try to be more like you guys! {significant glance at Developer}
: {glances at code, winces, shakes head, closes program, sits back and
waaaaaaits for it}
(Fast forward to the morning after go-live)
(!!!!!!BOOM!!!!!!)
: OH MY GAHD THE SERVICE IS DOWN! NOBODY CAN DO ANYTHING! EVERYONE IS LOCKED OUT! WHAT HAS HAPPENED! IT’S THE END OF LIFE AS WE KNOW IT!
FIRE ALL THE PEOPLE! FIRE ALL THE PEOPLE!!
: I am $DEITY! Someone will fix this!
: WOW THERE’S LIKE NO DATA ANYWHERE LOL IT’S LIKE THE DATABASE ISN’T EVEN THERE ROFL IS THAT BAD ROFLMAO!
: Ya, I
told you guys this would happen…
: No you didn’t. You recited poems about pickles and said it was very simple.
: And that you taught Ms. QA everything she used to build the package that blew out the transaction logs, locked all the resources, corrupted half the tables and then threw the entire app tier into recovery mode.
: At 2:15 in the morning.
: Which was what time the REST of us got paged to come in and deal with it, by the way, just sayin’.
: WHAT’S RECOVERY MODE LOL? IS THAT LIKE FOR DRUGS OR SOMETHING ROFL?!
: I am $DEITY. You will take the database out of recovery mode immediately.
: Can’t. That level of access is restricted to the DBA group. As I’m sure you recall, they don’t report to You, sir, they report to Herself. I’m sure you’ll be hearing from
Her soon. Oh hey, is that your pager going off?
:
…mother…of…Me…
: Yeah, pretty much. Have fun with that, sir.
:
…whimper…
:
Aaaaaaaaaand, I’m BACK! What’d I miss?!
: Nothin’ much. The usual.
: Hokay then! If you need me, I’ll be unavailable due to meetings!
: Cool. Later man.
: So. While we’re working on fixing the problems
your bad code caused…
: …wait,
wut…?!
: YEAH LOL B/C THAT’S LIKE REALLY BAD CODE ROFL YOU SHOULD HAVE KNOWN BETTER THAN TO DO THAT ROFLMAO!
: …the
@^*&@?!?!?!
: No worries, man. Everybody screws up big time now and then. Except me. I’ll clean up your mess for you. You’re welcome, bro.
: Don’t touch it. Seriously. Look at my eyes: I will
literally, with malice aforethought and
quite thoroughly
, kill you dead if you even
open the package and look at it.
: No, no, hey: it’s cool. I’m
here for you buddy. Gotta mentor the juniors, kinda part of the gig for the #1 Developer. {points at cup}
: YOU ARE NOT THE…
: …aaaaaaaanyway, I’m gonna need your sizing on these items for the next release…we don’t exactly have any requirements documentation just yet, but I think what we’re trying to accomplish is
plenty clear without them, we just want to put the thing back in the other thing if a bunch of stuff is or isn’t a certain way. So! Now that you understand the whole
vision there…whaddya think? Three hours? Four?
: I quit. I mean it this time. I quit. Wait. You want to put
what things in
which place? Hang on, because, where are you going to get the…
{becomes absorbed with interesting problem to solve, forgets is quitting, quotes time estimate based on phone conversation with Princess about project because
never, EVER learns, like, EVER}
{go back to top and start over, because this is an
infinite loop that repeats itself over and over and over and over and over and over and…}