Clickings of the Neon Tapir

Case Study: Release Planning using Theme Estimation

Sat, 18 Feb 2012 11:44:50 -0700

This is a story about how I applied my CSM experience and training to help a mid-sized project get off the ground.

Beginnings

When I came onto the project, the company was struggling with agile. They had been doing software engineering for a while, but in the context of controlling hardware. This project, an item state management system, was a departure for them. They already had a product in this space which they were essentially reselling from a consulting firm. The existing product was awkward to use, its feature set was limited, and the consulting firm did not wish to re-write the product to scale.

So, the company decided to write a replacement. They chose C# and .NET as the language. “Mason”, the principal engineer on the project, was a Java programmer by trade, so they needed some help. I was hired onto the team as a senior .NET engineer who could help mature Mason’s technical capabilities in .NET, and they had already hired a mid-level .NET engineer who started a few weeks before I did. It quickly became apparent that my CSM knowledge would be just as valuable.

The team had the desire but not the field experience. Mason had written a prototype, and after quick examination of the code, it was clear that his architecture would not support the ambition of the product.

The project manager, “Jason”, had chosen Scrum as the delivery methodology, yet he had no experience with it. He was on the right track with sprint-level work management, but he was having trouble with release-level planning. he asked me if I could help get the project back on track. The product owner, “Kathy”, already had a comprehensive set of use cases outlined. While they were not fully-baked, they contained enough meat to discuss in detail what the project is trying to accomplish.

Theme Sizing

I introduced the idea of themes. Kathy had already grouped the use cases into broad categories, so it seemed natural to think of these categories as themes. Jason and the product owner went off and grouped the use cases. They did it more fine-grained than I’d anticipated, putting them into a couple dozen smaller categories that I’d call epics.

I then introduced the idea of T-shirt sizing for themes. The goal of this was to talk about which parts of the application contained the greatest effort. Some use cases have some risk — we as a team aren’t sure how we’ll tackle them. Others don’t have enough definition. Still others represent a lot of work.

Over the course of a couple of afternoons, the team got them sized. The challenge remained, how do you use T-shirt sizes to create something that resembles the Gantt chart the project sponsors were comfortable with?

Theme Points

It hit me, these T-shirt sizes were a scale, just not a numeric scale. I needed to quantify T-shirt sizes.

Mike Cohn has an article about relating story points to hours in which he shows story points graphed as Bell curves. So I wrote a scale like this on the whiteboard:

Onesie  |  Small  |  Medium  |   Large  |     XL

I picked an arbitrary number to represent a Medium. I wanted it to be on a scale that couldn’t be mistaken for user story points, so I chose 500.

Onesie  |  Small  |  Medium  |   Large  |     XL
                      500

Of course, a Medium is not exactly 500 points. These represent a broad range, so I declared that a Medium was 500±50%, or 250-750.

Onesie  |  Small  |  Medium  |   Large  |     XL
                  |   500    |
                 250        750

I extrapolated to determine the rest of the scale.

Onesie  |  Small  |  Medium  |   Large  |     XL
  60    |   180   |   500    |   1500   |    5000
       125       250        750        2500

This worked out to roughly powers of 3.

Onesie  |  Small  |  Medium  |   Large  |     XL
   1        x3         x8         x25        x83

Jason took these ranges and our commentary during sizing that Theme X seemed bigger than Y and assigned values to each theme.

So When Will That Be Ready Again?

By this time, we had already completed a sprint or two, enough that we had an idea that our velocity was going to be somewhere near 30. We had enough information to try to correlate these themes against a calendar. Projecting when we’d complete the epic’s functionality and armed with our velocity, Jason threw the numbers into a spreadsheet and started trying out numbers.

Our current epic was rated at 800 theme points. We thought we’d be done with it in 6-7 ideal days, which means 3 developers can get through about 750 theme points in an ideal week. This became 250 points/dev/week, a velocity of sorts.

Thanks to the spreadsheet, we were immediately able to see that calculated out to about 12 months to write the application.

Better yet, we were able to forecast what general area of the application we’d be working on months from now, which allowed us to project some milestones. We need to have this epic done by Halloween to be on track, that set of themes by Thanksgiving, and so on.

Of course, these numbers had a high margin of error. We communicated it would be more like 6-18 months to write it. We reasoned that as the team got a epic or two under our belts, Jason and I should have been able to refine the conversion rate and get a more realistic idea.

What If…

The real fruits of this exercise were that we now had a few variables that we could experiment with.

For example, that velocity of 30 was measured while we were storming as a team and 2/3 of us were new to the domain. Maybe we could expect the work would get easier as we learned the domain. This would be expressed as an increased velocity. If you increase the conversion rate from 250 to 300 after a month, you gain about a month over the life of the project.

A similar gain would be realized if a 4th developer is added after a couple of months. Other projections were made about the impact of adding a QA engineer early, which the project didn’t have until much later.

Course Correction

Of course, the project sponsors wanted it faster. To them, time to market was crucial. The company saw a dwindling market for their current offerings and were looking for ways to increase their reach. Their goals were compelling — it seemed like a good opportunity. This product would do things that others don’t.

Management needed a product that was feature-compatible with its competitors in the marketplace before the venture capital expired, so we were asked to come up with a plan that might possibly work.

We were able to use the spreadsheet to estimate how much of the application we’d have to de-scope to get it done in the timeframe we want, and Kathy was able to identify several themes as not mission critical to beta testing. The idea was that development would continue while the product was in beta, which I had discarded as too risky but Jason felt was viable.

Presentation

We presented the plan. It was, in the parlance of a senior company executive, a “success-oriented plan”. No one was comfortable with it, so some difficult conversations followed. The project was in a bad position.

There were two senior executives who vied for control of the project. One felt that the company should continue to sell their poor solution until the new one was ready. The other felt that the poor solution could not be maintained, should not be sold, and the new one needed to be rushed to market.

The second executive won and demanded a crushing work pace, complete with extended work hours. I knew from experience that such a pace was not sustainable and was prone to error on a product whose core had to be solid the first time. The customers in that market would not forgive a poor first showing.

I could not in good conscience proceed with the project as outlined, so I left. I don’t know exactly how the story ends. The other experienced .NET engineer on the team left for similar reasons. I have spoken to Mason, and he told me the project is very behind the schedule Jason and I laid out.

Conclusion

I think the exercise was worthwhile. Although the schedule Jason and I concocted was wildly speculative, it provided a frame of reference so that valuable conversations could take place. It got prompt and appropriate management attention to the project. The technique allowed us to do the analysis in a timeframe where its recommendations could be acted on.

Many of you reading this article have probably already formed opinions. It’s clear the project is in trouble. For me, it was that no project plan could overcome the fact that the delivery team did not have enough technical experience nor the business knowledge to rapidly develop such a complex product.

Without doing the analysis we did, I don’t think the errors in the original project plan would have surfaced until much later.

Revenge of the Secret Santa Code Kata - F#

Fri, 17 Feb 2012 10:11:10 -0700

Here’s another solution to the Secret Santa code kata, this time a scouting mission into functional programming with F#.

To recap:

My first solution, written in PowerShell, relied on selecting random pairs of people and using a “hill-climbing” algorithm to avoid getting stuck.
My second solution I constrained to be deterministic — no randomness.

This one was more about trying to write something meaningful in F# using a problem I’m by now familiar with. Take a look at the code:

open System
open System.IO

// TODO: what's the accepted way to configure an F# program at runtime, App.config?
let lines = File.ReadAllLines(@"C:\temp\names_santa.txt") |> List.ofArray

let surname (x : string) = (x.Split ' ').[1]

// TODO: how do I make this generic enough to use in both cases below?
let lastItem (a : string[]) = a.[a.Length - 1]

let randomizer = new Random(DateTime.Now.Millisecond)

let swap (a: _[]) x y =
    let tmp = a.[x]
    a.[x] <- a.[y]
    a.[y] <- tmp

// shuffle an array (in-place), borrowed from a code snippet site
let shuffle a = 
    let len = Array.length a
    Array.iteri (fun i _ -> swap a i (randomizer.Next(i, len))) a

let linesArray = Array.ofList lines

// TODO: this "do-while" loop, I should be able to rewrite it as a recursive function, but how?
shuffle linesArray 
while surname linesArray.[0] = surname linesArray.[(Array.length linesArray) - 1] do shuffle linesArray

let rec pairings = linesArray
                    |> Seq.windowed 2
                    |> Seq.choose (fun (x:string[]) -> 
                                        match x with
                                        | x when surname x.[0] <> surname x.[1] -> Some(x)
                                        | _ -> None
                                   ) 

let pairingsList = List.ofSeq pairings

List.append pairingsList [[|pairingsList.[pairingsList.Length - 1].[1]; pairingsList.[0].[0]|]] |> 
    Seq.iter (fun (x:string[]) -> printfn "%s gives a gift to %s" x.[0] x.[1])

Console.WriteLine "Done, press Enter to exit"
Console.ReadLine() |> ignore

As you can see, I left myself a few to-do items. I boxed the effort at three hours, which is why I went ahead and borrowed the shuffle logic from the internet.

The first hour was learning to speak F# again, especially the method signatures. I played with F# when it was in beta, so I didn’t go into the kata stone cold. However, it took me a while to troubleshoot why my methods didn’t have the signature I expected.

I spent the most time writing the pairings function. Once I had the insight of using the match operator, the algorithm came together very quickly. I decided that I wasn’t happy with it being deterministic. I found that if I provided a file that was already in an appropriate order, pairings regurgitated the list. Dissatisfied, I went about trying to shuffle the array. Not being familiar with unit testing in F#, I used the F# Interactive console. I got frustrated with trying to implement the shuffle algorithm, so I decided to study a working copy.

The hard part came in integrating it with my code. I resorted to using a while loop, which I know from reading about functional programs means that I’m not thinking of the problem through. I believe I couldn’t convert this into a recursive function because shuffle returns a unit (that is, it does the work inline). If it returned an array, I bet I’ve have more luck. I’ll have to experiment further.

My attempts to rewrite the loop badly broke the program. I used comments as poor man’s source control, and I got lucky. I really should have check in the first working version into my local Subversion repository.

Some time passed. I managed to look at the do-while problem again, and after about 90 minutes, I solved both it and another issue that bugged me.

Here’s the second revision:

open System
open System.IO

let randomizer = new Random(DateTime.Now.Millisecond)

let surname (x : string) = (x.Split ' ').[1]
let lastItem (a : 'a list) = a.[a.Length - 1]

let shuffle items =
    let upperBound = (List.length items) * 100
    let randomlyWeightedItems = List.map (fun item -> item, randomizer.Next(0, upperBound)) items
    let sortedByWeight = List.sortWith (fun (_, leftWeight) (_, rightWeight) -> leftWeight - rightWeight) randomlyWeightedItems
    List.map (fun (item, _) -> item) sortedByWeight

let shuffleUntilEndsDiffer theList =    
    let rec loop a =        
        let shuffled = shuffle a        
        if surname shuffled.Head = surname (lastItem shuffled) then             
            loop shuffled    
    loop theList

// Main part of the program, need to find out how to separate code into files

let lines = List.ofArray (File.ReadAllLines(@"names_santa.txt"))

shuffleUntilEndsDiffer lines

let rec pairings = lines
                    |> Seq.windowed 2
                    |> Seq.choose (fun (x:string[]) -> 
                                        match x with
                                        | x when surname x.[0] <> surname x.[1] -> Some(x)
                                        | _ -> None
                                    )
                    |> List.ofSeq 

let veryFirstPerson = pairings.Head.[0]
let veryLastPerson = (lastItem pairings).[1] 

List.append pairings [[|veryLastPerson; veryFirstPerson|]] 
    |> Seq.iter (fun (x:string[]) -> printfn "%s gives to %s" x.[0] x.[1])

Console.ReadLine() 
    |> ignore

The do-while loop was solved by making the shuffling function return the shuffled contents instead of unit (void in C# parlance).

The second change was to get away from arrays. Arrays in F# are mutable, and I wanted to only use immutable data structures as much as possible.

The shuffler was the main barrier. Once I hit upon the idea to associate each person with external data, in this case a random weighting, the rest fell into place. You may also notice some language usage improvements, such as using a.Head instead of a.[0]

Questions? Comments?

Secret Santa Code Kata Redux in C#

Fri, 10 Feb 2012 22:03:27 -0700

Here’s another solution to the Secret Santa code kata, this time in my “native” programming language of C#. This is the second solution I’ve written this week to this problem. Code katas are about experimenting with different approaches to simple but not trivial problems.

My first solution, written in PowerShell, relied on selecting random pairs of people and using a “hill-climbing” algorithm to avoid getting stuck. This time, I gave myself the constraint that the solution had to be deterministic — no randomness. I had been toying with the idea of using an Abelian ring. A couple of common examples are the hours on a clock or the modulus (remainder) operator in math. But I couldn’t decide how I’d handle iterating through the members of that ring without duplicates. I determined that I’d need to re-sort the list.

I wrote this solution using test-driven development (TDD), meaning I wrote my tests before implementation — I won’t show all of those tests today. As it turned out, I didn’t need to code a Ring class. I find when I do TDD, I often don’t end up writing half the code I thought I’d need!

Unlike my PowerShell solution which just used strings, I knew I wanted to create a data object to store the incoming rows of data in, which I named the Participant class:

using System;

namespace SantaKataApp
{
    public class Participant
    {
        private Participant(string firstName, string lastName, string email)
        {           
            FirstName = firstName;
            LastName = lastName;
            Email = email;
        }

        public string FirstName { get; private set; }
        public string LastName { get; private set; }
        public string Email { get; private set; }

        public static Participant Create(string descriptor, int id)
        {
            var parts = descriptor.Split();
            var email = parts[2].Replace("<", "").Replace(">", "");
            return new Participant(parts[0], parts[1], email);
        }

        public override string ToString()
        {
            return string.Format("{0} {1} <{2}>", FirstName, LastName, Email);
        }
    }
}

For a while, I thought I might need to implement IEquatable<Participant>, Equals(), and GetHashCode() in order to compare Participants, but again, TDD proved me wrong.

The plan for this approach was to: 1. Parse the incoming list 2. Resort the list 3. Display the results

The act of resorting took the majority of the development time. I created a ListResorter class to do the work.

I started by writing tests….

using System;
using System.Collections;
using System.Collections.Generic;
using System.Linq;
using Microsoft.VisualStudio.TestTools.UnitTesting;
using SantaKataApp;

namespace SantaKataTests
{
    [TestClass]
    public class ListResorterTests
    {
        [TestMethod]
        public void SwapItemsInList_Succeeds()
        {
            var list = new List<int>(new[] { 3, 2, 1 });
            var listResorter = new ListResorter<int>(list, null);
            listResorter.Swap(0, 1);
            CollectionAssert.AreEqual(list.ToArray(), new[] { 2, 3, 1 });
        }

        [TestMethod, ExpectedException(typeof(InvalidOperationException))]
        public void SwapWithInvalidIndex_Throws()
        {
            var list = new List<int>(new[] { 3, 2, 1 });
            var listResorter = new ListResorter<int>(list, null);
            listResorter.Swap(74, 1);
        }

        [TestMethod]
        public void CanAdjoin_WithTwoItems_DeterminesIfTheyCanAdjoin()
        {
            var list = new List<int>(new[] { 3, 1, 2 });
            var listResorter = new ListResorter<int>(list, (x,y) => x % 2 != y % 2);
            Assert.IsTrue(listResorter.CanAdjoin(1, 2)); // list[1] = 1, list[2] = 2
            Assert.IsFalse(listResorter.CanAdjoin(0, 1));
        }

        // ... and so on ...
    }
}

These are the first two tests I wrote. I decided the simplest operation I needed to resort a list was the ability to swap two items in that list. Once I had that working, I picked the next piece, the ability to decide if two things can adjoin (“to be close to or in contact with”), and I proceeded from there.

Notice that the LineResorter<T> constructor takes a list to operate against, and a function that compares two instances of type T and determines if they can be adjoining in the list. In the case of my unit tests, I used (x,y) => x % 2 != y % 2, which is a fancy way of saying that two odds or two evens can’t be next to each other in the list. I wanted to use a different type (int) than I’d be using in my real use case to make sure I didn’t make any assumptions about the nature of type T. This comparison was the first one for two numbers that came to mind.

Each time I needed functionality out of the ListResorter, I wrote a test. I watched it fail, then I made all the tests pass. If I saw opportunities to refactor, I took them whenever all my tests were green (passing). By the time I was done, I had about a dozen tests and this class:

using System;
using System.Collections.Generic;
using System.Linq;

namespace SantaKataApp
{
    public class ListResorter<T>
    {
        private readonly List<T> _list;
        private readonly Func<T, T, bool> _canAdjoin;

        public ListResorter(List<T> list, Func<T, T, bool> canAdjoin)
        {
            _list = list;
            _canAdjoin = canAdjoin;
        }

        internal void Swap(int index1, int index2)
        {
            ThrowIfIndexesAreInvalid(index1, index2);
            T temp = _list[index2];
            _list[index2] = _list[index1];
            _list[index1] = temp;
        }

        private void ThrowIfIndexesAreInvalid(int index1, int index2)
        {
            if (_list.Count < index1 - 1 || _list.Count < index2 - 1)
                throw new InvalidOperationException("An index is beyond the length of the array");
        }

        internal bool CanAdjoin(int index1, int index2)
        {
            ThrowIfIndexesAreInvalid(index1, index2);
            return _canAdjoin(_list[index1], _list[index2]);
        }

        public int GetNextIndex(int i)
        {
            if (i >= _list.Count)
                throw new InvalidOperationException("Invalid index");
            if (i == _list.Count - 1)
                return 0;
            return i + 1;
        }

        internal bool CanAllAdjoin()
        {
            return !_list.Where((t, i) => !CanAdjoin(i, GetNextIndex(i))).Any();
        }

        public List<T> Resort()
        {
            var list = _list;

            while (! CanAllAdjoin())
            {
                for (int i=0; i < list.Count; i++)
                {
                    int j = GetNextIndex(i);
                    int k = GetNextIndex(j);
                    if (! CanAdjoin(i, j) && CanAdjoin(i, k))
                        Swap(j, k);
                }
            }
            return list;
        }
    }
}

This class has two public methods, GetNextIndex() and Resort(). That Abelian ring idea still lives in the GetNextIndex() method, which says that the next member after the last in the list is the first, making the list circular. Resort() does what you would expect.

The other methods in the class are marked internal, so that my TDD tests can access them via the [Assembly:InternalsVisibleTo()] attribute in the Assembly.cs code file. After design is done, I would consider rewriting the test methods that talk to internal methods so that they are exercised through the public method. I don’t want my unit tests to be break should someone decide to change the internal implementation of these methods. You can see a bit of this thinking in the ThrowIfIndexesAreInvalid() method. I pulled this method out to avoid duplication of code during refactoring, where I already had unit tests in place and thus I didn’t need to write new ones.

Once I had ListResorter working, writing a console wrapper was easy:

using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;

namespace SantaKataApp
{
    class Program
    {
        static void Main(string[] args)
        {
            string[] fileContents = File.ReadAllLines(args[0]);
            Func<Participant, Participant, bool> surnamesDontMatch = (x, y) => x.LastName != y.LastName;
            List<Participant> names = fileContents.Select(Participant.Create).ToList();

            var listResorter = new ListResorter<Participant>(names, surnamesDontMatch);
            List<Participant> sorted = listResorter.Resort();

            DisplayParticipants(listResorter, sorted);
        }

        internal static void DisplayParticipants(ListResorter<Participant> listResorter, IList<Participant> sorted)
        {
            for(int i=0; i < sorted.Count; i++)
                Console.WriteLine(Display(sorted[i], sorted[listResorter.GetNextIndex(i)]));
        }

        internal static string Display(Participant name, Participant giveToName)
        {
            var giveTo = giveToName != null ? giveToName.ToString() : "NONE";
            return string.Format("{0} gives a gift to {1}", name, giveTo);
        }
    }
}

Most of the work done in the console app is formatting the output for display. The heavy lifting is done by the ListResorter class.

This program outputs data like:

PS C:\temp> .\SantaKataApp.exe .\names_santa.txt
Luke Skywalker <luke@theforce.net> gives a gift to Toula Portokalos <toula@manhunter.org>
Toula Portokalos <toula@manhunter.org> gives a gift to Leia Skywalker <leia@therebellion.org>
Leia Skywalker <leia@therebellion.org> gives a gift to Gus Portokalos <gus@weareallfruit.net>
...

I met my goal: the output is the same every time given the same input file.

I time-boxed the effort at two hours and I came close to reaching my limit. I initially began by writing an iterator, but abandoned it when I realized that I only wanted to traverse the collection once.

There is more that could be done to this program, of course. For example, I’m not happy about DisplayParticipants using GetNextIndex(). The only reason that DisplayParticipants() needs a ListResorter is to use its ring-like GetNextIndex() method. This functionality should have been extracted into its own class.

Comments? Questions?

"Secret Santa" Code Kata in Powershell

Wed, 25 Jan 2012 21:36:48 -0700

The team at work is doing a Code Kata, the Secret Santa kata from Ruby Quiz. In it, you are given a list of names and emails and need to pair givers with receivers, with the only constraint being that a giver and receiver can’t have the same surname.

I wrote my solution in PowerShell. I wanted to use native PowerShell functionality, so I didn’t import any .NET libraries. I know this post is brief, but I wanted to get my solution posted with a little commentary while it was still fresh in my mind.

When run like this:

PS> .\santa.ps1 .\names.txt

It gives output like:

Giver                                                Receiver
-----                                               ---------
John Smith <jsmith@example.com>         Jane Doe <jdoe@example.com>
...

Here’s the code:

 param($file)

 function ConvertTo-Array($hashtable) 
 {   
     $result = @()
     $hashtable.GetEnumerator() | % {
         $object = New-Object PSObject   
         Add-Member -inputObject $object -memberType NoteProperty -name Giver -value $_.Name 
         Add-Member -inputObject $object -memberType NoteProperty -name Receiver -value $_.Value 
         $result += $object
     }   
     $result
 }

 $none = "NONE"
 $names = @{}
 gc $file | % { $names.Add($_, $none) }
 $namesToMatch = $names.Keys.Count
 if ($namesToMatch -lt 2) {  throw "Need at least two names to match" }

 while (($names.Keys | ? { $names.$_ -eq $none }).Length -gt 0) 
 {
     $from = $names.Keys | ? { $names.$_ -eq $none } | Get-Random
     $to = $names.Keys | ? { $_ -ne $from -and $names.Values -notcontains $_ } | Get-Random
     #"DEBUG: $from, $to"
     if ($from -ne $null -and $to -ne $null -and $from.split()[1] -ne $to.split()[1]) 
     {
         $names.$from = $to
     }
     else
     {
         $undoMatch = $names.Keys | ? { $names.Values -ne $none `
             -and $from.split()[1] -ne $_.split()[1]} | Get-Random
         #"DEBUG: unset $undoMatch"
         if ($undoMatch -ne $null) 
         {
             $names.$undoMatch = $none
         }
     }
     $percentComplete = 100 * ($names.Values | ? { $names.$_ -ne $none }).Length / $namesToMatch
     Write-Progress -activity "Match in Progress" -status "% Complete:" -percentcomplete $percentComplete
 }
 $results = ConvertTo-Array $names
 $results | ft

(I apologize in advance for the formatting. I will look into another solution like Pastebin if I do this more often. Please comment if you have recommendations.)

It reads the contents of the file and writes each line to a hashtable, where all the receivers are $none (a special value I created which I used like $null).

As long as there are unmatched people, I go through an algorithm to select a random gift giver who doesn’t have a receiver ($from) and a receiver who doesn’t have a giver ($to). The split(” “)[1] yields a person’s surname.

I used a variant on the “Hill Climbing” algorithm mentioned on the site. If I run into a situation where there are no more matches, I take a random person who could have been a match and throw them back into the pool of candidates. In this way, the algorithm never gets stuck.

At the end, I call my ConvertTo-Array function to beautify the output. Without it, the hashtable columns are labeled “Name” and “Value”, and that didn’t satisfy me. (I couldn’t get Format-Table to help.)

I added a progress bar to see how often this part of the code is invoked, and it gets called once or twice per run on average. However, the script itself has decent performance. It takes a second or so to match 40 names.

Please let me know what you think or ask questions. I’m happy to go into more detail.

Getting off the Ground

Mon, 23 Jan 2012 07:07:00 -0700

“You don’t need more time, you just need to decide” - Seth Godin

At work, there are a lot of things that I’ve identified for improvement.

Where I work is largely a Java shop, but there are a couple of large applications that are .NET and a growing group of .NET developers. And there are a lot of amazing opportunities here.

The company has a good track record of agile delivery using Scrum. However, I see areas for improvement also. The job of scrum master has become institutionalized as just a meeting facilitator. The responsibility of the scrum master as team coach has been lost in translation from training done there a few years ago.

Our current scrum master wanted the team to attend introductory scrum training. As a certified scrum master, I was anxious to hear what the agile coach on staff would say, and I was gratified to agree with his curriculum. It’s a matter of practice in our group.

Fortunately, I am well-suited to help move the team toward a steady cadence of delivery. We need to get our product owner away from firefighting mode and to get what he and our customers need within the confines a fixed sprint backlog. Having a strong scrum master will help.

And we need to get beyond our large estimates for small projects due to fear of the unknown with this application — turnover has left most of the delivery team new to the code base.

There’s a desire for a .NET community of practice. I’ve started the ball rolling with a book club. The first book we’ll read together is Working Effectively with Legacy Code by Michael Feathers, which is all about getting pre-written systems under unit test. We’re also starting our first code kata this week, an old Ruby Quiz about Secret Santas.

Both the book club and katas are exercises to start talking about software design in a simpler context. These .NET applications grew organically and were written by developers who were more comfortable with SQL than .NET. The application has shortcomings that you might expect: lots of logic in stored procedures, poor separation of concerns between SQL and .NET, and a presentation layer (ASP.NET web forms) that’s highly coupled to the database.

And the application has a lot of room to grow. It is ripe for becoming a data warehouse solution. I have some mixed experience setting up data warehousing from a previous position, but this time, the team is blessed with a talented database developer onboard to guide us through the tough parts. I can’t wait!

The concern I have is being spread too thin. As you can see, there are a number of vectors to pursue. I need to choose one, maybe two of these and drive them to completion before taking on others. I haven’t talked about the nine months of architecture changes that are needed to get the application able to scale to handle the desired demand or some of the expanded features they want for this application this year.

If I have learned anything over the past couple of years, it is that I cannot be all things to all people. I need to pace myself and be ready for some false starts. I also find a way to handing off these projects when (if?) they get off the ground. Here’s where having a good manager will help me, and I am blessed to have one.

I feel energized, and excited about the possibilities!

Parsing large files using PowerShell

Tue, 17 Jan 2012 15:54:00 -0700

At work, I had the need to parse through a large pipe-delimited file to count the number of records whose 5th column meets and doesn’t meet my criteria.

PS C:\temp> gc .\items.txt -readcount 1000 | `
  ? { $_ -notlike "HEAD" } | `
  % { foreach ($s in $_) { $s.split("|")[4] } } | `
  group -property {$_ -ge 256} -noelement | `
  ft -autosize

This command does what I want, returning output like this:

  Count Name
  ----- ----
1129339 True
2013703 False

Here’s some explanation in English, for those of you who don’t know PowerShell.

The first command is gc (Get-Content), which reads the file in 1000 (readcount) lines at a time.

The second command is ? (Where-Object), which filters out the HEAD row.

The next command % is an alias for Foreach-Object, where object in this case is a 1000-line chunk. The inner loop is another foreach loop, which is slightly different from Foreach-Object in ways that are unimportant to the matter at hand. Point is, you can’t nest % blocks. The block of the foreach loop splits each line by pipe delimiter and returns just the 5th column (first column is numbered 0).

The next command in the chain is group, an alias for Group-Object, in this case we’re grouping by a calculated property, whether the output of the previous command is greater than or equal to 256. By saying “-noelement”, I’m saying I don’t need an enumerated list of the values, which in this case are unimportant.

Finally, we get to ft (Format-Table). It is necessary because the Count column may be over 99999, in which case the value is truncated. The option “-autosize” causes PowerShell to make it fit instead.

However, for a 500 MB test file, this command takes about 5.5 minutes to run as measured by Measure-Command. A typical file is over 2 GB, where waiting 20+ minutes is undesirably long.

I posted a query to StackOverflow for some ideas.

While I waited, I discovered that 2500 was the optimum value for -ReadCount, getting the command execution time down to about 3.5 minutes.

Within minutes, I got a helpful hint from Gisli to look into using the .NET StreamReader. Here’s what that script looks like:

=== begin Show-SourceCounts.ps1 ===
param($file = $(Read-Host -prompt "File"))
$fullName = (Get-Item "$file").FullName
$sr = New-Object System.IO.StreamReader("$fullName")
$trueCount = 0; 
$falseCount = 0; 
while (($line = $sr.ReadLine()) -ne $null) {
      if ($line -like 'HEAD|') { continue }
      if ($line.split("|")[4] -ge 256) { 
            $trueCount++
      }
      else {
            $falseCount++
      }
}
$sr.Dispose() 
write "True count:   $trueCount"
write "False count: $falseCount"
=== end file ===

This script yields the same results as the first command, but in about a minute. Quite an improvement!

PowerShell to the rescue!

Wed, 14 Dec 2011 20:55:00 -0700

I encountered a situation today at work that lent itself to PowerShell. One of the developers asked for some information about the PowerShell line I wrote to diagnose this problem, and I thought it might make a good blog post.

There was a pipe-delimited file that was failing to import into a database via our program, and the QA person was having trouble figuring out why. The process yielded a stack trace, which pointed to a certain stored procedure and a SQL error:

System.Data.SqlClient.SqlException: Error during [stored procedure]: [stored procedure]; 4; String or binary data would be truncated. at System.Data.SqlClient.SqlConnection.OnError (SqlException exception, Boolean breakConnection)

The stack trace clearly suggests that an input is exceeding a field length. Question was which one. As it turned out, there was only one column being inserted by this procedure. I did a quick schema lookup and found out that the column in question allows 50 characters.

So, I wrote:

gc failing_input_file.txt | % {$_.split("|")[4] } `
  | ? { $_.Length -gt 50 } | select –unique | measure –line

The query yielded 255 offending rows, and led to an alteration of the INSERT stored procedure to truncate the value from the flat file.

The rest of this post is copied wholesale from that email.

Let me restate it a command at a time, and expand the aliases.

Most common PowerShell commands have an alias, which you can look at via Get-Alias. In general, if you want to know about a command, you type “Get-Help command-name”. I put a backtick after each line to continue the line, as is the PowerShell custom.

Many cmdlets (read: “command-lets”) do pipeline processing, which if you haven’t seen it means that the previous command’s output is handed to the next piece as input. The special variable $_ is the current whatever — in this case, the current line. With most commands like with select and measure, $_ is inferred.

PowerShell offers full access to .NET. The strings it returns are .NET strings, which is why Split() just works.

Get-Content failing_input_file.txt `      
| ForEach-Object {$_.split("|")[4] } `    
| Where-Object { $_.Length -gt 50 } `     
| Select-Object –unique `                 
| Measure-Object –line

The first line with Get-Content reads in the contents of the file. For each line of the file, the second part with the Foreach-Object splits that line by “|” and grabs the 5th item. The Select-Object command omits duplicates. The Measure-Object command counts the number of lines.

If you want the contract names themselves, leave off the Measure-Object call:

gc failing_input_file.txt `
| % {$_.split("|")[4] } ` 
| ? { $_.Length -gt 50 } `                                                       
| select –unique

Ubiquitous Capture

Wed, 26 Oct 2011 18:02:24 -0600

The idea behind ubiquitous capture is that you can dump things on your mind into a storage medium (notebook, what have you) so that you don’t have to tie up attention re-remembering things to keep them fresh in your mind. I have David Allen and Merlin Mann to thank for convincing me this is a good idea.

Today, after listening to Brett Terpstra on Mac Power Users, I’m trying a new (to me) ubiquitous capture workflow. It’s slightly different than what’s discussed on the show.

The lingua franca is Markdown. The more I use it, the more I like it.

Software

nvALT 2.1, Brett Terpstra’s fork of Notational Velocity
Byword 1.4, a minimalist and sexy Markdown-friendly text editor ($9.99)
Dropbox
Nocs for iPhone, a Markdown-friendly editor with Dropbox support

Setup

Change the note storage location for nvALT to a Dropbox folder
Add .md as a text extension in nvALT (see this Skitch)
Tell nvALT to store and read notes on disk as Plain Text Files

Let me know what you think.

Quotes From Some Folks

Mon, 24 Oct 2011 22:30:27 -0600

Quotes From Some Folks: Quotes From Some Folks Timeless Information

Hey all. It is me, Armen, also known as the coolest guy on and off the internet, back with another post~. I do it for the people. This one is about quotes some fellow bloggers(I might as well use that term for a change, but I’m not a blogger, I’m a site-writer) use on a regular basis in their thoughts, or that they use as a motto, or that they are quite fond of.

As you read these quotes, you might want to think about a quote that you have running in your thoughts at regular times. It could be a short one like “Just do it” that you use to get things started or completed, or “what goes around comes around” to deal with someone wronging you. Whatever it may be, it can be good to know what quotes are in your thoughts on a regular basis.

My first quote is from Mike King, writer at Learn This, and his quote of usage is:

You miss 100% of all the shots you never take. - Wayne Gretzky

Mike knows what’s up. Are you a guy? Did you not talk to that girl at the coffee shop? Guess what. That was a shot and you missed it. Are you a girl? Did you not talk to that guy at the coffee shop? Guess what. Well, society expects him to talk to you so I guess this doesn’t relate to the quote so well. Are you a student? Did you leave something out of your essay because you thought it might not be well-received, even though you had an inkling it might make it one of the top essays? That shot was missed folks. Are you a parent? Did you not sign your daughter up for the musical instrument class because you weren’t sure they could handle it? That shot was also missed.

As we go through our day, we see shot after shot. Even people that do barely anything see opportunities they can take advantage of, but instead, they miss out on them. Who wants to miss a shot? Noone, that’s who. 22% is higher than 0% most days of the week. 31% is also higher than 0%. I can do more of these math problems but my calculator will run out of ink if I keep using it here.

A doctor might be wavering between two methods of treatment for a patient, with one having 40% chance of success and the other having a 15% chance of success. However, if he keeps deciding for hours until the patient is no longer functional, he now has a 0% chance of successful treatment, and that’s a fact you can take to the bank or your local credit union if you aren’t a fan of the recent big bank monthly fee additions.

Onto the next blogger and their quote—

Used by Bud Hennekes, writer at A Boundless World we have:

This too shall pass.

This quote is a useful one for many different situations that can come up. It is more for the hard times than for the good times(I think it is almost only for the hard times). Let’s say you are in a period of unemployment, or lack a partner, or have an illness, or have just experienced grief. Then, this quote fits your situation. You must remember that terrible things have come and gone for all people through all of history. Thinking that your ‘thing’ is the end-all is about as realistic as thinking that teen singers are going to bring forth a powerful voice.

This quote is not one you should use during a good time. Actually, it is. At that time, it will remind you that your good time will pass, and that you need to make sure you take care of the foundational parts so that you can return to a regular mood in a stable fashion. Sometimes, when you are in a good mood, you do things that would not make sense if you knew the good mood would just last another 10 minutes.

Let’s say you start a new job. The first two weeks might be tough because you know so little about the environment and other people and requirements. That is the harder period. That period shall pass, and you will then be in a position and have competency in it, so here is another example where you should use this quote.

To the third quote–

This one is used by Farnoosh Brock of Prolific Living:

The lust for comfort kills the passions of the soul. – Khalil Gibran

Let’s first discuss the lust for comfort. Who is looking for comfort? Most people are. You’re probably looking for comfort. We seek that feel-good feeling of safety, relaxation, and well-being. While this is great, it is actually terrible. When you strive for that boring comfort that we all know is the comfort most people are thinking of, you give up risk-taking, and risk-taking is necessary for the “passions of the soul” that is present in the 2nd half of this quote. To see that lounge chair in the sun with no worries type of comfort, one sets aside the roughness and toughness required to have enough of a thick skin to follow one of those passions of the soul. Find me some person who followed one of their passions who had an easy time with it. Whether this is Eminem, Tony Hawk, Sheryl Swoops, or J.K. Rowling, following your passion basically requires you to walk around with a big piece of armor across your chest, to protect from stupid people trying to cut away at your self-confidence, so that you quit making them look weak in comparison.

I hope you liked that last line.

So, therefore, we have to cut away those weak thoughts of comfort-seeking when we have them. It’s that simple. When the thought of some idealistic future comfort comes in your mind, tell yourself “this is stupid”, and move on.

Speaking of the passions of the soul, it is sort of interesting how so many people have them. We sometimes think only 5% of people have big passions because they are the only ones who make it, but there are a huge chunk of people, possibly like yourself, who have big passions and don’t touch them out of fear of whatever. They have big passions too, but they just don’t act on them. Near the end of people’s lives, they could care less how much money they have in the bank, or other stupid things, but people do regret not following their passion.

Our next quote comes from Peter Clemens, creator of The Change Blog:

In matters of style, swim with the current; in matters of principle, stand like a rock. - Thomas Jefferson

Let me tell you. Rocks are great. They are like the symbol of stability and strength. We will get back to that.

First, let’s discuss the first half of this quote by the honorable Thomas Jefferson. Regarding style, swim with the current, and be as others are. This means to wear red if that is the thing that is happening, or to add some extra tone to your voice if that is what people are doing. Style is something you bring for others, because the clothes you wear are for others to see, and the style you put in your craft is for others to enjoy and get fulfillment from. This is why style is to be maintained in according with what people are currently doing and wanting to see.

Now, as far as the second part of this quote, principle is something you don’t mess with. Various folks have said “if you don’t stand for something, you’ll fall for anything.” This is quite true. Your principles have to remain whether times are good or bad. That is why they are principles. If you toss them away at the first sign of difficulty, what was the point of maintaining the charade of having them up until then? If your principles involve being a strong family person, but you forget your family when some emergency occurs, how is that the case? If you are against drug usage, and allow a peer to influence you into using one, what are you really against? This list goes on and on. If you are a person who speaks about environmental protection, but who starts bending the rules when it costs a little bit, why put up an act? You’re not fooling anyone if you don’t live up to a principle you’ve set forth.

It can get pretty heated up in here.

The next quote is used by Vincent Tan, who writes over at Health Money Success(one of the most direct site names you will see):

Reaching your destination is all about taking one step at a time.

This quote reminds me of my one audio post about taking the first step for whatever you are working on. Smart people break things into steps. Giving a speech to 100 people might be a 12 step process involving steps for writing the speech, steps for acquiring the audience, steps for securing a venue, and steps for follow-up. Big things get small real quickly when you break them apart. A tree can have 100 branches and seem huge, but if they are cut off one at a time, the task is not so difficult. Steps are the way we go from the first floor of our complex item to the second floor of the process.

Now, if you take one step at a time, that means you aren’t try to ‘cheat the system’ or skip ahead. Trying to speed by or find some secret faster path is a good way to lose time. You do things one by one, one step at a time, and you pass the people who are either try to cheat their way through, or who aren’t doing anything.

Making a great salad is as easy as some 10 steps that are done one by one, and the end result is healthy food you can eat for days.

Our last quote here is one used by Celestine Chua from The Personal Excellence Blog:

If you want to get something you have never gotten before, you have to do something you’ve never done before.

Now, Celes has done a lot of things she’d never done before, and now she has gotten a lot of things she had not gotten before, so this quote certainly fits her.

We have to understand this quote and then absorb it. Let’s say what you are doing is called X, and it gives you Y. Doing more X is going to give you more Y. That is how the world works. If you want M, you are going to need to do something different, like Z. X is not going to get you M. This makes life seem more appropriate and fair. Instead of thinking doing more X will get you Y, and also M, and also whatever else, you will know that doing more X will just get you more Y, whatever that Y is.

It is when you do or create something different that you get something different, and that you learn something new about yourself.

If you are wanting to get a new client, you are going to need to reach out to potential clients in a new way. Sitting there like you may have been will get you a new client as much as buying a lemon will help you make orange juice.

I hope you all enjoyed these various quotes and my commentary upon them.

Maybe I will hear from some of you folks/readers in the comments. Which quote did you like best? What is your own quote of usage? Your feedback is cool and also gets you to learn more for yourself.

The difference between management and leadership

Mon, 24 Oct 2011 22:30:25 -0600

The difference between management and leadership: The difference between management and leadership Seth’s Blog

Managers work to get their employees to do what they did yesterday, but a little faster and a little cheaper.

Leaders, on the other hand, know where they’d like to go, but understand that they can’t get there without their tribe, without giving those they lead the tools to make something happen.

Managers want authority. Leaders take responsibility.

We need both. But we have to be careful not to confuse them. And it helps to remember that leaders are scarce and thus more valuable.

The new frontier

Tue, 18 Oct 2011 16:31:14 -0600

The new frontier: The new frontier Seth’s Blog

What, exactly, is wrong with the old frontier?

When Google + launched, millions of formerly optimistic people became optimistic again. Maybe this was going to be the one, the social network with just the smart people and none of the lame stuff, none of the spam or the pitches or the people we’re trying to avoid.

And the same thing is true when the pack runs to the new nightclub, the new technology, the new suburban subdivision. Maybe this will be the one…

Of course, it rarely is. So much disappointment and so much bitterness. It’s never as great as you hoped it would be. Ennui and then, eventually, waiting for yet another new frontier.

It’s the old frontier that actually presents the most interesting opportunities, because the shine has worn off. This is your platform for real innovation, innovation in a place or a market or a situation that truly is ready for it.

When Not to Use Scrum

Tue, 18 Oct 2011 16:31:13 -0600

When Not to Use Scrum: When Not to Use Scrum The Scrum Crazy Blog

Don’t get me wrong, people! I am CRAZY about Scrum. Just ask my wife(she gets sick of hearing about it sometimes)! I’m probably the biggest Scrum fan on the planet!

However, I’ve encountered certain situations where it doesn’t seem like Scrum works well. They all pretty much boil down to two generic cases:

Applying Scrum to a problem domain it was not designed for, or
Applying Scrum where people or processes actively or passively work against Scrum principles.

IMO, Scrum may not be the best fit for:

Companies who will actively or passively work against Scrum and its major principles.
- This includes companies that do Faux Scrum.
- If the Company doesn’t care what process is used at the team level, and won’t constantly act against Scrum, then Scrum is fine at the team level.
Companies who are expecting a lot of benefits from Scrum but cannot commit to doing Scrum in a good faith, holistic, way.
- How does one do Scrum in a holistic way? Look to the Scrum Guide first
Companies that like to matrix numerous people into numerous projects.
- Matrixing at the Product Owner or ScrumMaster level *may* be ok(but probably not optimum) if the people are experienced/highly knowledgeable in those roles.
Teams who cannot commit to a week of fairly fixed scope (say, where < ~70-80% of the scope is fixed for a week).
- Examples of teams that sometimes cannot hold to this are generally interrupt driven type organizations, like network support, production support, software operation support, etc. A Kanban type of approach can be considered with such highly variable scope, but I’m not a fan of the Kanban movement yet because it is still incubating IMO.
- Be careful on this one. If your team experiences high scope churn, it may just be a really bad dysfunction that needs rectifying. Look closely before discarding Scrum here.
Teams where members and/or leaders will actively or passively work against Scrum and its major principles.
- This includes companies that do Faux Scrum.
- Stealth approaches are fine, but if there isn’t enough consensus to work within the Scrum framework(especially from team leaders), then results will probably be mediocre.
- I have seen some team leads(or functional managers) who embrace the principle of self organization (always to a point, especially if the team lead has more command and control responsibility from above), and I have seen team leads who just can’t let go of the authority. In cases where a team leader cannot let go of the authority, Scrum may be a disaster, especially for those advocating for it. If you’re in one of these situations and you really love Scrum, find a new team or organization to work for that truly embraces Scrum.
Something other than software development.
- It’s my personal view that true Scrum is for software development. One can adapt Scrum to other domains, but then it’s not really Scrum any more. One can certainly use “adapted pseudo Scrum” effectively, but that’s only if the person guiding the adaptations is a Scrum and/or process expert. I strongly believe that many Scrum concepts can be “lifted out” and used very successfully elsewhere — but again, that’s not really Scrum any more IMO.

Again, please don’t take this the wrong way and use these scenarios as an excuse to “give up on Scrum.” Scrum is about the “art of the possible,” and the only time it doesn’t work is when it is not ‘possible.’ I believe the situations above to be when Scrum *may* not be possible (or at least very difficult). In pretty much any other situations than the ones described above, I will defend Scrum heavily as possible and potentially highly beneficial.

Below are some examples of interesting situations where I didn’t think Scrum was the best fit.

Example #1

I had a company call me once for advice on Scrum. In short, they were doing the typical consulting firm thing where everyone works on ~3+ projects at once, which meant ~3+ different teams for each team member. I told her they either need to re-org into teams that work as teams, or forget about Scrum and find something else. I love Scrum, but matrix hell is Scrumbut City and it won’t work.

Example #2

(in this one I didn’t recommend against Scrum per se)

I interviewed to be a Scrum Coach(as a consultant) with a company. They had a couple of major Scrumbuts that they were unwilling to change, yet they expected to obtain significant productivity gains from using Scrum and wanted me to make that happen. An hour or so after the interview, I called the person who set up the interview and I told them I didn’t believe they would reap the benefits they sought because they weren’t really willing to embrace Scrum. As such, since I didn’t think it would be a successful engagement, I politely declined.

Oct 7, 2011 - Ten Years Later

Tue, 11 Oct 2011 23:15:52 -0600

Oct 7, 2011 - Ten Years Later:

Open conversations (or close them)

Tue, 11 Oct 2011 22:45:44 -0600

Open conversations (or close them): Open conversations (or close them) Seth’s Blog

A guy walks into a shop that sells ties. He’s opened the conversation by walking in.

Salesman says, “can I help you?”

The conversation is now closed. The prospect can politely say, “no thanks, just looking.”

Consider the alternative: “That’s a [insert adjective here] tie you’re wearing, sir. Where did you buy it?”

Conversation is now open. Attention has been paid, a rapport can be built. They can talk about ties. And good taste.

Or consider a patron at a fancy restaurant. He was served an old piece of fish, something hardly worth the place’s reputation. On the way out, he says to the chef,

“It must be hard to get great fish on Mondays. I’m afraid the filet I was served had turned.”

If the chef says, “I’m sorry you didn’t enjoy your meal…” then the conversation is over. The patron has been rebuffed, the feedback considered merely whining and a matter of personal perspective.

What if the chef said instead, “what kind of fish was it?” What if the chef invited the patron back into the kitchen to take a look at the process and was asked for feedback?

Open conversations generate loyalty, sales and most of all, learning… for both sides.

Event Sourcing as a strategic advantage

Tue, 11 Oct 2011 22:31:10 -0600

Event Sourcing as a strategic advantage: Event Sourcing as a strategic advantage Jimmy Bogard’s Blog

Very often you hear Domain-Driven Design recommended as an approach that should not be applied except in a few key strategic scenarios. The reasons for this are quite simple: DDD is expensive. Not in the time it takes to code the patterns (typing is cheap), but in the time it takes to build an understanding of the domain, form a model, and refine that model over time. The code is cheap, the conversations leading up to the code are expensive.

That said, DDD is an effective approach when the system you’re building is part of the strategic advantage of the business. If the system is important to the company because its revenue depends on how well the system aligns with the company’s short- and long-term objectives, then it will probably (hopefully) receive enough investment to realize that strategic advantage.

If you’re already going the DDD route, you’ll eventually come to a choice on what mechanism you’d like to use to model your domain. Currently, you have two major choices:

Persisting current state
Persisting as a stream of events

The first approach is the “traditional” DDD approach, most thoroughly explained in the “Applying Domain-Driven Design and Patterns” book. This is where you’ll find technical discussions on entities, aggregates, repositories and so on.

If you’ve already taken the plunge to “do DDD”, then Event Sourcing as a means of enhancing the strategic advantage that DDD brings is something that should be seriously considered.

Strategic advantages of event sourcing

Event sourcing is simply a mechanism of realizing state by capturing a stream of events. The final picture of state is available by replaying sequence of events that led to that final state.

In television crime drama shows, the investigators walk into a crime scene, which represents the final state of a system. Then there’s some sort of flashback where the crime is replayed as the series of events that led to that final result. Without the television magic of replaying the events, it’s difficult to deduce why the current state is where it is. But if you’re trying to understand why the current state is what it is, the final state is not nearly as useful as the series of events that led to that endpoint.

In critical business systems, state is often not nearly as important as the behavior that led to that current state. This behavior could be users purchasing items, orders being shipped, etc. These events are what reflect the health of the business, as they represent activities that actually result in revenue for the business.

As a business matures, you often see that in order to promote growth of the business, their strategic advantage lies in how well they understand the why of where their revenue comes from. It’s not enough that we sold 10000 widgets. Why did we sell that many? How can we sell more? Events provide insight into how the system came to be in its current state. It’s replaying the crime scene to determine the guilty. Without replaying, we can only make an educational guess.

Because businesses care so much about why the current state is what it is, explicitly modeling state as a stream of events aligns with the business’s goals.

Event sourcing in the enterprise

As the business grows, more and more areas want to react more quickly to changes happening in the system. Instead of being fed a daily feed of data, they want to be fed events. Don’t give me a feed of users every day, give me a list of new users, users who have moved, users who have cancelled their account. Don’t force me to reverse-engineer the crime scene, just tell me what happened.

In order for ancillary services in the enterprise to most effectively react to changes, they need to be aware of the business-level events that have happened. Just having access to data doesn’t describe what has happened. With events, individual services can determine how they want to react to business-scoped events, like “UserRegistered” or “AccountCancelled”. A single feed that tries to capture all this information in a single stream then puts the burden on the consumer to try to figure out what actually happened.

With event sourcing, our events are explicit in the system, and much more easily exposed to the enterprise. Furthermore, for historical purposes, it’s also much easier to perform data mining of business events when we capture events explicitly.

Event sourcing as an investment

Modeling state as a stream of events means you’re committing to capturing behavior explicitly, instead of capturing state. It’s an investment, not because it’s hard to capture events, but because of the conversations required to surface what the events and behavior should be. State is easy to figure out. Behavior is not.

However, if the business isn’t making an investment in a system that you believe needs event sourcing/DDD, it’s likely because:

Ignorance of the strategic investment these approaches bring
Lack of capital to actually make the investment (cart before the horse scenario)
The system isn’t as important as you think it is

Be careful that you’re not falling into the last one. Sometimes the system you’re working on just isn’t a part of the business’s strategic goals, and it’s just a tactical response. Otherwise, it’s just a matter of having a conversation with the business to see if they’re more interested in the current picture, or the sequence of events and behaviors that led to the current state. Current state is easy with event sourcing, it just matters if the business is interested in how they got there.

Steve Jobs (1955-2011)

Wed, 05 Oct 2011 23:30:45 -0600

Steve Jobs (1955-2011): Steve Jobs (1955-2011) Presentation Zen

Steve Jobs passed away today. He was just 56. Steve often talked about changing the world, and he did change the world in a huge way. His incredible dedication to detail and to simplicity and aesthetics raised the bar for technology, business, and design, and beyond. He even raised the bar for presentations. He was a true master. He was a true sensei. All his presentations were great, but my favorite one of Steve’s is not his usual Apple presentation, but rather a short 15-minute speech delivered from behind a lectern at Stanford University in the spring of 2005. There is nothing I can say that has not been said before about this legendary man. I have no words. I hope you’ll have a chance to listen to Steve’s Stanford speech once again. (日本語 version.)

Steve’s words are inspiring. Steve Jobs was—and will long remain—an inspiration for so many. Good bye, Sensei. We miss you.

“Your time is limited, so don’t waste it living someone else’s life. Don’t be trapped by dogma — which is living with the results of other people’s thinking. Don’t let the noise of others’ opinions drown out your own inner voice. And most important, have the courage to follow your heart and intuition. They somehow already know what you truly want to become. Everything else is secondary.”

— Steve Jobs

Creativity

Wed, 05 Oct 2011 23:16:15 -0600

Creativity: Creativity Agile & Business

A friend was reading Daniel Pink’s book called Drive. And so I started reading it.

I had this series of thoughts.

First, I have for a long time thought that the key thing about our jobs is creativity. Takeuchi and Nonaka call it knowledge creation, but to me it remains essentially the same thing. And it covers many domains: business, technical, people, customer, marketing, etc, etc. We must be, as a team at least, creative in all these different domains. Innovative, inventive, combining things in a new and interesting way, just coming up with something that seems weird at first, seeing something new that is elegant.

I find that there are, in overly simple terms, two types of people I run into.
People who are ok with creativity and chaos, and people who are not.

The people in that second group do not create much in the exercises we do in the Scrum classes. They tend to want to ‘edit’ or restrict work too soon. They tend to both give and take restrictions on the ‘scope’ far too soon. They tend to ask me to restrict or define the exercise, as though they were afraid of chaos.

And Daniel Pink’s book makes me think that they (those in the second group) have not realized how their own internal thoughts and feelings inhibit their creativity. They want to be too orderly, too soon. They want to edit themselves before they have gotten a useful body of ideas out on the table to play with. They fear that play is not real work. They worry much too soon about ‘are we on target’. They fear the subconscious, and want everything to be arrived at logically.

In other words, they have told themselves a bunch of lies about their own motivation (to focus on the subject of Daniel Pink’s book). They are not inherently less creative, at least according to this (my) hypothesis. They just have ‘rewarded’ themselves so wrongly that less creativity comes out.

Now, it may be that some teams are just less creative. Period.

But before deciding that, let’s give them dance lessons. Let’s have them play. Let’s ask them to break their own mental boundaries in some meaningful way. And see what they can really do once the blinders come off.

If you run into a team that basically is in the second category, I would work with them to identify their ideas and concerns about creativity. My hope is that by talking it through, they can start to allow themselves more chaos and creativity. And also more fun!

In closing, let me repeat here what Jeff Sutherland has often said: If they are not having more fun, then they are not playing Scrum the right way.

I think he means this in two ways;
- only by having fun will their best come out
- fun, serious fun, is a value for human beings in and of itself. It does not need to be justified in any other way.

Note: I do not think Jeff would condone ‘trivial fun’. My example of that might be: eating a whole bowl of candy on Halloween. Fun in a certain use of the word, but not good for you. I mean, and I think he means, fun that gives deeper satisfactions than that.

Note: The picture or mindmap above comes from this page:
http://bit.ly/nDZpld

Subscribe in a reader

Crop Guidelines for Portrait Photography

Wed, 05 Oct 2011 23:16:14 -0600

Crop Guidelines for Portrait Photography: Crop Guidelines for Portrait Photography PetaPixel

Here’s a helpful illustration that shows acceptable places to crop when shooting portraits. Cropping at green lines should be fine, while cropping at red lines might leave you with an awkward looking photograph.

Image credit: Don’t Chop at the red by J. Southard Photography

The forever recession (and the coming revolution)

Wed, 05 Oct 2011 23:16:12 -0600

The forever recession (and the coming revolution): The forever recession (and the coming revolution) Seth’s Blog

There are actually two recessions:

The first is the cyclical one, the one that inevitably comes and then inevitably goes. There’s plenty of evidence that intervention can shorten it, and also indications that overdoing a response to it is a waste or even harmful.

The other recession, though, the one with the loss of “good factory jobs” and systemic unemployment—I fear that this recession is here forever.

Why do we believe that jobs where we are paid really good money to do work that can be systemized, written in a manual and/or exported are going to come back ever? The internet has squeezed inefficiencies out of many systems, and the ability to move work around, coordinate activity and digitize data all combine to eliminate a wide swath of the jobs the industrial age created.

There’s a race to the bottom, one where communities fight to suspend labor and environmental rules in order to become the world’s cheapest supplier. The problem with the race to the bottom is that you might win…

Factories were at the center of the industrial age. Buildings where workers came together to efficiently craft cars, pottery, insurance policies and organ transplants—these are job-centric activities, places where local inefficiencies are trumped by the gains from mass production and interchangeable parts. If local labor costs the industrialist more, he has to pay it, because what choice does he have?

No longer. If it can be systemized, it will be. If the pressured middleman can find a cheaper source, she will. If the unaffiliated consumer can save a nickel by clicking over here or over there, then that’s what’s going to happen.

It was the inefficiency caused by geography that permitted local workers to earn a better wage, and it was the inefficiency of imperfect communication that allowed companies to charge higher prices.

The industrial age, the one that started with the industrial revolution, is fading away. It is no longer the growth engine of the economy and it seems absurd to imagine that great pay for replaceable work is on the horizon.

This represents a significant discontinuity, a life-changing disappointment for hard-working people who are hoping for stability but are unlikely to get it. It’s a recession, the recession of a hundred years of the growth of the industrial complex.

I’m not a pessimist, though, because the new revolution, the revolution of connection, creates all sorts of new productivity and new opportunities. Not for repetitive factory work, though, not for the sort of thing ADP measures. Most of the wealth created by this revolution doesn’t look like a job, not a full time one anyway.

When everyone has a laptop and connection to the world, then everyone owns a factory. Instead of coming together physically, we have the ability to come together virtually, to earn attention, to connect labor and resources, to deliver value.

Stressful? Of course it is. No one is trained in how to do this, in how to initiate, to visualize, to solve interesting problems and then deliver. Some see the new work as a hodgepodge of little projects, a pale imitation of a ‘real’ job. Others realize that this is a platform for a kind of art, a far more level playing field in which owning a factory isn’t a birthright for a tiny minority but something that hundreds of millions of people have the chance to do.

Gears are going to be shifted regardless. In one direction is lowered expectations and plenty of burger flipping… in the other is a race to the top, in which individuals who are awaiting instructions begin to give them instead.

The future feels a lot more like marketing—it’s impromptu, it’s based on innovation and inspiration, and it involves connections between and among people—and a lot less like factory work, in which you do what you did yesterday, but faster and cheaper.

This means we may need to change our expectations, change our training and change how we engage with the future. Still, it’s better than fighting for a status quo that is no longer. The good news is clear: every forever recession is followed by a lifetime of growth from the next thing…

Job creation is a false idol. The future is about gigs and assets and art and an ever-shifting series of partnerships and projects. It will change the fabric of our society along the way. No one is demanding that we like the change, but the sooner we see it and set out to become an irreplaceable linchpin, the faster the pain will fade, as we get down to the work that needs to be (and now can be) done.

This revolution is at least as big as the last one, and the last one changed everything.

What to do next

Wed, 05 Oct 2011 19:01:21 -0600

What to do next: What to do next Seth’s Blog

This is the most important decision in your career (or even your day).

It didn’t used to be. What next used to be a question answered by your boss or your clients.

With so many opportunities and so many constraints, successfully picking what to do next is your moment of highest leverage. It deserves more time and attention than most people give it.

If you’re not willing to face the abyss of choice, you will almost certainly not spend enough time dancing with opportunity.