Philly ETE, Day 2: Session 5

Chef: Saving Time (and Money) with Automated Provisioning

-- Trotter Cashion

Trotter starts by talking about his own user experience with automating deployment. His company looked at Chef, decided it would be too complicated, and decided to create the automation themselves using Bash. It was a big win at first, but over time it became way too difficult to maintain, adding new machines was time consuming, and there were still too many manual tasks involved.

So eventually they moved to Chef.

Chef is...

written by Opscode

written in Ruby

in some sort of controversy with Puppet

driven by a Ruby DSL

Chef has a concept of 'cookbooks.' It's a set of instructions to install any piece of software that are published by Opscode. It can also keep machine and code in lock-step. So you can deploy your software for any environment ("QA", "performance," "development") and configuration may well vary among them. Chef helps you keep all of the necessary elements by organizing all of the environments with a particular set of cookbooks.

One main difference between Capistrano and Chef is that Chef eschews the concept of SSH. Chef assumes that it is already on the box on which it is installing software. It also assumes that it has root access.

According to Trotter, Chef provisioning takes between 15 and 30 minutes and deploys take under 2 minutes.

Using Chef with Spatula:

git clone http://github.com/opscode/chef-repo

gem install spatula (this is Trotter's tool)

spatula prepare db-server.yourcompany.com

spatula install my_database #this will look up the appropriate cookbook and install

...Rest of instructions are on the slides and I don't want to copy them word for word. They should soon be up on the Philly ETE Site.

The Chef Directory is really simple. It contains 'config,' 'cookbooks', 'roles', and 'custom-cookbooks' subdirectories. The cookbook directory contains recipes, files and templates (static or dynamic files to copy to places on your machine), attributes, and some others.

Trotter says the best possible place to start learning Chef is on their website, though not necessarily at the home page. Start with http://wiki.opscode.com/display/chef/Resources. This has the most relevant definitions and lots of example code to get you started.

Philly ETE, Day 2: Session 4

Barbara, Demeter, and Don: notes on some CS precepts from a non-scientist programmer

-- David A. Black

This talk will focus on three ideas: Liskov Substitution Principle (Barbara), the Knuth "premature optimization" principle (Don), and the Law of Demeter.

Liskov: "Let q(x) be a property provable about objects x of type T. Then q(y) should be true for objects y of type S where S is a subtype of T."

How about some psuedocode?

type Bicycle {

attribute "wheels" = 2

}

type Tricycle inherits from Bicycle {

attribute "wheels" = 3

}

That's a simple example of a violation of the LSP because the Tricycle changes the core attribute of its parent. The Ruby way of looking at LSP is to eschew a heavy reliance on inheritance hierarchy.

type != class in Ruby. So what is an object's type? "For any Ruby object obj, the type of obj is: the type that objects of the type that obj is of are of." Glad we cleared that up. David borrows the term "stereotyping" and repurposes it to mean trying to determine what an object's ancestry in order to determine if it is suited to a certain task. This is the wrong approach. Duck typing is more effective, clean, and in keeping with 'the Ruby Way.'

In Ruby, David points out, we only care about objects and what they can do 'in the moment.' We want to send them a message and have them respond properly at a particular point in time. So inspecting the object's ancestry is unnecessary. Duck typing will tell you whether or not the object is suited to task.

So is Ruby compliant with LSP? We talk about the issues of type and object substitution frequently as Ruby programmers and we do it in a way that may or may not be orthogonal to LSP. I will be exploring this further as I continue through my Uncle Bob Payroll Case Study exercise.

Knuth:

"We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil." -- The full 'premature optimization' quote from Knuth and/or Sir Tony Hoare.

David states it straight out: this quote gets misused frequently. His first statement in an attempt to ground us is "not all optimization is premature." He also uses an example of the 'match' method versus the '=~' method to illustrate. Why should we feel hesitant or anxious over choosing one or another based on performance. Doesn't this fall into the 97% of small efficiencies? Probably. Other problems exist with the syntax of the statement itself. 'Optimal' is akin to a superlative. Maybe we should try to incrementally 'ameliorate' your code instead.

Demeter:

"The goal of the Law of Demeter is to organize and reduce dependencies between classes. Informally, one class depends on another class when it calls a function defined in the other class."

The example used is a Ruby implementation of an archiving program. Then, interestingly enough, he shows us something that usually is (wrongly) seen as a violation of the Law of Demeter:

a = Adder.new

a.add(3).multiply(6).add(1).subtract(5)

puts a.sum

Since this object chains methods that all call itself, it is not by definition a violation of the Law of Demeter. A real violation involves one object reaching into another object in a way that unnecessarily couples the two objects. David then showed an example of a clear Demeter violation that did not do any "dot" method chaining.

http://haacked.com/archive/2009/07/14/law-of-demeter-dot-counting.aspx is a post recommended by David to further explain how Demeter violations are not just identifiable by counting dots.

Philly ETE, Day 2: Session 3

Demystifying HTML5: Understanding the Emerging Web

-- Molly Holzschlag

The history: WHAT-WG is a company that broke away from the W3C because they did not believe in the future of XHTML. Now, of course, XHTML is dying off and is no longer supported. However, what comes out of that group is not considered an "open standard." The only official open web standards are issued by the W3C.

Today, W3C says they are committing future resources to HTML5 and are no longer supporting XHTML.

Google, Microsoft (!!!), Mozilla, Opera, and WebKit all, for the first time ever, believe in HTML5 and want to see it succeed. This may be the first time in the web's history that this ubiquitous acceptance happened.

Lots of companies are already jumping on board -- Netflix, MySpace, YouTube.

HTML5 is an attempt to advance the web's core language and to embrace a rich, interactive, forward-thinking web. It is also being designed to support full backward-compatibility. HTML5 will replace XHTML 1.0, DOM2HTML and, of course, all previous versions of HTML.

One of the core design principles for HTML5 is explicit specification for error handling and more graceful degradation (for example, if you deploy to an older browser that does not support HTML5, the application should fail gracefully).

The W3C's objective is to evolve HTML rather than recreate it. They are trying to avoid reinvention. They are also trying to build on real-world use and test cases.

HTML5 Syntax:

There is no documentation type definition (DTD). So there is no special declaration for version 5.

HTML syntax is still served using text/html. However, you can use HTML5 with XML syntax and XHTML 1.0.

Of course, when we implement in XML notation we must close our tags the proper way (of course, we should always be rigorous and write well-formed HTML as well as XML).

The W3C will soon support SVG universally (they're waiting on -- shocker -- IE to build support into IE9).

There are all new elements that Molly runs through with us. You should read her slides to get the full list of new elements and definitions. She said she will soon post them on the Philly ETE site.

'Input' is a particularly interesting new element. It allows us to to declare and execute embedded scripts in a more terse, expressive manner without all the clunkiness of the way we declare them now.

'Require' is another great new element. It allows us validate form input without any using any scripts! That's pretty darn cool.

Embedded media is also coming. Such as canvas (the HTML5 drawing API), video, and audio (embedded audio and video without the reliance on plug-ins). These are works in progress. Even further down the road (but still realistic future features) are localStorage (client-side data storage that is persistent across sessions and uses client-side SQL storage) and applicationStorage (enabling offline applications).

One of the best places, believe it or not, to find information about web standards and HTML5 is Wikipedia. So feel free to check it out there or, of course from the horse's mouth at www.w3.org

Philly ETE, Day 2: Session 2

Hardwired for deception means trouble with estimates!

-- Linda Rising

My first experience of Linda Rising was through her fabulous book, "Fearless Change." That is the most inspiring book I've ever read about organizational change and is packed with strategies and patterns for doing so. Ariel and I continue to use the language developed in the book to talk about our own work in affecting change on our client.

She's here today to give us "some bad news" and tell us "some bad things about" us. Much of the information she's going to present will be too difficult for us to absorb and the natural reaction will be to "explain it away."

I'm strangely intrigued.

Linda's definition of deception: "consciously or unconsciously leading another or yourself to believe something that is not true." We'll start here. Her claim is that we naturally deceive ourselves or others - constantly. So it's a natural extension of this to conclude that we are constantly screwing up our estimates.

We are hardwired to deceive. We are hardwired to be optimistic, to see what we want to see. Then we rationalize away any incongruent results between our optimistic choice and reality. We are all biased, prejudiced, etc... There is no way around this. According to Linda, we cannot do anything about it.

Along with the confidence in your own intelligence comes the illusion of rationality. But you are not rational and you are certainly not more or less rational than anyone more or less intelligent than yourself. In fact, the smarter you are, the more you deceive yourself and the more clever your rationalizations become.

One example goes back to a scientific discovery made in 1847 that doctors who washed their hands between performing autopsies and delivering babies saved more lives of women an infants. But this discovery did not affect many doctors' decisions to wash their hands. They continued their non-hand-washing practices even in the face of scientific evidence.

"A new scientific truth does not triumph by convincing its opponents and making them see the light, but rather because its opponents eventually die, and a new generation grows up that is familiar with it." -- Max Planck. Wow, that is grim.

Back to the world of everyday work. How about hiring? In an experiment, a number of interviewers were given the same information for a job candidate. Some interviewers were given some 'positive' information about the candidate first, and the others were given somewhat 'negative' information first. Those interviewers who received positive information first about a candidate gave significantly higher predictions of success. In fact, this tendency was exaggerated when the element of time pressure is added.

More stats on deception: On average, during a typical conversation there are 3 lies every 10 minutes. A survey of college professors revealed that 93% believed they were above average. And so on.

OK, this presentation is really about the real world. The practice of deception is passed onto our children. We teach them to deceive in a socially acceptable manner (i.e., pretend to like every present you get from family members on Christmas). We eat more from larger containers, when served larger portions, or when at an all-you-can-eat buffet. Yet we deceive ourselves by underestimating how much we eat. Sedentary people (such as programmers) show an increase in the hormone ghrelin, which increases appetite. So internally, even your own body deceives you into thinking you need to eat more than you actually do.

Getting back to our software story estimates, we can now draw conclusions about our (in)ability to estimate. Linda used to believe that mathematical models and tons of data from past projects would point the way to better estimates. But she saw that she was wrong. In her experience she was never able to make better estimates based on any of her tool, models, or truckload of data.

So should we even bother? Should we build in some mechanism for tuning our estimates to a less optimistic number? Linda has one suggestion: stop using numbers. Estimates represented by numbers are inherently deceptive. They give the illusion of being something that can be added, subtracted and multiplied. But estimates do not have these properties by nature. So getting away from numbers will alleviate some of the tendency to deceive oneself and others into confusing an estimate with a calculation. "Use t-shirt sizes or gummy bear colors," she says. Anything but numbers.

Linda's goal is to get us thinking about this and for me she has succeeded. And there has to be more than just estimating that is getting colored by our constant deception of ourselves and others - interviewing practices (as noted above), team dynamics, and decisions made during pairing are a few that spring to mind.

Crap! I just realized Cyrus is giving out estimating decks at our booth. They have numbers all over them! I must stop them! I'm off!

Philly ETE, Day 2: Session 1

I Think I Finally Understand Mocks

-- Brian Marick

When it comes to testing, there are a few people whose advice I seek out more often than others. Brian Marick is one of them (Martin Fowler and Steve Freeman are two others I can think of off the top of my head). So I'm excited to see what Brian has to say about Mocks.

"Object-oriented design is teaching objects what to say to each other." Nicely put.

Brian has a long and acrimonious history with mocks. He claims he finally got it... about six months ago.

"Mocks are about faster design and smoother pacing" relative to normal, everyday TDD.

Brian picks up his first example at the point that an AJAX request hits the server in a Sinatra application. The request will hit a JSON controller, which will go through several routines and then spit out some well-formed JSON over the wire. He uses Shoulda to test the JSON controller. He mocks out each object that the controller must interact with. Brian uses FlexMock (written by Jim Weirich) to mock out his objects. I wonder how this differs from Mocha. Should investigate this a little further.

After showing us the tests, he points out he can write code to pass the test without worrying about the creation or test-driving the objects that the controller utilizes. The idea here is that contrary to many people's usual one-way drive through TDD, mocking out the controller's objects first gives us the choice of what to create next.

As an aside, he's using something called Prezi Desktop, which I've never seen in action before. It seems like a cool, digitalized way of whiteboarding design.

For example, Brian next picks out the internalizer object. He test-drives that object, and then iteratively makes the controller actually "work" in the real world. The big win here is that as the internal functionality of the objects changes, your controller tests won't have to. The mock test simply ensures that the contract between the controller and the objects it interacts with stays broken. Checking out his slides (which will eventually be posted here.) will do better justice to this part of the talk than I can in prose. But I'll keep trying.

One interesting discovery Brian made was that when he finally felt comfortable with the way he was using mocks, he went back and replaced some of his tests with mock tests and was amazed at how much setup code and other cruft just "went away." I could definitely get down with that.

According to Brian: "Mocks are not about the final structure of the application. They are about the process by which you arrive at the final structure of the application." That seemed worth writing verbatim.

Faster Design: As the functionality of one of your controller's dependent objects grows, the size of the file grows too big. Probably the right thing to do here is to fork off another object, and continue to keep doing this (no shocker here: objects become too big or do too many things and must then be split into other objects). What Brian claims is that mocking "at the top" makes the construction of loosely coupled objects easier to design test-first.

Better Pacing: Again, this comes down to choice. Mock all of the objects that a controller must interact with on the controller test, and then defer the decision of which objects to construct, decompose, or change in any other way to the last responsible moment. Very cool idea. And very well illustrated in one particular slide. I need to try some of this out soon.

Mocks encourage you to start at the very top. Start at the view, or the controller, and drill down to the objects, the persistence layer, etc...

Brian points out the dirty little secret of mocks: in all likelihood it will force you to rewrite tests more than you are used to. But, he says, the rewriting is easier and even starts to feel like a natural part of the TDD flow. A bold statement, and one I'm willing to put to the test in my own work.

Philly ETE: Session 5

Clojure's Approach to State and Identity

-- Rich Hickey

Rich starts by mentioning that this is not a Clojure-specific talk until the very end. What we'll have is, in essence, a philosophical talk about programming practices followed by a practical application of these in Clojure

"Pure" functions: depend only on their arguments, given the same arguments, always returns the same value, has no effect on the rest of the world, has no notion of time. Rich wants to reduce the definition of a function to what is essentially a mathematical one.

To the extent that you write functions as he just described, you will write better programs - easier to test, safer, cleaner. But in large respect, most programs are not functions and do not consist of a majority of functions. For the purposes of this talk, Rich defines anything that is not a pure function as a process.

A process may include the notion of time. They may have effects on the outside world, might produce different answers at different times even given the same arguments (for example, a search engine).

"Variables in traditional languages are predicated on a single thread of control, one timeline." Of course, as soon as you add concurrency you introduce problems. Your variables are no longer atomic, they may require locks, they may require workarounds to deal with the problem of time.

It behooves each language to have a model for Time. Something must be built into the language to check if one event happens before, after, or at the same time as some other event. In fact, relativity is what we are really after with respect with time.

I'm wavering here. Last session and I'm getting a little bombarded by the philosophical discussion. I needed to hear this earlier in the day. Good thing I'm going to the Clojure Pragmatic Studio and learn more!

Example: Race-walker foul detector. If left and right foot are off the ground at the same time, that is a foul. This is another demonstration of how time must be taken into account. We need a snapshot of the world at a particular time in order to make this decision. We also can't solve this problem with locking (in this real world example, this would mean stopping the runner to inspect his foot position). If we had a value that included time, I could solve this problem.

Core to Programming Approach

- Programming with values is critical

- manage the succession of values (states) by eschewing changing values in place

Using Persistent Data Structures

- Immutable composite values are the key. Change is thereby merely another function that takes one value and returns another. The value is immutable so we don't have to worry about change outside the world of the function.

- The collection maintains its performance. It does not degrade as the program is developed further.

Rich has a way of instilling confidence in you, the developer, when implementing functional programming languages, particularly his own. His message: "Don't worry about threads, don't worry about state, don't worry about concurrency gotchas. I'm worrying about all of that for you so that you don't have to." That makes me feel nice, and like I need to spend more time exploring Clojure than I have already.

Philly ETE: Session 4

Opinionated is Relative: choice and modularity in Rails applications

-- David A. Black

Rails choices seems like a new or paradoxical concept. We've heard forever that Rails is opinionated software and "forces" you into design choices.

But David sees this paradigm shifting. He attributes the origins of this shift to Josh Susser and his blog post, "The Tyranny of Choice."

In fact, David sees signs of these choices even in the earliest versions of Rails. He is even writing a book about those choices called, appropriately, "Rails Choices" (Pragmatic Bookshelf, forthcoming). And this is where he will focus this particular talk.

Rails 3 is more modular than any of its predecessors. Modularity is slightly different than choice in that modularity has a bigger effect on architecture. So we'll first go through some everyday choices and then we will dip into modularity, specifically in respect to "choosing" a DB layer other than ActiveRecord.

To complete the transparency of his book, this talk, and the motives behind it, David goes on to describe his goals:

-- avoid "this sux/this rulez" - just deliver factual information so that the attendees can make up their own minds.

-- Factor in organizational realities (if he didn't before, he now has his fair share after spending three months on a client site as a Cyrus developer). Who gets to see what? Who gets to change what? Are some decisions already made (like in any legacy application)? In this case we need to evaluate the costs vs. the benefits of change. Choice, then, is loaded in a way that it is not in a brand new project.

-- Provide a good number of techniques. This is not an in-depth how-to. But the book should touch on things the reader isn't intimately familiar with. For example, and in-depth discussion of ORM validations vs. DB constraints. David also mentioned Liquid architecture as an alternative.

-- Stick close to the framework. This is not a rundown on every plugin or gem. It's not a how-to on adding these programs to your framework. However, some plugins get 'privileged' treatment such as RSpec, HAML, and Liquid.

In Rails 3, many things that used to just come wrapped up with the package are now installed via plugins. David also wants to open the subject of "supplemental code" - add-ons, overrides, project-specific libraries, etc... I like this coverage. For the longest time on my first Rails project I just stuck everything that wasn't an M, V, or C into the lib directory. Things got messy in a hurry, as you might expect. I'd like to learn more about this. Application organization best practices.

An aside: David HATES scaffolding. As he sees it, scaffolding obscures the real work that must be done to set up your first piece of functionality. One should be able to get through the preliminaries without being tied into scaffolding.

David has set up a site called http://anecdotes.rubypal.com/. This is supposed to augment or supplement his new book. From the site itself:

What's the "anecdote" thing?

I'd like to include some stories from people who've developed Rails applications and have real-life experience making Rails choices. Each anecdote will be about 1/3 to 1/2 a page long.

Why did you choose RSpec over TestUnit? Are you using HAML or Liquid instead of RHTML? Does your team use migrations, or does your DBA do it all?

And why? That's what I'm interested in: why you made a particular choice, in the context of your project and your team.

If your anecdote is chosen for inclusion in the book, you'll get a free copy of it when it's published.

Act 2 of David's Talk: Choice and Modularity

The demo will be a replacement of ActiveRecord with GDBM in Rails 3. Live coding here. (In GDBM, for each record, you have a file.) David shows us a Rails application console where a model acts exactly the same way it would were it interacting with ActiveRecord. Only it's not AR; it's GDBM under the hood. In showing how the replacement was implemented, David points out the freedom he has to write his own class methods from which his models extend. He then shows us the abstract_oxm.rb file that is responsible for "gluing" the object mapping layer to ActiveModel. This is the part where Rails 3 becomes very modular. You really have the choice to utilize one configuration file to map the rest of Rails to whatever database mapping layer you want. There are a few methods that "have" to be implemented in order for ActiveModel to play nice. The rest is up to you. Very cool!

David puts sums up succinctly. To paraphrase: there is a very small contract to which you must adhere in order to implement your own choice of persistence layer. And this is not limited to database layers. Templating and the ability to more quickly and easily work with Rack are other examples of where Rails 3 provides modularity.

Philly ETE: Slog

Internet.... bandwidth limit... being reached... web access.... sloooooooww.

Resorting to wireless broadband.

Philly ETE: Session 3

Agility and Architecture

-- Robert C. Martin

Yes, I'm back with Bob. There were some competitors for my attention for this session, but I've decided to play it safe. The fact that I've never heard Uncle Bob talk until this morning's keynote also influenced my decision. I'll maximize my exposure while I can.

The one thing I can count on with Uncle Bob is that I will be entertained. There's something to be said for that.

One question posed right off the bat: Does agile mean the abandonment of architecture? Does it mean no design?

Architecture - we don't know exactly what it is. It is defined differently by many different people. "Design" is a similarly ambiguous term.

"The whole 30 day thing kind of failed. Too much can go wrong in 30 days. My favorite at the moment is 2 weeks and I'm hoping to get to one week very, very soon."

Velocity tracking and estimating are seriously flawed practices. Things "jitter around," according to Uncle Bob. You will not achieve precision with these practices. They are also measures of how fast or efficient teams are getting things done. But they in no way track iterative architectural development.

This type of slice-by-slice development, we only implement "just enough" of the 'grand vision' done (where 'grand vision' means, in general, the entire architectural space of a project). In keeping our heads down in the smaller iteration, we can overlook the overall architecture and how it's shaping up. We may not be evaluating correctly.

The better solution is to think of the small iterations as drafts of a paper. When we're done with the first iteration or 'draft,' we start our second iteration and see what has to be reworked, thrown out, improved. When we complete our second iteration and have a new, improved "draft."

The message so far seems to be "throw away" what's working or fitting with the 'grand vision.' It generally only takes half the time to recreate something that we've already engineered.

It's worth pausing here to note that Uncle Bob just tried to slide in a reference to "An American Tale." That almost blew my mind. I'm still reeling. Ok, onward...

The best thing to do with architectural decisions is to defer them as long as possible. Don't choose your database until the last responsible moment. (Side note: I violated this rule almost immediately in trying to implement Uncle Bob's examples in "Agile Software Development." You can see that in one of my earlier posts. One of the first real-world lessons I learned from the book.)

Uncle Bob's example was his experience with Fitnesse. They knew "all along" that they needed a database. But they deferred and deferred until they realized that they actually never needed a database. They wrote from RAM to flat files over and over, iteration after iteration, until they realized that the database wasn't needed at all. The original Fitnesse, and the version which still exists today, is not db-dependent. To paraphrase Uncle Bob, they pushed off the database requirement, and pushed it off, and pushed it off until it fell off a cliff. What a metaphor!

Design - quick design sessions are useful. Whiteboarding is essential. But the best place to keep all of those resulting design decisions, dependencies, etc... is in the developers' heads. This is the goal, difficult though it may be to achieve.

Lots and LOTS of talk about why to use TDD. I thought that Uncle Bob was past telling people why to use the principles of good design. TDD, CI, pairing - we already know these are good, we are done convincing people, you are being ridiculous if you don't implement these practices. That's what I gleaned from Uncle Bob's keynote. I liked that. I'm tired of explaining why they are effective and I'm tired of having someone explain it to me. If you've been developing for over a year, have heard of all these terms, and you haven't been sold, I'm sorry. I don't want to fight with you anymore. You're probably just not that good and I certainly would regard you with skepticism if I had to work with you. Let's just put it out there. Ok, digression finished.

What I do enjoy from Uncle Bob's speaking is that he seems to still grasp real-world concepts and experiences "from the front lines." People that have been speaking for years on end are susceptible to losing this ability to relate. Uncle Bob has not.

And here we get another push to get QA requirements put into automated acceptance tests and implemented before the feature is developed. We're trying, Bob, we're trying!

Uncle Bob described a client with a QA manager with 80,000 manual tests. He came from his boss who one day told him he had to cut his budget by 50%. Basically, they had to throw away tests to achieve this. "How did they get into this mess?" Uncle Bob asks. They depended on manual testing. They did not instrument their system to allow for automated acceptance tests.

The deeper issue is that this client was using their resources (their workers) in the wrong fashion. People are not very good at following rote instructions. They make odd decisions. They are imperfect. They may take shortcuts. We want people leveraging computers to make automated tests that run the same way every time.

And one more thing, which you can happily throw in my face anytime: "If a system cannot be tested, the design sucks." Noted. I shall do my best.

There is still a place for architects in an agile environment, according to Uncle Bob. However, they must also right code with the team for some portion of the time. They must have the experience the same pain and success that everyday developers experience.

Philly ETE: Seen and Heard, 4/8/10 11:43 AM

-- I've now seen two people creating PowerPoint/Keynote as a way of taking notes for sessions. I'd like to talk to one of them to see what they see the advantages are.

-- 2 Apple iPad sightings. One funny moment was when someone working on an iPad a row over from me was trying to type (keyboard looks like an enormous iPhone keyboard). Damon leaned over to me and whispered "I think he's trying to type something..." The iPad owner was clearly struggling.

Philly ETE: Session 2

Airplanes to Application Development

-- John Kern

ObjectMentor guy talks big mechanical architecture and boils it down to incremental application development. At least that's my take up front. I'm also interested since I now consult at Boeing. Why not check out their most successful product in history (actually I have no idea if that's true, but it is pretty damn successful).

John is an aerospace engineer by training. This is "his angle" and affects the approach he takes to software development.

Rough timeline: early involvement and initial requirements were set in early 1990. The first commercial flight occurred in 1995! That's amazing to me. It's taken us half a year to successfully release a version of a product to the customers of my current client. We're not even talking about a green field project here. The Boeing 777 went from someone's head to a commercial flight in 6 years!

The development team consisted of 240 Design teams with up to 40 members each! Holy crap! They quickly learned lessons about communication to help them cope with such a massive overall team.

Risk mitigation was one of the most carefully adhered-to priciples during the building of the Boeing 777. They had a bunch of huge tests for awhile, and then were able to abandon them after they came across, learned and popularized the concept of CAD modeling. The result was that the Virtual 777 could be assembled, simulated, interference checked. Pretty amazing. Of course, this is actually easier to do with software. Sure, time has to be invested to create good, functional development and staging environments. But these are essential in my mind, not optional ways of adding value to the project.

(As an aside, I'm trying to eat my own dog food by creating a development environment for a project involving an MS Office client developed using C# and communicating with a Rails server on a remote host. It's not trivial. I'm not 100% there yet, but I believe it's important and I'll get there.)

What we end up with is a set of rhetorical questions gleaned from the success of the Boeing 777 project. It looks suspiciously like a checklist for agile software development ("Do you involve the client? test the process? deliver frequently?"). But I don't want to soft peddle this. John relates these questions to his own experience and has interesting takes on their real world application. Also, I learned how few of these things I'm actually doing at my client. Do I take a page out of Uncle Bob's book and just do it because it's the right thing to do? It gets complicated pretty quickly. Yes, I want to release more frequently. But I'm on just one team of many. Can I get just my team to release? What does that even mean in the context of my project? As it turns out, John's simple set of questions have started my mind grapes churning.

John mentioned something I had never heard of, called Semat. This looks suspiciously like YASM (Yet Another Software Manifesto -- I just made that up. You can use it if you want and call it your own. I don't mind). But I'll give it it's due and take a look before I throw it under a bus. (Note: there's also a Semat blog that might give a more informal but informative view of what Semat is all about). It's actually tough to tell if John supports Semat. I have noted that he is not listed as a signatory. OK, enough about Semat.

Thanks to John Kern for all of his insight.

Philly ETE: Session 1

Creating Multithreaded Code Using TDD

-- Venkat Subramaniam

I've chosen this session over the others, mainly because I think Venkat is a seriously smart programmer with a great deal of interesting knowledge to share. I'm curious to see how TDDing multithreaded code diverges from typical TDD practices.

Other questions I have before we start --

If I program in Ruby, should I ever do this, even if the TDD practices are awesome?

Will we get some TDD examples in an FP language like, for example, Clojure? That would be awesome. But I think I have a good enough grasp of Scala to follow along there too.

Interestingly enough, Venkat started out trying to disprove the validity of TDD in multithreaded programming. But in his attempt, he found that he actually WAS able to. Not only that, but the TDD methods he employed saved hours of debugging multithreaded code to find a couple of bugs.

Lots of talk about the importance of TDD. Probably a nice talk for inexperienced TDDers, but for me and the two Cyrus developers to my left, Venkat is preaching to the choir.

There won't be any slides during this talk. I have the highest respect for speakers who run their talks this way. It's risky and sometimes ugly, but it's the closest thing to real-world programming that we can get at a conference. And application to the real-world is what we're all trying to get out of these two days. But I digress.

We're up to the "Let's give it a shot" section. I'm excited. Running this example using Java on IntelliJ IDEA.

First we start with a "canary" test. It is a stupid test that does something like "assertTrure(true); I do something similar with my new Ruby test files, but I don't keep them. There might be a reason to keep them around, particularly when the application is multithreaded. We're starting our example with a couple of classes, MultiValueMap and MultiValueMapTest.

I'm doing my best to keep up with the code. However, I'm using vi (don't have IDEA installed here) so it's going to be rough around the edges. I might post what I have at some later point after I clean it up. No promises though.

Best argument for TDD "it prevents Whack-a-Mole software development."

Now to the multithreaded stuff. If we make a threading testcase, and to make the test pass we make our public void class a public synchronized void class, we run into a wall. We can't test this easily. We need to avoid sticking 'synchronized' in this case.

The next tool to reach for is mocking. But Venkat suggests a more stripped down version of mocking. The result: MyLockMock which extends, not a mocking library, but ReentrantLock.

After we get some threading tests passing, we look into exception handling. He forgoes putting 'try' and 'catch' statements until he can write a properly failing test. I lost the ability to keep up coding along with Venkat at this time, but I got to witness some interesting incremental TDD.

The message is repeated in a number of ways: make it as simple is possible, move slowly, test one discreet piece of functionality at a time. Simple advice that is much better given through coding that through a bunch of slides or beating us over the heads with it.

Venkat never created a single thread to test his multithreaded design. This was the 'revelation' that led him to believe that TDD was possible, even simple, with multithreaded code.

For C#, Venkat recommends anonymous delegates. Good to know in some of my own toy projects.

A personal aside:

Venkat is awesome! He's funny, engaging, takes the right amount of questions, rolls with the changes and questions of the audience and reflects it in his live coding. What an accomplished speaker. I'd watch this guy give a talk on milk pasteurization (or something else I care equally little about).

Philly ETE: A blow-by-blow Account

For the next two days I will try to keep a running log of my time here at the Philly Emerging Tech Conference. The blog posts will likely be rough around the edges because I won't spend much time editing. My goal here is to capture the most important parts of the talks I attend, information I pick up, and things I experience throughout the conference.

Enjoy and please feel free to comment, ask questions, or slam my opinions at your leisure.

Philly ETE : Keynote 1

Bad Code, Craftsmanship, Engineering, and Certification

-- Robert C. Martin

My first experience seeing Uncle Bob martin speak, and I'm very excited. I've read his recent blog post about certification so I'm prepared for something in this vein.

Bad Code: A Crap Odyssey - a video of a single file called two.java scrolling for minutes on end. 30,000 lines of code.

Bob's oft-stated axiom of programming: "The only way to go fast is to go well."

Boy Scout Rule - when you check a module in, it must be cleaner than the one you checked out. Check out the code, make your modifications and clenaup, and then perform one "random act of kindness" on them.

Length of functions - After a history of where the "20 lines max" suggestion came from, he suggested somewhere around 4 - 6 lines. He prefers lots and lots of little functions. Even in Ruby I probably don't adhere to a maximum of 6 lines per function. So a challenge is brought (at least to me) not 30 minutes into this conference.

Functions of extreme length, complexity, obscure variable names, etc... might work for one person. But as soon as the software team grows past the size of 1, any complexity is irresponsible to your fellow programmers.

Uncle Bob knows his function is small enough when it is no longer possible to extract a method. Using an IDE is the easiest way to "extract till you drop."

Number of Arguments - again, the answer to how many arguments a function should take is "as few as possible." But other strong statements were made -- "five is insane," "never pass booleans into your functions" (by the way this is because you are blatantly violating the rule of having each function do only one thing) -- and finally settles on no more than three arguments.

Variable names - nice, long names that describe what they do are preferable in private functions. If the variable has a very large scope (meaning it is accessed from many parts of the application), the name should be more terse. Use a sliding scale of scope to determine how long the variable name should be, but of course do not sacrifice understandability.

Classes - find ways to extract classes when you see large functions - especially is you see many indentations and conditionals. Uncle Bob's practice is to promote local variables to fields when extracting methods is not possible because the scope "returns multiple values." This obviates opportunities to create classes.

Craftsmanship - "it's not the next big thing." It's been around forever. It's simply committing yourself to doing a good job. It's understanding the rules and applying them, over and over, every day. A surprise to me is that there is actually a craftsmanship manifesto. Bob suggests signing it. I'll give it a close read and maybe sign it myself.

Discipline - Hardware has been changed and improved literally dozens of orders of magazine. Software development, however, has not changed nearly so much. We still rely on "assignment statements, if statements, and while loops." Functional programming is a movement to remove the assignment statement. But FP has a bigger view. The movement is towards multicore safety, multithreaded programming that will work through our processors. The functional language that Uncle Bob supports is.... drumroll... Clojure. Hey, me too!

TDD - "the controversy is long gone." Bob likens it to surgeons washing their hands before surgery. It used to be a controversial issue but "the jury's been in on this one for a long time."

Pairing, CI, etc... are also now axioms of best practices in software development. Bob's point here seems to be telling us how obvious it should be for developers to adopt these practices. Noted. I hope it has an effect on the 75% of the audience that does not even practice TDD.

"QA should find Nothing." -- this is the goal. Make them worry about their jobs.

Maybe the most inspiring part of the talk is his last point - "convincing management." Bob's advice -- "don't even ask them." It is the developer's job to do the best work she can. She can't do that without TDD, CI, Pairing. So do them. Don't ask permission from anybody. This is your career and your responsibility to design software the best way you know how.