Philly ETE: Slog

Internet.... bandwidth limit... being reached... web access.... sloooooooww.

Resorting to wireless broadband.

Philly ETE: Session 3

Agility and Architecture

-- Robert C. Martin

Yes, I'm back with Bob. There were some competitors for my attention for this session, but I've decided to play it safe. The fact that I've never heard Uncle Bob talk until this morning's keynote also influenced my decision. I'll maximize my exposure while I can.

The one thing I can count on with Uncle Bob is that I will be entertained. There's something to be said for that.

One question posed right off the bat: Does agile mean the abandonment of architecture? Does it mean no design?

Architecture - we don't know exactly what it is. It is defined differently by many different people. "Design" is a similarly ambiguous term.

"The whole 30 day thing kind of failed. Too much can go wrong in 30 days. My favorite at the moment is 2 weeks and I'm hoping to get to one week very, very soon."

Velocity tracking and estimating are seriously flawed practices. Things "jitter around," according to Uncle Bob. You will not achieve precision with these practices. They are also measures of how fast or efficient teams are getting things done. But they in no way track iterative architectural development.

This type of slice-by-slice development, we only implement "just enough" of the 'grand vision' done (where 'grand vision' means, in general, the entire architectural space of a project). In keeping our heads down in the smaller iteration, we can overlook the overall architecture and how it's shaping up. We may not be evaluating correctly.

The better solution is to think of the small iterations as drafts of a paper. When we're done with the first iteration or 'draft,' we start our second iteration and see what has to be reworked, thrown out, improved. When we complete our second iteration and have a new, improved "draft."

The message so far seems to be "throw away" what's working or fitting with the 'grand vision.' It generally only takes half the time to recreate something that we've already engineered.

It's worth pausing here to note that Uncle Bob just tried to slide in a reference to "An American Tale." That almost blew my mind. I'm still reeling. Ok, onward...

The best thing to do with architectural decisions is to defer them as long as possible. Don't choose your database until the last responsible moment. (Side note: I violated this rule almost immediately in trying to implement Uncle Bob's examples in "Agile Software Development." You can see that in one of my earlier posts. One of the first real-world lessons I learned from the book.)

Uncle Bob's example was his experience with Fitnesse. They knew "all along" that they needed a database. But they deferred and deferred until they realized that they actually never needed a database. They wrote from RAM to flat files over and over, iteration after iteration, until they realized that the database wasn't needed at all. The original Fitnesse, and the version which still exists today, is not db-dependent. To paraphrase Uncle Bob, they pushed off the database requirement, and pushed it off, and pushed it off until it fell off a cliff. What a metaphor!

Design - quick design sessions are useful. Whiteboarding is essential. But the best place to keep all of those resulting design decisions, dependencies, etc... is in the developers' heads. This is the goal, difficult though it may be to achieve.

Lots and LOTS of talk about why to use TDD. I thought that Uncle Bob was past telling people why to use the principles of good design. TDD, CI, pairing - we already know these are good, we are done convincing people, you are being ridiculous if you don't implement these practices. That's what I gleaned from Uncle Bob's keynote. I liked that. I'm tired of explaining why they are effective and I'm tired of having someone explain it to me. If you've been developing for over a year, have heard of all these terms, and you haven't been sold, I'm sorry. I don't want to fight with you anymore. You're probably just not that good and I certainly would regard you with skepticism if I had to work with you. Let's just put it out there. Ok, digression finished.

What I do enjoy from Uncle Bob's speaking is that he seems to still grasp real-world concepts and experiences "from the front lines." People that have been speaking for years on end are susceptible to losing this ability to relate. Uncle Bob has not.

And here we get another push to get QA requirements put into automated acceptance tests and implemented before the feature is developed. We're trying, Bob, we're trying!

Uncle Bob described a client with a QA manager with 80,000 manual tests. He came from his boss who one day told him he had to cut his budget by 50%. Basically, they had to throw away tests to achieve this. "How did they get into this mess?" Uncle Bob asks. They depended on manual testing. They did not instrument their system to allow for automated acceptance tests.

The deeper issue is that this client was using their resources (their workers) in the wrong fashion. People are not very good at following rote instructions. They make odd decisions. They are imperfect. They may take shortcuts. We want people leveraging computers to make automated tests that run the same way every time.

And one more thing, which you can happily throw in my face anytime: "If a system cannot be tested, the design sucks." Noted. I shall do my best.

There is still a place for architects in an agile environment, according to Uncle Bob. However, they must also right code with the team for some portion of the time. They must have the experience the same pain and success that everyday developers experience.

Philly ETE: Seen and Heard, 4/8/10 11:43 AM

-- I've now seen two people creating PowerPoint/Keynote as a way of taking notes for sessions. I'd like to talk to one of them to see what they see the advantages are.

-- 2 Apple iPad sightings. One funny moment was when someone working on an iPad a row over from me was trying to type (keyboard looks like an enormous iPhone keyboard). Damon leaned over to me and whispered "I think he's trying to type something..." The iPad owner was clearly struggling.

Philly ETE: Session 2

Airplanes to Application Development

-- John Kern

ObjectMentor guy talks big mechanical architecture and boils it down to incremental application development. At least that's my take up front. I'm also interested since I now consult at Boeing. Why not check out their most successful product in history (actually I have no idea if that's true, but it is pretty damn successful).

John is an aerospace engineer by training. This is "his angle" and affects the approach he takes to software development.

Rough timeline: early involvement and initial requirements were set in early 1990. The first commercial flight occurred in 1995! That's amazing to me. It's taken us half a year to successfully release a version of a product to the customers of my current client. We're not even talking about a green field project here. The Boeing 777 went from someone's head to a commercial flight in 6 years!

The development team consisted of 240 Design teams with up to 40 members each! Holy crap! They quickly learned lessons about communication to help them cope with such a massive overall team.

Risk mitigation was one of the most carefully adhered-to priciples during the building of the Boeing 777. They had a bunch of huge tests for awhile, and then were able to abandon them after they came across, learned and popularized the concept of CAD modeling. The result was that the Virtual 777 could be assembled, simulated, interference checked. Pretty amazing. Of course, this is actually easier to do with software. Sure, time has to be invested to create good, functional development and staging environments. But these are essential in my mind, not optional ways of adding value to the project.

(As an aside, I'm trying to eat my own dog food by creating a development environment for a project involving an MS Office client developed using C# and communicating with a Rails server on a remote host. It's not trivial. I'm not 100% there yet, but I believe it's important and I'll get there.)

What we end up with is a set of rhetorical questions gleaned from the success of the Boeing 777 project. It looks suspiciously like a checklist for agile software development ("Do you involve the client? test the process? deliver frequently?"). But I don't want to soft peddle this. John relates these questions to his own experience and has interesting takes on their real world application. Also, I learned how few of these things I'm actually doing at my client. Do I take a page out of Uncle Bob's book and just do it because it's the right thing to do? It gets complicated pretty quickly. Yes, I want to release more frequently. But I'm on just one team of many. Can I get just my team to release? What does that even mean in the context of my project? As it turns out, John's simple set of questions have started my mind grapes churning.

John mentioned something I had never heard of, called Semat. This looks suspiciously like YASM (Yet Another Software Manifesto -- I just made that up. You can use it if you want and call it your own. I don't mind). But I'll give it it's due and take a look before I throw it under a bus. (Note: there's also a Semat blog that might give a more informal but informative view of what Semat is all about). It's actually tough to tell if John supports Semat. I have noted that he is not listed as a signatory. OK, enough about Semat.

Thanks to John Kern for all of his insight.

Philly ETE: Session 1

Creating Multithreaded Code Using TDD

-- Venkat Subramaniam

I've chosen this session over the others, mainly because I think Venkat is a seriously smart programmer with a great deal of interesting knowledge to share. I'm curious to see how TDDing multithreaded code diverges from typical TDD practices.

Other questions I have before we start --

If I program in Ruby, should I ever do this, even if the TDD practices are awesome?

Will we get some TDD examples in an FP language like, for example, Clojure? That would be awesome. But I think I have a good enough grasp of Scala to follow along there too.

Interestingly enough, Venkat started out trying to disprove the validity of TDD in multithreaded programming. But in his attempt, he found that he actually WAS able to. Not only that, but the TDD methods he employed saved hours of debugging multithreaded code to find a couple of bugs.

Lots of talk about the importance of TDD. Probably a nice talk for inexperienced TDDers, but for me and the two Cyrus developers to my left, Venkat is preaching to the choir.

There won't be any slides during this talk. I have the highest respect for speakers who run their talks this way. It's risky and sometimes ugly, but it's the closest thing to real-world programming that we can get at a conference. And application to the real-world is what we're all trying to get out of these two days. But I digress.

We're up to the "Let's give it a shot" section. I'm excited. Running this example using Java on IntelliJ IDEA.

First we start with a "canary" test. It is a stupid test that does something like "assertTrure(true); I do something similar with my new Ruby test files, but I don't keep them. There might be a reason to keep them around, particularly when the application is multithreaded. We're starting our example with a couple of classes, MultiValueMap and MultiValueMapTest.

I'm doing my best to keep up with the code. However, I'm using vi (don't have IDEA installed here) so it's going to be rough around the edges. I might post what I have at some later point after I clean it up. No promises though.

Best argument for TDD "it prevents Whack-a-Mole software development."

Now to the multithreaded stuff. If we make a threading testcase, and to make the test pass we make our public void class a public synchronized void class, we run into a wall. We can't test this easily. We need to avoid sticking 'synchronized' in this case.

The next tool to reach for is mocking. But Venkat suggests a more stripped down version of mocking. The result: MyLockMock which extends, not a mocking library, but ReentrantLock.

After we get some threading tests passing, we look into exception handling. He forgoes putting 'try' and 'catch' statements until he can write a properly failing test. I lost the ability to keep up coding along with Venkat at this time, but I got to witness some interesting incremental TDD.

The message is repeated in a number of ways: make it as simple is possible, move slowly, test one discreet piece of functionality at a time. Simple advice that is much better given through coding that through a bunch of slides or beating us over the heads with it.

Venkat never created a single thread to test his multithreaded design. This was the 'revelation' that led him to believe that TDD was possible, even simple, with multithreaded code.

For C#, Venkat recommends anonymous delegates. Good to know in some of my own toy projects.

A personal aside:

Venkat is awesome! He's funny, engaging, takes the right amount of questions, rolls with the changes and questions of the audience and reflects it in his live coding. What an accomplished speaker. I'd watch this guy give a talk on milk pasteurization (or something else I care equally little about).

Philly ETE: A blow-by-blow Account

For the next two days I will try to keep a running log of my time here at the Philly Emerging Tech Conference. The blog posts will likely be rough around the edges because I won't spend much time editing. My goal here is to capture the most important parts of the talks I attend, information I pick up, and things I experience throughout the conference.

Enjoy and please feel free to comment, ask questions, or slam my opinions at your leisure.

Philly ETE : Keynote 1

Bad Code, Craftsmanship, Engineering, and Certification

-- Robert C. Martin

My first experience seeing Uncle Bob martin speak, and I'm very excited. I've read his recent blog post about certification so I'm prepared for something in this vein.

Bad Code: A Crap Odyssey - a video of a single file called two.java scrolling for minutes on end. 30,000 lines of code.

Bob's oft-stated axiom of programming: "The only way to go fast is to go well."

Boy Scout Rule - when you check a module in, it must be cleaner than the one you checked out. Check out the code, make your modifications and clenaup, and then perform one "random act of kindness" on them.

Length of functions - After a history of where the "20 lines max" suggestion came from, he suggested somewhere around 4 - 6 lines. He prefers lots and lots of little functions. Even in Ruby I probably don't adhere to a maximum of 6 lines per function. So a challenge is brought (at least to me) not 30 minutes into this conference.

Functions of extreme length, complexity, obscure variable names, etc... might work for one person. But as soon as the software team grows past the size of 1, any complexity is irresponsible to your fellow programmers.

Uncle Bob knows his function is small enough when it is no longer possible to extract a method. Using an IDE is the easiest way to "extract till you drop."

Number of Arguments - again, the answer to how many arguments a function should take is "as few as possible." But other strong statements were made -- "five is insane," "never pass booleans into your functions" (by the way this is because you are blatantly violating the rule of having each function do only one thing) -- and finally settles on no more than three arguments.

Variable names - nice, long names that describe what they do are preferable in private functions. If the variable has a very large scope (meaning it is accessed from many parts of the application), the name should be more terse. Use a sliding scale of scope to determine how long the variable name should be, but of course do not sacrifice understandability.

Classes - find ways to extract classes when you see large functions - especially is you see many indentations and conditionals. Uncle Bob's practice is to promote local variables to fields when extracting methods is not possible because the scope "returns multiple values." This obviates opportunities to create classes.

Craftsmanship - "it's not the next big thing." It's been around forever. It's simply committing yourself to doing a good job. It's understanding the rules and applying them, over and over, every day. A surprise to me is that there is actually a craftsmanship manifesto. Bob suggests signing it. I'll give it a close read and maybe sign it myself.

Discipline - Hardware has been changed and improved literally dozens of orders of magazine. Software development, however, has not changed nearly so much. We still rely on "assignment statements, if statements, and while loops." Functional programming is a movement to remove the assignment statement. But FP has a bigger view. The movement is towards multicore safety, multithreaded programming that will work through our processors. The functional language that Uncle Bob supports is.... drumroll... Clojure. Hey, me too!

TDD - "the controversy is long gone." Bob likens it to surgeons washing their hands before surgery. It used to be a controversial issue but "the jury's been in on this one for a long time."

Pairing, CI, etc... are also now axioms of best practices in software development. Bob's point here seems to be telling us how obvious it should be for developers to adopt these practices. Noted. I hope it has an effect on the 75% of the audience that does not even practice TDD.

"QA should find Nothing." -- this is the goal. Make them worry about their jobs.

Maybe the most inspiring part of the talk is his last point - "convincing management." Bob's advice -- "don't even ask them." It is the developer's job to do the best work she can. She can't do that without TDD, CI, Pairing. So do them. Don't ask permission from anybody. This is your career and your responsibility to design software the best way you know how.