"Managing Ruby Teams" - Luke Melia
First thing's first: what makes rubyists different?
- lots of independent thinking
- all open source with no dominant conglomerate running frameworks, dev techniques, etc... (see C#, Java)
- rubyists constantly embrace change (for example, Rails has 1500 contributors and consistently breaks backward compatibility - and everyone seems cool with this).
The Key Principles:
- self-organizing team (core of XP practices); this extends to empowering the team to maximize its own effectiveness.
- personal growth: simply being successful on a team, for most developers, is not enough. So a manager's job is to make sure they grow as professionals while working on the team.
- hire well (Cyurs: check)
- servant leadership - Jess Hottenstein may actually know what he's talking about.
- inspirational leadership
Practices:
1) Sit with your team. Listen and observe, but think twice before interrupting. Let the team try to hash out their issue.
2) Management by coding around. Pair program with different people, walk around the team's area and observe information radiators. Observe the "key feedback loop."
3) Tools and workspace. Share editors (if possible), tools, scripts, etc... amongst the entire team. And, obviously, sit together in an open area.
4) A Policy for Policies. You will have to put some rules in place but try to put in as few as possible. Fit each policy to the frequency and severity of the target problem.
5) Retrospectives and Kaizen. Luke doesn't distinguish between the two, but I think both are separate and important activities.
6) Delegation. Identify discreet, repetitive tasks and automate them. When you can't automate, delegate. They should be shared amongst the team. (an example from my team is tracking and troubleshooting (if necessary) the twice-daily push to our staging server).
7) The Andon Cord. The stop-the-line cord. (This is from Toyota Production System). Give your team this power. (And definitely stop the line if the build breaks!)
8) Be patient. Be aware of the forming, storming, norming, performing cycle.
9) One-on-ones. Simply a meeting between the project lead and the team member. Make the time pre-planned to privately address concerns and questions. You can also take this time to give feedback.
10) Motivation. Understand what motivates each employee. It can vary widely. For rubyists, don't underestimate the "mastery" motivator.
11) Feedback. Learn how to deliver it effectively. On our project we are actively trying to become better at giving each other feedback. It's not easy, but it really promotes a healthy team.
12) Open Source. Use it; share your changes. Using git submodules is a great way to contribute to open source while still addressing the team's concerns.
13) Hiring. Do it like Cyrus does it (Note: This is not exactly what Luke said). But one interesting thing that we don't do is "get 100% of the team on board." This can be tricky; we don't always know where the new hire is going or what team they'll be on. But that doesn't mean it's not a good goal to shoot for.
14) The Us-Them Relationship. Fight hard against the "our team versus the company/other project teams" relationship. Cultivate it externally - carefully. This means looking into the community to find other teams or projects that your team can learn from.
15) Be honest. Don't lie to our team. Share your passion. This is the 15th bullet point, but Luke makes it sound like the most important one. "Destroy trust, and the whole thing comes tumbling down."
Check out the reading list at: http://bit.ly/gorucobooks for some more info. Also Manager Tools, Agile Executive, and Agile Toolkit are all podcasts that helped Luke become a better manager.
15)
Saturday, May 22, 2010
GoRuCo Pace U. - Session 2
"memprof: the ruby level memory profiler" - Aman Gupta
Addressing the ruby GC is a way to get Ruby to run faster. (Aman is speaking of MRI Ruby, which is the most commonly one used.)
When the GC is running, it "stops the world." None of your code runs while this happens. The GC is conservative, which makes it difficult to tweak. It also utilizes a design decision called "mark and sweep" which is a less-than-optimal one.
To improve performance:
- avoid leaked references: you create an object and you don't need it anymore, but you're still holding onto the reference - the GC will not clean it up but always has to look at it.
- create fewer objects: obviously
The tools:
1. ObjectSpace.each_object: When it's sorted and printed out you can get a sense of what's going on in the VM. An example:
types = Hash.new(0)
ObjectSpace.each_object do |obj|
types[obj.class] += 1
end
pp types.sort_by{ |klass, num| num }
2. gdb.rb: gdb hooks for REE. Written by Aman. It's "a bunch of python code" that understands how your ruby code works. gdb can find and fix leaks and has a number of different processes.
3. bleak_house: installs a custom patched version of ruby and tells you what is leaking and WHERE the leak is happening. The drawback is that you have to reinstall ruby and all of your gems. Aman also wrote a patch that includes more verbose information about the leak and where it's happening. It is a "heap dumping" patch.
4. memprof. ease of use, detailed information, and simple analysis. Thus it allows processing via various languages and databases using simple JSON data format. You can follow all the under-the-hood specifics on www.timetobleed.com.
MemProf.track: like bleak_house but only for a given block of code (not the entire code base). So we can drill down and get more specific into areas of your code base.
** You'll want to try these tools out in production mode. Development mode has more overhead and your results will be skewed **
memprof.com: a web-based visualizer and leak analyzer. Aman used it to find a leak in the development mode of Rails 3 beta.
Addressing the ruby GC is a way to get Ruby to run faster. (Aman is speaking of MRI Ruby, which is the most commonly one used.)
When the GC is running, it "stops the world." None of your code runs while this happens. The GC is conservative, which makes it difficult to tweak. It also utilizes a design decision called "mark and sweep" which is a less-than-optimal one.
To improve performance:
- avoid leaked references: you create an object and you don't need it anymore, but you're still holding onto the reference - the GC will not clean it up but always has to look at it.
- create fewer objects: obviously
The tools:
1. ObjectSpace.each_object: When it's sorted and printed out you can get a sense of what's going on in the VM. An example:
types = Hash.new(0)
ObjectSpace.each_object do |obj|
types[obj.class] += 1
end
pp types.sort_by{ |klass, num| num }
2. gdb.rb: gdb hooks for REE. Written by Aman. It's "a bunch of python code" that understands how your ruby code works. gdb can find and fix leaks and has a number of different processes.
3. bleak_house: installs a custom patched version of ruby and tells you what is leaking and WHERE the leak is happening. The drawback is that you have to reinstall ruby and all of your gems. Aman also wrote a patch that includes more verbose information about the leak and where it's happening. It is a "heap dumping" patch.
4. memprof. ease of use, detailed information, and simple analysis. Thus it allows processing via various languages and databases using simple JSON data format. You can follow all the under-the-hood specifics on www.timetobleed.com.
MemProf.track: like bleak_house but only for a given block of code (not the entire code base). So we can drill down and get more specific into areas of your code base.
** You'll want to try these tools out in production mode. Development mode has more overhead and your results will be skewed **
memprof.com: a web-based visualizer and leak analyzer. Aman used it to find a leak in the development mode of Rails 3 beta.
Friday, May 21, 2010
GoRuCo Pace U. - Session 1
"Grease Your Suite: Tips and Tricks for Faster Testing" - Nick Gauthier
Nick uses, Rails, Shoulda, Factory Girl, and Paperclip. He favors Factory Girl over fixtures, as do I. Shoulda is interesting because it can be integrated with Test::Unit but still give it a BDD feel. I've used it a bunch of times but the jury is still out for me.
In his "vanilla" test suite, the one with standard unit and functional tests (and probably other), takes 13 minutes, 15 seconds.
Implementing Shoulda with nested contexts took the test time down to about 5 minutes.
Using Paperclip saves time during functional testing. By using Paperclip to return negligibly sized images during
parallel_tests, specjour, deep-test and tickle are tools to fork your testing process across multiple cores. According to Nick, these are good tools but they don't do the "balancing" across the cores very well. parallel_tests is for Test::Unit. It requires more setup in that you have to have multiple test databases running.
But Nick recommends hyrdra. It will run with Test::Unit (still tops in my mind) and doesn't require multiple db setup. On top of that, it has "smarter" balancing across multiple cores. It will also run with cucumber.
Some background on environment loading. Test::Unit will actually load the env 4 times. Cucumber does 2 and RSpec tests just does it once. Hydra also only boots the env once even for multiple test frameworks.
Hydra brought the testing time back to 1 minute, 26 seconds.
The default behavior for filesystem mounting on most 'nix systems is journalling enabled. This means that all ata is committed into the journal prior to being written into the main filesystem. Another option is journal_data_writeback. This means that data may be written into the main filesystem after its metadata has been commited to the journal. This will increase speed, but at the cost of total safety of your metadata. You don't care about this in your tests, so try it. But make sure you don't take this to production!
atime - totally useless (according to Nick) so turn it off in your filesystem.
After this the total test time is only 50 seconds.
Next, Nick switched to using RVM to install Ruby EE. He then implemented tcmalloc. He bumped up RUBY_HEAP_MIN_SLOTS as a setting on ruby EE from the default (10,000) to 1,000,000. This says "take a huge chunk of memory" to run our tests with. He bumped RUBY_GC_MALLOC_LIMIT up to 1,000,000,000. RUBY_HEAP_FREE_MIN went from 4,096 to 500,000. You can simply set this in your profile for testing.
After this, the total test time is 18 seconds. Quite a jump! And it looks relatively simple to implement. Right now, it only takes our project about 2 minutes to run our test suites. But once we get into Cucumber, time is going to increase rapidly.
Nick uses, Rails, Shoulda, Factory Girl, and Paperclip. He favors Factory Girl over fixtures, as do I. Shoulda is interesting because it can be integrated with Test::Unit but still give it a BDD feel. I've used it a bunch of times but the jury is still out for me.
In his "vanilla" test suite, the one with standard unit and functional tests (and probably other), takes 13 minutes, 15 seconds.
Implementing Shoulda with nested contexts took the test time down to about 5 minutes.
Using Paperclip saves time during functional testing. By using Paperclip to return negligibly sized images during
parallel_tests, specjour, deep-test and tickle are tools to fork your testing process across multiple cores. According to Nick, these are good tools but they don't do the "balancing" across the cores very well. parallel_tests is for Test::Unit. It requires more setup in that you have to have multiple test databases running.
But Nick recommends hyrdra. It will run with Test::Unit (still tops in my mind) and doesn't require multiple db setup. On top of that, it has "smarter" balancing across multiple cores. It will also run with cucumber.
Some background on environment loading. Test::Unit will actually load the env 4 times. Cucumber does 2 and RSpec tests just does it once. Hydra also only boots the env once even for multiple test frameworks.
Hydra brought the testing time back to 1 minute, 26 seconds.
The default behavior for filesystem mounting on most 'nix systems is journalling enabled. This means that all ata is committed into the journal prior to being written into the main filesystem. Another option is journal_data_writeback. This means that data may be written into the main filesystem after its metadata has been commited to the journal. This will increase speed, but at the cost of total safety of your metadata. You don't care about this in your tests, so try it. But make sure you don't take this to production!
atime - totally useless (according to Nick) so turn it off in your filesystem.
After this the total test time is only 50 seconds.
Next, Nick switched to using RVM to install Ruby EE. He then implemented tcmalloc. He bumped up RUBY_HEAP_MIN_SLOTS as a setting on ruby EE from the default (10,000) to 1,000,000. This says "take a huge chunk of memory" to run our tests with. He bumped RUBY_GC_MALLOC_LIMIT up to 1,000,000,000. RUBY_HEAP_FREE_MIN went from 4,096 to 500,000. You can simply set this in your profile for testing.
After this, the total test time is 18 seconds. Quite a jump! And it looks relatively simple to implement. Right now, it only takes our project about 2 minutes to run our test suites. But once we get into Cucumber, time is going to increase rapidly.
Subscribe to:
Posts (Atom)