Automated Tests and Deployment are Real Code

Dale Emery has posted a great set of slides. I would recommend this to all software developers, testers, business analysts and everyone in the software management food chain to read.

Slide #9 from Dale's deck

This has reminded ne of a conversation with a tester not too long ago. I was doing some work with him on their test system. We pair-programmed the change - it was his first experience in pairing. It was the also only way we could achieve this tricky modification. The system has grown into a big code base. It was not well documented, apart from some Word documents that were not up to date. There was also a number of methods that mostly did the same thing.

I explained to him that the test system itself should be modular and unit-tested. His jaw dropped: "We should have tests for our tests?". "Indeed", I replied, "you are using this to functionally test the product as if it is installed in a live environment. You need to have the confidence that changes you make will not adversely affect the results."

This was an eye-opener for him.  He has never considered test code to be real code. Surely we sometimes do things in test code that we won't do in production code, but that is intentional as the code is fit for purpose. It still needs to be maintained and structured properly for it will probably live as long as the product itself would. Just as there is a cost to not doing TDD on any sizeable production of code, there is a cost to not doing it for any sizeable test code base. This test code will probably live as long as the product itself, therefore it needs to be maintained with the same fervour as the production code. Good developers understand this about unit tests, but I am seeing that the biggest breakdown still occurs where people need to build test systems to perform functional or integration testing.

The same applies for your deployment code. The DevOps movement tells us that infrastructure is code. Not only do you need to look after your production code and test code, you also need to look after your deployment code with the same zeal. It is hard to do, it requires discipline, but that discipline is what is going to allow you to relax on the weekend, instead of fighting fires in the office. I would call that prosperity:

Agile teams - Prosperity Focused, Values-driven from Schalk Cronjé

I have recently heard of a team that used to work to the mantra of building walking skeletons first, including building the skeleton deployment code first.  Of late, they have been told to manually deploy because THERE_IS_NO_TIME_FOR_WRITING_AUTO_DEPLOYMENT code. Now that is just wrong. It is a mutation of Dale's Automation Last Zombie (see slide #12 of Dale's deck).  We can even call it the Deployment Last Zombie. I am sure that team will pay for this short-term thinking pressed upon them.

Just as test-driven development has done a lot for design and testability we really need Deployment-driven development. Dare I call it DDD? Preferably not, there are too many buzz acronyms around already. Regardless of what we call it, this is what DevOps and the next level of effective software development brings to the table.

Just remember, treat your test automation code and your deployment code with the same respect that you treat your production code with.
 

Upgrading to Mandriva Powerpack 2011 - Resolving the Pain Points

I recently had to upgrade three Lenovo G550 laptops to Mandriva Powerpack 2011. I have been a Mandriva user since the 90s, but I cannot say that this has been an easy upgrade - so much so that I have considered switching to Ubuntu instead. Anways I hope that these notes might help others solving some issues on the 2011 release.

Fails to start on first boot after installation:

Boot with failsafe the first time, the 2nd time normal boot will work. I don't know why it works, it is a stupid workaround.

Fixing segmentation faults in Mozilla Thunderbird:

This issue only occurs when using LDAP for login authentication. Starting nscd works around the problem, but for some reason nscd does not want to start at boot time. I have never been a fan of nscd anyway, I after having hours trying to get debug why nscd does not start I gave up and found a workaround in a BUG 291127.

cd /usr/lib64/mozilla-thunderbird-8.0
mv libldap60.so libldap60.so.REAL
ln -s /usr/lib64/libldap-2.4.so.2 libldap60.so

This is not really a satisfactory solution, as everytime I upgrade Thunderbird, I might need to fix the symlink again, but it got it going.

Autofs cannot mount NFS locations:

This appears to be name resolving bug in autofs 5.0.6. Solved it by doing

rpm -Uvh --oldpackage autofs-5.0.5-2mdv2010.1.x86_64.rpm

CD-ROM only works during installation:

Delete /dev/cdrom line from /etc/fstab as per Mandriva Errata.

Network errors on eth0:

If the ethernet adaptor according to lspci is "Ethernet controller: Broadcom Corporation NetLink BCM5906M Fast Ethernet PCI Express (rev 02)" then set the MTU=1000 (in contrast to default of 1500) as per a previous posting.

BCM4312 wifi driver:

If you are unlucky still to have Broadcom wireless hardware in your G550, then some work lies ahead. This specific hardware identifies itself to lspci as

Network controller: Broadcom Corporation BCM4312 802.11b/g LP-PHY (rev 01)

Mandriva gives you the option to use the new dkms-broadcom-wl package, but it simply did not want to install on the default 2.6.39 kernel due to DKMS build failures. I decided to settle for using the firmware-dependent b43 driver instead. For this the b43-fwcutter package had to be installed and the firmware downloaded from openwrt and unpacked - just follow the Mandriva instructions. It was also necessary to edit /etc/modprobe.conf and add the line

options b43 pio=1 qos=0 

(Src: OpenSuse Forums)

Finally setting MTU=1000 as was the case for the ethernet hardware, solved the dropping of packets.

WS-Security Username Tokens in Groovy

The  simplest of WS-Security tokens:  just send a username and password in the SOAP header. Ignoring the usual arguments about how insecure this might be, there are a number of systems that actually utitlise this and if you are using groovy-wslite, you might need to add it. The following code snippet generates the appropriate XML fix can be placed in the header.

If you place this in a seperate closure, the just call it from your message closure as using mkp.yieldUnescaped.

Kicking off Multiple Processes from Groovy

The following is a code snippet for kicking of a number of similar processes from a Groovy script and waiting for them to finish. The stdout & stderr streams are discarded.

Normally creating interfaces with maps in Groovy when there are overloaded methods present, are non-trivial, but in this case we throw away all content. This leads to a simple nullAppender.

I just used touch as a simple illustration, replace it with whatever you need. If you want to execute your process in anotehr directory than the current worknig directory or want to customise the environment, remember that execute can take two parameters to do exactly that.

How Management Can Help Continuous Improvement

When doing some house clearing, I can across and old note book from 2003/2004. Some notes I made about Continuous Improvement (CI) really struck me as they are as relevant today as they were when I wrote them. IN this blog, you will not any earth-shattering new information, just five simple principles that managers or thought leaders within an organisation should keep in mind.


Support the process through allocation of time, money, space + other resources
Support experimentation. Do not punish mistakes, but encourage learning from mistakes. Making a mistake is not failure,



When major organisational changes are planned or coming your way, assess the impact on the existing CI system.
Incorporate it in your planning. It is therefore important to align your CI system with your strategy.



Facilitate CI co-operation across functional boundaries.
Apply shared-problem solving. If only one group improves, it is just local optimisation. But co-operating across teams and functional boundaries, optimisation can go company global. This is also an underlying principle of rightshifting.


Become a learning organisation.
As a manager you must accept that learning will take place and, where applicable, act on all the learning that have taken place. The organisation itself, needs articulate and consolidate the learning of individual and groups. It is not of much help if only individuals learn, but that learning must be shared in some form or another.


The CI system itself must also be continuously improved.
It is important that sufficient resources are made available. If necessary, lobby higher levels of management to understand this. Try to use Reinertsen's Economic model to quantify what value your CI is process providing.




Note: Unfortunately I cannot remember some of the references I was reading at the time, when I made the notes. If anyone recognises something, please let me know.

Remotely Monitor WSO2 Carbon Instance with Groovy

This is a quick Groovy script that demonstrates how to print the memory usage of any remote JVM.

// Replace HOSTNAME,PORT,USERNAME,PASSWORD as appropriate.

import groovy.jmx.builder.JmxBuilder 
def jmx = new JmxBuilder()
def conn = jmx.connectorClient (
    protocol:"rmi",
    url:"service:jmx:rmi:///jndi/rmi://HOSTNAME:PORT/server"
)

conn.connect([ 'jmx.remote.credentials' :  [ 'USERNAME','PASSWORD' ] as String[] ])

def mem = new GroovyMBean(    conn.getMBeanServerConnection(),'java.lang:type=Memory');

println "Heap:"
println "===="
println 'Committed: ' + mem.HeapMemoryUsage.committed
println '     Used: ' + mem.HeapMemoryUsage.used
println '      Max: ' + mem.HeapMemoryUsage.max

println "\nNon-heap:"
println "========"
println 'Committed: ' + mem.NonHeapMemoryUsage.committed
println '     Used: ' + mem.NonHeapMemoryUsage.used
println '      Max: ' + mem.NonHeapMemoryUsage.max

conn.close()

WSO2 Carbon:
In order to use this to monitor a WSO2 Carbon instance (ESB, Data Services etc.), edit repository/conf/carbon.xml. In <Ports> uncomment the <JMX> section. Restart the WSO2 Carbon process. The username used for access must have the admin role.

My Rules of Thumb for Milestone Estimation

Tumbled 5 personal rules I use for estimating milestone dates

http://ysb33r.tumblr.com/post/1480037891/my-rules-of-thumb-for-milestone-estimation