Monday, October 31, 2011

INVEST for requirements

A very useful acronym for use when thinking about requirements. This simple discipline helps a great deal.

Each and every requirement must be independent. It can exist in itself. If it's too large then reduce it.

Requireemts are not contracts. They are points of conversation and they can evolve. Clients often think that a documented requirement is the end of the story. This causes problems and must be avoided.

The requirement should be presented as a value to the client. It makes sense to the client.

Requirements are use in planning. Requirements must be granular enough for understanding and estimation.

Sized Appropriately
In Agile, a requirement must be around 50% of the iteration.

All requirements should be testable to ensure the requirement has been achieved.

Saturday, October 29, 2011

Microsoft's Distributed systems. A potted history.

I began to wonder how many technologies for distributed system that we have moved through since the 90's when I first started using DCOM. COM in VB was my first foray. Happy days.

In the 90's, when programming with Microsoft, we had Distributed COM, DCOM and COM+ to play with if we wanted to get computers to communicate and work together as a system. These technologies were RPC based, worked only with Windows and required the DCOM to be infrastructure available on each node. The COM acronym, Component Object Model, gives us a clue that these were component based technologies.

With .NET came a wrapper for DCOM and Com+ and that was Enterprise Services. This therefore had the same paradigm attributes as pre-.NET, it was RPC and worked only on Windows machines.

.NET Remoting came along and was very simple and useful. It was ideal for new developments on new contained systems but it required each node to have the .NET CLR installed. This therefore reduced the effectiveness to only managed code systems running on Windows. It too was RPC in style.

Running alongside these technologies has been Microsoft Message Queue, MSMQ. It has been around since NT 4 and Windows 95. This differed from the other technologies in that rather than the RPC model it concentrated on Messages. It was not component based. But it did still require the nodes to have the MSMQ infrastructure on each node.   

So we have:
.NET Remoting

In the late 90's web-services became prevalent and this enables any platform to communicate to another. Linux to Mac to Windows to whatever. Utilising the ubiquitous HTTP protocol we can use XML for the message formats and they can be in a variety of types - SOAP or RSS etc. 

SOAP (Simple Object Access Protocol) was the most common protocol specification used and is still very popular.
  • SOAP is simple.
  • It is an XML file.
  • The message contains a SOAP ENVELOPE  element as the root.
  • The Envelope contains a Body element and a Head element.
  • Your payload goes in the Body and the some controlling specifications can go into the Header. The controlling specifications are for Security, Reliable Messaging and Transaction features.

The main differing feature between SOAP and RESTful services is tat SOAP is specified to be used on any transport mechanism. REST however is tends only to be used on HTTP.

The SOAP stack

SOAP can therefore be used on any platform and requires the service contract to be coded in a similar way to RPC

REST (Representation State Transfer) 
With REST we treat services as Resources with unique identifies and we use the  HTTP defined interface of 
  • GET, 
  • POST, 
  • PUT, 
  • DELETE and 
  • HEAD.

ASP.NET Web Services 
Microsoft provided ASP.NET Web Services (ASMX) which provided early basic support for SOAP.

Web Services Enhancements
Soon after the ASMX came WSE. This provided some support for the Security, RM and Tx protocols. Also supported were some TCP support. There were however no support for REST. 

Microsoft started to create a single model for all their communication frameworks. This became the Windows Communication Framework.

Now we can use use WCF to write the communication logic and then we can use whatever features we need. 
This is a huge step forward. The developer can now write against one single framework even if the requirement is for binary communication over TCP, or SOAP or REST. 

Thursday, October 20, 2011

The very best Bayes Explanation?

Is by the very interesting Eliezer S. Yudkowsky. Spend some time on his site. It's good to think.

Why is Bayes important?
Here's a story problem about a situation that doctors often encounter:

1% of women at age forty who participate in routine screening have breast cancer.  80% of women with breast cancer will get positive mammographies.  9.6% of women without breast cancer will also get positive mammographies.  A woman in this age group had a positive mammography in a routine screening.  What is the probability that she actually has breast cancer?

Next, suppose I told you that most doctors get the same wrong answer on this problem - usually, only around 15% of doctors get it right. Most doctors estimate the probability to be between 70% and 80%, which is wildly incorrect.

What do you think the answer is?
(see below for the answer. See Yudkowsky for why.)

R - Cheat sheet - Matrices

Return to the R Cheat sheet index page

cbind()  - Column bind - Equal length vectors can be combined to create a matrix.

> x<-c(1,2,3)
> y<-c(4,5,6)
> x
[1] 1 2 3
> y
[1] 4 5 6
> mm<-cbind(x,y);mm
     x y
[1,] 1 4
[2,] 2 5
[3,] 3 6

rbind() - As above but Row binding.

> mr<-rbind(x,y);mr
  [,1] [,2] [,3]
x    1    2    3
y    4    5    6

t( ) - Transpose the matrix.

> mm
     x y
[1,] 1 4
[2,] 2 5
[3,] 3 6
> t(mm)
  [,1] [,2] [,3]
x    1    2    3
y    4    5    6

Build a matrix in one instruction.
> m<-matrix(c(1,2,3,4,5,6), nrow=2);m

     [,1] [,2] [,3]
[1,]    1    3    5
[2,]    2    4    6


> m<-matrix(c(1,2,3,4,5,6), nrow=2, byrow=T);m
     [,1] [,2] [,3]
[1,]    1    2    3
[2,]    4    5    6

Matrix multiplication
m1 %*% m2 - multiply matrix m1 with matrix m2

> m1<-matrix(c(1,5,10,12), nrow=2);m1
     [,1] [,2]
[1,]    1   10
[2,]    5   12
> m2<-matrix(c(3,4,7,9), nrow=2);m2
     [,1] [,2]
[1,]    3    7
[2,]    4    9
> m1%*%m2
     [,1] [,2]
[1,]   43   97
[2,]   63  143

and using the * operator gives scalar multiplication.
> m1*m2
     [,1] [,2]
[1,]    3   70
[2,]   20  108

Solve the matrix

> m1
     [,1] [,2]
[1,]    1   10
[2,]    5   12
> ms<-solve(m1)
> ms
           [,1]        [,2]
[1,] -0.3157895  0.26315789
[2,]  0.1315789 -0.02631579
> ms%*%m1
     [,1] [,2]
[1,]    1    0
[2,]    0    1

Eigenvalues and vectors

> m1
     [,1] [,2]
[1,]    1   10
[2,]    5   12
> eigen(m1)
[1] 15.458236 -2.458236

           [,1]       [,2]
[1,] -0.5688428 -0.9450825
[2,] -0.8224463  0.3268319


Getting elements from a matrix

diag(mm) - Diagonal

R - Cheat sheet - Vectors

Return to the R Cheat sheet main index page

Create a vector. (Note the c)
> x<-c(1,2,5,9,15)
> x
[1]  1  2  5  9 15

Create a vector with elements of consecutive integer values.
> x<-1:7
> x
[1] 1 2 3 4 5 6 7

> x<-1:10
> x<-x*2
> x
 [1]  2  4  6  8 10 12 14 16 18 20

Some commands
min(x) - minimum of the elements in x.
max(x) - maximum of the elements in x.

sort(x, decreasing=T)
> y<-c(5,8,2,0,2)
> sort(y, decreasing=T)
[1] 8 5 2 2 0

length(x) - length of vector
x[n] - get nth element from vector. first element is position 1, not position 0.
x[3:7] - get range of elements

> x<-1:10
> x
 [1]  1  2  3  4  5  6  7  8  9 10
> x[3:7]
[1] 3 4 5 6 7

x[-5] Get all elements from the vector except the 5th element.

x[x>5]  Get the elements whose values are greater than 5.

x > 5 Show which elements in the vector have values greater than b5.

> x
 [1]  1  2  3  4  5  6  7  8  9 10
> x> 5

Creating sequences

> seq(10)
 [1]  1  2  3  4  5  6  7  8  9 10
> seq(0,1,length=10)
 [1] 0.0000000 0.1111111 0.2222222 0.3333333 0.4444444 0.5555556 0.6666667
 [8] 0.7777778 0.8888889 1.0000000
> seq(0,1,by=0.2)
[1] 0.0 0.2 0.4 0.6 0.8 1.0

R - Cheat sheet - Basic operations

Return to the R Cheat sheet main index

> # this is a comment line
> 2 - 3 
> log(10) # Natural log
> 3^3 # raising to the power of three
> (2*3)+6 
> sqrt(81)
> exp(4) # exponent function

Complex numbers
Re(x) - the Real part of an imaginary number.
Im(x) - the imaginary component of an imaginary number.

> # assign an imaginary number to x
> x <-8+2i
> Re(x)
[1] 8
> Im(x)
[1] 2

atan2(y, x)
Angles are in radians, not degrees (90deg is π/2).

R - Cheat sheet - The HELP in R

Return to R Cheat sheet main index

help() - this will bring up help information.
help(topic) - will deliver decimation on the topic.
help.start() - will start the HTML version of the Help.

ls() - will show objects in the path.
dir() - will show files in the current directory.

R cheat sheet. Some functions and commands in R

I've decided to break this Cheat Sheet over a few blog entries to make it easier to read.
The R project for statistical computing

Link to using Help in R
Basic operations
Input and Output

Why did I bother to do this, there are better resources out there.

Wednesday, October 19, 2011

Install R on Mac OSX

How to install R onto a Mac. It's very easy.

The R site is here:

The Mac OS FAQ can be found here:
The FAQ answers questions such as, What machines does R for Max OS X run on?, How to build from source code?
There is also a section on the - that is the GUI application for the Mac.

What releases of  OS X
R will run on Mac OS X version 10.2 (Jaguar) or higher.
The GUI application for the Mac requires Mac OS X 10.3 or higher (Panther).

Download and Install
To download R for the mac goto a mirror. The list is on the CRAN page.
Visit the R site is here:

When you have found a suitable mirror site, download the pkg file.

Run the pkg: and get to this screen and follow the installation through.

Run the R app as normal to start the application.

Job done
That's it. A 32-bit and 64-bit version will be installed.
Getting started
R can be used a simple calculator, for example:
> 2+3
> log(10)
> 3^3
> (2*3)+6

The cheat sheet can be found here. It's useful in getting started.

R used as a simple calculator

Council could probably do with a test script.

1. Stand back and look at sign.
2. Assert.IsVisible(sign);
3. Assert.IsCorrectlyOrientated(sign);
4. Assert.IsEqual(SignRequested, SignSlungUpByCouncilMuppet);

A spokesman for Oldham Council said "a genuine error was made by staff".
A "genuine error".
"Was made by staff". People that go around putting up the Council's signs are indeed staff of the Coucil.

Monday, October 17, 2011

Running a marathon when you're 100.

Mr Singh said: "The secret to a long and healthy life is to be stress-free. Be grateful for everything you have, stay away from people who are negative, stay smiling and keep running."

He puts his stamina down to ginger curry, tea and "being happy".

No reminder list App in OSX Lion but on iOS5 iPad and iPhone

With the latest IOS upgrade there was a simple but effective Reminders app for the iPad.

This can sync with iCloud and therefore be synced across all devices. But where is the app on the Mac?

It can be found in the Calendar app as shown below.

This will be shown in the Calendar app as:

Wednesday, October 12, 2011

Code first in Entity Framework database not being created

There are a few excelent resources on the web that help a great deal as tutorials for the Entity Framework but many are a version too old-ish.
If your connection string is correct, if the contact class and DbSet class code correct, and if the project has a reference to the correct version of EntityFramework, and the database is still not being created, here are a few pointers to look for.

(But, do look at the date of this post and versions as this will be out of date quite soon.)

One thing to look for:
If you think you have created the context class correctly, and web config strings, then close the ASP.NET development server icon in the icon tray. If this server is running, it seems that Code First will be prevented from creating a new database. There may be multiple instances of this server loaded. Simply STOP them down and then re-run your project. Then check the database.

Another thing to look for:
Check that CFCodefirst.1.1 is in the Packages folder. You will probably need to look of this in the file system at the root of the project / solution . Also check that it is entered in the packages.config file.
Something like:

<package id="EntityFramework" version="4.1.10331.0" />
<package id="EFCodeFirst" version="1.1" />

Make sure that you ask for some values from the database context. That is, use a property.
And, make sure the return value is correct. If the property fetch fails, you may not know about it. It'll fail and you'll get no data + you won't get your database made. I suggest you use:

var model = _db.DummyBooks;

If you have the data code in its own project, for some kind of layering. Then beware of giving the Entities the same name as other Domain objects. I know this is vague advice, but if you name the Entitiy something like Customer, and if there is a Domain object called Customer, then the Code First might get confused and fail to create the database. It often does this without telling you why.

If all else fails,
Sometime Visual Studio and the SQL Server Management Studio just seem to play up. They say the database isn't there, sometimes it's not. Sometimes putting a breakpoint after the call to the Context property seems to work. It seems random. But, if you go step by step, from the simplest to adding more and more complexity, shutting down VS and the db connection, progress can be had.
I'm sure this nightmare doesn't always happen. As yet I don't know what causes it.

Saturday, October 8, 2011

What really is Agile development anyway

I want to write my thoughts, maybe slightly random though they are, whilst I think about it. This isn't a well researched article, or in fact an article at all. It serves only as an outlet for my thoughts and mild frustration.

Firstly I should say that I am definitely PRO the use of Agile, and definitely PRO the use of SCRUM. I am also VERY much PRO AGILE, without total SCRUM. You'll see what I mean:

I have recently been looking at the job-boards as I decided it was time to move. Now, along with the usual technology buzzwords on the buzzword list, is the now almost ubiquitous requirement for Agile, or SCRUM, and slightly less so, Kanban or XP.
It seems that every job wants Agile. That's pretty good but what do they want and what does it mean?

Some companies I guess are very serious. They are prescriptive and formal with SCRUM. For those it can be applied to everything - ordering the coffee or creating their next geo-stationary satellite. Others are just 'kind of' in the space. They like bits of it, do bits of it, and are doing ok. How do you apply this SCRUM thing to the web-site dev team or perhaps the team doing the cargo shipping management system. The teams have different cultures and dynamics, different feelings, perhaps a different attitude to their trade.

I'm not sure a lot of people know what they are talking about when they talk about Agile. For me, Agile is a general set of ideas that benefit and tend to create successful software projects. I think that Agile is a type of thinking.

A place to start is the Agile manifesto. I'm sure this site will change over time, I hope so, but my first feeling when seeing this is not a good one. Before I even get to the merits of the text and the list of luminaries in the list of Authors, I'm thinking what a hideous thing to look at. I'm also thinking that it has a  bit of a self-righteous feel to it. The Principles is where you should end up really. That's the meat and veg of the site. Forget that it looks like a prayer for some religious order.

The Principles behind the Manifesto 
We follow these principles: 
Our highest priority is to satisfy the customer through early and continuous deliveryof valuable software. 
Welcome changing requirements, even late in development. Agile processes harness change for the customer's competitive advantage. 
Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale. 
Business people and developers must work together daily throughout the project. 
Build projects around motivated individuals.  Give them the environment and support they need, and trust them to get the job done. 
The most efficient and effective method of conveying information to and within a development team is face-to-face conversation. 
Working software is the primary measure of progress. 
Agile processes promote sustainable development.  The sponsors, developers, and users should be able to maintain a constant pace indefinitely. 
Continuous attention to technical excellence and good design enhances agility. 
Simplicity--the art of maximizing the amount of work not done--is essential. 
The best architectures, requirements, and designs emerge from self-organizing teams. 
At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.
Link to the manifesto

This is the heart of it all. This for me, after years of fiddling with projects, is not only the fashionable thinking of our time, but it's the best chance of success.

What about SCRUM, DSDM...? Well for me, these are not what Agile is about. They are ways of playing at Agile for sure, but they're not what Agile is? For me, in a sense, they take away some of the agile ethos, yet they do deliver good projects. I'll explain.

Objectives of Agile
I think that the objectives of Agile are things like quick thinking, quick reactions, the stuff in the principles. It is about getting the right stuff done for the client and knowing that all of us, clients and project producers, will constantly be making mistakes in our understanding and communication.

Methods of being Agile
I think the SCRUM, the XP and the DSDMs of this world are the methods, the prescribed and prescriptive methods to go from the tools and techniques to the result. The result is the product developed by using the objectives.

Techniques for using in our pursuance of Agile
The tools - well I think they're things like TDD, Unit testing, User stories, DDD.

I think the three previous sections are a bit general and debatable, and probably a bit wrong. In a few weeks, if I review what I have written here. I may well change it but I think the principle is correct.

People say that the Agile thing started around '86 when Scrum was first mooted. I don't think it was used then though. It was much later.
I remember using DSDM in the late '90s, and then XP. I'm sure I didn't hear the term Agile though. One day it suddenly became very trendy to talk about Agile. It was an invented term to group these new ways to develop software. Kanban arrived in 2004, a few years before that the manifesto was created.

At one end of the Agile spectrum is the philosophical, the cultural aspect of developing software. At the other end is where the philosophical approach has been distilled and implemented with tools to become a prescriptive process. The prescribed process is something losing it's free spirit. Sounds a bit hippy, but the free spirit is dangerous. In the hands of the careless it is another lost project. In the hands of the lazy its an excuse. That is why we need SCRUM. SCRUM is therefore good. A force for good but it is not, in itself, the thing that is AGILE.

Monday, October 3, 2011

The Five Dysfunctions of a Team - Patrick Lencioni


As difficult as teamwork can be to achieve, it is not complicated. And so, if I can't describe it in a page or two, then I've probably made it too complex. Here goes.
The true measure of a team is that it accomplishes the results that it sets out to achieve. To do that on a consistent, ongoing basis, a team must overcome the five dysfunctions listed here by embodying the behaviors described for each one.
  • Dysfunction #1: Absence of Trust: Members of great teams trust one another on a fundamental, emotional level, and they are comfortable being vulnerable with each other about their weaknesses, mistakes, fears, and behaviors. They get to a point where they can be completely open with one another, without filters. This is essential because ...
  • Dysfunction #2: Fear of Conflict: ... teams that trust one another (they have #1 above) are not afraid to engage in passionate dialogue around issues and decisions that are key to the organization's success. They do not hesitate to disagree with, challenge, and question one another, all in the spirit of finding the best answers, discovering the truth, and making great decisions. This is important because ...
  • Dysfunction #3: Lack of Commitment... teams that engage in unfiltered conflict (they have #2 above) are able to achieve genuine buy-in around important decisions, even when various members of the team initially disagree. That's because they ensure that all opinions and ideas are put on the table and considered, giving confidence to team members that no stone has been left unturned. This is critical because ...
  • Dysfunction #4: Avoidance of Accountability: ... teams that commit to decisions and standards of performance (they have #3 above) do not hesitate to hold one another accountable for adhering to those decisions and standards. What is more, they don't rely on the team leader as the primary source of accountability, they go directly to their peers. This matters because ...
  • Dysfunction #5: Inattention to Results: ... teams that trust one another, engage in conflict, commit to decisions, and hold one another accountable (they have #4 above) are very likely to set aside their individual needs and agendas and focus almost exclusively on what is best for the team. They do not give in to the temptation to place their departments, career aspirations, or ego-driven status ahead of the collective results that define team success.

I dislike the book really. Probably because  I'm too lazy to read it. The above snippet is in Chaper 42. In Chapter 43 there's a team assessment thing.

Saturday, October 1, 2011

Modulus of a Complex number

For, z = a + bi , the modulus is
|z| = sqrt(a^2 + b^2)

z = (1 + 2i)^2
z = (1 + 2i) . (1 + 2i)
  = 1 + 2i + 2i + 4i^2    (remember that, i^2 = -1 )
  = 1 + 4i + 4(-1)
  = -3 + 4i

|z| = sqrt( -3^2 + 4^2)
    = sqrt(9 + 16)
    = sqrt(25)
|z| = 5

Zombie - NSZombie

Leicester city council might not be ready for the Zombie attack. But setting NSZombie to YES in the XCode environment helps in finding memory leaks.

A bit more on Complex numbers

We know that, 
i = sqrt (-1)

and therfore, 
i^2 = -1

Also, sqrt (9) = 3

 sqrt (-9) = sqrt(-1).sqrt(9) = i.sqrt(9) = 3i
We can check this. 
(3i)^2 = 3^2.i^2 = 9.-1 = -9

(3i)^2 = -9
3i = sqrt(-9)

3i is an imaginary number.

Complex number are imaginary and real numbers together.

6 + 3i is a complex number.

Addition of complex numbers 
Here we have two complex numbers zi and z2

zi = a + bi
z2 = c + di
we add the real parts then add the imaginary parts.

zi + z2 = (a + c) + (bi + di)
        = (a + c) + (b + d)i

Subtraction of complex numbers 
Here we have two complex numbers zi and z2
zi = a + bi
z2 = c + di
we subtract the real parts then subtract the imaginary parts.
zi - z2 = (a - c) + (bi - di)
        = (a - c) + (b - d)i

Multiplication of complex numbers 
Again we have two complex numbers zi and z2
zi = a + bi
z2 = c + di

zi . z2 = (a + bi) . (c + di)
        = a(c+di) + bi(c+di)
        = ac+adi + cbi + (bi.di)          ....... (eqn 1)
let's sort out the (bi.di)
(bi.di) = bd.i^2
we know that i^2 = -1
so, db.i^2 = -bd
Returning to where we were in eqn (1),
zi . z2 ac + adi + cbi - (bi.di)
        = ac + adi + cbi - bd
        = (ac - bd) + (adi + cbi)
        = (ac - bd) + (ad + cb).i

Division of complex numbers 
Again we have two complex numbers zi and z2
zi = a + bi
z2 = c + di

zi / z2 = (a+bi)/(c+di)

We can use the rule:
(a+b).(a-b) = a^2-b^2

The Conjugate of a complex number is a reverse of the direction of the imaginary number.
The Conjugate of (a + bi)is (a - bi)
The Conjugate is written with a bar over the top, so the conjugate of z1 is written z1 bar. 
How do I type that here? I don't know!

When we multiply an imaginary number by its conjugate we get a Real number. Here's the trick:

zi     (a+bi)   c-di   ac-adi + bci-bdi^2
--- =  ------ . ---- = -------------------
z2     (c+di)   c-di   c^2 + d^2

      ac-adi + bci-bdi^2         [remember that , i^2 = -1]
    = ------------------- 
         c^2 + d^2

      ac-adi + bci+bd
    = ----------------- 
         c^2 + d^2

       (ac+bd) + (bc-ad)i
    = --------------------- 
         c^2 + d^2

       ac+bd         bc-ad
    =  --------  +  ------- . i
       c^2+d^2      c^2+d^2


1+2i    (1+2i)   2-3i   (1.2) + (1.(-3i)) + (2.2i) + (2i.(-3i))

---- =  ------ . ---- = ---------------------------------------
2+3i    (2+3i)   2-3i   (2.2) + (2.(-3i) + (3i.(2) + (3i.(-3i))

       2 - 3i + 4i - 6i^2
     = ------------------
       4 - 6i + 6i + 9i^2

        2 - 3i + 4i + 6

     =  ------------------
             4 + 9
        8 + i
     =  -----
     =  8      1
       ---  + --- i     
       13     13