Embedding date and time into a file in windows - "But it worked last night"

I'm patching a tiny project at work with duct tape until we can get an enterprise solution. A small batch job runs on the windows scheduler doing something like this:

MyCommand.exe  --output=Saturday1201amUS_%date:~10,4%-%date:~4,2%-%date:~7,2%-%time:~0,2%-%time:~3,2%-%time:~6,5%.txt

I ran it yesterday afternoon and it worked great. It created a file named
Saturday1201amUS_2009-12-21-15-46-00.21.log. This morning I did another test with my client and it failed miserably after creating a file named 'Saturday1201amUS_2009-12-22-'. A little investigation turned up that windows does not prepend a zero in front of the hours before noon so my executable file was getting a space in front of 9 oclock:

MyCommand.exe  --output=Saturday1201amUS_Saturday1201amUS_2009-12-22- 9-47-20.31.log

It worked great after putting quotes around the file name.

MyCommand.exe  --output="Saturday1201amUS_%date:~10,4%-%date:~4,2%-%date:~7,2%-%time:~0,2%-%time:~3,2%-%time:~6,5%.txt"

Bitter Refactoring

Kimberley Bitter
Originally uploaded by kamoda

After reading Martin Fowler's book on Refactoring I've been a big fan of the practice. We all know that adding in tests to insure the refactoring doesn't change the results is essential.
But this last week the real world crept into our world of programming paradigms. We refactored a complicated section of code to make it simpler, more readable, and faster - a big win; except that after the code was fielding, one of our customers reported a problem.
Their use of our product actually depended on a bug in the old version. Refactoring the code fixed the bug and caused our customers grief.
I'm still a big fan of refactoring and continue to do it, but now I'm a little wiser that refactoring carries more risk than I'd thought before.

Xeno's Paradox and Software Delivery

Achilles
Originally uploaded by Jennifurr-Jinx

The ancient Greek philosopher Zeno of Elea composed a paradox to support his teacher, Parmenides's view of motion and change in the universe. His story is about a race between Achilles and a turtle. The turtle gets a small head start, but before Achilles can overtake the turtle, he must catchup with the turtle. Once Achilles has caught up, the turtle has moved forward, so Achilles must move a little further to where the turtle is now. But by now the turtle is ahead and Achilles must again catchup etc... so Achilles never really reaches the turtle.

I've noticed this same phenomenon in software development with regard to last minute features being requested to be thrown into the release. With three days to go before release, we get a small 4-hour feature request to be added. Then two days before the release another smaller item - really tiny, wouldn't take much effort at all - just two hours. One day to go, and another almost infinitesimally small item to be added - it will just delay the release by one hour and it's really needed.
In Zeno's world, the software will never get released.

Keep Swimming or You Die

Some shark species must keep swimming to create a flow of water through its gills, or they will suffocate. Our aquatic friends of the deep can teach us something about software development.
Last night I stuck up a conversation with someone at my daughter's athletic event. Come to find out, he was an underemployed software developer. He was ruing the fact that for eight years he kept his head down and simply created java applications for his employer. He didn't investigate developing apps for this wacky internet web-thingy which is quite popular with the younger crowd. When he was laid-off, he found his excellent java application skills not much in demand in a sparse job market. Admirably he is going back to school and getting the skills needed for today's web-connected world.
The lesson from the fishies: Keep learning or your career will die.

The Toyota Way

I just finished listening to "The Toyota Way" while commuting in my Camry as a way to get an introduction to Lean Software Methodology. A few things that struck me in the book (in no particular order):

1. Counter Intuitive. Many of the ideas just don't sound right like stopping production if a problem is spotted on the line, or not using machines to their capacity. Although the ideas sound crazy at times, you can't argue with the success of Toyota.
   
2. Set Based Solutions. Don't just create one solution to a problem, create several solutions and discuss the merits of each with many people. Implement swiftly.
   
3. Continuous Improvement - "Kaizen". The production process should always be improving. People need to be committed to adapting to change and initiating change.
   
4. Commitment to Training. Employees should be trained well. I was amazed at the amount of training that Toyota offers its people.
   
5. Flow. Don't just optimize parts of the process - optimize the total system.
   
6. Fix the Process, "Pokayoke". When something does not perform as expected, don't just fix it, find out why it failed. Pokayoke means to find the root problem and take action so the problem does not occur again.
   
7. Easy to Copy Badly. The Toyota Production System is easy to copy badly with disastrous results. The example given was the magic chord that any employee can pull to stop the entire line when she discovers a defect in the process. Some plants tried to copy this method, but found it not to work well. In the Toyota plants pulling the chord doesn't immediately stop the line, but gives the team leaders some time to resolve the situation before stopping the complete line.
   
8. Focus on "Standard Work". Each step of production has written steps on how to do the work. The employees should be doing the process exactly as written. When defects in the line occur, the line managers focus on how the defect occured. Was the standard work followed? If not, why not? If it was followed, the standard work needs to be changed. Standard Work is the first step to continuous improvement. You cannot effectively improve an ad hoc process.
   
9. Eliminate Waste, "Muda". "anything other than the minimum amount of equipment, materials, parts, and workers (working time) which are absolutely essential to production" is waste. Seven classes of waste: overproduction, delay, transportation, processing, inventory, wasted motion, and defective parts. In software development I see a lot of inventory - written and tested software which has not been released because it's hard to install mission critical software.
   
10. Visual Displays, "Andon". It's important for everyone to see progress, so Toyota emphasizes big signs where everyone can see problems and production.
    
11. Hands-On Approach, Genchi Gengustu "Go see the problem". Management must visit the factory floor and see for themselves the problems.
    

15 Helpful Tips from Scott Bellware's Behavior Driven Class

On March 24th, last Tuesday, I attended Scott Bellware's helpful seminar on "Test-Driven Development and Behavior-Driven Development". A few of my take-aways:

1. One bug takes a unit of time to fix, but with multiple bugs the time goes up exponentially since the bugs can interact. Moral: Code in steps and only introduce one bug at a time.
   
2. Ask if the test you are about to write is a valuable test.
   
3. Code-DB impendence mismatch example: If orders have links to customer in the db, in code the customer has links to the orders.
   
4. Using "static" methods between layers is bad, it's like welding the objects together.
   
5. Inheritance is strong coupling, to be used as a last resort.
   
6. Scott and his team removed an entire layer of testing when they realized it provided less value than it took to maintain.
   
7. Databases are places for dead objects, not logic.
   
8. Someone at the meeting said Microsoft doesn't really do domain applications like we do, so they cannot really offer much advice on how to code.
   
9. "Reuse" is not a good goal.
   
10. You can overwrite tests, simple things are tested in more complicated tests.
    
11. Don't use prepopulated databases with test data. Start with an empty database, and end with an empty database.
    
12. When Scott asked the participants "Who does unit testing?" 80% of the 25 people raised their hands. "Who uses MSTest?", one hand.
    
13. Regression tests are a side effect of Test Driven Development, not the primary goal.
    
14. Doing traditional OO analysis (nouns, verbs,...) creates more objects than needed.
    
15. Don't directly expose an object's collections; use helper functions on the object. (Use Customer.Add(obj) instead of do Customer.list.add(obj))
    
    

Major Version Upgrade with No User Downtime Using Virtuals Machines

This last week we upgraded our web-based software to a new major version without any downtime for our users. It was a great success. This only works if the databases backing the site are upwardly compatible. Here's the steps:
1. It all started last year when we hosted our application in a virtual machine (who doesn't these days anyway) even though our application was the only one running on the server.
2. On each of the production boxes we recently installed a parallel shadow virtual machine and loaded the new version of the software. We created test copies of the databases on our database server farm.
3. We created a shadow farm address (www.siteB.OurFarm.com instead of www.siteA.OurFarm.com) in our local router/load balancer and pointed the siteB domain to our shadow virtuals. Since all the nunit and functional tests were all done in our internal test farm, this round of testing was for configuration issues (can all the machines see all the databases? Can the farm machines see each other? Can outside users hit all the shadow boxes?).
4. We upgraded the databases on the live system by adding our new tables. The old system ran on the upgraded databases, but did not use all the tables. We incrementally pointed our shadow virtuals to the three live databases and tested them.
5. On the big transition day, we simply switched our domain address "SiteA" to point to our new shadow virtuals and all was golden.

PostScript: If any errors had occurred with the new version, we could back out by pointing "SiteA" back to the original servers (assuming no damage to the databases). We had made copies of the databases before had so we could switch back to them if something had corrupted the databases.

SqlServer Timeout Expired Exception

One of our users emailed and said he got the following exception with a reporting query:

System.Data.SqlClient.SqlException: Timeout expired.  
The timeout period elapsed prior to completion of the operation or the server is not responding.   
at System.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection)

The exception just started happening because the database has slowly been increasing to where the report query fails to finish in a timely manner. The timely manner is 30 seconds, the default for .Net database connections.

You can increase the default time of 30 seconds by using the "CommandTimeout" field on the "Command" object as shown below:

using (connection) {
    SqlCommand sqlcommand = connection.CreateCommand();
    sqlcommand.CommandTimeout = 60; //default is 30 seconds
    sqlcommand.CommandText = sqlText;
    ... 

Don't be misled by the database connection string option, "Connection Timeout", e.g., "Data Source=myMachine; Integrated Security=SSPI; database=Northwind;Connection Timeout=60". That timeout is for getting the connection, not for how long it queries the database.

PeopleWare by DeMarco and Lister

I've just finished reading Peopleware by DeMarco and Lister and have put it on my Recommended Reading list. Buy one and read it.

This is a great book, it's 20 years old, but still relevant. I thought the highlights were these:

1. Projects fail not for technical reasons, but for sociological reasons.
   
2. Our successes stem from good human interactions, not from incremental technical improvements.
   
3. Making errors should be encouraged - everyone should have a few mistakes during the project when we tried something and it didn't work. When the answer to "How many mistakes did you make?" is "none"; that's a bad sign people aren't trying enough new things.
   
4. We need people who make the team "jell" - they are worth more than coders.
   
5. We spend too much time trying to get things done, and not enough time asking if the thing is worth doing.
   
6. People really like to make very high quality stuff, but the market wants high quantity.
   
7. Parkinson's law, "work expands to fill the alloted time", doesn't apply well to software developers.
   
8. Beware of claims to increase software productivity by 100% because most time is not spent programming.
   
9. When testing productivity of programmers, some were 10 times more productive than others. This is not quite what it sounds like since not all a person's time is taken with programming.
   
10. During "programming games", language did not affect the outcome (except assembly language) and experience didn't matter (if they had more than six months experience).
    
11. Quiet places for work are extremely important to allow programmers to get in the "flow" of their work. (This agrees with Joel Spolsky, but not the Agile wing of development).
    
12. Corporations have entropy. Differences, and creativity, in departments are slowly ground out of them to create a flat, bleak landscape of sameness.
    
13. Big Methodologies don't work; better to use training, tools, and peer reviews.
    
14. Fujitsu encourages all projects to do some aspect in a nonstandard way.
    
15. Letting teams strive for less than perfect quality is the "death knell" for the team. Teams really want to do high quality work.
    
16. "Sociology matters more than technology or even money."
    
17. Overtime is destructive and counter-productive.
    
18. People hate change. We are agents of change.
    
19. Try to keep people on one project at a time, not spread over several at one time. They are less efficient and more frustrated.
    
20. Don't waste your peoples time in meetings and overstaffing projects early.
    
21. Create a fun, creative, productive team environment - somehow.
    

Their recommendation for having quiet offices struck me. As a result we got the loud cell phone in the office turned down and I bought one of my people, who is affected by the noise, some noise-canceling headphones to see if that would help with the noise. It didn't.

Falling

As a programmer do you feel like you are falling behind in the current technology? If you don't, you aren't paying attention.
Isaac Newton came to the revelation that gravity held the heavenly bodies in orbit by theorizing a cannon ball being shot fast enough that it was always just falling over the horizon thereby putting it in orbit. That's what software development is today - we are always falling over a body of knowledge but never quite resting on it.

Creating software is changing rapidly:

• new Agile techniques for organizing and managing projects
  
• new Model-View-Controller frameworks for Java, C#, and python
  
• new language features, especially C# 3.0 has some wonderful new toys
  
• new rebirth of functional programming
  
• new ActiveRecord type object-relational mapping tools
  
• new AJAX toolkits
  
• oh, yeah, that new handset stuff for the iPhone and Android.
  

How do you keep up? Here are a few thoughts:

• Read technical blogs. My list is here.
  
• Do code reviews or paired programming. At our company we do biweekly code reviews for 30 minutes. I've learned so much from seeing how my compatriots code. Do the new CodingDojo thing with friends.
  
• Be involved with your local user groups. Austin is fortunate in having many fine groups such as a great Java User's Group, Dot Net User's Group and an Agile group. See a more complete listing at my del.icio.us austin+tech bookmarks.
  
• Get together with friends from companies you used to work at and see what their company is doing. How do they do builds? Unit and integration testing? What are the problems they have?
  
• Buy a domain name and play around.
  
• Learn a new language every two years.
  
• Read good classic books. Some of my favorites are here.
  

If you aren't learning everyday, you're falling behind.