IMVVHO, the right approach lies midway, no jumping into the water before knowing it's too cold or hot (Agile), or sitting on the side for 2 hours and testing the water before jumping in (Waterfall).

Should I be so lucky, I would love to go thru IDesign's process someday in a full scale project. Sounds my type.

Thanks for the clarification, Brian.
As for the up-front design - I have to admit that that is the message I got and this could have been a simple misunderstanding on my part. Put it that way it all sounds a lot more logical and iterative, and makes more sense to me.
As for Unit tests - I do subscribe to the TDD method, simply because I've seen its results in action. And no - TDD and up front design are not mutually exclusive. You can have a big design and SRS and DTDs for you class, and they could even be written with the appropriate list of minimal unit tests that the design must pass. The developer then writes the *tests* before the code. It worked out great on several projects I've seen/been on.

In any case, I thought Juval's POV is fascinating and insightful and there's certainly lots to be learned from it.

Cheers,
Roy.

Thanks Brian! Disclaimer: I have a lot of respect for Roy O. and this is not directed to him, but I am certain that we disagree on several points and that's OK.

I am so fed up with listening to how agile is the next silver bullet. I split "agile" into two categories: engineering practice and methodology/philosophy. The former simply includes many mature software engineering practices that have nothing to do with agile such as TDD (arguably), nightly builds, staged delivery, etc, etc. The latter is where it gets kind of nuts and where my contentions are many.

I have been a practitioner of many of the IDesign Method techniques over the last 2 years and have been enormously successful. These techniques include publishing a collaborative Architecture & Design Report (ADR) which at an architectural level (with some design) addresses the big picture of an enterprise platform: Design goals (scalability, availability, performance, reliability, security, etc) and how they align with the platforms and technology choices proposed within the report. I go into Application Architecture, System Architecture, providing models that communicate the process and message flow aspects of the system. Platform and technology aspects are modeled including authentication boundaries and transaction isolation models. This is not a big bang approach- instead it is a rational collaboration led by myself -as the lead architect- of application, infrastructure and release teams to collaboratively identify these critical points up front. What is the OLTP database isolation level? What should it be? How are we doing transactions? What is the difference between an atomic and long running transaction and where are the trade-offs? What is our security model? Are we going Trusted Subsystem or Impersonation/Delegation? Can the infrastructure support the latter or the former? Are we doing Kerberos or NTLM? What is the packaging and deployment going to look like? Oh, don't tell me we can't answer these basic questions before we write any code! This is quite frankly infuriating to me.

Sure, agelists will frown at the anywhere from 40 to 160 man-hours that will go into this, but on a $3M enterprise project consisting of over 3,500 man-hours, this is minimal and the payoff is significant. In the end, the document is not perfect, but it gels and serves to lend conceptual integrity (as defined by Brooks) throughout the entire system. Another important point is that architecture is all about compromise and many times, the design goals (set by the customer) can serve to be tie breakers in choosing between a synchronous or asynchronous calling pattern for example.

From there we do release planning, taking a staged delivery approach. Stages don't have to necessarily fit into a 4 week increment, some may be 2 weeks, some may be 8 weeks, that's OK. We pick the best developers available, but inevitably, skill levels will vary. Some developers will embrace the architecture (making some recommendations, of course) and lead. Others will follow and learn. This isn't NASA after all. This is reality.

Nightly builds are critical such that you can release at a moments notice. TDD applied in a pragmatic sense is also useful, and I also make extensive use of test harnesses, preferring Smart Clients (WinForms). The latter is critical when you have a system consisting of over 70 assemblies and at least 120 components. The harness communicates tangible progress to managers and stakeholders, while exercising a component that is integrated deep within the system.

I also make extensive use of earned value tracking, keeping the work updated in a portfolio view I call the EVS or Earned Value Schedule. Agelists will feel right at home here, because it is similar to a burndown chart, but this is nothing new. Not only does this provide an up to date "project plan", but it is real, and accurate within 1/100th of a percent.

Here's my gripe with agile. Like communism, in theory, agile is a wonderful ideal. In practice, in execution, it is severely flawed. Sure, it humanizes software engineering, but you just can't generalize everything. There is too much of a gap between skill sets within the industry to make up for the critical architecture & design step. YES! I do advocate up front architecture. Two to 4 weeks on a significant project, less on smaller ones. There, I said it. And no, this isn't waterfall. This is iterative with the bulk of the architecture effort done before the first stage and then some spread throughout (because some of the maps will change with the terrain, but not much). The key is balance. We learned a lot from waterfall in that there exists a continuum between predictive (waterfall) and adaptive ("Agile"), but the true sweet spot is somewhere in the middle. You can't just will architecture and design away and hope that somehow it works out. Not if your talking multi-million dollar enterprise projects being worked on by ordinary people. This is an industry wide people problem, not necessarily a process or methodology problem. Elitist? Maybe, but real. This is why architects are critical. And I don't mean this stupidity about "agile architects". I mean traditional (yes, I said traditional) architects who have the passion, discipline and integrity to ensure conceptual integrity throughout a system. They live, eat and breathe this stuff. Concurrency and synchronization keeps them up at night. Interface factoring and modeling is like breathing. They do code reviews. They nag developers to name their artifacts to standard. They audit the builds. They challenge vendors, in B2B EAI scenarios- perhaps for the first time- to think about transport security and authentication boundaries perhaps for the first time and will walk out the door before they put their name on an insecure implementation. This is ethics. This is professionalism. I think its time for the industry to grow up and move into a maturing engineering discipline and I am confident I am not the only one that shares this opinion.

Here is food for thought: Agile embraces the concept of delivering value to the customer as the top priority. I wholeheartedly agree. So, why in the world should the customer pay for n iterations for the "team" to get the architecture right? How is developer's OJT being faithful to the customer? If you don't want to cheat your customer, hire a competent architect.

More ramblings here: http://rickgaribay.net/posts/292.aspx