Thursday, November 30, 2006

The cost of politics on software development

My gut feeling is that organizational politics introduce a 25% overhead on all software development projects. I strongly feel that a formula can be devised to determine the cost of politics on software development. As a matter of fact when I pondered pursuing a Ph.D. in software engineering I thought this may be a worthwhile topic to explore.

The reason this subject matter is brought up in this blog is because politics are unspoken constraints on a project.

The software architecture practitioners are finally maturing to a point where system’s quality attributes are measured in a meaningful way. For example, to express performance requirements of a certain aspect of a system an architect would create a scenario that may sound something like this: “8500 users stochastically add one item per minute to their shopping cart while the system operates in the ‘holiday shopping mode’ and the requests are processed with an average latency of 3 seconds.” A series of such scenarios provide a working reference for an architect to aid him in design a system to meet these quantifiable requirements. But how to measure the politics quality attribute?

Finding a way to measure politics in some useful and meaningful way is important, because politics present design related constraints and risks that must be mitigated. In my recent architecture analysis task I noticed that a cluster of database servers had to be two thousand miles apart from the web and application servers. When I inquired about the reasoning I was tacitly told that the two different bosses from two different datacenters wanted a stake in a high visibility project. I was assured, however, that a “very fast backbone” will provide sufficient latency between the locations.

The first step to tackling this problem is to define what one means by politics and in what units can this property be measured. Because politics are unspoken in most organizations the architect would have to decide how to represent this constraint and mitigation strategies on paper. Or can this be done on paper? Or is this something that an architect should document privately? If so, who else should be aware of this information?

Your thoughts are welcome.

- Firebrand Architect on duty: CK

Sunday, November 26, 2006

Listen. Observe. Think. Speak. When passion limits your potential.

This post is a reflection on the past behavior that landed this firebrand architect into a counterproductive situation and, temporally, put him in a penalty box. Let this be a warning to others.

Before joining this project on a full time basis I was invited to spend time evaluating the existing architecture and make recommendations before the development phase would commence. During the architecture review & analysis process I have identified a number of risks, in light of project constraints, including application’s laborious authentication and authorization requirements. To address this risk I proposed integration of a well known .NET compatible COTS software access control and authentication component.

Luckily the organization’s Enterprise Architecture (EA) has already authorized utilization of such component, and even already possessed an enterprise license for its use.

A large (eighteen person) meeting was called to discuss future placement of the solution in development, test, and production environments. Heads of various environments as well as select technical staff were present. Utilization of various EA resources, including the access control component, was on the agenda.

During the meeting the technical lead representing the component stated that only the authentication part of the component is usable within the enterprise. Naturally, I questioned why the wealth of authorization functionality is not available for the project’s use. The technical lead didn’t have a good answer and responded by saying that nobody is using it. I passionately pushed further explaining how much this project would benefit from such functionality, I went through the key features, and even volunteered to assist the tech lead with configuring the component to make the authorization functionality work.

After the meeting the product’s chief architect and I reflected on the stubbornness of the technical lead to even consider looking into utilization of the access control functionality of the component. We both felt that the technical lead took the stance of “if it’s not broken don’t touch it” and was afraid to look into our request.

Soon after the meeting the program manager requested that this firebrand architect not be taken to other “high level” meetings. It turns out that what I thought was a passionate appeal to utilize desired functionality was perceived as arrogance.

Reflecting back on this event, quite some time later, I see the obvious mistake. It’s not that the action of the appeal for desired functionality that was wrong; it’s the presentation and timing that was off. After receiving a mediocre answer from the technical lead I should have documented my case and presented it offline to the program manager who was also present at the meeting. Since this was my first meeting with these high level representatives I had no sense of the existing political situation and pre-existing issues. A smarter move would have been to listen, observe, think, and then speak – offline. This was not the first time I participated in such a high visibility meeting, but this was the first time I ran into a group of highly sensitive individuals.

Three months later, after I have regained full confidence of the program manager, I was placed into a leadership role that required me to participate in similar high visibility meetings.

The moral of the story is to listen, observe, think, and understand the environment and the political situation first; and only then to speak.

- Firebrand Architect on duty: CK

Saturday, November 25, 2006

Information sharing / architecture documentation sharing

In a survey taken by SoftwareArchitectures.com in the spring of 2006 over 65% of the visitors were most interested in seeing practical examples of software architectures. The architecture of the real world enterprise grade software systems is proprietary as it represents the competitive advantage of an organization. Therefore it is difficult to obtain software architecture documents of such caliber, which consequently hinders information sharing between software architecture professionals.

But it is still possible to share the essence of software / system architecture without compromising organizational secrets. To achieve this, first, complete anonymity of the submitter must be assured. Second, the proprietary information must be rewritten or removed (e.g. IP addresses, company name, product name, interface systems, etc.). Third, the business problem solved by a system must be generalized. Fourth, implementation specific information must be removed to a level that does not give away owner’s identity.

Who will invest their free time into doing this? Only those who want to obtain feedback on the architecture from the greater software architecture community. Caveat: constructive feedback is not likely as long as the architect and the reviewers are not utilizing the same architectural paradigm while discussing the document.

SoftwareArchitectures.com has a library of usable, complete, and well-documented software architecture design documents available for free. See the Case Studies link on the main page.

- Firebrand Architect on duty: JP

Sunday, November 19, 2006

One year of experience many times

While at the Carnegie Mellon University pursuing a Masters degree in IT - Software Engineering one of my professors said: “You can have 20 years of experience or you can have one year of experience twenty times.”

It is easy to become a victim of monotony at an organization with well established processes where all software projects move in a similar cycle. This is especially true in government, non-R&D, and non-startup organizations. Further, once an architect becomes comfortable with a given technology or a set of tools he or she may be unwilling to consider alternative approaches to solving a slightly different problem. For example, if a software architect just finished designing a complex system using the Model-View-Controller (MVC) architectural pattern with a team of experienced software engineers, he or she may automatically select the same approach for a subsequent system. But what if the subsequent system is simpler and is staffed with software engineers freshly out of college who are unable to quickly pickup more complex patterns? What if the subsequent system needs to be more loosely coupled and the Presentation-Abstraction-Control (PAC) architectural pattern is more suitable? The architect should take a step back not only to reflect on a recently designed project, but also to allocate enough time to take on the next problem from a fresh perspective.

It takes effort and discipline to make time to evaluate the past experience and learn from it. It is especially hard when an architect is not provided an opportunity by the management to reflect on the lessons learned. Architects who worked on the projects that were painful (unreasonable deadlines, poor management, lack of resources or training, etc.) are more likely to reflect on how things should have been different. It’s true that working on a project in distress, with no chance for success, can be a good learning experience, but only as long as one vows to remember the mistakes of the past so that they are not repeated in the future.

It is easy to repeat the mistakes of the past, complain during the process, and then do nothing about it at the end. That’s how one acquires 1 year of (painful) experience twenty times. An architect is likely to reflect on the past experience if it was not a pleasant one (e.g. a failed project), so it takes effort and discipline to learn from the mistakes on the mediocre project or to see ways to improve from a successful project.

- Firebrand Architect on duty: CK

Tuesday, November 14, 2006

Different paths of ascension

It is common to hear software architects draw parallels between physical building design and software system design. The analogy suffices to a certain extent and is a subject of a different post. However, the author of this post is interested in the path of ascensions between the two professions.

The contrasting difference is that a software architect almost always starts as a programmer learning & testing the composition of the building blocks. A building architect starts his or her path on a high level - normally without spending time working as a bricklayer at a construction site. The curriculum of a building architect includes courses on structure, design, environment, and history followed by hands-on learning of CAD tools that support the work.

Now looking at a standard computer science curriculum one would notice that very few courses actually cover the concept of software architecture. Such courses are taught on a graduate level and are normally part of a graduate level software engineering degree.

Is it possible or will it ever be possible to become an effective software architect without the rigorous understanding of the computer science discipline? Will the separation of concerns in the building industry (e.g. structural engineers, plumbing, architects, etc.) transfer over to the software engineering industry with pure software architecture companies? Thoughts in a subsequent post.

Constantin Kostenko
Firebrand Architect®
SoftwareArchitectures.com
FirebrandArchitect.com

Wednesday, November 08, 2006

When you’re positioned for a sprint

There are some projects that run like a well oiled machine. The necessary resources are available, the management fully supports the initiative (i.e. fully funded), all team members understand and faithfully execute their roles, and the planned software system victoriously emerges with tolerable defects.

And then there are projects that are dysfunctional. For example, team members are forced to develop software without receiving any training on the chosen technology, schedules are created with little consideration for the envisioned architecture, and fundamental project management practices are not exercised… There are many case studies (examples available per request) documenting eventual demise of such projects unless a champion rises to the challenge and takes the lead. As an architect, especially if you’re an aspiring architect within your community, you may be provided with an opportunity, or forced, to take the challenge of saving the day.

As an architect with leadership responsibility the first thing you should do is to assess who can help you achieve your mission and who will waste your time. In a typical non-commercial project, there will be people who don’t provide any real value to the progress, but who are protected by management and have to be part of the team. In case of consulting organizations such people are consultants who are temporarily helping a project until their next engagements starts. In government organizations such people may be employees who do a mediocre job and operate on a strict 8 hour mentality versus on the demands of the current lifecycle phase of the project.

The next step is to identify the core features of the system and to ensure that you, as the architect, understand the whole picture. At this point of time you won’t have time to document the as-is and to-be states of the system, but you need to come up with some means to explain the architecture to the core group of people who will help you build the system. The core team needs to have a common understanding of the problem and the path to achieve the objective. The roles and responsibilities must be clearly defined, for example, in a case of a web-based system, the expertise among team members may be broken down by logical tiers (e.g. front end, business logic, persistence).

With core team in place you’ll be functioning as the lubricant during intense integration and string testing periods. You will also be forced to find ways to entertain non-contributors so that they don’t interfere with the execution of the vision. You may be required to defend selected solution for the sake of stability. Request that non-contributors think about the features of the next phase of the product, ask them to investigate some high level problem that the current release experiences, or in a worst case scenario ask them solve the Rubik’s Cube problem.

Living up to the champion status is hard since by now the existence of the project may be in jeopardy. However success can be materialized if you have a vision for near term, able to communicate the selected architecture, and able to identify and motivate a small team of core developers. Don’t forget that the exit criteria for saving the day (i.e. scope of end product and your responsibilities during the sprint) must be established before you begin, otherwise the sprint will turn into a marathon and you shall become ineffective.

Monday, October 30, 2006

Wiki for a software architecture document

Wikipedia has revolutionized the way information is generated, edited, and made accessible to various Internet audiences. Wikipedia, a publicly edited encyclopedia, is based on a concept of wiki where any user can edit any information at any time. This ad hoc approach sounds like a bad idea, but it may be useful for designing and documenting software architecture – democratically. The primary goal of the wiki solution is to make the system’s software architecture document a living document so that it can accessed and used effectively by all system stakeholders.

Compared with the industry’s “standard” approach to documenting software architectures (advanced word processor [MS Word], drawing software [Visio], and source control [CVS]) the wiki approach provides the following benefits. The document is accessible via any web browser, transclusion allows for repetition of information defined earlier in a document, source control and mechanism for merging & discussing changes are built-in. The wiki approach, however, suffers from lack of built-in image editing functionality. This means frequent copying and pasting from another image editor, which may or may not be available to all wiki user base.

The wiki approach is also a paradigm shift to managing software architecture. In an ordinary hierarchical organization software architecture design is done by a single person or by a very small group of software architects. The wiki approach levels the hierarchy and allows any team member to make changes. From one point of view the allocation of ownership may empower team members to contribute, but on the other side not a single person is accountable for the whole design. This may pose a challenge to a more traditional accountability structures and to people with fragile egos.

A technical note (TN) released by the Software Engineering Institute (SEI) shares the experience of using a wiki web-based tool by a small team of software engineers while working on a project for the Siemens corporation. For a more detailed discussion please see the TN page or TN in PDF.

Thursday, October 26, 2006

Deployment view: documenting the essence of the target environment


Sufficiently documenting software design, including source code, is essential. But sufficiently documenting fundamental software architecture views is critical. There are many views to consider during design and analysis of a system. This time a basic deployment view is discussed.

A simplified deployment view is a high level tangible mapping between system elements (e.g. messaging queue) and physical hardware spread across network topography.

One of SoftwareArchitectures.com colleagues, PJ, observed the following situation at his current client engagement.

The task of overseeing the deployment of a significant iterative release into the test environment was assigned to the associate architect of a system. The client’s environment features institutionalized configuration management policies that can be circumvented when pushed, but the deployment manager has no access the system directly. The test environment administrator, who physically controls the test environment, is only available by phone and email.

The build and deployment process went smoothly across most of the tiers of the test environment with the exception of the application tier. Specifically the configuration of the messaging framework and the business logic residing in this tier was not deployed successfully.

Having no access to the environment where an unstable release is being deployed is challenging. What made this situation dire is lack of usable deployment documentation. Yes, there were some back and forth emails containing information about various machines across environments, but none of the information could be readily used as a reference. This ultimately resulted in the following fundamental problem: people involved in the process of configuring and troubleshooting the environment did not have a shared view of the solution. The deployment timeframe stretched from that afternoon to late evening and then extended onto the next few days and eventually weeks until full smoke test could be run.

In an ad hoc environment, no matter how big or small, it is the responsibility of the architect to always have a readily available deployment view in the back of his pocket.

Software architects and designers may feel lax when it comes to recording decisions on the daily basis: a conscious and a well thought design may materialize with little documentation overtime. Not surprisingly the deployment view is the last thing that comes to mind in a deadline oriented ad hoc environment. But it’s the deployment view that represents architect’s view of a target environment and lubricates communication channels between the design and development team and the colleagues who manage the test environment. Like a fire escape plan on the door of a hotel room a basic deployment view must show the components of a system being deployed specific to the target environment.

Tuesday, October 03, 2006

This blog: what's in it for you?

The motive behind this blog is to expose the human aspects of the software architecture discipline and how they fit into the overall software development process. Due to purposeful omission of technology specific details the blog targets programmers, developers, project managers, students, interns, junior and seasoned software architects alike.

Some of the observations and conclusions may seem obvious and familiar as they have been documented in classical software engineering books such as Fred P. Brooks’s The Mythical Man-Month: Essays on Software Engineering, Roger S. Pressman’s Software Engineering: A Practitioner’s Approach, and Ian Sommerville’s Software Engineering. Further, some of the problems & solutions may not seem to be architectural in nature at all. That’s precisely the point. The goal of the prose is to expose the obvious and to prod the reader to ask oneself if a given scenario or a point of view is applicable to own situation. If just one reader sees personal value in pursuing a solution, or asking subsequent questions, then the objective of the blog post has been fulfilled.

The architecture of a software system is only as good as its implementation. In a fictional world an architect may pause the time and retreat from the real world to come up with the perfect architecture. The software engineering world in the trenches is far from that. A practicing architect is subject to tangible forces such as budgets, schedules, resources, and intangible forces such as “the bus problem” and politics. It is our hope that through concrete solutions and through philosophical discussion the reader will have appropriate ammunition tackle the problems are relevant to his or her role in the software development lifecycle.

Sunday, October 01, 2006

What is firebrand?

DESCRIPTION OF THE FIREBRAND

“The firebrand is the person who recognizes what is natural, what is organic, what is alive and vital in life, the person who dares to live, to be, and to create, often in the face of interference, rejection, deceit, and betrayal. The firebrand is a burning ember, life that is in each of us and that provides the spark and energy to speak against what distorts, hides, and denies our being and truth. It is that which awakens within us, when we must declare our independence or when we discover a new formula for living. It is the path that enables us to participate in the mystery of creation, uniquely and individually. The firebrand expresses her- or himself in two basic ways: as the torch that lights up the darkness, and the as the carrier of the torch, throwing light into the darkness, and often disturbing complacency and brewing trouble. Being a firebrand is a way of raising temperatures and creating conflict, turbulence, and dissension.

The motive of the firebrand is not to attack or destroy others but to bring to light a basic truth, to take a stand, and to declare and own who one is, especially in the face of perceived violations of one’s values and rights and interferences with one’s goals, purposes, and meanings.

In relation to one’s self, the firebrand engages in reflection and self-dialogue that evokes awareness of ideas, projects, and goals, insights, into one’s deviance from others, and particularly from the mainstream people. The firebrand chooses to be different when being different represents a truth, when being different guides the fulfillment of basic human values and actualization of one’s potentials.

In relation to others, the firebrand seeks to maintain what is unique and distinctive, what will enrich a relationship and keep it alive in fundamental ways. The firebrand avoids roles, categories, classifications, hierarchies, fixed routines, and practices but rather seeks to create rituals, searches for new rhythms and connections with others, keeps secrets and confidences, and engages in conflicts and intimacies when these are true to experience, when these are ways of enhancing life. The firebrand is concerned with being open, honest, adventurous, and creative. If none of these processes are viable, the firebrand terminates the activity or relationship and moves on.”

- Moustakas, Clark. (1995). “Being-In, Being-For, Being-With” pp. 5-6.

Thursday, September 07, 2006

Blockchain learning path for Enterprise Software colleagues

I wrote this post to document my learning path of blockchain concepts and Ethereum technologies while keeping my “new to blockchain” collea...