1. Technology and Product as Partner Organizations
I’m a technologist first but I’d truly be bored if that was the only hat I wore. I’m a product-minded organisation builder that can conceive a technology solution given a well-stated business goal, and as an analyst can also help elicit a business solution given a set of vaguely stated organisational challenges stated in terms of a desired outcome. Without my product hat I’d be a lot less valuable because I’d not be able to see this from end to end, and I’d certainly feel a lot less challenged.
I believe the Product function provides the most pure, concise expression of business intent, and staffing a solid Product function is critical to success in execution. More critically though, the relationship between Product and Technology must be well defined and well oiled, and as a technology executive you must support them; they are your customer. Issues in this relationship are wholesale the number one problem I’ve witnessed to varying degrees at almost every company at which I’ve worked, either as an employee, management consultant, or executive. Although there are proven patterns that make these problems eminently solvable, issues here can be some of the most intractable as they’re often organisational and non-trivial. It’s a mindset concern; not everyone shares the same mindset, priorities vary between functions, and some people are more easily convinced than others.
Issues here pervade the whole software development life cycle. Even with strong Product and Technology organisations, poorly defined expectations here are a root issue; it creates issues all the way from engineering efficiencies to quality and customer support. An organisation will never reach maturity until this is addressed. It’s for this reason that even as a technologist, a significant portion of my focus has historically been advocating for and helping structure this relationship, be it as an owner of the product organisation, or as a partner to one.
On the more well-oiled end of the scale, I’ve seen companies that completely understood the value of a solid Product organisation, had put good money into staffing it, could get some meaningful work done, and yet sometimes still struggled translating vision into execution. This most often came down to difficulty concisely articulating their real needs; problem statements tended toward the overly broad, user journeys were not always completely thought through and hard to visualize, and sometimes understanding of the relevant data was vague. In the absence of this, there existed a vacuum in those organisations. Technology therefore sometimes struggled to write meaningful code, and even more critically, those organisations risked rolling out a product that didn’t necessarily even align to true user needs in all areas. This is no way to compete. It doesn’t have to be like this; as a technology executive you can do a lot to support Product by structuring the way in which your teams need information.
At the other end of the spectrum, and more prevalent in technology-enabled companies than technology providers, this function is under-developed, represented instead by part or full-time business analysts trying hard to extract requirements from operational business users who don’t have time to be providing their input, and who in any event may not have practiced Product mindsets. In this case, frustrating to most of course is that everyone realizes a solid technology-enabled process could solve their bandwidth and data redundancy problems and allow the business to focus on more value-driven activities - but this is just where a lot of companies are. This is a broader organisational problem.
1.1. Product Orgs in Technology Enabled vs Technology Provider Companies
1.1.1. Technology Provider Organisations
Experientially, I’ve found the challenges of product organisations directly correlate to the role of technology in the organisation. Organisations selling technology (B2B SaaS products or platforms, for example) are started by business visionaries with a well understood value proposition, they understand how their idea distinguishes them, they understand the minimum product that will meet demand, drive value, and generate traction, and all else being equal, they have the potential to drive an engineering initiative with laser sharp focus. These are outward looking product organisations because they’re market and opportunity driven.
Practically ubiquitous among rapidly growing start-ups is the challenge of Product balancing the roadmap to original value proposition, “ility”-enabling work, and emerging needs of customers; staying true to selling software as a service and not product customization as a service is important. Of course, the balance chosen will vary with the personality and risk appetite of the company. There is no right nor wrong approach here - it’s just a matter of costs, risks and opportunities, and the tailoring of the company’s mindset and value chains to be in sympathy with those. Especially as it relates to the “ilities”, managing this balance takes a skilled partnership between Technology and Product, and lack of effective negotiation here can cause unharmonious relationships and a beleaguered product. I’ve spent a lot of time on how to sell technical debt initiatives already in this essay and won’t go into that any further here.
1.1.2. Technology Enabled Organisations
The imperatives for technology-enabled companies that don’t sell technology as their product or service are different. Their technology initiatives tend to meet the needs of internal users in the name of automation, simplified workflows, and perhaps making their business-critical data deduplicated and more readily available, and expressions of patterns in the data more meaningful. The challenge for these types of organisations is one of being able to conceive a holistic organisation-wide solution and being good at translating ideas into executable items, and they need to be driven by a different kind of business visionary. This is an inward-looking product organisation.
Challenging for the internal product organisation in this scenario is that problems stated in terms of automation and efficiency tend to be very broad, and they are not approached from the perspective of an invoking, clear product idea; they need a massive amount of analysis to work out what problems you really have and need to solve. Even in a situation where a business analysis effort yields useful information from business users, it is often stated through the lens of current legacy process, competing philosophies, and differing views and uses of data. This is to be expected. Much maligned users, devoid of adequate tooling and automation, are required to perform mental gymnastics and create bespoke, disconnected solutions just to get their jobs done. Asked for solution ideas they understandingly tend toward self-centric views of their own needs, and taken as a stated, can cause tunnel-vision while solutioning. Product's challenge then is to negotiate that holistic solution and convince all stakeholders that it can satisfy their needs, even if it doesn't present in the way they might have expected.
It is especially true in this case that a firm institutional commitment to and support of the Product function is required to break cross-functional deadlocks, to create the right culture from top to bottom, to start their evolution, and thereafter to accelerate. Failure to propagate a continual improvement mindset across the whole organisation can cause any implemented output to incompletely solve the overall problem, leaving some users struggling with new activities that don’t efficiently contribute to the value chain any more than they did before. The responsibility for setting that mindset starts at the top if Product is to succeed.
1.2. Customer-Defined Functionalities and Impact on Platform Sustainability
Naturally the more immediate imperatives for technology provider companies are driven less by internal pressures and more by customer pressures. In a B2C scenario the company has a mind to unilaterally implement their ideas as they see them, with perhaps little major directional influence from customers (save maybe A and B testing and controlled customer feedback mechanisms). The company may choose to implement any suggestions they receive or not - with little consequence and change to customer loyalty because they can play a numbers game and basically just force it on their consumers. As long as their value proposition remains relevant, their implementation competitive, and the consumer experience positive, they may experience little turnover. The customer acquisition and cancellation costs are relatively low. Note here that for the sake of my definition of B2C I include companies that actually do have business customers, but are providing less core commodity type services such as task management and office tooling, and not their main line-of-business applications.
In a B2B company the customer dynamic tends to be different, especially in a growing start-up. The functionalities provided tend to be the more complex line-of-business applications that are more deeply rooted within their core operations and value chains. The customer may still make use of several commodity online offerings to streamline their work, but they can be more easily changed out based on simpler considerations of functionality and cost. Line-of-business applications however have a very high exit cost, and they represent a real commitment to the service provider. As a result, enterprise customers may be less willing to accept a product as-is if it introduces difficulties executing their various value streams or meeting regulatory requirements (GDPR for example). Especially when they know that as a service provider you are a start-up, the customer may assert that they need the software to work a certain way other than as provided to land a sale. On the other hand, as a fully established large provider, customers know they have little leverage. Either way, if the product is to succeed, Sales and Product need to get talking right away to understand the relevance and implications of any requests they do receive.
1.2.1. Customers Are Often Poor at Articulating Their Own Needs
I’ve found that customers are commonly unable to clearly articulate their needs in a visionary fashion – problems yes, but needs - no. They don’t have a product mindset and are likely struggling with process and data fragmentation and duplication issues. They will often phrase their “needs” in the context of their existing system’s structure, approach, and limitations, and thus envision their needs in output terms instead of outcome terms. If the company is prepared to extend the system to assist, a very real danger exists that a product’s functions will end up being defined in terms of that customer’s needs through solely their lens.
This outcome may not be so costly when a singular customer exists (such as in the case of an internal customer), but in a case where you have a SaaS platform or are attempting to monetize something written for a singular case but that really has potentially broader application, it creates a problem because the product’s behaviours will have been mostly focused around a single mindset. Forcing other customers into that mindset probably won’t work, and save work done by Product to define a broader model, in the hands of the engineers they’ll “extend” the product or platform as a configuration-driven system to provide flexibility that can accommodate multiple behaviours, or just outright implement changed logic and box the product into a corner.
1.2.2. Customisation Creates Complexity
Adding configuration makes sense, up to a point, but taken too far it quickly adds complexity. The system now demands a higher degree of training of internal users to configure it. With your newly configurable system there will possibly be a protracted customer requirements process. Gathering configuration requirements can reveal needs for new configuration options the system doesn’t currently support, and because you’ve done it before you’ll happily do it again. Complexity trends upward and the problem is exacerbated.
Quickly, internal users and engineers lose track of how to configure the system; some configuration options may be incompatible, causing edge cases to be revealed that were not planned for, and these may or may not be well enforced in code or well documented. Because a customer’s business logic is not purely expressed in code anymore as much as configuration, the now non-deterministic nature of the system makes testing an order of magnitude more difficult to do truly comprehensively, both for the code itself and for customer configurations. It forces testing to consider every combination of settings (compatible or otherwise) and scenarios are invariably going to be missed. I have seen this in almost every SaaS start-up I’ve ever been associated with or integrated to. Again, this is not necessarily wrong – it’s a balance of cost, risk, and opportunity, and business leaders know this – but know that the downstream cost to platform sustainability is really high, and one day a business leader is going to be asking you as the technology leader the uncomfortable question of how you got them there.
1.2.3. Not Every Requirement is a Requirement – Question Everything
Ultimately, customers fundamentally want to be outcome-driven and not output-driven, even if they’re specifying requirements in terms of outputs. If you can convince a customer that they can attain the outcomes they are after without being married to how they get there, you can create some space to implement “requirements” in way that makes sense for the platform. If a customer is insistent on a particular approach you need a way to get them aligned with a general alternative that gets them the outcome they’re after. Sometimes regulatory considerations differ across national, state, or regional geographies that force your hand and you may have little choice but to implement them anyway, but generally this is very possible to achieve.
Simply – submit everything to the five whys process, and dig into everything:
- What outcomes are you trying to create with your processes?
- Where does your data come from?
- Who needs what data, and who should be allowed to see it?
- Why do they need it and when?
- What decisions are the data driving?
- What’s the best way to present data (this is often a place where the most opinions are provided and objections need to be overcome)
- Does the data drive higher levels of business intelligence, and how?
- Are your workflows optimal and how much change can you accept?
In order to succeed at this, these conversations need to be started very early on in a customer proposal process. Sales is often not equipped to do this, and Product and Professional Services need to be engaged. If throughout this process a new outcome is required that the product doesn’t currently envision, or if an outcome is planned for but is further right on roadmap, at least now you know what you’re truly working toward. You’ve hopefully gotten to the root goal and removed the noise that might have been masking the true need, and you’ve reduced the risk of surprises appearing down the road.
This is a simple statement that belies the true difficulty of the task it describes. This is hard. Dealing with the consequences of not doing this though is harder. This requires an institutional commitment at all levels from which the organisation cannot stray, and should be one of the main enabling goals of a technology executive to establish.
1.2.4. Allow Product to Be the Enforcer
Customer customization needs can quickly displace your product roadmap, so if the business elects to do that at all, Product has to make sure it’s providing real value in a way compatible with the product vision. If they don’t, the customers are really fulfilling the Product function, and the Product function can degrade to that of a project management function.
At the end of the day it really comes down to this; no matter what the model, it's crucial to success that a creative product-minded group of people look at the problem set as “what it could become”, not “what it is”, or “what you’re being asked for”. Don’t just throw the problem statement at your engineers and expect a relevant output, and don’t just blindly follow the business stakeholders’ assertions and perspectives and expect an extensible output. Engineers’ nature is to solve problems and given to a team without controls they may not implement what is best for the product. Product themselves must decide how much complexity they’re prepared to accept, how much drift from their standard model they’re prepared to accept, and how far they’re prepared to go to meet a customer’s needs without affecting platform sustainability.
1.3. Failing Late with User Acceptance Testing
For the purposes of this discussion, I refer to User acceptance testing (UAT) as the waterfall, downstream specification-based activity that ostensibly ensures that what is expected is what was delivered. I exclude from this discussion any UAT of customer-specific configuration activities that might be performed on behalf of a customer before they go live. I’ve seen this in all B2B businesses within which I’ve worked in one form or another.
You would be surprised how many companies claim to be agile yet still have an indoctrinated UAT practice.1 For me, this is tantamount to a built-in assumption that features can be legitimately written the wrong way for a very long time before problems are detected. Naturally this results in massive pressure waves all up and down the leadership hierarchy, across the release process, and into the customer experience. The mere presence of right-loaded UAT of development efforts in your process is a sign that something is seriously wrong with the relationship between Technology and Product.
A disclaimer: I’m an Agilist, but I’m not here as Agile zealot, I am not here to reiterate the Agile manifesto, and I am not here to browbeat Waterfall (mostly). This is a product of the organisation as a whole not being fully bought into an Agile mindset. I only want to stay here long enough though to continue to make my point about the importance of the Product organisation and its place in the SDLC. Call your process what you will, but if engineering is working under this expectation, it makes it hard for Product to course correct the development effort. Written and managed incrementally, Product can routinely check in on functionality and much lessen the chance of veering off-course. It’s for this reason that Product needs to be truly integrated as a key component of your development team, as Scrum lays out. Check early and check often.
The only jab I’ll take at Waterfall is that it is really hard to do well, but if you can make it work – all the power to you. It is predicated on the assumption that those writing a specification have thought every single aspect of a problem and designed a truly comprehensive business solution to speak to it. It’s a pretty bold team that can come out of an extended project planning phase for a long-term project believing they’ve truly achieved that. This is not to say that it cannot work, but in 20+ years in the professional software industry I’ve never seen it work. Issues always reveal themselves, and some are directionally life changing. This is a lot of risk for a company to adopt.
In closing, as a technology executive it’s crucial that the Product function be seen as a partner, and if your relationship with them is not well-oiled, it’s probably the first thing you will need to fix to get your department heading in the right direction.
Next Up
With the exception of my personal introduction, most of the content has been outwardly focused - relative to oneself as an executive - in terms of understanding others, relationships, risks, process, and more. At the end of the day, however, your ability to support your staff in understanding and achieving goals in these areas is only going to be as good as your ability to take care of yourself and sustain a high level of function over the long haul. It's for this reason that I feel it desirable to discuss some of my own realisations that helped me in staying true to myself and developing stamina.
You can find the final blog post in this series titled "Management of Self" here.
1. I even had a client for which there was such distrust between Engineering and the business that they had two completely separate QA functions – one in each department – with entirely unrelated test cases.
No comments:
Post a Comment