In a society struggling with deep-rooted structural inequalities, are metrics good enough?
Last year, we visited Channapatna, a rural town about 100 km from Bangalore, to work with Maya, an NGO that works with local community health workers. In one of the conversations about women’s mental health in the community, one health care worker stated: “Washing Machines are one of the worst things that have happened to the women here”. I was surprised to hear that statement. As a bachelor, I could not imagine a single day without it, and yet, here is someone who finds it to be the bane of the village. On inquiring further, this is what she had to say :
“Previously, our women would finish their morning chores, get together to visit a nearby lake and wash their clothes. It was the only time where the women could be completely free of judgement, laughing and sharing their lives with each other, away from the glances of the men in the village. This was their space. But now with washing machines, none of the women ever leave their houses. The women do not speak anymore”.
My intent here is to not romanticize the gendered labor practice of manually washing clothes or “villain”-ise the washing machine, but to recognize how the introduction of artefacts into a community alters social dynamics and often reinforces existing power structures, the essence of which is not captured by the reductive-ness of a number.
Having recently started working as a Senior Product Consultant with a Big 4 Firm, I hear many conversations on defining success metrics to help measure impact. The field has plenty of resources, guides and frameworks for crafting the “right” success measures. (Looking at you Google HEART framework). Yet, there are many instances where technological solutions have created more problems in society than organizations intended to address. For example, the Facebook induced genocide in Myanmar or the struggles of Swiggy delivery workers in India or the more recent Zoom-bombing incidents and vaccine denial. Are these successful products with high DAU/MAU? Yes. Have they benefited all the communities they intended to serve? No.
While companies promised utopia, we are currently living a technological nightmare.
It is easy to argue that none of the original makers of the software could anticipate such misplaced use of technology. After all, Zuckerburg’s original intention was “to give people the power to build community and bring the world closer together.” And yet, it has played a pivotal role in fueling a humanitarian crisis and causing significant disruption to democratic values in several others, including the USA. With tech regulations yet to catch up to the blinding pace of software development, founders wash their hands off the mess they make, citing ignorance. We cannot continue to look the other way while the products we make wreak havoc on a complex society, regardless of our intentions.
“Product Metrics” or “Success Metrics” are one way in which organizations try to make sense of the impact they create. Typical metrics include Daily Active Users, Monthly Active Users, Conversion Rate, Churn Rate. The abstraction of impact into quantifiable metrics enables businesses to track and take decisions on the course of action. Unfortunately, this is an oversimplification of the real-world contexts where products, services and people are entangled. They present a static picture of people’s experiences which are inherently dynamic, constantly evolving and thus remove any opportunity for negotiation, reiteration, revision of their positions.Metrics alone cannot tell you who uses these products and towards what intentions. What starts as a people-centered product development quickly shifts to a bottom line-centered approach. The continual pursuit of larger numbers on the screen eclipses the original intentions, without due regard to what actual human activities lead to these numbers.
Why does this happen? Should we not assume best intentions and expect people will use the designed platforms in the way it was intended? The question of who Designs and who benefits from Design is necessary to consider here. Herbert Simon, in his book “Sciences of the Artificial”, outlines Design as :
Everyone designs who devises courses of action aimed at changing existing situations into preferred ones. The intellectual activity that produces material artefacts is no different from the one that prescribes remedies for a sick patient or the one that devises a new sales plan for a company or a social welfare policy for a state.
Simon’s definition of Design is broad yet accurate. We dream of living in our fantasies of what the world could be. When corporations control the design process (who are the actors, what ideas get chosen, how they are implemented and what gets narrated as success), they monopolize the vision of a preferred future. The future is not collectively defined and instead imposed onto people and cultures, which results in significant conflict. Success metrics further contribute to this hegemony, often used to re-emphasize the narrative of how tech companies alleviated yet another humanitarian crisis. What metrics cannot capture is the inherent political tension that an artefact creates between the various actors — a continuous struggle to impose a specific worldview.
What metrics cannot capture is the inherent political tension an artefact creates between various actors —a continuous struggle to impose a specific worldview.
We solve problems that are visible to us. Metrics trick us into believing that attributes like engagement, retention and growth are the problems worth solving. When viewing the world with such tunnel vision, organizations find themselves in a rabbit hole, often far too late until they can acknowledge or recognize the tangible impact products & services have in the complex realities of their “users”. Given the uncertainty of predicting the future, how can we be more conscious of the products we create?
I do not wish to propose solutions for how this can be achieved, as I do not believe there is a single way to address the complexity of the topic. I will instead conclude with some thoughts on how we ought to extend conversations about product consequences in our workplaces :
- Look beyond metrics and the user to craft your definition of success as a product. Identify the various actors involved in your ecosystem and learn what success would look like for them. Co-defining success can help shift from a zero-sum mindset to creating a win-win situation for all people/entities involved.
- Create spaces within your organization to understand the consequences of “un/re-designing” the old system. When an artefact is introduced in the world, it induces rules and ways of being, not just between the object and the person, but also in relationships between various actors linked by the object. Develop frameworks to interrogate these new social structures and learn how they benefit or harm certain people. Tarot Cards in Tech is a thoughtful and fun way to introduce these topics for discussion.
- Develop interdisciplinary design practices with extensive post launch research drawing from and not restricted to anthropology, sociology, systems design and other relevant fields that can help decipher the rapidly changing interplay between technology, people and communities. This could allow for early reorganizing of social structures before inequalities become embedded as standardized accepted rituals.
As product managers, we do not swear by a Hippocratic oath, however we must act responsibly and hold ourselves accountable when given the power to Design structures that can make or break lives.
If you wish to engage in these conversations, you can find me on Twitter!
