Jim Harris just wrote an excellent post titled The Point of View Paradox on the importance of considering how multiple points of view can lead to multiple perceptions of truth.
The basic thesis is that the background, history, and perceptions we bring to a situation, any situation, will impact what we perceive as "truth" in that moment.
What does this have to do with data quality? Or 'technical' projects of any type for that matter? More than 15 years technology experience tell me it's a critical concept. Some examples:
Software development: How many times have we joked that users say "it's just what we asked for but not what we wanted." How often do developers and users read the same specification document and build a completely different mental image? Um, nearly every time! Successful development projects necessarily include checkpoints to ensure all parties' "truth" re what's being built have a chance to sync up.
DBA: Users want all the data available, all the time, at the push of a button. Administration wants to meet business requirements at the minimal cost. Security officers (and auditors) want thorough separation of data access based on business roles. DBAs want to ensure stable systems with adequate hardware and staffing for backups, redundancy, and maintenance. All important priorities -- yet each priority impacts the ability to meet the other priorities. (Assuming we don't have unlimited funds....)
Data Quality: The U.S. only has 50 states. I know this to be a truth. I also know of a system that has 51 states -- one of which is MassHampshire. Why? Because the company has an employee working in New Hampshire while living in Massachusetts. The system's "truth" of 51 states reflected its need to manage a cross-border set of regulations. When planning the data migration to a new ERP system, what originally appeared to be bad data was actually a use case that needed to be addressed in the new system's configuration.
Jim's post is worth taking a few moments to internalize. We don't have to agree with another's "truth" or point of view, but we generally do best when we at least make an attempt to understand the logic behind it. Even in technology.
5 comments:
Excellent Beth,
You hit the nail right on the head. So many projects and so many views / opinions.
Thanks for sharing.
Great post Beth,
Thanks for referencing and excellently extending the commentary about the point of view paradox.
When asked what truly makes the difference on successful projects, I always respond by emphasizing communication and collaboration. Many people frown at me as if I was avoiding their question because these factors are often taken as obvious.
However, communication and collaboration are typically under-prioritized because everyone assumes the entire team is on the same page since everyone appears to be agreeing with the documented requirements and specifications. But like you said, their "agreement" is very likely based on a completely different mental image.
As Stephen Covey wrote:
"We see the world not as it is, but as we are."
Of course, this doesn't mean that communication and collaboration are futile efforts, it just means we need to realize our inherent biases and like you said, plan for regular checkpoints to ensure all parties' "truth" has a chance to sync up.
Thanks and Best Regards,
Jim
Beth,
Isn't your 51 states example rather an example of bad data(base) management than different versions of the truth? The system with 51 states is trying to use one piece of data to represent two realities (where does a person live; where does a person work)?
Equally, why would a system limit the number of answers to a state in which a person lives to one? People may live in buildings that straddle state lines. In fact, people live in houses that straddle country boundaries ... I can give good examples on the Dutch/Belgian border, where the inhabitants have two addresses and can use either depending on the situation.
Graham, I agree that the proper scenario is to design the system in such a manner that boundaries may be straddled. Indeed, the new ERP system (to which the data was migrated) was configured to allow such scenarios and the organization was able to decommission MassHampshire from existence.
As to how the state came into existence: the older system was not amenable to change (i.e. the costs to rework the system were deemed prohibitive financial when weighed against the gain) and thus this approach was chosen.
Would MassHampshire have been my recommendation? Heavens no. But then I wasn't the person responsible for the IT budget in that organization, just the consultant who came in years later to help with the migration.
My point was less about the quality of their solution -- we're agreed that it wasn't optimal -- but that data which at first glance appears to be utterly erroneous can potentially represent a business use case that needs to be assessed.
Ah, gotcha! Thanks for the clarification.
We tend to build systems to match what we know or to fit most data. We then have to find creative solutions to all the exceptions - and there will always be many. In trying to fit reality into the straight-jacket of computer systems we will always end up with data quality issues .... as with your example :-)
Post a Comment