Software Quality: A Definition

What software quality really means, why it has two interconnected faces, and how proven practices help teams achieve both

Here's a question that haunts every software team: What does "quality" actually mean?

Ask a product manager, and they'll talk about user satisfaction and business outcomes. Ask a developer, and they'll mention clean code and maintainable architecture. Ask a user, and they just want something that works reliably and doesn't frustrate them.

They're all right—and that's exactly the problem. Quality isn't one thing. It's a multifaceted concept that encompasses both what users experience and how systems are built to deliver that experience sustainably over time.

The teams that truly understand quality don't just ship better software—they build systems that can evolve, adapt, and improve continuously. But getting there requires rethinking what quality means from the ground up.

The Deming Revolution: Quality as a System Outcome

W. Edwards Deming revolutionised manufacturing with a simple but profound insight: "One cannot inspect quality into a product." Quality doesn't happen at the end of the assembly line—it emerges from the entire system.

This applies even more to software. We can't test quality into code that's fundamentally broken. We can't patch over architectural rot with better deployment scripts. Quality flows from a reliable, managed system that reduces variation and gets feedback fast.

It's not just about testing at the end, but designing everything—architecture, development pipelines, team practices—to keep quality in from the start.

Whose Requirements Matter? The Relativity of Quality

Philip Crosby gave us the classic definition: "Quality means conformance to requirements." Clean, measurable, actionable.

But Gerald Weinberg pushed deeper: Whose requirements?

Quality is always relative to someone's expectations. A banking system and a social media app have completely different quality requirements. Even within the same product, different stakeholders care about different things:

• Users want reliability, speed, and intuitive interfaces

• Product managers want features that drive business metrics

• Developers want maintainable, testable code

• Operations want systems that deploy reliably and don't break at 3 AM

Without explicitly defining who our "user" is for each quality decision, "meeting requirements" becomes hollow and untestable.

The Two Faces: External vs Internal Quality

This brings us to the crucial distinction that most teams miss:

External Quality: What Users Experience

External quality covers everything the end-user experiences:

• Reliability: Does it work when I need it?

• Performance: Is it fast enough?

• Usability: Can I figure out how to use it?

• Security: Does it protect my data?

• Functionality: Does it do what it promises?

This is Juran's "fitness for use" concept. The product doing what it says on the tin.

Internal Quality: How the System Evolves

Internal quality is about the codebase, architecture, and development practices:

• Maintainability: Can we modify it safely?

• Testability: Can we verify changes quickly?

• Modularity: Are components properly separated?

• Readability: Can new team members understand it?

• Architecture: Does the design support future needs?

As Dave Farley argues, fast feedback loops are critical here. Tests that return within minutes (not hours) are essential for sustainable development. Long-running tests don't help because we don't learn quickly, so quality decays.

The Hidden Connection

Here's what most teams miss: internal and external quality are deeply interconnected.

Without internal quality, external quality inevitably decays. That messy codebase becomes impossible to change safely. Bug fixes introduce new bugs. Adding features becomes exponentially harder.

But internal quality without external quality is just expensive craftsmanship. Beautiful code that doesn't solve user problems is worthless.

The best teams understand this symbiosis and invest in both simultaneously.

A Framework for Thinking About Quality

At the time of writing, the ISO/IEC 25002:2024 standard gives us a structured way to think about this:

Internal Metrics: Static code measures, modularity, complexity, maintainability indices External Metrics: Runtime correctness, performance, reliability
Quality-in-Use: Real-world user satisfaction and business outcomes

Each level builds on the one below. We can't fake external quality with good internal practices alone, but we can't sustain external quality without them.

The Engineering Mindset: Measure What Matters

Tom Gilb's approach to software engineering economics pushes us to make quality measurable. Instead of vague goals like "make it reliable," define specific targets:

• Availability: 99.9% uptime during business hours

• Performance: Page loads under 2 seconds for 95% of requests

• Maintainability: New features can be implemented in under 5 days on average

• Defect rate: Less than 1 critical bug per 1000 lines of new code

The Vanity Metrics Trap

Before diving deeper into improvement strategies, we need to address a critical pitfall: vanity metrics. These are measurements that look impressive but don't actually correlate with quality outcomes.

Test coverage is the classic example. Teams obsess over hitting 90% or 95% coverage, but high coverage doesn't guarantee quality—it just means our tests exercise our code. We can have perfect coverage with terrible tests that don't catch real bugs, or we can have modest coverage with excellent tests that prevent critical failures.

Lines of code is another vanity metric. More code isn't better—often it's worse. The best solutions are frequently the simplest ones that solve the problem with minimal complexity.

Number of features shipped can also mislead. Shipping features that users don't want or can't figure out how to use doesn't create value—it creates confusion and maintenance burden.

Story points completed measures team activity, not outcomes. A team can complete lots of story points whilst building the wrong thing or accumulating technical debt that will slow them down later.

Number of tickets moved to "Done" creates similar problems. Teams start breaking down work artificially to inflate their completion numbers, or they rush through tickets without proper validation. The focus shifts from solving problems well to closing tickets quickly.

Number of commits is perhaps the most misleading of all. More commits could indicate good incremental development practices, or it could mean a developer is making tiny, meaningless changes to look busy. Some of the most valuable work involves deleting code, not adding it, and a single well-crafted commit might be worth more than dozens of small ones.

The danger of vanity metrics isn't just that they're useless—they actively distort behaviour. When teams optimise for test coverage, they write meaningless tests. When they optimise for lines of code, they write verbose, complex solutions. When they optimise for feature count, they build shallow, poorly-integrated capabilities. When they optimise for ticket throughput, they fragment work unnecessarily. When they optimise for commit frequency, they create noise instead of signal.

Instead, focus on metrics that correlate with actual quality outcomes: user satisfaction scores, defect escape rates, mean time to recovery, deployment frequency, and lead time for changes. These metrics reflect both sides of quality and drive behaviours that actually improve it.

The DORA (DevOps Research and Assessment) metrics are particularly valuable here. They provide a research-backed framework for measuring software delivery performance that correlates strongly with both business outcomes and quality:

• Lead Time for Changes: How long from commit to production

• Deployment Frequency: How often we deploy to production

• Mean Time to Recovery: How quickly we restore service after incidents

• Change Failure Rate: Percentage of deployments causing production failures

These metrics help us keep quality "under control" in Deming's sense. They measure system performance, not individual performance, and they encourage behaviours that improve both internal and external quality simultaneously.

Quality becomes something we plan, measure, and trade off consciously. Not something left to gut feeling or after-the-fact testing.

Finding Our Bottlenecks

Eli Goldratt's Theory of Constraints applies perfectly to software quality. Our development process has bottlenecks. Maybe it's the deployment pipeline, code review process, or testing infrastructure.

Unless we identify and fix these constraints, everything else stalls. It doesn't matter how clean our code is if we can't deploy it safely. It doesn't matter how fast our tests run if code reviews take a week.

The five focusing steps apply:

1. Identify the constraint in our quality system

2. Exploit it—get maximum value from that bottleneck

3. Subordinate everything else to support fixing it

4. Elevate the constraint—invest to remove it

5. Repeat—find the next constraint

This way, both internal and external quality improve iteratively and systemically, not through random patches.

The Human Factor

Weinberg also emphasised something often overlooked: quality is fundamentally about people. Technical practices fail without the right culture.

Egoless programming means being open to feedback and admitting mistakes. Psychological safety means team members can raise quality concerns without fear. Shared ownership means everyone feels responsible for the codebase, not just their own corner.

The best quality initiatives I've seen started with building trust and collaboration in teams, not with adopting new tools or methodologies.

A Working Definition

Bringing it all together, here's a comprehensive definition:

Quality is the degree to which a software product consistently meets the needs and expectations of its users (external quality), delivered through a development system (practices, architecture, pipelines, teamwork) that enables safe, rapid, sustainable evolution (internal quality).

This definition acknowledges:

• Conformance is non-negotiable (Crosby)

• Quality emerges from the system, not inspection (Deming)

• Fast feedback and measurable attributes keep quality resilient (Farley, Gilb)

• Focus improvement where constraints bind flow (Goldratt)

• Culture and collaboration are foundational (Weinberg)

Practical Implications

So what does this mean for our team?

Define quality goals explicitly. Don't just say "make it better." Specify what better means for our users and our codebase.

Invest in fast feedback. If our tests take hours to run, we're not learning. If deployment takes days, we can't iterate. Speed up our feedback loops.

Monitor both sides of quality. Track user-facing metrics like performance and reliability, but also internal metrics like technical debt and cycle time.

Find our constraints. What's really slowing down our ability to deliver quality? Fix that first.

Build quality culture. Invest in practices that encourage collaboration, learning, and shared responsibility.

Iterate continuously. Quality isn't a destination. It's an ongoing practice of improvement.

The Long Game

Teams that understand quality's dual nature don't just ship better software. They build sustainable competitive advantages. They can respond to market changes faster. They attract and retain better developers. They create user experiences that competitors can't easily replicate.

But it requires thinking beyond the next sprint or quarterly deadline. It means investing in both faces of quality, understanding their interconnection, and building systems that support both.

The teams that get this right don't just survive. They thrive.

---

References

W. Edwards Deming - Out of the Crisis (1986). MIT Press. Pioneer of quality management and statistical process control.

Philip Crosby - Quality is Free (1979). McGraw-Hill. Introduced "conformance to requirements" definition and zero-defects philosophy.

Gerald Weinberg - The Psychology of Computer Programming (1971). Van Nostrand Reinhold. Quality Software Management series (1992-1997). Dorset House. Emphasised the human and cultural aspects of software quality.

Joseph Juran - Quality Control Handbook (1951). McGraw-Hill. Developed "fitness for use" concept and quality trilogy (planning, control, improvement).

Dave Farley - Continuous Delivery (2010, with Jez Humble). Addison-Wesley. Modern Software Engineering (2021). Addison-Wesley. Advocate for fast feedback loops and continuous delivery practices.

Tom Gilb - Software Engineering Economics (1981). Prentice Hall. Competitive Engineering (2005). Butterworth-Heinemann. Pioneer in quantitative software quality measurement.

Eli Goldratt - The Goal (1984). North River Press. Theory of Constraints (1990). North River Press. Developed constraint-based thinking for system optimization.

Herbert Simon - Administrative Behavior (1947). University of Chicago Press. Models of Bounded Rationality series (1982). MIT Press. Developed concept of bounded rationality in organisational decision-making.

Daniel Kahneman & Amos Tversky - Thinking, Fast and Slow (2011). Farrar, Straus and Giroux. Research on cognitive biases and behavioural economics, including present bias and temporal discounting.

Friedrich Nietzsche - On the Genealogy of Morals (1887). Various editions. Analysis of how values are shaped by power structures and social contexts.

Niccolò Machiavelli - The Prince (1532). Various editions. Political treatise on power, strategy, and the practical application of virtues.

ISO/IEC 25002:2024 - Systems and software engineering — Systems and software Quality Requirements and Evaluation (SQuaRE) — Quality model overview and usage. International Organization for Standardization.

DORA (DevOps Research and Assessment) - State of DevOps Report series (2014-present). Research program measuring software delivery performance and organizational capabilities.