The Validation Nightmare: When Type Safety Goes Wrong

Today, I want to share a particularly frustrating development experience that perfectly illustrates how well-intentioned engineering practices can lead to significant problems when misapplied. Our team encountered a maddening issue in our Mercury dashboard where comparison trail data was present in the backend response but consistently failed to display in the UI.

The Symptoms

The issue was deceptively simple: users would see an empty comparison trail section with an error message stating "No valid comparison rounds found for this tournament" despite backend logs clearly showing the data was being sent to the frontend. This led to confusion among users and frustration among developers.

What made this particularly challenging was that:

1. Data was correctly formatted in the backend
2. Network traffic showed the data arrived at the frontend
3. TypeScript compiler showed no errors
4. React components rendered without runtime errors
5. Yet users saw only error messages instead of data

The Root Cause: Validation Hell

After hours of debugging, we found the culprit: excessive validation layers. Our codebase had accumulated multiple layers of data validation that created what I call a "validation gauntlet" - a series of increasingly strict checks that data had to pass before being displayed.

// Just one of MANY validation checks in our components
if (!comparisonTrail || !comparisonTrail.comparisons) {
  return <ErrorMessage message="Invalid comparison trail data." />;
}

if (comparisonTrail.comparisons.length === 0) {
  return <ErrorMessage message="No comparison data available." />;
}

// Group comparisons by round
const roundMap = {};
comparisonTrail.comparisons.forEach(comp => {
  if (!comp || typeof comp.roundNumber !== 'number') {
    console.error('Invalid comparison object:', comp);
    return;
  }
  // More validation...
});

// Sort rounds
const sortedRounds = Object.keys(roundMap).map(key => parseInt(key));

// Even more validation after all this
if (sortedRounds.length === 0) {
  return <ErrorMessage message="No valid comparison rounds found." />;
}

This pattern repeated throughout our codebase, with each component adding its own validation layer:

1. The API client validated and potentially rejected data
2. The position page component validated the data again
3. The TournamentAnalysis component performed yet more validation
4. The ComparisonRound component had its own validation
5. Even the ComparisonDetail component rejected data that didn't meet its standards

The result? Data that was perfectly usable was being rejected at some point in this chain, even though it had the necessary information to display. Users saw error messages instead of their data.

How Did This Happen?

This misguided approach evolved for several reasons:

1. Defensive Programming Gone Wrong: The team adopted an extreme version of defensive programming where every component protected itself against potentially malformed data.

2. Type Confusion: Despite using TypeScript, we mixed runtime and compile-time validations, creating redundant checks that added no safety but increased complexity.

3. Feature Isolation: Components were developed in isolation, so each developer added their own validation without realizing the cumulative effect.

4. Mixed Validation Responsibilities: The frontend tried to enforce data correctness rather than focusing on flexible presentation of whatever data was available.

The Solution: Back to Basics

The fix was remarkably simple yet required an overhaul of our thinking:

1. Trust the Type System: Rely on TypeScript's static types rather than excessive runtime checks.
2. Graceful Degradation: Components should display whatever data is available rather than showing nothing if it's imperfect.
3. Single Validation Point: Consolidate validation to a single point in the API client rather than repeating it throughout the component hierarchy.
4. Fallback Values: Use nullish coalescing and default values instead of rejecting entire data structures.

// After: A much more resilient component
const ComparisonDetail = ({ comparison }: { comparison: IComparisonResponse }) => {
  // Basic protection against null
  if (!comparison) return null;

  return (
    <div className="mb-6 border border-gray-700 bg-gray-900/30 rounded-lg p-4">
      <h3 className="text-lg font-semibold mb-2">
        <span className={comparison.winner === comparison.itemA ? "text-green-400" : "text-gray-400"}>
          {comparison.itemA || 'Market A'}
        </span>
        <span className="mx-2 text-gray-400">vs</span>
        <span className={comparison.winner === comparison.itemB ? "text-green-400" : "text-gray-400"}>
          {comparison.itemB || 'Market B'}
        </span>
      </h3>
      {/* Content that gracefully handles missing data */}
    </div>
  );
};

Top 3 Development Approaches We Should Have Used

Looking back, three key approaches could have prevented this issue:

1. Proper Separation of Concerns

The frontend should focus on presenting data, not enforcing data correctness. Validation belongs in specific layers:

• Backend API: Ensure data correctness before sending to clients
• API Client: Parse and normalize data once when receiving from the backend
• Components: Focus on rendering whatever data is available, with fallbacks for missing elements

2. Progressive Enhancement

Instead of an all-or-nothing approach, we should have embraced progressive enhancement:

• Show core data even if some parts are missing
• Gracefully degrade functionality rather than showing error messages
• Use optional chaining and nullish coalescing for cleaner handling of partial data

3. Centralized Type Management

Our types were scattered across multiple files with inconsistent definitions:

• Create a single source of truth for data types
• Use TypeScript interfaces to enforce consistency
• Separate runtime validation from type checking
• Generate types from API schemas where possible

Top 3 Libraries That Could Have Helped

Several libraries could have helped us avoid these issues:

1. zod

Zod provides runtime validation with static type inference, avoiding duplication between validation logic and type definitions.

// Define schema once, get both validation and types
const ComparisonSchema = z.object({
  id: z.string(),
  itemA: z.string(),
  itemB: z.string(),
  winner: z.string(),
  // ...other fields
});

// Type is inferred from schema
type Comparison = z.infer<typeof ComparisonSchema>;

// Validation with clear errors
function parseComparisonTrail(data: unknown) {
  return ComparisonSchema.parse(data);
}

2. io-ts

Similar to Zod but with a focus on functional programming patterns, io-ts would have helped us handle validation results more elegantly.

import * as t from 'io-ts';
import { fold } from 'fp-ts/lib/Either';

// Define codec
const ComparisonCodec = t.type({
  id: t.string,
  // ...other fields
});

// Use in API client
const validateComparison = (data: unknown) =>
  fold(
    (errors) => ({ ...defaultComparison, _errors: errors }), // Handle errors gracefully
    (validData) => validData, // Use valid data
  )(ComparisonCodec.decode(data));

3. react-query

React Query would have simplified our data fetching and prevented many of these issues through its structured caching and error handling.

function useComparisonTrail(positionId: string) {
  return useQuery(
    ['comparisonTrail', positionId],
    () => fetchComparisonTrail(positionId),
    {
      // Structured error handling
      onError: (error) => console.error('Comparison trail error:', error),

      // Transform response once, not in every component
      select: (data) => transformComparisonTrail(data),

      // Graceful error UI
      useErrorBoundary: false,
    },
  );
}

Lessons Learned

This experience taught us several important lessons:

1. Less is More with Validation: Validate at boundaries, not in every component.
2. Trust Your Types: If using TypeScript, let the type system do its job and minimize runtime checks.
3. Design for Resilience: Components should handle whatever data they receive, not require perfect data.
4. Test with Real Data: Many of these issues would have been caught if we tested with actual production data.
5. Review Full Stack Flows: Review how data flows through the entire system, not just individual components.

By stepping back and simplifying our approach—removing excessive validation, trusting our type system, and focusing on resilient UI components—we solved a problem that had frustrated users and developers alike.

Remember: The best code is often the code you remove, not the code you add. Sometimes we need to clear the path rather than build more barriers.