Skip to main content

Shadow Portfolio Feature Development: Reflections and Learnings

· 3 min read
Max Kaido
Max Kaido
Architect

From wrestling with race conditions in metric updates to discovering elegant state management patterns, this deep dive reveals hard-won insights from developing our shadow portfolio feature. Whether you're building financial systems, working with real-time metrics, or just interested in robust testing approaches, these battle-tested learnings will help you navigate similar challenges in your own projects.

Overview​

This post reflects on the development process of the shadow portfolio feature, analyzing challenges faced, solutions found, and potential patterns for future implementations.

1. Hardest Challenges​

a) Metric Synchronization​

  • Getting exact PnL values in metrics when positions close
  • Ensuring metrics are updated before being read
  • Dealing with async nature of metric updates

b) Maximum Movement Tracking​

  • Correctly capping max values at TP/SL
  • Handling the timing of updates vs position closure
  • Maintaining consistency between metrics and state

c) Test Expectations​

  • Different assumptions between test and implementation
  • Cascading test failures when fixing one issue
  • Timing-sensitive assertions

2. Root Causes​

a) Metric Synchronization Issues​

  • Prometheus-style metrics are fire-and-forget by design
  • We needed immediate consistency for tests
  • Multiple metrics needed to be updated atomically
  • Reading metrics right after updates led to race conditions

b) Movement Tracking Complexity​

  • Business logic wasn't initially clear (cap at TP/SL vs keep last value)
  • State updates needed to happen before position closure
  • Multiple moving parts (price updates, state tracking, closure conditions)

c) Test Issues​

  • Tests were written with implicit assumptions
  • Lack of clear documentation about expected behavior
  • Tests checking implementation details rather than behavior

3. Potential Patterns and Solutions​

a) Event Sourcing​

// Instead of direct state updates:
interface PositionEvent {
  type: 'PRICE_UPDATED' | 'MAX_UPDATED' | 'POSITION_CLOSED';
  data: any;
}

// Process events in sequence
async processEvent(event: PositionEvent) {
  switch(event.type) {
    case 'PRICE_UPDATED':
      await this.updateMaxValues();
      break;
    case 'POSITION_CLOSED':
      await this.updateMetrics();
      break;
  }
}

Benefits:

  • Clear state transitions
  • Easier testing
  • Better debugging

Drawback: Adds complexity

b) Command Query Responsibility Segregation (CQRS)​

// Separate write operations
interface PositionCommands {
  updatePrice(price: number): Promise<void>;
  closePosition(reason: ExitReason): Promise<void>;
}

// From read operations
interface PositionQueries {
  getMetrics(): Promise<Metrics>;
  getState(): Promise<PositionState>;
}

Benefits:

  • Clearer separation of concerns
  • Easier to handle async updates
  • Better testing isolation

Drawback: Might be overkill for simple features

c) State Machine Pattern​

enum PositionState {
  OPEN,
  PENDING_CLOSURE,
  CLOSED,
}

class PositionStateMachine {
  async transition(to: PositionState) {
    // Validate transition
    // Update state
    // Emit events
  }
}

Benefits:

  • Clear state transitions
  • Predictable behavior
  • Easier testing

Drawback: Adds boilerplate

4. Cost-Benefit Analysis​

Current Scale​

  • Current approach is working
  • Tests are passing
  • Code is maintainable
  • Issues were solved with minimal complexity

Future Scale Considerations​

  • If we add more complex position states → State Machine
  • If metrics become more critical → Event Sourcing
  • If read/write separation needed → CQRS

Key Learnings​

Metric Updates​

  • Always verify metric updates in sequence
  • Use exact values for TP/SL
  • Keep metric labels consistent

State Management​

  • Update max values before closure
  • Cap at TP/SL levels
  • Clear state transitions

Testing​

  • Test behavior, not implementation
  • Clear documentation of expectations
  • Handle async operations properly

Recommendations​

Current Implementation​

  1. Keep current implementation for now
  2. Focus on documentation and clarity
  3. Maintain consistent patterns

Future Considerations​

  1. State machine if adding more position states
  2. Event sourcing if audit/replay needed
  3. CQRS if scale demands it

Immediate Improvements​

  1. Better documentation of expected behaviors
  2. Clear test descriptions
  3. Consistent naming conventions

Conclusion​

While more complex patterns exist that could solve our challenges differently, the current implementation strikes a good balance between functionality and complexity. The overhead of implementing full patterns isn't justified yet, but keeping these patterns in mind will help us evolve the code if needed.

The key is not to over-engineer but to be aware of available patterns and apply them when the complexity justifies their use.

Tags​

#development #patterns #testing #metrics #learnings #shadow-portfolio