Skip to main content

Adventures in Kubernetes: When Your Node Changes Data Centers

· 2 min read

Today we faced an interesting challenge after migrating our prod26-k8s node to a new data center using snapshots. What seemed like a straightforward move turned into a fascinating journey through Kubernetes tolerations, cert-manager, and ingress configurations.

The Initial State

Our setup was relatively standard:

  • Single-node Kubernetes cluster running on prod26-k8s
  • Mercury bot deployment with nginx-ingress
  • Let's Encrypt certificates managed by cert-manager
  • All workloads running on the control-plane node

The Challenge

After restoring the node from a snapshot in the new DC, everything seemed fine at first glance. However, we quickly ran into several issues:

  1. Ingress Controller Pod: The nginx-ingress controller pod wouldn't schedule because it needed tolerations for the control-plane taint.
  2. Certificate Management: cert-manager's HTTP01 solver pods were stuck in Pending state.
  3. ACME Challenges: Let's Encrypt validation was failing with 503 errors.

TypeScript Declaration Pitfalls: 10 Costly Decisions to Avoid

· 4 min read
Max Kaido
Architect

When building a TypeScript project, type declarations are your first line of defense against bugs and maintainability issues. However, certain type declaration patterns can actually work against you, creating technical debt and making your codebase harder to maintain. Let's explore 10 specific anti-patterns we've identified in our analysis of real-world TypeScript projects.

Elegant Refactoring: A Journey to Better Domain-Driven Architecture

· 15 min read
Max Kaido
Architect

In any growing project, there comes a time when the initial architecture starts showing signs of strain. As features expand and requirements evolve, what once seemed like a clear structure can become muddled. Today, I want to share our journey of refactoring TON Arcana, focusing on creating a more cohesive, domain-driven architecture.

Here are 10 elegant improvements we're implementing to make our codebase cleaner, more maintainable, and better aligned with domain concepts.

Mastering Ansible Tags: A Practical Guide to Infrastructure Management

· 2 min read
Max Kaido
Architect

When managing complex infrastructure with Ansible, the ability to selectively run specific parts of your playbooks becomes crucial. This is where Ansible tags come in - they allow you to organize and execute tasks with surgical precision. Let's explore how we implemented a comprehensive tagging system in our infrastructure.

The Problem

Our infrastructure includes various components:

  • AI services (Ollama, OpenWebUI)
  • Monitoring systems
  • Development environments
  • Basic system configurations

Running the entire playbook for small changes was:

  • Time-consuming
  • Potentially disruptive
  • Resource-intensive

ATLAS: Building an AI-Powered Trading Analysis Framework

· 3 min read
Max Kaido
Architect

Introduction

ATLAS (AI Trading Logic and Analysis System) represents a new approach to algorithmic trading by combining traditional technical analysis with modern AI capabilities. Unlike conventional trading systems that rely solely on predefined rules or indicators, ATLAS leverages large language models to provide contextual, adaptive trading strategies while maintaining the rigor of systematic trading.

The Hidden Complexity of Dokku Networking: A Tale of Redis and DNS

· 2 min read
Max Kaido
Architect

Today, we faced a classic example of the hidden complexity in Dokku's networking model. What seemed like a simple task - connecting a NestJS application to Redis - turned into an adventure through container networking, DNS resolution, and service discovery.

The Challenge

Our Mercury Bot service needed to communicate with both Redis and ChromaDB on the same network. Simple, right? Not quite. The initial setup led to DNS resolution errors:

Error: getaddrinfo EAI_AGAIN dokku-redis-mercury

This cryptic error message was just the beginning of our journey into Dokku's networking internals.

Product Books

· 3 min read
Max Kaido
Architect

Product Books


1. Running Lean by Ash Maurya

  1. Validate your ideas through experiments before scaling.

  2. Focus on identifying and solving a single core problem for users.

  3. Use the Lean Canvas as a visual tool to capture and iterate on your business model.

  4. Build MVPs (Minimum Viable Products) to test assumptions quickly and cheaply.

  5. Prioritize learning over execution in the early stages.