“Deployment successful” doesn’t mean production is running what you deployed.
Saw a post about ArgoCD explaining the security angle - no kubeconfig sharing, everything through Git.
Valid. But I want to focus on what breaks at 3am.
Question I get: “Why ArgoCD? Our Jenkins (GitHub Actions, GitLab CI, CircleCI, etc.) deploys fine?”
Works fine. Until you’re fixing production at 3am.
Picture this:
Push-based is like working on a thesis with a friend via USB drive. “Here’s thesis_v3_final_fix_last_checked.doc”. He edits, gives it back. Then you change your copy. Then he changes his. A week later you have 5 versions, nobody remembers which is current, and you submit the wrong one.
Pull-based is shared Google Doc. One file, one truth. Someone messes up locally - sync fixes it. Change history always there.
The real difference:
Jenkins deployed. kubectl apply exited 0. Build green. Beautiful.
But:
- Someone manually edited deployment in cluster? Jenkins doesn’t know.
- HPA changed replica count? Jenkins doesn’t see it.
- Another pipeline overwrote config? Jenkins doesn’t care.
- Manifest in Git is outdated? Jenkins applied what’s there.
kubectl apply is fire and forget.
ArgoCD lives in the cluster. Every 3 minutes checks: “Does Git match cluster state?”
No? Fixes it.
But that’s not the main point.
Main point - who controls your production?
Jenkins, GitHub Actions, GitLab CI - external tools. Managed by someone else. Their rules, their pricing, their breaking changes.
Remember Google URL Shortener? Bitnami MongoDB images? GitHub Actions pricing changes?
When CI/CD is directly integrated into your workflow - you depend on it as critical infrastructure. But don’t control it.
ArgoCD is part of your cluster. Under your control. Like a code library, not an external service.
My experience:
Seen both approaches in production.
Push-based breaks silently. Everything green, production down. Because what you deployed != what’s actually running.
Pull-based breaks loudly, immediately. Drift detection shows divergence. Git log - full audit trail. Rollback = git revert.
Yes, ArgoCD is another component. Learning curve, setup, maintenance.
But for Kubernetes in production - can critically improve reliability.
Not always. Sometimes it’s overkill, or your app doesn’t depend on git that much - then push-based is fine.
What usually breaks your deployments at night - manual edits, pipeline conflicts, or something else?
Inspired by Sai P.’s post