Fix Ansible 'until' Loops: Solving the 'non-zero return code' Retry Error

The 2:00 AM Pipeline CrashIt’s late, and you’re pushing a critical hotfix to a production microservice. The Ansible playbook starts smoothly but suddenly hangs during a health check. After 60 seconds of dead air, your terminal explodes with an ANSI-red failure block:

FAILED! => {"attempts": 5, "changed": false, "msg": "non-zero return code"}

This error triggers when you use the until keyword to poll a resource and Ansible hits its retries limit. Essentially, the command inside your loop kept failing, or your success condition never became true before the timer ran out.

Why Your 'until' Condition is StallingAnsible evaluates both the module's success and your custom until logic during every loop iteration. Usually, one of three things is going wrong:

• The Success Gap: Your service takes 90 seconds to initialize, but your loop only waits for 50 seconds. The task isn't broken; it's just impatient.- Hidden Whitespace: You’re checking stdout for the word "ready," but the system is actually returning "ready\n". That single newline character will break a string comparison every time.- The Shell Trap: When using the shell module, any non-zero exit code (like a 404 from curl) makes Ansible view the attempt as a failure, even if you intended to wait for a 200 OK.## The Fix: Smarter Polling Logic### 1. Handling Temporary Command FailuresIf you use curl to check a booting API, the command itself will return an error code until the port is open. You must ensure the loop continues even when the command fails. While until naturally retries on failure, using the uri module is much cleaner than raw shell commands.

- name: Wait for Spring Boot API to report UP
  uri:
    url: "http://localhost:8080/actuator/health"
    status_code: 200
  register: api_result
  until: api_result.status == 200
  retries: 20
  delay: 10

# For legacy shell scripts:
- name: Check legacy service status
  shell: "/opt/bin/check_app_status.sh"
  register: shell_out
  until: shell_out.rc == 0
  retries: 15
  delay: 5
  failed_when: false # Prevents the task from crashing before retries finish

2. Debugging with High VisibilityIf your condition looks correct but still fails, you need to see exactly what Ansible sees. A common mistake is assuming stdout is a clean string. Use the trim filter or the search test to make your logic more resilient against extra spaces or log timestamps.

- name: Wait for log to confirm 'Database Migration Complete'
  shell: tail -n 5 /var/log/deploy.log
  register: log_check
  # Using search is safer than an exact string match
  until: log_check.stdout is search("Complete")
  retries: 12
  delay: 10

3. Calibrating the Polling WindowMath matters here. If a heavy database migration takes 5 minutes to run, but your Ansible task uses retries: 5 and delay: 10, you’ve only allowed 50 seconds of buffer. Always calculate the worst-case scenario and add a 20% safety margin.

- name: Wait for heavy DB migration (Max 6 minutes)
  command: /usr/bin/verify_db_schema.sh
  register: db_status
  until: db_status.rc == 0
  # 36 attempts * 10s delay = 360 seconds
  retries: 36
  delay: 10

Verification: Confirming the SolutionRun your playbook with the -v flag to watch the attempts in real-time. You should see the retry counter incrementing without the playbook stopping:

FAILED - RETRYING: Wait for API (20 retries left).
FAILED - RETRYING: Wait for API (19 retries left).
...
changed: [app_server] => {"attempts": 12, "changed": true, "status": 200}

If the task eventually turns green, your timing and logic are synced. If it hits attempt 36 and still dies, the problem lies with your application service, not your Ansible code.

Best Practices for Stable Loops- Prefer wait_for: Use the wait_for module to check if a port is open before running complex until loops. It’s faster and uses fewer resources.- Check Return Codes: Testing until: result.rc == 0 is almost always more reliable than parsing stdout strings.- Sanitize Strings: If you must match text, use | trim to remove invisible newline characters that cause comparison failures.- Avoid ignore_errors: Don't use ignore_errors: yes as a band-aid. It hides legitimate failures. Use failed_when or refine your until condition instead.