#9616 fix: Implemented catalog locking to allow multiple instances of the same tap to run in parallel without race conditions
Pull request opened by
edgarrmondragon on <!date^1761847375^{date_short}|2025-10-30T18:02:55Z>
Description
Related Issues
•
#8763
•
#8794
Summary by Sourcery
Introduce file-based catalog locking to prevent race conditions when running multiple instances of the same tap in parallel. Add a catalog_lock context manager and apply it to key catalog operations (copying, discovery, rule application, and run_discovery) in the Singer Tap implementation.
New Features:
• Add catalog_lock context manager using fasteners.InterProcessLock to coordinate catalog file access across processes.
Enhancements:
• Wrap catalog copy, discovery, run_discovery, and apply_catalog_rules steps with catalog_lock to prevent concurrent read/write conflicts.
Tests:
• Add comprehensive tests for catalog_lock covering lock acquisition/release, timeout behavior, exception cleanup, sequential locking, and file operations.
meltano/meltano