<#9412 feat: Add meltano logs command for viewing ...
# meltano-repo-activity
g
#9412 feat: Add meltano logs command for viewing job logs Pull request opened by tayloramurphy ## Summary Implements issue #2184 by adding a new
meltano logs
command that provides CLI access to job logs, serving as the command-line equivalent to the "Job Log" modal in the Meltano UI. ### Key Features •
meltano logs list
- Shows a table of recent job runs with metadata including status, timing, and duration •
meltano logs show <log_id>
- Displays log content for specific runs identified by UUID • JSON output support - Both commands support
--format json
for programmatic use • Tail functionality -
--tail N
to show last N lines of logs • Status indicators - Visual markers: ✓ PASS (success), ✗ FAIL (failed), → RUN (running) • Large file handling - Confirmation prompts for logs over 2MB ### Implementation Details The implementation leverages existing Meltano infrastructure: • Uses
Job
model and
JobLoggingService
for log access • Integrates with current job tracking and state management • Compatible with existing log storage structure in
.meltano/logs/elt/
• Filters out
STATE_EDIT
operations to show only actual job runs • Forward-compatible with the planned UUIDv7 migration ### UX Design Simple two-command workflow: 1.
meltano logs list
to see recent runs and their status 2. Copy the LOG ID from the list 3.
meltano logs show <log_id>
to view specific log content ### Example Usage # List recent job runs (shows 10 by default) $ meltano logs list Recent job runs (showing 10 of last 10): STATUS LOG ID JOB NAME STARTED DURATION ------------------------------------------------------------------------------------------------ ✓ PASS 550e8400-e29b-41d4-a716-446655440000 tap-gitlab target-postgres 2024-01-15 102345 2.5m ✗ FAIL 660e8400-e29b-41d4-a716-446655440001 tap-salesforce target-jsonl 2024-01-15 091520 0.8m → RUN 770e8400-e29b-41d4-a716-446655440002 tap-stripe target-postgres 2024-01-15 113000 running Use 'meltano logs show <LOG_ID>' to view a specific log. # Show specific log content $ meltano logs show 550e8400-e29b-41d4-a716-446655440000 Job: tap-gitlab target-postgres Run ID: 550e8400-e29b-41d4-a716-446655440000 State: SUCCESS Started: 2024-01-15 102345.123456+00:00 Ended: 2024-01-15 102615.789012+00:00 Trigger: schedule:daily-gitlab-sync Log content: ---------------------------------------- [2024-01-15 102345] [INFO] Starting tap-gitlab... [2024-01-15 102346] [INFO] Connecting to GitLab API... ... # Show last 20 lines of a log $ meltano logs show 660e8400-e29b-41d4-a716-446655440001 --tail 20 # Get JSON output for programmatic use $ meltano logs list --format json { "runs": [ { "log_id": "550e8400-e29b-41d4-a716-446655440000", "job_name": "tap-gitlab target-postgres", "state": "SUCCESS", "started_at": "2024-01-15T102345.123456+00:00", "ended_at": "2024-01-15T102615.789012+00:00", "duration_seconds": 150.665556, "trigger": "schedule:daily-gitlab-sync" } ], "total": 1 } ## Test Plan • Unit tests for both commands covering success and error cases • JSON output format validation • Large file confirmation flow • Tail functionality with exact line count verification • Manual testing with actual Meltano projects ## Documentation • CLI reference documentation added • Usage examples and workflow guidance • Command options documented Fixes #2184 ## Summary by Sourcery Add a new "meltano logs" command group with "list" and "show" subcommands to provide CLI access to job logs, supporting JSON output, tailing, and large-file confirmation. New Features: • Implement "meltano logs list" to list recent job runs with metadata (status, timing, duration) • Implement "meltano logs show <log_id>" to display job log content with optional tailing • Support JSON output for both commands with the --format json option • Prompt for user confirmation before displaying logs larger than 2MB Documentation: • Add CLI reference documentation for the new logs commands • Extend the logging guide with examples and usage of the meltano logs commands Tests: • Add unit tests for listing runs, showing logs, JSON output, tail functionality, large-file confirmation, and error handling scenarios ## Summary by Sourcery Add a new "meltano logs" command group with "list" and "show" subcommands to enable CLI access to job logs. New Features: • Add "meltano logs list" to list recent job runs with status, timing, duration, trigger, and JSON output support • Add "meltano logs show <log_id>" to display logs for a specific run, with optional tailing and JSON metadata output • Prompt for user confirmation before displaying log files larger than 2MB Enhancements: • Add exhaustive-match to mypy enabled error codes • Update project_engine return typing for SQLAlchemy sessions Documentation: • Add CLI reference documentation for the new logs commands • Update logging guide to include usage of the meltano logs commands Tests: • Add unit tests for listing runs, showing logs, JSON output, tail functionality, large-file confirmation, missing logs, and error scenarios meltano/meltano